How to Self-Host Coolify with Docker Compose: The Open-Source Heroku and Vercel Alternative for Your Homelab and VPS

DevOps engineer managing self-hosted Coolify PaaS with Docker Compose and Traefik reverse proxy
How to Self-Host Coolify with Docker Compose: The Open-Source Heroku and Vercel Alternative for Your Homelab and VPS 3

Modern cloud development has been revolutionized by Platform-as-a-Service (PaaS) providers like Heroku, Vercel, Netlify, and Render. These platforms eliminated the friction of server administration by allowing software engineers to connect a Git repository and deploy full-stack applications with automatic TLS certificates, continuous integration previews, and managed database provisioning. However, as applications scale or corporate security compliance takes precedence, commercial PaaS pricing structures quickly become exorbitant. Incurring thousands of dollars monthly for bandwidth egress, compute tier upgrades, and persistent volume add-ons forces engineering organizations and homelab operators to seek sovereign alternatives.

Coolify is the undisputed leader in self-hosted, open-source PaaS technology. Often described as the self-hosted Heroku and Vercel alternative, Coolify allows developers to deploy applications, managed databases (PostgreSQL, MySQL, Redis, MongoDB, ClickHouse), and one-click open-source services across any bare-metal Linux server, VPS (Hetzner, DigitalOcean, Linode), or local homelab cluster. With built-in support for Nixpacks, Dockerfiles, Docker Compose stacks, GitHub/GitLab webhooks, PR preview deployments, and automated SSL termination, Coolify brings developer ergonomics back to infrastructure you own.

In this technical walkthrough, you will learn how to manually deploy, configure, and harden Coolify v4 using Docker Compose. While Coolify offers a single-line automated bash installer, understanding and controlling the underlying Docker Compose architecture is crucial for production environments. You will inspect the multi-container control plane, configure Traefik as the dynamic edge router, orchestrate background workers via Redis, secure the Docker daemon socket, and set up automated S3 backups.

Coolify Architecture & Execution Model

Unlike monolithic control panels, Coolify separates its administrative UI and orchestration engine from the applications it manages. Coolify interacts with the host Docker engine via the Docker socket (or over remote SSH for multi-server clusters). When you trigger a deployment, Coolify builds your container using Nixpacks or Docker BuildKit, registers dynamic routing labels on the application container, and instructs Traefik to request and terminate Let’s Encrypt TLS certificates.

+-----------------------------------------------------------------------+
|                    Public Traffic & Webhook Ingress                   |
|           (Developers, End Users, GitHub/GitLab Webhooks)             |
+-----------------------------------------------------------------------+
                                   |
                                   | HTTPS (Port 443 / 80)
                                   v
+-----------------------------------------------------------------------+
|                       Edge Router: Traefik v3                         |
|   - Dynamic Docker Socket Discovery (Label-based routing)             |
|   - Automatic Let's Encrypt ACME TLS Management                      |
|   - Routes traffic to Coolify Core UI (Port 8000) & Deployed Apps     |
+-----------------------------------------------------------------------+
         |                                           |
         | Internal Routing                          | Dynamic Application Proxy
         v                                           v
+----------------------------+             +----------------------------+
|     Coolify Core Engine    |             |    User Deployed Apps      |
|  - Laravel / PHP 8.3 FPM   |             |  - Next.js / Node / Go     |
|  - REST API & Web UI       |             |  - Python / FastAPI / Rust |
+----------------------------+             |  - Managed PostgreSQL/Redis|
         |                                 +----------------------------+
         +-------------------+                       ^
         | Job Queues        | Database State        | Docker Run / Build
         v                   v                       |
+-----------------+ +-------------------+            |
|  Redis 7 Queue  | | PostgreSQL 16 DB  |            |
|  - Deployments  | | - App metadata    |            |
|  - Cron Workers | | - Environment vars|            |
+-----------------+ +-------------------+            |
         ^                                           |
         | Horizon Worker Tasks                      |
+----------------------------------------------------+------------------+
|                          Coolify Realtime Agent                       |
|   - Executes Docker build commands via /var/run/docker.sock           |
|   - Streams real-time build logs via WebSockets to Admin Dashboard    |
+-----------------------------------------------------------------------+

Core components of the stack include:

  • Coolify Core (coolify): The primary administrative API and dashboard powered by a high-performance Laravel application runtime.
  • PostgreSQL 16: Stores user configurations, server definitions, encrypted environment variables, deployment history, and project schemas.
  • Redis 7: Manages asynchronous background jobs (such as cloning Git repositories, compiling images, health check polls, and automated database backups) through Laravel Horizon.
  • Traefik v3: The reverse proxy that continuously monitors Docker container events. When an application container is spun up with specific Traefik labels, Traefik automatically issues TLS certificates and routes subdomains without requiring server reloads.
  • Realtime / Socket Engine: Provides bi-directional WebSocket streaming so you can watch live deployment terminal logs directly in your browser.

Prerequisites & Server Preparation

Before deploying Coolify, verify that your server meets the following operational criteria:

  • Operating System: Ubuntu 22.04 / 24.04 LTS, Debian 12, or AlmaLinux 9 (64-bit x86_64 or ARM64).
  • Hardware Sizing: Minimum 2 vCPUs and 4 GB RAM. If you plan to build modern frontend frameworks (e.g., Next.js, Nuxt) with Nixpacks or compile Rust/Go microservices, 8 GB RAM or a configured 4 GB swap file is strongly recommended to prevent out-of-memory compiler termination.
  • Storage: Minimum 30 GB SSD/NVMe storage to accommodate base build images and container caches.
  • Network & DNS: Wildcard DNS record (e.g., *.yourdomain.com) or dedicated A/AAAA records pointing to your server IP address. Ports 80 and 443 must be open and unblocked by external firewalls.

Create the required directory hierarchy on your host server:

sudo mkdir -p /data/coolify/{source,ssh,applications,databases,backups,services}
sudo mkdir -p /data/coolify/proxy/dynamic
cd /data/coolify

Ensure that Docker Engine 26+ and Docker Compose v2.26+ are installed. Also ensure that SSH key pairs exist for local host automation:

# Generate an internal deployment SSH key if none exists
if [ ! -f /data/coolify/ssh/id_rsa ]; then
    sudo ssh-keygen -t ed25519 -N "" -C "coolify-internal" -f /data/coolify/ssh/id_rsa
fi
sudo chmod 600 /data/coolify/ssh/id_rsa

Step 1: Configure Environment Variables (.env)

Coolify requires several cryptographic keys to encrypt sensitive environment secrets, credentials, and Git private keys stored in the database. Generate high-entropy keys using openssl rand -base64 32:

cat << 'EOF' > /data/coolify/source/.env
# ==============================================================================
# Coolify Control Plane Production Configuration
# ==============================================================================

# Core Application Keys (Generate unique values)
APP_ID=coolify-primary-node
APP_NAME=Coolify
APP_ENV=production
APP_KEY=base64:GENERATE_BASE64_KEY_HERE_32_BYTES==
APP_DEBUG=false

# Root Dashboard URL
APP_URL=https://coolify.yourdomain.com

# PostgreSQL Database Backend
DB_CONNECTION=pgsql
DB_HOST=coolify-db
DB_PORT=5432
DB_DATABASE=coolify
DB_USERNAME=coolify
DB_PASSWORD=generate_super_secure_pg_password_here_32chars

# Redis Cache & Job Queues
REDIS_HOST=coolify-redis
REDIS_PASSWORD=generate_super_secure_redis_password_here_32chars
REDIS_PORT=6379

# Queue Management
QUEUE_CONNECTION=redis

# Traefik Dashboard & Reverse Proxy Configuration
TRAEFIK_ACME_EMAIL=admin@yourdomain.com

# Docker Engine Integration
DOCKER_HOST=unix:///var/run/docker.sock
AUTOUPDATE=false
EOF
chmod 600 /data/coolify/source/.env

Step 2: Deploy Coolify Stack via Docker Compose

Create the /data/coolify/source/docker-compose.yml file. This production compose manifest defines the database, queue broker, Coolify core application, and Traefik edge proxy:

services:
  # ----------------------------------------------------------------------------
  # PostgreSQL Database Storage
  # ----------------------------------------------------------------------------
  coolify-db:
    image: postgres:16-alpine
    container_name: coolify-db
    restart: unless-stopped
    volumes:
      - /data/coolify/databases/postgres:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: ${DB_DATABASE}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    networks:
      - coolify_internal
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME} -d ${DB_DATABASE}"]
      interval: 5s
      timeout: 5s
      retries: 5

  # ----------------------------------------------------------------------------
  # Redis Queue & Realtime Cache
  # ----------------------------------------------------------------------------
  coolify-redis:
    image: redis:7-alpine
    container_name: coolify-redis
    restart: unless-stopped
    command: ["redis-server", "--requirepass", "${REDIS_PASSWORD}", "--appendonly", "yes"]
    volumes:
      - /data/coolify/databases/redis:/data
    networks:
      - coolify_internal
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
      interval: 5s
      timeout: 5s
      retries: 5

  # ----------------------------------------------------------------------------
  # Coolify Orchestration Core & Web Interface
  # ----------------------------------------------------------------------------
  coolify:
    image: ghcr.io/coollabsio/coolify:latest
    container_name: coolify
    restart: unless-stopped
    depends_on:
      coolify-db:
        condition: service_healthy
      coolify-redis:
        condition: service_healthy
    volumes:
      - /data/coolify/source/.env:/var/www/html/.env:ro
      - /data/coolify/ssh:/root/.ssh:ro
      - /var/run/docker.sock:/var/run/docker.sock
      - /data/coolify:/data/coolify
    env_file:
      - /data/coolify/source/.env
    networks:
      - coolify_internal
      - coolify_edge
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.coolify.rule=Host(`coolify.yourdomain.com`)"
      - "traefik.http.routers.coolify.entrypoints=websecure"
      - "traefik.http.routers.coolify.tls=true"
      - "traefik.http.routers.coolify.tls.certresolver=letsencrypt"
      - "traefik.http.services.coolify.loadbalancer.server.port=80"

  # ----------------------------------------------------------------------------
  # Traefik v3 Edge Proxy (Dynamic Let's Encrypt SSL)
  # ----------------------------------------------------------------------------
  coolify-proxy:
    image: traefik:v3.1
    container_name: coolify-proxy
    restart: unless-stopped
    command:
      - "--api.dashboard=false"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.file.directory=/traefik/dynamic"
      - "--providers.file.watch=true"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.websecure.http.tls=true"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge=true"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web"
      - "--certificatesresolvers.letsencrypt.acme.email=${TRAEFIK_ACME_EMAIL}"
      - "--certificatesresolvers.letsencrypt.acme.storage=/traefik/acme.json"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /data/coolify/proxy/acme.json:/traefik/acme.json
      - /data/coolify/proxy/dynamic:/traefik/dynamic
    networks:
      - coolify_edge

networks:
  coolify_internal:
    internal: true
  coolify_edge:
    name: coolify
    external: false

Before starting the containers, prepare the ACME security storage file with strict read/write permissions required by Traefik:

touch /data/coolify/proxy/acme.json
chmod 600 /data/coolify/proxy/acme.json

Step 3: Initializing and Bootstrapping Coolify

Launch the control plane in detached mode:

cd /data/coolify/source
docker compose up -d

Observe the startup logs to verify database migrations and Traefik SSL certificate acquisition:

docker compose logs -f --tail=100 coolify

Once initialization finishes, open your browser and navigate to https://coolify.yourdomain.com. On your first visit, Coolify prompts you to register the root administrative account (Name, Email, and Password). After completing registration, you are greeted by the Coolify v4 Command Center.

Step 4: Connecting Git Providers & Deploying Your First App

Coolify supports direct GitHub App integration, GitLab, custom Gitea servers, or raw Git repository URLs over SSH. To configure an automated CI/CD pipeline:

  1. Navigate to Sources: Click Sources in the navigation sidebar, choose GitHub, and click Create GitHub App. Coolify will automatically redirect you to GitHub with pre-filled webhook permissions.
  2. Create a Project: Go to Projects > Create New Project > select your environment (e.g., production).
  3. Add Resource: Choose Private Repository (with GitHub App), select your repository, and pick your target deployment branch (e.g., main).
  4. Select Build Pack: Coolify auto-detects your application language. Choose Nixpacks for zero-config builds (Node.js, Python, Go, PHP, Ruby) or Dockerfile for custom container logic.
  5. Define Domains & Ports: Enter your desired application subdomain (e.g., https://api.yourdomain.com) and the internal port your code listens on (e.g., 3000 or 8080).
  6. Deploy: Click Deploy. Coolify will pull the repository, run the build pipeline, execute container healthchecks, and configure Traefik TLS routing in seconds.

Step 5: Resource Quotas & Container Isolation (Hardening)

In a multi-tenant or multi-application environment, an unconstrained container with a memory leak or infinite loop can exhaust host resources and crash the entire server. Coolify allows you to enforce strict cgroups resource limits directly from the Web UI or via Compose configuration:

  • Memory Limits: Under your application settings, configure Memory Limit (e.g., 512M or 2G) and Memory Reservation (e.g., 256M).
  • CPU Cores: Restrict CPU allocation (e.g., 1.5 cores) to prevent single-threaded spinlocks from starving the kernel.
  • Auto-Restart Policy: Enforce unless-stopped or configure healthcheck-driven automatic container restarts.

Step 6: Automating Encrypted Database Backups to S3

One of Coolify’s most formidable capabilities is native, automated backup management for all deployed database instances (PostgreSQL, MySQL, MariaDB, Redis, and MongoDB). Rather than maintaining custom host crontabs, you can configure scheduled dumps pushed to any S3-compatible object store (such as MinIO, AWS S3, Cloudflare R2, or Backblaze B2):

  1. In the Coolify dashboard, navigate to Destinations > Storages > Add S3 Storage.
  2. Input your S3 Endpoint (e.g., https://s3.eu-central-1.amazonaws.com), Bucket Name, Access Key, and Secret Key.
  3. Open any managed database instance within your projects, select the Backups tab, and enable Scheduled Backups.
  4. Define a standard cron expression (e.g., 0 2 * * * for 02:00 AM daily) and select your configured S3 storage target. Coolify automatically creates compressed, timestamped dumps and uploads them offsite.

Troubleshooting Common Coolify Issues

1. Traefik Fails to Obtain Let’s Encrypt Certificate

Symptom: Accessing your deployed application URL results in a browser SSL warning (ERR_CERT_AUTHORITY_INVALID or Default Traefik Certificate).

Root Cause: Port 80 is either blocked by a cloud provider firewall, the DNS record does not match the server’s public IPv4/IPv6, or /data/coolify/proxy/acme.json has incorrect permissions.

Resolution: Verify external port 80 accessibility and check Traefik logs:

# Verify Traefik logs for ACME challenge errors
docker logs -f coolify-proxy | grep -i acme

# Ensure acme.json has strict permissions
sudo chmod 600 /data/coolify/proxy/acme.json

2. Build Process Fails with Exit Code 137 (OOM Killer)

Symptom: Complex frontend compilation (e.g., npm run build in Next.js or Vite) abruptly terminates with Killed or error: script 'build' exited with code 137.

Root Cause: Node.js build processes and compilers exceed available physical RAM, triggering the Linux kernel Out-Of-Memory (OOM) killer.

Resolution: Create a persistent 4 GB swap file on your Linux host to absorb memory spikes during heavy compilation:

sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

3. Permission Denied Accessing Docker Socket (/var/run/docker.sock)

Symptom: The Coolify dashboard shows “Docker daemon is unreachable” or container startup logs throw permission denied while trying to connect to the Docker daemon socket.

Root Cause: Group ownership or SELinux policies prevent the container process from communicating with the UNIX socket.

Resolution: Verify socket permissions and group membership on the host:

# Check docker socket ownership
ls -l /var/run/docker.sock

# Ensure standard permissions
sudo chmod 660 /var/run/docker.sock
sudo chown root:docker /var/run/docker.sock

Conclusion & Next Steps

Self-hosting Coolify provides modern developer agility without sacrificing infrastructure control or paying unpredictable cloud bills. With automatic SSL issuance, Git-driven CI/CD builds, managed databases, and multi-node scalability, your private cloud PaaS is ready to serve production workloads.

To further advance your self-hosted platform, explore connecting multiple worker nodes via Coolify’s remote server manager, setting up private container registries using Harbor, or tunneling internal dashboards securely through Cloudflare Zero Trust or Tailscale.

Related Posts