Self-Host Immich with Docker Compose: High-Performance Photo and Video Backup with Machine Learning

Systems engineer managing Immich photo backup dashboard and machine learning pipeline in homelab
Self-Host Immich with Docker Compose: High-Performance Photo and Video Backup with Machine Learning 3

Modern mobile smartphones and digital cameras generate terabytes of high-resolution RAW images, 4K 60fps video clips, and live photos. For over a decade, consumer cloud ecosystems such as Google Photos, Apple iCloud, and Microsoft OneDrive have served as default backup targets. However, escalating subscription costs, arbitrary storage tier limitations, automated algorithmic account suspensions, and deep privacy concerns over unencrypted media scanning have pushed self-hosters and IT professionals toward sovereign alternatives.

Among self-hosted photo management suites, Immich stands out as the definitive open-source standard. Unlike legacy archival solutions, Immich provides native iOS and Android mobile synchronization, automated background backups, high-performance facial recognition, reverse geocoding, and semantic natural language search powered by state-of-the-art CLIP machine learning models. In this comprehensive technical guide, you will deploy a battle-tested, production-ready Immich stack using Docker Compose, PostgreSQL with vector acceleration, Redis caching, hardware-accelerated video transcoding, and a Caddy reverse proxy.

Architectural Overview & Core Components

Immich is designed around a decoupled, microservices-based architecture built to handle hundreds of thousands of digital assets without blocking UI interactions. Understanding the lifecycle of an uploaded photo or video ensures reliable operations and predictable performance tuning.

+-----------------------------------------------------------------------------------+
|                            Client Layer (iOS / Android / Web)                     |
+-----------------------------------------------------------------------------------+
                                         |
                                         | HTTPS (TLS Termination & WebSockets)
                                         v
+-----------------------------------------------------------------------------------+
|                        Reverse Proxy (Caddy / Let's Encrypt)                      |
+-----------------------------------------------------------------------------------+
                                         |
                                         | Port 2283 (HTTP Stream & Multipart Uploads)
                                         v
+-----------------------------------------------------------------------------------+
|                    immich-server (Node.js API & Microservices)                    |
|    - Asset Ingestion, Hash Deduplication, EXIF Extraction, Multi-threading        |
+-----------------------------------------------------------------------------------+
          |                                  |                           |
          | Read/Write Metadata              | Queue Background Jobs     | Vector Inference
          v                                  v                           v
+-----------------------+          +-------------------+       +--------------------+
| PostgreSQL 16 DB      |          | Redis 7.2         |       | immich-machine-    |
| - pgvector Extension  |          | - BullMQ Queues   |       | learning (Python)  |
| - Relational Metadata |          | - Thumbnail Tasks |       | - InsightFace      |
| - Geo Coordinates     |          | - Transcoding     |       | - CLIP Search      |
+-----------------------+          +-------------------+       +--------------------+
          |                                                                |
          +-------------------------------+--------------------------------+
                                          |
                                          v
+-----------------------------------------------------------------------------------+
|                           Persistent Storage Layer (ZFS / SSD)                     |
|           /opt/immich/library  |  /opt/immich/upload  |  /opt/immich/profile       |
+-----------------------------------------------------------------------------------+

The core building blocks include:

  • immich-server: The unified central service handling API endpoints, client authentication, live WebSockets, metadata indexing, and internal job execution (formerly separated into server and microservices).
  • immich-machine-learning: A dedicated Python container running ONNX Runtime models for facial detection (InsightFace) and natural language semantic text-to-image queries (ViT/CLIP).
  • PostgreSQL with pgvector: Persistent relational storage augmented with vector indexing capabilities. This allows sub-millisecond similarity vector lookups for smart search without requiring an external vector database.
  • Redis: High-throughput in-memory message broker that orchestrates background job pipelines, including video transcoding, thumbnail rendering, metadata backfilling, and facial clustering.

Step 1: Host Preparation and Storage Topography

Immich requires deterministic directory permissions and appropriate storage separation. It is best practice to place database files on low-latency NVMe/SSD storage while directing bulk original photo and video libraries to high-capacity storage pools (such as a ZFS dataset or enterprise RAID array).

Create the dedicated directory hierarchy on your host system:

# Create deployment directories
sudo mkdir -p /opt/immich/{library,postgres,model-cache,upload}

# Create a dedicated non-root service account
sudo groupadd -g 1024 immich
sudo useradd -u 1024 -g immich -m -s /bin/false immich

# Grant ownership to the immich service user
sudo chown -R 1024:1024 /opt/immich

Step 2: Environment Configuration (.env)

Immich uses an environment file to decouple credentials, host paths, and runtime flags from the compose definition. Navigate to /opt/immich and create .env:

sudo nano /opt/immich/.env

Paste the following production configuration, ensuring you generate strong, random strings for database credentials and cryptographic tokens:

# ==============================================================================
# Immich Core Environment Configuration
# ==============================================================================

# Software Release Version
IMMICH_VERSION=release

# Local Storage Directory Paths
UPLOAD_LOCATION=/opt/immich/library
DB_DATA_LOCATION=/opt/immich/postgres
MODEL_CACHE_LOCATION=/opt/immich/model-cache

# System Localization
TZ=UTC

# Database Credentials
DB_HOSTNAME=immich_postgres
DB_USERNAME=immich_admin
DB_DATABASE_NAME=immich
DB_PASSWORD=SecureDbPassphrase_ChangeMe_982341209

# Application Security & Token Secrets
# Generate using: openssl rand -base64 32
JWT_SECRET=W3v+5N9uL82xY1Zk4jP0mQ7aB6cE5gH2jK9lP4mN1wE=

# Machine Learning Concurrency & Hardware Acceleration
# Options: cpu, cuda (NVIDIA), openvino (Intel CPU/iGPU), armnn (ARM)
IMMICH_MACHINE_LEARNING_ACCELERATION=cpu
MACHINE_LEARNING_WORKERS=2
MACHINE_LEARNING_REQUEST_THREADS=4

# Network Bindings
IMMICH_SERVER_PORT=2283

Secure the environment file permissions so unprivileged system users cannot inspect database credentials:

sudo chmod 600 /opt/immich/.env
sudo chown 1024:1024 /opt/immich/.env

Step 3: Production Docker Compose Definition

Create the orchestrator file at /opt/immich/docker-compose.yml. This configuration specifies isolated bridge networking, healthchecks, automatic restarts, and persistent volume bindings:

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION}
    restart: unless-stopped
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - "127.0.0.1:${IMMICH_SERVER_PORT}:2283"
    depends_on:
      redis:
        condition: service_healthy
      database:
        condition: service_healthy
    networks:
      - immich_internal
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:2283/api/server-info/ping"]
      interval: 15s
      timeout: 5s
      retries: 5
      start_period: 30s

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION}
    restart: unless-stopped
    volumes:
      - ${MODEL_CACHE_LOCATION}:/cache
    env_file:
      - .env
    networks:
      - immich_internal
    healthcheck:
      test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:3003/ping')"]
      interval: 20s
      timeout: 5s
      retries: 3
      start_period: 45s

  redis:
    container_name: immich_redis
    image: docker.io/redis:7.2-alpine
    restart: unless-stopped
    command: ["redis-server", "--appendonly", "yes", "--save", "900", "1"]
    volumes:
      - redis_data:/data
    networks:
      - immich_internal
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 5

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg16-v0.2.1
    restart: unless-stopped
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: "--data-checksums"
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    networks:
      - immich_internal
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME} -d ${DB_DATABASE_NAME}"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s

volumes:
  redis_data:
    driver: local

networks:
  immich_internal:
    driver: bridge

Step 4: Optional Hardware Acceleration (NVIDIA / Intel QuickSync)

Immich processes massive volumes of video transcoding (H.264, HEVC/H.265, AV1) and AI inference. Offloading these tasks from the CPU to an integrated GPU (Intel QuickSync) or dedicated graphics card (NVIDIA NVENC/Tensor Cores) drastically reduces CPU saturation and cuts job execution times.

Option A: Intel QuickSync (VA-API / QSV)

If your host runs an Intel CPU with integrated graphics (e.g., UHD/Iris Xe), mount the hardware render device into immich-server:

  immich-server:
    # ... existing configuration ...
    devices:
      - /dev/dri:/dev/dri

Ensure the Docker host permissions allow access to /dev/dri/renderD128 by verifying that the user belongs to the render or video group: sudo usermod -aG render $USER.

Option B: NVIDIA GPU Acceleration (CUDA)

For dedicated NVIDIA GPUs, install the NVIDIA Container Toolkit and expose GPU resources to both the server and machine learning containers:

  immich-machine-learning:
    # ... existing configuration ...
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

In your .env file, update: IMMICH_MACHINE_LEARNING_ACCELERATION=cuda.

Step 5: Initializing the Stack and Launch Verification

Launch the containers in detached mode and follow the startup logs to ensure database migrations and vector extensions initialize without errors:

cd /opt/immich
docker compose up -d

# Verify running container states
docker compose ps

Check the container health statuses:

NAME                      IMAGE                                            STATUS
immich_postgres           tensorchord/pgvecto-rs:pg16-v0.2.1               Up (healthy)
immich_redis              docker.io/redis:7.2-alpine                       Up (healthy)
immich_machine_learning   ghcr.io/immich-app/immich-machine-learning       Up (healthy)
immich_server             ghcr.io/immich-app/immich-server                 Up (healthy)

Step 6: Production Reverse Proxy with Caddy & TLS

Because Immich handles large video uploads and bi-directional real-time WebSocket communication, standard reverse proxy timeouts and file size limits must be configured properly. Caddy is ideal due to its automated Let’s Encrypt certificate lifecycle and native HTTP/2 + HTTP/3 support.

Add the following virtual host block to your /etc/caddy/Caddyfile:

photos.yourdomain.com {
    # Automatic TLS & HSTS
    encode zstd gzip

    # Remove upload size limits for multi-gigabyte 4K video clips
    request_body {
        max_size 50GB
    }

    # Reverse proxy upstream to Immich Server
    reverse_proxy 127.0.0.1:2283 {
        # Streaming and WebSocket support
        flush_interval -1
        header_up X-Forwarded-Proto {scheme}
        header_up X-Real-IP {remote_host}
    }

    # Security Headers
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "SAMEORIGIN"
        Referrer-Policy "strict-origin-when-cross-origin"
    }
}

Validate the syntax and reload Caddy:

sudo caddy validate --config /etc/caddy/Caddyfile
sudo systemctl reload caddy

Security Hardening and Disaster Recovery

A self-hosted photo library holds irreplaceable personal memories. Implementing automated database dumps and strict storage access policies is non-negotiable.

Automating Daily PostgreSQL Vector Backups

While backing up the library folder preserves raw files, database records contain critical tags, facial bounding boxes, user albums, and vector representations. Create an automated backup script at /usr/local/bin/backup-immich-db.sh:

#!/usr/bin/env bash
set -euo pipefail

BACKUP_DIR="/var/backups/immich"
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
mkdir -p "${BACKUP_DIR}"

# Execute PostgreSQL Dump via Docker
docker exec -t immich_postgres pg_dumpall -c -U immich_admin | gzip > "${BACKUP_DIR}/immich_db_${TIMESTAMP}.sql.gz"

# Retain only the last 14 days of backups
find "${BACKUP_DIR}" -type f -name "immich_db_*.sql.gz" -mtime +14 -delete

echo "[$(date -u)] Immich database backup completed: immich_db_${TIMESTAMP}.sql.gz"

Make the script executable and schedule it in the root crontab:

sudo chmod +x /usr/local/bin/backup-immich-db.sh
# Run every night at 02:30 AM
echo "30 2 * * * root /usr/local/bin/backup-immich-db.sh >> /var/log/immich-backup.log 2>&1" | sudo tee /etc/cron.d/immich-backup

Troubleshooting Common Operational Failures

Issue 1: Machine Learning Container Killed (OOMKilled)

Symptom: During initial photo uploads or bulk facial clustering, the immich_machine_learning container stops unexpectedly. docker inspect immich_machine_learning reports an exit code of 137 with OOMKilled: true.

Cause: High-resolution batch inference models (CLIP and InsightFace) exhaust system RAM when multiple parallel worker threads process 48-megapixel photos simultaneously.

Resolution: Lower the thread and worker concurrency in /opt/immich/.env:

MACHINE_LEARNING_WORKERS=1
MACHINE_LEARNING_REQUEST_THREADS=2

Restart the machine learning service: docker compose up -d --force-recreate immich-machine-learning.

Issue 2: Database Initialization Error (“extension ‘vectors’ does not exist”)

Symptom: immich_server logs show repeated crash loops with: QueryFailedError: type "vector" does not exist or failure to create index vectors.

Cause: Standard PostgreSQL images (e.g., official postgres:16-alpine) lack the required pgvecto.rs binary extension. Immich strictly requires PostgreSQL compiled with vector math support.

Resolution: Ensure your docker-compose.yml explicitly uses the verified vector image: tensorchord/pgvecto-rs:pg16-v0.2.1. If you previously started with vanilla Postgres, export your data, wipe the volume, and let the pgvecto-rs image re-initialize the database cluster.

Issue 3: Mobile Sync Upload Timeouts on Large 4K Videos

Symptom: The Immich mobile app fails when syncing videos over 1 GB, throwing Network Error 413: Payload Too Large or timing out after 60 seconds.

Cause: Default reverse proxy ingress parameters (such as Nginx client_max_body_size or Cloudflare free tier 100 MB limits) truncate multipart chunk streams.

Resolution: In Caddy, set max_size 50GB inside the request_body directive as shown in Step 6. If routing through Cloudflare, bypass Cloudflare proxying (DNS-only gray cloud) or use a direct VPN/Tailscale connection for original file sync to circumvent Cloudflare’s non-configurable HTTP payload limits.

Conclusion & Post-Deployment Checklist

You now have a fully private, autonomous photo and video backup infrastructure running on your own hardware. Your next operational steps should include:

  1. Admin Setup: Navigate to your domain (e.g., https://photos.yourdomain.com) to register the primary administrative account.
  2. Mobile App Pairing: Install the Immich mobile client on iOS or Android and enable foreground background backup over Wi-Fi.
  3. Face Clustering: Run the initial facial recognition and CLIP smart search job from Administration > Jobs.
  4. External Libraries: If you have an existing directory of family archives on NAS storage, mount it read-only in the Docker compose file and scan it directly without duplicating files.

With automated PostgreSQL vector backups, Redis-driven queues, and self-hosted machine learning inference, your personal and family media remains sovereign, accessible, and protected under your direct control.

Related Posts