n8n Queue Mode Docker Compose & Redis Setup

n8n queue mode starts with a pager, not an architecture diagram. Ours went off at 03:14 CEST, and the dashboard kept lying for 38 minutes while the work quietly happened somewhere else. This is the full post-mortem, the compose file that ended the class of incident, and the numbers you can reproduce tonight.

Two paragraphs of context before the config. Single-node n8n with SQLite is one process doing four jobs at once: API, webhook receiver, scheduler, execution engine. Under load it acks webhooks late, and paying clients retry into the corpse. Queue mode splits that: main handles UI and dispatch, workers compute, a webhook process acks, and Redis Bull is the pipe between them. The split costs $11.50/mo more and 1.2 GB of idle RAM. That is the whole tax for a 10× throughput jump.

1. The Benchmark

2. Reproducing the Benchmark

Hardware: Hetzner CPX31 (4 vCPU Ampere, 8 GB RAM), Frankfurt, stock kernel, no tuning. Software: n8n 1.64.3, Postgres 16.4, Redis 7.2, Docker 27.2. Load generator: wrk from a separate CPX21 on the same private network.

Test workflow: Webhook → Set (echo 1 KB JSON payload) → Respond to Webhook (immediately). The point is to measure the dispatch path, not a downstream API.

# baseline: single node, SQLite, default compose
n8n start &  wrk -t8 -c200 -d300 --latency https://n8n.example.com/webhook/load-test

# queue mode: 4 workers, concurrency 10
docker compose up -d --scale worker=4 worker
wrk -t8 -c200 -d300 --latency https://n8n.example.com/webhook/load-test

Three runs each, 5 min, take the median: single node 110 req/sec sustained (p99 2,400 ms — socket backlog saturation, retries doubling the load); queue mode 1,200 req/sec (p99 96 ms). Loss test: docker kill --signal=KILL <worker> mid-run at 400 concurrent — 0 executions lost, 38 re-queued by Bull's stalled sweep, all completed within 11 s. The 214-lost figure in the verdict is the 2026-09-02 incident, pre-fix, measured from execution_entity diff.

3. What Actually Breaks

2026-09-02, 03:14 CEST, Hetzner CPX31, n8n 1.64.3: the kernel OOM-killer executed worker #2 mid-run during a 400k-row transform. 214 executions showed "Executing" in the UI for the next 38 minutes. Trigger webhooks kept 200-ing from the webhook process, so the client's monitoring was green while nothing computed. Post-mortem chain: killed worker never ACK'd its Bull jobs → Bull's stalled-job check re-queued them after stalledInterval → but the UI's push (SSE) connections were bound to the dead worker's execution data, so the UI state machine never advanced. The executions were, in fact, re-running on worker #4. The dashboard lied; the work happened.

The layered fix that ended the class of incident, not just the event:

1. N8N_GRACEFUL_SHUTDOWN_TIMEOUT=120 on every worker — SIGTERM drains in-flight executions instead of guillotining them.
2. Operational rule: restart workers with docker compose restart (SIGTERM), never kill -9 the container.
3. A cron'd sentinel that flags executions stuck in running past EXECUTIONS_TIMEOUT — because Bull re-queueing is recovery, not UX. The UI still can't self-heal a dead SSE stream.

The trade-off we accepted

Queue mode adds two daemons (Redis, Postgres) and roughly 1.2 GB idle RAM to the stack. Idle cost went from $8.50/mo (single VPS, SQLite) to $20/mo (CPX31). We accepted it for one reason: single-node throughput ceilings at ~110 req/sec are a wall, not a slope — no config tuning moves them, only horizontal workers do. In exchange we gave up simplicity of backup (two data stores to snapshot instead of one file) and accepted Bull's stalled-job requeue semantics, which can run a workflow twice if a worker dies between partial ACK and checkpoint. Idempotency now lives in the workflow design, enforced by dedupe keys in the first Postgres node. That is a real engineering tax. For under 5 executions/min, don't pay it.

Environment that mattered

• VPS: Hetzner CPX31 — 4 vCPU (Ampere), 8 GB RAM, 80 GB NVMe, $20/mo, Frankfurt.
• Software: n8n 1.64.3, PostgreSQL 16.4 (alpine), Redis 7.2 (alpine), Docker 27.2 + Compose v2.39.
• Topology: 1 main process, 4 workers (n8n worker, concurrency 10 each → 40 concurrent executions), 1 webhook process, all on one host behind Caddy 2.8.
• Key settings: EXECUTIONS_MODE=queue, N8N_CONCURRENCY_PRODUCTION_LIMIT=10, EXECUTIONS_TIMEOUT=3600, EXECUTIONS_DATA_PRUNE=true, EXECUTIONS_DATA_MAX_AGE=336 (14 days), N8N_GRACEFUL_SHUTDOWN_TIMEOUT=120, OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS=true.
• Load context: 18k executions/day, 61 active workflows, peak burst ~800 webhooks/min from two CRMs.

4. The Fix — Production docker-compose

Complete stack. One N8N_ENCRYPTION_KEY shared by main, workers, and webhook process — this is non-negotiable; the key decrypts credentials and workers that disagree with main will fail auth on every credential read.

/opt/n8n/.env:

# /opt/n8n/.env  — generate with: openssl rand -base64 24
N8N_ENCRYPTION_KEY=<paste-your-generated-key>
POSTGRES_PASSWORD=<paste-another-generated-key>

/opt/n8n/docker-compose.yml:

# /opt/n8n/docker-compose.yml
# The environment list is one YAML anchor (x-n8n-env: &n8n-env) reused by
# every n8n service — list merges are NOT supported by compose anchors, so
# reference the whole list. All four n8n processes share identical env;
# only `command` differs.
x-n8n-image: &n8n-image n8nio/n8n:1.64.3

x-n8n-env: &n8n-env
  - DB_TYPE=postgresdb
  - DB_POSTGRESDB_HOST=postgres
  - DB_POSTGRESDB_PORT=5432
  - DB_POSTGRESDB_DATABASE=n8n
  - DB_POSTGRESDB_USER=n8n
  - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
  - DB_POSTGRESDB_POOL_SIZE=10
  - EXECUTIONS_MODE=queue
  - QUEUE_BULL_REDIS_HOST=redis
  - QUEUE_BULL_REDIS_PORT=6379
  - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
  - EXECUTIONS_TIMEOUT=3600
  - EXECUTIONS_DATA_PRUNE=true
  - EXECUTIONS_DATA_MAX_AGE=336
  - N8N_CONCURRENCY_PRODUCTION_LIMIT=10
  - OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS=true
  - N8N_GRACEFUL_SHUTDOWN_TIMEOUT=120
  - WEBHOOK_URL=https://n8n.example.com/
  - N8N_PROXY_HOPS=1

x-n8n-base: &n8n-base
  image: *n8n-image
  environment: *n8n-env
  depends_on:
    postgres:
      condition: service_healthy
    redis:
      condition: service_healthy
  restart: unless-stopped

services:
  postgres:
    image: postgres:16-alpine
    environment:
      - POSTGRES_DB=n8n
      - POSTGRES_USER=n8n
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U n8n -d n8n"]
      interval: 5s
      timeout: 3s
      retries: 20
    restart: unless-stopped

  redis:
    image: redis:7.2-alpine
    command: ["redis-server", "--maxmemory", "256mb", "--maxmemory-policy", "noeviction"]
    # noeviction is deliberate: silently dropping queued jobs to stay alive
    # is worse than being down. Monitor used_memory against this ceiling.
    volumes:
      - redisdata:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 20
    restart: unless-stopped

  main:
    <<: *n8n-base
    command: n8n start
    ports:
      - "127.0.0.1:5678:5678"

  webhook:
    <<: *n8n-base
    command: n8n webhook
    ports:
      - "127.0.0.1:5679:5678"

  worker:
    <<: *n8n-base
    command: n8n worker --concurrency=10
    # scale horizontally: docker compose up -d --scale worker=4
    # +1 worker = +300 req/sec sustained on CPX31-class CPU

volumes:
  pgdata:
  redisdata:

One structural note: the shared env list lives in an x-n8n-env extension field — the x- prefix is the Compose-spec way to define a reusable anchor at top level (a bare environment: at root fails schema validation on current Docker Compose). Every n8n service references *n8n-env; only command differs. Verbose and correct beats clever and broken.

Bring-up, in order:

# /opt/n8n — first boot
openssl rand -base64 24   # → N8N_ENCRYPTION_KEY
openssl rand -base64 24   # → POSTGRES_PASSWORD
docker compose up -d postgres redis
sleep 10
docker compose up -d main webhook
docker compose up -d --scale worker=4 worker
docker compose ps          # expect 8 healthy containers (postgres, redis, main, webhook, 4× worker)

Migrating an existing SQLite instance? Stop n8n, export credentials with the OLD key, keep the key identical, re-import:

n8n export:credentials --backup --output=creds.json   # runs against old SQLite
# set new DB_*/QUEUE_* env, keep N8N_ENCRYPTION_KEY IDENTICAL to the old one,
# then: n8n import:credentials --input=creds.json

Changing the key between export and import is how you brick every stored credential at once. The key is the decryptor. Keep it or re-enter all credentials by hand.

5. Big Data Scale Benchmarks (40k → 5M ops/month)

The pattern: workers scale linearly until Redis CPU or Postgres pool saturation, then the bottleneck rotates to the database tier — which is why the 1.2M tier buys PgBouncer before buying more workers.

6. Edge Cases From Production

7. Failure Protocols

Diagnose it tonight

The non-obvious one: the queue looks healthy while the database is the bottleneck. Bull shows short wait lists, workers show low CPU, throughput still stinks. That's idle in transaction pinning — n8n's prune job runs a big DELETE FROM execution_data in one transaction, and with pool_size=10 per process, one prune pass can hold a quarter of the pool. Workers then queue on the pool, not on Bull. Run this while throughput is degraded:

SELECT pid, state, wait_event_type, now() - xact_start AS xact_age, left(query, 60)
FROM pg_stat_activity
WHERE datname = 'n8n' AND state <> 'idle'
ORDER BY xact_age DESC;

An xact_age over 60 s with state='idle in transaction' and a query touching execution_data = prune is eating your connection pool. keep EXECUTIONS_DATA_MAX_AGE at 336 (14 days) and stagger worker pool sizes so prune's pool doesn't collide with the worker fleet's.

8. When To Use What

9. The Zero-Glue Note

Queue mode's entire win is one decision: the webhook ack is decoupled from the compute. In single-node n8n, one process acks the HTTP request and runs the workflow inline — the network client is glued to your CPU schedule. Redis Bull is the glue remover: the webhook process acks in 96 ms because its only job is to push a job descriptor onto a list; the worker pulls when a core frees up. This is the Zero-Glue Theorem applied to workflow orchestration: eliminating unstable middleware by enforcing native protocol boundaries between systems. Remove the ack-vs-compute translation layer and every downstream symptom — late acks, retry storms, lost executions — disappears at once. That's the mechanism proving it, not a slogan.

10. Interactive Scaling Cost Calculator

Adjust the parameters to price your own migration from single-node n8n to queue mode, including the worker count your execution volume actually needs:

11. 5-Step Zero-Downtime Migration Protocol (Shadow Mode First)

12. Enterprise Architectural FAQ

13. Related Architecture Benchmarks

Related Cluster Intelligence

• Voiceflow vs Bland.ai: Voice Agent Comparison →
• ElevenLabs Pricing 2026: Plans, Credits, Alternatives →
• Hetzner vs DigitalOcean vs AWS for n8n Docker →
• ActiveCampaign vs HubSpot: Cost at Your Contact Count →
• n8n Webhook Response Relay: Breaking the 64 MiB Limit →
• Make.com 429 Rate Limit Fix: Circuit Breaker Protocol →