← Back to Blog
Nov 1, 2026 · 12 min read

Self-Host n8n in Queue Mode: Postgres, Redis, and an AI Agent That Survives Real Traffic

n8nAI AgentsMCPSelf-HostedPostgres

A webhook fires, the agent starts, and the model decides it needs four tool calls. Ninety seconds later the workflow finishes. On a default n8n install, that same process was also serving the editor, the API, and every other trigger you own. Open a second tab and the UI stalls while it waits.

Queue mode splits those jobs apart. One process takes webhooks and timers, workers pull executions off a Redis queue and run them, and Postgres holds the state both sides need. Add a second worker later and throughput goes up without touching the workflow.

What follows is the setup I would deploy today: n8n 2.45.7, Postgres 18, Redis, one worker, and task runner sidecars. Then an AI Agent workflow that calls tools from an external MCP server, and the reverse case where Claude Desktop calls your n8n workflows as MCP tools.

Prerequisites

  • Docker Engine with Compose v2. Check with docker compose version.
  • A host with 4 GB RAM and 2 vCPU minimum. Each n8n process gets its own task runner sidecar, so budget for it.
  • An LLM API key for the agent step.
  • A domain and TLS if webhooks or the MCP endpoint need to be reachable from outside your machine.

Step 1: secrets first

mkdir -p ~/n8n && cd ~/n8n
openssl rand -hex 32   # N8N_ENCRYPTION_KEY
openssl rand -hex 24   # N8N_RUNNERS_AUTH_TOKEN

Put the results into .env:

N8N_VERSION=2.45.7
POSTGRES_USER=n8n
POSTGRES_PASSWORD=change-me
POSTGRES_DB=n8n
N8N_ENCRYPTION_KEY=replace-with-64-hex-chars
N8N_RUNNERS_AUTH_TOKEN=replace-with-48-hex-chars
N8N_WEBHOOK_URL=https://n8n.example.com
TZ=Asia/Jakarta

The encryption key deserves more attention than it usually gets. On first boot n8n generates one and writes it into /home/node/.n8n. Every worker and webhook process has to share that exact key, or it cannot decrypt the credentials your nodes reference. Set it explicitly, store it in your secret manager, and treat it as unrecoverable if lost: you would be re-entering every credential by hand.

Step 2: compose.yml

x-n8n-env: &n8n-env
  DB_TYPE: postgresdb
  DB_POSTGRESDB_HOST: postgres
  DB_POSTGRESDB_PORT: "5432"
  DB_POSTGRESDB_DATABASE: ${POSTGRES_DB}
  DB_POSTGRESDB_USER: ${POSTGRES_USER}
  DB_POSTGRESDB_PASSWORD: ${POSTGRES_PASSWORD}
  EXECUTIONS_MODE: queue
  QUEUE_BULL_REDIS_HOST: redis
  QUEUE_BULL_REDIS_PORT: "6379"
  N8N_ENCRYPTION_KEY: ${N8N_ENCRYPTION_KEY}
  N8N_WEBHOOK_URL: ${N8N_WEBHOOK_URL}
  GENERIC_TIMEZONE: ${TZ}
  TZ: ${TZ}
  N8N_RUNNERS_MODE: external
  N8N_RUNNERS_AUTH_TOKEN: ${N8N_RUNNERS_AUTH_TOKEN}
  N8N_RUNNERS_BROKER_LISTEN_ADDRESS: 0.0.0.0

services:
  postgres:
    image: postgres:18
    restart: always
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
      PGDATA: /var/lib/postgresql/data
    volumes:
      - db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
      interval: 5s
      retries: 10

  redis:
    image: redis:7-alpine
    restart: always
    command: ["redis-server", "--appendonly", "yes"]
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      retries: 10

  n8n-main:
    image: docker.n8n.io/n8nio/n8n:${N8N_VERSION}
    restart: always
    ports:
      - "5678:5678"
    environment:
      <<: *n8n-env
    volumes:
      - n8n_data:/home/node/.n8n
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy

  n8n-main-runner:
    image: n8nio/runners:${N8N_VERSION}
    restart: always
    environment:
      N8N_RUNNERS_AUTH_TOKEN: ${N8N_RUNNERS_AUTH_TOKEN}
      N8N_RUNNERS_TASK_BROKER_URI: http://n8n-main:5679
    depends_on:
      - n8n-main

  n8n-worker:
    image: docker.n8n.io/n8nio/n8n:${N8N_VERSION}
    restart: always
    command: ["worker", "--concurrency=10"]
    environment:
      <<: *n8n-env
      QUEUE_HEALTH_CHECK_ACTIVE: "true"
    volumes:
      - n8n_data:/home/node/.n8n
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy

  n8n-worker-runner:
    image: n8nio/runners:${N8N_VERSION}
    restart: always
    environment:
      N8N_RUNNERS_AUTH_TOKEN: ${N8N_RUNNERS_AUTH_TOKEN}
      N8N_RUNNERS_TASK_BROKER_URI: http://n8n-worker:5679
    depends_on:
      - n8n-worker

volumes:
  db_data:
  redis_data:
  n8n_data:

Three details in that file are easy to get wrong.

PGDATA on Postgres 18. The official image moved its default data directory in v18. Without the variable, the volume mount does not cover where Postgres actually writes, and you get an empty database after every restart.

The runner sidecar per process. In external mode, n8nio/runners holds the JS and Python task runners, and each n8n process needs its own. Queue mode with one worker means two sidecars: one for main, one for the worker. That is the isolation layer between user-supplied Code node code and the rest of n8n, so it is worth the extra container.

Shared n8n_data volume. Both processes mount the same volume because both need the encryption key and instance config. The database holds the workflows and executions; the volume holds the keys.

Step 3: start it and prove it works

docker compose up -d
docker compose ps
curl -sf localhost:5678/healthz && echo " main ok"
docker compose logs -f n8n-worker

Open http://localhost:5678, create the owner account, then go to Settings and pick Workers. Your worker should appear with its concurrency and current job count. An empty list means the worker failed to reach Redis, or it started with a different encryption key than main. Both failures look identical from the UI, so check docker compose logs n8n-worker before you start editing environment files at random.

Workers expose /healthz and /healthz/readiness only when QUEUE_HEALTH_CHECK_ACTIVE=true. If a load balancer needs to probe them, publish the port and point the health check there.

Step 4: an agent that calls an external MCP server

Build a workflow with a Webhook trigger and an AI Agent node. Attach two sub-nodes: a chat model (OpenAI Chat Model, Anthropic Chat Model, whichever you have a key for) and an MCP Client Tool.

On the MCP Client Tool node:

  • SSE Endpoint: the URL of your MCP server, for example https://mcp.example.com/sse.
  • Authentication: None, Bearer, Header, Multiple Headers, or OAuth2. Pick Bearer and paste a token for anything you do not fully trust.
  • Tools to Include: All, Selected, or All Except.

Choose Selected and list only what the agent needs. Every exposed tool gets described in the model's context on each turn, so a server with forty tools spends real money on tool descriptions and makes tool choice worse. Filtering at the node is cheaper than fixing it in the prompt.

Then constrain behavior in the agent's system message:

You are an operations assistant. Use the MCP tools for anything factual.
Rules:
1. Never guess a value a tool can return.
2. If a tool fails twice, stop and report the error text verbatim.
3. Answer in at most five sentences unless asked for detail.

One version note: the agent type setting on the AI Agent node was deprecated in n8n 1.82.0, and every AI Agent node now behaves as a Tools Agent. Version 1 of that node disappears in n8n 3.0, so upgrade old nodes before you migrate. If you find a tutorial telling you to pick "SQL Agent" or "Conversational Agent", it predates that change; use a database tool sub-node with a current Agent node instead.

Step 5: expose n8n itself as an MCP server

This is the direction most people miss. Drop an MCP Server Trigger into an empty workflow, attach a Custom n8n Workflow Tool pointing at the workflow you want to expose, and set authentication to Bearer. n8n gives you two URLs: a test URL that streams data into the editor while you build, and a production URL that works once the workflow is published.

n8n does not speak stdio, only SSE and streamable HTTP. For Claude Desktop, bridge it with mcp-remote:

{
  "mcpServers": {
    "n8n": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://n8n.example.com/mcp/your-path",
        "--header",
        "Authorization: Bearer ${AUTH_TOKEN}"
      ],
      "env": {
        "AUTH_TOKEN": "your-bearer-token"
      }
    }
  }
}

If nginx sits in front, the MCP location needs buffering off. SSE keeps a connection open, and a proxy that buffers will hold events until the response ends, which is never.

location /mcp/ {
    proxy_http_version          1.1;
    proxy_buffering             off;
    gzip                        off;
    chunked_transfer_encoding   off;
    proxy_set_header            Connection '';
}

Where queue mode actually bites

  • SQLite is not supported in queue mode. Run Postgres or MySQL, full stop.
  • Webhook responses travel from worker to main inside a Redis message, capped at 64 MiB by default through N8N_WEBHOOK_RESPONSE_RELAY_SIZE_MAX. The same cap applies to tool results coming back from an MCP Trigger workflow, and those cannot be offloaded.
  • N8N_WEBHOOK_URL replaced WEBHOOK_URL in n8n 2.30.0. Older versions ignore the new name.
  • n8n 2.0 removed QUEUE_WORKER_MAX_STALLED_COUNT entirely and made N8N_RUNNERS_ENABLED unnecessary.
  • With multiple webhook replicas, route every /mcp* request to a single dedicated replica. Persistent connections and round-robin do not mix.
  • Concurrency defaults to 10 per worker. Very low values spread across many workers exhaust the Postgres connection pool, which shows up as random processing delays rather than an obvious error.

When to use this vs alternatives

Pick n8n when the work is mostly integrations plus a few AI steps, and when people who do not write code need to edit the workflow. That is its real advantage, and queue mode makes it hold up under load.

Pick Dify or Flowise when the product is a chat or RAG app and the UI should be about prompts, knowledge bases, and model settings rather than HTTP nodes.

Pick Temporal or LangGraph when the workflow is core business logic that belongs in version control, with tests, code review, and no visual editor in the loop. You trade the node library for full control.

On licensing: n8n's code is source-available under the Sustainable Use License, which n8n itself does not call open source because the license restricts commercial use. Internal business use is free. White-labeling it, reselling it as an automation product, or letting your customers build workflows through your own UI is not.

If you only remember one thing

The failure mode in self-hosted n8n is rarely the workflow. It is the single process trying to serve the editor, hold webhook connections, and run a long agent chain at the same time. Split execution into workers, share the encryption key correctly, and keep the runner sidecars in external mode. The editor stays responsive and long tool calls stop being everyone else's problem.

References

Need Help Implementing This?

I help teams design and build scalable cloud infrastructure, DevOps pipelines, and production-grade systems.

Book a Free Consultation