Transmission 024 ended with a clear list of what was missing. PostgreSQL was in the stack, idling, nothing written to it. No schema, no status endpoint, no way to ask what happened to a task after the worker ACKed it. Today was supposed to close that gap.

It took three separate blockers before the containers ran.

The constraint

The persistence layer needed four things: a schema, an async database helper module, status transitions in the worker, and a new GET /tasks/{task_id} endpoint in the gateway. None of it is complicated. The work was adding asyncpg as a connection pool, writing the SQL, wiring the lifecycle into FastAPI’s lifespan context manager, and converting the worker’s main loop from synchronous to asyncio so it could await database calls without blocking the Redis read.

The Dockerfiles were the problem. They had been written on day one, before the database module existed. Each one named its files explicitly:

# gateway/Dockerfile
COPY main.py .

# worker/Dockerfile
COPY worker.py .

db.py was never going to make it into either image.

The proof

What was built

postgres/init.sql — the schema. One table, mounted into /docker-entrypoint-initdb.d/ so Postgres runs it on first container start.

CREATE TABLE IF NOT EXISTS tasks (
    task_id    VARCHAR(64)  PRIMARY KEY,
    name       VARCHAR(128) NOT NULL,
    status     VARCHAR(32)  NOT NULL DEFAULT 'pending',
    payload    JSONB,
    result     JSONB,
    created_at TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ  NOT NULL DEFAULT NOW()
);

CREATE INDEX IF NOT EXISTS idx_tasks_status ON tasks (status);

gateway/db.py and worker/db.py — async helpers built on asyncpg.create_pool. The gateway gets create_task, update_task_status, and fetch_task. The worker gets only update_task_status — it never inserts or queries, it only transitions state. asyncpg was chosen over psycopg3 here not for performance reasons but because asyncpg has the cleaner pool API for this exact pattern: one call to create_pool, then async with pool.acquire() as conn per query, no connection juggling.

gateway/main.py — rewritten to async. FastAPI’s lifespan context manager initialises the pool at startup and closes it on shutdown. POST /tasks now awaits a database insert before calling xadd — the task is in Postgres as pending before it hits the Redis stream. A new GET /tasks/{task_id} endpoint queries Postgres and returns the full row, or HTTP 404 if not found.

worker/worker.py — converted to asyncio. The main loop runs under asyncio.run(). The three state transitions are explicit:

# 1. Message delivered from Redis
await db.update_task_status(task_id, "processing")

try:
    result = await process_task(task_id, fields)

    # 2a. Success
    client.xack(STREAM_NAME, CONSUMER_GROUP, msg_id)
    await db.update_task_status(task_id, "completed", result=result)

except Exception as exc:
    # 2b. Failure — do NOT xack, message stays in PEL
    await db.update_task_status(task_id, "failed", result={"error": str(exc)})

docker-compose.yml — two changes. The postgres service got the init.sql volume mount. The worker service got Postgres environment variables and a postgres: service_healthy condition added to depends_on. Previously the worker only waited for Redis.

First build — 234 seconds, containers exited immediately

The first docker compose down -v && docker compose up --build ran clean. Both images built in 234 seconds — the cold download of asyncpg and its C extension dominated.

[+] up 9/9
 ✔ Image nexusflow-gateway         Built   234.8s
 ✔ Image nexusflow-worker          Built   234.8s
 ✔ Network nexusflow_nexusflow_net Created
 ✔ Volume nexusflow_postgres_data  Created
 ... 5 more

Then:

$ curl -X POST http://localhost:8000/tasks \
  -H "Content-Type: application/json" \
  -d '{"name": "generate_report", "payload": {"user_id": 42}}'
curl: (7) Failed to connect to localhost:8000 after 2278 ms: Could not connect to server

$ docker compose ps
NAME      IMAGE     COMMAND   SERVICE   CREATED   STATUS    PORTS

Empty. Both application containers had exited the moment they started.

Blocker one — db.py was not in the image

The gateway logs told the story immediately:

  File "/app/main.py", line 23, in <module>
    import db
ModuleNotFoundError: No module named 'db'

Same from the worker:

  File "/app/worker.py", line 40, in <module>
    import db
ModuleNotFoundError: No module named 'db'

Both Dockerfiles had been written on day one. The runtime stage of each one copied only what existed at the time:

# gateway — day one
COPY main.py .

# worker — day one
COPY worker.py .

db.py was created today. It was never added to either COPY instruction. The images went out with /app/main.py and nothing else.

The fix was switching from explicit per-file copies to COPY . . — the entire build context in one instruction. To keep the images clean, .dockerignore files went into each service directory rather than the repo root. The build contexts are ./gateway and ./worker, so Docker looks for .dockerignore relative to those paths, not the project root.

# after — both Dockerfiles
COPY . .

# gateway/.dockerignore  (worker/.dockerignore is identical)
__pycache__/
*.pyc
*.pyo
*.pyd
venv/
.venv/
.env
*.env.*
.git/
.DS_Store
.idea/
.vscode/
.pytest_cache/
htmlcov/
.coverage

With COPY . ., any module added to the service directory is automatically in the next image. No Dockerfile update required.

Blocker two — PostgreSQL crash recovery after unclean shutdown

After the Dockerfile fix, docker compose build --no-cache rebuilt both images cleanly, then docker compose up -d gave this:

[+] up 4/4
 ✔ Container nexusflow-worker-1   Recreated
 ✔ Container nexusflow-gateway-1  Recreated
 ✘ Container nexusflow-postgres-1 Error  dependency postgres failed to start  50.0s
 ✔ Container nexusflow-redis-1    Healthy
dependency failed to start: container nexusflow-postgres-1 is unhealthy

The docker compose logs postgres output explained it. The session earlier had killed Postgres mid-operation — docker compose down -v without giving it time to write a clean shutdown checkpoint. On restart it had to recover:

postgres-1  | 2026-08-25 06:59:09.614 UTC [28] LOG:  database system was interrupted;
             last known up at 2026-08-25 06:32:17 UTC
postgres-1  | 2026-08-25 07:00:20.459 UTC [28] LOG:  syncing data directory (fsync),
             elapsed time: 70.75 s, current path: ./base/1/2670

The fsync walk took 90 seconds. Docker’s health check fires every 10 seconds with 5 retries and a 10-second start period — 60 seconds total before it declares the container unhealthy. Postgres was still syncing at second 61. The gateway and worker tried to start, saw postgres unhealthy, and refused.

docker compose restart postgres let it finish the recovery. After that:

$ docker compose up -d
[+] up 2/2
 ✔ Container nexusflow-redis-1    Healthy
 ✔ Container nexusflow-postgres-1 Healthy

Then docker compose up -d again brought the gateway and worker up.

Blocker three — the curl health check has nothing to curl

$ docker compose ps
NAME                   STATUS
nexusflow-gateway-1    Up (unhealthy)
nexusflow-postgres-1   Up (healthy)
nexusflow-redis-1      Up (healthy)
nexusflow-worker-1     Up

The gateway was running but marked unhealthy. The logs showed it was fine:

gateway-1  | INFO:     Application startup complete.
gateway-1  | INFO:     Uvicorn running on http://0.0.0.0:8000

The health check in docker-compose.yml was the original from day one:

test: ["CMD-SHELL", "curl -f http://localhost:8000/health || exit 1"]

python:3.12-slim does not ship curl. The probe ran, got command not found, exited non-zero, and Docker marked the container unhealthy on every cycle. The gateway had been “unhealthy” since day one — it just had not mattered until now because the worker’s depends_on never referenced it.

Replaced it with a Python stdlib one-liner that requires nothing beyond what is already in the image:

test: ["CMD-SHELL", "python -c \"import urllib.request, sys; urllib.request.urlopen('http://localhost:8000/health', timeout=4); sys.exit(0)\""]

urlopen raises on non-2xx responses or connection failures. Python exits with code 1 from the uncaught exception. Docker reads that as unhealthy. No extra package, no shell dependency.

Final build — gateway only, 50 seconds, all layers cached

Only the gateway needed a rebuild for the health check change. The COPY . . layer was already cached — the source files had not changed. Docker reused everything and finished in 50 seconds.

[+] up 4/4
 ✔ Image nexusflow-gateway        Built                                           50.5s
 ✔ Container nexusflow-postgres-1 Healthy                                         17.6s
 ✔ Container nexusflow-redis-1    Healthy                                         17.6s
 ✔ Container nexusflow-gateway-1  Recreated                                       16.8s

NAME                   IMAGE              STATUS                        PORTS
nexusflow-gateway-1    nexusflow-gateway  Up About a minute (healthy)   0.0.0.0:8000->8000/tcp
nexusflow-postgres-1   postgres:16-alpine Up 18 minutes (healthy)       5432/tcp
nexusflow-redis-1      redis:7-alpine     Up 22 minutes (healthy)       6379/tcp
nexusflow-worker-1     nexusflow-worker   Up 16 minutes

All four containers up. Gateway healthy. Postgres healthy. Redis healthy.

Where this sits

The stack is running. The schema is there, the connection pools are open, the state machine is wired. What is not confirmed yet is whether a task submitted through the gateway actually lands in the database, gets picked up by the worker, and ends up in the tasks table with status = 'completed'. That is one curl command and one psql query away.