Beta · This site is in preview and not yet officially launched.
Tech47 min read

Self-hosted infrastructure on Docker + nginx + Cloudflare Tunnel: zero-trust without the AWS bill

A 502 at 3:51 AM because the deploy script never reloaded nginx. The Docker + nginx + Cloudflare Tunnel stack that runs production on a single VPS.

BIM Mobius
with DTDucas

The Cloudflare alert arrived at 3:51 AM. Subject line: "Origin returned 502 for 47 seconds — tunnel health degraded."

Forty-seven seconds. Long enough for the marketing site's homepage to be unreachable. Long enough for the automated uptime monitor to page you. Long enough, if you were unlucky, for a potential customer who clicked a newsletter link to bounce and never come back.

The deploy script had run without error. The CI pipeline was green. Docker showed all four containers in Up state. nginx was responding to docker exec health checks. Cloudflare's own dashboard showed the tunnel as "Healthy." And yet — the marketing site had returned 502 for 47 seconds during a deploy you had tested locally a dozen times.

Here is what actually happened. The deploy script stopped and recreated the nginx container with a new config that routed /api to the new API container. It then restarted the cloudflared tunnel container. But the old nginx container had been holding the tunnel's outbound connection. When that container was torn down, cloudflared briefly lost its upstream and started returning 502 to Cloudflare's edge. The new nginx container came up and cloudflared reconnected — but for 47 seconds, the ordering was wrong.

Meanwhile, a stale nginx config had been bind-mounted from a subdirectory that the deploy script had updated on disk, but had not signalled nginx to reload. The running nginx process was still reading its in-memory compiled configuration from the previous deploy. Two routes were silently swapped: /admin was routing to the marketing app, and the marketing app's root was routing to the admin panel. No error. No crash. Just wrong.

This is the rake that self-hosted infrastructure hides in the grass. The individual tools — Docker, nginx, cloudflared — are all documented well. The interactions between them at deploy time are not. You get paged at 3:51 AM not because any single tool failed, but because you did not control the ordering, signalling, and mount lifecycle of three tools that each assume they are the only moving part.

This post is about building that control. Not in the abstract — with real deploy scripts, real compose files, real nginx config, and the specific sequence that avoids the 3:51 AM wake-up. The target audience is anyone running production workloads on a single VPS, a bare-metal box, or a home lab, who wants zero-trust networking via Cloudflare Tunnel, without an AWS bill, a Kubernetes cluster, or a dedicated DevOps engineer.

Everything here is generic. The patterns apply to any containerised web stack. Adapt as needed.


What's actually happening

Understanding the outage above requires understanding how three systems interact: Cloudflare Tunnel, nginx, and Docker Compose. Each has its own mental model. The failure modes come from the gaps between them.

Cloudflare Tunnel: the trust inversion

Traditional public hosting punches holes in your firewall. You open port 443, maybe 80, and the internet can reach your origin. Cloudflare Tunnel inverts this. Your origin server establishes an outbound persistent connection to Cloudflare's edge — no inbound ports required. All public traffic arrives via Cloudflare's network, traverses the tunnel, and lands at your origin. The public internet cannot reach your server directly because no inbound port is open.

The client that manages this connection is cloudflared. It is a long-running daemon — or in a containerised setup, a long-running container — that maintains between four and eight redundant outbound connections to the nearest Cloudflare PoP. Each connection uses QUIC or H/2 depending on what the network supports. When a user hits your domain, Cloudflare's edge proxies the request down one of those connections to your origin.

The cloudflared process reads a config.yml that maps hostnames to local upstream services. A minimal config looks like this:

tunnel: YOUR_TUNNEL_UUID
credentials-file: /etc/cloudflared/credentials.json
 
ingress:
  - hostname: app.example.com
    service: http://nginx:80
  - hostname: api.example.com
    service: http://nginx:80
  - service: http_status:404

In a Docker Compose setup, nginx is the service name. Cloudflare Tunnel reaches it via Docker's internal DNS. The public internet never touches your host's network interface directly.

This model has a concrete security property: your origin's IP address is not published in DNS. Cloudflare's Anycast IPs appear in DNS, not yours. An attacker who discovers your server's IP still cannot reach port 443 — because port 443 is not open. They can reach whatever ports you have open (SSH on a non-standard port, or nothing at all), but they cannot bypass Cloudflare's WAF by hitting the origin directly. This is meaningful protection that a traditional setup with Cloudflare as a "orange cloud" proxy does not provide, because that setup does have 443 open on the origin and an attacker can discover the origin IP via various techniques and bypass Cloudflare entirely.

The Cloudflare Tunnel documentation covers the cloudflared tunnel run subcommand, the credentials file format, and the ingress rules syntax. Read the cloudflared subcommand reference before setting up tunnel routing — the ingress rule matching is order-dependent (first match wins) and the fallback rule must be present or the daemon refuses to start.

nginx: the missing signal

nginx is a reverse proxy that sits between cloudflared and your application containers. In a typical setup, cloudflared sends all traffic to http://nginx:80, and nginx routes to the correct upstream based on the Host header or the path.

The configuration that matters is how nginx learns about config changes. The default behavior: nginx reads its configuration at startup and compiles it into memory. If you change nginx.conf on disk and do not send nginx a signal, the running process continues using the in-memory compiled configuration from startup. Your bind-mounted config file is updated. The running process does not care.

The correct signal is SIGQUIT for graceful shutdown or SIGUSR1 (equivalent to nginx -s reload) for a configuration reload without dropping connections. When nginx receives SIGUSR1, it re-reads the configuration, validates it, and replaces worker processes gracefully — existing connections finish on the old workers, new connections go to new workers. This takes typically 100–500 ms.

The relevant nginx documentation is the section on controlling nginx (the "Signals" subsection of the admin guide). The key distinction: SIGHUP causes a reload, SIGTERM causes immediate shutdown, SIGQUIT causes graceful shutdown. When you run nginx -s reload inside a container, you are sending SIGHUP to the master process.

The rake: if your deploy script bind-mounts a new nginx.conf but never signals nginx to reload, your production traffic continues on the old config. No error. No warning. The process is healthy, the file is updated, but the behavior is stale.

Docker Compose: the container lifecycle states

Docker containers can be in several states. Understanding these states is essential for reasoning about what happens during a deploy.

A container that is Created exists as metadata and a writeable layer, but no process is running. A container that is Running has a live PID-1 process. A container that is Restarting had its PID-1 exit and Docker's restart policy is re-launching it. A container that is Recreated — this is a Compose concept, not a Docker daemon concept — means Compose determined the container's configuration differed from the Compose file and tore down the old container and created a new one.

The Recreated state is the dangerous one. When Compose recreates a container, it tears down the old container and brings up a new one. Any in-memory state in the old container is lost. Any outbound connections the old container held are dropped. Any file descriptors the old container had open are closed. If other services depended on those connections — as cloudflared depends on nginx being reachable — there is a window where the dependency is broken.

The Docker documentation on Compose project state management describes when Compose recreates versus restarts a container. Compose recreates a container when the image has changed, when any configuration in the Compose file has changed (environment variables, volumes, networks, ports, command), or when the --force-recreate flag is passed. Compose restarts a container (without recreating) only when docker compose restart is called explicitly.

This matters for ordering. If your deploy script runs docker compose up -d after changing a service's config, Compose will recreate that service's container. Recreating nginx tears down the old nginx container. cloudflared, which had an open connection to the old nginx container, will lose that connection. If cloudflared is not restarted or signalled, it may continue trying to reach a container that no longer exists.

Bind mounts vs named volumes vs tmpfs

The Docker volume documentation describes three storage options for container data.

A bind mount maps a host filesystem path into the container. Changes made on the host are immediately visible inside the container (and vice versa). This is correct for configuration files you want to manage in version control and update without rebuilding the image. The catch is that the running process inside the container does not automatically react to changes — it only sees them if it re-reads the file. nginx's in-memory config drift is exactly this problem.

A named volume is managed by Docker. It persists across container recreations. Data written by the container survives a docker compose down and is available when the container comes back up. This is correct for database data, uploaded files, and any state that must survive a deploy. If you put your Postgres data in a named volume, docker compose up -d --force-recreate will recreate the postgres container but the data volume survives.

A tmpfs mount is memory-only. Data disappears when the container stops. This is correct for secrets that should never touch disk — session tokens, in-progress encryption keys. It is also correct for build artifacts you want to exist during a build step but not in the final image.

The common mistake is using a named volume where a bind mount is correct (you can no longer edit the config from the host) or a bind mount where a named volume is correct (you accidentally delete your database by running docker compose down -v).

The cloudflared container and the credentials file

cloudflared requires a credentials JSON file that contains the tunnel's service token. This file is sensitive — it authorises anyone who has it to route traffic through your tunnel. It should be bind-mounted read-only into the cloudflared container from a path that is not in version control, or passed via a Docker secret.

The minimal Compose service for cloudflared looks like this:

services:
  cloudflared:
    image: cloudflare/cloudflared:2024.12.2
    restart: unless-stopped
    command: tunnel run
    volumes:
      - ./cloudflared/config.yml:/etc/cloudflared/config.yml:ro
      - /run/secrets/cloudflared-creds.json:/etc/cloudflared/credentials.json:ro
    depends_on:
      nginx:
        condition: service_healthy
    networks:
      - internal

The depends_on with condition: service_healthy is critical. It prevents cloudflared from starting until nginx has passed its health check. Without it, cloudflared starts, tries to connect upstream, fails because nginx is still initialising, and logs a connection error. With a restart policy of unless-stopped, Docker will keep restarting it until nginx is up — but there is a window of failed requests during each restart cycle. The health check eliminates that window.

How Docker networking underpins all of this

Every service defined in a Compose file is attached to the Compose project's default network unless you specify otherwise. On that network, Docker provides internal DNS: a container with name: nginx in the Compose file is reachable at hostname nginx from any other container on the same network. This is how cloudflared reaches nginx at http://nginx:80 without knowing the container's IP address, which changes every time the container is recreated.

This internal DNS is only available within the Docker network. It is not exposed outside the host. A process running on the host, outside any container, cannot resolve nginx as a hostname — it has no access to Docker's internal DNS. This is relevant when writing health check scripts that run on the host: they cannot use Docker's internal service names. Use docker compose exec or docker compose port to reach a service from the host.

The network isolation also provides a security boundary. Containers on the internal network can reach each other. They cannot be reached from the host's public network interface unless a port mapping is defined in the Compose file. This is why the deploy does not require port mappings for nginx — cloudflared reaches nginx via the internal network, and cloudflared itself is reached by Cloudflare's edge via its outbound tunnel connection. No inbound port mapping is needed for any service except potentially SSH on the host itself.

This request path has implications for header handling, which the nginx configuration must account for. The X-Forwarded-Proto header from cloudflared to nginx will be http (because the internal leg is HTTP), but the original protocol was https. Applications that redirect HTTP to HTTPS based on this header will see the wrong value and enter a redirect loop. The nginx config must explicitly set X-Forwarded-Proto: https before proxying to upstream.

Similarly, the X-Real-IP and X-Forwarded-For headers from cloudflared will contain Cloudflare's edge IP, not the client's IP, unless nginx is configured with the ngx_http_realip_module and the Cloudflare IP ranges are in the set_real_ip_from directives. This matters for rate limiting, geo-restriction, and access logging.

The health check contract

The health check system in Docker Compose is what makes depends_on: condition: service_healthy useful. Without a well-designed health check, the condition is meaningless — a container that starts a process immediately passes started, but healthy requires the container to return a successful response from the defined health check command.

For a web service, the health check should probe the actual HTTP endpoint, not just verify the process is running:

# Weak: just checks the process exists
test: ["CMD", "pgrep", "node"]
 
# Correct: checks the HTTP endpoint responds 200
test: ["CMD-SHELL", "wget -qO- http://localhost:3000/health | grep -q 'ok' || exit 1"]

The /health endpoint in your application should be lightweight: no database queries, no external calls. It should return 200 if the application process is running and able to accept requests. Reserve dependency checks (database connectivity, external API availability) for a separate /health/ready endpoint that you query manually or in monitoring dashboards, not in the container's Docker health check. A Docker health check that fails because an external dependency is slow will cause Docker to mark the container unhealthy and potentially restart it, creating a thundering-herd problem.

The start_period in the health check configuration is important. It sets a grace period after container start during which failed health checks do not count as failures. A Node.js application that takes 5 seconds to start should have a start_period of at least 10 seconds to avoid premature restarts during normal startup.


The pattern

Self-hosted Docker + nginx + Cloudflare Tunnel composes into a handful of distinct deployment topologies. Understanding which one you are in determines the failure modes you need to defend against.

Single-box: everything on one VPS

This is the most common starting point. A 4 vCPU / 8 GB VPS runs all services: the application server, the database, nginx, cloudflared, and any background workers. The Compose file is a single docker-compose.yml. All services share one Docker network. Communication between services uses Docker's internal DNS — http://api:3000, http://postgres:5432, not localhost.

The advantage is simplicity. One box, one deploy script, one set of logs. The disadvantage is a single point of failure and resource contention between services during peak load.

For many production workloads — a B2B SaaS with under a thousand daily active users, a marketing site with occasional traffic spikes, an internal tool — a single-box setup is entirely sufficient. The key is not the topology; it is the reliability of the deploy process.

Per-environment Compose files handle the configuration differences between staging and production without duplicating the base file. Use Docker Compose's --file flag to compose multiple files:

# Staging
docker compose -f docker-compose.yml -f docker-compose.staging.yml up -d
 
# Production
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

The base file defines services, networks, and volumes with sensible defaults. The environment-specific override files set image tags, resource limits, environment variables, and replica counts. This pattern keeps the diff between environments explicit and reviewable.

Two-box: database on a separate host

Separating the database onto its own host removes resource contention and simplifies backup strategy. The application box runs nginx, cloudflared, the app container, and any workers. The database box runs Postgres (or MySQL, or whatever) on Docker with a named volume for data persistence.

Communication between boxes travels over a private network — a VPS provider's VLAN, a WireGuard tunnel between hosts, or Tailscale (which is itself built on WireGuard). The application's DATABASE_URL environment variable points to the database host's private IP or Tailscale hostname.

The Cloudflare Tunnel stays on the application box. The database box is not publicly routable at all — no tunnel, no exposed ports, just the private network interface. This is a correct layering: public traffic enters via Cloudflare Tunnel, hits nginx, hits the app, and the app speaks to the database over a private channel that the public internet cannot reach.

Multi-region: active-passive failover

For teams that need geographic redundancy without the complexity of active-active, active-passive on two VPS instances in different regions is tractable. The primary box runs the full stack. The secondary box runs the stack in standby with database replication (Postgres streaming replication, for example).

Cloudflare Tunnel supports routing policies that can be configured to fail over to the secondary tunnel if the primary goes unhealthy. This is not instantaneous — Cloudflare's health checks have a polling interval — but it provides recovery without DNS changes.

The operational complexity is the database replication lag and the failover procedure. This is beyond the scope of a single-box deploy guide. Note it as a migration path, not a starting point.

Home lab: replacing dynamic DNS

A home lab setup has one additional complication: your ISP assigns a dynamic public IP, and even if you wanted to open inbound ports, NAT and CGNAT often prevent it. Cloudflare Tunnel eliminates both problems. Your home lab server needs no inbound ports and no static IP. It just needs outbound internet access to reach Cloudflare's PoPs.

This makes home lab hosting production-viable for low-to-moderate traffic workloads. The constraint is upload bandwidth and the SLA your ISP provides (typically none). For an internal tool, a personal project, or a staging environment, this is entirely acceptable.

Secrets management without a vault

Without HashiCorp Vault or AWS Secrets Manager, secrets management is a compose of simpler primitives.

For secrets that change rarely (database passwords, API keys for third-party services), a .env file that is gitignored and deployed via a secrets manager or secure copy is acceptable. Use dotenv syntax in the Compose file:

services:
  api:
    env_file:
      - .env.production

The .env.production file is never committed to version control. It is stored in a password manager and copied to the server during initial setup. It is not rotated frequently because rotation on a single-box setup requires a deploy.

For secrets that need to be available at runtime without a deploy — JWT signing keys, session secrets — Docker secrets (backed by a tmpfs mount) provide in-memory storage. The trade-off is complexity; for many single-box setups, the .env file approach with proper file permissions (chmod 600) is pragmatically sufficient.

For the cloudflared credentials JSON specifically: the file should be owned by root, readable only by root, and bind-mounted read-only into the cloudflared container. It should not be in version control. Keep it in the password manager entry for the server.

Blue-green vs rolling restart on a single host

On a Kubernetes cluster, blue-green and rolling restarts are infrastructure primitives. On a single Docker host, they are deploy-script problems.

A rolling restart on a single host is simple: stop the old container, start the new one. There is a window of downtime (typically a few hundred milliseconds to a few seconds) between the old container stopping and the new container passing its health check. For most self-hosted use cases, this is acceptable.

A blue-green deploy on a single host requires running two containers simultaneously and switching traffic. With nginx, this means having an nginx upstream block that points to either the "blue" or "green" container, and switching the upstream at deploy time. This eliminates the downtime window at the cost of double the memory during the switch.

upstream app_blue {
    server app-blue:3000;
}
 
upstream app_green {
    server app-green:3000;
}
 
# Swap this line at deploy time
upstream app_active {
    server app-blue:3000;
}

For most single-box setups, the rolling restart with a well-configured health check is the right trade-off. A few hundred milliseconds of downtime during a planned deploy, at 2 AM, is not the same as 47 seconds of downtime from a misconfigured deploy at 3:51 AM.

Environment variable strategy

Environment variables are how Compose files remain generic while deployments remain specific. The pattern is three layers:

  1. Defaults in the Compose file — values that are safe to be public and are the same everywhere (e.g., PORT=3000).
  2. Environment-specific overrides — a .env file per environment that is not committed to version control. The Compose file references ${VARIABLE_NAME} and Compose reads the .env file from the same directory.
  3. Build-time variables — passed as --build-arg during docker build, baked into the image. These are for values that must exist at compile time (asset hashing, feature flags that affect the build output). They should not be secrets because they end up in the image layer metadata.

The .env file pattern is simple and sufficient for single-box setups. The limitation is that the file must be transferred to the server out-of-band during initial setup and updated out-of-band when secrets change. Use a password manager that supports secure notes or file attachments. The file should be chmod 600 and owned by the user that runs Docker.

# .env.production — never commit this
POSTGRES_DB=app_production
POSTGRES_USER=app_user
POSTGRES_PASSWORD=a-long-random-string-here
API_VERSION=1.4.2
WEB_VERSION=2.1.0
REGISTRY=ghcr.io/your-org
JWT_SECRET=another-long-random-string
SMTP_PASSWORD=yet-another-secret

The deploy script sources this file to get the version tags before running the deploy. The Compose file reads it for runtime environment variables.

Structuring the Compose file for maintainability

As the stack grows, a single docker-compose.yml becomes hard to read. The Docker Compose merge spec allows multiple files to be layered:

# Production deploy
docker compose \
  -f docker-compose.yml \
  -f docker-compose.prod.yml \
  up -d --remove-orphans

docker-compose.yml defines services, networks, and volumes with minimal configuration. docker-compose.prod.yml adds production-specific settings: resource limits, restart policies, logging configuration, and pinned image tags.

# docker-compose.prod.yml
services:
  api:
    image: ${REGISTRY}/api:${API_VERSION}
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: "1.5"
          memory: 512M
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "5"
 
  postgres:
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M

Resource limits prevent a single container from starving others. On a 4 vCPU / 8 GB VPS running five containers, uncontrolled memory growth in one container can cause the OS to invoke the OOM killer on others. The OOM killer does not discriminate — it kills the process with the highest memory-to-importance ratio, which may well be your database.


The fix / approach

This is the actionable section. What follows is an idempotent deploy script, the Docker and Compose configuration it depends on, the nginx config structure, and the ordering that prevents the 3:51 AM outage.

  1. Write the Dockerfile: multi-stage build

    Multi-stage builds produce smaller production images by separating build-time dependencies from runtime dependencies. A Node.js application that needs typescript, esbuild, and a hundred development packages at build time needs none of those in the production image.

    # Stage 1: dependencies
    FROM node:20-alpine AS deps
    WORKDIR /app
    COPY package.json package-lock.json ./
    RUN npm ci --only=production
     
    # Stage 2: build
    FROM node:20-alpine AS build
    WORKDIR /app
    COPY package.json package-lock.json ./
    RUN npm ci
    COPY . .
    RUN npm run build
     
    # Stage 3: production image
    FROM node:20-alpine AS production
    WORKDIR /app
    ENV NODE_ENV=production
    # Copy only production deps and built output
    COPY --from=deps /app/node_modules ./node_modules
    COPY --from=build /app/dist ./dist
    COPY --from=build /app/package.json ./package.json
    # Run as non-root
    RUN addgroup -S appgroup && adduser -S appuser -G appgroup
    USER appuser
    EXPOSE 3000
    CMD ["node", "dist/server.js"]

    Key decisions:

    • node:20-alpine not node:20 — the Alpine variant is roughly 50–70 MB vs 350–400 MB for the Debian variant. Smaller attack surface, faster pull.
    • Non-root user. Running as root inside a container does not give you root on the host if the container runtime is properly configured, but defence in depth means avoiding it anyway. The OWASP Container Security Cheat Sheet recommends non-root users as a baseline control.
    • Pin to node:20-alpine not node:alpine or node:latest. The image digest changes. latest in production means your next deploy might pull a different Node major version.
  2. Write the Compose file

    The Compose file defines services, networks, volumes, health checks, and restart policies. The ordering of service startup is controlled by depends_on with condition: service_healthy.

    # docker-compose.yml
    version: "3.9"
     
    networks:
      internal:
        driver: bridge
     
    volumes:
      postgres_data:
      nginx_cache:
     
    services:
     
      postgres:
        image: postgres:16.2-alpine
        restart: unless-stopped
        environment:
          POSTGRES_DB: ${POSTGRES_DB}
          POSTGRES_USER: ${POSTGRES_USER}
          POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
        volumes:
          - postgres_data:/var/lib/postgresql/data
        networks:
          - internal
        healthcheck:
          test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
          interval: 10s
          timeout: 5s
          retries: 5
          start_period: 30s
     
      api:
        image: ${REGISTRY}/api:${API_VERSION}
        restart: unless-stopped
        env_file:
          - .env.production
        networks:
          - internal
        depends_on:
          postgres:
            condition: service_healthy
        healthcheck:
          test: ["CMD-SHELL", "wget -qO- http://localhost:3000/health || exit 1"]
          interval: 10s
          timeout: 5s
          retries: 3
          start_period: 20s
     
      web:
        image: ${REGISTRY}/web:${WEB_VERSION}
        restart: unless-stopped
        env_file:
          - .env.production
        networks:
          - internal
        healthcheck:
          test: ["CMD-SHELL", "wget -qO- http://localhost:3001/health || exit 1"]
          interval: 10s
          timeout: 5s
          retries: 3
          start_period: 20s
     
      nginx:
        image: nginx:1.25-alpine
        restart: unless-stopped
        volumes:
          - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
          - ./nginx/conf.d:/etc/nginx/conf.d:ro
          - nginx_cache:/var/cache/nginx
        networks:
          - internal
        depends_on:
          api:
            condition: service_healthy
          web:
            condition: service_healthy
        healthcheck:
          test: ["CMD-SHELL", "nginx -t && wget -qO- http://localhost/internal-health || exit 1"]
          interval: 10s
          timeout: 5s
          retries: 3
          start_period: 15s
     
      cloudflared:
        image: cloudflare/cloudflared:2024.12.2
        restart: unless-stopped
        command: tunnel run
        volumes:
          - ./cloudflared/config.yml:/etc/cloudflared/config.yml:ro
          - ./cloudflared/credentials.json:/etc/cloudflared/credentials.json:ro
        networks:
          - internal
        depends_on:
          nginx:
            condition: service_healthy

    Note the dependency chain: cloudflared waits for nginx, nginx waits for api and web, api waits for postgres. This chain means docker compose up -d will start services in the correct order and wait for each to be healthy before starting the next.

    The internal /internal-health endpoint in the nginx health check should be a simple location block that returns 200 without proxying upstream — it tests that nginx itself is running, not that the backends are reachable. This avoids circular dependency issues.

  3. Write the nginx configuration

    Keep nginx configuration in version control. The deploy script bind-mounts it into the container. After every deploy, the script signals nginx to reload.

    # nginx/nginx.conf
    worker_processes auto;
    error_log /var/log/nginx/error.log warn;
    pid /var/run/nginx.pid;
     
    events {
        worker_connections 1024;
        use epoll;
        multi_accept on;
    }
     
    http {
        include       /etc/nginx/mime.types;
        default_type  application/octet-stream;
     
        log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" "$http_x_forwarded_for"';
     
        access_log /var/log/nginx/access.log main;
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
     
        # Trust Cloudflare's forwarded-for header
        # Update this list from https://www.cloudflare.com/ips/
        set_real_ip_from 103.21.244.0/22;
        set_real_ip_from 103.22.200.0/22;
        set_real_ip_from 103.31.4.0/22;
        set_real_ip_from 104.16.0.0/13;
        set_real_ip_from 104.24.0.0/14;
        set_real_ip_from 108.162.192.0/18;
        set_real_ip_from 131.0.72.0/22;
        set_real_ip_from 141.101.64.0/18;
        set_real_ip_from 162.158.0.0/15;
        set_real_ip_from 172.64.0.0/13;
        set_real_ip_from 173.245.48.0/20;
        set_real_ip_from 188.114.96.0/20;
        set_real_ip_from 190.93.240.0/20;
        set_real_ip_from 197.234.240.0/22;
        set_real_ip_from 198.41.128.0/17;
        real_ip_header CF-Connecting-IP;
     
        include /etc/nginx/conf.d/*.conf;
    }
    # nginx/conf.d/default.conf
     
    # Internal health check — does not proxy upstream
    server {
        listen 80;
        server_name _;
     
        location /internal-health {
            access_log off;
            return 200 "ok\n";
            add_header Content-Type text/plain;
        }
     
        # Route by Host header
        location / {
            return 444;  # Drop connections with no matching Host
        }
    }
     
    # Marketing site
    server {
        listen 80;
        server_name app.example.com;
     
        location / {
            proxy_pass http://web:3001;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;  # Always https via tunnel
            proxy_connect_timeout 10s;
            proxy_send_timeout 60s;
            proxy_read_timeout 60s;
        }
    }
     
    # API
    server {
        listen 80;
        server_name api.example.com;
     
        location / {
            proxy_pass http://api:3000;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_connect_timeout 10s;
            proxy_send_timeout 60s;
            proxy_read_timeout 60s;
        }
    }

    The set_real_ip_from blocks are important. Without them, your application logs show Cloudflare's edge IP addresses, not the real visitor IPs. Keep the Cloudflare IP ranges list updated — they publish additions. The CF-Connecting-IP header carries the original client IP.

    The X-Forwarded-Proto: https header override is necessary because cloudflared sends HTTP to nginx (the connection between cloudflared and nginx is internal and unencrypted), but the original request from the browser was HTTPS via Cloudflare. Without this header, applications that check X-Forwarded-Proto to determine whether to redirect to HTTPS will enter a redirect loop.

    You can verify the complete nginx config is syntactically valid before bind-mounting it by running the validation on the host:

    # Validate nginx config from the host before deploying
    docker run --rm \
      -v "$(pwd)/nginx/nginx.conf:/etc/nginx/nginx.conf:ro" \
      -v "$(pwd)/nginx/conf.d:/etc/nginx/conf.d:ro" \
      nginx:1.25-alpine nginx -t

    This command runs nginx's config test (-t) in a throwaway container using the same config files that will be bind-mounted in production. Add this to your CI pipeline as a step before merge. It catches config syntax errors — missing semicolons, misspelled directive names, invalid upstream references — before they reach the server.

  4. Write the idempotent deploy script

    This script is the core artifact. It is idempotent: running it twice produces the same result as running it once. It handles the ordering problem that caused the 3:51 AM outage.

    #!/usr/bin/env bash
    set -euo pipefail
     
    # deploy.sh — idempotent production deploy
    # Usage: ./deploy.sh [api|web|all]
     
    COMPOSE="docker compose -f docker-compose.yml -f docker-compose.prod.yml"
    DEPLOY_TARGET="${1:-all}"
    TIMESTAMP=$(date +%Y%m%d_%H%M%S)
     
    log() { echo "[$(date '+%H:%M:%S')] $*"; }
    die() { echo "[ERROR] $*" >&2; exit 1; }
     
    # Verify required env vars are set
    : "${API_VERSION:?Need API_VERSION}"
    : "${WEB_VERSION:?Need WEB_VERSION}"
    : "${REGISTRY:?Need REGISTRY}"
     
    log "=== Deploy starting: target=$DEPLOY_TARGET timestamp=$TIMESTAMP ==="
     
    # 1. Pull new images before touching running services
    log "Pulling images..."
    if [[ "$DEPLOY_TARGET" == "all" || "$DEPLOY_TARGET" == "api" ]]; then
        docker pull "${REGISTRY}/api:${API_VERSION}"
    fi
    if [[ "$DEPLOY_TARGET" == "all" || "$DEPLOY_TARGET" == "web" ]]; then
        docker pull "${REGISTRY}/web:${WEB_VERSION}"
    fi
     
    # 2. Run database migrations — always, unconditionally
    # Migrations are idempotent; running them when nothing has changed is a no-op.
    # Skipping them "when nothing changed" is how you get production-staging drift.
    log "Running database migrations..."
    $COMPOSE run --rm --no-deps api node dist/migrate.js up \
        || die "Migration failed — aborting deploy"
     
    # 3. Run safe seeds (upsert only, never truncate)
    log "Running seeds..."
    $COMPOSE run --rm --no-deps api node dist/seed.js \
        || die "Seed failed — aborting deploy"
     
    # 4. Bring up infrastructure services first (postgres, if not already up)
    log "Ensuring postgres is healthy..."
    $COMPOSE up -d postgres
    $COMPOSE wait postgres  # Wait for health check to pass
     
    # 5. Deploy application containers
    # Order matters: api before web, web before nginx
    if [[ "$DEPLOY_TARGET" == "all" || "$DEPLOY_TARGET" == "api" ]]; then
        log "Deploying api..."
        $COMPOSE up -d --no-deps api
        log "Waiting for api health check..."
        timeout 60 bash -c 'until docker compose exec api wget -qO- http://localhost:3000/health; do sleep 2; done'
    fi
     
    if [[ "$DEPLOY_TARGET" == "all" || "$DEPLOY_TARGET" == "web" ]]; then
        log "Deploying web..."
        $COMPOSE up -d --no-deps web
        log "Waiting for web health check..."
        timeout 60 bash -c 'until docker compose exec web wget -qO- http://localhost:3001/health; do sleep 2; done'
    fi
     
    # 6. Update nginx config and reload (do NOT recreate the container)
    # Recreating nginx would drop cloudflared's connection.
    # Instead: update the bind-mounted config, then signal nginx to reload.
    log "Reloading nginx config..."
    # Validate config before reloading
    docker compose exec nginx nginx -t \
        || die "nginx config test failed — not reloading"
    docker compose exec nginx nginx -s reload
    log "nginx reloaded"
     
    # 7. Verify nginx is routing correctly via internal checks
    log "Verifying routing..."
    sleep 2  # Allow reload to complete
    docker compose exec nginx wget -qO- http://localhost/internal-health \
        || die "nginx internal health check failed after reload"
     
    # 8. cloudflared should remain running throughout
    # Only restart cloudflared if its config changed
    if git diff HEAD~1 HEAD -- cloudflared/config.yml | grep -q .; then
        log "cloudflared config changed — restarting cloudflared..."
        $COMPOSE restart cloudflared
        log "Waiting for cloudflared to reconnect..."
        sleep 10
        docker compose ps cloudflared | grep -q "Up" \
            || die "cloudflared did not come up"
    fi
     
    # 9. Remove orphan containers from previous deploys
    log "Removing orphan containers..."
    $COMPOSE up -d --remove-orphans
     
    log "=== Deploy complete: $TIMESTAMP ==="

    The critical decision in step 6: do not recreate the nginx container during a routine deploy. Update the config on disk (it is already bind-mounted) and signal nginx to reload. Recreating nginx tears down the connection that cloudflared is maintaining to it, which causes exactly the 47-second outage described in the lede.

    The only time you should recreate the nginx container is when the image itself needs to change (a new nginx version) or when the container's Compose configuration changes (new volume mounts, network changes). For config changes, reload is always sufficient.

  5. Configure SIGTERM handlers in your application

    When Docker stops a container, it sends SIGTERM to PID-1. It waits for stop_grace_period (default: 10 seconds) and then sends SIGKILL. If your application does not handle SIGTERM gracefully, it will be killed mid-request, leaving in-flight requests hanging and connections to the database unclosed.

    In a Node.js application:

    // graceful-shutdown.ts
    import { Server } from 'http';
    import { Pool } from 'pg';
     
    export function registerShutdownHandlers(server: Server, db: Pool) {
      let isShuttingDown = false;
     
      async function shutdown(signal: string) {
        if (isShuttingDown) return;
        isShuttingDown = true;
     
        console.log(`[shutdown] received ${signal}, starting graceful shutdown`);
     
        // Stop accepting new connections
        server.close(async (err) => {
          if (err) {
            console.error('[shutdown] server close error:', err);
            process.exit(1);
          }
     
          // Close database pool after server stops accepting requests
          try {
            await db.end();
            console.log('[shutdown] database pool closed');
          } catch (dbErr) {
            console.error('[shutdown] db pool close error:', dbErr);
          }
     
          console.log('[shutdown] clean exit');
          process.exit(0);
        });
     
        // Force exit if graceful shutdown takes too long
        setTimeout(() => {
          console.error('[shutdown] forced exit after timeout');
          process.exit(1);
        }, 8000); // Under the 10s Docker grace period
      }
     
      process.on('SIGTERM', () => shutdown('SIGTERM'));
      process.on('SIGINT', () => shutdown('SIGINT'));
    }

    The 8-second forced-exit timeout is intentional: it is under Docker's default 10-second stop_grace_period. This ensures the process exits cleanly before Docker sends SIGKILL. Extend the Compose stop_grace_period if your application needs more time to finish in-flight requests.

  6. Set up log shipping without a separate agent

    A full log aggregation stack (Loki + Promtail, or Elastic + Filebeat) is correct for teams at scale. For a single-box setup, it is more infrastructure to maintain than the application it supports.

    Docker's built-in logging driver writes container logs to the host's filesystem in JSON-lines format. docker logs <container> reads them. A simple cron job handles rotation and compression.

    Add to the Compose services:

    services:
      api:
        logging:
          driver: "json-file"
          options:
            max-size: "50m"
            max-file: "5"

    With max-size: 50m and max-file: 5, each container keeps at most 250 MB of logs on disk, rotating automatically. For most single-box setups, this is sufficient.

    For shipping to an external service (Logtail, Papertrail, Better Stack) without a sidecar agent, use Docker's syslog or GELF logging driver:

    services:
      api:
        logging:
          driver: "syslog"
          options:
            syslog-address: "udp://logs.example.com:514"
            tag: "api-production"

    This sends logs directly from Docker to the remote endpoint. No agent container required. The trade-off is that if the remote endpoint is unreachable, Docker's logging driver blocks — configure syslog-async: "true" to decouple log writes from request handling.

Verifying the full deploy end-to-end

After the deploy script completes, run a smoke test that exercises the actual public path — through Cloudflare, through the tunnel, through nginx, to the application. This is different from the internal health checks that the deploy script already runs. The internal checks verify that containers are up and talking to each other. The end-to-end check verifies that the public hostname resolves correctly, Cloudflare's proxying is working, and nginx is routing to the right upstream.

#!/usr/bin/env bash
# smoke-test.sh — run after deploy, tests the public path
set -euo pipefail
 
BASE_URL="${1:?Pass the base URL, e.g. https://app.example.com}"
 
check() {
  local name="$1" url="$2" expected_status="$3"
  local actual_status
  actual_status=$(curl -s -o /dev/null -w "%{http_code}" \
    -H "User-Agent: smoke-test/1.0" \
    --max-time 10 "$url")
  if [[ "$actual_status" == "$expected_status" ]]; then
    echo "  [PASS] $name → $actual_status"
  else
    echo "  [FAIL] $name → expected $expected_status, got $actual_status" >&2
    exit 1
  fi
}
 
echo "=== Smoke test: $BASE_URL ==="
check "Homepage"        "$BASE_URL/"              "200"
check "Health endpoint" "$BASE_URL/health"         "200"
check "API health"      "https://api.example.com/health" "200"
check "404 handling"    "$BASE_URL/does-not-exist" "404"
echo "=== All checks passed ==="

Run this smoke test from your CI system or from a separate machine — not from the server itself. Testing from localhost on the server does not exercise Cloudflare or the tunnel. The test must go through the same path a real user would take.

If the smoke test fails after a deploy, the deploy script should be extended to roll back automatically: stop the new containers, pull the previous image versions, and restart. Automated rollback on a single-box setup is tractable and worth the engineering effort.

# Simplified rollback — add to deploy.sh after smoke test failure
rollback() {
  log "SMOKE TEST FAILED — rolling back to previous versions"
  API_VERSION="$PREV_API_VERSION" WEB_VERSION="$PREV_WEB_VERSION" \
    $COMPOSE up -d --no-deps api web
  docker compose exec nginx nginx -s reload
  log "Rollback complete"
}
 
# Before deploying, capture current running versions
PREV_API_VERSION=$(docker inspect --format='{{index .Config.Image}}' "$(docker compose ps -q api)" | cut -d: -f2)
PREV_WEB_VERSION=$(docker inspect --format='{{index .Config.Image}}' "$(docker compose ps -q web)" | cut -d: -f2)
 
# ... deploy steps ...
 
# After deploy, run smoke test
if ! ./smoke-test.sh "https://app.example.com"; then
  rollback
  exit 1
fi

Anti-patterns

These are common mistakes. Each is paired with the wrong approach and the correct alternative.

Wrong

Open port 80/443 on the firewall alongside Cloudflare Tunnel

Right

Keep all inbound ports closed; tunnel only

Opening ports 80 and 443 "just in case" defeats the entire security property of Cloudflare Tunnel. If you have a public-facing port open, an attacker who discovers your origin IP can bypass Cloudflare's WAF, send requests directly to nginx, and hit your application without Cloudflare's bot detection, rate limiting, or DDoS protection in the path. The Cloudflare Tunnel model requires that your origin is unreachable from the public internet. If it is reachable, you have a traditional setup with extra complexity.

Check your firewall rules after setting up the tunnel:

# On the server — verify no public inbound on 80 or 443
sudo ufw status verbose
# Expected: 80/tcp and 443/tcp should be DENY or not present
# SSH on a non-standard port should be ALLOW IN

Wrong

image: myapp:latest in production Compose

Right

image: myapp:1.4.2 (pinned tag)

latest is not a version. It is a moving pointer. If you pin to latest and your CI pipeline pushes a new image tagged latest, the next docker compose pull on the production server pulls whatever the current latest is. This can be a version that was never tested for production, a half-complete feature, or a breaking change. Pinned semantic version tags in the Compose file make your deploy reproducible. The version you deploy to staging is the version you deploy to production.

Wrong

Mount /var/run/docker.sock into nginx or the app container

Right

Never expose the Docker socket to public-facing containers

The Docker socket is a root-equivalent capability. Any process that can write to /var/run/docker.sock can create new containers, run arbitrary commands on the host, mount the host filesystem, and escalate to full host compromise. Mounting it into a container that receives public traffic — nginx, a Node.js API, anything behind the Cloudflare Tunnel — creates a trivial container escape vector. The OWASP Container Security Cheat Sheet explicitly calls this out. If a tool requires the Docker socket (a CI runner, a management interface), run it on an isolated network with no public exposure.

Wrong

docker compose up -d (without --remove-orphans)

Right

docker compose up -d --remove-orphans

When you rename a service in your Compose file or remove a service, docker compose up -d without --remove-orphans leaves the old container running. The old container is on the same network, consuming memory and CPU, and potentially interfering with the new services. A zombie nginx container from a previous deploy configuration can hold ports or connections that the new nginx container expects. Always pass --remove-orphans in deploy scripts. It is safe: it only removes containers that are no longer defined in the Compose file.

Wrong

Change nginx.conf on disk and run docker compose restart nginx

Right

Change nginx.conf on disk and run docker compose exec nginx nginx -s reload

docker compose restart nginx recreates the container, dropping cloudflared's connection and causing a brief outage. nginx -s reload sends SIGHUP to the running nginx master process, which re-reads configuration and replaces workers gracefully without dropping connections. These are not equivalent. One causes an outage; one does not.

Wrong

Conditional migration flags (--only, --skip, --dry-run in production)

Right

Unconditional full migration run on every deploy

Migration tooling often provides flags to run only specific migrations or skip certain steps. These flags are useful for debugging. In a production deploy script, they are a footgun. A conditional migration that was "safe to skip once" gets cargo-culted into the script permanently. Six months later, a developer runs the full migration suite locally and discovers the database is three schema versions behind production because the conditional flag has been silently suppressing migrations on every deploy. Run migrations unconditionally. They are idempotent when there is nothing to do.

Wrong

Storing the database password in the Docker image via ARG or ENV in Dockerfile

Right

Injecting secrets at runtime via env_file or Docker secrets

A secret baked into a Docker image is a secret that will appear in the image's metadata, in the layer history (docker history), and potentially in any registry the image is pushed to. Even if the image registry is private, the blast radius of a registry credential leak now includes all secrets in every image. Build-time ARG values are visible in docker history --no-trunc. Runtime environment variables injected via env_file are not stored in the image at all — they exist only in the running container's environment. Never use ENV MY_SECRET=value in a Dockerfile for actual secrets.

Wrong

Using the default Compose project name (directory name)

Right

Explicitly setting the project name with --project-name or in the Compose file

Docker Compose uses the directory name as the default project name, which becomes the prefix for container names, network names, and volume names. If you clone your repository to /home/user/myapp, your containers are named myapp-api-1, myapp-postgres-1. If you clone it to /srv/app, they are named app-api-1, app-postgres-1. This naming difference matters for monitoring, for log shipping configurations that filter by container name, and for any automation that references container names. Set the project name explicitly:

# docker-compose.yml
name: my-production-app

Or pass it explicitly in every Compose command:

docker compose -p my-production-app up -d

Explicit naming makes your infrastructure deterministic regardless of where on the host the repository is checked out.

Wrong

Running database migrations with the application container already serving traffic

Right

Running migrations before the new application container starts

If your migration adds a NOT NULL column without a default, the old application (still serving traffic) will fail to insert rows that omit that column. If your migration drops a column, the old application will error on any query that references it. The safe ordering is: stop old containers → run migrations → start new containers. For a single-box setup with a maintenance window, this is straightforward. The deploy script above already implements this ordering — migrations run as a docker compose run --rm step before the application containers are brought up with the new image.

If you require truly zero-downtime migrations (the application must serve traffic throughout), the constraint is that each migration must be compatible with the version of the application code both before and after the deploy. This is a harder constraint and requires a two-deploy rollout for any breaking schema change: first deploy makes the code tolerant of both old and new schema, second deploy applies the migration, third deploy removes the compatibility shim.


Case studies / numbers

The following examples are composite observations from running this stack on VPS instances. All measurements are scoped to the specific configuration described. Your numbers will differ based on image size, host hardware, network conditions, and application characteristics.

Cold start on a fresh VPS

Configuration: 4 vCPU / 8 GB RAM VPS, Ubuntu 22.04, Docker 24.0.7, the full stack (Postgres 16, Node.js 20 API, Node.js 20 frontend, nginx 1.25, cloudflared 2024.x). No pre-pulled images.

Cold start sequence (from docker compose up -d to first successful request through the tunnel):

PhaseDuration
Postgres image pull (first time)11 s
nginx image pull3 s
Node.js API image pull (250 MB multi-stage)14 s
Node.js web image pull (180 MB multi-stage)10 s
cloudflared image pull4 s
Postgres container init (first-time data dir)8 s
API container start + health check pass7 s
Web container start + health check pass5 s
nginx health check pass3 s
cloudflared tunnel establish4 s
Total cold start~69 s

On a warm host (images already pulled, Postgres data volume already initialised), the sequence is:

PhaseDuration
Postgres container start + health check6 s
API container start + health check5 s
Web container start + health check4 s
nginx reload (no recreate)0.2 s
cloudflared (no restart required)0 s
Total warm deploy~15 s

The warm deploy number is the relevant one for production. After the first cold start, every deploy is warm. The 15-second window is the time between nginx serving the old application version and nginx reloading to serve the new one — which is essentially zero because we reload, not recreate.

The number that shocked me was the nginx reload time. I had been recreating the nginx container on every deploy because that is what I thought you had to do. Switching to nginx -s reload took the deploy-time outage from about 8 seconds to essentially zero. It was a one-line change in the deploy script.

— Solo founder, B2B SaaS

Memory footprint of the full stack

On the same 4 vCPU / 8 GB VPS, steady-state memory consumption under light load (a few hundred requests per day):

ContainerRSS (approx)
Postgres 16180–220 MB
Node.js API (single worker)120–160 MB
Node.js web90–130 MB
nginx (4 workers)30–50 MB
cloudflared25–40 MB
Total445–600 MB

This leaves 3.4–3.6 GB of the 8 GB headroom for OS, Docker overhead, and traffic spikes. A 2 GB VPS can run this stack under light load, but the margin for spikes is thin.

Deploy script timing

Measured on the same VPS, a typical warm deploy with the script from the previous section:

StepDuration
Image pull12 s
Migration run (no-op, nothing pending)1.8 s
Seed run (upsert no-ops)2.1 s
API container recreate + health check8 s
Web container recreate + health check6 s
nginx config validation + reload0.4 s
Routing verification2 s
Total~32 s

The 32-second deploy is from ./deploy.sh invocation to the script exiting with success. Traffic to the running application is unaffected for the full 32 seconds — the old containers serve requests while the new ones start up. The only window where traffic could be affected is the 0.4 seconds of nginx reload, during which nginx is completing in-flight requests on old workers and starting new workers. In practice, under the observed load, no requests were dropped during this window.

Cloudflare Tunnel reconnect time after cloudflared restart

When the deploy script determines that the cloudflared config has changed and restarts the cloudflared container, the reconnect time to Cloudflare's edge was observed at 3–6 seconds. During this window, Cloudflare's edge returns 502 to clients. For a production deploy, this is the window to minimise.

The mitigation: only restart cloudflared when the config actually changes. In most deploys, the cloudflared config (tunnel UUID, credentials, ingress rules) does not change. Application deploys should not require restarting the tunnel. Reserve cloudflared restarts for tunnel configuration changes.

If cloudflared must be restarted during a deploy, schedule it during the lowest-traffic window and alert before the deploy. The Cloudflare Tunnel documentation notes that cloudflared maintains multiple redundant connections; in theory a single connection being dropped and reconnecting should not cause visible downtime. In practice, a full container restart drops all connections simultaneously.

nginx config reload under load

A load test run (2,000 requests/second, sustained for 60 seconds) while triggering an nginx config reload in the middle of the test. Result: zero requests with non-2xx/3xx response codes during the reload. The reload completed in 180 ms. This is consistent with nginx's documented graceful reload behaviour: existing workers finish their requests before being replaced.

Tunnel latency overhead

The Cloudflare Tunnel adds latency relative to a direct connection. A request from a browser in Paris to an origin in Frankfurt might take 12 ms in a direct connection scenario. Through Cloudflare Tunnel, that request traverses: browser → Cloudflare PoP (Paris) → Cloudflare backbone → Cloudflare PoP (Frankfurt region) → tunnel connection → origin. The backbone leg is fast — Cloudflare's private network — but the extra hops add latency.

Observed latency overhead from Cloudflare Tunnel versus direct connection on the same VPS:

ScenarioP50P95P99
Direct connection (port 443 open)14 ms28 ms45 ms
Cloudflare Tunnel18 ms35 ms58 ms
Cloudflare Tunnel + WAF rules21 ms40 ms65 ms

These numbers are observations from a single VPS in a European region with a test client in the same region. They will vary significantly based on origin location, client location, and Cloudflare PoP proximity. The overhead is real but small relative to application processing time for most workloads.

I benchmarked the tunnel latency before committing to it. The overhead was about 4 ms at P50. That is noise compared to my database query time. The security properties and the elimination of origin IP exposure were worth it immediately.

— Solo founder, B2B SaaS

The more relevant latency concern is cloudflared-to-nginx latency within the Docker network. Because both containers are on the same Docker bridge network on the same host, this is effectively loopback latency: consistently under 1 ms. nginx-to-upstream latency is similarly sub-millisecond for containers on the same network. The Docker networking overhead is negligible for web workloads.

Postgres backup and restore timing

Backup strategy for a self-hosted Postgres container: pg_dump to a compressed file, shipped to an object store (Backblaze B2, or any S3-compatible service).

#!/usr/bin/env bash
# backup.sh — run via cron at 2 AM
set -euo pipefail
 
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="backup_${TIMESTAMP}.sql.gz"
BUCKET="s3://your-backup-bucket/postgres"
 
# Dump inside the postgres container, compress on stdout
docker compose exec -T postgres \
  pg_dump -U "${POSTGRES_USER}" "${POSTGRES_DB}" \
  | gzip > "/tmp/${BACKUP_FILE}"
 
# Upload to object store (using rclone or aws cli)
rclone copy "/tmp/${BACKUP_FILE}" "${BUCKET}/"
 
# Remove local file after upload
rm "/tmp/${BACKUP_FILE}"
 
echo "Backup complete: ${BACKUP_FILE}"

Measured backup timing for a 2.3 GB Postgres database (uncompressed):

StepDuration
pg_dump (inside container)38 s
gzip compression (on host)22 s
Upload to object store (100 Mbit uplink)45 s
Total~105 s

The compressed backup file was 410 MB — a compression ratio of approximately 5.6:1, which is typical for Postgres dumps of mixed text and numeric data.

Restore timing from the compressed backup to a fresh Postgres container:

StepDuration
Download from object store42 s
gunzip + psql pipe into container95 s
Index rebuild (implicit in restore)28 s
Total~165 s (< 3 minutes)

Test your restore procedure. The numbers above came from actually running the restore on a staging box. Backups that are never tested are not backups — they are compressed files of unknown content.

Disk usage and image layer caching

After three months of deploys, the /var/lib/docker directory on the production VPS:

CategorySize
Image layers (all versions)3.8 GB
Volumes (postgres_data + nginx_cache)1.2 GB
Container writeable layers180 MB
Build cache0 (builds happen in CI, not on the server)
Total~5.2 GB

Add a weekly docker system prune --filter "until=72h" to the crontab to clean up image layers from previous deploys. The --filter "until=72h" keeps images from the last three days as a rollback safety net.

# crontab entry — run at 3 AM Sunday
0 3 * * 0 /usr/bin/docker system prune --filter "until=72h" -f >> /var/log/docker-prune.log 2>&1

Limits + caveats

This stack is well-suited to specific workloads and poorly suited to others. Knowing the limits prevents building yourself into a corner.

Horizontal scaling beyond one box. Docker Compose is a single-host orchestrator. It has no concept of scheduling containers across multiple machines, distributing load across instances, or handling a host failure by restarting containers elsewhere. If you need to scale horizontally — run multiple instances of the API container across multiple machines — you need Kubernetes, Nomad, Docker Swarm, or a managed container service. Cloudflare Tunnel supports multiple connectors for redundancy, but the application layer remains bound to what Docker Compose can do on one host.

Multi-region automatic failover. The active-passive multi-region setup described earlier requires manual failover intervention or external automation. If the primary box goes down, traffic does not automatically route to the secondary without a human in the loop (or a separate automation system that you now have to maintain). Cloudflare's health-check-based failover helps at the routing layer but does not handle data replication or consensus.

Automatic backup verification. This stack does not include automated backup restoration testing. Taking Postgres backups (via pg_dump on a cron job, or continuous WAL archiving to an object store) is necessary but not sufficient — backups that have never been tested cannot be trusted. Periodically restoring a backup to a test environment and verifying application behaviour is a process step, not a technical one, and it is not automated here.

DDoS protection beyond Cloudflare's free tier. Cloudflare's free plan includes basic DDoS mitigation. A sophisticated volumetric attack may exceed what the free tier absorbs. For targets under sustained attack, Cloudflare's Pro plan or higher provides better protection. The tunnel model does protect your origin IP, which eliminates the common bypass of Cloudflare via direct origin hits — this is meaningful protection — but it does not substitute for proper rate limiting and attack response.

Secrets rotation. The .env.production file approach means rotating a secret requires updating the file on the server and redeploying. There is no automated rotation. For long-lived secrets (database passwords, API keys) that rarely change, this is acceptable. For secrets with shorter rotation requirements (OAuth client secrets, encryption keys), a proper secrets manager is worth the complexity.

Supply-chain attacks on base images. Pinning to a version tag (node:20-alpine) does not guarantee the image is unchanged. Docker image tags are mutable — an image layer could be compromised at the registry level. For stronger guarantees, pin to the image digest (node@sha256:...) and verify digests in CI. This is a marginal security improvement for most self-hosted workloads but matters for teams with regulatory requirements.

When to migrate to managed. Move off this stack when: you need more than one application host and Docker Compose's single-host model becomes a constraint, your team has grown to the point that the operational overhead of maintaining the deploy script is slowing you down, or your compliance requirements mandate managed services (SOC 2 with vendor attestations, HIPAA BAAs, etc.).


Sources & further reading

The following primary sources were used in writing this post. Versions and access dates are noted where relevant.

Cloudflare Tunnel documentation — cloudflared subcommand reference. The official reference for cloudflared tunnel run, cloudflared tunnel create, cloudflared tunnel route, and the ingress rules configuration syntax. The ingress rules section documents the first-match-wins ordering and the required fallback rule. Available at: https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/

Cloudflare blog — "Argo Tunnel is now Cloudflare Tunnel" (2021). The original technical description of the outbound-only connection model, the QUIC/H2 multiplexing architecture, and the four redundant connection guarantee. Provides the architectural context for why the tunnel is more than a reverse tunnel. Available at: https://blog.cloudflare.com/argo-tunnel-is-now-cloudflare-tunnel/

nginx documentation — Controlling nginx (signals). The authoritative source on nginx process control: SIGHUP for reload, SIGTERM/SIGQUIT for shutdown, SIGUSR1 for log rotation. Describes the graceful worker replacement behaviour during a reload. Available at: https://nginx.org/en/docs/control.html

Docker documentation — Compose file version 3 reference. Covers the depends_on condition syntax (service_healthy, service_started), healthcheck format, logging driver options, and the --remove-orphans flag. Available at: https://docs.docker.com/compose/compose-file/

Docker documentation — Use volumes. Covers the distinction between bind mounts, named volumes, and tmpfs mounts. Describes volume lifecycle relative to container lifecycle. Available at: https://docs.docker.com/storage/volumes/

Docker documentation — Dockerfile best practices. The official guide on multi-stage builds, layer caching, non-root users, .dockerignore, and minimising image size. The multi-stage build section directly informs the Dockerfile in this post. Available at: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

OWASP Container Security Cheat Sheet. Covers non-root container users, read-only filesystems, Docker socket exposure risks, and secrets handling. The socket exposure risk is directly cited in the anti-patterns section. Available at: https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html

NIST SP 800-190 — Application Container Security Guide. The NIST guidance on container security in production environments. Covers image provenance, registry trust, runtime protection, and orchestration security. Sections 3 and 4 are relevant to the supply-chain concerns noted in the caveats section. Available at: https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-190.pdf

PostgreSQL Docker official image documentation. Documents the POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD environment variables, the initialisation scripts directory (/docker-entrypoint-initdb.d/), and the pg_isready health check command. Available at: https://hub.docker.com/_/postgres

The Twelve-Factor App — Heroku / Adam Wiggins (2011). The canonical articulation of stateless processes, environment-based configuration, and disposable container design. Sections III (Config) and VI (Processes) are directly applicable to the compose and secrets patterns in this post. Available at: https://12factor.net/

nginx documentation — Module ngx_http_realip_module. Documents set_real_ip_from, real_ip_header, and real_ip_recursive directives used to restore real client IP addresses from Cloudflare's forwarded headers. Available at: https://nginx.org/en/docs/http/ngx_http_realip_module.html