Docker Compose Tutorial

Docker Compose Tutorial: Multi-Container Apps Made Simple

Introduction: The Problem with Single Containers

In Module 1, you learned to containerize a single Flask app. But here's the reality: real applications never run alone.

A typical web application stack looks like this:

• Django API — handles business logic
• PostgreSQL — stores user data
• Redis — caches frequent queries
• Nginx — serves static files and routes traffic
• Next.js — renders the frontend

Without Docker Compose, you'd run five separate docker run commands, manually create a network, map ports, and pray nothing conflicts. It's tedious, error-prone, and impossible to share with your team.

Docker Compose solves this by letting you define your entire stack in a single YAML file and start everything with one command.

The Restaurant Manager Analogy

Docker Compose is like a restaurant manager.

Instead of hiring each chef individually (running containers one by one), you hand the manager a menu (docker-compose.yml). The manager reads the menu, knows exactly how many chefs you need, what ingredients to prep, and how the kitchen should flow. One command — docker compose up — and the entire restaurant is operational.

Without the manager:

bashCopy
1# The painful way: 5 separate commands
2docker network create my-network
3docker run -d --name postgres --network my-network -v pgdata:/var/lib/postgresql/data postgres:16
4docker run -d --name redis --network my-network redis:7
5docker run -d --name django --network my-network -p 8000:8000 my-django-app
6docker run -d --name nextjs --network my-network -p 3000:3000 my-nextjs-app
7docker run -d --name nginx --network my-network -p 80:80 -p 443:443 my-nginx

With Docker Compose:

bashCopy
1# The beautiful way: one command
2docker compose up -d

Visual Architecture: What We're Building

This diagram shows the DeployMart stack — a real-world application with five services orchestrated by a single docker-compose.yml:

Key insight: Only Nginx exposes ports to the outside world. Django, Next.js, PostgreSQL, and Redis communicate internally through Docker's private network. This is a security best practice — your database should never be reachable from the internet.

The Complete docker-compose.yml

Here's the production-ready docker-compose.yml for the DeployMart stack. Read it carefully — every line is explained below.

yamlCopy
1# docker-compose.yml
2# ============================================
3# VERSION: Use the latest Compose specification
4# ============================================
5version: "3.9"
6
7# ============================================
8# SERVICES: Each service = one container
9# ============================================
10services:
11  # ----------------------------------------
12  # 1. PostgreSQL Database
13  # ----------------------------------------
14  postgres:
15    image: postgres:16-alpine           # Official image, Alpine variant (small)
16    container_name: deploymart-db       # Friendly name for logs
17    restart: unless-stopped             # Auto-restart on crash, not on manual stop
18    
19    environment:                        # Config via env vars (12-factor app)
20      POSTGRES_DB: deploymart
21      POSTGRES_USER: deploymart_user
22      POSTGRES_PASSWORD: ${DB_PASSWORD} # From .env file - NEVER hardcode secrets!
23    
24    volumes:
25      - pgdata:/var/lib/postgresql/data # Named volume: data survives container restarts
26    
27    networks:
28      - backend                         # Only backend services can reach DB
29    
30    healthcheck:                        # Docker checks if Postgres is ready
31      test: ["CMD-SHELL", "pg_isready -U deploymart_user -d deploymart"]
32      interval: 10s
33      timeout: 5s
34      retries: 5
35      start_period: 30s
36
37  # ----------------------------------------
38  # 2. Redis Cache
39  # ----------------------------------------
40  redis:
41    image: redis:7-alpine
42    container_name: deploymart-cache
43    restart: unless-stopped
44    
45    volumes:
46      - redis_data:/data                # Persist cache data (optional)
47    
48    networks:
49      - backend
50    
51    healthcheck:
52      test: ["CMD", "redis-cli", "ping"]
53      interval: 10s
54      timeout: 5s
55      retries: 5
56
57  # ----------------------------------------
58  # 3. Django API
59  # ----------------------------------------
60  django:
61    build:
62      context: ./django                 # Build from Dockerfile in ./django/
63      dockerfile: Dockerfile.prod         # Use production Dockerfile
64    container_name: deploymart-api
65    restart: unless-stopped
66    
67    environment:
68      DATABASE_URL: postgres://deploymart_user:${DB_PASSWORD}@postgres:5432/deploymart
69      REDIS_URL: redis://redis:6379/0
70      DJANGO_SETTINGS_MODULE: myproject.settings.prod
71      SECRET_KEY: ${DJANGO_SECRET_KEY}
72    
73    volumes:
74      - django_static:/app/staticfiles   # Share static files with Nginx
75      - django_media:/app/mediafiles     # Share uploaded files with Nginx
76    
77    networks:
78      - backend                          # Can talk to DB and Redis
79      - frontend                         # Can be reached by Nginx
80    
81    depends_on:                          # Start order (doesn't wait for healthy!)
82      postgres:
83        condition: service_healthy      # Wait until Postgres passes health check
84      redis:
85        condition: service_healthy
86    
87    healthcheck:
88      test: ["CMD", "curl", "-f", "http://localhost:8000/health/"]
89      interval: 30s
90      timeout: 10s
91      retries: 3
92      start_period: 40s
93
94  # ----------------------------------------
95  # 4. Next.js Frontend
96  # ----------------------------------------
97  nextjs:
98    build:
99      context: ./nextjs
100      dockerfile: Dockerfile.prod
101    container_name: deploymart-web
102    restart: unless-stopped
103    
104    environment:
105      NODE_ENV: production
106      NEXT_PUBLIC_API_URL: http://django:8000  # Uses Docker DNS!
107    
108    volumes:
109      - nextjs_static:/app/.next/static  # Share built static files with Nginx
110    
111    networks:
112      - frontend                         # Only Nginx can reach this
113    
114    depends_on:
115      django:
116        condition: service_healthy
117    
118    healthcheck:
119      test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:3000/api/health"]
120      interval: 30s
121      timeout: 10s
122      retries: 3
123
124  # ----------------------------------------
125  # 5. Nginx Reverse Proxy
126  # ----------------------------------------
127  nginx:
128    image: nginx:1.25-alpine
129    container_name: deploymart-proxy
130    restart: unless-stopped
131    
132    ports:
133      - "80:80"                         # HTTP
134      - "443:443"                       # HTTPS
135    
136    volumes:
137      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro           # Main config (read-only)
138      - ./nginx/conf.d:/etc/nginx/conf.d:ro                   # Virtual hosts
139      - django_static:/var/www/django/static:ro               # Django static files
140      - django_media:/var/www/django/media:ro                 # Django uploads
141      - nextjs_static:/var/www/nextjs/static:ro               # Next.js static files
142      - certbot_data:/etc/letsencrypt:ro                     # SSL certificates
143      - certbot_www:/var/www/certbot:ro                     # Certbot challenges
144    
145    networks:
146      - frontend                       # Can reach Django and Next.js
147      # Note: Nginx does NOT connect to backend (security!)
148    
149    depends_on:
150      django:
151        condition: service_healthy
152      nextjs:
153        condition: service_healthy
154
155# ============================================
156# NETWORKS: Isolate traffic between layers
157# ============================================
158networks:
159  frontend:
160    driver: bridge                      # Default bridge network
161    # No external access = more secure
162  
163  backend:
164    driver: bridge
165    internal: true                      # CRITICAL: No external access at all
166                                        # Only containers on this network can communicate
167
168# ============================================
169# VOLUMES: Persistent storage
170# ============================================
171volumes:
172  pgdata:                              # PostgreSQL data (survives restarts)
173  redis_data:                           # Redis persistence
174  django_static:                        # Collected static files (shared with Nginx)
175  django_media:                         # User uploads (shared with Nginx)
176  nextjs_static:                        # Next.js build output (shared with Nginx)
177  certbot_data:                         # SSL certificates
178  certbot_www:                          # Certbot validation

Deep Dive: How Docker Compose Networking Works

The Magic of Docker DNS

When you define services in docker-compose.yml, Docker automatically:

1. Creates a private bridge network (named deploymart_backend and deploymart_frontend)
2. Assigns each container an IP address (e.g., 172.20.0.2, 172.20.0.3)
3. Runs a DNS server that resolves service names to these IPs

This means your Django app can connect to PostgreSQL using just the service name:

pythonCopy
1# Django settings.py
2DATABASES = {
3    'default': {
4        'ENGINE': 'django.db.backends.postgresql',
5        'NAME': 'deploymart',
6        'USER': 'deploymart_user',
7        'PASSWORD': os.getenv('DB_PASSWORD'),
8        'HOST': 'postgres',        # <-- Just the service name! No IP needed!
9        'PORT': '5432',
10    }
11}

javascriptCopy
1// Next.js API route
2const response = await fetch('http://django:8000/api/products');
3//                    ^^^^^^ Just the service name!

Why Hardcoded IPs Are Dangerous

Real-World Lesson: At a startup I worked with, a developer hardcoded 172.18.0.2 for the database host. It worked on their machine. When deployed to staging, the database got 172.18.0.3. The app crashed for 2 hours until someone checked the logs. Always use service names.

Understanding Docker Compose Volumes

Three Types of Storage

Named Volumes in Production

yamlCopy
1volumes:
2  pgdata:/var/lib/postgresql/data

• Managed by Docker — stored in Docker's data directory (/var/lib/docker/volumes/)
• Survives container restarts — your database data is safe
• Easy backup — docker run --rm -v pgdata:/data -v $(pwd):/backup alpine tar czf /backup/pgdata.tar.gz -C /data .
• Not accessible from host — more secure

Bind Mounts in Development

yamlCopy
1volumes:
2  - ./django:/app          # Edit code on host, see changes instantly
3  - ./nextjs/src:/app/src  # Hot-reload for Next.js

• Direct host folder sync — changes reflect immediately
• Great for development — no rebuild needed
• Never use in production — performance issues, security risks

tmpfs for Security

yamlCopy
1services:
2  django:
3    tmpfs:
4      - /tmp:noexec,nosuid,size=100m  # Session data in RAM only

• Never touches disk — perfect for sensitive session tokens
• Auto-cleaned on restart — no data leakage
• Fastest I/O — it's RAM

Environment Variables: The 12-Factor Way

Never hardcode secrets in docker-compose.yml. Use .env files:

bashCopy
1# .env (add this to .gitignore!)
2DB_PASSWORD=your-super-secret-password
3DJANGO_SECRET_KEY=django-insecure-change-me-in-production
4REDIS_PASSWORD=redis-secret

yamlCopy
1# docker-compose.yml
2services:
3  postgres:
4    environment:
5      POSTGRES_PASSWORD: ${DB_PASSWORD}  # Reads from .env automatically

bashCopy
1# .env.example (commit this - no real values)
2DB_PASSWORD=changeme
3DJANGO_SECRET_KEY=changeme
4REDIS_PASSWORD=changeme

Why this matters: If you accidentally commit a .env file with real passwords to GitHub, attackers scan public repos within minutes. Always use .env for secrets and .env.example as a template.

Hands-On Lab: Build the DeployMart Stack

Step 1: Create Project Structure

bashCopy
1mkdir deploymart && cd deploymart
2mkdir django nextjs nginx nginx/conf.d
3touch docker-compose.yml .env .env.example .gitignore

Step 2: Write the docker-compose.yml

Use the complete file from the "Complete docker-compose.yml" section above.

Step 3: Create Environment Files

bashCopy
1# .env (NEVER commit this)
2DB_PASSWORD=deploymart_secret_2024
3DJANGO_SECRET_KEY=django-insecure-dev-only-key-change-in-prod

bashCopy
1# .env.example (COMMIT this)
2DB_PASSWORD=changeme
3DJANGO_SECRET_KEY=changeme

bashCopy
1# .gitignore
2.env
3__pycache__/
4node_modules/
5*.pyc

Step 4: Start the Stack

bashCopy
1# Build images and start all services
2docker compose up -d --build
3
4# Watch the magic happen
5[+] Running 7/7
6 ✔ Network deploymart_backend     Created
7 ✔ Network deploymart_frontend    Created
8 ✔ Volume deploymart_pgdata       Created
9 ✔ Container deploymart-db        Started
10 ✔ Container deploymart-cache     Started
11 ✔ Container deploymart-api       Started
12 ✔ Container deploymart-web       Started
13 ✔ Container deploymart-proxy     Started

Step 5: Verify Everything Works

bashCopy
1# Check all services are running
2docker compose ps
3
4# View logs for a specific service
5docker compose logs -f django
6
7# View logs for all services (follow mode)
8docker compose logs -f
9
10# Check service health
11docker inspect --format='{{.State.Health.Status}}' deploymart-db
12# Output: healthy
13
14# Test internal DNS resolution
15docker exec deploymart-api ping -c 1 postgres
16# Output: PING postgres (172.20.0.2): 56 data bytes
17
18# Enter a running container
19docker compose exec django bash
20
21# Scale Django to 3 instances (for testing)
22docker compose up -d --scale django=3

Step 6: Useful Commands Cheat Sheet

bashCopy
1# Start everything
2docker compose up -d
3
4# Start with build (after Dockerfile changes)
5docker compose up -d --build
6
7# Stop everything
8docker compose down
9
10# Stop and remove volumes (WARNING: deletes data!)
11docker compose down -v
12
13# Restart a single service
14docker compose restart django
15
16# View resource usage
17docker compose stats
18
19# Run a one-off command
20docker compose run --rm django python manage.py migrate
21
22# Check configuration
23docker compose config  # Validates and shows merged config

Common Mistakes & How to Fix Them

❌ Mistake 1: Forgetting depends_on

yamlCopy
1# WRONG: Django starts before Postgres is ready
2services:
3  django:
4    # ...
5  postgres:
6    # ...

The Problem: Django tries to connect to Postgres before Postgres has finished initializing. Django crashes. You restart Django. Postgres is still starting. It crashes again. You enter a restart loop.

The Fix:

yamlCopy
1services:
2  django:
3    depends_on:
4      postgres:
5        condition: service_healthy  # Wait for health check, not just "started"

Important: depends_on only controls startup order, not readiness. Always pair it with condition: service_healthy.

❌ Mistake 2: Using Bind Mounts for Databases

yamlCopy
1# WRONG: Bind mount for database (slow + risky)
2services:
3  postgres:
4    volumes:
5      - ./pgdata:/var/lib/postgresql/data  # DON'T DO THIS

The Problem:

• Performance: Bind mounts use the host filesystem, which is slower than Docker's native volume driver
• Permissions: File ownership conflicts between host user (UID 1000) and container user (UID 999 for Postgres)
• Corruption risk: If your host OS crashes, the bind-mounted database might get corrupted

The Fix:

yamlCopy
1services:
2  postgres:
3    volumes:
4      - pgdata:/var/lib/postgresql/data  # Named volume = correct way

❌ Mistake 3: Committing .env Files

bashCopy
1# WRONG: .env is tracked in Git
2git add .env
3git commit -m "Add config"
4# Your secrets are now on GitHub forever

The Fix:

bashCopy
1# 1. Add .env to .gitignore BEFORE committing
2echo ".env" >> .gitignore
3
4# 2. Commit .env.example instead
5cp .env .env.example
6# Edit .env.example to remove real values
7git add .env.example .gitignore
8git commit -m "Add environment template"

❌ Mistake 4: Exposing Database Ports

yamlCopy
1# WRONG: Database accessible from outside
2services:
3  postgres:
4    ports:
5      - "5432:5432"  # Anyone can connect to your DB!

The Problem: If your server has a public IP, port 5432 is now open to the entire internet. Attackers scan for open database ports constantly.

The Fix:

yamlCopy
1services:
2  postgres:
3    # NO ports section!
4    networks:
5      - backend  # Only internal network access

If you MUST access the database from outside (for debugging), use port forwarding temporarily:

bashCopy
1# Forward local port 5433 to container port 5432 (temporary, secure)
2docker compose exec -T postgres psql -U deploymart_user -d deploymart

❌ Mistake 5: No Health Checks

Without health checks, Docker thinks your container is "running" even if your app is stuck in a deadlock or can't connect to the database.

The Fix: Add health checks to every service (shown in the complete docker-compose.yml above).

Best Practices Checklist

Before pushing your docker-compose.yml to production, verify:

• .env is in .gitignore — Secrets never committed
• .env.example exists — New developers know what to configure
• Database has no ports — Not exposed to the internet
• backend network is internal: true — Database isolation
• All services have healthcheck — Auto-recovery works
• depends_on uses condition: service_healthy — Correct startup order
• Named volumes for persistence — Data survives restarts
• Bind mounts only for dev — Never in production
• restart: unless-stopped — Auto-recovery on crash
• Images pinned to versions — postgres:16-alpine not postgres:latest

Mini Project: Build a Task Manager with Django + PostgreSQL + Redis

Requirements

Build a Task Manager API with the following architecture:

docker-compose.yml
├── Django API (Gunicorn, port 8000)
├── PostgreSQL 16 (persistent data)
├── Redis 7 (task caching)
└── Nginx (port 80, proxies to Django)

API Endpoints (Django)

Technical Requirements

1. Django uses postgres as the database host (Docker DNS)
2. Django caches task lists in Redis (cache for 60 seconds)
3. PostgreSQL uses a named volume pgdata
4. Nginx proxies /api/ to Django and serves static files
5. All services have health checks
6. Backend network is internal (no external access)

Verification Steps

bashCopy
1# 1. Start the stack
2docker compose up -d --build
3
4# 2. Verify all services healthy
5docker compose ps
6# STATUS should show "healthy" for all
7
8# 3. Test API through Nginx
9curl http://localhost/api/tasks/
10# Output: []
11
12# 4. Create a task
13curl -X POST http://localhost/api/tasks/ \
14  -H "Content-Type: application/json" \
15  -d '{"title": "Learn Docker Compose", "done": false}'
16
17# 5. Verify Redis caching
18curl http://localhost/api/tasks/
19# Second request should be faster (cached)
20
21# 6. Test persistence
22docker compose down
23docker compose up -d
24curl http://localhost/api/tasks/
25# Task should still exist!
26
27# 7. Verify database is NOT exposed
28curl http://localhost:5432
29# Should FAIL (connection refused)

Starter Files

pythonCopy
1from flask import Flask, request, jsonify
2import os
3import uuid
4import redis
5
6app = Flask(__name__)
7
8# Redis connection
9cache = redis.Redis(
10    host='redis',  # Docker DNS!
11    port=6379,
12    decode_responses=True
13)
14
15# In-memory storage (use PostgreSQL in real app)
16tasks = {}
17
18@app.route('/health')
19def health():
20    return jsonify({
21        "status": "healthy",
22        "db": "connected" if cache.ping() else "disconnected"
23    }), 200
24
25@app.route('/api/tasks/', methods=['GET'])
26def list_tasks():
27    # Try cache first
28    cached = cache.get('tasks_list')
29    if cached:
30        return jsonify({"source": "cache", "data": eval(cached)}), 200
31    
32    result = list(tasks.values())
33    cache.setex('tasks_list', 60, str(result))  # Cache for 60s
34    return jsonify({"source": "database", "data": result}), 200
35
36@app.route('/api/tasks/', methods=['POST'])
37def create_task():
38    data = request.get_json()
39    if not data or 'title' not in data:
40        return jsonify({"error": "Title is required"}), 400
41    
42    task = {
43        "id": str(uuid.uuid4()),
44        "title": data['title'],
45        "done": data.get('done', False)
46    }
47    tasks[task['id']] = task
48    cache.delete('tasks_list')  # Invalidate cache
49    return jsonify(task), 201
50
51@app.route('/api/tasks/<task_id>', methods=['GET'])
52def get_task(task_id):
53    task = tasks.get(task_id)
54    if not task:
55        return jsonify({"error": "Task not found"}), 404
56    return jsonify(task), 200
57
58@app.route('/api/tasks/<task_id>', methods=['DELETE'])
59def delete_task(task_id):
60    if task_id not in tasks:
61        return jsonify({"error": "Task not found"}), 404
62    del tasks[task_id]
63    cache.delete('tasks_list')
64    return jsonify({"message": "Task deleted"}), 200
65
66if __name__ == '__main__':
67    app.run(host='0.0.0.0', port=5000)

flask==3.0.3
gunicorn==23.0.0
redis==5.0.0
psycopg2-binary==2.9.9

nginxCopy
1upstream django_backend {
2    server django:5000;
3}
4
5server {
6    listen 80;
7    server_name localhost;
8
9    location /api/ {
10        proxy_pass http://django_backend;
11        proxy_set_header Host $host;
12        proxy_set_header X-Real-IP $remote_addr;
13    }
14
15    location / {
16        root /var/www/static;
17        try_files $uri $uri/ =404;
18    }
19}

What You've Learned

Next Steps

In Module 3, you'll learn Nginx as a Reverse Proxy — the critical piece that routes traffic from the internet to your Django and Next.js containers. You'll discover why Nginx belongs in the same docker-compose.yml (not a separate one), and how to configure multi-domain hosting on ports 80 and 443.

Preview: We'll take the DeployMart stack from this module and add Nginx in front of it, handling SSL termination, static file serving, and load balancing — all with a single docker compose up -d.

Additional Resources

• Docker Compose Documentation: docs.docker.com/compose
• Compose Specification: compose-spec.io
• 12-Factor App: 12factor.net
• Dive (image inspector): Analyze what's inside your images
• Hadolint: Lint your Dockerfiles for best practices

Completed the mini project? Share your docker-compose.yml and image size in the comments. In the next module, we'll add Nginx and make this stack production-ready.