Session 04 · 2h

Docker Compose

Every flag you typed by hand in Lab 03 — declared once, in one file, under version control.

The Compose file The Compose CLI Developer loop Bridge to Kubernetes Lab 04

Session 04

What we'll cover today

1 · The Compose file

The Compose Specification: services, networks, volumes, build vs image, environment and interpolation, depends_on with health conditions.

2 · The Compose CLI

Ports and the project network, up -d --wait, ps, logs -f, exec, --scale, profiles, down vs down -v.

3 · Loop & limits

docker compose watch, override files, compose config, Compose in CI — and where Compose runs out of road.

4 · Lab 04

Pinboard in one file: healthchecks, depends_on conditions, .env, live editing, three API replicas.

The Compose file

One declarative YAML file describing a whole application — services, networks and volumes together.

Motivation

Why Compose

Lab 03, by hand

Three long docker run lines, in the right order
Network and volume created separately, first
Nothing waits for Postgres to be ready
Teardown is a checklist you will get wrong
The knowledge lives in your shell history

With Compose

One file describes every container, network and volume — with dependencies and health conditions.

docker compose up -d --wait
docker compose down -v

Reviewable in a pull request. Identical on every laptop and in CI.

Say the two words that matter: declarative and reproducible. Compose is also the on-ramp to the Kubernetes mindset — you stop describing steps and start describing a desired state. Mention that Compose v2 is a Go plugin to the Docker CLI, so it is docker compose with a space; the old Python docker-compose binary reached end of life in 2023 and should be uninstalled.

The format

The Compose Specification

An open spec, versioned by the spec itself — no version: key. If you see one, the file is from 2020.
Default filenames: compose.yaml (preferred), then docker-compose.yml
name: sets the project name — the prefix on every container, network and volume
Top-level keys: services, networks, volumes, configs, secrets

Modern

docker compose up (a space)
compose.yaml, no version:
name: pinboard

Legacy — don't copy from old blogs

docker-compose up (a hyphen)
version: "3.8"
links:, COMPOSE_FILE gymnastics

The version key is the single most common thing students copy from the internet; Compose v2 ignores it and prints a warning. Explain the project name concretely: with name: pinboard the containers are pinboard-db-1, pinboard-api-1, the network is pinboard_default. Without it, Compose uses the directory name, which is why the same file behaves differently in two folders.

Anatomy

Services, volumes, networks

name: pinboard
services:
  db:
    image: postgres:17-alpine
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-pinboard}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-pinboard-secret}
      POSTGRES_DB: ${POSTGRES_DB:-pinboard}
    volumes:
      - pinboard-data:/var/lib/postgresql/data
      - ./db/init.sql:/docker-entrypoint-initdb.d/init.sql:ro

volumes:
  pinboard-data:

Map every line back to Lab 03: image is the last argument of docker run, environment is -e, the first volumes entry is --mount type=volume and the second is a bind mount. Point at the top-level volumes: block — a named volume must be declared there or Compose will treat it as a host path. Note that Compose creates a default network for the project automatically, so there is no networks: block to write for a simple stack.

Where the image comes from

`build` vs `image`

image: alone — pull and run
build: alone — build an anonymous image
Both — build, and tag the result with that name
docker compose build, or up --build to force a rebuild

services:
  api:
    build:
      context: ./pinboard-api
      args:
        APP_VERSION: "1.0"
    image: pinboard-api:1.0

  web:
    build: ./pinboard-web
    image: pinboard-web:1.0

Always set image: next to build: — you get a predictable tag you can push, scan, or kind load later.

This is exactly the API service from src/compose.yaml: same build context, same APP_VERSION build arg they used in Lab 02, now declared instead of typed. Mention that Compose does not rebuild automatically when sources change — up reuses the existing image, which is the number-one "why is my change not showing" question, and the reason watch exists.

Configuration

environment, env_file, interpolation

services:
  api:
    env_file:
      - .env                       # read by Compose AND passed to the container
    environment:
      APP_THEME: ${APP_THEME:-emerald}
      APP_GREETING: ${APP_GREETING:-Welcome to Pinboard (Compose)}
      DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}

${VAR} is substituted by Compose, from the shell and from .env, before the file is parsed
${VAR:-default} falls back; ${VAR:?message} fails loudly if it is unset
To pass a literal $ into the container, double it: $${POSTGRES_USER}
.env holds credentials, is git-ignored, and ships as .env.example

Draw the line between the two dollar-sign worlds: interpolation happens on your machine when Compose reads the file, environment variables happen inside the container at run time. The $$ escape matters in the very next slide, where pg_isready must expand the variable inside the database container, not on your laptop. Show .env.example in src/ and tell them to copy it to .env in Lab 04.

Startup order

`depends_on` + `healthcheck`

services:
  db:
    image: postgres:17-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
      interval: 5s
      timeout: 3s
      retries: 10

  api:
    image: pinboard-api:1.0
    depends_on:
      db:
        condition: service_healthy

Plain depends_on: [db] only waits for the container to start. "Started" is not "accepting connections".

This is the slide that fixes the classic race where the API starts before Postgres finishes initialising. Conditions available: service_started, service_healthy, and service_completed_successfully for one-shot migration containers. Point at the $$ again. Add the honest caveat: the API should still retry its connection, because in Kubernetes nothing orders your Pods for you.

Start from an empty compose.yaml with just name: pinboard. Add db, run docker compose up -d, show docker compose ps. Add api with depends_on, then web with ports. Say that no .env is needed here because every variable in the file has a ${VAR:-default} — writing one is Lab 04's job. Then docker compose up -d --wait and point at the terminal pausing until db reports healthy. Open http://localhost:8080, post a note, then docker compose down and up -d --wait again to show the note survived in pinboard-data.

Networking, ports and the CLI

How services reach each other, and the ten commands you will use every day.

Discovery

`ports` vs `expose`

Compose creates pinboard_default, a user-defined bridge — so DNS by service name works out of the box
ports publishes to the host: the only door from your browser
expose is documentation between services
Add a networks: block only when you want isolation, e.g. a backend network the web tier cannot reach

services:
  api:
    expose:
      - "8080"          # api:8080, internal only

  web:
    environment:
      API_URL: http://api:8080
    ports:
      - "8080:8080"     # host:container

Note the service name, not the container name, is the DNS name — api, not pinboard-api-1. That indirection is what makes --scale work two slides from now. Show docker compose exec web wget -qO- http://api:8080/api/info to prove the internal hop needs no published port, and remind them the database has no ports at all on purpose.

Runtime policy

restart and resource limits

restart: is the same policy set as docker run --restart
Limits live under deploy.resources — honoured by docker compose on a single host
Other deploy keys (replicas, placement, update policy) are Swarm territory
Still no self-healing: a limit that is hit kills the container, and only the restart policy brings it back

services:
  api:
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 128M
          cpus: "0.50"

Connect back to session 03: this is --memory=128m --cpus=0.5 --restart unless-stopped, declared. Mention that hitting the memory limit still gives exit code 137 and OOMKilled, visible with docker compose ps -a. Flag deploy.replicas as a trap — docker compose up respects it, but everything else under deploy is ignored outside Swarm.

Daily driver

The commands you'll actually type

docker compose up -d --wait          # create everything, block until healthy/started
docker compose ps                    # state, health, published ports
docker compose logs -f api           # follow one service (no -f: dump and exit)
docker compose exec db psql -U pinboard -d pinboard
docker compose restart api
docker compose stop                  # keep containers, keep data
docker compose down                  # remove containers + the project network
docker compose down -v               # …and named volumes: the notes are gone

down -v is the destructive one. down keeps pinboard-data; down -v deletes it.

Emphasise --wait: without it up -d returns as soon as containers are created, which makes scripts and CI flaky. It exits non-zero if a service never becomes healthy. Also show docker compose exec versus run: exec enters a running service, run --rm starts a new one-off container, which is how you run migrations or tests.

More of the same thing

Scaling a service

docker compose up -d --scale api=3 --wait
docker compose ps
# pinboard-api-1  pinboard-api-2  pinboard-api-3

for i in $(seq 6); do
  docker compose exec -T web wget -qO- http://api:8080/api/info
done   # the "hostname" field rotates between the three containers

Compose DNS returns all three container IPs for api — clients round-robin
A service with a fixed ports mapping cannot be scaled: only one container can own host port 8080
This is manual scaling on one machine. Nothing rebalances, nothing reacts to load

Run the loop live and let them watch the hostname change — it is the first time in the course they see load balancing, and it is the same mental model as a Kubernetes Service in session 07. Explain why web keeps the published port and api does not. Then land the third bullet hard: three replicas is a number you typed, and it stays until you type a different one.

Optional pieces

Profiles

Services in a profile are not started unless the profile is on
Keep debug tools, seeders and test runners in the same file without slowing everyone down
Enable with --profile or the COMPOSE_PROFILES environment variable

docker compose --profile tools up -d
COMPOSE_PROFILES=tools docker compose up -d

services:
  adminer:
    image: adminer:5
    ports:
      - "8081:8080"
    depends_on:
      - db
    profiles:
      - tools

Profiles are how one file serves dev, CI and demos. A useful pattern: a seed service in the tools profile that runs psql against the DB and exits. Note that a service in a profile can still be started explicitly by name, and that depends_on pulls in its dependencies automatically.

Type docker compose up -d --scale api=3 --wait then docker compose ps. Refresh http://localhost:8080 a few times with the browser dev tools open on /api/info, or loop docker compose exec -T web wget -qO- http://api:8080/api/info. Then docker stop pinboard-api-2 and keep looping: some requests fail until nginx re-resolves — a perfect setup for readiness probes in session 06. Finish with docker compose up -d --scale api=1.

The developer loop — and the ceiling

Live reload, override files, CI — then an honest list of what Compose will never do for you.

Inner loop

`docker compose watch`

sync — copy changed files into the running container
rebuild — rebuild the image and replace the container
sync+restart — copy, then restart the process (config files)
Run it with docker compose watch; it takes over the terminal and streams what it does

services:
  web:
    develop:
      watch:
        - action: sync
          path: ./pinboard-web/html
          target: /usr/share/nginx/html
        - action: rebuild
          path: ./pinboard-web/Dockerfile
  api:
    develop:
      watch:
        - action: rebuild
          path: ./pinboard-api

This is src/compose.override.yaml verbatim. Contrast it with the bind mount from session 03: a bind mount hides the image contents behind host files, while sync copies into the image's own filesystem, so what you test is much closer to what you ship. Note ignore: for excluding node_modules-style folders, and that a rebuild rule on the Dockerfile keeps the loop correct when the image itself changes.

Run docker compose watch in one pane and edit src/pinboard-web/html/index.html in another — change the page title, save, refresh the browser. Then add a LABEL line to src/pinboard-web/Dockerfile and let them watch the rebuild-and-replace scroll past. Point out that the override file was picked up automatically, which is the topic of the next slide.

Composition

Multiple files & overrides

compose.yaml + compose.override.yaml are merged automatically
-f takes explicit control — first file is the base, later files override
Scalars are replaced, mappings are merged, most sequences are appended
Paths stay relative to the first file

# dev: base + override, automatically
docker compose up -d --wait

# prod-ish: base + an explicit second file
docker compose \
  -f compose.yaml \
  -f compose.prod.yaml \
  up -d --wait

Keep the base file environment-neutral. Put ports, bind mounts and develop: rules in the override.

Warn that the merge rules are subtle enough that you should never guess — render the result with compose config, which is the next slide. Mention include: as the modern way to pull in another team's Compose file, and extends: for sharing a service definition. Note that -f disables the automatic override pickup, so you must list it yourself.

Trust, then verify

`compose config` renders the truth

docker compose config                 # merged + interpolated + validated
docker compose config --services      # just the names
docker compose config --images        # what would be pulled or built

docker compose -f compose.yaml -f compose.prod.yaml config > rendered.yaml

Shows the file after variable substitution — the fastest way to catch an empty ${POSTGRES_PASSWORD}
Fails with a clear error on invalid YAML or unknown keys: a free lint step in CI
Reveals exactly how your override files merged, so you stop guessing

Demonstrate it with an unset variable: compose config prints a warning and an empty value, which is far easier to debug than a Postgres container that refuses to start. Suggest adding docker compose config -q as a pre-commit or pipeline check. This is the Compose equivalent of terraform validate or kubectl apply --dry-run=server.

Automation

Compose in CI

docker compose config -q                     # lint the file
docker compose build --pull
docker compose up -d --wait                  # non-zero exit if never healthy
docker compose run --rm --no-deps tests      # exit code = test result
docker compose logs --no-color > compose-logs.txt
docker compose down -v --remove-orphans      # always, even on failure

--wait plus health checks removes every sleep 30 from your pipeline
Alternative: up --abort-on-container-exit --exit-code-from tests
Set name: or -p per build so parallel jobs never collide

The ceiling

What Compose cannot do → Kubernetes

What you need	Compose	Kubernetes
Run across many machines	One Docker host. Full stop.	A scheduler places Pods on any node in the cluster
Survive a machine dying	Restart policy — on that same host	Controllers reschedule the workload elsewhere
Zero-downtime updates	`up` stops the old container, then starts the new one	Deployment rolling update, readiness gates, `rollout undo`
Scale with demand	`--scale api=3`, typed by a human	HorizontalPodAutoscaler on CPU or custom metrics
Secrets at scale	A `.env` file on someone's laptop	Secret objects, RBAC, encryption at rest, external stores

Compose is excellent for one machine: development, CI, small single-host deployments. Everything above needs an orchestrator.

The landscape

Where Swarm, Nomad and Kubernetes fit

Docker Swarm

Built into the engine, reads Compose files with docker stack deploy. Easiest step up — but a small ecosystem and little momentum.

HashiCorp Nomad

One binary, schedules containers and plain binaries or VMs. Simple and solid; a niche compared to K8s.

Kubernetes

The de-facto standard: every cloud offers it managed, and the whole CNCF ecosystem targets its API. Sessions 05–10.

Compose files do not become Kubernetes manifests. Converters like kompose give you a starting point, never a production result.

Tell them to write the file from scratch rather than copying src/compose.yaml — the reference is there to check against at the end. Watch for three things: a named volume used without the top-level volumes: block, a missing .env copied from .env.example, and single-dollar signs in the pg_isready health check. Last step is deliberate: down, restart, notes still there; down -v, restart, notes gone.

Session 04

Key takeaways

One file, one command

The Compose Specification — no version:, docker compose with a space, up -d --wait.

Health drives startup

healthcheck plus depends_on: condition: service_healthy ends the startup race.

Great on one host

watch, overrides and config for the dev loop and CI. Multi-host, self-healing and autoscaling are out of scope.

Next up

Session 05 — Kubernetes architecture, kubectl and Pods.

Recap in one line: Compose is declarative single-host orchestration, and it is the last stop before the cluster. Remind them Lab 05 starts with kind create cluster, so their machine should have kind installed and a few gigabytes free before next session.

Questions?

Up next — Session 05: Kubernetes — architecture, kubectl, Pods. One host was never going to be enough.