View on GitHub

Containers & Kubernetes Tutorial

Session 08 demos – ConfigMaps, Secrets, volumes, StatefulSets

Demos 1–2 belong to Block A (ConfigMaps and Secrets), Demos 3–6 to Block B (StorageClass, PVC, StatefulSet, emptyDir). Everything runs in the demo namespace of the pinboard kind cluster. Files next to this one:

File Used in
configmap.yaml Demo 1 — the ConfigMap both Pods consume
pod-cm-env.yaml Demo 1 — consumed as environment variables
pod-cm-volume.yaml Demo 1 — the same keys, mounted as files
secret.yaml Demos 2, 4 — Postgres credentials
pvc-standalone.yaml Demo 3 — a PVC with no Pod: Pending on purpose
db-statefulset.yaml Demos 4–5 — headless Service + StatefulSet + PVC template
pod-emptydir.yaml Demo 6 — two containers sharing a volume

Before the session

  1. Images and namespace (≈ 4 min — postgres:17-alpine is 100 MB, pre-load it):

    cd ~/docker-kubernetes-training/demos/session08

docker pull postgres:17-alpine && docker pull busybox:1.37
kind load docker-image postgres:17-alpine busybox:1.37 --name pinboard
kubectl create namespace demo --dry-run=client -o yaml | kubectl apply -f -
kubectl config set-context --current --namespace=demo

  2. Demo 1 has a 60-second wait in the middle. Plan the slide you will talk over (the “update propagation” table) — or start the demo before the slide.

  3. Two panes: 1 for typing, 2 for kubectl -n demo logs -f cm-volume (started during Demo 1).

Demo 1: The same ConfigMap, as env vars and as files (≈ 8 min)

What to show / say

Two ways to consume one object, and one difference that decides your restart strategy: env vars are a snapshot taken at container start; mounted files are live.

Steps

  1. kubectl apply -f configmap.yaml -f pod-cm-env.yaml -f pod-cm-volume.yaml
kubectl -n demo get pods

    NAME        READY   STATUS    RESTARTS   AGE
cm-env      1/1     Running   0          8s
cm-volume   1/1     Running   0          8s

  2. Env side:

    kubectl -n demo exec cm-env -- env | grep -E 'APP_|GREETING'

    APP_GREETING=Welcome to Pinboard on Kubernetes
APP_THEME=emerald
LOG_FORMAT=json
GREETING_COPY=Welcome to Pinboard on Kubernetes

    👉 envFrom took all three keys as-is; configMapKeyRef took one and renamed it. 👉 note there is no PINBOARD_ prefix magic — the key is the variable name (envFrom.prefix exists if you want one).

  3. Volume side:

    kubectl -n demo exec cm-volume -- ls -l /config
kubectl -n demo exec cm-volume -- cat /config/APP_THEME

    total 0
lrwxrwxrwx 1 root root 19 May 12 09:12 APP_GREETING -> ..data/APP_GREETING
lrwxrwxrwx 1 root root 16 May 12 09:12 APP_THEME -> ..data/APP_THEME
lrwxrwxrwx 1 root root 17 May 12 09:12 LOG_FORMAT -> ..data/LOG_FORMAT
emerald

    👉 one file per key — and they are symlinks into a hidden ..data directory. That is how the kubelet swaps the whole set atomically: you never read a half-updated config.

  4. Start pane 2 (the Pod re-reads the file every 5 s so the change shows itself):

    kubectl -n demo logs -f cm-volume

    09:13:41 Welcome to Pinboard on Kubernetes
09:13:46 Welcome to Pinboard on Kubernetes

  5. Change the ConfigMap — one command, live:

    kubectl -n demo patch configmap pinboard-api-config \
  -p '{"data":{"APP_GREETING":"Config changed at '"$(date +%T)"'"}}'

    kubectl -n demo edit cm pinboard-api-config if you prefer to show the editor.

  6. Immediately check the env Pod:

    kubectl -n demo exec cm-env -- env | grep APP_GREETING

    APP_GREETING=Welcome to Pinboard on Kubernetes

    👉 unchanged, and it will stay unchanged until the container is recreated. Env vars are copied into the process at exec() time — Kubernetes cannot reach into a running process and change them.

  7. Now wait, and 👉 point at pane 2. Somewhere between 30 and 70 seconds later:

    09:14:31 Welcome to Pinboard on Kubernetes
09:14:36 Config changed at 09:14:12
09:14:41 Config changed at 09:14:12

    👉 the file updated by itself. The delay is the kubelet’s sync period plus its cache TTL (~1 min by default), not a network hiccup. And note the honest half of it: Kubernetes updated the file; if your app read it once at startup, nothing happened. Mounted config only helps apps that watch or re-read.

  8. The fix for the env Pod, and the command students will actually use:

    kubectl -n demo delete pod cm-env && kubectl apply -f pod-cm-env.yaml
sleep 5 && kubectl -n demo exec cm-env -- env | grep APP_GREETING

    APP_GREETING=Config changed at 09:14:12

    👉 for a Deployment it is kubectl rollout restart deploy/pinboard-api — a rolling restart, no downtime (Session 06, Demo 6). 👉 mention immutable: true on a ConfigMap: it forbids edits, so you create a new name per version and the rollout is the config change. Kustomize does that for you (Session 10).

Demo 2: Secrets — encoded, not encrypted (≈ 5 min)

What to show / say

The generator flag everyone should know, and the 15 seconds it takes to read any Secret you have get on.

Steps

  1. Generate the YAML instead of hand-writing base64:

    kubectl -n demo create secret generic pinboard-db \
  --from-literal=POSTGRES_USER=pinboard \
  --from-literal=POSTGRES_PASSWORD=pinboard-secret \
  --from-literal=POSTGRES_DB=pinboard \
  --from-literal=DATABASE_URL='postgres://pinboard:pinboard-secret@pinboard-db:5432/pinboard' \
  --dry-run=client -o yaml

    apiVersion: v1
data:
  DATABASE_URL: cG9zdGdyZXM6Ly9waW5ib2FyZDpwaW5ib2FyZC1zZWNyZXRAcGluYm9hcmQtZGI6NTQzMi9waW5ib2FyZA==
  POSTGRES_DB: cGluYm9hcmQ=
  POSTGRES_PASSWORD: cGluYm9hcmQtc2VjcmV0
  POSTGRES_USER: cGluYm9hcmQ=
kind: Secret
metadata:
  name: pinboard-db
  namespace: demo
type: Opaque

    👉 --dry-run=client -o yaml is the general trick: it turns any imperative kubectl create into a manifest you can commit. Works for deployments, services, jobs, cronjobs, everything.

  2. Apply the file version, which uses stringData instead:

    kubectl apply -f secret.yaml
grep -A4 stringData secret.yaml

    👉 stringData is write-only sugar. Read it back:

    kubectl -n demo get secret pinboard-db -o yaml | head -12

    apiVersion: v1
data:
  DATABASE_URL: cG9zdGdyZXM6Ly9waW5ib2FyZDpwaW5ib2FyZC1zZWNyZXRAcGluYm9hcmQtZGI6NTQzMi9waW5ib2FyZA==

  3. The demolition:

    kubectl -n demo get secret pinboard-db -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d ; echo

    pinboard-secret

    Say it clearly: “base64 is encoding. It exists so binary values survive JSON, not to protect anything. There was no password prompt, no audit, no key.”

  4. What actually protects it — name the four, briefly:

    kubectl auth can-i get secrets -n demo

    yes

    👉 (1) RBAC on secrets in that namespace, (2) encryption at rest for etcd (EncryptionConfiguration, off by default on plain clusters), (3) not putting them in git — External Secrets Operator, Secrets Store CSI driver, SOPS/age, (4) short-lived credentials instead of static ones where you can. 👉 and the decode one-liner for all keys at once, worth stealing:

    kubectl -n demo get secret pinboard-db -o go-template='=\n'

Demo 3: StorageClass and dynamic provisioning (≈ 5 min)

What to show / say

Nobody creates PersistentVolumes by hand any more. You ask for a size and a class; a provisioner produces the volume. And the most common “stuck PVC” is not an error.

Steps

  1. kubectl get storageclass

    NAME                 PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
standard (default)   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  3h

    👉 four columns, four decisions: who provisions, what happens to the data when the PVC is deleted (Delete — on AKS/EKS/GKE you often want Retain), when the volume is created, and whether it can grow later. On a cloud cluster you would see gp3, managed-csi, standard-rwo, several classes, one default.

  2. kubectl apply -f pvc-standalone.yaml
kubectl -n demo get pvc

    NAME      STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
scratch   Pending                                      standard       5s

  3. Do not let anyone think it is broken — read the events:

    kubectl -n demo describe pvc scratch | tail -5

    Events:
  Type    Reason                Age   From                         Message
  ----    ------                ----  ----                         -------
  Normal  WaitForFirstConsumer  8s    persistentvolume-controller  waiting for first consumer to be created before binding

    👉 WaitForFirstConsumer: the provisioner will not carve out storage until it knows which node the Pod is scheduled on — otherwise it could create a volume in a zone where the Pod cannot run. Exactly the same behaviour with EBS and Persistent Disk. The next demo shows a claim that does bind, because it has a Pod.

Demo 4: StatefulSet — stable names, stable storage (≈ 7 min)

What to show / say

A Deployment’s Pods are cattle with random names. A StatefulSet’s Pods are numbered, started in order, and each keeps its own volume across restarts.

Steps

  1. kubectl apply -f db-statefulset.yaml
kubectl -n demo get pods -w

    pinboard-db-0   0/1   Pending             0     0s
pinboard-db-0   0/1   ContainerCreating   0     2s
pinboard-db-0   1/1   Running             0     14s

    👉 pinboard-db-0 — an ordinal, not a hash. Ctrl-C when it is Running.

  2. The claim that did bind:

    kubectl -n demo get pvc,pv

    NAME                                      STATUS    VOLUME                                     CAPACITY   STORAGECLASS
persistentvolumeclaim/data-pinboard-db-0  Bound     pvc-6f0d1a4e-8c5b-4a12-9a77-2f1c9b0d3e55   1Gi        standard
persistentvolumeclaim/scratch             Pending                                                         standard

NAME                                                        CAPACITY   RECLAIM POLICY   STATUS   CLAIM
persistentvolume/pvc-6f0d1a4e-8c5b-4a12-9a77-2f1c9b0d3e55   1Gi        Delete           Bound    demo/data-pinboard-db-0

    👉 name it: data-pinboard-db-0 = <volumeClaimTemplate name>-<pod name>. The PV was created automatically, by the provisioner, the moment a Pod needed it — while scratch is still waiting for a consumer.

  3. Ordered scale-up:

    kubectl -n demo scale statefulset/pinboard-db --replicas=3
kubectl -n demo get pods -l app.kubernetes.io/name=pinboard-db -w

    pinboard-db-1   0/1   ContainerCreating   0   1s
pinboard-db-1   1/1   Running             0   12s
pinboard-db-2   0/1   Pending             0   0s
pinboard-db-2   1/1   Running             0   13s

    👉 -1 only starts after -0 is Ready, -2 after -1. That is what ordered rollout means, and it is why a StatefulSet rollout is slower and safer. 👉 be honest: these are three independent Postgres instances, not replication. The StatefulSet gives you identity and storage; clustering is your database’s job (or an operator’s).

  4. Stable DNS — the reason the headless Service exists:

    kubectl -n demo run dnsprobe --rm -it --image=busybox:1.37 --restart=Never -- \
  nslookup pinboard-db-0.pinboard-db.demo.svc.cluster.local

    Server:    10.96.0.10
Address:   10.96.0.10:53

Name:      pinboard-db-0.pinboard-db.demo.svc.cluster.local
Address:   10.244.1.14

    And the Service name itself returns all the Pods, no VIP:

    kubectl -n demo run dnsprobe --rm -it --image=busybox:1.37 --restart=Never -- \
  nslookup pinboard-db.demo.svc.cluster.local

    Name:      pinboard-db.demo.svc.cluster.local
Address:   10.244.1.14
Name:      pinboard-db.demo.svc.cluster.local
Address:   10.244.2.9
Name:      pinboard-db.demo.svc.cluster.local
Address:   10.244.1.15

    👉 clusterIP: None → DNS hands back Pod IPs instead of one virtual IP. Compare with pinboard-api from Session 07, which returned a single 10.96.x.x.

  5. Ordered scale-down, in reverse:

    kubectl -n demo scale statefulset/pinboard-db --replicas=1
kubectl -n demo get pvc

    NAME                  STATUS   VOLUME                                     CAPACITY   STORAGECLASS
data-pinboard-db-0    Bound    pvc-6f0d1a4e-...                           1Gi        standard
data-pinboard-db-1    Bound    pvc-91b3c027-...                           1Gi        standard
data-pinboard-db-2    Bound    pvc-0ad55e6c-...                           1Gi        standard

    👉 highest ordinal dies first — and the PVCs stay. Scale back to 3 and pinboard-db-1 gets its own old data back. Deleting a StatefulSet does not delete its PVCs either: storage is deliberately harder to lose than compute. Clean-up is a manual kubectl delete pvc (see Reset).

Demo 5: Kill the database Pod, keep the data (≈ 5 min)

What to show / say

The whole point of a PVC, in ninety seconds of screen time.

Steps

  1. Write something durable:

    kubectl -n demo exec pinboard-db-0 -- psql -U pinboard -d pinboard -c \
  "CREATE TABLE IF NOT EXISTS notes (id BIGSERIAL PRIMARY KEY, text TEXT NOT NULL);
   INSERT INTO notes (text) VALUES ('written before the Pod was deleted');"

    INSERT 0 1

  2. kubectl -n demo exec pinboard-db-0 -- psql -U pinboard -d pinboard -c "SELECT * FROM notes;"

     id |                text
----+------------------------------------
  1 | written before the Pod was deleted
(1 row)

  3. Destroy it:

    kubectl -n demo delete pod pinboard-db-0
kubectl -n demo get pod pinboard-db-0 -w

    pinboard-db-0   0/1   ContainerCreating   0   2s
pinboard-db-0   1/1   Running             0   9s

    👉 same name — a Deployment would have given you a new random one. And it was scheduled back to the same node, because its PVC is bound to a local volume there.

  4. kubectl -n demo exec pinboard-db-0 -- psql -U pinboard -d pinboard -c "SELECT * FROM notes;"

     id |                text
----+------------------------------------
  1 | written before the Pod was deleted
(1 row)

    👉 compare with Lab 01, where docker rm on the Postgres container took the data with it. Same image, same database — the difference is one volumeClaimTemplate.

Demo 6: emptyDir — sharing between containers in a Pod (≈ 4 min)

What to show / say

Not all volumes persist. emptyDir exists so two containers in one Pod can share a filesystem — the sidecar pattern, in eight lines of YAML.

Steps

  1. kubectl apply -f pod-emptydir.yaml
kubectl -n demo get pod shared-scratch

    NAME             READY   STATUS    RESTARTS   AGE
shared-scratch   2/2     Running   0          9s

    👉 2/2 — two containers, one Pod, one network namespace, one set of volumes.

  2. kubectl -n demo logs shared-scratch -c reader --tail=4

    09:41:02
09:41:04
09:41:06
09:41:08

    👉 reader never talked to writer. It is tailing a file that the other container is appending to, on a directory the kubelet created when the Pod was scheduled.

  3. Where it lives, and what that means:

    kubectl -n demo exec shared-scratch -c writer -- ls -l /shared
kubectl -n demo delete pod shared-scratch

    👉 emptyDir dies with the Pod — not with the container. A container that crashes and restarts finds its files still there; a Pod that is rescheduled to another node starts empty. Use it for caches, scratch space and hand-offs; use a PVC for anything you would miss. 👉 medium: Memory (commented in the file) makes it a tmpfs — fast, and it counts against the Pod’s memory limit.

Reset

Lab 08 has students build the Secret, ConfigMap and StatefulSet themselves in the pinboard namespace. Empty demo — including the PVCs, which outlive everything else on purpose.

kubectl -n demo delete pod cm-env cm-volume shared-scratch dnsprobe --ignore-not-found
kubectl delete -f db-statefulset.yaml -f configmap.yaml -f secret.yaml --ignore-not-found
kubectl -n demo delete pvc --all
kubectl -n demo get all,pvc,cm,secret

No resources found in demo namespace.

cm/kube-root-ca.crt is created by the cluster in every namespace — leaving it is correct.

kubectl get pv                        # expect: no released volumes left behind
kubectl config set-context --current --namespace=default