How-to Guides

Scope

Task recipes for Docker Engine 29.x, Buildx, and Compose v5: installing and upgrading, building and securing images, running containers and Compose stacks, networking, monitoring, cleanup, and troubleshooting. Look-up tables (versions, limits, daemon keys) are in Reference; background is in Explanation.

Install Docker Engine on Ubuntu

Use Docker's own apt repository, not the distribution's docker.io package, to get current 29.x releases. Supported: Ubuntu 22.04, 24.04, and 26.04 LTS (64-bit).

# Remove conflicting distro packages first (safe if none are installed)
for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt remove -y $pkg; done

# Add Docker's GPG key
sudo apt update
sudo apt install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository (deb822 format)
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/docker.asc
EOF
sudo apt update

# Install Engine, CLI, containerd, Buildx and Compose plugins
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Verify
sudo systemctl status docker
sudo docker run hello-world
docker version --format '{{.Server.Version}}'

Firewall interaction

Published container ports bypass ufw and firewalld rules. Docker supports only rulesets created with iptables/ip6tables (iptables-nft or iptables-legacy) while the default iptables backend is active.

To pin a version, list candidates with apt list --all-versions docker-ce and install docker-ce=<VERSION_STRING> docker-ce-cli=<VERSION_STRING>.

Upgrade to Engine 29 Safely

Engine 29.0 (2025-11-10) contains breaking changes. Check these before upgrading a host from 28.x or older:

Change in 29.x	What to check	Action
Daemon requires API v1.44+	Old SDK clients or tools pinned to API < 1.44 (Docker < 25.0)	Upgrade clients, or set DOCKER_API_VERSION only to 1.44+
containerd 2.x sets nofile to 1024	Databases, proxies, or JVMs that need many file descriptors	Add --ulimit nofile=... or default-ulimits in daemon.json
Docker Content Trust removed from the CLI	CI jobs using DOCKER_CONTENT_TRUST=1 or docker trust	Move to Cosign signing and verification
Legacy link env vars not injected	Apps reading DB_PORT_5432_TCP_ADDR-style variables	Use DNS names on user-defined networks; temporary escape hatch: DOCKER_KEEP_DEPRECATED_LEGACY_LINKS_ENV_VARS=1 on the daemon
cgroup v1 deprecated	Hosts still booting with cgroup v1	Plan migration to cgroup v2 (support continues until at least May 2029)
macvlan / IPvlan-L2 get no default gateway	Networks relying on an implicit gateway	Add --gateway to the network IPAM config
containerd image store only for fresh installs	Upgraded hosts stay on overlay2	Optionally migrate (next section)

# Check the cgroup version and current storage backend before upgrading
stat -fc %T /sys/fs/cgroup/          # cgroup2fs = v2, tmpfs = v1
docker info -f '{{ .Driver }} {{ .DriverStatus }}'

# Upgrade in place from the apt repository
sudo apt update && sudo apt install --only-upgrade -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Enable the containerd Image Store on an Upgraded Host

Fresh Engine 29 installs already use it; upgraded hosts keep overlay2 until you switch.

# 1. Save anything you cannot re-pull (images become hidden after switching)
docker save -o /srv/backup/local-images.tar myapp:latest

# 2. Enable the feature
sudo tee /etc/docker/daemon.json <<'EOF'
{
  "features": {
    "containerd-snapshotter": true
  }
}
EOF
sudo systemctl restart docker

# 3. Verify: expect io.containerd.snapshotter.v1
docker info -f '{{ .DriverStatus }}'

# 4. Restore saved images
docker load -i /srv/backup/local-images.tar

Merge, don't overwrite

The tee above replaces daemon.json. On a host with existing settings, add the features key to the existing file instead. Also check disk space: containerd keeps compressed and unpacked layers, under its own root (/var/lib/containerd).

Switch the Firewall Backend to nftables (Experimental)

Only for non-Swarm hosts on Engine 29.0 or newer.

# Docker will not enable forwarding with nftables, so enable it (and filter it) yourself
echo 'net.ipv4.ip_forward=1' | sudo tee /etc/sysctl.d/99-docker-forward.conf
echo 'net.ipv6.conf.all.forwarding=1' | sudo tee -a /etc/sysctl.d/99-docker-forward.conf
sudo sysctl --system

# Set the backend (merge into an existing daemon.json if you have one)
echo '{ "firewall-backend": "nftables" }' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker

# Verify
docker info | grep -i firewall
sudo nft list tables | grep docker-bridges

Rules you kept in the iptables DOCKER-USER chain are ignored after a reboot. Re-create them as your own nftables table, and use --bridge-accept-fwmark if you need to override Docker's drop rules.

Run the Daemon Rootless

# Prerequisites: uidmap package and >= 65,536 subordinate IDs
sudo apt install -y uidmap
grep ^$(whoami): /etc/subuid /etc/subgid

# Optional: stop the rootful daemon
sudo systemctl disable --now docker.service docker.socket

# Install and start the per-user daemon (script ships with docker-ce packages)
dockerd-rootless-setuptool.sh install
systemctl --user enable --now docker
sudo loginctl enable-linger $(whoami)      # keep it running after logout

# Point the CLI at the rootless socket
docker context use rootless
docker info -f '{{ .SecurityOptions }}'    # should include name=rootless

Since 29.5.0 the default rootless network driver is gvisor-tap-vsock. To bind ports below 1024, set net.ipv4.ip_unprivileged_port_start=0 via sysctl or grant CAP_NET_BIND_SERVICE to rootlesskit.

Authenticate to Docker Hub and Avoid Pull Limits

Anonymous pulls are limited to 100 per 6 hours per IPv4 address (or IPv6 /64), and Personal accounts to 200 per 6 hours (limits).

# Log in with a personal access token (not your password)
echo "$DOCKERHUB_TOKEN" | docker login -u myuser --password-stdin

# Check remaining pulls (headers ratelimit-limit / ratelimit-remaining)
TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)
curl -s --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest | grep -i ratelimit

For CI fleets, add a pull-through cache (Harbor, Nexus, or a registry mirror) and configure "registry-mirrors" in daemon.json, or use a paid plan with unlimited pulls.

Build Images

Multi-Stage Build for Node.js

# syntax=docker/dockerfile:1
FROM node:22-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm npm ci
COPY . .
RUN npm run build && npm prune --omit=dev

FROM node:22-alpine AS runtime
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
USER node
CMD ["node", "dist/main.js"]

The build stage installs all dependencies (the build needs dev dependencies), then prunes them before the runtime stage copies node_modules. The older npm ci --only=production flag is deprecated in npm 7+ in favor of --omit=dev.

Multi-Stage Build for Go with Cache Mounts

# syntax=docker/dockerfile:1

# ---- Build stage ----
FROM golang:1.25-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
    go mod download
COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    CGO_ENABLED=0 go build -ldflags="-s -w" -o /app/server .

# ---- Runtime stage ----
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /app/server /server
EXPOSE 8080
USER nonroot:nonroot
ENTRYPOINT ["/server"]

Build Optimization Checklist

Strategy	Typical impact	Notes
Multi-stage builds	Large size reduction (often 50-90%)	Separate build and runtime stages
Alpine or distroless base	Much smaller than full Debian/Ubuntu	Alpine uses musl libc; test for compatibility
.dockerignore	Faster context transfer	Exclude node_modules, .git, build output
Order instructions by change frequency	Far more cache hits on rebuild	Copy lockfiles and install deps before copying source
RUN --mount=type=cache	Persistent package manager caches	Survives across builds on the same builder
RUN --mount=type=secret	Secrets never land in layers	Pass with --secret id=npmrc,src=$HOME/.npmrc

Buildx Recipes

# BuildKit is the default builder since Engine 23.0; DOCKER_BUILDKIT=1 is no longer needed
docker build --target production -t myapp:latest .

# Registry cache for CI (needs a builder that supports cache export, e.g. docker-container driver)
docker buildx create --name ci --driver docker-container --use
docker buildx build \
  --cache-from type=registry,ref=registry.example.com/myapp:buildcache \
  --cache-to type=registry,ref=registry.example.com/myapp:buildcache,mode=max \
  -t registry.example.com/myapp:1.4.2 --push .

# Multi-platform build with SBOM and provenance attestations
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --sbom=true --provenance=mode=max \
  -t registry.example.com/myapp:1.4.2 --push .

# Inspect what was pushed
docker buildx imagetools inspect registry.example.com/myapp:1.4.2

# Scaffold a Dockerfile and compose.yaml for an existing project
docker init

Scan and Sign Images

# Vulnerability scanning
docker scout quickview myapp:latest
docker scout cves --only-severity critical,high myapp:latest
trivy image --severity HIGH,CRITICAL myapp:latest

# Sign and verify with Cosign (replaces Docker Content Trust, removed from the CLI in v29)
cosign sign --key cosign.key registry.example.com/myapp@sha256:<digest>
cosign verify --key cosign.pub registry.example.com/myapp@sha256:<digest>

# Keyless signing in CI (OIDC identity + Rekor transparency log)
cosign sign registry.example.com/myapp@sha256:<digest>

Use Docker Hardened Images as Base Images

DHI Community images are free (Apache 2.0) and served from dhi.io. Authenticate with your Docker ID, then reference them like any other base image. Check the catalog for exact repository and tag names.

docker login dhi.io
docker pull dhi.io/python:3.13

Run Containers

# Run detached with a published port
docker run -d --name web -p 8080:80 nginx:1.27

# Run with resource limits and a restart policy
docker run -d --name web \
  --cpus="2.0" --memory="512m" --memory-swap="1g" \
  --pids-limit=100 \
  --restart=unless-stopped \
  nginx:1.27

# Hardened run: read-only root, tmpfs scratch, no extra capabilities
docker run -d --name api \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --cap-drop ALL --cap-add NET_BIND_SERVICE \
  --security-opt no-new-privileges \
  --user 10001:10001 \
  registry.example.com/myapp:1.4.2

# Set the umask for the main process, execs, and healthchecks (Engine 29.8.0+)
docker run -d --umask 0027 registry.example.com/myapp:1.4.2

# Exec, logs, stats
docker exec -it web /bin/sh
docker logs -f --tail 100 web
docker stats web

# Stop gracefully (30 s before SIGKILL), then remove
docker stop --timeout 30 web
docker rm web

# Container IP per network (top-level NetworkSettings.IPAddress is deprecated)
docker inspect -f '{{range $n, $c := .NetworkSettings.Networks}}{{$n}}={{$c.IPAddress}} {{end}}' web

Health Checks

HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
  CMD curl -f http://localhost:8080/health || exit 1

# Show health state (the .HealthStatus placeholder exists since Engine 29.5.0)
docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.HealthStatus}}'

Use Docker Compose

Development Stack

# compose.yaml (no top-level "version:" key; it is obsolete)
services:
  web:
    image: nginx:1.27
    ports:
      - "8080:80"
    volumes:
      - ./html:/usr/share/nginx/html:ro
    depends_on:
      db:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 256M

  db:
    image: postgres:17
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
    secrets:
      - db_password

volumes:
  pgdata:

secrets:
  db_password:
    file: ./secrets/db_password.txt

docker compose up -d
docker compose ps
docker compose logs -f web
docker compose watch          # sync/rebuild on file changes (develop.watch section)
docker compose down           # keep volumes
docker compose down -v        # also remove named volumes (destroys data)

Single-Host Production Overrides

# compose.prod.yaml, used as: docker compose -f compose.yaml -f compose.prod.yaml up -d
services:
  web:
    image: registry.example.com/myapp:${TAG}
    restart: unless-stopped
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: "2.0"
          memory: 512M
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 5s
      retries: 3

Replicas and published ports

With replicas: 3 on one host, do not publish a fixed host port for the service (the second replica would conflict). Put a reverse proxy (Traefik, nginx) in front instead, or publish a port range.

Configure Networking

# User-defined bridge with explicit subnet
docker network create --driver bridge --subnet 10.10.0.0/24 appnet

# Or let Docker pick a subnet of a given size from its default pools (v29+)
docker network create --subnet 0.0.0.0/24 appnet2

# Attach a running container, then inspect members
docker network connect appnet web
docker network inspect appnet -f '{{range .Containers}}{{.Name}} {{.IPv4Address}}{{"\n"}}{{end}}'

# Name resolution between containers on the same user-defined network
docker exec web getent hosts db

# Publish only on localhost (not reachable from other hosts, not affected by ufw bypass)
docker run -d -p 127.0.0.1:5432:5432 postgres:17

# macvlan network (v29+: set --gateway explicitly if containers need a default route)
docker network create -d macvlan --subnet 192.168.50.0/24 --gateway 192.168.50.1 -o parent=eth0.50 vlan50

Secure the Daemon and Containers

# Remote access over SSH instead of an exposed TCP socket
docker context create prod --docker "host=ssh://deploy@prod-host.example.com"
docker --context prod ps

# Mutual TLS on TCP (only if SSH is not an option)
dockerd --tlsverify --tlscacert=ca.pem --tlscert=server-cert.pem --tlskey=server-key.pem -H=0.0.0.0:2376

# Confirm the security options in effect
docker info -f '{{ .SecurityOptions }}'
docker inspect -f '{{ .HostConfig.SecurityOpt }} {{ .HostConfig.CapDrop }}' api

Do not work around the CVE-2026-31431 seccomp hardening with unconfined

Engine 29.4.2 and later block AF_ALG sockets. If a 32-bit workload breaks, follow the release-note workaround (a targeted moby/profiles seccomp profile for that container only, on a patched kernel) rather than --security-opt seccomp=unconfined.

Monitor Containers

# Real-time resource usage
docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.NetIO}}"

# Logs since a point in time
docker logs --since 1h --tail 100 -f web

# Event stream filtered to container deaths and OOM kills
docker events --since '2026-09-24T00:00:00' --filter type=container --filter event=die --filter event=oom

# Disk usage by images, containers, volumes, build cache
docker system df -v

Clean Up Disk Space

# Selective cleanup
docker container prune              # stopped containers
docker image prune -a --filter "until=24h"
docker volume prune                 # unused anonymous volumes (add -a for named ones)
docker network prune
docker buildx prune --filter until=72h

# Everything unused, including volumes (destructive)
docker system prune -a --volumes

Move Images Between Hosts (Air-Gapped)

docker save myapp:latest | gzip > myapp.tar.gz
docker load -i myapp.tar.gz

# v29+: save or load selected platforms of a multi-platform image
docker image save --platform linux/amd64,linux/arm64 -o myapp-multi.tar myapp:latest

Troubleshoot Common Issues

Issue	Diagnosis	Fix
Container OOMKilled	docker inspect -f '{{.State.OOMKilled}}' web	Raise --memory or fix the leak
"Too many open files" after upgrading to v29	docker exec web sh -c 'ulimit -n' shows 1024	Set --ulimit nofile=65536:65536 or default-ulimits
Disk space exhausted	docker system df	Prune (see above); on the containerd store check /var/lib/containerd too
Images "disappeared" after enabling containerd store	docker info -f '{{ .DriverStatus }}'	Expected: switch back, or re-pull / docker load
DNS resolution fails between containers	docker exec web getent hosts db	Use a user-defined network (embedded DNS 127.0.0.11), not the default bridge
toomanyrequests on pull	Docker Hub pull limit reached	docker login, use a mirror, or a paid plan
Daemon fails to start with nftables	journalctl -u docker mentions IP forwarding	Enable forwarding sysctls or set "ip-forward": false
Port conflict	docker port web; ss -ltnp	Change the host port mapping
Slow builds	Frequent cache misses	Reorder Dockerfile, add cache mounts, use registry cache

# Debug an image that will not start
docker run --rm -it --entrypoint /bin/sh myapp:latest

# Attach a debug shell to a running (even distroless) container, Docker Desktop / Docker Debug
docker debug web

# What changed in the container filesystem
docker diff web

# Copy files out
docker cp web:/var/log/app.log ./app.log

# Daemon logs
journalctl -u docker --since "10 min ago"

docker debug

docker debug ships with Docker Desktop and has been free for all users since Desktop 4.49.0 (2025-10-23). On plain Engine hosts without it, use docker run --rm -it --pid container:web --network container:web busybox.

Sources

• Install Docker Engine on Ubuntu
• Docker Engine v29 release notes
• containerd image store
• Docker with nftables
• Rootless mode
• Docker Hub usage and limits
• Docker CLI reference
• Dockerfile reference
• Buildx cache backends
• Compose file reference
• Docker Hardened Images