Building Docker Images and Dockerfiles

TL;DR

You'll learn how a Dockerfile's instructions turn into image layers, and how to order them so builds are fast and cacheable. You'll write a real multi-stage Dockerfile that produces a small, secure production image. You'll also know the common mistakes that bloat images or leak secrets, and how to avoid them.

1. The Mental Model

A Dockerfile is a recipe; each instruction bakes one layer on top of the last, like stacking transparent sheets. Docker only redoes the sheets that changed since your last build — everything above an unchanged sheet gets reused instantly. Layers are immutable and shared across images, which is why Docker is fast and disk-efficient. An image is just a stack of cached, read-only diffs, and your job is to order that stack so the expensive diffs sit at the bottom.

2. The Core Material

2.1 How layers and the build cache actually work

Photo by Abhishek Gupta on Pexels

Every instruction in a Dockerfile that changes the filesystem — FROM, RUN, COPY, ADD — creates a new layer. Instructions like ENV, WORKDIR, EXPOSE, CMD, LABEL only change image metadata, so they're cheap but still invalidate cache for anything after them if their value changes.

Docker builds top-to-bottom. For each instruction it computes a cache key from the instruction text (and, for COPY/ADD, a hash of the files being copied). If the key matches a previous build's layer, Docker reuses that layer without re-executing anything. The moment one instruction misses the cache, every instruction after it also misses, even if those later instructions are identical to before. This is the single most important fact for writing fast Dockerfiles: put things that change rarely (base image, system packages) near the top, and things that change often (your application source code) near the bottom.

# BAD: cache busts on every source change, re-installs deps every time
FROM node:20-slim
WORKDIR /app
COPY . .
RUN npm install
CMD ["node", "server.js"]

# GOOD: dependency install is cached unless package*.json changes
FROM node:20-slim
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY . .
CMD ["node", "server.js"]

In the bad version, editing one line of server.js invalidates COPY . ., which invalidates RUN npm install right after it — so every code change triggers a full dependency reinstall. In the good version, npm ci only reruns when the lockfile actually changes.

Check how big your context is and what's being sent to the daemon:

docker build -t myapp .
# Sending build context to Docker daemon  45.2MB

That context includes everything in your build directory unless excluded — more on that below.

2.2 Writing a real Dockerfile: instructions that matter

Photo by Malte Luk on Pexels

• FROM picks your base image and starting layer set. Prefer specific, minimal tags (python:3.12-slim over python:3.12 or, worse, python:latest) — smaller attack surface, fewer dependencies, and reproducible builds since latest can silently change underneath you.
• WORKDIR sets the working directory for subsequent instructions and creates it if missing. Always use this instead of RUN cd ..., because RUN spawns a new shell each time and cd doesn't persist across instructions.
• COPY vs ADD — use COPY by default. ADD can auto-extract tar archives and fetch URLs, which is surprising, magical behavior you rarely want. Only reach for ADD when you specifically need that extraction.
• RUN executes a command and commits the result as a layer. Chain related commands with && and clean up in the same RUN line — if you apt-get install in one layer and rm -rf /var/lib/apt/lists/* in a later layer, the earlier layer still contains those files on disk; only deleting within the same layer actually shrinks the image.

RUN apt-get update \
    && apt-get install -y --no-install-recommends curl \
    && rm -rf /var/lib/apt/lists/*

• ARG vs ENV — ARG is a build-time-only variable (not present in the final container unless you also set it as ENV); ENV persists into the running container and into every subsequent layer. Use ARG for things like version pins passed via --build-arg, and ENV for runtime configuration like NODE_ENV=production.
• USER switches the effective user for subsequent RUN/CMD/ENTRYPOINT instructions. Default is root, which is a real risk if the container is compromised — always drop to a non-root user before your final CMD.
• CMD vs ENTRYPOINT — ENTRYPOINT defines the fixed executable; CMD supplies default arguments that can be overridden at docker run. Use ENTRYPOINT for "this image always runs this binary" and CMD for the default behavior users can override.

2.3 Multi-stage builds: separating "build" from "ship"

Photo by Erik Mclean on Pexels

A multi-stage build uses several FROM statements in one Dockerfile. Earlier stages can have compilers, dev dependencies, and test tools; only the artifacts you explicitly COPY --from=<stage> make it into the final image. This is how you get a 20MB production image out of a build environment that's 1GB+.

flowchart LR
    A["Stage: builder\n(FROM node:20)"] -->|"npm ci + npm run build"| B["dist/ artifacts"]
    B -->|"COPY --from=builder"| C["Stage: runtime\n(FROM node:20-slim)"]
    C --> D["Final image\n(no dev deps, no source, no compiler)"]

# --- Stage 1: build ---
FROM node:20 AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build        # produces /app/dist

# --- Stage 2: runtime ---
FROM node:20-slim
WORKDIR /app
RUN useradd --create-home appuser
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
USER appuser
EXPOSE 3000
CMD ["node", "dist/server.js"]

The builder stage never ships — its 700MB of devDependencies, compilers, and source maps are discarded. Only dist/ and runtime node_modules cross into the final image. You can verify the size difference directly: docker images will show the final tag dramatically smaller than an equivalent single-stage build.

You can also target an intermediate stage for debugging without building the rest:

docker build --target builder -t myapp:debug .

3. Worked Example

Let's build a Python Flask API from scratch and measure the impact of each optimization.

Attempt 1 — naive Dockerfile:

FROM python:3.12
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "app.py"]

docker build -t api:v1 .
docker images api:v1
# REPOSITORY   TAG   SIZE
# api          v1    1.02GB

Problems: python:3.12 (full Debian base, ~1GB on its own) instead of slim; COPY . . before installing dependencies means every code edit reinstalls all packages; running as root; no .dockerignore so .git, __pycache__, and local virtualenvs get sent into the build context.

Attempt 2 — fixed:

First, add a .dockerignore next to the Dockerfile:

.git
__pycache__
*.pyc
.venv
.env
tests/

FROM python:3.12-slim AS base
WORKDIR /app

# Dependencies layer — only rebuilds when requirements.txt changes
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Application layer — rebuilds on every code change, but that's cheap
COPY . .

RUN useradd --create-home appuser
USER appuser

EXPOSE 5000
CMD ["python", "app.py"]

docker build -t api:v2 .
docker images api:v2
# REPOSITORY   TAG   SIZE
# api          v2    152MB

That's a 6.7x size reduction, purely from choosing slim, excluding junk via .dockerignore, and using --no-cache-dir (pip's own download cache, not Docker's layer cache — it avoids storing .whl files twice). Now edit only app.py and rebuild:

docker build -t api:v3 .
# => [3/5] RUN pip install ... CACHED
# => [4/5] COPY . .            <-- only this reruns

The dependency install shows CACHED because requirements.txt didn't change — the build finishes in under a second instead of reinstalling 15 packages.

4. Production Pitfalls & Best Practices

4.1 Real-World Best Practices

Photo by Yoga Vidya Mandiram on Pexels

• Pin base image versions and digest where reproducibility matters: FROM python:3.12.4-slim@sha256:abc123... guarantees the exact same base bytes every build, not just "whatever 3.12.4-slim resolves to today."
• Order instructions from least-changing to most-changing — OS packages, then language dependencies, then app code — to maximize cache hits.
• Use .dockerignore aggressively: exclude .git, node_modules, build artifacts, and secrets so they never enter the build context (faster builds, and they literally can't leak into a layer).
• Use multi-stage builds for any compiled or transpiled language (Go, TypeScript, Java, Rust) so the final image has only the binary/artifact, not the toolchain.
• Run as a non-root USER in the final stage; combine with --read-only and dropped Linux capabilities at docker run time for defense in depth.
• Use build secrets via --mount=type=secret, never ARG/ENV, for API keys or credentials needed only during build (e.g., private package registry tokens).
• Label your images with LABEL org.opencontainers.image.source=... and version metadata — invaluable once you have dozens of images in a registry.

4.2 Common Bugs & Anti-Patterns

1. Secrets baked into layers via ARG/ENV:

# BAD: visible forever in image history, even after a later RUN removes the file
ARG NPM_TOKEN
RUN echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > .npmrc && npm ci && rm .npmrc

# FIXED: secret never becomes part of any layer
# syntax=docker/dockerfile:1
RUN --mount=type=secret,id=npmtoken \
    NPM_TOKEN=$(cat /run/secrets/npmtoken) npm ci
# build with: docker build --secret id=npmtoken,src=./token.txt .

2. Unpinned, mutable tags causing "works on my machine":

# BAD
FROM node:latest

# FIXED
FROM node:20.11.1-slim

3. Cleanup in a later layer doesn't shrink the image:

# BAD: apt cache from layer 2 is still physically stored, even though layer 4 deletes it
RUN apt-get update && apt-get install -y build-essential
RUN make && make install
RUN rm -rf /var/lib/apt/lists/*

# FIXED: delete within the same RUN/layer that created the files
RUN apt-get update && apt-get install -y build-essential \
    && make && make install \
    && apt-get purge -y build-essential \
    && rm -rf /var/lib/apt/lists/*

4. Missing .dockerignore leaking .env or .git into the image:

# BAD: no .dockerignore — COPY . . grabs .env with real DB passwords
COPY . .

# FIXED: .dockerignore contains
.env
.git
*.log

4.3 When to Use What (Decision Matrix)

5. Now Try It

Build a Dockerfile for a small script that prints "hello" using this app.py:

import os
print(f"Hello from {os.environ.get('NAME', 'Docker')}!")

Task: Write a multi-stage-ready, cache-friendly Dockerfile using python:3.12-slim, that: (1) creates and uses a non-root user, (2) sets ENV NAME=Container so the output reads "Hello from Container!" by default, (3) includes a .dockerignore excluding .git and __pycache__. Build it twice — the second time after touching only app.py — and confirm in the build log that no step unnecessarily reruns.

Success looks like: docker run --rm yourimage prints Hello from Container!, docker images shows a size under 180MB, and your second build log shows at least one CACHED step.