Layer Caching, Multi-stage Builds and BuildKit
Fast, small and reproducible images come from understanding how the build cache keys each instruction, keeping heavy work out of the final stage, and using BuildKit features such as cache, secret and bind mounts and external cache backends.
Key points
- 1
Each instruction is a cache step: COPY and ADD are keyed on a checksum of the copied files (mtime ignored), RUN on its command string. A miss invalidates every step after it.
- 2
Order steps from least to most frequently changing: base image, system packages, dependency manifests and install, then source code.
- 3
Layers are additive: delete files in the same RUN that created them, or build in a separate stage and COPY --from only the artefacts into a slim final stage.
- 4
BuildKit builds only the stages the target depends on, and runs independent stages in parallel.
- 5
Cache mounts persist package downloads in the builder without entering the image. Secret mounts expose credentials to one RUN and are never stored, unlike build args, which show in docker history.
- 6
Ephemeral CI runners start cold: export and import layer cache with --cache-to/--cache-from (registry, gha, s3, local). Use mode=max to keep builder-stage layers.
- 7
For multi-platform images, run compilers natively with FROM --platform=$BUILDPLATFORM and cross-compile using TARGETOS/TARGETARCH instead of slow QEMU emulation.
Common traps
A separate
RUN apt-get updateline is cached forever: combine it withapt-get installin one RUN.An ARG declared early invalidates every later RUN when its value changes, even RUN steps that never reference it.
Changing a build secret's value does not invalidate the cache; pass a cache-busting build arg if the step must rerun.
Test yourself on Layer Caching, Multi-stage Builds and BuildKit
Ten questions, with the answer and explanation after each one.