Skip to content

Latest commit

 

History

History
129 lines (107 loc) · 8.71 KB

container-builds.md

File metadata and controls

129 lines (107 loc) · 8.71 KB

Building containers with a persistent Cargo cache

The root Dockerfile requires BuildKit. With a current Docker installation, docker build uses it by default; clients that report a missing Buildx plugin need their Docker distribution's client plugin installed. The default target remains the production image. No CI service, registry, remote cache, or particular machine size is required.

docker build --target verification .
docker build -t hatchdoor:local .

verification runs cargo test --locked as an unprivileged build user, including permission-sensitive tests. It runs the default-feature suite, just like the same Cargo command outside Docker; feature-gated model/evaluation tests are not implicitly enabled. Production builds do not depend on this target. Callers that require tests before publication must run it first and propagate failure. The suite needs four things beyond the Rust sources: the git program, which chef installs; frontend/vite.config.ts, which the stage copies so the SPA test can compare the backend reserved prefixes with the service-worker denylist; frontend/public/link-preview.png, which the link-preview test reads to check the actual image's dimensions and size; and frontend/src/lib/noteComments.cases.json, the shared cases the Rust link-preview test uses to check comment stripping against the frontend. A new test that reads another file outside src/ and docs/user-vault needs its own COPY here. verification derives from chef rather than from the dependency stage, so its first run on a builder compiles the test profile from scratch even when a release build is already cached. That is the expected cost of a separate profile, not a regression.

Cargo registry downloads, Git checkouts, and /app/target use BuildKit cache mounts shared across Cargo stages, separated by target platform. The cache namespace defaults to hatchdoor; --build-arg CARGO_CACHE_NAMESPACE=<name> selects an isolated namespace. Keep it stable across source and lockfile changes for reuse. Test/dev and release artifacts occupy different profile directories; the first use of each profile still compiles its dependencies.

The chef recipe and release dependency cook precede the real source COPY. An ordinary edit therefore retains the dependency step. Manifest, lockfile, and Cargo target changes can invalidate it. The final Cargo build uses --locked and validates its own inputs even when cache contents have been removed. The binary is copied outside the target mount during the same RUN, so the runtime image never depends on a mounted cache at runtime.

Optional build controls

Build argument Default Effect
CARGO_BUILD_JOBS Unset (Cargo default) Limit simultaneous Cargo compilation jobs, including chef installation.
CARGO_PROFILE_RELEASE_INCREMENTAL false Opt into compiler reuse within changed release crates.
CARGO_PROFILE_RELEASE_CODEGEN_UNITS 16 Preserve Cargo's normal non-incremental release value when opting into incrementality.
CARGO_CACHE_NAMESPACE hatchdoor Isolate persistent caches on a shared builder.
GIT_SHA Empty The commit the image was built from. Written to the org.opencontainers.image.revision label and compiled into the binary, where a build that is not a release shows it in its version, as 2.8.0 (dev abc123).
HATCHDOOR_IMAGE Empty published marks the published image. Compiled into the binary; the opt-in usage report then sends image: podman when the container has Podman's /run/.containerenv and image: docker otherwise. docker and podman count as published. Any other value, or none, reports source.
VERSION Empty The release version, such as 2.8.0, written to the org.opencontainers.image.version label. It also marks the build as a release: when it equals the version in Cargo.toml, the binary reports that plain version and the opt-in usage report says channel: stable. Without it, or with any other value, the build is a development build and reports dev. Pass it only for a release: a nightly or test image that passes the current version would report itself as that release. The version number itself always comes from Cargo.toml.

Image labels

The runtime stage labels every image, whatever builds it. Six OCI labels are fixed in the Dockerfile: title, description, licenses, source, url and documentation under org.opencontainers.image.. Update notifiers, scanners and Renovate read source to find the repository behind battermanz/hatchdoor. io.modelcontextprotocol.server.name is the name the MCP registry compares with a server.json to prove who owns the image, so the two must change together.

version and revision come from the VERSION and GIT_SHA build arguments and are empty strings when a build passes neither:

docker build --build-arg VERSION=2.8.0 --build-arg GIT_SHA="$(git rev-parse HEAD)" -t hatchdoor:local .
docker inspect hatchdoor:local --format '{{json .Config.Labels}}'

description, licenses and source repeat Cargo.toml. Nothing runs the comparison for you, so after changing a label or one of those Cargo.toml fields, run node --test scripts/dockerfile-labels.test.mjs.

Resource-limited builders

For a constrained builder, a caller can select one Cargo job and prebuild the Rust stage before the full image, preventing Rust/frontend build overlap:

docker build --target verification --build-arg CARGO_BUILD_JOBS=1 .
docker build --target rust-builder --build-arg CARGO_BUILD_JOBS=1 \
  --build-arg CARGO_PROFILE_RELEASE_INCREMENTAL=true .
docker build --build-arg CARGO_BUILD_JOBS=1 \
  --build-arg CARGO_PROFILE_RELEASE_INCREMENTAL=true -t hatchdoor:local .

Use identical context and build arguments between the last two commands, and retain the same builder cache. The default build leaves BuildKit free to schedule independent stages. Cargo jobs do not bound every rustc/linker thread or the frontend. Incremental release builds consume additional disk and can still recompile the changed crate and relink; they do not promise a fixed speedup or identical runtime performance. Normal release optimization defaults remain in effect unless a caller opts in.

Cache lifetime and verification

Cache mounts belong to the selected builder. With Docker's default driver they live under that daemon's data root, so retaining that storage permits reuse between client/job containers. A persistent disk does not prevent BuildKit pruning/garbage collection. Exporting ordinary image layers is not a guarantee that mutable cache mounts follow to another builder. Correct builds must also work with empty caches.

Prune the layer cache and the Cargo cache mounts together, or prune neither. docker buildx prune clears both, which costs the next build a cold compile and nothing worse. A mount-only prune (--filter type=exec.cachemount) leaves the dependencies stage reported as CACHED while /app/target is empty, so cargo chef cook never runs and the final Cargo build compiles every dependency instead. Measured on a 4-core amd64 builder, one edited source line costs 13s with a warm mount and 614s in that mixed state, against 107s on the Dockerfile that predates these cache mounts. BuildKit's own garbage collection can produce the same mix without anyone asking for it, so treat a build that unexpectedly compiles hundreds of crates as a sign the mount was evicted, and clear the layer cache before looking any further.

Use --progress=plain to inspect cached build steps. An unchanged-source build can skip an entire Cargo RUN, including a cached verification RUN. To exercise Cargo again, make a source edit in a disposable checkout. Confirm chef's cook stays cached and unchanged dependencies do not compile again. Also verify a real dependency/lockfile update and an isolated empty namespace. Never substitute an unreviewed lockfile edit for --locked validation.

Report wall time with cache state, source, toolchain, profile, and jobs. Docker layer hits, Cargo fresh crates, and rustc internal incremental reuse are different metrics. Stable Cargo output does not expose a single compiler cache-hit rate. For a remote Docker daemon, client RSS is not build memory; sample the build cgroup or the daemon's enclosing cgroup and state whether page cache, swap, and other jobs are included.

References: Docker cache mount semantics, Docker default driver, Cargo profiles.