Repository navigation
feat(docker): prune builder installations using buildcache check - #460
Conversation
Add containers/eic/check-buildcache.py to check concrete specs against configured OCI binary buildcaches and output missing spec DAG hashes. In builder_installation_default and builder_installation_custom stages of containers/eic/Dockerfile, check if all packages already exist in the buildcache. If so, skip spack install; if only a subset of packages are missing, install only those specs (which Spack then builds and autopushes to the buildcache). This speeds up container builds by avoiding redundant package downloads and compilation in the builder track, while preserving the invariant that the buildcache is fully populated for the runtime track.
There was a problem hiding this comment.
🔵 Needs a closer look
It changes the core multi-stage build pipeline in ways whose correctness depends on registry auth, autopush timing, and concurrency that can only be validated by passing both the GitHub Actions and GitLab CI builds.
2 open issues
What changed in this PR
This PR adds a buildcache pre-check optimization to the multi-stage EIC container build (containers/eic/Dockerfile). A new helper script queries each concrete, non-external spec of a Spack environment against the configured OCI binary mirrors (preferring autopush mirrors), and the builder installation stages use its output to skip spack install entirely when everything is cached, install only the missing spec hashes when partially cached, or fall back to a full environment install if the check fails. This reduces build time and disk I/O in the builder layers while keeping the runtime stages' --use-buildcache only invariant intact (missing specs are compiled and autopushed before runtime stages run).
Changes:
- New
containers/eic/check-buildcache.pythat auto-detects OCI mirrors and reports missing spec DAG hashes using Spack's internal OCI API, with graceful exits. builder_installation_defaultandbuilder_installation_customgatespack installon the check result via/tmp/rebuild.txt, with a full-install fallback.- The script is copied once into
builder_concretization_defaultso it is available to all builder stages but absent from runtime images.
| File | Description |
|---|---|
| containers/eic/check-buildcache.py | New script that checks environment specs against OCI buildcaches and emits missing DAG hashes; uses Spack internal OCI helpers. |
| containers/eic/Dockerfile | Copies the script into builder stages and wraps spack install in both builder installation stages with a buildcache pre-check and fallback. |
Review effort: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
It changes the critical dual-CI container build path, depends on unverifiable private Spack OCI internals, and contains a fallback path that hard-fails instead of falling back as described, so it warrants human review.
1 open issue
2 resolved since last review
Review effort: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
It reworks the core multi-stage builder install flow (which must pass both GitHub and GitLab CI) and relies on private Spack OCI internals that can't be verified here, and it contains a set -e/fallback logic gap that should be resolved by a human reviewer.
Review effort: Balanced
Summary
This PR applies the buildcache pre-check optimization to the Docker container build in
containers/eic/Dockerfile.Changes
containers/eic/check-buildcache.pywhich queries concrete non-external specs against configured OCI binary mirrors (auto-detecting mirrors from Spack configuration) using Spack's internal OCI API.containers/eic/Dockerfile:check-buildcache.pyinto/usr/local/bin/inbuilder_concretization_default(accessible to all builder stages, absent from runtime images).builder_installation_defaultandbuilder_installation_custom, checks the buildcache for existing packages:spack installcompletely, saving significant build time and disk I/O in builder layers.Invariant Maintained
The runtime stages rely on
--use-buildcache only. The builder stages guarantee that any missing packages are compiled and pushed before runtime stages execute.Upstreaming
This functionality is under consideration for upstreaming, but no PR is available for linking yet.