Builds: validate inputs and preserve CI progress

[tdejager]

We want to fix the correctness problems in the current build setup before changing the architecture in #24. Retrying a build should keep the work it already finished, and an older run should not be able to replace a newer release.

Before

A full-rebuild retry could delete the packages it had just rebuilt. Delayed generators could replace newer build branches, and old jobs could still upload. Local commands could use stale recipes, while patch checks missed inherited shared patches. CI installation used frozen mode without consistently requiring the lockfile to match the manifest.

After

• Cache keys include the build inputs. Unchanged retries keep their completed packages; changed inputs start a fresh distro/platform cache. This deliberately invalidates dependents too, rather than trying to reuse packages that may no longer be compatible.
• Generation uses the latest main after acquiring the distro lock. Branch updates use a lease, and build jobs check freshness before building and before each upload. Publishers for the same distro/platform do not overlap. queue: max keeps an old rerun from replacing the newest pending build. If the 100-run queue fills, rerun the current build-branch head after it drains.
• Local build commands reject missing or stale recipes and tell you how to regenerate them. Patch checks use Vinca's shared/distro resolution. Repository-wide sorting and pin rendering use Python instead of Unix shell loops.
• CI uses locked execution and waits for sanity and regression checks before starting the build matrix. Auto-merge reads current labels and requires an App/PAT to enable it. Only automerge label jobs enter the per-PR queue, so unrelated label changes cannot discard a pending disable run. Upload credentials go through the environment, and generated jobs explicitly grant checkout read access.

Build branches, stage barriers and channel-based dependency handoff stay as they are. The planning/building/publishing refactor belongs in #24.

Testing

34 regression tests passed in an isolated checkout of this commit, including real local patch builds and Git race cases:

pixi run --locked python -m unittest discover -s tools/tests -v

On the completed series, pixi run --locked rs check, lock validation, sorting and pin rendering passed. Command-level checks exercised cache retries, stale-lock rejection, single-package rendering, inherited patch application, and macOS/Windows workflow generation. A follow-up run generated the Jazzy/osx-arm64 workflow with queue: max; the Git freshness tests also passed. actionlint -shellcheck= -pyflakes= .github/workflows/automerge.yaml and all 17 CI tests passed after the auto-merge queue fix. GitHub's hosted queue scheduling was not exercised. No full ROS build, native Windows run or real upload was performed.

Closes #25
Closes #26
Closes #27
Closes #29
Closes #35
Closes #36
Closes #37
Closes #38

Independent PR

This PR is based directly on main and can merge independently of #40 (maintenance) and #41 (imports/naming). Its tests pass without either of those branches. Git merge checks verified all six merge orders: each combines without conflicts and produces the same intended tree. All 78 regression tests pass when the independent branches are combined.

---

PR description written by Astra on 2026-10-09. Sol reviewed the local CLI and patch-check changes.

Follow-up updated by Astra on 2026-10-09. A separate gpt-6-astra:high agent reviewed all three PRs, found the additional auto-merge queue race in #39, and confirmed its fix. No actionable findings remained; the review was read-only.

[Tobias-Fischer]

Thanks! I'd split this. The small fixes are worth keeping, but the cache-epoch and freshness machinery costs more than it saves:

• Keep:
  • automerge reads the live label;
  • locked pixi installs;
  • the Anaconda token passed via env instead of argv;
  • patch checks through vinca's resolver;
  • the generate-<distro> concurrency group.
• Cache epoch (tools/ci_config.py): every shared/distro file and the generated recipes go into the key, so any push or merge from main wipes the whole PR cache. That's the opposite of "preserve CI progress". It also leaves evict_cache unused.
• release.py:
  • every staged job runs a full fetch and a main-freshness check, so any unrelated merge fails all in-flight uploads, and old runs can't be rerun;
  • queue: max plus the concurrency group already gives most of the ordering.
• Local builds: blocking them on a recipe fingerprint that hashes all of shared/ and .scripts/ is heavy for what it prevents.
• CI and conflicts: CI didn't run on this PR. It conflicts with Drop the distro mutex's run_constraints #23 (pytest vs unittest discover, same spot in testpr.yaml) and overlaps ABI check: compare pull-request builds with the released ones #20.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

[Tobias-Fischer]

This was closed automatically when we rewrote main's history into topic commits (GitHub closes PRs whose base history is replaced). The parts we're keeping from #39–#41 are in #46, with you as co-author. The reasons for leaving out the rest are in the review comments above. Thanks for the audit; it found several real bugs!

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer