Repository navigation
fix: restore TauCeti caches with atomic downloads and retries - #664
Merged
Merged
Conversation
Collaborator
Author
|
Please merge leanprover/lean-eval-submissions#2156 first. This PR exports |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TauCeti cache restoration can fall back to compilation after an artifact download fails, aborting regeneration even when dependency pins are correct. Interrupted single-file downloads can also leave partial cache entries.
Use verbose cache-only restoration with one whole-step retry, plus a curl wrapper with bounded download retries, connection/stall/transfer limits, and atomic publication. Preserve complete diagnostics on failure and export the wrapper for subsequent trusted builds.
Validation: all 62 benchmark Python tests pass with the real-Lake test enabled, including HTTP/partial-transfer tests and a real-Lake restoration fixture. The fixture proves an interrupted transfer leaves no partial cache artifact and an unchanged-cache retry restores and imports the module without compilation. Script-level tests cover restoration retry/exhaustion, export propagation, and the existing pin mismatch guard. A dedicated CI workflow runs the new tests. Full pinned TauCeti restoration also passed locally in cache-only mode (14,602 jobs, no TauCeti compilation). Bash syntax, workflow lint, action-pin audit, and diff checks pass.
Depends on leanprover/lean-eval-submissions#2156 for the optional
CURLexport. Merge that PR first.Refs leanprover/lean-eval-submissions#1894.