Skip to content

Use Lake comparator on Namespace evaluation runners - #2162

Merged
kim-em merged 2 commits into
mainfrom
migrate/lake-comparator-namespace
Oct 22, 2026
Merged

kim-em merged 2 commits into
mainfrom
migrate/lake-comparator-namespace

Conversation

@kim-em

@kim-em kim-em commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Replace the current submission workflow's standalone comparator/exporter, Landrun, and separately built NanoDa with the toolchain verifier provided by the benchmark. Run evaluations on the tested Namespace 16 CPU / 32 GB profile with privileged namespaces, retaining the evaluation time budget.

Port the cold Lean Pool dependency test from a hand-written Landrun invocation to real lake comparator, checking that the read-only shared layout fails while the adapter's private build outputs succeed with both Lean and NanoDa. Historical replay images and locks are unchanged.

Prerequisites #2161 and leanprover/lean-eval#665 are merged. The fresh Namespace adapter canary passed with exactly three attempts, two acceptances, and one expected rejection. Server intake receives this immutable workflow revision through the existing protected dispatch-tag and Worker promotion workflow.

Validation: 1,129 offline tests passed (4 skipped); the real cold-cache sandbox test passed locally with Lean 4.35.0-rc3 and both checkers; actionlint, pin audit, and diff checks passed. Final CI (including the shell cache wrapper) passed: https://github.com/leanprover/lean-eval-submissions/actions/runs/39858687818. Fresh Namespace adapter canary: https://github.com/leanprover/lean-eval/actions/runs/39859118231. Earlier Namespace compatibility evidence: https://github.com/leanprover/lean-eval/actions/runs/39724318459.

@kim-em

kim-em commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Prerequisite: leanprover/lean-eval#665 must merge first (bubblewrap provisioning #2161 is already merged). Keep this PR in draft until the controlled adapter canary passes and evaluations frozen to the old benchmark have drained.

@kim-em
kim-em marked this pull request as ready for review October 22, 2026 22:24
@kim-em
kim-em merged commit cf42de8 into main Oct 22, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant