Repository navigation
ci: run the x86-64 benchmarks on the size-attester7870-x64 runner - #527
Merged
kevaundray merged 4 commits intoOct 7, 2026
Merged
kevaundray merged 4 commits into
kevaundray merged 4 commits into
Conversation
The x86-64 bench testbed moves from ubuntu-latest to the self-hosted runner size-attester7870-x64 (an Intel Xeon Gold 6548N, 32 vCPUs, 64 GB). Base and PR build and run on that one machine, so the testbed is built for target-cpu=native instead of haswell and measures the AVX-512, VPCLMULQDQ and GFNI arms. A new step prints the CPU and those flags, and the rust-cache key includes the CPU model so a native build is never restored on another CPU. In a fork's own repository, which has no such runner, the testbed falls back to ubuntu-latest built for haswell. arm64 is unchanged.
A workflow_dispatch trigger with optional base and head inputs runs the same per-testbed comparison: head defaults to the branch the run is dispatched on, base to the head's first parent. scripts/ab.sh already takes --base in --ci mode; the workflow fetches the base ref and passes its commit. A dispatch run leaves artifacts only: bench-comment.yml posts for pull_request runs alone.
eth-act/zkevm-benchmark-runs, whose jobs these ghr runners do pick up, requests [size-attester7870-x64, self-hosted-ghr]; ask for the same set.
Benchmarks: base against this PR, on one runnercfd0147 merged into its base, against the base a4c8699: each benchmark run in turns on one runner, base and PR 5 times each. A size is shown when it changed; a time or a peak memory when it moved the same way in every round and its median by at least 1% (the change is the median ratio, with the rounds' range). Proving timeProver stages
Verifying time
Time per operation
Every resultProving time
Prover stages
Verifying time
Time per operation
Proof size
Peak memory
allocations
heap-peak
|
tcoratger
approved these changes
Oct 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the
x86-64bench testbed fromubuntu-latestto the self-hosted runnersize-attester7870-x64, so x86 benchmarks run on a fixed, known machine.The new machine adds three things: a faster CPU, more cores (32 vCPUs against the 4 of GitHub's runner) and the native build (the AVX-512, VPCLMULQDQ and GFNI paths the haswell pin disabled). The single-thread cases (
-1thread) and the kernels' per-operation times remove the core count, so they isolate per-core speed and the code paths. Times are the median with the range of the runs in brackets; speedup is old median over new median.Prover stages of a few cases
leanxmss-100
leanxmss-100-1thread
fibonacci-asm-2000000
shielded-258
This PR's own comparison. Base and PR are the same proving code here, so the bench comment's deltas are noise: it flags no proving time, and on x86 it shows only leansphincs-26's verifying time at +1.45% (+0.46% to +2.09%) and
ntt-forward-22at -5.01% (-8.66% to -1.37%).The base side also runs on the new runner, since ab.sh builds base and head in the same job, so comparisons stay fair from the first run.