CI perf: ci.yml e2e Gradle PR tier — all 4 Gradle lines on every PR push, 39% of a CI run (~9,800 Linux job-min/day) #1267

Measurement

Window: 2026-10-09 08:17–12:24 UTC. 25 sampled CI runs with every job measured: 8 merge_group, 3 push and 14 non-draft pull_request. Example runs: 37922159538 (PR), 37908333141 (merge_group), 37910917261 (push).

The e2e job's real-Gradle PR tier has 16 ubuntu legs: 4 Gradle lines × 4 legs (ci.yml:1242-1257). The 4 legs per line are agent+discovery+hosted_[345], hosted_[b-p], the hosted catch-all, and vendor+multi-project.

• A non-draft PR run of CI costs ~370 Linux job-min. The Gradle tier is ~39% of that.
• Of the 425 Gradle e2e jobs in the sample, 425 succeeded and none failed.
• PR volume: 100 CI pull_request runs between 03:57 and 12:24, of which 41 were full non-draft successes. That is about 130 full PR runs/day, plus partial cancelled ones.

Where the time goes

• Almost all of it is test time: across the sample, Run e2e tests took 3,525 of the ~3,650 Gradle-leg minutes.
• Per-job setup is small: Set up job 46 min, e2e binaries download 23 min, checkout 14 min, Gradle install < 5 min.
• So merging legs, as CI perf: e2e fan-out — 23 e2e-macos + ~118 Linux e2e jobs/run, 82–87% under 2 min, 30–58% of job time is setup (~1,200 macOS + ~6,000 Linux job-min/day) #1172 and CI perf: ci.yml e2e — ~120 sub-2.5-min Linux/Windows legs per run, 64% setup overhead (~12,000 job-min/day) #1178 propose, does not help here. The cost is the 4× repetition of the same ~36-min test set, once per Gradle line, on every PR push.

Root cause

• Every PR push runs every supported Gradle line.
• The middle lines, 7.6.6/JDK 17 and 8.14.3/JDK 21, run the same suites as the boundary lines: 6.9.4 is the oldest supported line and 9.8.0 the newest, and 9.8.0 shares 8.14.3's JDK 21.
• Both middle lines run again in:
  • every merge_group run (the queue gates on ci-ok)
  • every push to main
  • nightly in gradle-compatibility.yml (17 4 * * *)

Proposed fix

Run the boundary Gradle lines on PRs and all four lines in the merge queue. This follows the existing e2e-macos pattern.

1. In .github/workflows/ci.yml, move the 8 e2e rows for gradle: '7.6.6' and gradle: '8.14.3' (ci.yml:1246-1253) into a new job e2e-gradle-mid:
   • if: github.event_name != 'pull_request'
   • needs: [e2e-build]
   • env: *e2e-env, steps: *e2e-steps (the anchors already exist at ci.yml:1280/1285)
   • the same strategy.fail-fast: false
2. Add e2e-gradle-mid to ci-ok.needs. ci-ok already accepts skipped (ci.yml:2559-2560). The required check names (ci-ok, clippy) are unchanged.
3. Update scripts/tests/test_ci_gradle_prefixes.py, which reads rows from job_rows(..., "e2e") at lines 146/199/236, so that it reads the union of the e2e and e2e-gradle-mid rows. It must keep asserting that every Gradle test runs in exactly one leg per line.
4. Optional, as a follow-up: keep e2e-gradle-mid on PRs that touch Gradle code (crates/**/gradle*, scripts/install-gradle.sh, the Gradle e2e tests) through the paths filter outputs that PR Run compatibility matrices on PRs only for their own files #1206 adds.

Expected saving

• Linux: ~75 job-min per non-draft PR run × ~130 runs/day ≈ ~9,800 Linux job-min/day. Also ~1,000 fewer Linux job starts/day, which eases the Linux queue: its p90 was 3.9 min overall and 5.3 min in the 08h burst.
• macOS / Windows: 0.
• Critical path: none on merge_group. On PRs, the 8.14.3 hosted_[b-p] leg (12.2 min p50) is one of three paths that finish within ~2 min of each other before ci-ok, together with the Windows test shards (CI perf: test (windows/macos) — redundant cargo build step + full debuginfo linking ~240 test binaries (~930 job-min/day, ~300 macOS) #1173) and coverage-docker (sbt) → coverage-merge (CI perf: coverage-docker (sbt) — 5.6-min image rebuild warms 3 unused toolchains; merge-queue critical path (~2.5 min/merge-group run) #1225). Removing it alone does not move PR latency.

Coverage and risk

• Every test still runs on every merge_group run and every push, on all four lines, and nightly in gradle-compatibility.yml.
• Risk: a regression that only shows on 7.6.6 or 8.14.3 would surface in the merge queue rather than on the PR, and would evict one wave (about 287–474 job-min per affected run, from the 08:17 dashboard).
• In the 24h sample, none of the 425 Gradle e2e jobs failed. That suggests such evictions would be rare, but this is a coverage trade-off: a maintainer should sign off on it.

Effort

S–M. One new job reusing the existing anchors, plus a test-script update.

ROI

Saving ~9.8 (thousand weighted job-min/day, Linux ×1) × confidence 0.6 ÷ effort 1.5 ≈ 3.9.

Generated by Claude Code