Repository navigation
Run autoharness jobs only in the merge queue - #738
Conversation
Running autoharness only in the merge queue means a contributor's first autoharness result arrives after the PR is queued, and a failure ejects it hours later. workflow_dispatch does not help contributors: it needs write access and runs on a branch of this repo, not on a fork's PR. Add the run-autoharness label as a third way to run the autoharness and metrics jobs: while it is on a PR they run on every push. Adding any label now triggers the workflow, so the partitions skip label events (they already ran for that commit) and the autoharness jobs ignore labels other than run-autoharness.
feliperodri
added
the
run-autoharness
feliperodri
left a comment
feliperodri
left a comment
There was a problem hiding this comment.
Approving. I pushed two commits on top so contributors can ask for an autoharness run before the merge queue: adding the run-autoharness label runs autoharness and the metrics jobs on the PR, on every push while the label stays. workflow_dispatch doesn't cover this, because it needs write access and runs on a branch of this repo, not on a fork's PR.
Adding any label now triggers the workflow, so the partitions skip label events (they already ran for that commit), and the autoharness jobs ignore labels other than run-autoharness.
Tested on this PR:
- Push without the label: both autoharness checks pass in seconds with their steps skipped; List, Analyzer, Metrics and Build JSON are skipped; the partitions run (run).
run-autoharnessadded: autoharness runsRun Kani Verification, and Metrics, List and Analyzer run; the partitions skip (run).- The merge-queue path gets tested when this PR goes through the queue; I'll check that autoharness actually runs there.
Thanks @Tianshu-Huang!
Adding a label started a new run of kani.yml on the same commit, and GitHub reads required checks from the latest run. That run skipped the partitions (and, for other labels, the autoharness jobs), so the required checks went back to "expected" and the PR could not enter the merge queue. kani.yml no longer reacts to labels. kani-autoharness-request.yml runs on labeled/synchronize/reopened and, when the PR has run-autoharness, calls kani.yml with autoharness: true, which runs only the autoharness and metrics jobs. Its checks are prefixed with the caller job name, so they never collide with the required ones.
|
Correction to my review: the first version of the label trigger blocked merging. Adding a label started a new run of Fixed in a6df330: |
Problem
Five CI jobs (autoharness, metrics, list, analyzer, build JSON) run on every PR push and take about five runner-hours. Every PR waits on them. #730 asks to run them only before merging.
Fix
Add
if: merge_group || workflow_dispatchto those five jobs.Autoharness is a required check, so it cannot be skipped as a whole job: a skipped job loses the
(ubuntu-latest)part of its name and the required check never turns green. So for that job theif:is on each step. The job still runs, does nothing, and passes in seconds.Testing
On my fork, a manual run passed all 16 jobs (run), and a PR skipped the five jobs with the two autoharness required checks green (PR). Did not test
merge_group.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.