Skip to content

fix: tell shepherd planners about CI that misfires on stacked PRs - #507

Merged
scode merged 1 commit into
mainfrom
shepherd-stacked-ci
Nov 7, 2026
Merged

scode merged 1 commit into
mainfrom
shepherd-stacked-ci

Conversation

@scode

@scode scode commented Nov 7, 2026

Copy link
Copy Markdown
Owner

scode-shepherd delivers a goal as a stack of draft PRs, each based on the previous one's branch. A cheap worker model
writes them one at a time, and a runner script checks each one by running a list of commands from the plan before a
reviewer looks at it. Some repositories carry CI written for PRs against the default branch, and it misfires on such
a stack. On a real run the repository had a check whose only job is to fail any PR whose base is not main, kept on
purpose as a guard against merging a stacked PR through GitHub's web page. The goal file, written by scode-build-goal,
demanded that every applicable GitHub check pass. Nothing said the guard was expected to fail, so the worker finished
the second PR, saw the guard fail, and reported that the plan had to change. That stopped the run and woke the
supervisor model to work out that the failure was harmless.

Two skills plan such a run, and both now look at the repository's CI first: scode-build-goal when it writes the goal
file in shepherd mode, and scode-shepherd's supervisor when it turns a goal into the plan the worker follows. A base
guard (a job that does nothing but reject PRs on other bases) is named, by workflow and job, as expected to fail on a
PR whose base it does not allow, and for that reason only; if it fails any other way, or any other check fails, that
still counts. The same look also catches the opposite misfire: a workflow filtered to the default branch never runs on
a PR based on another branch, so whatever it tests would quietly go untested. Its commands go into the runner's list
instead, and when they cannot run there (they need an OS the runner's machine lacks, say), the planner asks the user whether the PRs may go
without them, unless the goal already says so.

scode-shepherd also reads goal files that are already written: a general demand that applicable GitHub checks pass no
longer covers a base guard on a PR it rejects, since the stack the goal asks for makes that failure certain. It also
no longer covers a skipped workflow whose commands the runner runs. A check the user demanded by name still binds, and if the stack
keeps it from passing, the user decides, since running the same commands locally is not that hosted check passing.
The rule is deliberately narrow, so that it cannot be used to excuse a real failure.

@scode
scode force-pushed the shepherd-worker-decisions branch from dca3924 to 875ce06 Compare November 7, 2026 14:07
Base automatically changed from shepherd-worker-decisions to main November 7, 2026 14:08
scode-shepherd delivers a goal as a stack of draft PRs, each based on the previous one's branch. A cheap worker model
writes them one at a time, and a runner script checks each one by running a list of commands from the plan before a
reviewer looks at it. Some repositories carry CI written for PRs against the default branch, and it misfires on such
a stack. On a real run the repository had a check whose only job is to fail any PR whose base is not `main`, kept on
purpose as a guard against merging a stacked PR through GitHub's web page. The goal file, written by scode-build-goal,
demanded that every applicable GitHub check pass. Nothing said the guard was expected to fail, so the worker finished
the second PR, saw the guard fail, and reported that the plan had to change. That stopped the run and woke the
supervisor model to work out that the failure was harmless.

Two skills plan such a run, and both now look at the repository's CI first: scode-build-goal when it writes the goal
file in shepherd mode, and scode-shepherd's supervisor when it turns a goal into the plan the worker follows. A base
guard (a job that does nothing but reject PRs on other bases) is named, by workflow and job, as expected to fail on a
PR whose base it does not allow, and for that reason only; if it fails any other way, or any other check fails, that
still counts. The same look also catches the opposite misfire: a workflow filtered to the default branch never runs on
a PR based on another branch, so whatever it tests would quietly go untested. Its commands go into the runner's list
instead, and when they cannot run there (they need an OS the runner's machine lacks, say), the planner asks the user whether the PRs may go
without them, unless the goal already says so.

scode-shepherd also reads goal files that are already written: a general demand that applicable GitHub checks pass no
longer covers a base guard on a PR it rejects, since the stack the goal asks for makes that failure certain. It also
no longer covers a skipped workflow whose commands the runner runs. A check the user demanded by name still binds, and if the stack
keeps it from passing, the user decides, since running the same commands locally is not that hosted check passing.
The rule is deliberately narrow, so that it cannot be used to excuse a real failure.
@scode
scode force-pushed the shepherd-stacked-ci branch from 11fae8d to 2bace9e Compare November 7, 2026 14:08
@scode
scode marked this pull request as ready for review November 7, 2026 14:08
@scode
scode merged commit 1692d63 into main Nov 7, 2026
5 checks passed
@scode
scode deleted the shepherd-stacked-ci branch November 7, 2026 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant