Repository navigation
fix: tell shepherd planners about CI that misfires on stacked PRs - #507
Merged
Merged
Conversation
scode
force-pushed
the
shepherd-worker-decisions
branch
from
November 7, 2026 14:07
dca3924 to
875ce06
Compare
scode-shepherd delivers a goal as a stack of draft PRs, each based on the previous one's branch. A cheap worker model writes them one at a time, and a runner script checks each one by running a list of commands from the plan before a reviewer looks at it. Some repositories carry CI written for PRs against the default branch, and it misfires on such a stack. On a real run the repository had a check whose only job is to fail any PR whose base is not `main`, kept on purpose as a guard against merging a stacked PR through GitHub's web page. The goal file, written by scode-build-goal, demanded that every applicable GitHub check pass. Nothing said the guard was expected to fail, so the worker finished the second PR, saw the guard fail, and reported that the plan had to change. That stopped the run and woke the supervisor model to work out that the failure was harmless. Two skills plan such a run, and both now look at the repository's CI first: scode-build-goal when it writes the goal file in shepherd mode, and scode-shepherd's supervisor when it turns a goal into the plan the worker follows. A base guard (a job that does nothing but reject PRs on other bases) is named, by workflow and job, as expected to fail on a PR whose base it does not allow, and for that reason only; if it fails any other way, or any other check fails, that still counts. The same look also catches the opposite misfire: a workflow filtered to the default branch never runs on a PR based on another branch, so whatever it tests would quietly go untested. Its commands go into the runner's list instead, and when they cannot run there (they need an OS the runner's machine lacks, say), the planner asks the user whether the PRs may go without them, unless the goal already says so. scode-shepherd also reads goal files that are already written: a general demand that applicable GitHub checks pass no longer covers a base guard on a PR it rejects, since the stack the goal asks for makes that failure certain. It also no longer covers a skipped workflow whose commands the runner runs. A check the user demanded by name still binds, and if the stack keeps it from passing, the user decides, since running the same commands locally is not that hosted check passing. The rule is deliberately narrow, so that it cannot be used to excuse a real failure.
scode
force-pushed
the
shepherd-stacked-ci
branch
from
November 7, 2026 14:08
11fae8d to
2bace9e
Compare
scode
marked this pull request as ready for review
November 7, 2026 14:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
scode-shepherd delivers a goal as a stack of draft PRs, each based on the previous one's branch. A cheap worker model
writes them one at a time, and a runner script checks each one by running a list of commands from the plan before a
reviewer looks at it. Some repositories carry CI written for PRs against the default branch, and it misfires on such
a stack. On a real run the repository had a check whose only job is to fail any PR whose base is not
main, kept onpurpose as a guard against merging a stacked PR through GitHub's web page. The goal file, written by scode-build-goal,
demanded that every applicable GitHub check pass. Nothing said the guard was expected to fail, so the worker finished
the second PR, saw the guard fail, and reported that the plan had to change. That stopped the run and woke the
supervisor model to work out that the failure was harmless.
Two skills plan such a run, and both now look at the repository's CI first: scode-build-goal when it writes the goal
file in shepherd mode, and scode-shepherd's supervisor when it turns a goal into the plan the worker follows. A base
guard (a job that does nothing but reject PRs on other bases) is named, by workflow and job, as expected to fail on a
PR whose base it does not allow, and for that reason only; if it fails any other way, or any other check fails, that
still counts. The same look also catches the opposite misfire: a workflow filtered to the default branch never runs on
a PR based on another branch, so whatever it tests would quietly go untested. Its commands go into the runner's list
instead, and when they cannot run there (they need an OS the runner's machine lacks, say), the planner asks the user whether the PRs may go
without them, unless the goal already says so.
scode-shepherd also reads goal files that are already written: a general demand that applicable GitHub checks pass no
longer covers a base guard on a PR it rejects, since the stack the goal asks for makes that failure certain. It also
no longer covers a skipped workflow whose commands the runner runs. A check the user demanded by name still binds, and if the stack
keeps it from passing, the user decides, since running the same commands locally is not that hosted check passing.
The rule is deliberately narrow, so that it cannot be used to excuse a real failure.