Skip to content

fix(check): let the queue's base binary skip the stub check - #658

Merged
kixelated merged 3 commits into
mainfrom
quest-check/no-stubs-flag
Nov 12, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest-check/no-stubs-flag

Conversation

@kixelated

@kixelated kixelated commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

quest check gains --no-stubs, which skips the comparison of the repository's skill stubs with the skills compiled into the binary. The stub check stays on by default. .github/workflows/queue-build.yml passes it in the base-binary step ("Check that the PR leaves its quest accurate"), with a comment saying why.

Since #655, main's binary rejected any PR that adds, removes or changes a skill's stub, because it compares stubs with main's skills. #649 failed the queue with "quest-unblock/SKILL.md is missing" and "quest-complete/SKILL.md is outdated". The PR's own quest check in nix develop --command just check, later in the same required job, still checks the stubs against the PR's skills.

Decisions

  • Maintainer (2026-11-12): add a quest check flag that skips the stub check (--no-stubs) and pass it in queue-build.yml's base-binary step; the stubs are checked by the PR's own binary in just check later in the same required job. The check stays on by default everywhere else.

  • Maintainer (2026-11-12): also pass --no-stubs in /quest-merge's preflight (assets/skills/merge.md), which runs quest check --base with a quest the PR cannot modify, so it rejected stub-changing PRs the same way. The maintainer asked for the skill edit.

  • Other callers of quest check with a binary that is not the PR's own, searched in the assets, src/cli/land*, and workflows: none has the same flaw. quest land's screening calls the check function in process with no stub comparison, quest land-bump runs quest init with the same binary before quest check --fix, and README.md's examples use the repository's own pin. The README's sentence on --no-stubs now names the quest-merge preflight beside the merge queue.

Caused by

  • moq-dev/quest#655 - added the stub check to quest check, which the queue runs with the base's binary against a PR whose stubs follow the PR's skills.

Public API

New flag quest check --no-stubs; default behavior and exit codes are unchanged. No wire impact.

Landing

queue.yml calls main's copy of queue-build.yml, so the workflow change takes effect only after this PR lands, and while this PR is in the queue the old step runs. This PR changes no stubs, so the old step passes it.

Validation

  • just check and just test pass.
  • quest check --help shows the flag; README updated to match. /quest-merge's preflight command in assets/skills/merge.md now passes --no-stubs; just check and just test pass on the head with it.
  • New assertion in test/init.test.ts runs the built CLI on a repository with no stubs: check exits 1 with "is missing", check --no-stubs exits 0 with no output.

Models

Size: XS
Implementer: Claude Code, claude-sonnet-5-5, high effort
Implementer: Claude Code, claude-sonnet-5-5, high effort (iterate, XS)

(Written by Claude Sonnet 5.5 with high)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Head: b17430d

  • Base: origin/main at 2729857

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1d0a4-e7c3-7280-bf35-b83f29bee3ce, round 1

  • Score: 9/10, The change cleanly skips base-binary stub comparisons while retaining head stub validation in the merge queue; code inspection found no actionable defects, and builds and tests were not run as instructed.

  • Findings: none

  • Reviewer: opencode, opencode-go/deepseek-v4.1-flash, high effort

  • Session: ses_e2f5b1193ffeEaSXnuIyoDIZk7, round 2

  • Score: 10/10, All three earlier findings are refuted by the queue's own stub check at queue-build.yml:57, the impossibility of a base binary checking a PR's stubs, the README's distinction between base-built and repository-installed binaries, and the maintainer's recorded decision to accept the preflight split, with no changes at this head introducing new issues.

  • Findings: none

  • The last round's findings, at b17430d:

    • Refuted: medium, assets/skills/merge.md:35: Merge preflight skips the only pre-queue stub check, and its stated justification is false here; The maintainer recorded on 2026-11-12 the decision to skip stubs in the preflight, and the queue build at .github/workflows/queue-build.yml:57 runs the PR's own `just check` stub check before landing, with `--direct` remaining the explicit-request path.
    • Refuted: low, .github/workflows/queue-build.yml:53: The queue's trusted stub check is delegated to PR-controlled code; A base binary compares the head's stubs with the base's skills, so only the PR's own binary can check them, and the maintainer's recorded decision chose to put that check in the queue's `just check` at .github/workflows/queue-build.yml:57.
    • Refuted: low, README.md:277: The README's PR job still runs the command the new paragraph says must change; README.md:283-288 limits `--no-stubs` to binaries built from the base, while the example at README.md:277 uses the repository's installed `quest`, whose skills match its stubs, so the example remains correct.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 12, 2026 16:27
@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Explain Change →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 28 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1e84489b-b5d1-4fd3-a42a-ea5f36f71b14
Commits

Reviewing files that changed from the base of the PR and between b17430d and 53c8058.

Files selected for processing: 5

  • .github/workflows/queue-build.yml
  • README.md
  • assets/skills/merge.md
  • src/cli/main.ts
  • test/init.test.ts

No actionable comments were generated in the recent review.

Walkthrough

quest check now compares repository skill stubs with the skills in its binary unless --no-stubs is set. Stub findings are collected only when the resolved root is the quest root and contribute to the command’s failure status. The queue workflow uses --no-stubs with its base binary, then uses the PR binary for just check. The README documents these checks, and a test covers the disabled-stub option.

Merge Risk: Minimal · previous review b1743

This adds an opt-out --no-stubs flag to quest check, and stub checking stays on by default. The queue's base-binary step and the merge preflight use the flag, while the PR's own checks still verify its stubs. No actionable merge-blocking risk remains.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: a fix that lets the queue's base binary skip the stub check. It matches the raw summary, which says the queue check now runs the base binary with check --no-stubs, a…
Description check Passed The description states that quest check gains --no-stubs, explains why the queue's base-binary step passes it, and lists the related README, merge skill, and test changes. It matches the changeset…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: a fix that lets the queue's base binary skip the stub check. It matches the raw summary, which says the queue check now runs the base binary with check --no-stubs, and the description, which adds the --no-stubs flag.

Full details: Description check

Explanation

The description states that quest check gains --no-stubs, explains why the queue's base-binary step passes it, and lists the related README, merge skill, and test changes. It matches the changeset in every file summary.

Finishing Touches

Simplify code

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 10e3ca92-80e7-48fa-b6e7-3e0fc302aa70
Commits

Reviewing files that changed from the base of the PR and between 9b4d96e and b17430d.

Files selected for processing: 2

  • README.md
  • assets/skills/merge.md
Files skipped from review as they are similar to previous changes: 1

  • README.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

kixelated and others added 2 commits November 12, 2026 08:43
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@kixelated
kixelated added this pull request to the merge queue Nov 12, 2026
Merged via the queue into main with commit e4b35f7 Nov 12, 2026
15 checks passed
@kixelated
kixelated deleted the quest-check/no-stubs-flag branch November 12, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant