Skip to content

docs(skills): trim the skills and fix what transcripts show agents misread - #657

Merged
kixelated merged 2 commits into
mainfrom
skills-trim
Nov 12, 2026
Merged

kixelated merged 2 commits into
mainfrom
skills-trim

Conversation

@kixelated

@kixelated kixelated commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Trims nine of the skills and fixes the sentences agents read two ways, from five days of this host's session transcripts (about 1,580 sessions that loaded a quest skill) and the current official skill guidance.

Changes

Composed size is what quest skill <name> prints: the skill, the skills it builds on, and its guide sections. Claude Code keeps only the first 5,000 tokens (about 20k characters) of an invoked skill after compaction, so a coordinator that compacts was losing the tail of its guide sections.

Skill Own text Composed
run 7,540 to 7,752 28,528 to 21,131
complete unchanged 28,778 to 21,381
spawn unchanged 29,061 to 21,664
review 7,208 to 6,239 20,350 to 18,205
plan 2,825 to 2,084 18,251 to 13,120
audit 1,936 to 1,910 20,177 to 15,020
import 1,265 to 1,168 19,438 to 14,210
recurring unchanged 19,843 to 14,712
unblock unchanged 25,406 to 20,275
iterate 2,822 to 2,824 15,358 to 15,040
merge 5,472 to 4,296 9,232 to 8,056
start 1,543 to 932 13,204 to 12,593
grill 2,049 to 1,729 3,238 to 2,918

Cut, because the CLI or the guide already says it:

  • start: the worktree write check (about 30 of 158 sessions spent a call on it and none failed), "update your checkout" (quest start builds the claim on the fetched upstream, and the git pull && quest start chain it invited was refused by the worktree hook in about 28 sessions), and the restatements of the guide's Execution section.
  • review: the --help pointers (quest review --help is 25k characters, and 36 of 136 sessions ran it through head), and the findings loop that each round's next line already directs.
  • merge: the landing paths (merge queue, quest land, ungated base), which quest merge's refusals spell out.
  • plan: the sizing rule and epic sentence the guide's Format and Creation sections give, and three sentences no session acted on.
  • plan no longer loads the guide's Execution section, nor run Execution and Prompting. Each kept the one thing it used from them: the quest-plan/<slug> branch, the open-PR overlap check, and what a brief holds.

Reworded, because sessions read them two ways:

  • review: a draft its starter delegated for review is reviewed without a hold (14 of 16 sessions guessed this, 2 stopped). gh pr ready is named for it (120 of 136 sessions had to work that out). --branch is passed whenever the checked-out branch is not the PR's head branch (15 sessions lost their record to record skipped). The ## Models lines are shown (12 sessions in two days searched other PRs for the format). Either reviewer's outage stops the review (4 sessions on 2026-11-12 split four ways when only the second reviewer failed).
  • iterate: the ending is /quest-review once nothing is open, draft or ready, and a plain quest review round only when stopping early on a ready PR. Of 185 sessions on a ready PR, 85 took one reading and 68 the other.
  • run: questions go one PR to a prompt, as moq-dev/quest#608 intended (half of the prompts carried 2 to 4 PRs). "Within the harness's agent limit" becomes "as many at once as the harness allows" (sessions capped themselves at 7 to 9). checks_pending, a reason no code prints any more, becomes checks_timed_out, which sent a green PR back to the user; it is retried once.
  • plan: the first round asks only the goal. It was confirmed alone in 2 of 33 sessions, and 14 of 51 goal answers changed it, so the questions bundled with it were asked again.
  • grill: a round takes as many questions as the tool allows (it caps a prompt at four), and facts are looked up before asking, in a sub-agent only for a broad search (3 of 34 sessions used one).
  • import: one comment per issue. Two overlapping sentences produced two comments in 3 of 8 sessions.
  • merge: the landable list now names the draft and failed-check conditions that sat in prose around it.
  • start, when it cannot prompt: leave the draft with ## Waiting. 11 sessions marked a PR ready with decisions open.

Left alone: afk, complete, spawn, delete, export, recurring (no evidence against their current text), and unblock, which moq-dev/quest#649 folds into complete.

Public API and wire impact

Shipped skill text changes; no CLI, format, or wire change. Descriptions are unchanged, so no stub changes.

Validation

  • just check and just test pass (1,248 tests).
  • Real surface: quest skill <name> for all 16 skills, with the sizes above. A scan of each composed output confirms every guide section a skill names by "the guide's X section" is still loaded with it.
  • No session has run the new text yet. The sentences that changed behavior are listed above so a reviewer can weigh each against its evidence.

Choices made without asking

Follow-ups

Not planned here, for the maintainer to weigh:

  • The guide's Coordination section tells coordinators to put "cannot prompt" and "wait in the foreground" in each brief, which 93% of 135 briefs restate and the shared rules already cover. quest agent ran in 2 of 107 coordinator sessions, and its paragraph is 1.1k characters in every coordinator's context.
  • /quest-spawn with no filter starts every ready quest in any milestone. One session walked into m3 with about 290 ready and was stopped by the user.
  • Milestone questions were answered with the recommendation 87% of the time, often citing a rule a past session recorded in the tree.

Caused by

No single PR: the wording accumulated across the skills' history.

Models

Size: L
Implementer: Claude Code, claude-fable-5-2, unknown effort

🤖 Generated with Claude Code

(Written by Claude Fable 5)

Review

Rewritten by quest review after each round.

  • Head: b95eadc

  • Base: origin/main at 907aeac

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort

  • Session: 01a1cf1f-147b-7361-8bdd-91c26d16b40a, round 2

  • Score: 10/10, Static review of the incremental documentation changes and supporting CLI code found no actionable regressions.

  • Findings: none

  • Reviewer: opencode, opencode-go/deepseek-v4.1-flash, high effort

  • Session: ses_e30e0e636ffeKzua9yGMuedg1N, round 2

  • Score: 10/10, Every earlier finding is fixed at the head, including the follow-up quest's trusted-binary rule and the bounded timeout retry, and no new problem is introduced.

  • Findings: none

  • The last round's findings, at 1bfc937:

    • Fixed: medium, quest/m0/token-waste/merge-check.md:28: The follow-up quest's plan drops the trusted-binary requirement for the head check; The quest's Plan now says the check is only as trusted as the `quest` that runs it and keeps the rule that `quest merge` must run with a `quest` the PR cannot modify, with a bullet weighing a refusal from the head's checkout, so the earlier bypass is closed.
    • Fixed: medium, assets/skills/run.md:30: Falling to the next model on any error contradicts the guide and README; run.md:30 restores the original fallback to a missing or logged-out CLI or a usage limit with nothing written, which agrees with the guide's Coordination and README.md:1277-1281.
    • Fixed: medium, assets/skills/merge.md:24: Under /quest-run, follow-up questions are asked twice; merge.md:24 now asks only about follow-ups the pick round has not asked and runs `/quest-iterate` with that round's answers, so `/quest-run` no longer re-asks questions it already put to the user.
    • Fixed: medium, assets/skills/run.md:60: `checks_timed_out` is retried without a bound; run.md:60 retries a `checks_timed_out` head once and leaves a second timeout of the same head to the user, bounding the re-queue loop.
    • Fixed: low, assets/skills/import.md:14: "Comment once" conflicts with the second comment at merge start, and the PR link is gone; import.md now gives each issue exactly one comment, posts a tracked issue's comment at merge start and a won't-fix one at close, and links both the quest and the PR.
    • Fixed: low, assets/skills/review.md:58: A reviewer that took no hold is still told to release a token; review.md:58 now conditions the release on having taken a hold, so the no-hold path is no longer told to run `quest release` without a token.
    • Fixed: low, assets/skills/run.md:3: run drops the Prompting guide section although it writes every delegation brief; run.md:13 now carries the Prompting rules that apply to a brief, setting settled decisions apart from recommendations, naming the task's reason and context, and forbidding prescribed steps and restated rules, leaving only instruction-file rules in the section.
    • Fixed: low, assets/skills/iterate.md:28: `--branch` is only paired with a detached checkout in iterate; iterate.md:28 now passes `--branch <head-branch>` whenever the checked-out branch is not the PR's head branch, matching review.md:15.

kixelated and others added 2 commits November 12, 2026 01:37
…sread

Cut sentences the CLI's own output or the guide already gives, drop guide
sections a skill never uses, and reword the sentences that five days of
session transcripts show agents reading two ways.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 12, 2026 10:07
@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 26 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d39e2894-e991-45cb-822d-cb57f2dbb9e5
Commits

Reviewing files that changed from the base of the PR and between e6fe441 and b95eadc.

Files selected for processing: 10

  • assets/skills/audit.md
  • assets/skills/grill.md
  • assets/skills/import.md
  • assets/skills/iterate.md
  • assets/skills/merge.md
  • assets/skills/plan.md
  • assets/skills/review.md
  • assets/skills/run.md
  • assets/skills/start.md
  • quest/m0/token-waste/merge-check.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated
kixelated added this pull request to the merge queue Nov 12, 2026
Merged via the queue into main with commit 1aa47c4 Nov 12, 2026
11 checks passed
@kixelated
kixelated deleted the skills-trim branch November 12, 2026 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant