[kixelated]

Finishes the routing charts quest. quest usage --json now carries every measure the cost-against-quality charts plot. The model check (quest/m1/model-check.md) holds the procedure and the published benchmark table.

What the charts quest delivers

The quest's non-goal rules out chart code in quest, so I read the deliverable as three things:

1. The data. Each axis is a field of quest usage --json, computed once in the CLI, so the model check and the readout don't each re-derive it with ad hoc joins. These were missing:

   • Writer cost per merged PR. Each accuracy row gains spent: its writing sessions' spend on its merged PRs, with spent.pulls counting the PRs where spend was found. Cost per merged PR divides by those, so PRs whose sessions no host reported don't dilute it. A PR's spend on a model goes to its Implementer: line of that model, of the same effort where both name one. Spend on a model no line names, such as a sub-agent on a smaller model, goes to the line that took the most.
   • Local reviewer cells. A new reviewers rollup groups by PR size, harness, model, effort, and shadow mark. It carries PRs reviewed, tallies (each branch counted once, as accuracy does), first rounds, and reviewing cost per PR.
   • First-round findings. The writer A/B and the readout compare these on their own. quest review's tally now keeps the first round's findings, and the record writes them as - Rounds: 3; first round 2 (medium 2); findings 4 …. Records from before this change still parse, with first: null, and saved review state from before keeps working. Rollups count first rounds only over PRs whose every counted tally records one (firstRounds.pulls).
   • Sub-agents. token-max exports each session's parent, which quest usage ignored. A sub-agent in a detached checkout went unattributed, and a Claude /code-review sub-agent counted as writing. Now a session that names no PR's branch takes its parent's PR, and takes its parent's role when both are on the same PR. On real data this moved $18 off "unattributed", and reviewing spend now reconciles.

2. The procedure, in the model check's Plan:

   • which field feeds each axis;
   • thin cells stay on the chart, hollow and labelled with their PR count, never dropped;
   • unknown-effort cells stay separate points;
   • each chart names its date range, its command, and its PR counts;
   • the PR body carries every plotted number as a table.

   The epic's readout points to it.

3. The benchmark table in the model check's Plan: the mypriors rows moved out of the prose, plus Anthropic's cost guide and Artificial Analysis, re-read today. Anthropic's guide now lists Opus 5.5 high (95.3%, $0.29 per task) and xhigh (96.7%, $0.74).

The charts themselves are a per-run artifact, so the first ones are below rather than in the tree.

First charts (2026-11-06 to 2026-11-13, this host only)

Command: quest usage --since 2026-11-06 --json, with token-max built into this worktree. No other host has token-max installed, so --ssh read nothing.

Every cell is too thin to act on:

• At most 5 priced merged PRs per writer cell, and most cells have none.
• No tally records a first round yet, so findings below are summed across all rounds.
• The opencode DeepSeek reviewer has no cost, since token-max does not read opencode sessions.
• Hand-backs count only drafts still open, so merged PRs read 0 (see the hand-backs follow-up).

Mermaid quadrant charts normalize both axes to 0-1, so each title gives the scale.

quadrantChart
  title Writers: cost per merged PR (x, 0 to $16) against rounds per reviewed PR (y, 0 to 8)
  x-axis Cheaper --> Costlier
  y-axis Fewer rounds --> More rounds
  XS Opus unknown n2: [0.19, 0.38]
  S Opus low n1: [0.09, 0.47]
  S Opus unknown n4: [0.28, 0.33]
  M Fable unknown n1: [0.94, 0.13]
  M Opus medium n5: [0.29, 0.54]
  L Fable medium n1: [0.01, 0.25]
  L Opus unknown n1: [0.03, 0.88]

Loading

quadrantChart
  title Local reviewers: cost per PR reviewed (x, 0 to $1) against findings per reviewed PR (y, 0 to 5)
  x-axis Cheaper --> Costlier
  y-axis Fewer findings --> More findings
  XS Sol medium n5: [0.06, 0.03]
  S Sol low shadow n2: [0.11, 0.0]
  S Sol medium n9: [0.15, 0.09]
  M Sol medium shadow n8: [0.18, 0.09]
  M Sol high n8: [0.34, 0.30]
  L Sol high shadow n1: [0.66, 1.0]
  L Sol xhigh n2: [0.91, 0.9]

Loading

Writers: every cell with a merged PR

Size	Writer	Effort	Merged	Priced	$ per merged PR	Reviewed	Rounds per reviewed PR	Findings per reviewed PR	Hand-backs	Escapes per merged PR
XS	claude-fable-5-2	high	1	0	-	0	-	-	0	0
XS	claude-fable-5-2	medium	1	0	-	0	-	-	0	0
XS	claude-haiku-5-5	high	7	0	-	0	-	-	0	0
XS	claude-haiku-5-5	unknown	1	0	-	0	-	-	0	0
XS	claude-opus-5-5	high	6	0	-	0	-	-	0	0
XS	claude-opus-5-5	low	6	0	-	3	6	9.33	0	0
XS	claude-opus-5-5	unknown	16	2	3.07	5	3	3.8	0	0
XS	claude-opus-5-5	xhigh	2	0	-	0	-	-	0	0
XS	claude-sonnet-5-5	high	3	0	-	1	3	0	0	0
XS	claude-sonnet-5-5	unknown	2	0	-	2	4	4.5	0	0
XS	gpt-6-astra	xhigh	1	0	-	0	-	-	0	0
XS	gpt-6.1-sol	low	1	0	-	0	-	-	0	0
XS	gpt-6.1-sol	xhigh	1	0	-	0	-	-	0	0
S	claude-opus-5-5	high	11	0	-	0	-	-	0	0
S	claude-opus-5-5	low	37	1	1.37	12	3.75	5.17	0	0
S	claude-opus-5-5	medium	6	0	-	1	2	5	0	0
S	claude-opus-5-5	unknown	24	4	4.52	11	2.64	6.09	1	0.04
S	claude-opus-5-5	xhigh	1	0	-	0	-	-	0	0
S	gpt-6.1-sol	high	1	0	-	0	-	-	0	0
M	claude-fable-5-2	high	3	0	-	0	-	-	0	0
M	claude-fable-5-2	medium	1	0	-	0	-	-	0	0
M	claude-fable-5-2	unknown	1	1	15.06	1	1	0	0	0
M	claude-opus-5-5	high	7	0	-	1	4	15	0	0
M	claude-opus-5-5	low	1	0	-	0	-	-	0	0
M	claude-opus-5-5	medium	24	5	4.64	6	4.33	5.83	0	0.13
M	claude-opus-5-5	unknown	12	0	-	9	4.67	9	0	0
M	claude-opus-5-5	xhigh	1	0	-	0	-	-	0	0
L	claude-fable-5-2	medium	1	1	0.21	1	2	2	0	0
L	claude-fable-5-2	unknown	1	0	-	0	-	-	0	0
L	claude-opus-5-5	high	7	0	-	0	-	-	0	0.14
L	claude-opus-5-5	low	1	0	-	0	-	-	0	0
L	claude-opus-5-5	unknown	2	1	0.5	1	7	7	0	0
L	claude-opus-5-5	xhigh	2	0	-	0	-	-	0	0
-	claude-opus-5-5	low	2	2	1.57	2	3	1.5	0	0
-	claude-opus-5-5	unknown	1	0	-	0	-	-	0	0

Local reviewers

Size	Reviewer	Model	Effort	Reviewed	Priced	$ per PR reviewed	Findings per reviewed PR	Share fixed
XS	Codex	gpt-6.1-sol	low, shadow	4	0	-	0.25	100%
XS	Codex	gpt-6.1-sol	medium	7	5	0.06	0.14	100%
XS	opencode	opencode-go/deepseek-v4.1-flash	high	4	0	-	4.75	79%
S	Codex	gpt-6.1-sol	high, shadow	1	0	-	0	-
S	Codex	gpt-6.1-sol	low, shadow	7	2	0.11	0	-
S	Codex	gpt-6.1-sol	medium	13	9	0.15	0.46	80%
S	Codex	gpt-6.1-sol	xhigh	1	0	-	6	83%
S	opencode	opencode-go/deepseek-v4.1-flash	high	8	0	-	10.13	89%
S	opencode	opencode/deepseek-v4.1-flash-free	high	1	0	-	0	-
M	Codex	gpt-6.1-sol	high	12	8	0.34	1.5	89%
M	Codex	gpt-6.1-sol	low, shadow	2	0	-	0.5	100%
M	Codex	gpt-6.1-sol	medium	2	0	-	1.5	100%
M	Codex	gpt-6.1-sol	medium, shadow	9	8	0.18	0.44	50%
M	opencode	opencode-go/deepseek-v4.1-flash	high	7	0	-	11	87%
L	Codex	gpt-6.1-sol	high, shadow	1	1	0.66	5	100%
L	Codex	gpt-6.1-sol	xhigh	2	2	0.91	4.5	89%
-	Codex	gpt-6.1-sol	high	1	1	0.09	0	-
-	Codex	gpt-6.1-sol	low, shadow	11	6	0.1	0.09	100%
-	Codex	gpt-6.1-sol	medium	12	6	0.1	0.67	75%
-	opencode	opencode-go/deepseek-v4.1-flash	high	5	0	-	3.6	83%

Online reviewers, over the 193 merged PRs:

Online reviewer	Coverage of merged PRs	Findings per reviewed PR	Accepted
coderabbitai[bot]	36/193 (19%)	2.22	50%
pullfrog[bot]	56/193 (29%)	0.43	88%
review (OpenAI)	100/193 (52%)	0.25	48%

The benchmark charts plot the table in quest/m1/model-check.md, one chart per suite. On SWE-bench Pro, Opus low and Sonnet high sit level at about $0.10 per task, and Opus medium, high, and xhigh trace the frontier above them. On Terminal-Bench 4.0, Sonnet high sits alone at the top, and Haiku xhigh is the cheap end. No PR cell is priced well enough yet to compare frontiers.

Validation

• just check and just test pass (1260 tests) after merging main at 4ab5dae (feat(review): give every PR a risk tier from its diff #716's risk tiers, feat(land)!: pin bumps follow the latest release #717).
• Ran the built CLI on main's real PRs and this host's sessions (quest usage --since 2026-11-06 --json). Checked the result a second way, straight from pulls[]:
  • Writing spend on merged PRs with implementers: $67.74, equal to the sum of accuracy[].spent.
  • Reviewing spend on PRs with a review record: $9.81, equal to the sum of reviewers[].spent.
• Hand-checked the fixture's routed Sol reviewer row across feat(skills): label planned issues when the PR opens #10, feat: read readiness through line branches #5, quest: plan the first public release #6, and quest: plan converging MoQ onto standalone Quest #3: high 4, medium 1, low 3; fixed 4; refuted 2; settled 1; open 1; 9 rounds.
• Tests cover these cases:
  • a detached sub-agent on a smaller model takes its parent's PR, its role, and its main writer's line;
  • a reviewer's sub-agent counts as reviewing;
  • spend of unknown effort and spend whose effort doesn't match a line both land on the line of the same model;
  • first-round parsing, including malformed first-round counts;
  • a first round stays frozen as later rounds regrade or settle its findings.

Public API

Breaking, as the ! says: the quest usage --json shape gains accuracy[].spent (with split), pulls[].sessions[].parent, accuracy[].firstRounds, a top-level reviewers, and pulls[].review[].tally.first. accuracy[].tally no longer carries first. The ## Review record's Rounds: line gains first round N (...), which older records lack. No migration is needed: nothing is in production, and older records and saved state still read.

Choices made without asking

• Charts are not code. The deliverable is the fields and the procedure. The CLI computes each axis as a rollup field, so every run joins the same way, and draws nothing.
• Cost per merged PR divides by priced PRs (spent.pulls), not all merged PRs. Most PRs have no sessions on the hosts read, and counting them would make every cell look cheap.
• A writer cell's cost is its own sessions' spend, matched by model, rather than the PR's whole writing cost. An iterate by another model is its own cell, as accuracy already counts it. Spend on models no line names, such as a sub-agent on a smaller model, goes to the PR's main writer. Spend that fits several lines of one model and effort goes to the first line when an implementing session spent it and to later lines when an iterating one did (review round 1).
• The procedure lives once, in the recurring model check. The epic's readout points to it, since the epic's README is deleted when the epic completes.
• Thin cells: the procedure ties "too thin" to the run's own variance estimate, which the readout already computes, rather than a fixed PR count.
• Benchmark rows with a stated effort but approximate values (Anthropic's Sonnet medium, and Sonnet high's cost per task derived from its cost per solved task) are kept and marked ~. Rows on max and rows without a cost per task are left out.
• Ambiguous spend is split and marked (review round 2). A reviewer gets the sessions its record names and their sub-agents; spend that still fits several seats, such as a routed and a shadow reviewer at one effort with no named session, is split evenly and reported as spent.split in the JSON.
• The risk classifier's spend is not writing (merging feat(review): give every PR a risk tier from its diff #716). Writer cost counts only implementing and iterating sessions, so the new classifying role stays out of it, as reviewing does.
• Parent attribution is in this PR because the cost axis depends on it. It falls back to the session's own branch first, so a coordinator's sub-agent working on another quest keeps that quest's PR.

Decisions

User, 2026-11-14, /quest-run:

• Where do charts go? ✅ Mermaid charts in the PR body (rejected: images in a secret gist; a claude.ai page made by hand). The model check's procedure now says so, with each title giving the scales and each label the PR count.
• Counting hand-backs on merged PRs? ✅ Add a follow-up quest in this PR (rejected: drop).
• token-max on every agent host? ✅ Install it through each host's Nix config, in kixel-ai/dotfiles' quest tree (rejected: wrap quest's Nix package; both). quest usage --ssh runs token-max on the remote host, so it must be on that host's PATH.
• Pricing the DeepSeek/opencode reviewer's sessions? ✅ Plan it, in kixel-ai/token-max's quest tree, since token-max doesn't read those sessions (rejected: drop).
• Design checkpoint at review round 3 (user, 2026-11-14, comment): ✅ Continue: show split in quest usage's text output (rejected: record each Implementer line's session ID in ## Models; leave PRs with two seats of one model and effort off the cost axis; merge as is). Done in round 4: the writer and local-reviewer text rollups append ~$X of it split with other seats.

Caused by

• moq-dev/quest#704 - ignored token-max's parent, so a sub-agent in a detached checkout went unattributed and a review's sub-agents counted as writing

Follow-ups

• Count hand-backs on merged PRs (S, new quest): read each PR's body history (GraphQL userContentEdits) so a merged PR that was once handed back counts. No open decisions; the Plan recommends body history over a ## Models record, since it covers PRs merged before it lands.
• kixel-ai/dotfiles: install token-max on every host that runs agents, through each host's Nix config, so quest usage --ssh can price their sessions. For /quest-plan there.
• kixel-ai/token-max: read opencode sessions, so the DeepSeek reviewer's sessions get a cost. For /quest-plan there.

Models

• Size: M
• Implementer: Claude Code, claude-opus-5-5, medium effort
• Implementer: Claude Code, claude-opus-5-5, medium effort (iterate, M)
• Implementer: Claude Code, claude-opus-5-5, low effort (iterate, S)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with medium)

Review

Rewritten by quest review after each round.

• Head: a44a1e5

• Base: origin/main at a5b1225

• Risk: high, Changes to PR review-record parsing and quest usage output could break metrics consumed by skills, Kixel Code, and scripts; reverting restores the code but leaves already-written PR records behind.

• Classified: high by Codex, gpt-6.1-sol, low effort, session 01a1d7c6-608c-75f0-ac9d-d5c68ae1f3fe

• Reviewer: Codex, gpt-6.1-sol, high effort

• Session: 01a1d78b-33f8-7160-8da3-e61bac5531a2, round 7

• Score: 10/10, The incremental change only moves a documentation comment to the function it describes; no issues were found, and builds and tests were not run as instructed.

• Rounds: 7; first round 1 (medium 1); findings 1 (medium 1): fixed 1, refuted 0, open 0

• Findings: none

• Reviewer: Codex, gpt-6.1-sol, medium effort, shadow

• Session: 01a1d78b-33f0-7753-983c-c0753cf657e5

• Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

[kixelated]

Maintainer decision (2026-11-14, design checkpoint at 5bba01f): ✅ continue: show split in quest usage's text output, then finish the review (rejected: record each Implementer line's session ID and divide writing cost by session; leave ambiguous PRs off the cost axis; merge as is).

(Written by Claude Opus 5.5)

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 193cb57a-ba67-4c89-8acd-59ec0d75f29f

Commits

Reviewing files that changed from the base of the PR and between a5b1225 and a44a1e5.

Files selected for processing: 11

• README.md
• quest/m0/routing-evidence/README.md
• quest/m0/routing-evidence/charts.md
• quest/m0/routing-evidence/handbacks.md
• quest/m1/model-check.md
• src/cli/main.ts
• src/cli/review-record.ts
• src/cli/review.ts
• src/cli/usage.ts
• test/review.test.ts
• test/usage.test.ts

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[pullfrog]

ℹ️ No critical issues — one edge case in the parent fallback, inline.

Reviewed changes

I reviewed the full diff at 04392af: the first-round tally, the parent-based attribution, the share allocator, and the new reviewers rollup. vitest run test/usage.test.ts test/review.test.ts passes locally (98 tests).

• First-round findings — tallied() freezes Tally.first on a seat's first round. The Rounds: line writes it as first round N (...) and parseCounts validates it. Records and saved state from before the change still parse, with first: null.
• Parent attribution — attribute() now places sessions through a memoized place(). A session that names no PR's branch takes its parent's PR, and a sub-agent on the same PR as its parent takes the parent's role. A seen set stops parent chains that loop.
• share() allocator — spend goes to a writer or reviewer seat in this order: the seat a record names, then model and effort, then role to break ties. Spend that still fits several seats is split evenly and reported as split. Spend on a model no seat names goes to the costliest seat.
• New rollup fields — each accuracy row gains spent (merged PRs only) and firstRounds. A new reviewers rollup groups by PR size, harness, model, effort, and shadow. The text rollups show cost per PR and any split spend.
• Docs and quests — this PR updates the README, --help, and the chart procedure and benchmark table in quest/m1/model-check.md. It adds the hand-backs follow-up quest and removes the finished charts.md quest.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using claude-opus-5-5 ｜ 𝕏}

[pullfrog]

ℹ️ The new merge commit adds no issues. The earlier inline finding on the parent-PR fallback is still open; it is now at src/cli/usage.ts:530.

Reviewed changes

This pass covers merge commit efc894f, which brings in main's risk tiers (#716) on top of 04392af. vitest run test/usage.test.ts test/review.test.ts passes (105 tests).

• The classifying role comes first — in place(), a session the record names as the risk classifier is classifying before the reviewer, parent-role, and time checks run. A classifier's sub-agents inherit that role through the parent rule.
• Classifier spend stays out of writer cost — accuracyRows now passes share() only implementing and iterating sessions. reviewerRows still takes only reviewing sessions, so the classifier's spend counts in neither rollup.
• The tiers rollup sits beside reviewers — the Usage shape, the README, and the --help text combine both PRs' wording without duplicates or lost clauses.
• Fixture change — OWNED is now the classifier session. The unknown-effort case it used to cover in the end-to-end test now has its own unit test in describe("share").

  ^{｜ Fix it ➔ ｜ View workflow run ｜ Using claude-opus-5-5 ｜ 𝕏}

[pullfrog]

ℹ️ No functional issues. The parent-fallback fix is correct. One misplaced doc comment from the merge is noted inline.

Reviewed changes

This pass covers merge commit 0e8da02. It brings in main through a5b1225, including the writer A/B arms from #720, and fixes the earlier parent-fallback finding. vitest run test/usage.test.ts test/review.test.ts passes (107 tests), and tsc --noEmit is clean.

• Fixed the parent-PR fallback across repositories. In place(), a session now inherits its parent's PR only when owned(session) is null or includes that PR's repository. The new test keeps a sub-agent in other/repo unattributed. The README and --help wording match.
• Merged the arms rollup into the shared outcome helper. Main's outcomes() closure gave way to this PR's outcome(row, each, latest) with tallied(). accuracyRows and armRows both use it, so ArmRow now carries firstRounds, and the arm text line shows first rounds.
• Combined the README and --help text. They now describe the writer A/B rollup next to the new writer-cost and local-reviewer rollups, with no clause duplicated or lost.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using claude-opus-5-5 ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes

This pass covers a44a1e5, the only commit since the prior Pullfrog review.

• Moved tallied()'s doc comment back above it. The comment now sits at src/cli/usage.ts:743 directly over tallied, and noOutcomes no longer carries a stale branch-tally doc. This resolves the earlier misplaced-comment finding.

  ^{｜ View workflow run ｜ Using claude-sonnet-5-5 ｜ 𝕏}