Skip to content

feat(usage): count hand-backs on merged PRs from earlier bodies - #723

Merged
kixelated merged 2 commits into
mainfrom
quest/m0/routing-evidence/handbacks
Nov 14, 2026
Merged

kixelated merged 2 commits into
mainfrom
quest/m0/routing-evidence/handbacks

Conversation

@kixelated

@kixelated kixelated commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m0/routing-evidence/handbacks.md.

Summary

quest usage now counts a PR as handed back if its current draft body or any earlier version had a ## Waiting / ## Decisions needed section. Before this, merged PRs always read 0, because quest ready removes the section before merge.

  • src/github/online-review.ts: readActivity can now also read userContentEdits { diff }, which returns every earlier full body. It uses the same batched, paged GraphQL query that readOnlineReviews already makes, so there are no extra round trips (10 PRs per query, as before). signals.ts doesn't ask for the edits, so its reads stay the same.
  • src/cli/usage.ts: a row counts as handed back when any earlier body passes handedBack from src/core/claim.ts. Each PR still counts once.
  • Test: merged PR feat(skills): label planned issues when the PR opens #10's edit history contains a ## Waiting body, and it now counts in both of its writers' accuracy rows.
  • Removed the quest file and its caveat in quest/m1/model-check.md.

Choices made without asking

  • "While it was a draft" is approximated. Any earlier body with a hand-back section counts, whether or not the PR was a draft at that moment. Checking draft state per edit would mean reading timeline events for every PR. By convention only drafts carry these sections.
  • The epic now has a size: quest/m0/routing-evidence/README.md is [L]. This was its last child, and quest check requires a size once an epic has none left. The remaining work is the experiments' readout and ending the A/B, which is mostly judgment.
  • Cost: a PR with a long edit history adds up to 100 bodies per page to the response. Tested on feat(usage)!: cost and first rounds per writer and reviewer cell, for the routing charts #721: 14 edits of about 16 KB each, GraphQL cost 1.

Models

  • Size: S
  • Implementer: Claude Code, claude-opus-5-5, low effort

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with low)

Review

Rewritten by quest review after each round.

  • Head: 4d26d19

  • Base: origin/main at 85d0a78

  • Risk: high, The changed quest/github return shape and historical hand-back counting can break Kixel Code and scripts that consume review records or quest usage output; reverting restores behavior but cannot undo decisions already made from those records.

  • Classified: high by Codex, gpt-6.1-sol, low effort, session 01a1d7d3-1160-7180-bddc-aaf620979ddc

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1d7d3-2836-7bd2-ad4a-1b0210b93a31, round 1

  • Score: 9/10, No actionable defects were found and the history lookup and rollups appear consistent, though builds and tests were not run as instructed.

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

  • Reviewer: Codex, gpt-6.1-sol, low effort, shadow

  • Session: 01a1d7d3-2836-73d2-937e-13c9f6ddae7a, round 1

  • Score: 8/10, No actionable regressions were identified and the implementation aligns with updated tests, although builds, tests, and live GitHub behavior were not exercised under the review constraints.

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

kixelated and others added 2 commits November 13, 2026 18:08
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Count hand-backs on merged PRs feat(usage): count hand-backs on merged PRs from earlier bodies Nov 14, 2026
@kixelated
kixelated marked this pull request as ready for review November 14, 2026 02:12
@coderabbitai

coderabbitai Bot commented Nov 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 5 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 3efc279d-cc15-4900-bfc3-d401f1674c87
Commits

Reviewing files that changed from the base of the PR and between 85d0a78 and 4d26d19.

Files selected for processing: 6

  • quest/m0/routing-evidence/README.md
  • quest/m0/routing-evidence/handbacks.md
  • quest/m1/model-check.md
  • src/cli/usage.ts
  • src/github/online-review.ts
  • test/usage.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

I reviewed the full diff: how quest usage reads hand-backs from a PR's body edit history, and the quest-tree cleanup.

  • Optional userContentEdits connection: readActivity takes an extra list, and the base CONNECTIONS leaves edits out, so reads in signals.ts are unchanged. The first query and the per-connection paging both work through the merged connections list.
  • readOnlineReviews return shape: it now returns { online, bodies }. Null diff entries, which come from deleted edits, are filtered out.
  • Hand-back from history: usage sets row.handedBack when any earlier body passes handedBack. This runs before accuracyRows and the other rollups, so each PR still counts once.
  • Test: #10's mocked edit history has a ## Waiting body and a null diff, and it now counts in both of its writers' accuracy rows. bun test test/usage.test.ts and tsc --noEmit pass locally.
  • Quest tree: the finished quest file is removed, the epic is sized [L], and the caveat in quest/m1/model-check.md is gone.

Edit history includes the PR's current body as its newest entry. So the old draft && guard no longer limits what counts: a ready PR whose current body still has a hand-back section now counts too. The PR body already calls this out as an accepted approximation.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 14, 2026
Merged via the queue into main with commit 224ea74 Nov 14, 2026
28 checks passed
@kixelated
kixelated deleted the quest/m0/routing-evidence/handbacks branch November 14, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant