Skip to content

feat(models): route XS writers to Claude Sonnet 5.5 high - #646

Merged
kixelated merged 3 commits into
mainfrom
models/xs-sonnet
Nov 12, 2026
Merged

kixelated merged 3 commits into
mainfrom
models/xs-sonnet

Conversation

@kixelated

@kixelated kixelated commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

XS quests' first writer moves from claude-haiku-5-5 at xhigh to claude-sonnet-5-5 at high; gpt-6.1-sol low stays the fallback. Decided by the maintainer on 2026-11-11 in a /quest-run session.

Why: two Haiku XS agents in a row on moq-dev/quest#643 died with Claude Code's "context refilled too quickly after compaction". The maintainer's settings cap Haiku's auto-compact window at 100k because Haiku 5.5 bills 5x past 100k input, and a quest agent's fixed load (system prompt, quest context, skill, guide) is roughly 40-60k tokens. Sonnet 5.5 is flat priced ($2/$10 per MTok) over a 1M window. Rejected: raising Haiku's window to 200k, removing the cap, leaving it.

Effort high (maintainer's pick over medium): Anthropic's cost guide shows Sonnet 5.5 medium about 7 points below high on its SWE-bench Pro subset and 15 below on its internal agentic-coding set, while Sonnet high matches Opus 5.5 low (87.6% vs 87.4%) at about the same cost per solved task. Artificial Analysis puts Sonnet high ahead of Opus low on Terminal-Bench 4.0.

Changes: WRITERS.XS and its comment (the unused haiku helper is replaced by sonnet), README routing table and quest models example, the CLI test, and quest/m1/model-check.md's record (new 2026-11-11 entry; XS Sonnet high to be tallied fresh, since mypriors has no Sonnet high row). quest/m0/routing-evidence/writer-ab.md is left to the coordinator's own PR.

Public API

quest models XS and quest unblocked --json now list claude-sonnet-5-5 high first for XS. No flag, schema, or wire change.

Validation

  • quest gate just check and quest gate just test: both exit 0 on the final head.
  • Built CLI (just build):
    $ ./dist/quest models XS
    claude	claude-sonnet-5-5	high
    codex	gpt-6.1-sol	low
    $ ./dist/quest models XS --json
    [{"harness":"claude","model":"claude-sonnet-5-5","effort":"high"},{"harness":"codex","model":"gpt-6.1-sol","effort":"low"}]
  • Model id: claude --help lists --effort with high, but a live claude -p --model claude-sonnet-5-5 run could not happen here (the sandbox's Claude CLI is not logged in), so the id is unverified against the live CLI.

Review

Local quest review --size S on c82ba13: Codex gpt-6.1-sol medium, score 10, 0 findings; opencode deepseek-v4.1-flash high, score 9, 2 low findings.

Head e80bee7: fixed both. (1) The new model-check entry re-parented the Nov 9 Fable and S sub-bullets; moved it below them. (2) No Sonnet high benchmark row and the comment called the 100k cap a model property; the entry now marks the cell tally-only with the cost guide's numbers, and the comment names the cap as a user setting.

Head dc177ef: merged origin/main (#641, #644, #647) to clear the merge queue's merge_conflict. The only conflict, quest/m0/routing-evidence/writer-ab.md, took main's version, which already routes XS to Sonnet 5.5 high in its A/B. git diff origin/main...HEAD matches the reviewed head's diff apart from blob indices, so the e80bee7 review covers it. quest gate just check and quest gate just test pass.

Models

Size: S
Implementer: Claude Code, claude-opus-5-5, low effort
Implementer: Claude Code, claude-opus-5-5, low effort (iterate, S)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with low)

kixelated and others added 2 commits November 11, 2026 16:36
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ap as a setting

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 57 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6e83b10f-436f-45cc-8b77-26320ed75b0d
Commits

Reviewing files that changed from the base of the PR and between 93c9c2e and dc177ef.

Files selected for processing: 4

  • README.md
  • quest/m1/model-check.md
  • src/cli/models.ts
  • test/cli.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated
kixelated added this pull request to the merge queue Nov 12, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 12, 2026
@moq-bot

moq-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

quest land watch: the merge queue removed this pull request at e80bee7 because it conflicts with main (merge_conflict).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

(written by quest land watch)

@moq-bot

moq-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

codex, gpt-6.1-sol, medium effort land fixer note:

The failure log states:

CONFLICT (content): Merge conflict in quest/m0/routing-evidence/writer-ab.md
Auto-merging quest/m1/model-check.md
Auto-merging src/cli/models.ts
Automatic merge failed; fix conflicts and then commit the result.

Merged base into this PR and resolved the paragraph by retaining the base's XS writer arm and the review wording already shared by both sides. Commit 261df56740567004a983a0df4b9f5be096429b38 is ready on pr-646. No protected paths were edited.

In the merged tree's Nix development shell, just install and just check passed. The installer regression scripts, just build, just compile, and lifecycle checks for dist/quest.js and dist/quest also passed.

just test reported a usage-test assertion failure at test/usage.test.ts:530: the expected Claude iterating session is missing. A focused run of bun run vitest run test/usage.test.ts -t 'ties each session' reproduced the same failure on both the merged tree and a separate base-only worktree. The fixture names /work as a deleted checkout, whereas this runner has a real repository there. The relevant implementation and test are unchanged from base.

The full suite also reported failure in the review test named 'Codex reaches a gateway at OPENAI_BASE_URL, whose persistent 429 is the pool's usage limit'. The suite was still running after those failures; a passing full test run has not been obtained. These validation blockers prevent reporting the conflict fix as fixed.

(written by codex, gpt-6.1-sol, medium effort)

@moq-bot

moq-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

codex, gpt-6.1-sol, medium effort land fixer: Conflict re·solved in commit 261df56740567004a983a0df4b9f5be096429b38, but nix develop --command just test reports FAIL test/usage.test.ts > usage > ties each session to its PR and role, and rolls up s. Recommendation: Retain the mechanical merge commit, re·solve the base test fixture's assumption that /work is a deleted checkout and investigate the reported review-test failure, then rerun nix develop --command just

CONFLICT (content): Merge conflict in quest/m0/routing-evidence/writer-ab.md
Auto-merging quest/m1/model-check.md
Auto-merging src/cli/models.ts
Automatic merge failed; fix conflicts and then commit the result.

(written by codex, gpt-6.1-sol, medium effort)

@kixelated
kixelated added this pull request to the merge queue Nov 12, 2026
Merged via the queue into main with commit 921efd0 Nov 12, 2026
18 checks passed
@kixelated
kixelated deleted the models/xs-sonnet branch November 12, 2026 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant