Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1266,7 +1266,7 @@ the same for every repository unless its `AGENTS.md` names others:

| Size | Writer | Fallback |
|---|---|---|
| XS | Claude Code `claude-haiku-5-5` xhigh | Codex `gpt-6.1-sol` low |
| XS | Claude Code `claude-sonnet-5-5` high | Codex `gpt-6.1-sol` low |
| S | Claude Code `claude-opus-5-5` low | Codex `gpt-6.1-sol` medium |
| M | Claude Code `claude-opus-5-5` medium | Codex `gpt-6.1-sol` high |
| L | Claude Code `claude-opus-5-5` high | Codex `gpt-6.1-sol` xhigh |
Expand All @@ -1288,7 +1288,7 @@ XS, and `quest models` prints the entry to run:

```console
$ quest models XS
claude claude-haiku-5-5 xhigh
claude claude-sonnet-5-5 high
codex gpt-6.1-sol low

$ quest models XS --writer claude-opus-5-5
Expand Down
14 changes: 13 additions & 1 deletion quest/m1/model-check.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ checkpoints and outages to count by hand:
cell with again:
- Haiku 5.5: high 30, $0.0034; xhigh 34.6, $0.0053.
- Sonnet 5.5: medium 30.7, $0.012; below the frontier at every effort,
so rejected for XS though Anthropic's
so rejected for XS then, though Anthropic's
[model guide](https://claude.dev/blog/which-claude-model-should-you-build-with/)
favors it for scoped coding.
- Opus 5.5: low 34.7, $0.015; medium 48, $0.037; high 51.7, $0.052;
Expand All @@ -99,6 +99,18 @@ checkpoints and outages to count by hand:
- S looks lopsided: Opus low scores well below its Sol medium fallback
(34.7 against 46.3) for $0.006 less a task. Check its tallies before
moving it.
- Changed again (maintainer, 2026-11-11): XS writes on Sonnet high
instead of Haiku xhigh, since two Haiku XS agents in a row on
[moq-dev/quest#643](https://github.com/moq-dev/quest/pull/643) died
refilling context right after compaction: Haiku 5.5 bills 5x past 100k
input, so the maintainer caps its auto-compact window there, barely above an
agent's 40-60k fixed load. Sonnet 5.5's price is flat over its 1M
window. mypriors has no Sonnet high row, so tally this cell rather than
compare it there. Anthropic's
[cost guide](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence)
puts Sonnet 5.5 high level with Opus 5.5 low (87.6% against 87.4%) at
about the same cost per solved task, and medium about 7 points below
high on its SWE-bench Pro subset.
- Compare both tables' models (in `src/cli/models.ts`) with each vendor's current models: the newest
model of each tier the tables pin. Bump any that are stale, and check that both CLIs
accept the new ids and the pinned efforts. `--deep` and its strongest-model
Expand Down
10 changes: 6 additions & 4 deletions src/cli/models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,18 @@ export const DEFAULT_SIZE: Size = "M";

const sol = (effort: string): Model => ({ harness: "codex", model: "gpt-6.1-sol", effort });
const opus = (effort: string): Model => ({ harness: "claude", model: "claude-opus-5-5", effort });
const haiku = (effort: string): Model => ({ harness: "claude", model: "claude-haiku-5-5", effort });
const sonnet = (effort: string): Model => ({ harness: "claude", model: "claude-sonnet-5-5", effort });

/**
* Each size's writers, ranked: a spawn starts the quest on the first that can
* run, falling through only while nothing is written, since the model that
* writes a PR runs all its iterates. XS runs Haiku at xhigh, which scores about
* as Opus low does for a third of its subscription cost per task.
* writes a PR runs all its iterates. XS runs Sonnet rather than the cheaper
* Haiku, whose surcharge past 100k input tokens leads users to cap its
* compaction window there, barely above an agent's fixed load of prompt,
* context, and skill.
*/
export const WRITERS: Record<Size, readonly Model[]> = {
XS: [haiku("xhigh"), sol("low")],
XS: [sonnet("high"), sol("low")],
S: [opus("low"), sol("medium")],
M: [opus("medium"), sol("high")],
L: [opus("high"), sol("xhigh")],
Expand Down
4 changes: 2 additions & 2 deletions test/cli.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ test("models ranks a size's writers, or the one entry that runs a PR's writer",
const tree = Tree.baseline();
expect(tree.run("models", "XS")).toEqual({
code: 0,
stdout: "claude\tclaude-haiku-5-5\txhigh\ncodex\tgpt-6.1-sol\tlow\n",
stdout: "claude\tclaude-sonnet-5-5\thigh\ncodex\tgpt-6.1-sol\tlow\n",
stderr: "",
});
expect(JSON.parse(tree.run("models", "XL", "--json").stdout)).toEqual([
Expand All @@ -196,7 +196,7 @@ test("models ranks a size's writers, or the one entry that runs a PR's writer",
expect(writer("M", "gpt-6.1-sol")).toBe("codex\tgpt-6.1-sol\thigh\n");
// A writer its size does not list runs at the nearest size that lists it.
expect(writer("XS", "claude-opus-5-5")).toBe("claude\tclaude-opus-5-5\tlow\n");
expect(writer("L", "claude-haiku-5-5")).toBe("claude\tclaude-haiku-5-5\txhigh\n");
expect(writer("L", "claude-sonnet-5-5")).toBe("claude\tclaude-sonnet-5-5\thigh\n");
expect(writer("XL", "gpt-6.1-sol")).toBe("codex\tgpt-6.1-sol\txhigh\n");
const unknown = tree.run("models", "S", "--writer", "unknown");
expect(unknown.code).toBe(2);
Expand Down
Loading