Skip to content

feat(vibe): Codify Codex budget rules - #312

Merged
shafty023 merged 5 commits into
mainfrom
feat/vibe-codex-budget-policy
Oct 20, 2026
Merged

shafty023 merged 5 commits into
mainfrom
feat/vibe-codex-budget-policy

Conversation

@shafty023

@shafty023 shafty023 commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fourteen vibe sessions on Codex exhausted two accounts' weekly limits. They ran xhigh writers, explorers on the sol default, broad local suites at every handoff, and containerized e2e stacks that collided on one machine. This codifies five budget rules in the vibe plugin.

1. Reasoning effort defaults to high. cl-sweep/scripts/model-policy.mjs now owns DEFAULT_REASONING_EFFORT = 'high', and DEFAULT_TICKET_REASONING_EFFORT reads it, so new cl-sweep ticket-owner sessions persist high. --reasoning-effort xhigh still works, and a persisted session keeps its stored effort on every turn (new test). The support-lane text in cl-sweep and cl-execute now says high, with xhigh only when raised explicitly.

2. Model by role, one source of truth per harness.

  • Codex: model-policy.mjs carries a role table (writer, reviewer, verifier on gpt-6-sol, the existing default; explorer on gpt-5.6-terra; mechanical on gpt-6-luna). native-app-server-orchestrator.mjs now reads its model constants from the policy instead of repeating them. The vibe, handoff and vibe-seed-refresh skills take each native spawn's model and reasoning_effort from the new vibe/scripts/worker-model.mjs codex <agent>, which maps every vibe agent to a role (tested against agents/*.md). Explorer and research support lanes that ran on sol move to terra; lanes already on luna stay there. Persisted sessions keep their stored binding, and the "never create a new gpt-5.6-* session" rule is now scoped to ticket-owner sessions.
  • Claude Code: agent frontmatter stays the source of truth. vibe-backend-worker (read-only advice) moves from opus to sonnet. The other explorers were already on sonnet, and writers, reviewers and verifiers keep their models. No vibe agent is a pure lookup, so none moves to haiku.

3. Targeted validation. The verify worker runs Biome on changed files, source gates, pnpm typecheck:affected, and only the tests code_tests_for returns for each changed file. It never runs pnpm test, pnpm verify, test:affected or a whole lane, and its full-suite mode is gone. The handoff publication recipe (tools/vibe-workers/src/main-sync-checks.ts) drops test:affected, lane execution, turbo test, pnpm verify, pnpm test, test:lint, test:skills, desktop test:renderer and storybook test. It keeps Biome, source gates, typecheck:affected, turbo typecheck ...[since] (changed packages and dependents), storybook validate:catalog and vibe storybook-diff. Lane discovery still runs, but only to record required browser and Desktop e2e lanes for CI evidence. Storybook guidance runs only the changed story files. The repo's own pre-push hook still runs pnpm verify --scope=push, and the broad suites run in CI and after merge. Rule text was amended where it lives (verify worker, handoff, guardrails, quality loop, graph table, environment, primitive and decomposer agents); cl-execute and cl-sweep validation text is unchanged.

4. Shared e2e stack. The plugin cannot own symphony-alpha's compose project name or ports, so it serializes instead. vibe/scripts/e2e-lock.mjs run <command> holds one machine-wide lock, waits for the stack to be free (exit 75 with the holder when it stays busy), and takes over a lock left by a dead process. The verify worker and handoff run only the specs the change touches, through the lock.

5. Usage awareness.

  • Codex: vibe/scripts/codex-usage.mjs check calls App Server account/rateLimits/read (verified against codex-cli 0.163.0). It exits 3 with status: stop when a limit is reached and ordinary usage is no longer allowed, or continues with --allow-credits while credits remain (LIMIT-STOPPED, with no credits left, stops even then). A reached limit that still allows ordinary usage continues (status: ok, state: LIMIT). The vibe skill runs it before spawning workers or forwarding a request, and when a turn stalls.
  • One reader for both skills: the read and its classification live in codex-fleet-monitor/scripts/ratelimit.mjs (merged in feat(vibe): Add codex-fleet-monitor skill #310). readRateLimits({ transport }) and classifyUsage return ok, LIMIT, LIMIT-SPENDING-CREDITS, LIMIT-STOPPED or unknown. codex-usage.mjs imports them and only maps the state to its gate; its exit codes and statuses are unchanged, and the output gains state. liveness.mjs now has one JSON-RPC client base with two framings. The monitor uses the daemon proxy. The gate uses a private stdio codex app-server, so a check inside a vibe session never starts the managed daemon. The classification table is tested once in ratelimit.test.mjs; codex-usage.test.mjs keeps the gate tests and adds the ordinary-usage-allowed case.
  • Claude Code: there is no read-only usage query to call before a turn. The owned launcher reads each turn's stream-json rate_limit_event instead. It returns the last one as rateLimit and stops a turn that hits rejected or starts paid overage with USAGE_LIMIT_REACHED, unless the launch input sets allowCredits. So on Claude, the check runs during each turn rather than before it.

Removed as unreachable after rule 3: main-sync-budget-ref-read.ts, the affected-test wrapper pin and its fixture, the test:lint budget, the test-selection report admission, the captured-main mapping of test:affected, and the budget variant of the native-failure helpers.

These tests were deleted because their only contract was budgeting or admitting test commands inside the recipe:

  • main-sync-budget-ref-read.test.ts
  • five main-sync-budget cases (aggregate test budgets, the full test:lint budget, the captured-main test pin)
  • the test cases of the report-admission selected-task test
  • the replay case of the input-boundary retained-report test
  • the budget variant of the native-failure cases
  • the lane test that stored 2,549 paths in recipe argv

These were rewritten instead:

  • the three handoff-matrix tests in main-sync.test.ts now pin the targeted matrix, with counterfactual "no test command runs" assertions
  • the lane, comparison and budget tests now assert the typecheck path

Not changed, possible follow-ups: cl-execute's "run the comprehensive validation once on the final tree" step still allows broad local suites. The prototype publication recipe still runs pnpm --filter prototypes test.

Vibe goes to 1.0.60 in both manifests, because #310 claims 1.0.59.

Validation

  • uv run ruff check . and uv run pyright: clean
  • After merging main (feat(vibe): Add codex-fleet-monitor skill #310, fix(code-review): keep findings outside the diff #315) and unifying the reader:
    • uv run pytest plugins/vibe plugins/code-review: 1758 passed, 3 skipped
    • every plugins/**/*.test.{mjs,cjs,js} (the CI node step): 454 passed
  • Before that merge: uv run pytest plugins/vibe 340 passed, vibe node:test 355 passed
  • tools/vibe-workers: typecheck clean, vitest 735 passed, bundles rebuilt
  • tools/plugin-manifests: 6 passed
  • codex-usage.mjs check against a live account returned stop (exit 3) on an exhausted weekly window, and credits_allowed with --allow-credits. After the unification it returned ok with state: ok on an open window, and the monitor printed codex-usage local 0% ok and, over ssh, codex-usage remote 50% ok

Risk

Handoff no longer runs broad local suites, so a regression outside the code_tests_for set first shows up in PR CI. The Claude usage stop can end a writer turn mid-way. The request stays queued for the same writer, as it does after a timeout.

- Default Codex workers to high reasoning effort; xhigh only when raised
  explicitly, persisted session effort never rewritten
- Bind worker models by role from one policy per harness: explorers on
  gpt-5.6-terra in Codex and sonnet in Claude Code, writers, reviewers
  and verifiers on the existing gpt-6-sol
- Run only the tests code_tests_for selects per changed file plus
  Biome, source gates and typecheck; drop the verify full-suite mode and
  the handoff recipe's test suites, with their now unused budgeting
- Serialize containerized e2e through a machine-wide lock and run only
  the specs the change needs
- Check Codex rate limits before spawning workers and on stalls, and
  stop Claude turns on a reached limit or paid overage, unless the
  person allowed credits
- Bump vibe to 1.0.60

Testing: uv run ruff check ., uv run pyright, uv run pytest plugins/vibe
(340 passed), vibe node:test suites (355 passed), vibe-workers vitest
(735 passed) and typecheck, plugin manifest parity (6 passed)

Risks: Handoff no longer runs broad local suites; CI and the repo's
pre-push hook remain the broad gate
… monitor

codex-fleet-monitor's ratelimit.mjs now owns readRateLimits and
classifyUsage (ok, LIMIT, LIMIT-SPENDING-CREDITS, LIMIT-STOPPED,
unknown). liveness.mjs gains a stdio transport next to the daemon proxy,
both on one JSON-RPC client base. vibe's codex-usage.mjs imports the
reader, keeps its private stdio App Server, and maps the state to its
gate: it now continues while ordinary usage is still allowed, and stops
otherwise unless credits were allowed. Exit codes and statuses are
unchanged; the output gains a state field.
LIMIT-STOPPED means neither ordinary usage nor credits remain, so
--allow-credits cannot make a turn run. The gate now returns stop there
regardless of --allow-credits, and the skill only offers paid credits
when the state is LIMIT-SPENDING-CREDITS.
@shafty023
shafty023 merged commit cd4c6ec into main Oct 20, 2026
10 checks passed
@shafty023
shafty023 deleted the feat/vibe-codex-budget-policy branch October 20, 2026 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant