Repository navigation
feat(review): quest review runs opencode reviewers - #603
Conversation
Quest-Claim: 10b2de8ca4e9f140
`quest review --harness opencode --model <provider/model> --effort <variant>` reviews read-only: opencode loads no project config or Claude Code files, every tool but its shell is denied, and the shell runs each command under sandbox-runtime with no network, reading only the checkout and its Git directories and writing only a private directory. The sandbox and the seccomp-built srt package are restored from kix's runner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 42 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: fec1772
[P1] src/cli/sandbox.ts:60–65: enforce the advertised checkout-only read boundary. allowRead grants exceptions to the listed denyRead roots; it does not deny reads elsewhere. For a checkout under /workspace, a shell command can still read another project's /workspace/other/.env (likewise user data under /mnt or /srv). The code also explicitly acknowledges that an existing shared /tmp/claude remains readable despite denying /tmp. A reviewed head can induce those reads, and shell output goes back to the provider even though the shell itself has no network. Restrict non-runtime reads to the checkout/Git/private-temp paths, and refuse or isolate the runtime's shared /tmp/claude mount instead of only denying writes. Add direct-shell negative tests for a sibling checkout outside the hidden roots and a planted file in /tmp/claude; the current home-read test misses both cases.
Overall direction: the harness integration and project-config suppression are well structured, but the read-isolation promise needs tightening before relying on this for untrusted changes. Verification limits: GitHub-only static review of the patches and sandbox configuration; no builds, tests or live sandbox probes run independently. The PR itself reports incomplete full-suite validation. Rechecked open/non-draft state, full head SHA and reviews immediately before posting.

Finishes quest/m2/review-harnesses.md:
quest review --harness opencodereviews a branch as the Claude Code and Codex reviewers do.Summary
quest review --harness opencode --model <provider/model> --effort <variant>runsopencode run --standalone --format jsonfor the review turn and resumes that session (--session) for the schema turn. opencode has no review mode, so the first turn is a review prompt plus the usual scope, REVIEW.md, and responses;opencode runtakes no schema, so the findings turn's prompt carries it and the reply is parsed and checked byverdictas before (a Markdown fence around the JSON is stripped).OPENCODE_DISABLE_PROJECT_CONFIGandOPENCODE_DISABLE_CLAUDE_CODEkeep out the checkout'sopencode.json,.opencode/plugins, MCP servers, shell,AGENTS.md, and skills. The base's instructions arrive as aninstructionsfile.shellis denied (*deny, thenskilldeny, thenshellallow). These rules come last inOPENCODE_CONFIG_CONTENT, so none of the user's own rules can outrank them. No LSP, formatter, snapshot, or share.srt): no network, reads only the checkout and its Git directories (the rest of/home,/tmp, and the user's home are hidden), writes only a private temp directory, starts from a cleared environment with no provider keys, and uses a PATH with no directory inside the checkout. On Linux, every round first probes thatsrtrefuses Unix sockets, which needs its seccomp helper, and refuses to run otherwise.src/cli/sandbox.tsandnix/sandbox-runtime.nixcome back from kix's runner (git show 7fe97931^:src/runner/sandbox.ts), trimmed to a read-only reviewer.opencode modelsdoesn't list the model, the round fails as logged out (exit 2). Aprovider.autherror also exits 2. opencode's limit errors (provider.quota,go-limit,free-limit,chatgpt-limit,budget-limit,model-budget-limit, or aLIMITmessage) exit 3 throughReviewLimitError, with the reset taken fromresetsAt/retryAfterMs. Aretryevent carrying a limit stops the CLI instead of letting it wait for the reset.run()setsPWDto the checkout for every reviewer CLI, because opencode runs inPWD, not in its working directory.srt(with the seccomp helper), and on Linuxbubblewrapandsocat.just review-sandboxcovers opencode. Its planted MCP server, two plugins, and shell all fire in the control run and none fire underquest review. A further test runs writes, a home read, and a network request through the sandbox shell directly (all blocked), and has the model report$SANDBOX_RUNTIMEto show its commands went throughsrt.QUEST_LIVE_OPENCODE_MODELpicks the model (defaultopencode-go/deepseek-v4.1-flash#high).Public API
quest review --harnessacceptsopencode, which no size's list picks, so it needs--modeland--effort. The## Reviewrecord names itopencode.REVIEW_HARNESSES,ReviewHarness,Reviewing,opencodeTurn,opencodeReset.reviewerEnvironmentnow takes the model. Nothing is on the wire. No breaking change for the existing harnesses.Choices made without asking
opencode runrather thanopencode serveplus@opencode/client(which kix used). This fits the existing spawn-per-turn reviewer and adds no dependency. What it gives up: there is no API check that no foreign config loaded. The live test's control run covers that instead.catandrgrun inside the OS sandbox, so the plugin isn't needed.REVIEWERS. Which model the council's DeepSeek member runs is council's call (it namesopencode-go/deepseek-v4-proat high).Validation
just check's steps all pass, run directly: tsc, prettier,quest checkon both trees, nixfmt, and actionlint. The gatedjust check/just testwaited more than 2 hours for host room (load average 100 to 145) and were stopped, sojust testas a whole, including the install scripts and lifecycle builds, did not run here. These suites pass with--testTimeout 60000: review, sandbox, cli, snapshot, and usage (143 tests). At the default 5s, five review tests timed out on the loaded host.just review-sandboxwith opencode 2.2.6, codex, and Claude Code: opencode 4/4 and Codex 3/3 pass.ANTHROPIC_BASE_URL, copied logins can't authenticate. With that unset, 3 of 4 pass. The control fails only because Claude Code's plantedapiKeyHelperno longer fires. That failure is unrelated to this change: it fails the same way whether or not the control setsPWD, and this PR doesn't touch Claude's flags. Follow-up below.Limitations
bwrapmay be refused. When that happens the socket probe fails and the round stops with that error, rather than running unsandboxed.--effortalways becomes#variant.Follow-ups
apiKeyHelperfire (Claude Code 2.1.x print mode with a stored login). The model check says to plan a quest for anyjust review-sandboxfailure. Recommendation: plan it under the model check, to either drop that plant fromCONTROLor find the release change.Models
Size: M
Implementer: Claude Code, claude-opus-5-5, medium effort
(Written by Claude Opus 5.5 with medium)
Review
Rewritten by
quest reviewafter each round.01a1c52d-dc47-7d11-8108-b38af3e7b2da, round 1The implementation appears sound by inspection with no actionable defects identified, but builds, tests, and live sandbox checks were not run, as instructed.