Repository navigation
test(ts): check the differential harnesses' step lists, named cases, and kept references - #253
Conversation
Quest-Claim: 41d91de2d6de9d75
…and kept references The task harness logs each step's name on both sides, so a step list that drifts, or a name that no longer picks its step, fails the differential run. stock names its cases and its caller targets `buf::reserves` by name, as lang's does. lang's `refs` gains references kept in a tuple and read after the index or reference they were borrowed through changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 15 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed c247b8e7895e11feb58fcdd718863f24e7e22c86 against base e51ecadaf9ff14fb779915e75dd492e7be61a5ec.
No actionable bugs found. The differential runner's exhaustive single-op inputs make the new native/TS step-name logs detect drift in the current step lists, including steps that otherwise do nothing without a channel. Stock's named lookup preserves its case order and removes the fragile last-index assumption. The new reference fixture and generated snapshot preserve the indexed value and original reborrow targets across subsequent index/reference reassignment.
Direction: reusing differential output and a named case lookup is simpler than adding another native export or generating one step list from the other. The PR correctly distinguishes semantic regression coverage from proof that every lowering-predicate branch is independently necessary.
Verification limits: GitHub-only static review with an independent check. No compilation, differential runs, branch instrumentation, mutation experiments or timing measurements were performed; the reported local results and exhaustive branch-audit claims were not independently reproduced.
(Written by OpenAI)
Problem
The differential harnesses could silently test less than they claim:
harness.rs's andtests/task.ts's step lists by order only. A TS step whose name no longer matches the Rust step at its index still passes, because both sides run the same function, but a scripted case written by name then runs another step. For example, swapping only the nameshelpersandboxedintask.tspassesmain's task test.caller.tspicksbuf::reservesasall().length - 1. With a case appended after it,main's stock test still passes while its reserves inputs run the new case.Approach
step <name>on both sides, so the logs differ wherever the lists do. The exhaustive single-op cases run every index, so any drift fails, and the failure names both steps.harness::cases()lists(name, fn)pairs and gains the rootharness::op(name), as lang's does.caller.tstargetsop("buf::reserves"), which panics on a name that no longer exists.refsaudit: I instrumented the fix(ts): read a tuple's parts as they were when it was made #148 predicate (rereads/rvalue_rereads,ts/src/lower/mir/mod.rs:1699) and translated lang (instrumentation not committed). The table below shows which branchesrefsreaches.refs::keptadds the two shapes the review asked for: a tuple of references read after the array index they borrowed at moves on, and after the reference they reborrowed through is reassigned, directly, through a struct field, and via a copy.refsbeforerefsafterWhat the audit found, checked by forcing each branch to
trueand diffing lang's translation and differential run:Some(_) => false) is unreachable. The translator readsoptimized_mir, and the MIR validator only allowsDerefas a place's first projection. A reborrow through a reassigned reference arrives as aCopyForDereftemp plus a deref at the root.Expr::is_path, so an indexed part is always materialized. Even withis_pathadmitting them, the borrow indexes MIR's single-assignment copy of the index (_42), which is fixed.rereadsandrvalue_rereadsbranches totrueleaveskept's output unchanged: a held value that reads a later-written variable is flushed before the write. Sokeptpins the semantics against future changes to either mechanism, not today's predicate arms.falseanswers. Across all of lang, forcing the plain-place, copy, deref-at-root, index, or other-rvalue branches totruepasses the differential run, with no or behavior-identical output changes. Only the held, alias, and borrow-of-non-variable branches'falseanswers are load-bearing in lang.Each check fails on a broken harness first
helpersandboxedintask.ts:main:test result: ok35366 cases, 4988 panics, 8122 differences, firstinput 0e: nativestep helpers, tsstep boxedslicesafterbuf::reserveswithmain's caller:test result: ok. A probe asserting the picked case is stillreservesthrowsreserves is no longer the last case.buf::reserveon this branch:Panic: no case buf::reserves, test fails.refs: see the audit above. No predicate mutation changeskept's output, so the case guards semantics rather than one arm.Cost
cargo test --test fixtures task, user + sys of the process tree). That is within noise.kept: runs only on random inputs that pickrefs(about 1/90 of lang's cases).opruns once at import.Impact
None: test fixtures and the task test runner only. stock's ts.toml gains the root
harness::op, so its snapshot gainsopand namedcases. lang's snapshot gainsrefs::kept,Nest, andHeld.Alternatives
runalone, so this needs a second export or parsing source text. Logging names reuses the differential run and also labels every failure.cells,locks) too. Their literal bytes pick sub-ops modulo a fixedmatch, so an append moves none of them. Not done.Choices made without asking
step <name>, logged before the step runs.rereadsandrvalue_rereads, which decides how a reference part of an aggregate is lowered.Follow-ups
Review
quest review, Codexgpt-6.1-sol, medium effortLocal checks:
just cipassed on this head.🤖 Generated with Claude Code
(written by Claude Opus 5.5)