Repository navigation
chore(deps): bump ordeal =0.22.1 -> =0.36.0 and teach synth-verify the five 0.28 derived-op nodes - #1705
Merged
Conversation
…e five 0.28 derived-op nodes ordeal 0.28 (#192 phase 5) admitted bvnot/bvneg/bvrotl/bvsdiv/bvsrem to the closed fragment as their own BvTerm nodes with Lean-proven blasting rules, and ordeal::lowering now returns those nodes instead of a rewrite. That is the single cause of dependabot's 0.34.0 failures (15 panics at the term.rs canonicalize_bv catch-all, in Test and in Z3 Verification; its mutation-survey red was a runner shutdown). The catch-all fired as designed, so no verdict was ever minted over an unmodeled node. term.rs: bv_rank gets unique ranks 19..23, APPENDED so no existing commutative operand order or CNF moves; ord_bv gets the same-variant arms; canonicalize_bv recurses through the children and never reorders operands, since none of the five commutes; fmt_bv prints real SMT-LIB mnemonics. solver.rs: the Z3 oracle maps the nodes to Z3's native bvnot/bvneg/bvrotl/bvsdiv/bvsrem (bvsrem, not bvsmod). Every loud catch-all stays. Evidence, ordeal 0.22.1 vs 0.36.0, same machine: - synth-verify: 276/276 (default) and 278/278 (z3-solver,arm) pass on both versions. - Serial ordeal+Z3 differential (SYNTH_SOLVER_DIFF=1, --test-threads=1) over the whole synth-verify suite: the per-test verdict sequence is IDENTICAL, with 182 tests and 1283 verdicts (476 unsat/unsat pairs, 79 sat/sat, 0 disagreements; trap path 94 unsat / 59 sat). 40 of the differential queries carry a Rotl or Sdiv node. - Planted miscompiles: ARM ADD as SUB, and ARM ROR as ROTL, are each reported Invalid with the same counterexample on both versions, with Z3 agreeing. ARM SDIV as UDIV is rejected by the ordeal-only trap gate on both. Seam controls for all five nodes at widths 8 and 32 (true identities Unsat; sdiv-as-udiv, srem-as-urem, neg-as-not, rotl-as-rotr Sat) agree with Z3 on both versions. - Exec timing is flat. The serial differential took 81 s on 0.22.1 and 73 s on 0.36.0. synth-backend issue_1055_reread_window took 66.8 s on 0.22.1 and 67.8 s / 70.5 s on 0.36.0 (a 93.5 s reading was taken under load average 19). Every query keeps its wall-clock deadline (check_with_deadline, in both the seam and trap.rs); no call site changed. Supersedes dependabot #1693 (same 0.36.0, which fails 4 checks) and #1590. Refs #1693 Refs #1590 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
avrabe
added a commit
that referenced
this pull request
Nov 12, 2026
Since ordeal 0.36.0 (#1705), "Rust-test Line Coverage" is red on main 1757f10: under llvm-cov instrumentation the I64Popcnt expansion certification in crates/synth-backend/tests/i64_expansion_certification.rs returns "ordeal returned unknown (wall-clock deadline 300000 ms)". The same job was green on ef9897c (ordeal 0.22.1) and on #1705's own PR run, so the query sits at the deadline and runner load decides it. Measured locally under cargo llvm-cov, same host, deadline raised so nothing is cut off: the certification test takes 61.1 s on 0.22.1 and 71.6 s on 0.36.0 (+17 %); I64Popcnt certifies on both. Uninstrumented debug: 55.4 s -> 64.4 s. The CI rust-cpu runners are several times slower than this host. The coverage job now sets SYNTH_ORDEAL_DEADLINE_MS=1200000. The deadline bounds completeness only (solver.rs DEFAULT_DEADLINE_MS): a query past it is a loud Unknown, never a verdict, so this cannot admit a wrong proof. Test and Z3 Verification keep the 300 s default. Refs #1705 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
avrabe
added a commit
that referenced
this pull request
Nov 12, 2026
Since ordeal 0.36.0 (#1705), "Rust-test Line Coverage" is red on main 1757f10: under llvm-cov instrumentation the I64Popcnt expansion certification in crates/synth-backend/tests/i64_expansion_certification.rs returns "ordeal returned unknown (wall-clock deadline 300000 ms)". Bisected upstream (pulseengine/ordeal#308): the step is 0.31.0 -> 0.32.0, which made ordeal's trusted term check run on every verdict. That check is deliberate and stays; making it cheaper is ordeal#283. synth spends nearly all of this test inside ordeal, which the dev profile built unoptimized. [profile.dev.package.{ordeal,ordeal-lrat,ordeal-cert}] now sets opt-level = 3. Same host, default 300 s deadline: shipped_expansions_certify before after debug cargo test 64.4 s 15.1 s cargo llvm-cov test 71.6 s 19.1 s I64Popcnt certifies unsat, LRAT-checked, in both. The deadline is not touched (an earlier version of this PR raised it for the coverage job; this replaces that). Only dependency build settings change: no synth code generation, no release profile. synth-verify 276/0, clippy and fmt clean, cc_sweep 55/0, Cargo.lock unchanged (--locked). Refs #1705 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
avrabe
added a commit
that referenced
this pull request
Nov 12, 2026
Since ordeal 0.36.0 (#1705), "Rust-test Line Coverage" is red on main 1757f10: under llvm-cov instrumentation the I64Popcnt expansion certification in crates/synth-backend/tests/i64_expansion_certification.rs returns "ordeal returned unknown (wall-clock deadline 300000 ms)". Bisected upstream (pulseengine/ordeal#308): the step is 0.31.0 -> 0.32.0, which made ordeal's trusted term check run on every verdict. That check is deliberate and stays; making it cheaper is ordeal#283. synth spends nearly all of this test inside ordeal, which the dev profile built unoptimized. [profile.dev.package.{ordeal,ordeal-lrat,ordeal-cert}] now sets opt-level = 3. Same host, default 300 s deadline: shipped_expansions_certify before after debug cargo test 64.4 s 15.1 s cargo llvm-cov test 71.6 s 19.1 s I64Popcnt certifies unsat, LRAT-checked, in both. The deadline is not touched (an earlier version of this PR raised it for the coverage job; this replaces that). Only dependency build settings change: no synth code generation, no release profile. synth-verify 276/0, clippy and fmt clean, cc_sweep 55/0, Cargo.lock unchanged (--locked). Refs #1705 Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
chore(deps): bump ordeal
=0.22.1->=0.36.0and teach synth-verify the five 0.28 derived-op nodesRefs #1693. Refs #1590.
This PR supersedes dependabot #1693, which bumps ordeal to 0.36.0 and fails 4 checks, and dependabot #1590, the earlier 0.34.0 attempt. The version is the same as #1693's, with the API breaks repaired and measured evidence attached. The pin stays exact (
=).Why #1590 failed
The failures had one cause. ordeal 0.28 (#192 phase 5) admitted
bvnot,bvneg,bvrotl,bvsdivandbvsremto the closed fragment as their ownBvTermnodes, each with a Lean-proven blasting rule.ordeal::lowering::*now returns those nodes instead of a rewrite into core ops. synth-verify's loud catch-all incanonicalize_bv(term.rs:310) fired on them, which caused all 15 panics in Test and in Z3 Verification. It fired as designed, so no verdict was ever minted over an unmodeled node. The mutation survey red on #1590 was a runner shutdown (The runner has received a shutdown signal), not a finding.Changes
crates/synth-verify/src/term.rsbv_rank: unique ranks 19..23 forNot/Neg/Rotl/Sdiv/Srem. They are appended, so no existing commutative operand order, and therefore no CNF, moves.ord_bv: same-variant arms for all five.canonicalize_bv: recurses into the children and never reorders operands, because none of the five commutes.fmt_bv: real SMT-LIB mnemonics.crates/synth-verify/src/solver.rs: the Z3 oracle maps the five nodes to Z3's nativebvnot/bvneg/bvrotl/bvsdiv/bvsrem(bvsrem, notbvsmod). The oracle therefore checks ordeal's composite rules against an independent implementation.Cargo.toml: the pin rationale is extended.Cargo.lockgainsordeal-cert0.36.0, the certificate types split out in 0.36 and re-exported at their old paths, andordeal-lratmoves to 0.36.0.crates/BUILD.bazelreferences@crates//:ordeal, whichcrate.from_cargoreads fromCargo.lock.MODULE.bazel.locklists no ordeal entry, so there was nothing to edit.Other upstream changes read in CHANGELOG 0.23 to 0.36:
check*. A trusted rejection degrades toUnknown, never to a wrong answer.QueryTerm::flattensignature. 0.36 movesflattento a trait. synth uses neither.Solver::new/assert/check/check_with_limit/check_with_deadline,CheckResult,Model,ordeal::trap::*andordeal::eval::*.Evidence: ordeal 0.22.1 vs 0.36.0, same machine, same commands
cargo test -p synth-verify --no-fail-fast... --features z3-solver,arm(CI Z3 job)SYNTH_SOLVER_DIFF=1 --test-threads=1, per-test verdict sequencediffempty)--features verifysolver tests (6 binaries)i64_expansion_certification,issue_1055_reread_window)40 of the differential queries on 0.36.0 carry a
Rotl(39) orSdiv(1) node, so the new Z3 arms are exercised by the real suite.SremandNegreach the seam only through the controls below.Planted miscompiles. These were applied to a scratch state, never committed, and reverted with
git diff crates/empty afterwards. Each was run under ordeal alone and under the ordeal+Z3 differential, on both versions. The results are identical across versions.ADDmodelled asSUB:verify_i32_addgivesInvalid, counterexampleinput_1 = 0xC0000000. Z3 agrees (sat/sat).RORmodelled asROTL, which puts a Rotl node on both sides:verify_i32_rotl_transformationgivesInvalid, counterexample0x80000000. Z3 agrees.SDIVmodelled asUDIV:verify_i32_div_sis rejected by the trap gate. That path is ordeal-only, so Z3 does not see it, and it is reported here as such.Unsat:a = (a sdiv b)*b + (a srem b),neg a = not a + 1,rotl(a,b) = rotr(a,-b).Sat: sdiv as udiv, srem as urem, neg as not, rotl as rotr, and the srem identity with udiv.Timing. These are exec times with binaries prebuilt, on a shared machine.
z3-solver,arm(parallel)issue_1055_reread_windowOne 0.36.0 reading of
issue_1055_reread_windowwas 93.5 s, taken under load average 19 on 10 cores. A re-run gave 67.8 s and 70.5 s. No query lost its bound: the seam andtrap.rsstill callcheck_with_deadline, unchanged.Rebased onto z3 0.22.0 (#1702)
The evidence above was measured on a base with z3 0.21.1. This branch is now rebased onto ef9897c, which carries z3 0.22.0 / z3-sys 0.14.1. The rebase was clean,
cargo metadata --lockedaccepts the mergedCargo.lock, and on the rebased head:cargo test -p synth-verify --no-fail-fast: 276 pass / 0 fail.cargo test -p synth-verify --features z3-solver,arm --no-fail-fast: 278 / 0.cargo fmt --checkandcargo clippy --workspace --all-targets -- -D warnings: clean.cc_sweep.py:steps=55 failures=0.The serial ordeal+Z3 verdict-sequence differential and the planted-miscompile controls were not re-run against z3 0.22.0.
Gates (local)
cargo fmt --check,cargo clippy --workspace --all-targets -- -D warnings: clean.cargo test --workspace --no-fail-fast: 3271 passed / 0 failed (288 s).python3 scripts/mutation_survey.py ci:MUTANTS-CI subset=10 controls=3 non-killed=7 failures=0;ci_mutants_gate.pyok;mutant_residue_check.pyresidue=0; afterwardspgrep -fl mutation_surveyempty andgit diff crates/empty.cc_sweep.py):steps=55 failures=0. The commit subject is deliberatelychore(deps): ...: a lane-shapedv0.91 ORDEAL:subject reds R10-lane because noRQ-91-ORDEALartifact exists. Keep the PR titlechore(deps): ...too, or add the artifact first.The z3-feature build still warns
unused import: z3::ast::Ast(solver.rs:257). That warning is pre-existing on 0.22.1 and is not touched here.🤖 Generated with Claude Code
https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L