Skip to content

chore(deps): bump ordeal =0.22.1 -> =0.36.0 and teach synth-verify the five 0.28 derived-op nodes - #1705

Merged
avrabe merged 1 commit into
mainfrom
v091-ordeal036
Nov 11, 2026
Merged

avrabe merged 1 commit into
mainfrom
v091-ordeal036

Conversation

@avrabe

@avrabe avrabe commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

chore(deps): bump ordeal =0.22.1 -> =0.36.0 and teach synth-verify the five 0.28 derived-op nodes

Refs #1693. Refs #1590.

This PR supersedes dependabot #1693, which bumps ordeal to 0.36.0 and fails 4 checks, and dependabot #1590, the earlier 0.34.0 attempt. The version is the same as #1693's, with the API breaks repaired and measured evidence attached. The pin stays exact (=).

Why #1590 failed

The failures had one cause. ordeal 0.28 (#192 phase 5) admitted bvnot, bvneg, bvrotl, bvsdiv and bvsrem to the closed fragment as their own BvTerm nodes, each with a Lean-proven blasting rule. ordeal::lowering::* now returns those nodes instead of a rewrite into core ops. synth-verify's loud catch-all in canonicalize_bv (term.rs:310) fired on them, which caused all 15 panics in Test and in Z3 Verification. It fired as designed, so no verdict was ever minted over an unmodeled node. The mutation survey red on #1590 was a runner shutdown (The runner has received a shutdown signal), not a finding.

Changes

  • crates/synth-verify/src/term.rs
    • bv_rank: unique ranks 19..23 for Not/Neg/Rotl/Sdiv/Srem. They are appended, so no existing commutative operand order, and therefore no CNF, moves.
    • ord_bv: same-variant arms for all five.
    • canonicalize_bv: recurses into the children and never reorders operands, because none of the five commutes.
    • fmt_bv: real SMT-LIB mnemonics.
  • crates/synth-verify/src/solver.rs: the Z3 oracle maps the five nodes to Z3's native bvnot/bvneg/bvrotl/bvsdiv/bvsrem (bvsrem, not bvsmod). The oracle therefore checks ordeal's composite rules against an independent implementation.
  • Every loud catch-all stays.
  • Cargo.toml: the pin rationale is extended. Cargo.lock gains ordeal-cert 0.36.0, the certificate types split out in 0.36 and re-exported at their old paths, and ordeal-lrat moves to 0.36.0.
  • Bazel: crates/BUILD.bazel references @crates//:ordeal, which crate.from_cargo reads from Cargo.lock. MODULE.bazel.lock lists no ordeal entry, so there was nothing to edit.

Other upstream changes read in CHANGELOG 0.23 to 0.36:

  • From 0.32, the trusted term-DAG and query re-check runs on every check*. A trusted rejection degrades to Unknown, never to a wrong answer.
  • 0.34 changes the QueryTerm::flatten signature. 0.36 moves flatten to a trait. synth uses neither.
  • No other API that synth uses changed: Solver::new/assert/check/check_with_limit/check_with_deadline, CheckResult, Model, ordeal::trap::* and ordeal::eval::*.

Evidence: ordeal 0.22.1 vs 0.36.0, same machine, same commands

0.22.1 0.36.0
cargo test -p synth-verify --no-fail-fast 276 pass / 0 fail 276 / 0
... --features z3-solver,arm (CI Z3 job) 278 / 0 278 / 0
serial ordeal+Z3 differential, SYNTH_SOLVER_DIFF=1 --test-threads=1, per-test verdict sequence 182 tests, 1283 verdicts identical (diff empty)
differential pairs (unsat/unsat, sat/sat, disagreements) 476 / 79 / 0 476 / 79 / 0
ordeal-only trap path (unsat / sat) 94 / 59 94 / 59
synth-cli --features verify solver tests (6 binaries) 29 / 0 29 / 0
synth-backend solver-heavy tests (i64_expansion_certification, issue_1055_reread_window) 12 / 0 12 / 0

40 of the differential queries on 0.36.0 carry a Rotl (39) or Sdiv (1) node, so the new Z3 arms are exercised by the real suite. Srem and Neg reach the seam only through the controls below.

Planted miscompiles. These were applied to a scratch state, never committed, and reverted with git diff crates/ empty afterwards. Each was run under ordeal alone and under the ordeal+Z3 differential, on both versions. The results are identical across versions.

  • ARM ADD modelled as SUB: verify_i32_add gives Invalid, counterexample input_1 = 0xC0000000. Z3 agrees (sat/sat).
  • ARM ROR modelled as ROTL, which puts a Rotl node on both sides: verify_i32_rotl_transformation gives Invalid, counterexample 0x80000000. Z3 agrees.
  • ARM SDIV modelled as UDIV: verify_i32_div_s is rejected by the trap gate. That path is ordeal-only, so Z3 does not see it, and it is reported here as such.
  • Seam controls for all five nodes, at widths 8 and 32:
    • True identities give Unsat: a = (a sdiv b)*b + (a srem b), neg a = not a + 1, rotl(a,b) = rotr(a,-b).
    • Planted ones give Sat: sdiv as udiv, srem as urem, neg as not, rotl as rotr, and the srem identity with udiv.
    • All agree with Z3 on both versions.

Timing. These are exec times with binaries prebuilt, on a shared machine.

0.22.1 0.36.0
synth-verify serial differential 81 s 73 s
synth-verify z3-solver,arm (parallel) 47 s 41 s
issue_1055_reread_window 66.8 s 67.8 s / 70.5 s

One 0.36.0 reading of issue_1055_reread_window was 93.5 s, taken under load average 19 on 10 cores. A re-run gave 67.8 s and 70.5 s. No query lost its bound: the seam and trap.rs still call check_with_deadline, unchanged.

Rebased onto z3 0.22.0 (#1702)

The evidence above was measured on a base with z3 0.21.1. This branch is now rebased onto ef9897c, which carries z3 0.22.0 / z3-sys 0.14.1. The rebase was clean, cargo metadata --locked accepts the merged Cargo.lock, and on the rebased head:

  • cargo test -p synth-verify --no-fail-fast: 276 pass / 0 fail.
  • cargo test -p synth-verify --features z3-solver,arm --no-fail-fast: 278 / 0.
  • cargo fmt --check and cargo clippy --workspace --all-targets -- -D warnings: clean.
  • cc_sweep.py: steps=55 failures=0.

The serial ordeal+Z3 verdict-sequence differential and the planted-miscompile controls were not re-run against z3 0.22.0.

Gates (local)

  • cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings: clean.
  • cargo test --workspace --no-fail-fast: 3271 passed / 0 failed (288 s).
  • python3 scripts/mutation_survey.py ci: MUTANTS-CI subset=10 controls=3 non-killed=7 failures=0; ci_mutants_gate.py ok; mutant_residue_check.py residue=0; afterwards pgrep -fl mutation_survey empty and git diff crates/ empty.
  • Claim Check sweep (cc_sweep.py): steps=55 failures=0. The commit subject is deliberately chore(deps): ...: a lane-shaped v0.91 ORDEAL: subject reds R10-lane because no RQ-91-ORDEAL artifact exists. Keep the PR title chore(deps): ... too, or add the artifact first.

The z3-feature build still warns unused import: z3::ast::Ast (solver.rs:257). That warning is pre-existing on 0.22.1 and is not touched here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L

…e five 0.28 derived-op nodes

ordeal 0.28 (#192 phase 5) admitted bvnot/bvneg/bvrotl/bvsdiv/bvsrem to
the closed fragment as their own BvTerm nodes with Lean-proven blasting
rules, and ordeal::lowering now returns those nodes instead of a rewrite.
That is the single cause of dependabot's 0.34.0 failures (15 panics at
the term.rs canonicalize_bv catch-all, in Test and in Z3 Verification;
its mutation-survey red was a runner shutdown). The catch-all fired as
designed, so no verdict was ever minted over an unmodeled node.

term.rs: bv_rank gets unique ranks 19..23, APPENDED so no existing
commutative operand order or CNF moves; ord_bv gets the same-variant
arms; canonicalize_bv recurses through the children and never reorders
operands, since none of the five commutes; fmt_bv prints real SMT-LIB
mnemonics. solver.rs: the Z3 oracle maps the nodes to Z3's native
bvnot/bvneg/bvrotl/bvsdiv/bvsrem (bvsrem, not bvsmod). Every loud
catch-all stays.

Evidence, ordeal 0.22.1 vs 0.36.0, same machine:
- synth-verify: 276/276 (default) and 278/278 (z3-solver,arm) pass on
  both versions.
- Serial ordeal+Z3 differential (SYNTH_SOLVER_DIFF=1, --test-threads=1)
  over the whole synth-verify suite: the per-test verdict sequence is
  IDENTICAL, with 182 tests and 1283 verdicts (476 unsat/unsat pairs,
  79 sat/sat, 0 disagreements; trap path 94 unsat / 59 sat). 40 of the
  differential queries carry a Rotl or Sdiv node.
- Planted miscompiles: ARM ADD as SUB, and ARM ROR as ROTL, are each
  reported Invalid with the same counterexample on both versions, with
  Z3 agreeing. ARM SDIV as UDIV is rejected by the ordeal-only trap gate
  on both. Seam controls for all five nodes at widths 8 and 32 (true
  identities Unsat; sdiv-as-udiv, srem-as-urem, neg-as-not, rotl-as-rotr
  Sat) agree with Z3 on both versions.
- Exec timing is flat. The serial differential took 81 s on 0.22.1 and
  73 s on 0.36.0. synth-backend issue_1055_reread_window took 66.8 s on
  0.22.1 and 67.8 s / 70.5 s on 0.36.0 (a 93.5 s reading was taken under
  load average 19).

Every query keeps its wall-clock deadline (check_with_deadline, in both
the seam and trap.rs); no call site changed.

Supersedes dependabot #1693 (same 0.36.0, which fails 4 checks) and #1590.

Refs #1693
Refs #1590

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
@codecov

codecov Bot commented Nov 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 34.78261% with 15 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/synth-verify/src/term.rs 34.78% 15 Missing ⚠️

📢 Thoughts on this report? Let us know!

@avrabe
avrabe merged commit 1757f10 into main Nov 11, 2026
80 of 81 checks passed
@avrabe
avrabe deleted the v091-ordeal036 branch November 11, 2026 21:03
avrabe added a commit that referenced this pull request Nov 12, 2026
Since ordeal 0.36.0 (#1705), "Rust-test Line Coverage" is red on main
1757f10: under llvm-cov instrumentation the I64Popcnt expansion
certification in crates/synth-backend/tests/i64_expansion_certification.rs
returns "ordeal returned unknown (wall-clock deadline 300000 ms)". The same
job was green on ef9897c (ordeal 0.22.1) and on #1705's own PR run, so the
query sits at the deadline and runner load decides it.

Measured locally under cargo llvm-cov, same host, deadline raised so nothing
is cut off: the certification test takes 61.1 s on 0.22.1 and 71.6 s on
0.36.0 (+17 %); I64Popcnt certifies on both. Uninstrumented debug: 55.4 s ->
64.4 s. The CI rust-cpu runners are several times slower than this host.

The coverage job now sets SYNTH_ORDEAL_DEADLINE_MS=1200000. The deadline
bounds completeness only (solver.rs DEFAULT_DEADLINE_MS): a query past it
is a loud Unknown, never a verdict, so this cannot admit a wrong proof.
Test and Z3 Verification keep the 300 s default.

Refs #1705

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
avrabe added a commit that referenced this pull request Nov 12, 2026
Since ordeal 0.36.0 (#1705), "Rust-test Line Coverage" is red on main
1757f10: under llvm-cov instrumentation the I64Popcnt expansion
certification in crates/synth-backend/tests/i64_expansion_certification.rs
returns "ordeal returned unknown (wall-clock deadline 300000 ms)".

Bisected upstream (pulseengine/ordeal#308): the step is 0.31.0 -> 0.32.0,
which made ordeal's trusted term check run on every verdict. That check is
deliberate and stays; making it cheaper is ordeal#283.

synth spends nearly all of this test inside ordeal, which the dev profile
built unoptimized. [profile.dev.package.{ordeal,ordeal-lrat,ordeal-cert}]
now sets opt-level = 3. Same host, default 300 s deadline:

  shipped_expansions_certify   before    after
  debug cargo test              64.4 s   15.1 s
  cargo llvm-cov test           71.6 s   19.1 s

I64Popcnt certifies unsat, LRAT-checked, in both. The deadline is not
touched (an earlier version of this PR raised it for the coverage job; this
replaces that). Only dependency build settings change: no synth code
generation, no release profile. synth-verify 276/0, clippy and fmt clean,
cc_sweep 55/0, Cargo.lock unchanged (--locked).

Refs #1705

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
avrabe added a commit that referenced this pull request Nov 12, 2026
Since ordeal 0.36.0 (#1705), "Rust-test Line Coverage" is red on main
1757f10: under llvm-cov instrumentation the I64Popcnt expansion
certification in crates/synth-backend/tests/i64_expansion_certification.rs
returns "ordeal returned unknown (wall-clock deadline 300000 ms)".

Bisected upstream (pulseengine/ordeal#308): the step is 0.31.0 -> 0.32.0,
which made ordeal's trusted term check run on every verdict. That check is
deliberate and stays; making it cheaper is ordeal#283.

synth spends nearly all of this test inside ordeal, which the dev profile
built unoptimized. [profile.dev.package.{ordeal,ordeal-lrat,ordeal-cert}]
now sets opt-level = 3. Same host, default 300 s deadline:

  shipped_expansions_certify   before    after
  debug cargo test              64.4 s   15.1 s
  cargo llvm-cov test           71.6 s   19.1 s

I64Popcnt certifies unsat, LRAT-checked, in both. The deadline is not
touched (an earlier version of this PR raised it for the coverage job; this
replaces that). Only dependency build settings change: no synth code
generation, no release profile. synth-verify 276/0, clippy and fmt clean,
cc_sweep 55/0, Cargo.lock unchanged (--locked).

Refs #1705


Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant