Overhaul test setup by Krzmbrzl · Pull Request #743 · ValeevGroup/SeQuant

Krzmbrzl · 2026-10-28T10:49:23Z

This streamlines the unit test setup. It attempts to remove copy-paste tests and instead use parametrization, consolidate test naming (some of the test names are currently way too long) and remove redundancies in test coverage. Dead or broken tests are removed

Missing commas concatenated the string literals, so only two inputs were tested. Separate them, capture the failing input, and drop "T{}", which is a valid (empty) tensor and does not throw. AI-assisted

REQUIRE_NOTHROW(a == b) never evaluated the comparison. Use REQUIRE. The flattened_rangenest distance for the one-level case is 4 (two sequences of two operators), not the 2 that was written down but never checked. AI-assisted

Delete the dead blocks in test_eval_expr, test_eval_node, test_export (including the placeholder "tbd2" section), test_mbpt and test_wick. AI-assisted

The {0} and {1} lookups on "A + B" are already the first two rows of the table. AI-assisted

The fourth row repeats the second one verbatim. AI-assisted

The 4-tensor chain g{a1;i1} g{a1;a2} g{a2;a3} g{a3;i2} is the second entry of "DensePeakSize matches oracle over a battery of small networks", which runs the identical comparison. AI-assisted

The removed section extracted the same pure non-t product f*g with the same predicate as "non-t pred: pure-non-t root collapses, mixed root is pruned"; only its !leaf() check on the extracted node was not already covered, so add that check there. AI-assisted

With strict fill-once set on the manager, the section "non-persistent: a re-store with NO intervening reset() throws under strict fill-once" already checks the same store/store sequence unconditionally; the dropped section only repeated it, and returned early unless asserts throw. AI-assisted

Both sections built the same kinds of EvalExpr (tensor, constant, variable, power); one now constructs the union of their inputs and checks op_type() is unset. This also drops the unused c2/c3 locals. AI-assisted

"ordered executor asserts the cell table before evaluating" rebuilt the fixture of "evaluate_ordered_schedule matches forest descent..." verbatim and only added the cell-table size check, which now lives there. AI-assisted

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

AI-assisted

Capture the wide-string input through toUtf8 so failures print readably. AI-assisted

Capture wide-string inputs through toUtf8. AI-assisted

Catch2 cannot render std::wstring (or vectors of them) readably. AI-assisted

AI-assisted

Hidden test cases that pass now run by default; ones that failed or were skipped (blocked on removed or unfinished functionality) are removed along with the helpers and comments that only they used. AI-assisted

The OSV early-contraction reproducers now check that every t is contracted directly with its C transform. The C60 member-2 and the two resident-scan peak probes only printed cells (the latter documents a known-missing peak term) and are removed. AI-assisted

AI-assisted

The test passes; its comments still described the failing state. AI-assisted

The whole-residual water-20/C60 sweeps and the 40000-node deep-spine test take seconds and (for the latter) ~2.8 GB in a RelWithDebInfo build, which is too much for the Debug, sanitizer and valgrind configurations that set SEQUANT_SKIP_LONG_TESTS. Mark the helpers only they use [[maybe_unused]]. AI-assisted

ajay-mk · 2026-10-28T17:32:14Z

Maybe worth looking at:

The BTAS, TAPP and TA eval tests repeat the same core test suite (summation, product, symmetrization, and so on), with only the backend calls swapped. Could we write that suite once as a template over the backend, with each backend supplying its tensor type and a few small wrappers? Each backend file would then run the shared suite and keep only its own extra tests. TA would keep its larger set of extras (tensors of tensors, batching and so on).

I got the idea from Boost.Test fixtures (used in TA). Catch2’s counterparts are type-parametrised test cases and templated test fixtures.

Krzmbrzl · 2026-10-28T18:07:22Z

Yes, that's definitely something that would improve the testing harness quite a bit. I intentionally kept it out of this PR though as to not overblow it. That will be a larger change that might also require thinking a bit more about how to handle cases in which only some of the backends are available. 👀

Master's test overhaul (#743) gave test cases terse kebab-case names; give the test cases this branch added the same style. Tags are unchanged.

Krzmbrzl added the tests Related to unit or integration tests label Oct 28, 2026

Krzmbrzl force-pushed the test-overhaul branch from 2df7119 to 255e5cc Compare October 28, 2026 11:49

Krzmbrzl marked this pull request as ready for review October 28, 2026 11:50

Krzmbrzl linked an issue Oct 28, 2026 that may be closed by this pull request

Broken hidden test cases #622

Closed

Krzmbrzl added 26 commits October 28, 2026 15:55

Fix test_parse exception inputs

52099ee

Missing commas concatenated the string literals, so only two inputs were tested. Separate them, capture the failing input, and drop "T{}", which is a valid (empty) tensor and does not throw. AI-assisted

Remove disabled (#if 0) test code

09bacee

Delete the dead blocks in test_eval_expr, test_eval_node, test_export (including the placeholder "tbd2" section), test_mbpt and test_wick. AI-assisted

Drop duplicated TreeIndex test rows

3fa3f8a

The {0} and {1} lookups on "A + B" are already the first two rows of the table. AI-assisted

Drop duplicated tensor-network isomorphism row

8fa49e3

The fourth row repeats the second one verbatim. AI-assisted

Drop DensePeakSize oracle test subsumed by the battery

de89cbb

The 4-tensor chain g{a1;i1} g{a1;a2} g{a2;a3} g{a3;i2} is the second entry of "DensePeakSize matches oracle over a battery of small networks", which runs the identical comparison. AI-assisted

Merge EvalExpr constructor and op_type sections

28bf5f2

Both sections built the same kinds of EvalExpr (tensor, constant, variable, power); one now constructs the union of their inputs and checks op_type() is unset. This also drops the unused c2/c3 locals. AI-assisted

Shorten test names in test_basis

0b27597

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_basis_wick

83fa076

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_cache_manager

f791f5d

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_canonicalize

98fd1f9

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_cell_registry

c934bb3

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_cell_table

1d36471

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_dag_dot_dump

705170a

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_dag_scope

6a0352e

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_eval_btas

d71c246

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_eval_dryrun

08e8d1d

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_eval_expr

ebc8b1a

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_eval_ordered

8bda82a

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_eval_ta

944cf1e

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_eval_tapp

831ded4

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_extract_subtrees

18a2f34

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Shorten test names in test_legality

eeb82bd

Replace prose-style test and section names with terse kebab-case ones. AI-assisted

Krzmbrzl added 22 commits October 28, 2026 15:55

Share the product adjoint checks and split adjoint scopes into sections

0a23997

AI-assisted

Drive iroot checks from a table

39a1819

AI-assisted

Drive AsyCost text and LaTeX checks from tables

eb9ff6a

AI-assisted

Drive binary node visit checks from helpers and tables

a544f14

AI-assisted

Loop the variable node checks over product and sum

b0a4ea9

AI-assisted

Drive fermionic can_contract checks from a table

d832cbe

AI-assisted

Drive density-fit and hypercontraction rule checks from tables

6e2961d

Capture the wide-string input through toUtf8 so failures print readably. AI-assisted

Drive biorthogonalization checks from tables

02563c0

Capture wide-string inputs through toUtf8. AI-assisted

Print wide-string test inputs through toUtf8

a4ff981

Catch2 cannot render std::wstring (or vectors of them) readably. AI-assisted

Keep a custom comparer in the named-parameter Context check

98e2d5a

AI-assisted

Drive ITF K/J remap checks from a table

64e176c

AI-assisted

Unhide passing and drop failing hidden tests in test_eval_dryrun

d4a7987

Hidden test cases that pass now run by default; ones that failed or were skipped (blocked on removed or unfinished functionality) are removed along with the helpers and comments that only they used. AI-assisted

Unhide passing and drop failing hidden tests in test_eval_ta

6ef9ba3

Hidden test cases that pass now run by default; ones that failed or were skipped (blocked on removed or unfinished functionality) are removed along with the helpers and comments that only they used. AI-assisted

Unhide passing and drop failing hidden tests in test_legality

295a781

Hidden test cases that pass now run by default; ones that failed or were skipped (blocked on removed or unfinished functionality) are removed along with the helpers and comments that only they used. AI-assisted

Unhide passing and drop failing hidden tests in test_optimize

4570652

Hidden test cases that pass now run by default; ones that failed or were skipped (blocked on removed or unfinished functionality) are removed along with the helpers and comments that only they used. AI-assisted

Unhide passing and drop failing hidden tests in test_ordered_executor

f4da1f9

Hidden test cases that pass now run by default; ones that failed or were skipped (blocked on removed or unfinished functionality) are removed along with the helpers and comments that only they used. AI-assisted

Unhide passing and drop failing hidden tests in test_peak_profile

7249f84

Hidden test cases that pass now run by default; ones that failed or were skipped (blocked on removed or unfinished functionality) are removed along with the helpers and comments that only they used. AI-assisted

Update schedule-view README for the unhidden dry-run report test

4cba90f

AI-assisted

Skip env-driven generator tests when their variables are unset

e5eca86

AI-assisted

Drop stale RED-gate comments from the free-hoist charge test

8bbeb9e

The test passes; its comments still described the failing state. AI-assisted

Krzmbrzl force-pushed the test-overhaul branch from 46d168d to f0893d8 Compare October 28, 2026 14:55

Add more [[maybe_unused]] attributes

7dd07dd

evaleev added this to the 26.2 milestone Oct 28, 2026

Krzmbrzl merged commit 8d12735 into ValeevGroup:master Oct 29, 2026
10 checks passed

evaleev added a commit that referenced this pull request Oct 30, 2026

Rename branch test cases to the kebab-case convention

92d539d

Master's test overhaul (#743) gave test cases terse kebab-case names; give the test cases this branch added the same style. Tags are unchanged.

evaleev added a commit that referenced this pull request Nov 2, 2026

Rename branch test cases to the kebab-case convention

4a5e7c7

Master's test overhaul (#743) gave test cases terse kebab-case names; give the test cases this branch added the same style. Tags are unchanged.

Navigation Menu

Repository navigation

Overhaul test setup - #743

Overhaul test setup#743
Krzmbrzl merged 65 commits into
ValeevGroup:masterfrom
Krzmbrzl:test-overhaul

Krzmbrzl commented Oct 28, 2026

Uh oh!

ajay-mk commented Oct 28, 2026

Uh oh!

Krzmbrzl commented Oct 28, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Krzmbrzl commented Oct 28, 2026

Uh oh!

ajay-mk commented Oct 28, 2026

Uh oh!

Krzmbrzl commented Oct 28, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants