Repository navigation
BUG: Fix #1908 wrong-data items L73, L74, L77, L78 and L79 - #1981
imikejackson wants to merge 10 commits into
Conversation
Remove out-of-plane kernel offsets on single-slice images before the empty-kernel checks. Previously, an Annulus with only Z offsets bypassed those checks and reduced no image values. Preserve each operation's existing empty-kernel result and leave 3D kernels unchanged. Add independent copy-through and zero-output tests for all seven grayscale filters on both algorithm paths, and for the bounded composite pipeline. Part of BlueQuartzSoftware#1908 (item L73) Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Legacy DREAM3D shift files omit the top section's zero row. Importing the 116 legacy rows as cumulative shifts for 117 sections skips the first row and applies the remaining shifts to the wrong sections. Replace the align_sections_input.tar.gz registration with align_sections_input_v2.tar.gz, which prepends the missing zero row. The v1 archive is retired from this registration and remains immutable. Update both example pipelines to use v2, import all 117 rows, document the row-to-section mapping, and test generated legacy and corrected files. Part of BlueQuartzSoftware#1908 (item L78) Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
The seed triangle moves to the origin and contributes a zero row to the least-squares system. Small patches could therefore fit three or seven coefficients with too few usable neighbor rows and return arbitrary curvature. Count finite, nonzero non-seed rows for the selected fit and reuse the existing invalid-patch NaN outputs when there are too few. Add analytic paraboloid and minimum-size regressions and document the patch requirements. Part of BlueQuartzSoftware#1908 (item L74) Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Reference-slice mode stored reference-relative cumulative offsets in both shift arrays. Align Sections (List) accumulated those offsets again when reading Relative input, producing incorrect alignment from row 2 onward. Store differences of consecutive integer cumulative shifts in both Direct and Scanline paths. Preserve cumulative shifts and non-reference behavior. Add a literal shift oracle and List round-trip regression, and clarify the Relative and Cumulative Shifts documentation. The existing output test now derives Relative Shifts from the exemplar's Cumulative Shifts, preserving the original archive and the other exact exemplar comparisons. Part of BlueQuartzSoftware#1908 (item L77) Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
ReadImage compared decoded segment byte counts with page 0. Different component counts and sample types with the same byte width could pass and be copied using page 0's layout, producing incorrect pixel values. Add IImageIO::readPageLayouts returning only small page metadata structs. The TIFF implementation opens the file once, collects logical page directories once and reads their tags with one forward traversal. ValidatePageLayouts compares those layouts in memory, preserving TIFF's unsigned sample-format default and the existing byte-count and dimension checks. ReadImageFilter rejects mismatched layouts in preflight with error -50579. The regression cases assert this preflight error explicitly. Keep the execute check because ImportFijiMontageFilter calls ReadImage directly without ReadImageFilter preflight; removing that guard requires addressing this caller. Add generated TIFF regression cases and document the page layout requirement. Part of BlueQuartzSoftware#1908 (item L79) Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Read Image Stack reads only page 0 of each file. The page-layout check compared every page of a multi-page TIFF, so a stack file whose unread pages had a different layout failed with -50579, although only page 0 is decoded. Pass the number of pages the read uses to ValidatePageLayouts in preflight and execute, and add a single-page-read regression test. Related to BlueQuartzSoftware#1908 item L79 Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
FeatureFaceCurvature queued one worker per feature face even when the workers accessed shared out-of-core stores. These stores do not support concurrent access, so separate triangle ranges could still race and lose output values. Use ParallelTaskAlgorithm's residency gate with all input and output arrays in one requireArraysInMemory call. Run workers synchronously when any store is out-of-core and retain the existing TBB task queue when all stores are resident. Add multi-feature analytic plane tests with real disk-backed store witnesses and a deterministic queue-decision regression. Remove the filter from the knowledge base's ungated list. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
The page-layout check compared every page of a multi-page TIFF with page 0, so a Z-cropped read failed with -50579 when a page outside the crop had a different layout, although that page is never decoded. ValidatePageLayouts now takes the first page and the number of pages that the read decodes and compares only those pages with page 0, which sets the output array layout. Preflight checks after the crop is resolved, and execute checks with the same range before any output is written. Document exactly which pages are checked and add Z-crop regression cases. Related to BlueQuartzSoftware#1908 item L79 Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
- Add 2D out-of-plane kernel tests and doc notes for Opening and Closing by Reconstruction (Preserve Intensities on), which the shared MorphologyEngine change also fixes. - Add a 2D-kernel note to every grayscale morphology filter page. - Reuse the in-plane kernel helper in the binary composite pipeline. - Replace the raw TIFF sample-format code in PageMetadata with a format-neutral SampleFormat enum. - Add direct readPageLayouts tests, a test that reaches the execute-time page-layout check, and a curvature test that detects lost writes. - Fix doc wording, test tags and comments; drop unneeded LoadPlugins(). Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
The page-layout check rejects a TIFF whose pages differ in component count or data type in preflight with -50579, before the per-page byte count check can return -2007 in execute. Update the mixed RGB/grayscale test to expect -50579 from preflight and execute with no output array, and describe both errors on the Read Image page. Part of BlueQuartzSoftware#1908 (item L79) Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
| } | ||
|
|
||
| // Translation makes the seed row zero. Each coefficient needs a usable neighbor row. | ||
| if(usableRows < cols) |
There was a problem hiding this comment.
[Important] The L74 solvability check counts nonzero rows, but the fit needs full rank
usableRows counts neighbor rows that are finite and not all zero. Enough such rows does not make A full rank.
- Normals off (3 columns): neighbors collinear with the seed, for example (1,0), (2,0), (3,0), give rows (0.5x², 0, 0).
usableRowsis 3 but the rank is 1. - Normals on (7 columns): the seed lies on one line of centroids and the other neighbors on a parallel line
y = b(a one-triangle-wide strip, which voxel meshes produce). Theny(y−b)(cx+dy)is a nonzero combination of {xy, y², xy², y³} that is zero at every point, so the rank is at most 6 even with 7 or more usable rows.
colPivHouseholderQr().solve(b) then returns one basic solution from a null space, so the cross-strip curvature on a curved strip is arbitrary. That is the defect L74 says it fixes. The Patch solvability test has no rank-deficient patch with enough rows.
Fix: decide on the rank:
const Eigen::ColPivHouseholderQR<Eigen::MatrixXd> qr(A);
if(usableRows < cols || qr.rank() < static_cast<Eigen::Index>(cols))
{
markOutputsInvalid(triId);
continue;
}
// sln1 = qr.solve(b);Add collinear (normals off) and two-parallel-line (normals on) cases that expect NaN, and update the paragraph in FeatureFaceCurvatureFilter.md.
| { | ||
| return MakeErrorResult<std::vector<PageMetadata>>(k_ErrorReadMetadataFailed, fmt::format("Cannot read dimensions of page {} of TIFF '{}': {}", page, pathStr, file.errorMessage())); | ||
| } | ||
| TIFFGetFieldDefaulted(tiff, TIFFTAG_SAMPLESPERPIXEL, &components); |
There was a problem hiding this comment.
[Minor] readPageLayouts() accepts tag defaults that readMetadata() rejects
readMetadata() requires an explicit SamplesPerPixel tag (to avoid treating malformed multichannel data as grayscale) and gets the type from DetermineTiffDataType(). readPageLayouts() uses TIFFGetFieldDefaulted for SamplesPerPixel, BitsPerSample and SampleFormat. A later page with no SamplesPerPixel tag is reported as 1 component, passes -50579 against a 1-component page 0, and is decoded with the layout of page 0.
Fix: apply the same rules to each page. Require the tag and return k_ErrorReadMetadataFailed with the page index when it is missing. Get the type with DetermineTiffDataType(tiff), then derive bits and SampleFormat from it, or store the DataType in PageMetadata.
| stream.dstWidth, stream.dstHeight, stream.dstDepth, stream.srcWidth, stream.srcHeight, metadata.numPages)); | ||
| } | ||
| // Only the decoded pages are validated. Page 0 sets the output layout, so each decoded page is compared with it. | ||
| auto layoutResult = ReadImage::ValidatePageLayouts(*imageIO, inputFilePath, stream.singlePageOnly ? usize{0} : stream.zStart, stream.dstDepth); |
There was a problem hiding this comment.
[Minor] Execute repeats the preflight page-layout check, and a test calls the algorithm directly to reach it
This breaks the KB entry Run a filter, not its algorithm. preflightImpl() already runs ValidatePageLayouts(), and ReadRasterBackend runs it again. The reason given is ImportFijiMontageFilter's direct construction, but the entry says that caller must establish the precondition and name it at the call. ReadImageTest.cpp:1847 calls ReadImage::operator()() to reach an input check, which the entry's "What to look for" list names.
Fix (choose one): (a) Move the check to where ImportFijiMontageFilter prepares each tile, comment the precondition at the call, and test it through the filter. (b) Keep it as the read-time check that the entry allows (a file can change after preflight), say so in the comment, and make the test change the file between preflight and execute.
| - Slices (DataArray | 2 component) - The slice indices (stored as uint32s) | ||
| - Relative Shifts (DataArray | 2 component) - The slices shift relative to previous shift (stored as int64s) [*previously known as `newxshift` and `newyshift`*] | ||
| - Cumulative Shifts (DataArray | 2 component) - The slice's accumulated shift (stored as int64s) | ||
| - Relative Shifts (DataArray | 2 component) - The slice's shift relative to the slice above it (the slice with the next higher Z index), in both modes (stored as int64s). The first row has no slice above it, so its relative shift equals its cumulative shift. [*previously known as `newxshift` and `newyshift`*] |
There was a problem hiding this comment.
[Minor] Nonzero reference slice: Relative Shifts do not round-trip, and neither the docs nor the tests say so
With a nonzero Reference Slice, row 0 holds the top section's shift c₀, and AlignSectionsList::findShifts never reads row 0. With Relative input, lower slices get c_k − c₀ and the top slice gets 0. With Cumulative input, lower slices get c_k and the top slice gets 0. Neither reproduces Feature Centroid, and the two disagree. The new round-trip test uses only Reference Slice 0.
Fix: document on both pages what List does with row 0 for a nonzero reference. Add a round-trip case with a nonzero reference that asserts the documented outcome. The List limitation itself is pre-existing wrong data (report item 2.1).
| TEST_CASE("SimplnxCore::FeatureFaceCurvatureFilter: multi-feature planes", "[SimplnxCore][FeatureFaceCurvatureFilter]") | ||
| { | ||
| // Storage-aware array creation and the storage preferences need the application that LoadPlugins() creates. | ||
| UnitTest::LoadPlugins(); |
There was a problem hiding this comment.
[Minor] The in-core curvature test loads every plugin but only needs the Application
This breaks the KB entry Call UnitTest::LoadPlugins() only when the test looks up filters by UUID. The in-core multi-feature planes test constructs the algorithm directly and needs only the Application for the preferences sentinel. Use Application::GetOrCreateInstance(), as the entry prescribes, so the test does not fail when an unrelated plugin does not load. Keep LoadPlugins() in the real-OOC case, which needs the HDF5-OOC registration.
| per-feature output stores without a gate (PR #1645 made it serial). `ComputeFeatureFaceMisorientation` was fixed in | ||
| PR #1618. `CombineStlFiles` (issue #1876), `ConvertOrientations`, `PointSampleEdgeGeometry` and `FeatureFaceCurvature` | ||
| still launch workers that write shared stores without a gate. | ||
| PR #1618, and `FeatureFaceCurvature` in the change that fixed issue #1908 items L73 to L79. `CombineStlFiles` (issue |
There was a problem hiding this comment.
| * | ||
| * On a single-slice image, the Z offsets of the kernel are ignored, so this kernel is empty and the filter copies | ||
| * the input. The oracle is the input itself, and the same filter with the empty Annulus {0, 0, 0} must give the same | ||
| * values. Before #1908 L73, Preserve Intensities on returned INT32_MIN (opening) or INT32_MAX (closing) everywhere. |
There was a problem hiding this comment.
[Nit] Comments and tags narrate history
cpp-code-documentation says to describe the current code, without task numbers. Replace "Before #1908 L73, Preserve Intensities on returned INT32_MIN (opening) or INT32_MAX (closing) everywhere." with "With Preserve Intensities on, an empty kernel must not produce the INT32_MIN or INT32_MAX fill value." Also replace the [L73] tags in MorphologyEngineTest.cpp and the "(#1908 L73)" lines in the Reconstruction tests with behavior descriptions.
| std::error_code errorCode; | ||
| fs::remove(path, errorCode); | ||
| const auto result = imageIO.readPageLayouts(path); | ||
| REQUIRE(result.invalid()); |
There was a problem hiding this comment.
[Nit] The new readPageLayouts failure tests are loose
Use SIMPLNX_RESULT_REQUIRE_INVALID instead of REQUIRE(result.invalid()) (KB entry Assert a valid Result with SIMPLNX_RESULT_REQUIRE_VALID). Assert the code in the "not a TIFF" section. The two new k_ErrorReadMetadataFailed branches ("Cannot select page", "Cannot read dimensions") have no test that reaches them, as cpp-best-practices requires; add one where practical.
joeykleingers
left a comment
There was a problem hiding this comment.
This is the summary of my review of 2026-11-16. GitHub removed it when the review was submitted, so I post it here.
PR 1981 Code Review
Source: review of commit
2fa764d(#1981, "BUG: Fix #1908 wrong-data items L73, L74, L77, L78 and L79" by imikejackson,topic/issue1908_batch4→develop@8cad1e8). Coverage: all 36 changed files (C++, tests, filter docs, pipelines, CMake, knowledge base).
1. Findings
-
1.1 [Important] The L74 solvability check counts nonzero rows, but the fit needs full rank
File:src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CalculateTriangleGroupCurvatures.cpp:336-347(CalculateTriangleGroupCurvatures::operator())
CONFIRMED (logic), PLAUSIBLE (how often real meshes hit it). Found by the correctness and independent-evidence checks.
usableRowscounts neighbor rows that are finite and not all zero. Enough such rows does not makeAfull rank:- Normals off (3 columns): neighbors whose centroids are collinear with the seed in the local frame, for example (1,0), (2,0), (3,0). The rows are (0.5x², 0, 0), so
usableRowsis 3 but the rank is 1. - Normals on (7 columns): the seed lies on one line of centroids and the other neighbors lie on a parallel line
y = b. This is a one-triangle-wide strip, which voxel surface meshes produce. Theny(y−b)(cx+dy)is a nonzero combination of the monomials {xy, y², xy², y³} that is zero at every point, so the rank is at most 6 even when 7 or more rows count as usable.
In both cases
colPivHouseholderQr().solve(b)returns one basic solution from a null space. On a curved strip, the cross-strip curvature terms (xy,y²) are therefore arbitrary. That is the wrong-data behavior that L74 says it fixes. The newPatch solvabilitytest only exercises row counts (including duplicate seed rows) and has no rank-deficient patch with enough rows.
Fix: decide on the rank, using the Eigen QR that the code already runs:const Eigen::ColPivHouseholderQR<Eigen::MatrixXd> qr(A); if(usableRows < cols || qr.rank() < static_cast<Eigen::Index>(cols)) { markOutputsInvalid(triId); continue; } // ... sln1 = qr.solve(b);
This also avoids factoring
Atwice. AddPatch solvabilitycases with collinear neighbors (normals off) and two parallel centroid lines (normals on), and expect NaN. Update theFeatureFaceCurvatureFilter.mdparagraph: the fit also needs neighbors that determine every coefficient, not only enough usable rows. - Normals off (3 columns): neighbors whose centroids are collinear with the seed in the local frame, for example (1,0), (2,0), (3,0). The rows are (0.5x², 0, 0), so
-
1.2 [Minor]
readPageLayouts()accepts tag defaults thatreadMetadata()rejects
File:src/simplnx/Utilities/ImageIO/TiffImageIO.cpp:430-432(TiffImageIO::readPageLayouts)
readMetadata()requires an explicitSamplesPerPixeltag ("avoids treating malformed multichannel data as grayscale"), and it gets the data type fromDetermineTiffDataType().readPageLayouts()usesTIFFGetFieldDefaultedforSamplesPerPixel,BitsPerSampleandSampleFormat. Example: page 0 is a valid 1-component uint16 page and a later page has noSamplesPerPixeltag. That page is reported as 1 component, passes the -50579 check, and is decoded with the layout of page 0, which is the malformed-multichannel case the backend says it refuses. The two functions also describe the sample type in different ways: raw bits plus format here, theDataTypethat the decoder uses there.
Fix: for each page, apply the same rules asreadMetadata(). Require theSamplesPerPixeltag and returnk_ErrorReadMetadataFailedwith the page index when it is missing. Get the type fromDetermineTiffDataType(tiff), then derivebitsPerSampleandSampleFormatfrom thatDataType, or store theDataTypeinPageMetadata. Then the check compares exactly what the decoder will use. -
1.3 [Minor] Execute repeats the preflight page-layout check, and a test calls the algorithm directly to reach it
File:src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp:317andsrc/Plugins/NXImageProcessing/test/ReadImageTest.cpp:1847
This breaks the knowledge base entry Run a filter, not its algorithm.ReadImageFilter::preflightImpl()already runsValidatePageLayouts(), andReadRasterBackendruns it again in every execute. The reason given isImportFijiMontageFilter, which constructsReadImagedirectly. Under that entry, the direct caller is the one that must establish the precondition, with a comment at the call site. The testReadImage: TIFF_PageLayout rejected in executecallsReadImage::operator()()to reach an input check, which the entry's "What to look for" list names.
Fix (choose one): (a) Remove the execute call. RunValidatePageLayouts()whereImportFijiMontageFilterprepares each tile, name the precondition at theReadImage(...)call, and test it through the filter. (b) Keep it as the read-time file check that the entry allows ("a file can change after preflight"). Then say so in the comment atReadImage.cpp:316, and make the test change the file betweenpreflight()andexecute()instead of skipping the filter. -
1.4 [Minor] Stale comment says page layouts cannot be detected
File:src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp:355-357
The comment says "IImageIO exposes metadata only for page 0, so these checks cannot detect different component counts or scalar types with the same byte width." This PR addsIImageIO::readPageLayouts(), andValidatePageLayouts()now detects exactly that. Replace it with:// Destination offsets use the dimensions and pixel stride of page 0. ValidatePageLayouts() already compared the // component count and sample type of each decoded page with page 0. Here, check the segment byte counts and the // observed dimensions of each page.
-
1.5 [Minor] Align Sections (List): the "Relative" definition contradicts the new row-order text
File:src/Plugins/SimplnxCore/docs/AlignSectionsListFilter.md:23
Line 23 still says "the offset of section N relative to section N-1". The new text (lines 32 and 65) and the code (AlignSectionsList::findShifts, rows ordered from top to bottom, accumulated from row 1) define the relative shift of a section against the section above it (N+1). A user who builds a relative file from line 23 gets every shift paired with the wrong neighbor. Replace with: "Relative [0]: Each row gives the shift of its section relative to the section above it (the next higher Z index). The filter adds these shifts, starting from the top section, to get cumulative shifts before it applies them." -
1.6 [Minor] Nonzero reference slice: Relative Shifts do not round-trip, and neither the docs nor the tests say so
File:src/Plugins/SimplnxCore/docs/AlignSectionsFeatureCentroidFilter.md:65(and the new round-trip test inAlignSectionsFeatureCentroidTest.cpp)
With Use Reference Slice on and Reference Slice ≠ 0, row 0 holds the nonzero shift c₀ of the top section.AlignSectionsList::findShiftsnever reads row 0. With Relative input, each lower slice gets c_k − c₀ and the top slice gets 0. With Cumulative input, each lower slice gets c_k and the top slice gets 0. Neither result reproduces the Feature Centroid output, and the two inputs disagree with each other. The new round-trip test uses onlyReferenceSlice = 0, where c₀ = 0. The PR description says relative input now gives the same result, without this limit.
Fix: state on both pages what Align Sections (List) does with row 0 when the reference slice is not 0. Add a round-trip case with a nonzero reference slice that asserts the documented outcome. Ask the reviewer where the pre-existing List limitation should be fixed (see 2.1). -
1.7 [Minor] The in-core curvature test loads every plugin but only needs the Application
File:src/Plugins/SimplnxCore/test/FeatureFaceCurvatureTest.cpp:1184(multi-feature planes)
This breaks the knowledge base entry Call UnitTest::LoadPlugins() only when the test looks up filters by UUID. The test constructs the algorithm directly and usesForceInCore. Its comment says it needs the Application for the preferences sentinel and for storage-aware creation, and the entry's prescribed call for that isApplication::GetOrCreateInstance(). As written, the test fails when an unrelated plugin library does not load. KeepLoadPlugins()in thereal OOC storescase, which needs the HDF5-OOC registration. -
1.8 [Nit] The knowledge base edit cites "the change" instead of the PR, and names items the PR does not fix
File:docs/Coding_Knowledge_Base.md:1104
The file's own rules say to cite pull requests by number. "In the change that fixed issue #1908 items L73 to L79" also suggests that L75 and L76 are included. Replace with: "ComputeFeatureFaceMisorientationwas fixed in PR #1618 andFeatureFaceCurvaturein PR #1981." -
1.9 [Nit] Comments and tags narrate history
Files:src/Plugins/NXImageProcessing/test/ReconstructionFilterTestUtils.hpp:521; the[L73]tags inMorphologyEngineTest.cpp; the "(#1908 L73)" lines inOpening/ClosingByReconstructionImageFilterTest.cpp
cpp-code-documentationsays to describe the current code, without task numbers. For example, replace "Before #1908 L73, Preserve Intensities on returned INT32_MIN (opening) or INT32_MAX (closing) everywhere." with "With Preserve Intensities on, an empty kernel must not produce the INT32_MIN or INT32_MAX fill value." Use a behavior tag such as[MorphologyEngine]instead of[L73]. -
1.10 [Nit] The new
readPageLayoutsfailure tests are loose
File:test/TiffImageIOTest.cpp:555,567
They useREQUIRE(result.invalid()), but the knowledge base entry Assert a valid Result with SIMPLNX_RESULT_REQUIRE_VALID asks forSIMPLNX_RESULT_REQUIRE_INVALID. The "not a TIFF" section asserts no code. The two newk_ErrorReadMetadataFailedbranches ("Cannot select page", "Cannot read dimensions") have no test that reaches them, althoughcpp-best-practicesasks for a test for each new failure branch. Use the macro, assert -20100 or -20101 and the distinct message phrase, and add a case for a truncated IFD chain if one is practical. -
1.11 [Nit] New tests call
preflight()beforeexecute()
Files:ReconstructionFilterTestUtils.hpp(RequireZOnlyAnnulus2DPassthrough),MorphologyEngineTest.cpp(Grayscale morphology: 2D out-of-plane Annulus passthrough),FeatureFaceCurvatureTest.cpp(Patch solvability)
Run a filter, not its algorithm:execute()already runs preflight. Remove the separatepreflight()calls unless the test asserts something on the preflight result.
2. Pre-existing defects surfaced by this PR
- 2.1 Wrong data: yes.
AlignSectionsList::findShifts(AlignSectionsList.cpp:49-64) never reads row 0. Input: the Cumulative Shifts of Align Sections (Feature Centroid) with a nonzero Reference Slice. Effect: the top section stays unshifted while every other section moves by its cumulative shift, so the top section is misaligned by c₀ relative to the rest, and there is no error. This is not introduced here. The reviewer decides where to fix it (see 1.6).
3. Documentation
Covered in 1.4, 1.5, 1.6, 1.8 and 1.9. The nine morphology pages, the Read Image page (-50579 before the read, -2007 for dimensions, cropped and single-page scope) and the Feature Face Curvature page match the code. The Feature Face Curvature page will need the rank wording from 1.1.
4. Missing or failed evidence
- 4.1 clang-format 19.1.1: every changed C/C++ file is formatted. clang-tidy was not run (no build available), and no tests were run in this review. The pass results in the PR description (in-core, and out-of-core with the private companion PR) are the author's reports, not observations from this review.
- 4.2 I could not inspect
align_sections_input_v2.tar.gzor its SHA-512. WhetherAlignment_By_Misorientation_Shifts.txthas 117 rows with116 117 0 0 0 0as the first data row, and whetherexample_relative_shifts.txtis unchanged, is unverified. The pipeline tests are the only evidence. - 4.3
multi-feature planes with real OOC storesreturns with a WARN in builds without HDF5-OOC. In the public in-core CI, the out-of-core gate is therefore checked only by the companion out-of-core build. - 4.4 "CI passes on all platforms" is unchecked in the PR test plan.
5. Confirmed correct
- 5.1 L73:
detail::MakeInPlaneMorphologyKernel()returnsnulloptunless Z = 1 and the kernel has an offset with dz ≠ 0, so each redirect recurses only once. It is applied inMorphScanline,MorphDirect,MorphGradientScanline,MorphGradientDirectand both composite pipelines, after the original layout validation. The binary composite route now shares the same helper with no change in behavior. The tests compare with the input or with zero (an independent oracle) across both dispatch scenarios. - 5.2 L77: the in-core and out-of-core paths compute the same slice-to-slice differences. Row 0 is written only for a nonzero reference. The hand-checked expected values ({1,−1,−1,1}; {0,0,2,1,−1,2,2,−1}) and the empty-top case [0,0,−1,1] agree with the cumulative shifts. Negative differences that pass through the
usizeintermediates in the in-core path convert back exactly under C++20. - 5.3 Feature Face Curvature gate:
requireArraysInMemory()lists every store the workers touch (all 7 outputs, the 4 inputs, feature-face ids, vertices, faces).CheckArraysInMemoryskips thenullptroptional outputs. With a non-resident store the workers run inline. The test shows disk-backedHDF5-OOCstores and that the work-queue message is absent, and the sentinel-filled outputs catch lost writes. This satisfies Write out-of-core stores from one thread. - 5.4 L79:
ValidatePageLayouts()compares page 0 with exactly the decoded range:[max(first,1), min(size, first+count)). A read of page 0 alone short-circuits, and preflight and execute derive the same first page for voxel and physical Z crops. -50579 is unique in the repository, and the message names the file, the page and both layouts.readPageLayouts()usesCollectPageDirectories(), so pyramid and mask subfiles map the same way as inreadPixelDataRows(), and a test covers this. The check reads file metadata, not a placeholder count, so running it in preflight is allowed. - 5.5 L78: the new
_v2archive follows the immutable-archive rule. No reference to the old directory remains. The legacy-file test builds its own file and checks hand-computed mappings with and without the zero row.
6. Verdict
REQUEST CHANGES: 1.1 is open. The L74 check accepts rank-deficient patches and still writes arbitrary curvatures on collinear or strip-shaped patches, which is the defect class this PR says it fixes. The other items are Minor or Nit.
Checks and lenses with no further findings: CPU, memory, naming, const-correctness, error-message wording, placeholder-count check, test-data immutability.
Summary
Fixes five wrong-data defects from the #1908 sweep (items L73, L74, L77, L78 and L79), and a related out-of-core race in Feature Face Curvature. Each fix has regression tests with independent expected values and updates the filter documentation that states the changed behavior. 36 files changed, +1507 / −70 lines.
Companion PR: BlueQuartzSoftware/SimplnxOoc#44 (private repository). It updates one out-of-core test that expects the old reference-slice Relative Shifts (L77). Merge the two together; without it, that one SimplnxOoc test fails in the out-of-core build.
Test data: the L78 fix uses a new data archive,
align_sections_input_v2.tar.gz, which is already published on theData_Archiverelease. The originalalign_sections_input.tar.gzis unchanged and is no longer referenced.1. Grayscale morphology on 2D images (L73)
src/simplnx/Utilities/ImageProcessing/MorphologyEngine.hpp:detail::MakeInPlaneMorphologyKernel()src/Plugins/NXImageProcessing/docs/*ImageFilter.md: 9 morphology pagesOn an image with Z = 1, an out-of-plane kernel (for example an Annulus around Z) used to read outside the slice and write wrong values. The engine now drops every kernel offset with dz ≠ 0 when Z = 1. If no offset is left, the existing empty-kernel behavior applies: Dilate, Erode, Opening and Closing copy the input, and Morphological Gradient and the top-hats output zero. The fix applies to every grayscale route (Direct, Scanline, gradient and the composite pipelines). It therefore also fixes Opening and Closing by Reconstruction with Preserve Intensities on, which used to output INT32_MIN or INT32_MAX in this case. All 9 filter pages describe the 2D behavior.
2. Legacy align-sections shift file (L78)
src/Plugins/SimplnxCore/CMakeLists.txt: downloadsalign_sections_input_v2.tar.gzsrc/Plugins/SimplnxCore/pipelines/Align_Sections_Examples/*.d3dpipelinesrc/Plugins/SimplnxCore/docs/AlignSectionsListFilter.mdThe 6.5.171 shift file
Alignment_By_Misorientation_Shifts.txthas 116 rows for 117 sections. It is missing the zero row of the top section, so Align Sections (List) applied every shift to the wrong section. The v2 archive adds the row116 117 0 0 0 0, and the example pipelines read 117 rows from it. The documentation says that row 0 is the top section, that it must be a zero row for either input type, and that rows go from the top section to the bottom section. The unit test writes its own legacy file in code and does not use the archive.3. Feature Face Curvature: too few triangles for the fit (L74)
src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CalculateTriangleGroupCurvatures.cppThe seed triangle moves to the origin, so its row in the least-squares system is zero. A small patch could therefore fit 7 (with face normals) or 3 coefficients from fewer usable rows and return arbitrary curvatures. The algorithm now counts the usable neighbor rows (finite, not all zero). When there are fewer rows than coefficients, all requested outputs of that triangle are NaN, the same as for other invalid patches.
4. Feature Face Curvature: out-of-core race
src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/FeatureFaceCurvature.cppdocs/Coding_Knowledge_Base.mdThe algorithm ran one TBB task per feature face, and the tasks read and wrote shared stores. Out-of-core stores do not support concurrent writes, so values could be lost. A
ParallelTaskAlgorithmnow callsrequireArraysInMemory()with all 13 arrays that the workers use. The tasks run in parallel only when all of these arrays are in memory; otherwise they run one after another. The knowledge base no longer lists Feature Face Curvature as a filter without this gate.5. Align Sections (Feature Centroid): relative shifts in reference-slice mode (L77)
src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/AlignSectionsFeatureCentroid.cppsrc/Plugins/SimplnxCore/docs/AlignSectionsFeatureCentroidFilter.mdWith Use Reference Slice on, the Relative Shifts array held the shift relative to the reference slice, which is the same as the Cumulative Shifts. Read back into Align Sections (List) as relative input, these shifts were added up a second time. Relative Shifts now hold the slice-to-slice difference of the cumulative shifts in both the in-core and the out-of-core path. The first row has no slice above it, so its relative shift equals its cumulative shift. Cumulative Shifts and the consecutive-slice mode do not change. The test computes the expected relative shifts from the existing exemplar's cumulative shifts, so the exemplar archive does not change.
6. Read Image: multi-page TIFFs with different page layouts (L79)
src/simplnx/Utilities/ImageIO/IImageIO.hpp,TiffImageIO.hpp/.cpp: newIImageIO::readPageLayouts()andPageMetadatasrc/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp:ReadImage::ValidatePageLayouts()src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/ReadImageFilter.cppThe reader took the component count and data type from page 0 and read every page with that layout. A page with a different layout gave wrong values.
readPageLayouts()reads the size, component count, bits per sample and a format-neutralSampleFormatof every page in one pass. The default implementation inIImageIOreturns one page.ValidatePageLayouts()compares only the pages that the read uses (page 0 plus the pages inside a Z crop; one page for a single-page read) and returns error-50579, which names the page and both layouts. Preflight runs the check. Execute runs it again, which protects callers that use the algorithm directly for multi-page reads without the filter's preflight, such as Import Fiji Montage.The page-layout check runs before the per-page
-2007checks from #1915. A file with an 8-bit RGB page followed by an 8-bit grayscale page now fails in preflight with-50579, not in execute with-2007. The existing testMultiPage_TIFF_RGB_Then_Grayscale_Rejected_Before_Copyexpects this, and the Read Image page describes both errors. Pages with different dimensions still fail with-2007while the filter reads them.Test plan
NXImageProcessing::MorphologyEngine2D out-of-plane kernel tests (L73), and the 2D passthrough tests of Opening and Closing by Reconstruction.SimplnxCore::AlignSectionsListFilter: Legacy cumulative file requires zero row(L78).SimplnxCore::FeatureFaceCurvatureFilter: Patch solvability(L74): analytic paraboloid patches at and below the minimum size.SimplnxCore::FeatureFaceCurvatureFilter: multi-feature planesand... with real OOC stores: outputs are pre-filled with a sentinel, so a lost write fails.SimplnxCore::AlignSectionsFeatureCentroidFilter: Reference slice relative shifts round trip(L77): literal expected shifts, and Align Sections (List) gives the same result from relative and cumulative input.NXImageProcessing::ReadImageFilter: TIFF_PageLayout,TIFF_PageLayout_SinglePageOnly,TIFF_PageLayout_ZCrop,NXImageProcessing::ReadImage: TIFF_PageLayout rejected in execute, and theTiffImageIO:: readPageLayouts ...tests (L79).Nonzero reference top slice,Empty slices) check the new slice-to-slice Relative Shifts.MorphologyEngine|Grayscale|MorphologicalGradient|TopHat|Reconstruction|AlignSections|FeatureFaceCurvature|ReadImage|ImageIO|Tiff|ImportFijiMontage) and the Small IN100 and align-sections pipeline tests pass, run one at a time.ooclabel (1053 tests) pass.Related Issue
Related to #1908
Related to #1815