Skip to content

BUG: Fix #1908 wrong-data items L73, L74, L77, L78 and L79 - #1981

Open
imikejackson wants to merge 10 commits into
BlueQuartzSoftware:developfrom
imikejackson:topic/issue1908_batch4
Open

imikejackson wants to merge 10 commits into
BlueQuartzSoftware:developfrom
imikejackson:topic/issue1908_batch4

Conversation

@imikejackson

@imikejackson imikejackson commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes five wrong-data defects from the #1908 sweep (items L73, L74, L77, L78 and L79), and a related out-of-core race in Feature Face Curvature. Each fix has regression tests with independent expected values and updates the filter documentation that states the changed behavior. 36 files changed, +1507 / −70 lines.

Companion PR: BlueQuartzSoftware/SimplnxOoc#44 (private repository). It updates one out-of-core test that expects the old reference-slice Relative Shifts (L77). Merge the two together; without it, that one SimplnxOoc test fails in the out-of-core build.

Test data: the L78 fix uses a new data archive, align_sections_input_v2.tar.gz, which is already published on the Data_Archive release. The original align_sections_input.tar.gz is unchanged and is no longer referenced.

1. Grayscale morphology on 2D images (L73)

  • src/simplnx/Utilities/ImageProcessing/MorphologyEngine.hpp: detail::MakeInPlaneMorphologyKernel()
  • src/Plugins/NXImageProcessing/docs/*ImageFilter.md: 9 morphology pages

On an image with Z = 1, an out-of-plane kernel (for example an Annulus around Z) used to read outside the slice and write wrong values. The engine now drops every kernel offset with dz ≠ 0 when Z = 1. If no offset is left, the existing empty-kernel behavior applies: Dilate, Erode, Opening and Closing copy the input, and Morphological Gradient and the top-hats output zero. The fix applies to every grayscale route (Direct, Scanline, gradient and the composite pipelines). It therefore also fixes Opening and Closing by Reconstruction with Preserve Intensities on, which used to output INT32_MIN or INT32_MAX in this case. All 9 filter pages describe the 2D behavior.

2. Legacy align-sections shift file (L78)

  • src/Plugins/SimplnxCore/CMakeLists.txt: downloads align_sections_input_v2.tar.gz
  • src/Plugins/SimplnxCore/pipelines/Align_Sections_Examples/*.d3dpipeline
  • src/Plugins/SimplnxCore/docs/AlignSectionsListFilter.md

The 6.5.171 shift file Alignment_By_Misorientation_Shifts.txt has 116 rows for 117 sections. It is missing the zero row of the top section, so Align Sections (List) applied every shift to the wrong section. The v2 archive adds the row 116 117 0 0 0 0, and the example pipelines read 117 rows from it. The documentation says that row 0 is the top section, that it must be a zero row for either input type, and that rows go from the top section to the bottom section. The unit test writes its own legacy file in code and does not use the archive.

3. Feature Face Curvature: too few triangles for the fit (L74)

  • src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CalculateTriangleGroupCurvatures.cpp

The seed triangle moves to the origin, so its row in the least-squares system is zero. A small patch could therefore fit 7 (with face normals) or 3 coefficients from fewer usable rows and return arbitrary curvatures. The algorithm now counts the usable neighbor rows (finite, not all zero). When there are fewer rows than coefficients, all requested outputs of that triangle are NaN, the same as for other invalid patches.

4. Feature Face Curvature: out-of-core race

  • src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/FeatureFaceCurvature.cpp
  • docs/Coding_Knowledge_Base.md

The algorithm ran one TBB task per feature face, and the tasks read and wrote shared stores. Out-of-core stores do not support concurrent writes, so values could be lost. A ParallelTaskAlgorithm now calls requireArraysInMemory() with all 13 arrays that the workers use. The tasks run in parallel only when all of these arrays are in memory; otherwise they run one after another. The knowledge base no longer lists Feature Face Curvature as a filter without this gate.

5. Align Sections (Feature Centroid): relative shifts in reference-slice mode (L77)

  • src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/AlignSectionsFeatureCentroid.cpp
  • src/Plugins/SimplnxCore/docs/AlignSectionsFeatureCentroidFilter.md

With Use Reference Slice on, the Relative Shifts array held the shift relative to the reference slice, which is the same as the Cumulative Shifts. Read back into Align Sections (List) as relative input, these shifts were added up a second time. Relative Shifts now hold the slice-to-slice difference of the cumulative shifts in both the in-core and the out-of-core path. The first row has no slice above it, so its relative shift equals its cumulative shift. Cumulative Shifts and the consecutive-slice mode do not change. The test computes the expected relative shifts from the existing exemplar's cumulative shifts, so the exemplar archive does not change.

6. Read Image: multi-page TIFFs with different page layouts (L79)

  • src/simplnx/Utilities/ImageIO/IImageIO.hpp, TiffImageIO.hpp/.cpp: new IImageIO::readPageLayouts() and PageMetadata
  • src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp: ReadImage::ValidatePageLayouts()
  • src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/ReadImageFilter.cpp

The reader took the component count and data type from page 0 and read every page with that layout. A page with a different layout gave wrong values. readPageLayouts() reads the size, component count, bits per sample and a format-neutral SampleFormat of every page in one pass. The default implementation in IImageIO returns one page. ValidatePageLayouts() compares only the pages that the read uses (page 0 plus the pages inside a Z crop; one page for a single-page read) and returns error -50579, which names the page and both layouts. Preflight runs the check. Execute runs it again, which protects callers that use the algorithm directly for multi-page reads without the filter's preflight, such as Import Fiji Montage.

The page-layout check runs before the per-page -2007 checks from #1915. A file with an 8-bit RGB page followed by an 8-bit grayscale page now fails in preflight with -50579, not in execute with -2007. The existing test MultiPage_TIFF_RGB_Then_Grayscale_Rejected_Before_Copy expects this, and the Read Image page describes both errors. Pages with different dimensions still fail with -2007 while the filter reads them.

Test plan

  • New regression tests with independent expected values:
    • NXImageProcessing::MorphologyEngine 2D out-of-plane kernel tests (L73), and the 2D passthrough tests of Opening and Closing by Reconstruction.
    • SimplnxCore::AlignSectionsListFilter: Legacy cumulative file requires zero row (L78).
    • SimplnxCore::FeatureFaceCurvatureFilter: Patch solvability (L74): analytic paraboloid patches at and below the minimum size.
    • SimplnxCore::FeatureFaceCurvatureFilter: multi-feature planes and ... with real OOC stores: outputs are pre-filled with a sentinel, so a lost write fails.
    • SimplnxCore::AlignSectionsFeatureCentroidFilter: Reference slice relative shifts round trip (L77): literal expected shifts, and Align Sections (List) gives the same result from relative and cumulative input.
    • NXImageProcessing::ReadImageFilter: TIFF_PageLayout, TIFF_PageLayout_SinglePageOnly, TIFF_PageLayout_ZCrop, NXImageProcessing::ReadImage: TIFF_PageLayout rejected in execute, and the TiffImageIO:: readPageLayouts ... tests (L79).
  • The L73, L74, L77 and L79 tests, the Reconstruction passthrough tests, the execute-time page-layout test and the sentinel curvature test were run against the code before their fix and failed.
  • The existing reference-slice tests (Nonzero reference top slice, Empty slices) check the new slice-to-slice Relative Shifts.
  • macOS arm64 Release, in-core: affected tests (MorphologyEngine|Grayscale|MorphologicalGradient|TopHat|Reconstruction|AlignSections|FeatureFaceCurvature|ReadImage|ImageIO|Tiff|ImportFijiMontage) and the Small IN100 and align-sections pipeline tests pass, run one at a time.
  • macOS arm64 Release, out-of-core (DREAM3D-NX build with SimplnxOoc and the companion PR): the same affected tests and the full ooc label (1053 tests) pass.
  • CI passes on all platforms.

Related Issue

Related to #1908
Related to #1815

Remove out-of-plane kernel offsets on single-slice images before the
empty-kernel checks. Previously, an Annulus with only Z offsets bypassed
those checks and reduced no image values. Preserve each operation's
existing empty-kernel result and leave 3D kernels unchanged.

Add independent copy-through and zero-output tests for all seven grayscale
filters on both algorithm paths, and for the bounded composite pipeline.

Part of BlueQuartzSoftware#1908 (item L73)

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Legacy DREAM3D shift files omit the top section's zero row. Importing the
116 legacy rows as cumulative shifts for 117 sections skips the first row
and applies the remaining shifts to the wrong sections.

Replace the align_sections_input.tar.gz registration with
align_sections_input_v2.tar.gz, which prepends the missing zero row.
The v1 archive is retired from this registration and remains immutable.
Update both example pipelines to use v2, import all 117 rows, document the
row-to-section mapping, and test generated legacy and corrected files.

Part of BlueQuartzSoftware#1908 (item L78)

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
The seed triangle moves to the origin and contributes a zero row to the
least-squares system. Small patches could therefore fit three or seven
coefficients with too few usable neighbor rows and return arbitrary curvature.

Count finite, nonzero non-seed rows for the selected fit and reuse the
existing invalid-patch NaN outputs when there are too few. Add analytic
paraboloid and minimum-size regressions and document the patch requirements.

Part of BlueQuartzSoftware#1908 (item L74)

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Reference-slice mode stored reference-relative cumulative offsets in both
shift arrays. Align Sections (List) accumulated those offsets again when
reading Relative input, producing incorrect alignment from row 2 onward.

Store differences of consecutive integer cumulative shifts in both Direct
and Scanline paths. Preserve cumulative shifts and non-reference behavior.
Add a literal shift oracle and List round-trip regression, and clarify the
Relative and Cumulative Shifts documentation. The existing output test now
derives Relative Shifts from the exemplar's Cumulative Shifts, preserving
the original archive and the other exact exemplar comparisons.

Part of BlueQuartzSoftware#1908 (item L77)

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
ReadImage compared decoded segment byte counts with page 0. Different
component counts and sample types with the same byte width could pass and
be copied using page 0's layout, producing incorrect pixel values.

Add IImageIO::readPageLayouts returning only small page metadata structs.
The TIFF implementation opens the file once, collects logical page directories
once and reads their tags with one forward traversal. ValidatePageLayouts
compares those layouts in memory, preserving TIFF's unsigned sample-format
default and the existing byte-count and dimension checks.

ReadImageFilter rejects mismatched layouts in preflight with error -50579.
The regression cases assert this preflight error explicitly. Keep the execute
check because ImportFijiMontageFilter calls ReadImage directly without
ReadImageFilter preflight; removing that guard requires addressing this caller.

Add generated TIFF regression cases and document the page layout requirement.

Part of BlueQuartzSoftware#1908 (item L79)

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
Read Image Stack reads only page 0 of each file. The page-layout check
compared every page of a multi-page TIFF, so a stack file whose unread
pages had a different layout failed with -50579, although only page 0 is
decoded. Pass the number of pages the read uses to ValidatePageLayouts in
preflight and execute, and add a single-page-read regression test.

Related to BlueQuartzSoftware#1908 item L79

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
FeatureFaceCurvature queued one worker per feature face even when the
workers accessed shared out-of-core stores. These stores do not support
concurrent access, so separate triangle ranges could still race and
lose output values.

Use ParallelTaskAlgorithm's residency gate with all input and output
arrays in one requireArraysInMemory call. Run workers synchronously
when any store is out-of-core and retain the existing TBB task queue
when all stores are resident. Add multi-feature analytic plane tests
with real disk-backed store witnesses and a deterministic queue-decision
regression. Remove the filter from the knowledge base's ungated list.

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
The page-layout check compared every page of a multi-page TIFF with page 0,
so a Z-cropped read failed with -50579 when a page outside the crop had a
different layout, although that page is never decoded. ValidatePageLayouts
now takes the first page and the number of pages that the read decodes and
compares only those pages with page 0, which sets the output array layout.
Preflight checks after the crop is resolved, and execute checks with the
same range before any output is written. Document exactly which pages are
checked and add Z-crop regression cases.

Related to BlueQuartzSoftware#1908 item L79

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
- Add 2D out-of-plane kernel tests and doc notes for Opening and Closing
  by Reconstruction (Preserve Intensities on), which the shared
  MorphologyEngine change also fixes.
- Add a 2D-kernel note to every grayscale morphology filter page.
- Reuse the in-plane kernel helper in the binary composite pipeline.
- Replace the raw TIFF sample-format code in PageMetadata with a
  format-neutral SampleFormat enum.
- Add direct readPageLayouts tests, a test that reaches the
  execute-time page-layout check, and a curvature test that detects
  lost writes.
- Fix doc wording, test tags and comments; drop unneeded LoadPlugins().

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
The page-layout check rejects a TIFF whose pages differ in component
count or data type in preflight with -50579, before the per-page byte
count check can return -2007 in execute. Update the mixed RGB/grayscale
test to expect -50579 from preflight and execute with no output array,
and describe both errors on the Read Image page.

Part of BlueQuartzSoftware#1908 (item L79)

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
@imikejackson imikejackson self-assigned this Nov 14, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

}

// Translation makes the seed row zero. Each coefficient needs a usable neighbor row.
if(usableRows < cols)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Important] The L74 solvability check counts nonzero rows, but the fit needs full rank

usableRows counts neighbor rows that are finite and not all zero. Enough such rows does not make A full rank.

  • Normals off (3 columns): neighbors collinear with the seed, for example (1,0), (2,0), (3,0), give rows (0.5x², 0, 0). usableRows is 3 but the rank is 1.
  • Normals on (7 columns): the seed lies on one line of centroids and the other neighbors on a parallel line y = b (a one-triangle-wide strip, which voxel meshes produce). Then y(y−b)(cx+dy) is a nonzero combination of {xy, y², xy², y³} that is zero at every point, so the rank is at most 6 even with 7 or more usable rows.

colPivHouseholderQr().solve(b) then returns one basic solution from a null space, so the cross-strip curvature on a curved strip is arbitrary. That is the defect L74 says it fixes. The Patch solvability test has no rank-deficient patch with enough rows.

Fix: decide on the rank:

const Eigen::ColPivHouseholderQR<Eigen::MatrixXd> qr(A);
if(usableRows < cols || qr.rank() < static_cast<Eigen::Index>(cols))
{
  markOutputsInvalid(triId);
  continue;
}
// sln1 = qr.solve(b);

Add collinear (normals off) and two-parallel-line (normals on) cases that expect NaN, and update the paragraph in FeatureFaceCurvatureFilter.md.

{
return MakeErrorResult<std::vector<PageMetadata>>(k_ErrorReadMetadataFailed, fmt::format("Cannot read dimensions of page {} of TIFF '{}': {}", page, pathStr, file.errorMessage()));
}
TIFFGetFieldDefaulted(tiff, TIFFTAG_SAMPLESPERPIXEL, &components);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] readPageLayouts() accepts tag defaults that readMetadata() rejects

readMetadata() requires an explicit SamplesPerPixel tag (to avoid treating malformed multichannel data as grayscale) and gets the type from DetermineTiffDataType(). readPageLayouts() uses TIFFGetFieldDefaulted for SamplesPerPixel, BitsPerSample and SampleFormat. A later page with no SamplesPerPixel tag is reported as 1 component, passes -50579 against a 1-component page 0, and is decoded with the layout of page 0.

Fix: apply the same rules to each page. Require the tag and return k_ErrorReadMetadataFailed with the page index when it is missing. Get the type with DetermineTiffDataType(tiff), then derive bits and SampleFormat from it, or store the DataType in PageMetadata.

stream.dstWidth, stream.dstHeight, stream.dstDepth, stream.srcWidth, stream.srcHeight, metadata.numPages));
}
// Only the decoded pages are validated. Page 0 sets the output layout, so each decoded page is compared with it.
auto layoutResult = ReadImage::ValidatePageLayouts(*imageIO, inputFilePath, stream.singlePageOnly ? usize{0} : stream.zStart, stream.dstDepth);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] Execute repeats the preflight page-layout check, and a test calls the algorithm directly to reach it

This breaks the KB entry Run a filter, not its algorithm. preflightImpl() already runs ValidatePageLayouts(), and ReadRasterBackend runs it again. The reason given is ImportFijiMontageFilter's direct construction, but the entry says that caller must establish the precondition and name it at the call. ReadImageTest.cpp:1847 calls ReadImage::operator()() to reach an input check, which the entry's "What to look for" list names.

Fix (choose one): (a) Move the check to where ImportFijiMontageFilter prepares each tile, comment the precondition at the call, and test it through the filter. (b) Keep it as the read-time check that the entry allows (a file can change after preflight), say so in the comment, and make the test change the file between preflight and execute.

- Slices (DataArray | 2 component) - The slice indices (stored as uint32s)
- Relative Shifts (DataArray | 2 component) - The slices shift relative to previous shift (stored as int64s) [*previously known as `newxshift` and `newyshift`*]
- Cumulative Shifts (DataArray | 2 component) - The slice's accumulated shift (stored as int64s)
- Relative Shifts (DataArray | 2 component) - The slice's shift relative to the slice above it (the slice with the next higher Z index), in both modes (stored as int64s). The first row has no slice above it, so its relative shift equals its cumulative shift. [*previously known as `newxshift` and `newyshift`*]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] Nonzero reference slice: Relative Shifts do not round-trip, and neither the docs nor the tests say so

With a nonzero Reference Slice, row 0 holds the top section's shift c₀, and AlignSectionsList::findShifts never reads row 0. With Relative input, lower slices get c_k − c₀ and the top slice gets 0. With Cumulative input, lower slices get c_k and the top slice gets 0. Neither reproduces Feature Centroid, and the two disagree. The new round-trip test uses only Reference Slice 0.

Fix: document on both pages what List does with row 0 for a nonzero reference. Add a round-trip case with a nonzero reference that asserts the documented outcome. The List limitation itself is pre-existing wrong data (report item 2.1).

TEST_CASE("SimplnxCore::FeatureFaceCurvatureFilter: multi-feature planes", "[SimplnxCore][FeatureFaceCurvatureFilter]")
{
// Storage-aware array creation and the storage preferences need the application that LoadPlugins() creates.
UnitTest::LoadPlugins();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] The in-core curvature test loads every plugin but only needs the Application

This breaks the KB entry Call UnitTest::LoadPlugins() only when the test looks up filters by UUID. The in-core multi-feature planes test constructs the algorithm directly and needs only the Application for the preferences sentinel. Use Application::GetOrCreateInstance(), as the entry prescribes, so the test does not fail when an unrelated plugin does not load. Keep LoadPlugins() in the real-OOC case, which needs the HDF5-OOC registration.

per-feature output stores without a gate (PR #1645 made it serial). `ComputeFeatureFaceMisorientation` was fixed in
PR #1618. `CombineStlFiles` (issue #1876), `ConvertOrientations`, `PointSampleEdgeGeometry` and `FeatureFaceCurvature`
still launch workers that write shared stores without a gate.
PR #1618, and `FeatureFaceCurvature` in the change that fixed issue #1908 items L73 to L79. `CombineStlFiles` (issue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] The knowledge base edit cites "the change" instead of the PR

The file's rules say to cite pull requests by number, and "items L73 to L79" suggests that L75 and L76 are included. Replace with: "ComputeFeatureFaceMisorientation was fixed in PR #1618 and FeatureFaceCurvature in PR #1981."

*
* On a single-slice image, the Z offsets of the kernel are ignored, so this kernel is empty and the filter copies
* the input. The oracle is the input itself, and the same filter with the empty Annulus {0, 0, 0} must give the same
* values. Before #1908 L73, Preserve Intensities on returned INT32_MIN (opening) or INT32_MAX (closing) everywhere.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] Comments and tags narrate history

cpp-code-documentation says to describe the current code, without task numbers. Replace "Before #1908 L73, Preserve Intensities on returned INT32_MIN (opening) or INT32_MAX (closing) everywhere." with "With Preserve Intensities on, an empty kernel must not produce the INT32_MIN or INT32_MAX fill value." Also replace the [L73] tags in MorphologyEngineTest.cpp and the "(#1908 L73)" lines in the Reconstruction tests with behavior descriptions.

Comment thread test/TiffImageIOTest.cpp
std::error_code errorCode;
fs::remove(path, errorCode);
const auto result = imageIO.readPageLayouts(path);
REQUIRE(result.invalid());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] The new readPageLayouts failure tests are loose

Use SIMPLNX_RESULT_REQUIRE_INVALID instead of REQUIRE(result.invalid()) (KB entry Assert a valid Result with SIMPLNX_RESULT_REQUIRE_VALID). Assert the code in the "not a TIFF" section. The two new k_ErrorReadMetadataFailed branches ("Cannot select page", "Cannot read dimensions") have no test that reaches them, as cpp-best-practices requires; add one where practical.

@joeykleingers joeykleingers left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the summary of my review of 2026-11-16. GitHub removed it when the review was submitted, so I post it here.

PR 1981 Code Review

Source: review of commit 2fa764d (#1981, "BUG: Fix #1908 wrong-data items L73, L74, L77, L78 and L79" by imikejackson, topic/issue1908_batch4 → develop @ 8cad1e8). Coverage: all 36 changed files (C++, tests, filter docs, pipelines, CMake, knowledge base).

1. Findings

  • 1.1 [Important] The L74 solvability check counts nonzero rows, but the fit needs full rank
    File: src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CalculateTriangleGroupCurvatures.cpp:336-347 (CalculateTriangleGroupCurvatures::operator())
    CONFIRMED (logic), PLAUSIBLE (how often real meshes hit it). Found by the correctness and independent-evidence checks.
    usableRows counts neighbor rows that are finite and not all zero. Enough such rows does not make A full rank:

    • Normals off (3 columns): neighbors whose centroids are collinear with the seed in the local frame, for example (1,0), (2,0), (3,0). The rows are (0.5x², 0, 0), so usableRows is 3 but the rank is 1.
    • Normals on (7 columns): the seed lies on one line of centroids and the other neighbors lie on a parallel line y = b. This is a one-triangle-wide strip, which voxel surface meshes produce. Then y(y−b)(cx+dy) is a nonzero combination of the monomials {xy, y², xy², y³} that is zero at every point, so the rank is at most 6 even when 7 or more rows count as usable.

    In both cases colPivHouseholderQr().solve(b) returns one basic solution from a null space. On a curved strip, the cross-strip curvature terms (xy, y²) are therefore arbitrary. That is the wrong-data behavior that L74 says it fixes. The new Patch solvability test only exercises row counts (including duplicate seed rows) and has no rank-deficient patch with enough rows.
    Fix: decide on the rank, using the Eigen QR that the code already runs:

    const Eigen::ColPivHouseholderQR<Eigen::MatrixXd> qr(A);
    if(usableRows < cols || qr.rank() < static_cast<Eigen::Index>(cols))
    {
      markOutputsInvalid(triId);
      continue;
    }
    // ... sln1 = qr.solve(b);

    This also avoids factoring A twice. Add Patch solvability cases with collinear neighbors (normals off) and two parallel centroid lines (normals on), and expect NaN. Update the FeatureFaceCurvatureFilter.md paragraph: the fit also needs neighbors that determine every coefficient, not only enough usable rows.

  • 1.2 [Minor] readPageLayouts() accepts tag defaults that readMetadata() rejects
    File: src/simplnx/Utilities/ImageIO/TiffImageIO.cpp:430-432 (TiffImageIO::readPageLayouts)
    readMetadata() requires an explicit SamplesPerPixel tag ("avoids treating malformed multichannel data as grayscale"), and it gets the data type from DetermineTiffDataType(). readPageLayouts() uses TIFFGetFieldDefaulted for SamplesPerPixel, BitsPerSample and SampleFormat. Example: page 0 is a valid 1-component uint16 page and a later page has no SamplesPerPixel tag. That page is reported as 1 component, passes the -50579 check, and is decoded with the layout of page 0, which is the malformed-multichannel case the backend says it refuses. The two functions also describe the sample type in different ways: raw bits plus format here, the DataType that the decoder uses there.
    Fix: for each page, apply the same rules as readMetadata(). Require the SamplesPerPixel tag and return k_ErrorReadMetadataFailed with the page index when it is missing. Get the type from DetermineTiffDataType(tiff), then derive bitsPerSample and SampleFormat from that DataType, or store the DataType in PageMetadata. Then the check compares exactly what the decoder will use.

  • 1.3 [Minor] Execute repeats the preflight page-layout check, and a test calls the algorithm directly to reach it
    File: src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp:317 and src/Plugins/NXImageProcessing/test/ReadImageTest.cpp:1847
    This breaks the knowledge base entry Run a filter, not its algorithm. ReadImageFilter::preflightImpl() already runs ValidatePageLayouts(), and ReadRasterBackend runs it again in every execute. The reason given is ImportFijiMontageFilter, which constructs ReadImage directly. Under that entry, the direct caller is the one that must establish the precondition, with a comment at the call site. The test ReadImage: TIFF_PageLayout rejected in execute calls ReadImage::operator()() to reach an input check, which the entry's "What to look for" list names.
    Fix (choose one): (a) Remove the execute call. Run ValidatePageLayouts() where ImportFijiMontageFilter prepares each tile, name the precondition at the ReadImage(...) call, and test it through the filter. (b) Keep it as the read-time file check that the entry allows ("a file can change after preflight"). Then say so in the comment at ReadImage.cpp:316, and make the test change the file between preflight() and execute() instead of skipping the filter.

  • 1.4 [Minor] Stale comment says page layouts cannot be detected
    File: src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp:355-357
    The comment says "IImageIO exposes metadata only for page 0, so these checks cannot detect different component counts or scalar types with the same byte width." This PR adds IImageIO::readPageLayouts(), and ValidatePageLayouts() now detects exactly that. Replace it with:

    // Destination offsets use the dimensions and pixel stride of page 0. ValidatePageLayouts() already compared the
    // component count and sample type of each decoded page with page 0. Here, check the segment byte counts and the
    // observed dimensions of each page.
  • 1.5 [Minor] Align Sections (List): the "Relative" definition contradicts the new row-order text
    File: src/Plugins/SimplnxCore/docs/AlignSectionsListFilter.md:23
    Line 23 still says "the offset of section N relative to section N-1". The new text (lines 32 and 65) and the code (AlignSectionsList::findShifts, rows ordered from top to bottom, accumulated from row 1) define the relative shift of a section against the section above it (N+1). A user who builds a relative file from line 23 gets every shift paired with the wrong neighbor. Replace with: "Relative [0]: Each row gives the shift of its section relative to the section above it (the next higher Z index). The filter adds these shifts, starting from the top section, to get cumulative shifts before it applies them."

  • 1.6 [Minor] Nonzero reference slice: Relative Shifts do not round-trip, and neither the docs nor the tests say so
    File: src/Plugins/SimplnxCore/docs/AlignSectionsFeatureCentroidFilter.md:65 (and the new round-trip test in AlignSectionsFeatureCentroidTest.cpp)
    With Use Reference Slice on and Reference Slice ≠ 0, row 0 holds the nonzero shift c₀ of the top section. AlignSectionsList::findShifts never reads row 0. With Relative input, each lower slice gets c_k − c₀ and the top slice gets 0. With Cumulative input, each lower slice gets c_k and the top slice gets 0. Neither result reproduces the Feature Centroid output, and the two inputs disagree with each other. The new round-trip test uses only ReferenceSlice = 0, where c₀ = 0. The PR description says relative input now gives the same result, without this limit.
    Fix: state on both pages what Align Sections (List) does with row 0 when the reference slice is not 0. Add a round-trip case with a nonzero reference slice that asserts the documented outcome. Ask the reviewer where the pre-existing List limitation should be fixed (see 2.1).

  • 1.7 [Minor] The in-core curvature test loads every plugin but only needs the Application
    File: src/Plugins/SimplnxCore/test/FeatureFaceCurvatureTest.cpp:1184 (multi-feature planes)
    This breaks the knowledge base entry Call UnitTest::LoadPlugins() only when the test looks up filters by UUID. The test constructs the algorithm directly and uses ForceInCore. Its comment says it needs the Application for the preferences sentinel and for storage-aware creation, and the entry's prescribed call for that is Application::GetOrCreateInstance(). As written, the test fails when an unrelated plugin library does not load. Keep LoadPlugins() in the real OOC stores case, which needs the HDF5-OOC registration.

  • 1.8 [Nit] The knowledge base edit cites "the change" instead of the PR, and names items the PR does not fix
    File: docs/Coding_Knowledge_Base.md:1104
    The file's own rules say to cite pull requests by number. "In the change that fixed issue #1908 items L73 to L79" also suggests that L75 and L76 are included. Replace with: "ComputeFeatureFaceMisorientation was fixed in PR #1618 and FeatureFaceCurvature in PR #1981."

  • 1.9 [Nit] Comments and tags narrate history
    Files: src/Plugins/NXImageProcessing/test/ReconstructionFilterTestUtils.hpp:521; the [L73] tags in MorphologyEngineTest.cpp; the "(#1908 L73)" lines in Opening/ClosingByReconstructionImageFilterTest.cpp
    cpp-code-documentation says to describe the current code, without task numbers. For example, replace "Before #1908 L73, Preserve Intensities on returned INT32_MIN (opening) or INT32_MAX (closing) everywhere." with "With Preserve Intensities on, an empty kernel must not produce the INT32_MIN or INT32_MAX fill value." Use a behavior tag such as [MorphologyEngine] instead of [L73].

  • 1.10 [Nit] The new readPageLayouts failure tests are loose
    File: test/TiffImageIOTest.cpp:555,567
    They use REQUIRE(result.invalid()), but the knowledge base entry Assert a valid Result with SIMPLNX_RESULT_REQUIRE_VALID asks for SIMPLNX_RESULT_REQUIRE_INVALID. The "not a TIFF" section asserts no code. The two new k_ErrorReadMetadataFailed branches ("Cannot select page", "Cannot read dimensions") have no test that reaches them, although cpp-best-practices asks for a test for each new failure branch. Use the macro, assert -20100 or -20101 and the distinct message phrase, and add a case for a truncated IFD chain if one is practical.

  • 1.11 [Nit] New tests call preflight() before execute()
    Files: ReconstructionFilterTestUtils.hpp (RequireZOnlyAnnulus2DPassthrough), MorphologyEngineTest.cpp (Grayscale morphology: 2D out-of-plane Annulus passthrough), FeatureFaceCurvatureTest.cpp (Patch solvability)
    Run a filter, not its algorithm: execute() already runs preflight. Remove the separate preflight() calls unless the test asserts something on the preflight result.

2. Pre-existing defects surfaced by this PR

  • 2.1 Wrong data: yes. AlignSectionsList::findShifts (AlignSectionsList.cpp:49-64) never reads row 0. Input: the Cumulative Shifts of Align Sections (Feature Centroid) with a nonzero Reference Slice. Effect: the top section stays unshifted while every other section moves by its cumulative shift, so the top section is misaligned by c₀ relative to the rest, and there is no error. This is not introduced here. The reviewer decides where to fix it (see 1.6).

3. Documentation

Covered in 1.4, 1.5, 1.6, 1.8 and 1.9. The nine morphology pages, the Read Image page (-50579 before the read, -2007 for dimensions, cropped and single-page scope) and the Feature Face Curvature page match the code. The Feature Face Curvature page will need the rank wording from 1.1.

4. Missing or failed evidence

  • 4.1 clang-format 19.1.1: every changed C/C++ file is formatted. clang-tidy was not run (no build available), and no tests were run in this review. The pass results in the PR description (in-core, and out-of-core with the private companion PR) are the author's reports, not observations from this review.
  • 4.2 I could not inspect align_sections_input_v2.tar.gz or its SHA-512. Whether Alignment_By_Misorientation_Shifts.txt has 117 rows with 116 117 0 0 0 0 as the first data row, and whether example_relative_shifts.txt is unchanged, is unverified. The pipeline tests are the only evidence.
  • 4.3 multi-feature planes with real OOC stores returns with a WARN in builds without HDF5-OOC. In the public in-core CI, the out-of-core gate is therefore checked only by the companion out-of-core build.
  • 4.4 "CI passes on all platforms" is unchecked in the PR test plan.

5. Confirmed correct

  • 5.1 L73: detail::MakeInPlaneMorphologyKernel() returns nullopt unless Z = 1 and the kernel has an offset with dz ≠ 0, so each redirect recurses only once. It is applied in MorphScanline, MorphDirect, MorphGradientScanline, MorphGradientDirect and both composite pipelines, after the original layout validation. The binary composite route now shares the same helper with no change in behavior. The tests compare with the input or with zero (an independent oracle) across both dispatch scenarios.
  • 5.2 L77: the in-core and out-of-core paths compute the same slice-to-slice differences. Row 0 is written only for a nonzero reference. The hand-checked expected values ({1,−1,−1,1}; {0,0,2,1,−1,2,2,−1}) and the empty-top case [0,0,−1,1] agree with the cumulative shifts. Negative differences that pass through the usize intermediates in the in-core path convert back exactly under C++20.
  • 5.3 Feature Face Curvature gate: requireArraysInMemory() lists every store the workers touch (all 7 outputs, the 4 inputs, feature-face ids, vertices, faces). CheckArraysInMemory skips the nullptr optional outputs. With a non-resident store the workers run inline. The test shows disk-backed HDF5-OOC stores and that the work-queue message is absent, and the sentinel-filled outputs catch lost writes. This satisfies Write out-of-core stores from one thread.
  • 5.4 L79: ValidatePageLayouts() compares page 0 with exactly the decoded range: [max(first,1), min(size, first+count)). A read of page 0 alone short-circuits, and preflight and execute derive the same first page for voxel and physical Z crops. -50579 is unique in the repository, and the message names the file, the page and both layouts. readPageLayouts() uses CollectPageDirectories(), so pyramid and mask subfiles map the same way as in readPixelDataRows(), and a test covers this. The check reads file metadata, not a placeholder count, so running it in preflight is allowed.
  • 5.5 L78: the new _v2 archive follows the immutable-archive rule. No reference to the old directory remains. The legacy-file test builds its own file and checks hand-computed mappings with and without the zero row.

6. Verdict

REQUEST CHANGES: 1.1 is open. The L74 check accepts rank-deficient patches and still writes arbitrary curvatures on collinear or strip-shaped patches, which is the defect class this PR says it fixes. The other items are Minor or Nit.


Checks and lenses with no further findings: CPU, memory, naming, const-correctness, error-message wording, placeholder-count check, test-data immutability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants