Repository navigation
BUG: Fix selected ImageProcessing numeric conversions and ranges - #1888
Open
jmarquisbq wants to merge 93 commits into
Open
jmarquisbq wants to merge 93 commits into
jmarquisbq wants to merge 93 commits into
Conversation
jmarquisbq
force-pushed
the
fix/unsafe-numeric-conversions-1874
branch
2 times, most recently
from
November 3, 2026 16:01
decbffd to
549b553
Compare
This was referenced Nov 3, 2026
joeykleingers
added a commit
to joeykleingers/simplnx
that referenced
this pull request
Nov 12, 2026
Replace the knowledge-base entry that let the user choose only how a NaN input value is handled with one rule for NaN and both infinities, decided by where the value comes from. It follows the handling in BlueQuartzSoftware#1888: - Parameters: preflight rejects NaN, and rejects infinity unless the filter documentation gives it a meaning. Values stored into a floating-point array may be NaN or infinity. - Values read from a file: the reader rejects nonfinite values where the format requires finite ones. - Input array values that reach a comparison, range, histogram, threshold, color map, search, index or integer conversion: a Nonfinite Input Handling choice, Error (default) or Warn and Continue. - Element-wise floating-point math passes IEEE results through and documents them. - A computed nonfinite value for an integer output is an error naming the index and output type; finite out-of-range values are clipped with a warning, instead of NaN becoming 0. - An undefined result for degenerate input is a documented NaN. Update the saturating-cast and parameter-validation entries to match. Part of BlueQuartzSoftware#1873. Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
jmarquisbq
force-pushed
the
fix/unsafe-numeric-conversions-1874
branch
from
November 12, 2026 19:17
e23107c to
eff78a6
Compare
jmarquisbq
marked this pull request as ready for review
November 12, 2026 23:51
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
imikejackson
pushed a commit
to joeykleingers/simplnx
that referenced
this pull request
Nov 13, 2026
Replace the knowledge-base entry that let the user choose only how a NaN input value is handled with one rule for NaN and both infinities, decided by where the value comes from. It follows the handling in BlueQuartzSoftware#1888: - Parameters: preflight rejects NaN, and rejects infinity unless the filter documentation gives it a meaning. Values stored into a floating-point array may be NaN or infinity. - Values read from a file: the reader rejects nonfinite values where the format requires finite ones. - Input array values that reach a comparison, range, histogram, threshold, color map, search, index or integer conversion: a Nonfinite Input Handling choice, Error (default) or Warn and Continue. - Element-wise floating-point math passes IEEE results through and documents them. - A computed nonfinite value for an integer output is an error naming the index and output type; finite out-of-range values are clipped with a warning, instead of NaN becoming 0. - An undefined result for degenerate input is a documented NaN. Update the saturating-cast and parameter-validation entries to match. Part of BlueQuartzSoftware#1873. Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
imikejackson
added a commit
that referenced
this pull request
Nov 13, 2026
…1947) * BUG: Reject kernel radii that morphology filters cannot represent Kernel Radius is a UInt32 vector that preflight never checked. Execute narrowed each component to int32, so a radius of 2147483648 or more became negative, the structuring element came out empty and the filter copied its input (top-hats and the gradient wrote zeros). Large radii also overflowed the element-count and Safe Border size arithmetic. - Add ValidateKernelRadius() and MakeCheckedStructuringElement() to StructuringElement.hpp: each radius must be at most 2147483646 and the element count must fit the offset vector. Allocation failures are returned as errors. - Add ValidateMorphologyLayout() to MorphologyEngine.hpp. It checks the Safe Border padded extents and the signed coordinate, plane, volume and byte counts derived from the image and the radius. Preflight and every execution entry point call it before allocating. - Use the flat-offset fold only when its offsets and staged indices fit int64, and clamp the packed binary word end without overflow. - Negative radii in MakeStructuringElement() return an empty element before any negation, and its loops and shell arithmetic use int64. Fixed filters: BinaryDilateImageFilter, BinaryErodeImageFilter, BinaryMorphologicalClosingImageFilter, BinaryMorphologicalOpeningImageFilter, BinaryOpeningByReconstructionImageFilter, BlackTopHatImageFilter, ClosingByReconstructionImageFilter, DilateObjectMorphologyImageFilter, ErodeObjectMorphologyImageFilter, GrayscaleDilateImageFilter, GrayscaleErodeImageFilter, GrayscaleMorphologicalClosingImageFilter, GrayscaleMorphologicalOpeningImageFilter, MorphologicalGradientImageFilter, OpeningByReconstructionImageFilter, WhiteTopHatImageFilter. Part of #1873. Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net> * DOC: Decide NaN and infinity handling by where the value comes from Replace the knowledge-base entry that let the user choose only how a NaN input value is handled with one rule for NaN and both infinities, decided by where the value comes from. It follows the handling in #1888: - Parameters: preflight rejects NaN, and rejects infinity unless the filter documentation gives it a meaning. Values stored into a floating-point array may be NaN or infinity. - Values read from a file: the reader rejects nonfinite values where the format requires finite ones. - Input array values that reach a comparison, range, histogram, threshold, color map, search, index or integer conversion: a Nonfinite Input Handling choice, Error (default) or Warn and Continue. - Element-wise floating-point math passes IEEE results through and documents them. - A computed nonfinite value for an integer output is an error naming the index and output type; finite out-of-range values are clipped with a warning, instead of NaN becoming 0. - An undefined result for degenerate input is a documented NaN. Update the saturating-cast and parameter-validation entries to match. Part of #1873. Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net> * BUG: Clip adaptive histogram equalization neighbors to the image The Radius of AdaptiveHistogramEqualizationImageFilter is a UInt32 vector, and every neighbor loop walked the whole -r..r box and skipped the neighbors outside the image. A radius near UINT32_MAX made each voxel run billions of empty iterations, so the filter did not finish. Each of the four neighbor loops (the generic and lookup-table plane bodies and the generic and lookup-table 2D block bodies) now computes its bounds clipped to the image before it iterates. The skipped iterations are exactly the ones that were skipped before, in the same order, so the accumulated sum, the in-bounds count and every output value are unchanged. The linear uint8 paths already bounded their traversal and are unchanged. Part of #1873. Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net> * REV: Restore the nonfinite skip rule and clear new clang-tidy findings - Coding knowledge base: Warn and Continue excludes nonfinite values from the contributing samples and counts of reductions and neighborhood operations. The nonfinite-handling rules apply to new filters and to filters a change already fixes; bringing an existing filter's output type or failure behavior in line is a breaking change that its pull request names. - Adaptive histogram equalization: rename the signed coordinates xi and yi to signedX and signedY. - Morphology and structuring-element guards: camelCase local constants, checked std::array access, a kernel parameter name in the two new layout helpers, an explicit int64 slice product and parentheses in the bounding-box extents. No behavior change. * REV: Check kernel radii only in preflight Filter preflight is the one place a filter's inputs are checked; the shared morphology helpers do not repeat those checks. - MakeCheckedStructuringElement() and ValidateMorphologyLayout() no longer re-run ValidateKernelRadius(); they document a validated radius as a precondition. The structuring element still converts allocation failures to an error, and the layout checks on the final geometry dimensions still run in preflight and execute. - The kernel-radius tests also execute each filter with an invalid radius and check that it fails with the preflight code, creates no output and leaves the input unchanged. The test that called the helper directly with an invalid radius is removed. Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net> * TEST: Use the NXImageProcessing plugin names after the rebase Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net> --------- Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net> Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net> Co-authored-by: Michael Jackson <mike.jackson@bluequartz.net>
jmarquisbq
requested review from
imikejackson and
joeykleingers
and removed request for
imikejackson and
joeykleingers
November 13, 2026 16:38
joeykleingers
requested changes
Nov 13, 2026
joeykleingers
self-requested a review
November 13, 2026 20:47
imikejackson
force-pushed
the
fix/unsafe-numeric-conversions-1874
branch
from
November 13, 2026 21:14
5e080a8 to
4652ca7
Compare
Contributor
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Contributor
Author
Review feedback checklistTracks all 16 items from Joey’s November 13 review, posted against High priority
Low priority
Nits
|
joeykleingers
previously approved these changes
Nov 16, 2026
joeykleingers
left a comment
Contributor
There was a problem hiding this comment.
The review fixes are in 860e902. The companion revision is BlueQuartzSoftware/SimplnxOoc#11 at 42c38a9.
Manual test on macOS (Release, out-of-core with visualization): the checklist passed in Force In-Memory and Force Out-of-Core. This includes bookmarks 16, 08 and 19, save and reopen, the Otsu Multiple Thresholds and Write Image controls, and the removal of the Connected Component legacy option. The out-of-core results were bit-identical to the in-memory results. The Windows package checks were not run.
joeykleingers
force-pushed
the
fix/unsafe-numeric-conversions-1874
branch
from
November 16, 2026 21:10
860e902 to
f2b6042
Compare
Contributor
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Use the actual batch length to select serial labeling through 512 pixels, including a short final batch. Release measurements with one, two, four and eight thresholds on Intel and AMD CPUs established the cutoff. Check every output label across the boundary. Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Compute the inverse bin width once for each nonconstant histogram. Use it for both sample batches and compact integer frequencies. Construct the coordinate diagnostic only when the bin would be invalid. Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Correct unsafe numeric conversions and range calculations in 18 NXImageProcessing filters. This is partial C15 coverage: 19 approved findings are addressed, and the remaining C15 work stays outside this increment.
Changes since the previous review
Gaussian preflight checks X and Y sigma. Execution checks Z sigma when the final image has more than one Z slice. Masked Otsu prepares its output for writing before reserving label buffers, including when it reuses a reopened destination. Threshold Maximum Connected Components rejects nonempty stores paired with a zero-volume geometry. Square preserves a failed 64-bit range read’s diagnostics, and smoothing and color export retain transfer warnings in order.
Manual Windows review — Mike’s signoff is open
Build from the latest head of
fix/unsafe-numeric-conversions-1874in both simplnx and SimplnxOoc. Use your normal Windows Release builds withDREAM3DNX_ENABLE_OOC=OFFandON.Data/ImageProcessing_Examples. Save, close and reopen each result, then select and render the named arrays again.Powder Particles/Cell Data/Particle Labelsalign with the particle boundaries inmaterials/powder_particles.png.Tangent,Inverse SineandInverse Cosineretain fractional values and continuous gradients. Useam/calibration/phase_radians.mhaandnormalized_response.mha.Serial Sections/Cell Data/Section Intensityis 256 × 256 × 8 with spacing (0.5, 0.5, 1.0). Open all eight preview TIFFs and confirm that the input sections fromio/serial_stackappear in order 000–007.Eight-pixel control check using Python's standard library
Run this in a writable directory to create the two inputs:
Read each file with Read MHA/MetaImage File into a separate image geometry. Use the intensity array as the image input and the UInt8 array as the mask. Pixel numbers below run from 1 to 8.
[7, 8, 255, 255, 7, 8, 255, 255]. Only selected finite samples fit the histogram. Unmasked nonfinite pixels have no scientifically defined Otsu label.Tutorial 17 has a separate tracked issue with two unreadable negative-origin TIFF previews. The eight previews in bookmark 19 must open. simplnx #1914
This checklist verifies the source-built application. A shipping ZIP’s startup on a clean Windows machine remains a separate, open deployment check.
Companion merge order
Merge simplnx #1888 before SimplnxOoc #11. The disk-backed tests require the output types, parameters, and policies in the simplnx change. DREAM3DNX #1233 already merged and supplied the application revision used for the Windows tests below.
Historical review material
The October decision companion and earlier Windows review ZIPs document previous revisions. The companion includes superseded nonfinite-control proposals. Use the branch heads and checklist above for this review.
1. Math output types and integer limits
src/Plugins/NXImageProcessing/src/NXImageProcessing/utils/MathOutputUtilities.hppsupplies the selected output conversions.2. Morphology and smoothing
HConvex and HMinima preserve native integer Height arithmetic and reject fractional Height for integer images. HConvex checks signed contrast overflow. CurvatureFlow and SmoothingRecursiveGaussian check terminal integer conversions, report clipping and reject nonfinite integer results. Their help records working-precision limits.
3. Segmentation, export and pipeline compatibility
ConnectedComponent checks provisional label capacity. MorphologicalWatershedFromMarkers checks marker ID conversion. Both offer Error and Warn policies, with Error as the default. OtsuMultipleThresholds fits thresholds from selected finite samples. The optional Bool/UInt8 mask assigns the Excluded Label to masked-out tuples. Labels at unmasked nonfinite pixels have no scientific interpretation. Preflight checks the mask type and component shape and rejects class-label collisions. Otsu validates the final mask tuple count during execution, so pipelines can use masks resized by an upstream filter. A final mismatch fails before either array is read.
ThresholdMaximumConnectedComponents checks Upper Boundary conversion and range arithmetic, corrects the midpoint, and retains the two-intensity-unit stopping tolerance. Its masks and downstream segmentation can change because of the corrected midpoint. Threshold assigns Outside Value to nonfinite pixels. WriteImage assigns the first preset color to unmasked nonfinite pixels during color mapping and gives excluded pixels the chosen mask color. Supported raw floating exports preserve the original values.
Review fixes also add first-sample diagnostics, validate filter options in preflight, and pass validated native integer Heights to the reconstruction engine. Connected Component help explains the route-dependent effects of legacy label wrap. Shipped bookmarks, Python variants and generated help use the matching contracts and parameters. Consumer-chain tests check supported migration paths. Includes, generated-header references and test tags follow the NXImageProcessing plugin name.
4. SDK headers and solver documentation
Install
ImageProcessingOperationTypes.hppwith the public SDK so consumers can include the threshold engine. The Otsu solver documentation now states how it reports an empty threshold result.Test plan
Verification for this review
The results below were recorded at simplnx
d6ab00543and SimplnxOoc77d5b65e9using DREAM3D-NX8e6d7403.Earlier source and Windows archive validation
Earlier Windows review follow-up validation
The affected ImageProcessing test target built in Windows Release in both storage configurations and in in-core Debug. These results apply to the earlier simplnx
44875faf, SimplnxOoce191adcdand desktop8e6d7403revisions.-28908before either store is read.Mike’s signoff on the current source pair is tracked in the manual Windows checklist above.
Earlier review-fix validation
The previous runs used simplnx
4758c66d8bec, SimplnxOoc397fb6fb4050ora960d2ccf143as noted below, and DREAM3DNX8e6d7403df21.a960d2ccf143. Its later changes are private out-of-core tests, so the in-core binaries and coverage are unaffected.a960d2ccf143. After the companion test updates, the ImageProcessing target rebuilt and 615/615 focused cases and 1,131/1,131 non-benchmark out-of-core cases passed with the companion at397fb6fb4050.clang_format_prcheck passed.Earlier source verification — 12 November 2026
The coordinated pair for this section was simplnx
5e080a8ec87band SimplnxOoc1692f37b57e8, with DREAM3DNX2ad8cf5982d65.Two focused Gaussian declaration reports were reproduced on pinned develop for the exact parameter pairs. Whole-TU configured analysis remains non-clean and exits 1; a whole-project zero-warning result is not claimed. One earlier LLVM tool crash was preserved, then completed in an identical-input isolated retry. No analysis coverage gap remains.
Windows archive and bookmark checks — 12 November 2026
Launch PATH and profiles were isolated to the package and Windows. Static runtime identities match the owning final builds and configured dependencies. Core module sampling was complete in 33/36 runs; no observed module came from outside the package or Windows. Sampling does not prove every transient DLL load. The package campaign does not establish a new physical backing-file witness or a clean-VM/GUI pass. Actual disk-backed store assertions remain part of the OOC unit coverage.
The comparison baseline retains 22 retired source-path references after the plugin-folder rename. Each replacement pipeline was checked against the exact approved hash; these are recorded historical provenance gaps, not new output failures. The bounded comparator establishes compatibility, not independent mathematical correctness.
Earlier develop-integration regression evidence
Before the final mechanical/diagnostic/preflight corrections, the paired set at public
eff78a69975437cfe58e0c81b57668d20c2689d9and the unchanged companion revisions passed:These earlier selections overlap and are not recounted as fresh final-head tests. The numerical/storage contracts outside the final corrections are unchanged. In-memory validation uses algorithm path 0; OOC uses path 1. The broad selection excluded benchmarks and ITK-named entries; the C15 selections separately ran 35 ITK-named golden cases in each build. No benchmarks were included in those earlier selections.
Separate CI and deployment checks
5 November package record
Earlier package evidence — 5 November 2026
Earlier C15 archives passed 36/36 packaged nxrunner invocations across the 18 bookmarks and two modes. Output checks passed 30/36 rows. Six rows reproduced the inherited TIFF and copper pole-figure blockers below. A physical Serial-Section output witness matched all 524,288 stored UInt8 values to the saved result. Jessica Marquis reported the representative startup and GUI checks passed in both modes.
Those results and the previous GUI signoff belong to the earlier archives. Mike’s current source-build signoff and clean-machine shipping-package validation remain open.
The 5 November report recorded the Tutorial 17 TIFF issue and a copper pole-figure tuple-shape issue. The upstream copper correction subsequently passed the packaged comparisons. TIFF issue #1914 · Copper correction #1919
Related Issue
The C15 inventory and deferred scope are documented in the HTML review companion. This is partial coverage; no C15 issue closure is requested.
Related to #1870