diff --git a/docs/Coding_Knowledge_Base.md b/docs/Coding_Knowledge_Base.md index 15885e16cc..ed3cde86cb 100644 --- a/docs/Coding_Knowledge_Base.md +++ b/docs/Coding_Knowledge_Base.md @@ -1147,9 +1147,10 @@ set. **Rule:** a test calls `UnitTest::LoadPlugins();` only when it calls `createFilter()` on the filter list, loads a pipeline (`Pipeline::FromJson()`, `FromFile()`, `FromSIMPLJson()`, `FromSIMPLFile()`; every SIMPL backwards-compatibility test does), or runs a filter whose code gets other filters from the filter list (`ReadH5Ebsd` -does). A test that constructs a filter or algorithm directly does not call it. A test that needs it calls it itself: -ctest runs each visible, individually registered `TEST_CASE` in its own process (a few hidden groups, such as -`[.OocStoreContract]`, run together in one). +does). Each test that needs plugin loading calls it itself. CTest runs each visible, individually registered +`TEST_CASE` in its own process. A few hidden groups, such as `[.OocStoreContract]`, run together in one process. +`UnitTest::PreferencesSentinel` needs an existing `Application` instance. If the test only needs that instance, +call `Application::GetOrCreateInstance()` before constructing the sentinel. **Example of the broken rule:** the synthetic tuple-mismatch cases of `ComputeAvgCAxesTest.cpp` and `ReplaceElementAttributesWithNeighborValuesTest.cpp` load the plugins although they construct the filter directly, and @@ -1158,10 +1159,12 @@ ctest runs each visible, individually registered `TEST_CASE` in its own process **Why:** the call loads every plugin library and fails the test when one does not load, so an unneeded call makes a test fail for a plugin it does not use. -**What to look for:** `UnitTest::LoadPlugins()` in a test with no UUID lookup, pipeline load or registry-dependent -subfilter. +**What to look for:** `UnitTest::LoadPlugins()` in a test with no UUID lookup, pipeline load, registry-dependent +subfilter, or other need for plugin registration. A `PreferencesSentinel` constructed before the `Application` +instance exists dereferences a null pointer (PR #1915). -**Pattern to use:** direct filter construction without plugin loading, as in `test/GeometryTest.cpp`. +**Pattern to use:** construct filters directly as in `test/GeometryTest.cpp`. For a test that also uses a sentinel, +initialize the `Application` directly as in the projection warning cases in `MaximumProjectionImageFilterTest.cpp`. ### Give Catch2 a bool when you compare file times diff --git a/src/Plugins/NXImageProcessing/docs/BinaryProjectionImageFilter.md b/src/Plugins/NXImageProcessing/docs/BinaryProjectionImageFilter.md index 0eafc129e4..d3c441c9af 100644 --- a/src/Plugins/NXImageProcessing/docs/BinaryProjectionImageFilter.md +++ b/src/Plugins/NXImageProcessing/docs/BinaryProjectionImageFilter.md @@ -14,7 +14,7 @@ The **Projection Dimension** selects the collapsed axis: 0 = X, 1 = Y, 2 = Z. The **Foreground Value** and **Background Value** are supplied as floating-point numbers and cast to the input element type; the foreground equality test is performed in the input element type. This matches the legacy ITK Binary Projection accumulator, which initializes each column to background and switches it to foreground the first time a voxel equal to the foreground value is encountered. -When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection); the filter emits a preflight warning listing them. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. +When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection). Preflight warning `-8343` names the discarded objects by their full paths, including other children of the geometry such as Cell Feature Data. There is one exception when the input array is outside Cell Data. A Cell Data array with the same name as the input array is discarded but is not named in the warning. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. The input array must be single-component (scalar) and one of the scalar numeric types (`int8`, `uint8`, `int16`, `uint16`, `int32`, `uint32`, `int64`, `uint64`, `float32`, or `float64`). This is an ITK-free, out-of-core-capable reimplementation of the legacy ITK Binary Projection Image Filter. diff --git a/src/Plugins/NXImageProcessing/docs/MaximumProjectionImageFilter.md b/src/Plugins/NXImageProcessing/docs/MaximumProjectionImageFilter.md index 1d951fd8d8..ca1e8c83c4 100644 --- a/src/Plugins/NXImageProcessing/docs/MaximumProjectionImageFilter.md +++ b/src/Plugins/NXImageProcessing/docs/MaximumProjectionImageFilter.md @@ -12,7 +12,7 @@ Collapses the selected **Projection Dimension** of a scalar Image Geometry to a The **Projection Dimension** selects the collapsed axis: 0 = X, 1 = Y, 2 = Z. -When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection); the filter emits a preflight warning listing them. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. +When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection). Preflight warning `-8343` names the discarded objects by their full paths, including other children of the geometry such as Cell Feature Data. There is one exception when the input array is outside Cell Data. A Cell Data array with the same name as the input array is discarded but is not named in the warning. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. The input array must be single-component (scalar) and one of the supported types: `uint8`, `int16`, `uint16`, or `float32`. This is an ITK-free, out-of-core-capable reimplementation of the legacy ITK Maximum Projection Image Filter. diff --git a/src/Plugins/NXImageProcessing/docs/MeanProjectionImageFilter.md b/src/Plugins/NXImageProcessing/docs/MeanProjectionImageFilter.md index 12c05dcdf2..740cac7c6b 100644 --- a/src/Plugins/NXImageProcessing/docs/MeanProjectionImageFilter.md +++ b/src/Plugins/NXImageProcessing/docs/MeanProjectionImageFilter.md @@ -14,7 +14,7 @@ The **Projection Dimension** selects the collapsed axis: 0 = X, 1 = Y, 2 = Z. The output element type is always **Float64** (double), regardless of the input element type, matching the legacy ITK Mean Projection Image Filter (the mean of an integer column is generally not an integer). The mean is accumulated with a Kahan-compensated running sum so that long columns do not lose precision. -When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection); the filter emits a preflight warning listing them. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. +When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection). Preflight warning `-8343` names the discarded objects by their full paths, including other children of the geometry such as Cell Feature Data. There is one exception when the input array is outside Cell Data. A Cell Data array with the same name as the input array is discarded but is not named in the warning. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. The input array must be single-component (scalar) and one of the scalar numeric types (`int8`, `uint8`, `int16`, `uint16`, `int32`, `uint32`, `int64`, `uint64`, `float32`, or `float64`). This is an ITK-free, out-of-core-capable reimplementation of the legacy ITK Mean Projection Image Filter. diff --git a/src/Plugins/NXImageProcessing/docs/MedianProjectionImageFilter.md b/src/Plugins/NXImageProcessing/docs/MedianProjectionImageFilter.md index 419e89bdff..f4ebce71fe 100644 --- a/src/Plugins/NXImageProcessing/docs/MedianProjectionImageFilter.md +++ b/src/Plugins/NXImageProcessing/docs/MedianProjectionImageFilter.md @@ -12,7 +12,7 @@ Collapses the selected **Projection Dimension** of a scalar Image Geometry to a The **Projection Dimension** selects the collapsed axis: 0 = X, 1 = Y, 2 = Z. -When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection); the filter emits a preflight warning listing them. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. +When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection). Preflight warning `-8343` names the discarded objects by their full paths, including other children of the geometry such as Cell Feature Data. There is one exception when the input array is outside Cell Data. A Cell Data array with the same name as the input array is discarded but is not named in the warning. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. The input array must be single-component (scalar) and one of the supported types: `uint8`, `int16`, `uint16`, or `float32`. This is an ITK-free, out-of-core-capable reimplementation of the legacy ITK Median Projection Image Filter. diff --git a/src/Plugins/NXImageProcessing/docs/MinimumProjectionImageFilter.md b/src/Plugins/NXImageProcessing/docs/MinimumProjectionImageFilter.md index e6af8d6e1e..ae18506d7f 100644 --- a/src/Plugins/NXImageProcessing/docs/MinimumProjectionImageFilter.md +++ b/src/Plugins/NXImageProcessing/docs/MinimumProjectionImageFilter.md @@ -12,7 +12,7 @@ Collapses the selected **Projection Dimension** of a scalar Image Geometry to a The **Projection Dimension** selects the collapsed axis: 0 = X, 1 = Y, 2 = Z. -When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection); the filter emits a preflight warning listing them. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. +When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection). Preflight warning `-8343` names the discarded objects by their full paths, including other children of the geometry such as Cell Feature Data. There is one exception when the input array is outside Cell Data. A Cell Data array with the same name as the input array is discarded but is not named in the warning. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. The input array must be single-component (scalar) and one of the supported types: `uint8`, `int16`, `uint16`, or `float32`. This is an ITK-free, out-of-core-capable reimplementation of the legacy ITK Minimum Projection Image Filter. diff --git a/src/Plugins/NXImageProcessing/docs/ReadImageFilter.md b/src/Plugins/NXImageProcessing/docs/ReadImageFilter.md index 7291b5517e..7ec4d8d756 100644 --- a/src/Plugins/NXImageProcessing/docs/ReadImageFilter.md +++ b/src/Plugins/NXImageProcessing/docs/ReadImageFilter.md @@ -16,6 +16,10 @@ The following image types are supported: - TIFF / TIF, including multi-page TIFF read as a 3D Z-stack (via libtiff) - NRRD (`.nrrd` attached, `.nhdr` detached) (ITK-free, zlib only) +All pages of a multi-page TIFF must have the same dimensions, component count, and scalar type. The filter compares the number of bytes per pixel and the dimensions of each page it reads with page 0. It does not compare component counts or scalar types. A change in component count or scalar type can go undetected when the number of bytes per pixel stays the same. This can cause incorrect pixel values. + +If the first page is 8-bit RGB and a later page is 8-bit grayscale, the filter fails with error `-2007` when it reads that later page. The number of bytes per pixel differs (3 and 1), so it fails before it copies any pixels of that page. This intentionally differs from DREAM3D 6.5.171, which could crash on mixed-page TIFF files (KI-053). + ### NRRD Volumes For a [NRRD](https://teem.sourceforge.net/nrrd/format.html) input (attached single-file `.nrrd`, or detached `.nhdr` + external data file), the geometry is built directly from the NRRD header. Supported header features: diff --git a/src/Plugins/NXImageProcessing/docs/ReadImageStackFilter.md b/src/Plugins/NXImageProcessing/docs/ReadImageStackFilter.md index 026d2056da..96018b79d6 100644 --- a/src/Plugins/NXImageProcessing/docs/ReadImageStackFilter.md +++ b/src/Plugins/NXImageProcessing/docs/ReadImageStackFilter.md @@ -78,6 +78,8 @@ The *Output Data Type* parameter provides the following choices: - **uint16 [1]**: Convert image data to 16-bit unsigned integer. - **uint32 [2]**: Convert image data to 32-bit unsigned integer. +Preflight reads only the first file's image header. During execution, the filter checks each decoded slice kept by the Z crop before further processing or copying it into the stack. When *Set Image Data Type* is off, the first file in the list sets the stack's output type, also when the Z crop removes that file. If a slice cannot match the stack's output type, error `-23534` names the file and the expected and actual types. Earlier slices may already have been written when this error occurs. Mixed uint8 and uint16 slices can be read together when *Set Image Data Type* converts them to the same output type. This intentionally differs from the failure described in DREAM3D 6.5.171 errata KI-051. + ### Cropping Caveats The user can crop the incoming image geometry using the Cropping Options section. The cropping type options are: diff --git a/src/Plugins/NXImageProcessing/docs/StandardDeviationProjectionImageFilter.md b/src/Plugins/NXImageProcessing/docs/StandardDeviationProjectionImageFilter.md index a0be927d65..95f542bf4b 100644 --- a/src/Plugins/NXImageProcessing/docs/StandardDeviationProjectionImageFilter.md +++ b/src/Plugins/NXImageProcessing/docs/StandardDeviationProjectionImageFilter.md @@ -14,7 +14,7 @@ The **Projection Dimension** selects the collapsed axis: 0 = X, 1 = Y, 2 = Z. The standard deviation is the *sample* standard deviation: the column mean is formed with the population divisor *N*, but the sum of squared deviations is divided by *N − 1* before taking the square root, matching the legacy ITK Standard Deviation Projection Image Filter. A column of length 1 (or a degenerate projected axis) yields 0. The output element type is always **Float64** (double), regardless of the input element type. The statistic uses Welford's online update to avoid cancellation when large values have small differences. -When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection); the filter emits a preflight warning listing them. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. +When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection). Preflight warning `-8343` names the discarded objects by their full paths, including other children of the geometry such as Cell Feature Data. There is one exception when the input array is outside Cell Data. A Cell Data array with the same name as the input array is discarded but is not named in the warning. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. The input array must be single-component (scalar) and one of the scalar numeric types (`int8`, `uint8`, `int16`, `uint16`, `int32`, `uint32`, `int64`, `uint64`, `float32`, or `float64`). This is an ITK-free, out-of-core-capable reimplementation of the legacy ITK Standard Deviation Projection Image Filter. diff --git a/src/Plugins/NXImageProcessing/docs/SumProjectionImageFilter.md b/src/Plugins/NXImageProcessing/docs/SumProjectionImageFilter.md index 0e000400ab..dc937ead49 100644 --- a/src/Plugins/NXImageProcessing/docs/SumProjectionImageFilter.md +++ b/src/Plugins/NXImageProcessing/docs/SumProjectionImageFilter.md @@ -14,7 +14,7 @@ The **Projection Dimension** selects the collapsed axis: 0 = X, 1 = Y, 2 = Z. The output element type is always **Float64** (double), regardless of the input element type, matching the legacy ITK Sum Projection Image Filter (the sum of an integer column readily overflows a narrow integer type). The sum is accumulated with a Kahan-compensated running sum so that long columns do not lose precision. -When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection); the filter emits a preflight warning listing them. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. +When **Perform In-Place** is enabled, the input Image Geometry is replaced by the projected (collapsed) geometry containing the output array. Because the whole original geometry is replaced, any *other* arrays that were in its cell-data Attribute Matrix are discarded (they have the original, non-collapsed shape and cannot survive the projection). Preflight warning `-8343` names the discarded objects by their full paths, including other children of the geometry such as Cell Feature Data. There is one exception when the input array is outside Cell Data. A Cell Data array with the same name as the input array is discarded but is not named in the warning. Disable **Perform In-Place** to keep the original geometry: the input geometry is then preserved and the result is written to a newly created geometry named by **Created Image Geometry**, whose cell-data Attribute Matrix name is copied from the input geometry. The input array must be single-component (scalar) and one of the scalar numeric types (`int8`, `uint8`, `int16`, `uint16`, `int32`, `uint32`, `int64`, `uint64`, `float32`, or `float64`). This is an ITK-free, out-of-core-capable reimplementation of the legacy ITK Sum Projection Image Filter. diff --git a/src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp b/src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp index 03fa4e8e90..28232b156f 100644 --- a/src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp +++ b/src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImage.cpp @@ -330,13 +330,11 @@ Result<> ReadRasterBackend(DataStructure& dataStructure, const ReadImageInputVal const usize sourcePage = stream.singlePageOnly ? 0 : (stream.zStart + dz); const usize destSliceBase = dz * sliceElements; - // Per-page geometry check: the destination offsets are computed from page-0-derived dimensions, so a - // page whose dimensions / component count / scalar type differ would silently mis-map. Because the - // IImageIO interface exposes metadata only for page 0, we validate each page against the observed - // extent of its own decoded segments and reject any mismatch rather than mis-sizing the slice. + // Destination offsets use page 0's dimensions and pixel stride. Check each page's decoded segment + // byte counts and observed dimensions against page 0. IImageIO exposes metadata only for page 0, + // so these checks cannot detect different component counts or scalar types with the same byte width. usize observedRows = 0; usize observedColumns = 0; - bool pixelStrideMismatch = false; readResult = imageIO->readPixelDataRows( inputFilePath, @@ -362,7 +360,10 @@ Result<> ReadRasterBackend(DataStructure& dataStructure, const ReadImageInputVal observedColumns = std::max(observedColumns, sourceColumn + pixelCount); if(pixels.size() != pixelCount * bytesPerPixel) { - pixelStrideMismatch = true; + // Reject an incompatible segment before forming a span with page 0's pixel stride. + return MakeErrorResult(-2007, fmt::format("Page {} of '{}' has a pixel byte count that differs from page 0. All pages must have the same dimensions, component count, and " + "scalar type.", + sourcePage, inputFilePath.string())); } // Y crop: skip rows outside the destination window. @@ -408,10 +409,10 @@ Result<> ReadRasterBackend(DataStructure& dataStructure, const ReadImageInputVal return readResult; } - if(observedRows != stream.srcHeight || observedColumns != stream.srcWidth || pixelStrideMismatch) + if(observedRows != stream.srcHeight || observedColumns != stream.srcWidth) { - return MakeErrorResult(-2007, fmt::format("Page {} of '{}' has dimensions {}x{} (or a differing component count / scalar type) that do not match page 0 ({}x{}). Every page of a " - "multi-page volume must share identical dimensions, component count, and scalar type.", + return MakeErrorResult(-2007, fmt::format("Page {} of '{}' has dimensions {}x{} that do not match page 0 ({}x{}). All pages must have the same dimensions, component count, and " + "scalar type.", sourcePage, inputFilePath.string(), observedColumns, observedRows, stream.srcWidth, stream.srcHeight)); } if(singlePage) diff --git a/src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImageStack.cpp b/src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImageStack.cpp index a463278154..c2a1affa2c 100644 --- a/src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImageStack.cpp +++ b/src/Plugins/NXImageProcessing/src/NXImageProcessing/Filters/Algorithms/ReadImageStack.cpp @@ -298,6 +298,13 @@ Result<> ReadImageStackImpl(DataStructure& dataStructure, const ReadImageStackIn } } + DataPath srcImageDataPath = imageGeomPath.createChildPath(cellDataName).createChildPath(imageArrayName); + const DataType sliceType = importedDataStructure.getDataRefAs(srcImageDataPath).getDataType(); + if(sliceType != GetDataType()) + { + return MakeErrorResult(-23534, fmt::format("Slice {} ('{}') has data type {}, but the stack requires {}.", i, filePath, DataTypeToString(sliceType), DataTypeToString(GetDataType()))); + } + if(resample != k_NoResampleModeIndex) { if(resample == k_ScalingModeIndex) @@ -338,8 +345,6 @@ Result<> ReadImageStackImpl(DataStructure& dataStructure, const ReadImageStackIn destImageGeomPath = ResampledTemporaryGeometryPath(imageGeomPath); } - DataPath srcImageDataPath = imageGeomPath.createChildPath(cellDataName).createChildPath(imageArrayName); - bool validInputForGrayScaleConversion = importedDataStructure.getDataRefAs(srcImageDataPath).getDataType() == DataType::uint8; if(convertToGrayscale && validInputForGrayScaleConversion) { diff --git a/src/Plugins/NXImageProcessing/test/MaximumProjectionImageFilterTest.cpp b/src/Plugins/NXImageProcessing/test/MaximumProjectionImageFilterTest.cpp index f230c12200..cdd58058a6 100644 --- a/src/Plugins/NXImageProcessing/test/MaximumProjectionImageFilterTest.cpp +++ b/src/Plugins/NXImageProcessing/test/MaximumProjectionImageFilterTest.cpp @@ -3,6 +3,8 @@ #include "NXImageProcessing/Filters/MaximumProjectionImageFilter.hpp" +#include "simplnx/Core/Application.hpp" + #include using namespace nx::core; @@ -124,3 +126,98 @@ TEST_CASE("NXImageProcessing::MaximumProjectionImageFilter: ITK real-image golde { projection_test::RunProjectionMd5ItkGolden("Ramp-Up-Short.nrrd", /*projDim=*/1, /*removeOriginalGeometry=*/true, "5390344262c91e83bc9208b0991a2fc9"); } + +TEST_CASE("NXImageProcessing::MaximumProjectionImageFilter: Warning names discarded objects", "[NXImageProcessing][MaximumProjectionImageFilter]") +{ + Application::GetOrCreateInstance(); + const UnitTest::PreferencesSentinel prefsSentinel(DataStorageMode::ForceInCore, 0); + // No extra cell array, an extra one, and an old array with the output name (also discarded in place). + const std::string extraName = GENERATE(std::string(""), std::string("Extra"), std::string("Output")); + DataStructure ds; + const DataPath inputPath = ip_test::BuildRampImage(ds, 3, 0.0, 1.0); + const DataPath geomPath({"Image Geometry"}); + auto& imageGeom = ds.getDataRefAs(geomPath); + const DataPath featurePath = geomPath.createChildPath("Cell Feature Data"); + REQUIRE(AttributeMatrix::Create(ds, "Cell Feature Data", {2}, imageGeom.getId()) != nullptr); + const DataPath extraPath = inputPath.getParent().createChildPath(extraName.empty() ? "Extra" : extraName); + if(!extraName.empty()) + { + auto& cellAM = ds.getDataRefAs(inputPath.getParent()); + auto store = DataStoreUtilities::CreateDataStore(ds, extraPath, cellAM.getShape(), {1}); + REQUIRE(Int16Array::Create(ds, extraName, store, cellAM.getId()) != nullptr); + } + MaximumProjectionImageFilter filter; + Arguments args; + args.insertOrAssign(MaximumProjectionImageFilter::k_InputImageGeomPath_Key, std::make_any(geomPath)); + args.insertOrAssign(MaximumProjectionImageFilter::k_InputImageDataPath_Key, std::make_any(inputPath)); + args.insertOrAssign(MaximumProjectionImageFilter::k_RemoveOriginalGeometry_Key, std::make_any(true)); + args.insertOrAssign(MaximumProjectionImageFilter::k_OutputImageArrayName_Key, std::make_any("Output")); + const auto preflightResult = filter.preflight(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + std::string discardedWarning; + for(const auto& warning : preflightResult.outputActions.warnings()) + { + if(warning.code == -8343) + { + discardedWarning += warning.message; + } + } + // The geometry swap deletes the original children. The warning excludes the consumed input, + // but must name the feature matrix even when it is the only other discarded object, using full paths. + CHECK(discardedWarning.find(featurePath.toString()) != std::string::npos); + CHECK(discardedWarning.find(inputPath.toString()) == std::string::npos); + if(!extraName.empty()) + { + CHECK(discardedWarning.find(extraPath.toString()) != std::string::npos); + } + // Axis projection dispatches by store type; it has no Direct/Scanline override. + const auto executeResult = filter.execute(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + CHECK(ds.getData(featurePath) == nullptr); + if(extraName != "Output") + { + CHECK(ds.getData(extraPath) == nullptr); + } + REQUIRE(ds.getDataRefAs(geomPath).getDimensions() == SizeVec3{1, 3, 3}); + const auto& output = ds.getDataRefAs(inputPath.getParent().createChildPath("Output")); + REQUIRE(output.getNumberOfTuples() == 9); + for(usize slot = 0; slot < 9; ++slot) + { + // X columns are {3*s, 3*s+1, 3*s+2}: max = 3*s+2. + CHECK(output[slot] == static_cast(3 * slot + 2)); + } + UnitTest::CheckArraysInheritTupleDims(ds); +} + +TEST_CASE("NXImageProcessing::MaximumProjectionImageFilter: Warning excludes an input outside cell data", "[NXImageProcessing][MaximumProjectionImageFilter]") +{ + Application::GetOrCreateInstance(); + const UnitTest::PreferencesSentinel prefsSentinel(DataStorageMode::ForceInCore, 0); + DataStructure ds; + const DataPath cellInputPath = ip_test::BuildRampImage(ds, 3, 0.0, 1.0); + const DataPath geomPath({"Image Geometry"}); + auto& imageGeom = ds.getDataRefAs(geomPath); + // The selected input is a direct child of the geometry. It has the cell tuple count, so it is accepted. + const DataPath inputPath = geomPath.createChildPath("Geometry Input"); + auto store = DataStoreUtilities::CreateDataStore(ds, inputPath, ShapeType{3, 3, 3}, {1}); + REQUIRE(Float32Array::Create(ds, "Geometry Input", store, imageGeom.getId()) != nullptr); + MaximumProjectionImageFilter filter; + Arguments args; + args.insertOrAssign(MaximumProjectionImageFilter::k_InputImageGeomPath_Key, std::make_any(geomPath)); + args.insertOrAssign(MaximumProjectionImageFilter::k_InputImageDataPath_Key, std::make_any(inputPath)); + args.insertOrAssign(MaximumProjectionImageFilter::k_RemoveOriginalGeometry_Key, std::make_any(true)); + args.insertOrAssign(MaximumProjectionImageFilter::k_OutputImageArrayName_Key, std::make_any("Output")); + const auto preflightResult = filter.preflight(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + std::string discardedWarning; + for(const auto& warning : preflightResult.outputActions.warnings()) + { + if(warning.code == -8343) + { + discardedWarning += warning.message; + } + } + // The cell array is not the input, so it is named. The consumed input is not named. + CHECK(discardedWarning.find(cellInputPath.toString()) != std::string::npos); + CHECK(discardedWarning.find(inputPath.toString()) == std::string::npos); +} diff --git a/src/Plugins/NXImageProcessing/test/ReadImageStackTest.cpp b/src/Plugins/NXImageProcessing/test/ReadImageStackTest.cpp index 68712a55c2..314c0a4750 100644 --- a/src/Plugins/NXImageProcessing/test/ReadImageStackTest.cpp +++ b/src/Plugins/NXImageProcessing/test/ReadImageStackTest.cpp @@ -30,6 +30,7 @@ #include #include #include +#include #include #include #include @@ -1489,6 +1490,119 @@ TEST_CASE("NXImageProcessing::ReadImageStackFilter::MultiPage_FirstPagePerFile", REQUIRE(hasMultiPageWarning); } +TEST_CASE("NXImageProcessing::ReadImageStackFilter::MixedSliceDataTypes", "[NXImageProcessing][ReadImageStackFilter]") +{ + const bool firstSliceIsUint16 = GENERATE(false, true); + CAPTURE(firstSliceIsUint16); + const fs::path inputDir = fs::path(std::string(unit_test::k_BinaryTestOutputDir.view())) / "ReadImageStackMixedSliceDataTypes"; + fs::create_directories(inputDir); + for(usize slice = 0; slice < 2; ++slice) + { + const fs::path path = inputDir / fmt::format("slice_{}.tif", slice); + if((slice == 0) == firstSliceIsUint16) + { + TIFF* tif = TIFFOpen(path.string().c_str(), "w"); + REQUIRE(tif != nullptr); + REQUIRE(TIFFSetField(tif, TIFFTAG_IMAGEWIDTH, 2) != 0); + REQUIRE(TIFFSetField(tif, TIFFTAG_IMAGELENGTH, 2) != 0); + REQUIRE(TIFFSetField(tif, TIFFTAG_SAMPLESPERPIXEL, 1) != 0); + REQUIRE(TIFFSetField(tif, TIFFTAG_BITSPERSAMPLE, 16) != 0); + REQUIRE(TIFFSetField(tif, TIFFTAG_SAMPLEFORMAT, SAMPLEFORMAT_UINT) != 0); + REQUIRE(TIFFSetField(tif, TIFFTAG_ORIENTATION, ORIENTATION_TOPLEFT) != 0); + REQUIRE(TIFFSetField(tif, TIFFTAG_PLANARCONFIG, PLANARCONFIG_CONTIG) != 0); + REQUIRE(TIFFSetField(tif, TIFFTAG_PHOTOMETRIC, PHOTOMETRIC_MINISBLACK) != 0); + std::vector pixels = {1000, 1000, 1000, 1000}; + REQUIRE(TIFFWriteScanline(tif, pixels.data(), 0, 0) >= 0); + REQUIRE(TIFFWriteScanline(tif, pixels.data() + 2, 1, 0) >= 0); + TIFFClose(tif); + } + else + { + WriteMultiPageTiff(path, 2, 2, {{10, 10, 10, 10}}); + } + } + + GeneratedFileListParameter::ValueType fileList; + fileList.inputPath = inputDir.string(); + fileList.filePrefix = "slice_"; + fileList.fileSuffix = ""; + fileList.fileExtension = ".tif"; + fileList.startIndex = 0; + fileList.endIndex = 1; + fileList.incrementIndex = 1; + fileList.paddingDigits = 1; + fileList.ordering = GeneratedFileListParameter::Ordering::LowToHigh; + + ReadImageStackFilter filter; + DataStructure ds; + Arguments args; + args.insertOrAssign(ReadImageStackFilter::k_InputFileListInfo_Key, std::make_any(fileList)); + args.insertOrAssign(ReadImageStackFilter::k_ImageGeometryPath_Key, std::make_any(k_ImageGeomPath)); + + SECTION("Reject mismatched slice data types") + { + // With conversion disabled, uint8 and uint16 cannot share a typed stack array. + // Preflight reads only file 0. Execute must report the mismatched slice without throwing. + DataStructure preflightDs; + IFilter::PreflightResult preflightResult; + REQUIRE_NOTHROW(preflightResult = filter.preflight(preflightDs, args)); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + + IFilter::ExecuteResult executeResult; + REQUIRE_NOTHROW(executeResult = filter.execute(ds, args)); + SIMPLNX_RESULT_REQUIRE_INVALID(executeResult.result); + const auto& sliceError = executeResult.result.errors().front(); + CAPTURE(sliceError.code, sliceError.message); + CHECK(sliceError.code == -23534); + CHECK(sliceError.message.find("slice_1.tif") != std::string::npos); + CHECK(sliceError.message.find("uint8") != std::string::npos); + CHECK(sliceError.message.find("uint16") != std::string::npos); + } + + SECTION("Preflight does not read later slice headers") + { + // An existing file with an invalid TIFF header distinguishes header I/O from file-list checks. + { + std::ofstream invalidSlice(inputDir / "slice_1.tif", std::ios::binary | std::ios::trunc); + REQUIRE(invalidSlice.is_open()); + invalidSlice << "not a TIFF"; + REQUIRE(invalidSlice.good()); + } + auto preflightResult = filter.preflight(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + } + + SECTION("Explicit conversion permits mixed types") + { + args.insertOrAssign(ReadImageStackFilter::k_ChangeDataType_Key, true); + args.insertOrAssign(ReadImageStackFilter::k_ImageDataType_Key, std::make_any(1)); + auto executeResult = filter.execute(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + const auto& store = ds.getDataRefAs(k_ImageDataPath).getDataStoreRef(); + REQUIRE(store.getNumberOfTuples() == 8); + // Read Image's normalized conversion gives 10 * (65535 / 255) = 2570; + // uint16(1000) remains 1000 when the source and destination types match. + for(usize slice = 0; slice < 2; ++slice) + { + const uint16_t expected = ((slice == 0) == firstSliceIsUint16) ? 1000 : 2570; + for(usize pixel = 0; pixel < 4; ++pixel) + { + REQUIRE(store[slice * 4 + pixel] == expected); + } + } + } + + SECTION("Z crop excludes the mismatched slice") + { + auto crop = CreateCropOptions(CropGeometryParameter::CropValues::TypeEnum::VoxelSubvolume, false, false, true); + crop.zBoundVoxels = {0, 0}; + args.insertOrAssign(ReadImageStackFilter::k_CroppingOptions_Key, std::make_any(crop)); + auto executeResult = filter.execute(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + VerifyGeometryDimensions(ds, k_ImageGeomPath, 2, 2, 1); + } +} + namespace { /** diff --git a/src/Plugins/NXImageProcessing/test/ReadImageTest.cpp b/src/Plugins/NXImageProcessing/test/ReadImageTest.cpp index 5563e3a539..c52ea28ed9 100644 --- a/src/Plugins/NXImageProcessing/test/ReadImageTest.cpp +++ b/src/Plugins/NXImageProcessing/test/ReadImageTest.cpp @@ -736,6 +736,80 @@ TEST_CASE("NXImageProcessing::ReadImageFilter: MultiPage_TIFF_ZStack", "[NXImage } } +TEST_CASE("NXImageProcessing::ReadImageFilter: MultiPage_TIFF_RGB_Then_Grayscale_Rejected_Before_Copy", "[NXImageProcessing][ReadImageFilter]") +{ + const bool convertData = GENERATE(false, true); + CAPTURE(convertData); + const fs::path tiffPath = ReadImageOutputDir() / "rgb_then_grayscale.tif"; + constexpr uint32_t width = 3; + constexpr uint32_t height = 2; + { + std::unique_ptr tiff(TIFFOpen(tiffPath.string().c_str(), "w"), TIFFClose); + REQUIRE(tiff != nullptr); + for(const uint16_t components : {uint16_t{3}, uint16_t{1}}) + { + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_IMAGEWIDTH, width) == 1); + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_IMAGELENGTH, height) == 1); + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_BITSPERSAMPLE, 8) == 1); + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_SAMPLEFORMAT, SAMPLEFORMAT_UINT) == 1); + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_SAMPLESPERPIXEL, components) == 1); + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_PHOTOMETRIC, components == 3 ? PHOTOMETRIC_RGB : PHOTOMETRIC_MINISBLACK) == 1); + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_ORIENTATION, ORIENTATION_TOPLEFT) == 1); + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_PLANARCONFIG, PLANARCONFIG_CONTIG) == 1); + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_COMPRESSION, COMPRESSION_NONE) == 1); + REQUIRE(TIFFSetField(tiff.get(), TIFFTAG_ROWSPERSTRIP, height) == 1); + std::vector row(width * components, components == 3 ? uint8_t{255} : uint8_t{127}); + for(uint32_t y = 0; y < height; ++y) + { + REQUIRE(TIFFWriteScanline(tiff.get(), row.data(), y, 0) == 1); + } + REQUIRE(TIFFWriteDirectory(tiff.get()) == 1); + } + } + + const DataPath geomPath({"Mixed Pages"}); + const DataPath arrayPath = geomPath.createChildPath("Cell Data").createChildPath("Pixels"); + Arguments args; + args.insertOrAssign(ReadImageFilter::k_FileName_Key, tiffPath); + args.insertOrAssign(ReadImageFilter::k_ImageGeometryPath_Key, geomPath); + args.insertOrAssign(ReadImageFilter::k_CellDataName_Key, std::string("Cell Data")); + args.insertOrAssign(ReadImageFilter::k_ImageDataArrayPath_Key, std::string("Pixels")); + args.insertOrAssign(ReadImageFilter::k_ChangeOrigin_Key, false); + args.insertOrAssign(ReadImageFilter::k_ChangeSpacing_Key, false); + args.insertOrAssign(ReadImageFilter::k_ChangeDataType_Key, convertData); + args.insertOrAssign(ReadImageFilter::k_ImageDataType_Key, ChoicesParameter::ValueType{1}); + ReadImageFilter filter; + DataStructure dataStructure; + auto preflight = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflight.outputActions); + IFilter::ExecuteResult execute; + REQUIRE_NOTHROW(execute = filter.execute(dataStructure, args)); + SIMPLNX_RESULT_REQUIRE_INVALID(execute.result); + REQUIRE(execute.result.errors()[0].code == -2007); // The page-layout check must reject page 1, not another error path. + REQUIRE(dataStructure.getDataRefAs(geomPath).getDimensions() == SizeVec3{width, height, 2}); + + // Oracle: page 0 has 3*2*3 = 18 white components (255, or 65535 after conversion). + // Page 1 supplies only 3 bytes per row, but the RGB destination needs 3*3 = 9. + // Reject that segment before copying any bytes; its 18 destination components must + // retain their initial zero values. This detects a late error even without a sanitizer. + const auto checkPixels = [](const auto& pixels, auto white) { + REQUIRE(pixels.getSize() == 36); + for(usize i = 0; i < 18; ++i) + { + REQUIRE(pixels[i] == white); + REQUIRE(pixels[18 + i] == 0); + } + }; + if(convertData) + { + checkPixels(dataStructure.getDataRefAs(arrayPath), uint16_t{65535}); + } + else + { + checkPixels(dataStructure.getDataRefAs(arrayPath), uint8_t{255}); + } +} + TEST_CASE("NXImageProcessing::ReadImageFilter: Nrrd_Scalar3D", "[NXImageProcessing][ReadImageFilter]") { UnitTest::LoadPlugins(); diff --git a/src/Plugins/NXImageProcessing/test/StandardDeviationProjectionImageFilterTest.cpp b/src/Plugins/NXImageProcessing/test/StandardDeviationProjectionImageFilterTest.cpp index 4a28a79810..e8df7c469f 100644 --- a/src/Plugins/NXImageProcessing/test/StandardDeviationProjectionImageFilterTest.cpp +++ b/src/Plugins/NXImageProcessing/test/StandardDeviationProjectionImageFilterTest.cpp @@ -3,6 +3,7 @@ #include "NXImageProcessing/Filters/StandardDeviationProjectionImageFilter.hpp" +#include "simplnx/Core/Application.hpp" #include "simplnx/Utilities/ImageProcessing/AxisProjectionEngine.hpp" #include "simplnx/Utilities/ImageProcessing/ProjectionReducers.hpp" @@ -264,3 +265,65 @@ TEST_CASE("NXImageProcessing::StandardDeviationProjectionImageFilter: ITK real-i projection_test::RunProjectionBaselineItkGolden("RA-Float.nrrd", "BasicFilters_StandardDeviationProjectionImageFilter_z_projection.nrrd", /*projDim=*/2, /*tolerance=*/0.0001); } + +TEST_CASE("NXImageProcessing::StandardDeviationProjectionImageFilter: Warning names discarded objects", "[NXImageProcessing][StandardDeviationProjectionImageFilter]") +{ + Application::GetOrCreateInstance(); + const UnitTest::PreferencesSentinel prefsSentinel(DataStorageMode::ForceInCore, 0); + // No extra cell array, an extra one, and an old array with the output name (also discarded in place). + const std::string extraName = GENERATE(std::string(""), std::string("Extra"), std::string("Output")); + DataStructure ds; + const DataPath inputPath = ip_test::BuildRampImage(ds, 3, 0.0, 1.0); + const DataPath geomPath({"Image Geometry"}); + auto& imageGeom = ds.getDataRefAs(geomPath); + const DataPath featurePath = geomPath.createChildPath("Cell Feature Data"); + REQUIRE(AttributeMatrix::Create(ds, "Cell Feature Data", {2}, imageGeom.getId()) != nullptr); + const DataPath extraPath = inputPath.getParent().createChildPath(extraName.empty() ? "Extra" : extraName); + if(!extraName.empty()) + { + auto& cellAM = ds.getDataRefAs(inputPath.getParent()); + auto store = DataStoreUtilities::CreateDataStore(ds, extraPath, cellAM.getShape(), {1}); + REQUIRE(Int16Array::Create(ds, extraName, store, cellAM.getId()) != nullptr); + } + StandardDeviationProjectionImageFilter filter; + Arguments args; + args.insertOrAssign(StandardDeviationProjectionImageFilter::k_InputImageGeomPath_Key, std::make_any(geomPath)); + args.insertOrAssign(StandardDeviationProjectionImageFilter::k_InputImageDataPath_Key, std::make_any(inputPath)); + args.insertOrAssign(StandardDeviationProjectionImageFilter::k_RemoveOriginalGeometry_Key, std::make_any(true)); + args.insertOrAssign(StandardDeviationProjectionImageFilter::k_OutputImageArrayName_Key, std::make_any("Output")); + const auto preflightResult = filter.preflight(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + std::string discardedWarning; + for(const auto& warning : preflightResult.outputActions.warnings()) + { + if(warning.code == -8343) + { + discardedWarning += warning.message; + } + } + // The geometry swap deletes the original children. The warning excludes the consumed input, + // but must name the feature matrix even when it is the only other discarded object, using full paths. + CHECK(discardedWarning.find(featurePath.toString()) != std::string::npos); + CHECK(discardedWarning.find(inputPath.toString()) == std::string::npos); + if(!extraName.empty()) + { + CHECK(discardedWarning.find(extraPath.toString()) != std::string::npos); + } + // Axis projection dispatches by store type; it has no Direct/Scanline override. + const auto executeResult = filter.execute(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + CHECK(ds.getData(featurePath) == nullptr); + if(extraName != "Output") + { + CHECK(ds.getData(extraPath) == nullptr); + } + REQUIRE(ds.getDataRefAs(geomPath).getDimensions() == SizeVec3{1, 3, 3}); + const auto& output = ds.getDataRefAs(inputPath.getParent().createChildPath("Output")); + REQUIRE(output.getNumberOfTuples() == 9); + for(usize slot = 0; slot < 9; ++slot) + { + // X columns are {3*s, 3*s+1, 3*s+2}: sample variance = (1+0+1)/(3-1) = 1. + CHECK(output[slot] == Approx(1.0).margin(1.0e-12)); + } + UnitTest::CheckArraysInheritTupleDims(ds); +} diff --git a/src/Plugins/OrientationAnalysis/docs/ReadH5EspritDataFilter.md b/src/Plugins/OrientationAnalysis/docs/ReadH5EspritDataFilter.md index b84da017a3..cb4e393259 100644 --- a/src/Plugins/OrientationAnalysis/docs/ReadH5EspritDataFilter.md +++ b/src/Plugins/OrientationAnalysis/docs/ReadH5EspritDataFilter.md @@ -12,6 +12,8 @@ Each measurement is placed at the grid cell named by its own `X BEAM` and `Y BEA The scan stores an **orientation** at every pixel as three **Euler angles** (three angles, in the Bunge Z-X-Z convention, describing how the measured crystal at that pixel is rotated relative to the sample). +The **Scan Names** stacking order controls placement along the **Image Geometry** Z direction. **Low to High** places the first selected scan at the lowest Z slice. **High to Low** places it at the highest Z slice and places the last selected scan at the lowest Z slice. A single scan has the same placement in either order. High to Low output intentionally differs from DREAM3D 6.5.171, which ignored this setting (errata KI-011). + | User interface before entering a proper "Z Spacing" value and selecting which scans to include. | |-------| |![](Images/ReadEspritH5_1.png)| diff --git a/src/Plugins/OrientationAnalysis/docs/ReadH5OimDataFilter.md b/src/Plugins/OrientationAnalysis/docs/ReadH5OimDataFilter.md index 10f8082243..2c58275789 100644 --- a/src/Plugins/OrientationAnalysis/docs/ReadH5OimDataFilter.md +++ b/src/Plugins/OrientationAnalysis/docs/ReadH5OimDataFilter.md @@ -33,7 +33,9 @@ the user should **CHECK** the option **ON** for "Perform Slice By Slice Transfor ## Algorithm -Each selected scan is appended in selection order along the **Image Geometry** Z direction. Phase values are clamped and Euler components are interleaved into their output array; scalar measurements, including X and Y positions, are copied at that scan's tuple offset. Pattern data uses the same tuple offset multiplied by its pattern component count. Output arrays are written in bounded bulk transfers, which supports out-of-core output storage. +The **Scan Names** stacking order controls placement along the **Image Geometry** Z direction. **Low to High** places the first selected scan at the lowest Z slice. **High to Low** places it at the highest Z slice and places the last selected scan at the lowest Z slice. A single scan has the same placement in either order. High to Low output intentionally differs from DREAM3D 6.5.171, which ignored this setting (errata KI-011). + +Phase values are clamped and Euler components are interleaved into their output array; scalar measurements, including X and Y positions, are copied at that scan's tuple offset. Pattern data uses the same tuple offset multiplied by its pattern component count. Output arrays are written in bounded bulk transfers, which supports out-of-core output storage. ### X Position and Y Position diff --git a/src/Plugins/OrientationAnalysis/docs/WriteINLFileFilter.md b/src/Plugins/OrientationAnalysis/docs/WriteINLFileFilter.md index 99a1f86077..4c36826de5 100644 --- a/src/Plugins/OrientationAnalysis/docs/WriteINLFileFilter.md +++ b/src/Plugins/OrientationAnalysis/docs/WriteINLFileFilter.md @@ -18,7 +18,9 @@ This **Filter** writes out **Cell** data from an **Image Geometry** to a file fo - Phase Id -- the ensemble (phase) each cell belongs to - Symmetry -- the crystal-symmetry code for the cell's phase -The **Symmetry** column holds the integer crystal-symmetry code corresponding to the cell's crystal structure (for example, *43* denotes cubic m-3m and *62* denotes hexagonal 6/mmm). The codes follow the EBSDLib `CrystalStructure` enumeration and are taken from the **Crystal Structures** ensemble array. +The **Symmetry** column holds the TSL phase-symmetry code for the cell's crystal structure. For example, *43* denotes cubic m-3m and *62* denotes hexagonal 6/mmm. The filter converts the **Crystal Structures** values to these codes. The phase headers and data rows use the same mapping for all 11 Laue classes. Unknown or unsupported structures use *999*. Cells with phase zero also use *999*. + +This intentionally differs from DREAM.3D 6.5.171. That version wrote *999* in data rows for every Laue class except cubic-high and hexagonal-high. Some information about the phase is included in the header section of the file in addition to values for the origin, step size, and dimensions. The origin, **step sizes** (`X_STEP`, `Y_STEP`, `Z_STEP`), and min/max positions in the header are all in **microns**. The header also reports the number of **Features** in the file. diff --git a/src/Plugins/OrientationAnalysis/docs/WriteStatsGenOdfAngleFileFilter.md b/src/Plugins/OrientationAnalysis/docs/WriteStatsGenOdfAngleFileFilter.md index 9fe88d1efd..863710a565 100644 --- a/src/Plugins/OrientationAnalysis/docs/WriteStatsGenOdfAngleFileFilter.md +++ b/src/Plugins/OrientationAnalysis/docs/WriteStatsGenOdfAngleFileFilter.md @@ -62,6 +62,8 @@ The *Delimiter* parameter provides the following choices: - **: (colon) [3]**: Uses a colon as the column separator. - **\t (tab) [4]**: Uses a tab character as the column separator. +The Colon option now writes `:` between values. This intentionally differs from DREAM3D 6.5.171, which wrote double quotes for this option (errata KI-015). + ## Required Input Sources - **Euler Angles** -- typically read from EBSD data via [Read H5EBSD File](ReadH5EbsdFilter.md), [Read EDAX EBSD Data (.ang)](ReadAngDataFilter.md), or [Read Oxford Instr. EBSD Data (.ctf)](ReadCtfDataFilter.md). Supplied in radians (Bunge Z-X-Z convention); use the *Convert to Degrees* option only if **StatsGenerator** will import them as degrees. diff --git a/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/ReadH5EspritData.hpp b/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/ReadH5EspritData.hpp index 58912514de..e3f7480223 100644 --- a/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/ReadH5EspritData.hpp +++ b/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/ReadH5EspritData.hpp @@ -59,7 +59,7 @@ class ORIENTATIONANALYSIS_EXPORT ReadH5EspritData : public IEbsdOemReader copyRawEbsdData(int sliceIndex) override; diff --git a/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/WriteINLFile.cpp b/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/WriteINLFile.cpp index 85ecf7eb16..4360999eb0 100644 --- a/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/WriteINLFile.cpp +++ b/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/WriteINLFile.cpp @@ -30,7 +30,7 @@ constexpr usize k_EulerComponents = 3; /** * @brief Maps a simplnx crystal structure to the INL TSL symmetry code. * @param symmetry Simplnx crystal-structure value. - * @return Matching TSL phase symmetry, or UnknownCrystalStructure when unsupported. + * @return Matching TSL phase symmetry, or UnknownSymmetry when unsupported. */ uint32 mapCrystalSymmetryToTslSymmetry(uint32 symmetry) { @@ -59,7 +59,7 @@ uint32 mapCrystalSymmetryToTslSymmetry(uint32 symmetry) case ebsdlib::CrystalStructure::Trigonal_Low: return ebsdlib::Ang::PhaseSymmetry::Trigonal; default: - return ebsdlib::CrystalStructure::UnknownCrystalStructure; + return ebsdlib::Ang::PhaseSymmetry::UnknownSymmetry; } } } // namespace @@ -303,26 +303,7 @@ Result<> WriteINLFile::operator()() const float64 yPos = origin[1] + (static_cast(y) * res[1]); const float64 zPos = origin[2] + (static_cast(z) * res[2]); const int32 phaseId = cellPhasesChunk[chunkIndex]; - uint32 symmetry = crystalStructuresCache[phaseId]; - if(phaseId > 0) - { - if(symmetry == ebsdlib::CrystalStructure::Cubic_High) - { - symmetry = ebsdlib::Ang::PhaseSymmetry::Cubic; - } - else if(symmetry == ebsdlib::CrystalStructure::Hexagonal_High) - { - symmetry = ebsdlib::Ang::PhaseSymmetry::DiHexagonal; - } - else - { - symmetry = ebsdlib::Ang::PhaseSymmetry::UnknownSymmetry; - } - } - else - { - symmetry = ebsdlib::Ang::PhaseSymmetry::UnknownSymmetry; - } + const uint32 symmetry = phaseId > 0 ? mapCrystalSymmetryToTslSymmetry(crystalStructuresCache[phaseId]) : ebsdlib::Ang::PhaseSymmetry::UnknownSymmetry; textStream << phi1 << " " << phi << " " << phi2 << " " << xPos << " " << yPos << " " << zPos << " " << featureIdsChunk[chunkIndex] << " " << phaseId << " " << symmetry << "\n"; diff --git a/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/WriteStatsGenOdfAngleFile.cpp b/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/WriteStatsGenOdfAngleFile.cpp index 04fb2e6bf4..7ba831caed 100644 --- a/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/WriteStatsGenOdfAngleFile.cpp +++ b/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/WriteStatsGenOdfAngleFile.cpp @@ -26,9 +26,7 @@ namespace { constexpr usize k_ChunkTuples = 65536; constexpr usize k_EulerComponents = 3; -// Choice three is labeled Colon but currently writes a double quote. Preserve -// this mismatch until output-format behavior is changed explicitly. -constexpr std::array k_Delimiters = {',', ';', ' ', '"', '\t'}; +constexpr std::array k_Delimiters = {',', ';', ' ', ':', '\t'}; const std::array k_DelimiterStr = {"Comma", "Semicolon", "Space", "Colon", "Tab"}; void WriteHeader(std::ofstream& out, const WriteStatsGenOdfAngleFileInputValues& inputValues, int32 lineCount) diff --git a/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/utilities/IEbsdOemReader.hpp b/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/utilities/IEbsdOemReader.hpp index 8f4da4686d..df3d7fde2b 100644 --- a/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/utilities/IEbsdOemReader.hpp +++ b/src/Plugins/OrientationAnalysis/src/OrientationAnalysis/utilities/IEbsdOemReader.hpp @@ -73,7 +73,7 @@ class ORIENTATIONANALYSIS_EXPORT IEbsdOemReader IEbsdOemReader& operator=(IEbsdOemReader&&) noexcept = delete; /** - * @brief Loads and copies each selected scan in selection order. + * @brief Loads scans in selection order and copies them to the requested Z stacking order. * @return Reader, phase-metadata, or derived copy errors. * * The method sets the destination geometry unit to micrometers before import. @@ -100,7 +100,8 @@ class ORIENTATIONANALYSIS_EXPORT IEbsdOemReader return readResults; } - Result<> copyDataResults = copyRawEbsdData(index); + const int sliceIndex = m_InputValues->SelectedScanNames.stackingOrder == RefFrameZDir::k_HightoLow ? static_cast(scanNames.size()) - 1 - index : index; + Result<> copyDataResults = copyRawEbsdData(sliceIndex); if(copyDataResults.invalid()) { return copyDataResults; @@ -177,7 +178,7 @@ class ORIENTATIONANALYSIS_EXPORT IEbsdOemReader /** * @brief Copies the currently loaded scan into its destination volume range. - * @param index Zero-based scan index. + * @param index Zero-based destination Z slice. execute() computes it from the selection index and the stacking order. * @return Source conversion or destination transfer errors. */ virtual Result<> copyRawEbsdData(int index) = 0; diff --git a/src/Plugins/OrientationAnalysis/test/ReadH5OimDataTest.cpp b/src/Plugins/OrientationAnalysis/test/ReadH5OimDataTest.cpp index 90e250b1a3..9d6a02d6bb 100644 --- a/src/Plugins/OrientationAnalysis/test/ReadH5OimDataTest.cpp +++ b/src/Plugins/OrientationAnalysis/test/ReadH5OimDataTest.cpp @@ -17,6 +17,8 @@ #include #include +#include + #include #include #include @@ -83,6 +85,86 @@ void RequireAcquisitionCoordinates(const DataStructure& dataStructure, usize sca } } // namespace +TEST_CASE("OrientationAnalysis::ReadH5OimDataFilter: Scan Stacking Order", "[OrientationAnalysis][ReadH5OimDataFilter]") +{ + const UnitTest::TestFileSentinel testDataSentinel(unit_test::k_TestFilesDir, "7_read_oem_ebsd_h5_files_v3.tar.gz", "7_read_oem_ebsd_h5_files_v3"); + const auto sourceFile = fs::path(fmt::format("{}/7_read_oem_ebsd_h5_files_v3/EdaxOIMData.h5", unit_test::k_TestFilesDir)); + const auto inputFile = fs::path(unit_test::k_BinaryTestOutputDir.view()) / "ReadH5OimData_StackingOrder.h5"; + fs::copy_file(sourceFile, inputFile, fs::copy_options::overwrite_existing); + + constexpr usize k_TuplesPerScan = 186 * 151; + { + const hid_t fileId = H5Fopen(inputFile.string().c_str(), H5F_ACC_RDWR, H5P_DEFAULT); + REQUIRE(fileId >= 0); + H5Support::H5ScopedFileSentinel sentinel(fileId, true); + // The downloaded scans have identical IQ values. Give scan s the exact + // integer-valued signal IQ(s, p) = 100000*s + p, where p is the pixel index. + for(int32 scan = 1; scan <= 3; scan++) + { + std::vector values(k_TuplesPerScan); + for(usize pixel = 0; pixel < k_TuplesPerScan; pixel++) + { + values[pixel] = static_cast(100000 * scan + pixel); + } + const std::string datasetPath = fmt::format("/Scan {}/EBSD/Data/IQ", scan); + const hid_t datasetId = H5Dopen2(fileId, datasetPath.c_str(), H5P_DEFAULT); + REQUIRE(datasetId >= 0); + const herr_t writeResult = H5Dwrite(datasetId, H5T_NATIVE_FLOAT, H5S_ALL, H5S_ALL, H5P_DEFAULT, values.data()); + REQUIRE(H5Dclose(datasetId) >= 0); + REQUIRE(writeResult >= 0); + } + } + + OEMEbsdScanSelectionParameter::ValueType scanSelections = {inputFile, RefFrameZDir::k_LowtoHigh, {k_ScanName_2, k_ScanName_1, k_ScanName_3}}; + std::vector expectedScans; + // For N selected scans, selection index i occupies Z slice i (Low to High) + // or N-1-i (High to Low). Thus [2,1,3] becomes [2,1,3] or [3,1,2] in Z. + SECTION("Low to High") + { + expectedScans = {2, 1, 3}; + } + SECTION("High to Low") + { + scanSelections.stackingOrder = RefFrameZDir::k_HightoLow; + expectedScans = {3, 1, 2}; + } + SECTION("High to Low single scan") + { + scanSelections.stackingOrder = RefFrameZDir::k_HightoLow; + scanSelections.scanNames = {k_ScanName_2}; + expectedScans = {2}; + } + + DataStructure dataStructure; + ReadH5OimDataFilter filter; + Arguments args; + args.insertOrAssign(ReadH5OimDataFilter::k_SelectedScanNames_Key, std::make_any(scanSelections)); + args.insertOrAssign(ReadH5OimDataFilter::k_ZSpacing_Key, std::make_any(1.0f)); + args.insertOrAssign(ReadH5OimDataFilter::k_Origin_Key, std::make_any(std::vector(3, 0.0f))); + args.insertOrAssign(ReadH5OimDataFilter::k_ReadPatternData_Key, std::make_any(false)); + const DataPath geometryPath({ImageGeom::k_TypeName}); + args.insertOrAssign(ReadH5OimDataFilter::k_CreatedImageGeometryPath_Key, std::make_any(geometryPath)); + args.insertOrAssign(ReadH5OimDataFilter::k_CellAttributeMatrixName_Key, std::make_any(k_Cell_Data)); + args.insertOrAssign(ReadH5OimDataFilter::k_CellEnsembleAttributeMatrixName_Key, std::make_any(k_Cell_Ensemble_Data)); + + const auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions) + const auto executeResult = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result) + REQUIRE(dataStructure.getDataRefAs(geometryPath).getDimensions() == SizeVec3{186, 151, expectedScans.size()}); + const auto& imageQuality = dataStructure.getDataRefAs(geometryPath.createChildPath(k_Cell_Data).createChildPath(ebsdlib::Ang::ImageQuality)); + REQUIRE(imageQuality.getNumberOfTuples() == k_TuplesPerScan * expectedScans.size()); + for(usize slice = 0; slice < expectedScans.size(); slice++) + { + for(usize pixel = 0; pixel < k_TuplesPerScan; pixel++) + { + CAPTURE(slice, pixel); + REQUIRE(imageQuality[slice * k_TuplesPerScan + pixel] == static_cast(100000 * expectedScans[slice] + pixel)); + } + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("OrientationAnalysis::ReadH5OimDataFilter: Acquisition Coordinates", "[OrientationAnalysis][ReadH5OimDataFilter]") { UnitTest::LoadPlugins(); diff --git a/src/Plugins/OrientationAnalysis/test/WriteINLFileTest.cpp b/src/Plugins/OrientationAnalysis/test/WriteINLFileTest.cpp index a999007341..f8ceb3da30 100644 --- a/src/Plugins/OrientationAnalysis/test/WriteINLFileTest.cpp +++ b/src/Plugins/OrientationAnalysis/test/WriteINLFileTest.cpp @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -258,6 +259,116 @@ TEST_CASE("OrientationAnalysis::WriteINLFileFilter: Valid Filter Execution", "[O UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("OrientationAnalysis::WriteINLFileFilter: All Laue symmetry codes", "[OrientationAnalysis][WriteINLFileFilter]") +{ + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + CAPTURE(scenario); + UnitTest::AlgorithmTestScope scope(scenario); + + // Independent oracle: the TSL Laue codes (AngConstants.h) are m-3m=43, m-3=23, + // 4/mmm=42, 4/m=4, mmm=22, 2/m (c axis)=2, -1=1, 6/mmm=62, 6/m=6, -3m=32, + // and -3=3. Unknown/unsupported structures and phase zero must use TSL unknown=999. + // Give phase zero a known structure to verify that it remains an unknown phase. + const std::array structures = {ebsdlib::CrystalStructure::Cubic_High, + ebsdlib::CrystalStructure::Cubic_High, + ebsdlib::CrystalStructure::Cubic_Low, + ebsdlib::CrystalStructure::Tetragonal_High, + ebsdlib::CrystalStructure::Tetragonal_Low, + ebsdlib::CrystalStructure::OrthoRhombic, + ebsdlib::CrystalStructure::Monoclinic, + ebsdlib::CrystalStructure::Triclinic, + ebsdlib::CrystalStructure::Hexagonal_High, + ebsdlib::CrystalStructure::Hexagonal_Low, + ebsdlib::CrystalStructure::Trigonal_High, + ebsdlib::CrystalStructure::Trigonal_Low, + ebsdlib::CrystalStructure::UnknownCrystalStructure, + 12345}; + const std::array expectedCodes = {999, 43, 23, 42, 4, 22, 2, 1, 62, 6, 32, 3, 999, 999}; + const usize numCells = structures.size(); + + DataStructure dataStructure; + auto* imageGeom = ImageGeom::Create(dataStructure, "ImageGeometry"); + REQUIRE(imageGeom != nullptr); + imageGeom->setDimensions({numCells, 1, 1}); + imageGeom->setSpacing({1.0F, 1.0F, 1.0F}); + imageGeom->setOrigin({0.0F, 0.0F, 0.0F}); + auto* cellData = AttributeMatrix::Create(dataStructure, "Cell Data", {1, 1, numCells}, imageGeom->getId()); + REQUIRE(cellData != nullptr); + imageGeom->setCellData(*cellData); + auto* featureIds = Int32Array::CreateWithStore(dataStructure, "FeatureIds", {1, 1, numCells}, {1}, cellData->getId()); + auto* phases = Int32Array::CreateWithStore(dataStructure, "Phases", {1, 1, numCells}, {1}, cellData->getId()); + auto* eulers = Float32Array::CreateWithStore(dataStructure, "EulerAngles", {1, 1, numCells}, {3}, cellData->getId()); + REQUIRE(featureIds != nullptr); + REQUIRE(phases != nullptr); + REQUIRE(eulers != nullptr); + eulers->fill(0.0F); + auto* crystalStructures = UInt32Array::CreateWithStore(dataStructure, "CrystalStructures", {numCells}, {1}); + auto* numFeatures = Int32Array::CreateWithStore(dataStructure, "NumFeatures", {numCells}, {1}); + REQUIRE(crystalStructures != nullptr); + REQUIRE(numFeatures != nullptr); + numFeatures->fill(1); + REQUIRE(StringArray::CreateWithValues(dataStructure, "Materials", {numCells}, std::vector(numCells, "Phase")) != nullptr); + for(usize i = 0; i < numCells; ++i) + { + (*featureIds)[i] = static_cast(i); + (*phases)[i] = static_cast(i); + (*crystalStructures)[i] = structures[i]; + } + + const fs::path outputPath = fs::temp_directory_path() / "nx_all_laue_inl_writer.inl"; + auto outputFileGuard = MakeScopeGuard([&outputPath]() noexcept { + std::error_code errorCode; + fs::remove(outputPath, errorCode); + }); + WriteINLFileFilter filter; + Arguments args = filter.getDefaultArguments(); + args.insertOrAssign(WriteINLFileFilter::k_OutputFile_Key, std::make_any(outputPath)); + args.insertOrAssign(WriteINLFileFilter::k_ImageGeomPath_Key, std::make_any(DataPath({"ImageGeometry"}))); + args.insertOrAssign(WriteINLFileFilter::k_FeatureIdsArrayPath_Key, std::make_any(DataPath({"ImageGeometry", "Cell Data", "FeatureIds"}))); + args.insertOrAssign(WriteINLFileFilter::k_CellPhasesArrayPath_Key, std::make_any(DataPath({"ImageGeometry", "Cell Data", "Phases"}))); + args.insertOrAssign(WriteINLFileFilter::k_CellEulerAnglesArrayPath_Key, std::make_any(DataPath({"ImageGeometry", "Cell Data", "EulerAngles"}))); + args.insertOrAssign(WriteINLFileFilter::k_CrystalStructuresArrayPath_Key, std::make_any(DataPath({"CrystalStructures"}))); + args.insertOrAssign(WriteINLFileFilter::k_NumFeaturesArrayPath_Key, std::make_any(DataPath({"NumFeatures"}))); + args.insertOrAssign(WriteINLFileFilter::k_MaterialNameArrayPath_Key, std::make_any(DataPath({"Materials"}))); + const auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + const auto executeResult = scope.executeFilter(filter, dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + + std::ifstream inputStream(outputPath); + REQUIRE(inputStream.is_open()); + usize rowCount = 0; + usize headerCount = 0; + std::string line; + while(std::getline(inputStream, line)) + { + if(line.rfind("# Symmetry_", 0) == 0) + { + ++headerCount; + REQUIRE(headerCount < numCells); + CAPTURE(headerCount); + CHECK(line == fmt::format("# Symmetry_{}: {}", headerCount, expectedCodes[headerCount])); + } + if(line.empty() || line[0] == '#') + { + continue; + } + REQUIRE(rowCount < numCells); + CAPTURE(rowCount); + std::istringstream row(line); + float64 phi1 = 0, phi = 0, phi2 = 0, x = 0, y = 0, z = 0; + int32 featureId = -1, phaseId = -1; + uint32 symmetry = 0; + REQUIRE(static_cast(row >> phi1 >> phi >> phi2 >> x >> y >> z >> featureId >> phaseId >> symmetry)); + CHECK(phaseId == static_cast(rowCount)); + CHECK(symmetry == expectedCodes[rowCount]); + ++rowCount; + } + CHECK(headerCount == numCells - 1); + CHECK(rowCount == numCells); + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("OrientationAnalysis::WriteINLFileFilter: cancellation leaves the existing output untouched", "[OrientationAnalysis][WriteINLFileFilter]") { UnitTest::LoadPlugins(); diff --git a/src/Plugins/OrientationAnalysis/test/WriteStatsGenOdfAngleFileTest.cpp b/src/Plugins/OrientationAnalysis/test/WriteStatsGenOdfAngleFileTest.cpp index e4b201b137..772ae3dd2c 100644 --- a/src/Plugins/OrientationAnalysis/test/WriteStatsGenOdfAngleFileTest.cpp +++ b/src/Plugins/OrientationAnalysis/test/WriteStatsGenOdfAngleFileTest.cpp @@ -206,6 +206,51 @@ TEST_CASE("OrientationAnalysis::WriteStatsGenOdfAngleFileFilter: analytical Bool UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("OrientationAnalysis::WriteStatsGenOdfAngleFileFilter: Colon delimiter writes colon separated rows", "[OrientationAnalysis][WriteStatsGenOdfAngleFileFilter]") +{ + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + CAPTURE(scenario); + UnitTest::AlgorithmTestScope scope(scenario); + + DataStructure dataStructure = CreateAnalyticalDataStructure(); + scope.requireExpectedStore(dataStructure.getDataRefAs(k_AnalyticalEulersPath)); + + const fs::path outputFile = fs::path(fmt::format("{}/StatsGenODF_Colon.txt", unit_test::k_BinaryTestOutputDir)); + WriteStatsGenOdfAngleFileFilter filter; + Arguments args = filter.getDefaultArguments(); + args.insertOrAssign(WriteStatsGenOdfAngleFileFilter::k_OutputFile_Key, std::make_any(outputFile)); + args.insertOrAssign(WriteStatsGenOdfAngleFileFilter::k_Weight_Key, std::make_any(2.5F)); + args.insertOrAssign(WriteStatsGenOdfAngleFileFilter::k_Sigma_Key, std::make_any(3)); + args.insertOrAssign(WriteStatsGenOdfAngleFileFilter::k_Delimiter_Key, std::make_any(WriteStatsGenOdfAngleFileFilter::k_ColonDelimiter)); + args.insertOrAssign(WriteStatsGenOdfAngleFileFilter::k_ConvertToDegrees_Key, std::make_any(true)); + args.insertOrAssign(WriteStatsGenOdfAngleFileFilter::k_UseMask_Key, std::make_any(true)); + args.insertOrAssign(WriteStatsGenOdfAngleFileFilter::k_CellEulerAnglesArrayPath_Key, std::make_any(k_AnalyticalEulersPath)); + args.insertOrAssign(WriteStatsGenOdfAngleFileFilter::k_CellPhasesArrayPath_Key, std::make_any(k_AnalyticalPhasesPath)); + args.insertOrAssign(WriteStatsGenOdfAngleFileFilter::k_MaskArrayPath_Key, std::make_any(k_AnalyticalBoolMaskPath)); + + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions) + auto executeResult = scope.executeFilter(filter, dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result) + + // The mask keeps (0, pi/2, pi) in phase 1 and (pi/2, 0, pi/2) in phase 2. + // Multiplying by 180/pi gives (0, 90, 180) and (90, 0, 90) degrees. + // Append the supplied weight 2.5 and sigma 3, separating all five values with ':'. + const std::array expectedRows = {"0.00000000:90.00000000:180.00000000:2.50000000:3", "90.00000000:0.00000000:90.00000000:2.50000000:3"}; + for(usize phaseIndex = 0; phaseIndex < expectedRows.size(); phaseIndex++) + { + CAPTURE(phaseIndex); + const auto lines = ReadLines(outputFile.parent_path() / fmt::format("StatsGenODF_Colon_Phase_{}.txt", phaseIndex + 1)); + REQUIRE(lines.size() == 8); + CHECK(lines[3] == "# Angle Data is Colon delimited."); + CHECK(lines[4] == "# Euler angles are expressed in degrees"); + CHECK(lines[6] == "Angle Count:1"); + CHECK(lines[7] == expectedRows[phaseIndex]); + } + + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("OrientationAnalysis::WriteStatsGenOdfAngleFileFilter: InValid Filter Execution", "[OrientationAnalysis][WriteStatsGenOdfAngleFileFilter]") { UnitTest::LoadPlugins(); diff --git a/src/Plugins/SimplnxCore/docs/AddBadDataFilter.md b/src/Plugins/SimplnxCore/docs/AddBadDataFilter.md index e3be25073c..5d37c9795d 100644 --- a/src/Plugins/SimplnxCore/docs/AddBadDataFilter.md +++ b/src/Plugins/SimplnxCore/docs/AddBadDataFilter.md @@ -32,6 +32,8 @@ This means a volume fraction of 0.2 in boundary-only mode introduces far fewer b To flag a cell as bad, the filter sets the corresponding tuple in every **Attribute Array** in the same **Attribute Matrix** as the *Boundary Euclidean Distances* input to **0**. This includes Feature Ids, Phases, Quaternions, and any other cell-level data sharing that matrix. Be aware that "0" may be a legitimate value for some arrays (e.g., a phase index) and that the convention here is shared with experimental cleanup filters that interpret Feature Id 0 as bad. +The filter rejects StringArray and NeighborList objects in the geometry's Cell Data Attribute Matrix during preflight. The error identifies the array to move. DREAM3D 6.5.171 did not check these types before execution. + ### Prerequisite Steps This filter needs to know which cells are near a feature boundary. That information comes from a Manhattan-distance map, which must be computed and converted before this filter runs: diff --git a/src/Plugins/SimplnxCore/docs/AlignSectionsFeatureCentroidFilter.md b/src/Plugins/SimplnxCore/docs/AlignSectionsFeatureCentroidFilter.md index 6a3ea5284f..06a45b5efc 100644 --- a/src/Plugins/SimplnxCore/docs/AlignSectionsFeatureCentroidFilter.md +++ b/src/Plugins/SimplnxCore/docs/AlignSectionsFeatureCentroidFilter.md @@ -24,6 +24,12 @@ Unlike the misorientation-based and mutual-information-based alignment methods, If *Use Reference Slice* is enabled, each section is shifted to align with a single fixed reference slice rather than its immediate neighbor. This prevents small errors from accumulating across many sections. +When **Use Reference Slice** is enabled, **Reference Slice** must be between 0 and Z-1, where Z is the geometry's Z dimension. Preflight rejects larger values with error `-68073`. + +The highest-Z section is also shifted when the reference slice is not 0. This corrects DREAM3D 6.5.171 behavior, which left that section unchanged. + +An empty mask section has no finite centroid. If either centroid used for a shift is not finite, the relative shift is zero. In consecutive mode, the cumulative shift stays equal to the previous section's shift. In reference mode, the section stays unchanged. This defines behavior that was undefined in DREAM3D 6.5.171. + ## Algorithm The algorithm proceeds in two phases: diff --git a/src/Plugins/SimplnxCore/docs/ApplyTransformationToGeometryFilter.md b/src/Plugins/SimplnxCore/docs/ApplyTransformationToGeometryFilter.md index 605f6ea49e..a6a15ded40 100644 --- a/src/Plugins/SimplnxCore/docs/ApplyTransformationToGeometryFilter.md +++ b/src/Plugins/SimplnxCore/docs/ApplyTransformationToGeometryFilter.md @@ -21,6 +21,8 @@ For Image Geometries, transformation requires re-gridding because cell positions - **Interpolation Method** -- how cell data values are sampled from the old grid (see below). - **Cell Attribute Matrix** -- which attribute matrix holds the cell data to transform. +For transforms classified as grid rotations, each output axis uses its matched source axis spacing and scale. This also applies when uneven scaling includes a small shear within the grid tolerance. For example, the transform (x,y,z) to (x+5y,100000y,z) changes spacing (2,3,5) to (2,300000,5). + ### Image Geometry Caveat: Successive Transformations Applying multiple transformations one-at-a-time to an Image Geometry produces visible re-gridding artifacts because each intermediate step re-samples the data. The following example shows a 90-degree rotation done as a single step versus as two 45-degree steps: diff --git a/src/Plugins/SimplnxCore/docs/ComputeFeatureClusteringFilter.md b/src/Plugins/SimplnxCore/docs/ComputeFeatureClusteringFilter.md index e458a27f95..29302600b4 100644 --- a/src/Plugins/SimplnxCore/docs/ComputeFeatureClusteringFilter.md +++ b/src/Plugins/SimplnxCore/docs/ComputeFeatureClusteringFilter.md @@ -22,6 +22,8 @@ The **Filter** also outputs the clustering list (every inter-**Feature** distanc *Note:* Because the algorithm iterates over all **Features**, each distance is double-counted. For example, the distance from **Feature** 1 to **Feature** 2 is counted along with the identical distance from **Feature** 2 to **Feature** 1. +The random reference includes all pairs of distinct sample indices among its 1,000 sample points. DREAM3D 6.5.171 omitted pairs involving the first sample point. It omitted 1,998 of the 999,000 ordered reference pairs (0.2%). The correction can lower RDF values. The change in each bin depends on how many reference pairs were missing from that bin. + ### Bin Count **Number of Bins for RDF** is a dimensionless count and must be at least 1. Preflight rejects zero and negative counts. diff --git a/src/Plugins/SimplnxCore/docs/ComputeVectorColorsFilter.md b/src/Plugins/SimplnxCore/docs/ComputeVectorColorsFilter.md index a20d3b26c9..e9af209df6 100644 --- a/src/Plugins/SimplnxCore/docs/ComputeVectorColorsFilter.md +++ b/src/Plugins/SimplnxCore/docs/ComputeVectorColorsFilter.md @@ -12,6 +12,8 @@ This filter generates an RGB color for each **Cell** based on the vector assigne When *Apply to Good Voxels Only* is enabled, the supplied boolean or uint8 **Mask** array marks which **Cells** are valid; **Cells** flagged as bad are left black (RGB 0,0,0) instead of being colored. +A trend that rounds to 360 degrees selects the red hue, the same as 0 degrees. The plunge then blends this hue toward white. For horizontal vectors, this corrects the black output in DREAM3D 6.5.171 to red. + ### Required Input Sources - **Vector Attribute Array** -- a 3-component float32 per-**Cell** vector array. This may come from any filter that produces a per-**Cell** vector field (for example, a dipole, gradient, or orientation-derived vector array). diff --git a/src/Plugins/SimplnxCore/docs/ExecuteProcessFilter.md b/src/Plugins/SimplnxCore/docs/ExecuteProcessFilter.md index 497deff2a7..3987ba39e9 100644 --- a/src/Plugins/SimplnxCore/docs/ExecuteProcessFilter.md +++ b/src/Plugins/SimplnxCore/docs/ExecuteProcessFilter.md @@ -18,6 +18,8 @@ Simple command with arguments (for Windows users) > C:/Applications/DREAM3D-NX/nxrunner.exe --execute C:/Applications/DREAM3D-NX/Data/MyPipeline.d3dpipeline +Spaces between arguments are ignored. Spaces inside double quotes are preserved. This corrects DREAM3D 6.5.171 behavior, which could add a leading space to an argument after a quoted argument or repeated spaces. + For a command with spaces in the path > "C:/Program Files/DREAM3D-NX/nxrunner.exe" --execute "C:/Program Files/DREAM3D-NX/pipelines/OrientationAnalysis/Small_IN100_Processing/(01) Small IN100 Archive.d3dpipeline" diff --git a/src/Plugins/SimplnxCore/docs/FeatureFaceCurvatureFilter.md b/src/Plugins/SimplnxCore/docs/FeatureFaceCurvatureFilter.md index a8d7f03b19..bbc73366a3 100644 --- a/src/Plugins/SimplnxCore/docs/FeatureFaceCurvatureFilter.md +++ b/src/Plugins/SimplnxCore/docs/FeatureFaceCurvatureFilter.md @@ -16,7 +16,7 @@ The principal directions can have their signs flipped: they come from eigenvecto *Note*: Computing the Weingarten matrix values is an experimental feature, and there is no guarantee at this time that the values are correct. -The fitting neighborhood contains the triangle itself and its neighbors within the **Neighborhood Ring Count**. Only neighbors with the same pair of **Face Labels** are included. If this neighborhood initially contains fewer than two triangles, the filter returns an error. Otherwise, triangles with a NaN component in their face normal or face centroid are excluded from the fit. If the evaluated triangle is excluded, all of its requested outputs are NaN. The outputs are also NaN if this exclusion leaves fewer than two triangles. The evaluated triangle counts toward this minimum. +The fitting neighborhood contains the triangle itself and its neighbors within the **Neighborhood Ring Count**. Only neighbors with the same pair of **Face Labels** are included. If this neighborhood contains fewer than two triangles, all requested outputs for that triangle are NaN and the filter continues. This includes a Feature face with only one triangle. This differs from DREAM3D 6.5.171, which wrote invalid values for one-triangle faces. Triangles with a NaN component in their face normal or face centroid are excluded from the fit. If the evaluated triangle is excluded, all of its requested outputs are NaN. The outputs are also NaN if this exclusion leaves fewer than two triangles. The evaluated triangle counts toward this minimum. ### Units diff --git a/src/Plugins/SimplnxCore/docs/QuickSurfaceMeshFilter.md b/src/Plugins/SimplnxCore/docs/QuickSurfaceMeshFilter.md index f735d65805..9a3c949d43 100644 --- a/src/Plugins/SimplnxCore/docs/QuickSurfaceMeshFilter.md +++ b/src/Plugins/SimplnxCore/docs/QuickSurfaceMeshFilter.md @@ -28,6 +28,11 @@ The resulting mesh is "blocky" because each triangle aligns with a voxel face. T The user may select any number of **Cell Attribute Arrays** to transfer onto the new **Triangle Geometry**. Each output **Face** inherits the values of the **Cell** it came from. Only scalar and vector arrays can be transferred; multi-dimensional component shapes (e.g., N×M matrices) are not supported. +Selected feature arrays are transferred in the same way. Each **Face** has two sides, so a transferred array +with N components per tuple has 2N components per **Face**. The first N components come from the side of +Face Labels[0]. The last N components come from the side of Face Labels[1]. In DREAM3D 6.5.171, the two +halves were reversed on some internal faces. This filter corrects that, so its output is different there. + ### Face Labels Convention Each triangle gets a 2-component *Face Labels* attribute storing the two Feature IDs on either side of the boundary. The filter guarantees the smaller Feature ID is in component 0, so downstream filters can rely on a consistent ordering. diff --git a/src/Plugins/SimplnxCore/docs/ReadDREAM3DFilter.md b/src/Plugins/SimplnxCore/docs/ReadDREAM3DFilter.md index 4ad0f9161f..857188f87b 100644 --- a/src/Plugins/SimplnxCore/docs/ReadDREAM3DFilter.md +++ b/src/Plugins/SimplnxCore/docs/ReadDREAM3DFilter.md @@ -12,6 +12,8 @@ This filter reads data from a `.dream3d` file into the pipeline. A `.dream3d` fi The filter does not have to read the entire file. Through the *Import File Path* parameter, the user can browse the structure of the selected `.dream3d` file and choose exactly which data objects to import. Selecting only the objects that are needed avoids loading large arrays that the rest of the pipeline does not use, which saves memory and time. To import everything, simply select all objects. +If an **Include List** path does not exist in the file, preflight reports a warning that names the missing path. Existing ancestors of that path are still imported. They may be empty groups. DREAM3D 6.5.171 did not report this warning. + ### How Imported Objects Are Added The selected objects are added into the pipeline's current **DataStructure** at the same paths they had in the file. If an object being imported has the same path as an object already present in the current **DataStructure**, the existing object is replaced (overwritten) by the imported one, so take care when reading into a non-empty **DataStructure**. Any pipeline metadata stored in the file may also be read when the file is opened. diff --git a/src/Plugins/SimplnxCore/docs/ReadStlFileFilter.md b/src/Plugins/SimplnxCore/docs/ReadStlFileFilter.md index a6a0c961d4..6d397a6ba6 100644 --- a/src/Plugins/SimplnxCore/docs/ReadStlFileFilter.md +++ b/src/Plugins/SimplnxCore/docs/ReadStlFileFilter.md @@ -29,6 +29,8 @@ The filter inspects the header to try to determine the vendor of the STL file, b ## IMPORTANT NOTES +After validating the binary header and triangle count, preflight checks the minimum file size: 80 header bytes, 4 count bytes, and 50 bytes for each declared triangle. If this size check fails, preflight returns a file-length error before mesh arrays are allocated. This differs from DREAM3D 6.5.171, where a corrupt triangle count can cause a very large allocation or an allocation failure. + The reader follows the STL specification strictly and obeys the "Attribute byte count" value written after each triangle. If you are writing an STL file, set the "Attribute byte count" to *zero* (0) unless you are deliberately encoding extra data after each triangle, in which case the count must exactly match the number of extra bytes. An incorrect "Attribute byte count" is the most common cause of read failures, because the reader will skip the number of bytes the file claims are present. ## Known Vendors Who Write Out-of-Spec STL Files diff --git a/src/Plugins/SimplnxCore/docs/ReadVtkStructuredPointsFilter.md b/src/Plugins/SimplnxCore/docs/ReadVtkStructuredPointsFilter.md index 1ef98e6366..dff6524521 100644 --- a/src/Plugins/SimplnxCore/docs/ReadVtkStructuredPointsFilter.md +++ b/src/Plugins/SimplnxCore/docs/ReadVtkStructuredPointsFilter.md @@ -6,7 +6,7 @@ IO (Input) ## Description -This filter reads a *STRUCTURED_POINTS* 3D array from a legacy `.vtk` file and stores it in an **Image Geometry** (a regular grid of equally-sized voxels). A *STRUCTURED_POINTS* file is a general form of an **Image Geometry** in which data values can be attached either to each voxel or to the corner points of each voxel. The file can be either binary or ASCII. The currently supported VTK attribute types are *SCALARS* and *VECTORS*; other attribute types are not read correctly and may cause the filter to fail. +This filter reads a *STRUCTURED_POINTS* grid from a legacy `.vtk` file and stores it in an **Image Geometry** (a regular grid of equally-sized voxels). A *STRUCTURED_POINTS* file is a general form of an **Image Geometry** in which data values can be attached either to each voxel or to the corner points of each voxel. The file can be either binary or ASCII. The currently supported VTK attribute types are *SCALARS* and *VECTORS*; other attribute types are not read correctly and may cause the filter to fail. ### Point Data vs. Cell Data @@ -23,6 +23,10 @@ When both are read, two independent outputs are created: the **Cell Data** is pl The grid dimensions, spacing, and origin are read directly from the `.vtk` file's `DIMENSIONS`, `SPACING`, and `ORIGIN` records. The spacing and origin are plain numbers with no embedded unit, so their length unit is whatever the file's author intended (dimensionless to the reader). +Each cell dimension is one less than its point dimension, with a minimum of one. This supports 2D planes, 1D lines, and a single point. For example, `DIMENSIONS 10 10 1` has 81 cells in a `9 9 1` grid. Zero point dimensions return an error during preflight. These behaviors intentionally differ from DREAM3D 6.5.171, which rejects valid cell data on singleton axes and can underflow on zero dimensions. + +On each axis with one point, both output geometries use the file origin minus half the spacing to keep the voxel center at the point's physical position. + ### Example Input # vtk DataFile Version 2.0 diff --git a/src/Plugins/SimplnxCore/docs/RotateSampleRefFrameFilter.md b/src/Plugins/SimplnxCore/docs/RotateSampleRefFrameFilter.md index 52134fcaf6..6a31a158bc 100644 --- a/src/Plugins/SimplnxCore/docs/RotateSampleRefFrameFilter.md +++ b/src/Plugins/SimplnxCore/docs/RotateSampleRefFrameFilter.md @@ -42,6 +42,8 @@ The *Perform Slice By Slice Transform* option applies the rotation independently ## Notes +Spacing follows the source axis for each output axis. For example, a 120 degree rotation about (111) maps (x,y,z) to (z,x,y). Input spacing (2,3,5) becomes (5,2,3). This corrects DREAM3D 6.5.171 errata KI-008. The output intentionally differs from 6.5.171 for affected anisotropic rotations. + The rotation will most likely produce an origin that differs from the input geometry's origin (the transformed bounding box is repositioned). The *Keep Input Geometry's Origin* option preserves the original origin instead; by default it is OFF and the transform-derived origin is used. To reset the origin and spacing afterward, use the [Set Origin & Spacing](SetImageGeomOriginScalingFilter.md) filter. ## Example diff --git a/src/Plugins/SimplnxCore/docs/WriteOnScaleTableFileFilter.md b/src/Plugins/SimplnxCore/docs/WriteOnScaleTableFileFilter.md index 3ee712a04e..e563727ddc 100644 --- a/src/Plugins/SimplnxCore/docs/WriteOnScaleTableFileFilter.md +++ b/src/Plugins/SimplnxCore/docs/WriteOnScaleTableFileFilter.md @@ -46,7 +46,7 @@ The `name` data line in the generated file ends with one ASCII space after `Phas The legacy DREAM.3D exporter required dimensions in non-increasing order (X >= Y >= Z), and this filter keeps that rule. If an **Image Geometry** does not have this order, the filter determines the required 90-degree axis rotations and runs the Rotate Sample Reference Frame **Filter** internally. The exported dimensions, spacing, origin, and feature IDs come from the reordered copy. The input **Data Structure** does not change. -The filter validates two-axis reorders after rotation and reports an error if the rotated dimensions are not the sorted input dimensions. This problem occurs with anisotropic spacing because of a limitation in the current Rotate Sample Reference Frame implementation; resample to isotropic spacing or reorder the axes before this filter. +Two-axis reorders preserve anisotropic spacing and all cells. This corrects DREAM3D 6.5.171 errata KI-008. The filter still checks that the rotated dimensions match the sorted input dimensions before export. A **Rectilinear Grid Geometry** with dimensions that require reordering is not supported. The filter rejects this geometry during preflight because a 90-degree image rotation cannot preserve its non-uniform bounds arrays. diff --git a/src/Plugins/SimplnxCore/docs/WriteVtkRectilinearGridFilter.md b/src/Plugins/SimplnxCore/docs/WriteVtkRectilinearGridFilter.md index 39612f039c..fcbc6320e1 100644 --- a/src/Plugins/SimplnxCore/docs/WriteVtkRectilinearGridFilter.md +++ b/src/Plugins/SimplnxCore/docs/WriteVtkRectilinearGridFilter.md @@ -16,10 +16,14 @@ The user selects which **Cell** **Data Array**s from the **Image Geometry** are The *Write Binary File* parameter controls the on-disk encoding. When *false* (the default), the data is written as plain ASCII text, which is portable and easy to inspect but larger and slower to read. When *true*, the data is written in VTK's binary form, producing smaller files that load faster. +ASCII coordinates retain the precision of the computed float values, including small spacing such as `1e-7`. This corrects the six-decimal rounding in DREAM3D 6.5.171, so ASCII coordinate output can differ from that version. + +Arrays are written as legacy VTK `SCALARS`, which supports 1 to 4 components. Preflight rejects arrays with more than 4 components, including 6- and 9-component arrays. This corrects DREAM3D 6.5.171 behavior, which wrote these arrays with invalid `SCALARS` headers. + ### Required Input Sources - **Image Geometry** -- the geometry whose dimensions and spacing define the grid. -- **Cell Data Array(s) to write** -- any per-cell arrays on the **Image Geometry**. A common choice is a *FeatureIds* array produced by [Segment Features (Scalar)](ScalarSegmentFeaturesFilter.md). +- **Cell Data Array(s) to write** -- any per-cell arrays with 1 to 4 components on the **Image Geometry**. A common choice is a *FeatureIds* array produced by [Segment Features (Scalar)](ScalarSegmentFeaturesFilter.md). % Auto generated parameter table will be inserted here diff --git a/src/Plugins/SimplnxCore/docs/WriteVtkStructuredPointsFilter.md b/src/Plugins/SimplnxCore/docs/WriteVtkStructuredPointsFilter.md index 3ad3e1e31c..a150414e9f 100644 --- a/src/Plugins/SimplnxCore/docs/WriteVtkStructuredPointsFilter.md +++ b/src/Plugins/SimplnxCore/docs/WriteVtkStructuredPointsFilter.md @@ -20,10 +20,12 @@ Use **Write Vtk Structured Points** when the **Image Geometry** has uniform spac The *Write Binary File* parameter controls the on-disk encoding. When *false* (the default), the data is written as plain ASCII text, which is portable and easy to inspect but larger and slower to read. When *true*, the data is written in VTK's binary form, producing smaller files that load faster. +Arrays are written as legacy VTK `SCALARS`, which supports 1 to 4 components. Preflight rejects arrays with more than 4 components, including 6- and 9-component arrays. This corrects DREAM3D 6.5.171 behavior, which wrote these arrays with invalid `SCALARS` headers. + ### Required Input Sources - **Image Geometry** -- the geometry whose origin, spacing, and dimensions define the grid. -- **Cell Data Array(s) to write** -- any per-cell arrays on the **Image Geometry**. A common choice is a *FeatureIds* array produced by [Segment Features (Scalar)](ScalarSegmentFeaturesFilter.md). +- **Cell Data Array(s) to write** -- any per-cell arrays with 1 to 4 components on the **Image Geometry**. A common choice is a *FeatureIds* array produced by [Segment Features (Scalar)](ScalarSegmentFeaturesFilter.md). % Auto generated parameter table will be inserted here diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/AddBadDataFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/AddBadDataFilter.cpp index d982fa74fa..aa5f2e92e4 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/AddBadDataFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/AddBadDataFilter.cpp @@ -15,6 +15,7 @@ #include "simplnx/Parameters/NumberParameter.hpp" #include "simplnx/Utilities/DataArrayUtilities.hpp" +#include "simplnx/Utilities/DataGroupUtilities.hpp" #include "simplnx/Utilities/FilterUtilities.hpp" #include "simplnx/Utilities/SIMPLConversion.hpp" @@ -175,6 +176,20 @@ IFilter::PreflightResult AddBadDataFilter::preflightImpl(const DataStructure& da return {MakeErrorResult(-76237, fmt::format("Image geometry must have a valid cell Attribute Matrix"))}; } + const auto childArrayPaths = GetAllChildArrayDataPaths(dataStructure, imgGeomPtr->getCellDataPath()); + if(childArrayPaths.has_value()) + { + for(const auto& arrayPath : childArrayPaths.value()) + { + if(dataStructure.getDataAs(arrayPath) == nullptr) + { + return {MakeErrorResult( + -76239, fmt::format("Cell array '{}' is not a numeric DataArray. Move StringArray and NeighborList objects out of the geometry's cell Attribute Matrix before adding bad data.", + arrayPath.toString()))}; + } + } + } + { auto createAction = std::make_unique(DataType::uint64, std::vector{1}, std::vector{1}, DataPath({pSeedArrayNameValue})); resultOutputActions.value().appendAction(std::move(createAction)); diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/AlignSectionsFeatureCentroid.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/AlignSectionsFeatureCentroid.cpp index ad5f327065..d016d5ac7a 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/AlignSectionsFeatureCentroid.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/AlignSectionsFeatureCentroid.cpp @@ -1,4 +1,5 @@ #include +#include #include "AlignSectionsFeatureCentroid.hpp" @@ -11,6 +12,19 @@ using namespace nx::core; +namespace +{ +int64 ComputeCentroidShift(float32 centroid, float32 referenceCentroid, float32 spacing) +{ + // Empty mask slices have NaN centroids and contribute no relative shift. + if(!std::isfinite(centroid) || !std::isfinite(referenceCentroid)) + { + return 0; + } + return static_cast((centroid - referenceCentroid) / spacing); +} +} // namespace + AlignSectionsFeatureCentroid::AlignSectionsFeatureCentroid(DataStructure& dataStructure, const IFilter::MessageHandler& mesgHandler, const std::atomic_bool& shouldCancel, AlignSectionsFeatureCentroidInputValues* inputValues) : AlignSections(dataStructure, shouldCancel, mesgHandler) @@ -127,22 +141,22 @@ Result<> AlignSectionsFeatureCentroid::findShifts(std::vector& xShifts, s auto& centroidsStore = m_DataStructure.getDataAs(m_InputValues->CentroidsArrayPath)->getDataStoreRef(); // Convert physical centroid differences to integer voxel offsets. - for(usize iter = 1; iter < dims[2]; iter++) + for(usize iter = (m_InputValues->UseReferenceSlice && m_InputValues->ReferenceSlice != 0) ? 0 : 1; iter < dims[2]; iter++) { slice = (dims[2] - 1) - iter; if(m_InputValues->UseReferenceSlice) { // Reference-relative and cumulative shifts are identical in this mode. - relativexshift = static_cast((xCentroid[iter] - xCentroid[static_cast(m_InputValues->ReferenceSlice)]) / spacing[0]); - relativeyshift = static_cast((yCentroid[iter] - yCentroid[static_cast(m_InputValues->ReferenceSlice)]) / spacing[1]); + relativexshift = ComputeCentroidShift(xCentroid[iter], xCentroid[static_cast(m_InputValues->ReferenceSlice)], spacing[0]); + relativeyshift = ComputeCentroidShift(yCentroid[iter], yCentroid[static_cast(m_InputValues->ReferenceSlice)], spacing[1]); xShifts[iter] = relativexshift; yShifts[iter] = relativeyshift; } else { // Accumulate consecutive-slice offsets. - relativexshift = static_cast((xCentroid[iter] - xCentroid[iter - 1]) / spacing[0]); - relativeyshift = static_cast((yCentroid[iter] - yCentroid[iter - 1]) / spacing[1]); + relativexshift = ComputeCentroidShift(xCentroid[iter], xCentroid[iter - 1], spacing[0]); + relativeyshift = ComputeCentroidShift(yCentroid[iter], yCentroid[iter - 1], spacing[1]); xShifts[iter] = xShifts[iter - 1] + relativexshift; yShifts[iter] = yShifts[iter - 1] + relativeyshift; } @@ -165,13 +179,13 @@ Result<> AlignSectionsFeatureCentroid::findShifts(std::vector& xShifts, s } if(std::isnan(xCentroid[iter]) && !xWarning) { - std::string message = fmt::format("The X Centroid was NaN. All subsequent slices are probably wrong. Slice=", iter); + std::string message = fmt::format("The X Centroid was NaN. All subsequent slices are probably wrong. Slice={}", iter); m_MessageHandler.sendInfoMessage(message); xWarning = true; } if(std::isnan(yCentroid[iter]) && !yWarning) { - std::string message = fmt::format("The Y Centroid was NaN. All subsequent slices are probably wrong. Slice=", iter); + std::string message = fmt::format("The Y Centroid was NaN. All subsequent slices are probably wrong. Slice={}", iter); m_MessageHandler.sendInfoMessage(message); yWarning = true; } @@ -179,7 +193,7 @@ Result<> AlignSectionsFeatureCentroid::findShifts(std::vector& xShifts, s usize xIndex = iter * 2; usize yIndex = (iter * 2) + 1; slicesStore[xIndex] = slice; - slicesStore[yIndex] = slice + 1; + slicesStore[yIndex] = iter == 0 ? dims[2] - 1 - static_cast(m_InputValues->ReferenceSlice) : slice + 1; relativeShiftsStore[xIndex] = relativexshift; relativeShiftsStore[yIndex] = relativeyshift; cumulativeShiftsStore[xIndex] = xShifts[iter]; @@ -191,17 +205,17 @@ Result<> AlignSectionsFeatureCentroid::findShifts(std::vector& xShifts, s else { // Compute shifts without writing diagnostic arrays. - for(usize iter = 1; iter < dims[2]; iter++) + for(usize iter = (m_InputValues->UseReferenceSlice && m_InputValues->ReferenceSlice != 0) ? 0 : 1; iter < dims[2]; iter++) { if(m_InputValues->UseReferenceSlice) { - xShifts[iter] = static_cast((xCentroid[iter] - xCentroid[static_cast(m_InputValues->ReferenceSlice)]) / spacing[0]); - yShifts[iter] = static_cast((yCentroid[iter] - yCentroid[static_cast(m_InputValues->ReferenceSlice)]) / spacing[1]); + xShifts[iter] = ComputeCentroidShift(xCentroid[iter], xCentroid[static_cast(m_InputValues->ReferenceSlice)], spacing[0]); + yShifts[iter] = ComputeCentroidShift(yCentroid[iter], yCentroid[static_cast(m_InputValues->ReferenceSlice)], spacing[1]); } else { - xShifts[iter] = xShifts[iter - 1] + static_cast((xCentroid[iter] - xCentroid[iter - 1]) / spacing[0]); - yShifts[iter] = yShifts[iter - 1] + static_cast((yCentroid[iter] - yCentroid[iter - 1]) / spacing[1]); + xShifts[iter] = xShifts[iter - 1] + ComputeCentroidShift(xCentroid[iter], xCentroid[iter - 1], spacing[0]); + yShifts[iter] = yShifts[iter - 1] + ComputeCentroidShift(yCentroid[iter], yCentroid[iter - 1], spacing[1]); } if((xShifts[iter] < -sdims[0] || xShifts[iter] > sdims[0]) && !xWarning) @@ -222,13 +236,13 @@ Result<> AlignSectionsFeatureCentroid::findShifts(std::vector& xShifts, s } if(std::isnan(xCentroid[iter]) && !xWarning) { - std::string message = fmt::format("The X Centroid was NaN. All subsequent slices are probably wrong. Slice=", iter); + std::string message = fmt::format("The X Centroid was NaN. All subsequent slices are probably wrong. Slice={}", iter); m_MessageHandler.sendInfoMessage(message); xWarning = true; } if(std::isnan(yCentroid[iter]) && !yWarning) { - std::string message = fmt::format("The Y Centroid was NaN. All subsequent slices are probably wrong. Slice=", iter); + std::string message = fmt::format("The Y Centroid was NaN. All subsequent slices are probably wrong. Slice={}", iter); m_MessageHandler.sendInfoMessage(message); yWarning = true; } @@ -332,17 +346,17 @@ Result<> AlignSectionsFeatureCentroid::findShiftsOoc(std::vector& xShifts // Convert physical centroid differences to integer voxel offsets. bool xWarning = false; bool yWarning = false; - for(usize iter = 1; iter < dims[2]; iter++) + for(usize iter = (m_InputValues->UseReferenceSlice && m_InputValues->ReferenceSlice != 0) ? 0 : 1; iter < dims[2]; iter++) { if(m_InputValues->UseReferenceSlice) { - xShifts[iter] = static_cast((xCentroid[iter] - xCentroid[static_cast(m_InputValues->ReferenceSlice)]) / spacing[0]); - yShifts[iter] = static_cast((yCentroid[iter] - yCentroid[static_cast(m_InputValues->ReferenceSlice)]) / spacing[1]); + xShifts[iter] = ComputeCentroidShift(xCentroid[iter], xCentroid[static_cast(m_InputValues->ReferenceSlice)], spacing[0]); + yShifts[iter] = ComputeCentroidShift(yCentroid[iter], yCentroid[static_cast(m_InputValues->ReferenceSlice)], spacing[1]); } else { - xShifts[iter] = xShifts[iter - 1] + static_cast((xCentroid[iter] - xCentroid[iter - 1]) / spacing[0]); - yShifts[iter] = yShifts[iter - 1] + static_cast((yCentroid[iter] - yCentroid[iter - 1]) / spacing[1]); + xShifts[iter] = xShifts[iter - 1] + ComputeCentroidShift(xCentroid[iter], xCentroid[iter - 1], spacing[0]); + yShifts[iter] = yShifts[iter - 1] + ComputeCentroidShift(yCentroid[iter], yCentroid[iter - 1], spacing[1]); } if((xShifts[iter] < -sdims[0] || xShifts[iter] > sdims[0]) && !xWarning) @@ -361,12 +375,12 @@ Result<> AlignSectionsFeatureCentroid::findShiftsOoc(std::vector& xShifts } if(std::isnan(xCentroid[iter]) && !xWarning) { - m_MessageHandler.sendMessage(nx::core::IFilter::Message::Type::Info, fmt::format("The X Centroid was NaN. All subsequent slices are probably wrong. Slice=", iter)); + m_MessageHandler.sendMessage(nx::core::IFilter::Message::Type::Info, fmt::format("The X Centroid was NaN. All subsequent slices are probably wrong. Slice={}", iter)); xWarning = true; } if(std::isnan(yCentroid[iter]) && !yWarning) { - m_MessageHandler.sendMessage(nx::core::IFilter::Message::Type::Info, fmt::format("The Y Centroid was NaN. All subsequent slices are probably wrong. Slice=", iter)); + m_MessageHandler.sendMessage(nx::core::IFilter::Message::Type::Info, fmt::format("The Y Centroid was NaN. All subsequent slices are probably wrong. Slice={}", iter)); yWarning = true; } } @@ -379,7 +393,7 @@ Result<> AlignSectionsFeatureCentroid::findShiftsOoc(std::vector& xShifts auto& cumulativeShiftsStore = m_DataStructure.getDataAs(m_InputValues->CumulativeShiftsArrayPath)->getDataStoreRef(); auto& centroidsStore = m_DataStructure.getDataAs(m_InputValues->CentroidsArrayPath)->getDataStoreRef(); - for(usize iter = 1; iter < dims[2]; iter++) + for(usize iter = (m_InputValues->UseReferenceSlice && m_InputValues->ReferenceSlice != 0) ? 0 : 1; iter < dims[2]; iter++) { usize slice = (dims[2] - 1) - iter; int64 relativexshift = 0; @@ -391,14 +405,14 @@ Result<> AlignSectionsFeatureCentroid::findShiftsOoc(std::vector& xShifts } else { - relativexshift = static_cast((xCentroid[iter] - xCentroid[iter - 1]) / spacing[0]); - relativeyshift = static_cast((yCentroid[iter] - yCentroid[iter - 1]) / spacing[1]); + relativexshift = ComputeCentroidShift(xCentroid[iter], xCentroid[iter - 1], spacing[0]); + relativeyshift = ComputeCentroidShift(yCentroid[iter], yCentroid[iter - 1], spacing[1]); } usize xIndex = iter * 2; usize yIndex = (iter * 2) + 1; slicesStore[xIndex] = slice; - slicesStore[yIndex] = slice + 1; + slicesStore[yIndex] = iter == 0 ? dims[2] - 1 - static_cast(m_InputValues->ReferenceSlice) : slice + 1; relativeShiftsStore[xIndex] = relativexshift; relativeShiftsStore[yIndex] = relativeyshift; cumulativeShiftsStore[xIndex] = xShifts[iter]; diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CalculateTriangleGroupCurvatures.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CalculateTriangleGroupCurvatures.cpp index 02a346d3c0..42e3392097 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CalculateTriangleGroupCurvatures.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CalculateTriangleGroupCurvatures.cpp @@ -223,7 +223,8 @@ void CalculateTriangleGroupCurvatures::operator()() const FindNRingNeighbors::UniqueFaceIds_t triPatch = nRingNeighborAlg.getNRingTriangles(); if(triPatch.size() <= 1) { - throw std::runtime_error("NRingNeighbor failed to find more than one triangles."); + markOutputsInvalid(triId); + continue; } size_t beforeSize = triPatch.size(); diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeFeatureClustering.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeFeatureClustering.cpp index ee369df7a5..67dca53ef2 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeFeatureClustering.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeFeatureClustering.cpp @@ -71,7 +71,7 @@ std::vector GenerateRandomDistribution(float32 minDistance, float32 max distanceList.resize(largeNumber); - for(usize i = 1; i < largeNumber; i++) + for(usize i = 0; i < largeNumber; i++) { const float32 x = randomCentroids[3 * i]; const float32 y = randomCentroids[3 * i + 1]; diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeVectorColors.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeVectorColors.cpp index c98e4f54c4..664488f4ac 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeVectorColors.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ComputeVectorColors.cpp @@ -118,7 +118,7 @@ Result<> ComputeVectorColorsInChunks(const AbstractDataStore& vectors, g = 255.0f * ((120.0f - trend) / 120.0f); b = 255.0f * (trend / 120.0f); } - if(trend > 240.0f && trend < 360.0f) + if(trend > 240.0f && trend <= 360.0f) { trend -= 240.0f; r = 255.0f * (trend / 120.0f); diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ExecuteProcess.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ExecuteProcess.cpp index f3dbdafa27..c711ce82e2 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ExecuteProcess.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ExecuteProcess.cpp @@ -155,6 +155,10 @@ std::vector ExecuteProcess::splitArgumentsString(const std::string& std::vector argumentList; for(int i = 0; i < arguments.size(); i++) { + if(arguments[i] == ' ') + { + continue; + } if(arguments[i] == '\"') { i++; diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/QuickSurfaceMeshDirect.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/QuickSurfaceMeshDirect.cpp index 0ccf70cb84..f6bc812b46 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/QuickSurfaceMeshDirect.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/QuickSurfaceMeshDirect.cpp @@ -1179,9 +1179,11 @@ Result<> QuickSurfaceMeshDirect::createNodesAndTriangles(std::vectorquickSurfaceTransfer(triangleIndex, neigh1, point, faceLabelsStore); + tupleTransferFunction->quickSurfaceTransfer(triangleIndex, firstCell, secondCell, faceLabelsStore); } triangleIndex++; @@ -1201,7 +1203,7 @@ Result<> QuickSurfaceMeshDirect::createNodesAndTriangles(std::vectorquickSurfaceTransfer(triangleIndex, neigh1, point, faceLabelsStore); + tupleTransferFunction->quickSurfaceTransfer(triangleIndex, firstCell, secondCell, faceLabelsStore); } triangleIndex++; @@ -1291,9 +1293,11 @@ Result<> QuickSurfaceMeshDirect::createNodesAndTriangles(std::vectorquickSurfaceTransfer(triangleIndex, neigh2, point, faceLabelsStore); + tupleTransferFunction->quickSurfaceTransfer(triangleIndex, firstCell, secondCell, faceLabelsStore); } triangleIndex++; @@ -1313,7 +1317,7 @@ Result<> QuickSurfaceMeshDirect::createNodesAndTriangles(std::vectorquickSurfaceTransfer(triangleIndex, neigh2, point, faceLabelsStore); + tupleTransferFunction->quickSurfaceTransfer(triangleIndex, firstCell, secondCell, faceLabelsStore); } triangleIndex++; @@ -1403,9 +1407,11 @@ Result<> QuickSurfaceMeshDirect::createNodesAndTriangles(std::vectorquickSurfaceTransfer(triangleIndex, neigh3, point, faceLabelsStore); + tupleTransferFunction->quickSurfaceTransfer(triangleIndex, firstCell, secondCell, faceLabelsStore); } triangleIndex++; @@ -1425,7 +1431,7 @@ Result<> QuickSurfaceMeshDirect::createNodesAndTriangles(std::vectorquickSurfaceTransfer(triangleIndex, neigh3, point, faceLabelsStore); + tupleTransferFunction->quickSurfaceTransfer(triangleIndex, firstCell, secondCell, faceLabelsStore); } triangleIndex++; diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/QuickSurfaceMeshScanline.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/QuickSurfaceMeshScanline.cpp index 4df057b92f..0958a98720 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/QuickSurfaceMeshScanline.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/QuickSurfaceMeshScanline.cpp @@ -1453,7 +1453,7 @@ Result<> QuickSurfaceMeshScanline::createNodesAndTriangles(MeshIndexType nodeCou triBuffer.push_back(nid2); faceLabelBuf.push_back(curFeature); faceLabelBuf.push_back(neigh1Feature); - ttArgsBuf.push_back({triangleIndex, neigh1, point, curFeature, neigh1Feature}); + ttArgsBuf.push_back({triangleIndex, point, neigh1, curFeature, neigh1Feature}); } else { @@ -1472,7 +1472,7 @@ Result<> QuickSurfaceMeshScanline::createNodesAndTriangles(MeshIndexType nodeCou triBuffer.push_back(nid4); faceLabelBuf.push_back(curFeature); faceLabelBuf.push_back(neigh1Feature); - ttArgsBuf.push_back({triangleIndex, neigh1, point, curFeature, neigh1Feature}); + ttArgsBuf.push_back({triangleIndex, point, neigh1, curFeature, neigh1Feature}); } else { @@ -1560,7 +1560,7 @@ Result<> QuickSurfaceMeshScanline::createNodesAndTriangles(MeshIndexType nodeCou triBuffer.push_back(nid3); faceLabelBuf.push_back(curFeature); faceLabelBuf.push_back(neigh2Feature); - ttArgsBuf.push_back({triangleIndex, neigh2, point, curFeature, neigh2Feature}); + ttArgsBuf.push_back({triangleIndex, point, neigh2, curFeature, neigh2Feature}); } else { @@ -1579,7 +1579,7 @@ Result<> QuickSurfaceMeshScanline::createNodesAndTriangles(MeshIndexType nodeCou triBuffer.push_back(nid3); faceLabelBuf.push_back(curFeature); faceLabelBuf.push_back(neigh2Feature); - ttArgsBuf.push_back({triangleIndex, neigh2, point, curFeature, neigh2Feature}); + ttArgsBuf.push_back({triangleIndex, point, neigh2, curFeature, neigh2Feature}); } else { @@ -1667,7 +1667,7 @@ Result<> QuickSurfaceMeshScanline::createNodesAndTriangles(MeshIndexType nodeCou triBuffer.push_back(nid2); faceLabelBuf.push_back(curFeature); faceLabelBuf.push_back(neigh3Feature); - ttArgsBuf.push_back({triangleIndex, neigh3, point, curFeature, neigh3Feature}); + ttArgsBuf.push_back({triangleIndex, point, neigh3, curFeature, neigh3Feature}); } else { @@ -1686,7 +1686,7 @@ Result<> QuickSurfaceMeshScanline::createNodesAndTriangles(MeshIndexType nodeCou triBuffer.push_back(nid4); faceLabelBuf.push_back(curFeature); faceLabelBuf.push_back(neigh3Feature); - ttArgsBuf.push_back({triangleIndex, neigh3, point, curFeature, neigh3Feature}); + ttArgsBuf.push_back({triangleIndex, point, neigh3, curFeature, neigh3Feature}); } else { diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadVtkStructuredPoints.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadVtkStructuredPoints.cpp index 3169d9244c..c0558347d9 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadVtkStructuredPoints.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadVtkStructuredPoints.cpp @@ -9,10 +9,12 @@ #include #include +#include #include #include #include #include +#include #include using namespace nx::core; @@ -702,7 +704,7 @@ Result<> ReadVtkStructuredPoints::readFile() { return result; } - // Cell dimensions subtract one from each point dimension. + // Read point dimensions before deriving cell dimensions. line = std::string(buf.data()); auto tokens = StringUtilities::split(line, ' '); if(tokens.size() != 4) @@ -737,9 +739,16 @@ Result<> ReadVtkStructuredPoints::readFile() pointDims[2] = convertResultSizeT.value(); CreateImageGeometryAction::DimensionType cellDims(3, 0); - cellDims[0] = pointDims[0] - 1; - cellDims[1] = pointDims[1] - 1; - cellDims[2] = pointDims[2] - 1; + constexpr std::array k_AxisNames = {"X", "Y", "Z"}; + for(usize axis = 0; axis < 3; axis++) + { + if(pointDims[axis] == 0) + { + return MakeErrorResult(to_underlying(ErrorCodes::DimsValueErr), + fmt::format("The {} value for the {} axis is 0. Each {} value must be at least 1.", k_DimensionsKeyword, k_AxisNames[axis], k_DimensionsKeyword)); + } + cellDims[axis] = pointDims[axis] > 1 ? pointDims[axis] - 1 : 1; + } std::fill(buf.begin(), buf.end(), '\0'); result = ReadLine(in, buf.data(), kBufferSize); @@ -820,6 +829,15 @@ Result<> ReadVtkStructuredPoints::readFile() } origin[2] = convertResultF32.value(); + // Center the one-voxel slab on the VTK point along each singleton axis. + for(usize axis = 0; axis < 3; axis++) + { + if(pointDims[axis] == 1) + { + origin[axis] -= 0.5f * spacing[axis]; + } + } + if(m_InputValues->ReadPointData && m_Preflight) { auto createImageGeometryAction = std::make_unique(m_InputValues->PointGeomPath, pointDims, origin, spacing, m_InputValues->PointAttributeMatrixName); diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadVtkStructuredPoints.hpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadVtkStructuredPoints.hpp index 0d87b866c7..ae7812485c 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadVtkStructuredPoints.hpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadVtkStructuredPoints.hpp @@ -94,7 +94,8 @@ class SIMPLNXCORE_EXPORT ReadVtkStructuredPoints AsciiDataReadErr = -268, ///< Reports truncated or failed ASCII data. AsciiTokenTooLongErr = -269, ///< Reports a token longer than 1,024 characters. AsciiStreamPositionErr = -270, ///< Reports failure to restore the next header position. - NumberConvertErr = -10351 ///< Reports numeric text conversion failure. + NumberConvertErr = -10351, ///< Reports numeric text conversion failure. + DimsValueErr = -10354 ///< Reports a zero point dimension. }; /** diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/WriteOnScaleTableFile.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/WriteOnScaleTableFile.cpp index 5a65b1b4ad..92f837ae72 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/WriteOnScaleTableFile.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/WriteOnScaleTableFile.cpp @@ -370,15 +370,13 @@ Result<> WriteOnScaleTableFile::operator()() const SizeVec3 reorderedDimensions = exportGeometry->getDimensions(); SizeVec3 expectedDimensions = inputDimensions; std::sort(expectedDimensions.begin(), expectedDimensions.end(), std::greater()); - // ImageRotationUtilities::CreateRotationArgs selects output spacing from matrix columns, which show where each old axis lands. - // New-axis spacing requires matrix rows, or equivalently the columns of the transposed matrix. + // Check that the rotation preserved the intended axis permutation before exporting. if(reorderedDimensions != expectedDimensions) { - return MakeErrorResult( - k_UnexpectedReorderedDimensionsError, - fmt::format("Rotate Sample Reference Frame produced dimensions ({}) for geometry '{}' but the OnScale reorder expected ({}). This happens for two-axis reorders of geometries with " - "anisotropic spacing because the rotation utility assigns output spacing from the wrong matrix axis. Resample to isotropic spacing or reorder the axes before this filter.", - StringUtilities::formatDimensions3D(reorderedDimensions), m_InputValues->InputGeometryPath.toString(), StringUtilities::formatDimensions3D(expectedDimensions))); + return MakeErrorResult(k_UnexpectedReorderedDimensionsError, fmt::format("Rotate Sample Reference Frame produced dimensions ({}) for geometry '{}' but the OnScale reorder expected ({}). " + "The rotated geometry cannot be exported with the required axis order.", + StringUtilities::formatDimensions3D(reorderedDimensions), m_InputValues->InputGeometryPath.toString(), + StringUtilities::formatDimensions3D(expectedDimensions))); } exportFeatureIds = &rotatedDataStructure.getDataRefAs(k_ScratchCellDataPath.createChildPath(inputFeatureIds.getName())); m_MessageHandler.sendInfoMessage( diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/WriteVtkRectilinearGrid.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/WriteVtkRectilinearGrid.cpp index 306b741db7..bcf436c542 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/WriteVtkRectilinearGrid.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/WriteVtkRectilinearGrid.cpp @@ -7,6 +7,8 @@ #include "SimplnxCore/utils/VtkUtilities.hpp" +#include + using namespace nx::core; WriteVtkRectilinearGrid::WriteVtkRectilinearGrid(DataStructure& dataStructure, const IFilter::MessageHandler& mesgHandler, const std::atomic_bool& shouldCancel, @@ -130,7 +132,7 @@ Result<> WriteVtkRectilinearGrid::writeCoords(FILE* outputFile, const std::strin for(int idx = 0; idx < nPoints; ++idx) { d = idx * step + min; - fprintf(outputFile, "%f ", d); + fprintf(outputFile, "%.*g ", std::numeric_limits::max_digits10, static_cast(d)); if(idx % 20 == 0 && idx != 0) { fprintf(outputFile, "\n"); diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/AlignSectionsFeatureCentroidFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/AlignSectionsFeatureCentroidFilter.cpp index 53b6550dc7..db3715a531 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/AlignSectionsFeatureCentroidFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/AlignSectionsFeatureCentroidFilter.cpp @@ -29,6 +29,7 @@ constexpr nx::core::int32 k_OutputRepresentationTypeError = -68002; constexpr nx::core::int32 k_InputComponentDimensionError = -68003; constexpr nx::core::int32 k_InputComponentCountError = -68004; constexpr nx::core::int32 k_OutOfRangeReferenceSliceValue = -68064; +constexpr nx::core::int32 k_ReferenceSliceExceedsZDimension = -68073; } // namespace @@ -156,17 +157,21 @@ IFilter::PreflightResult AlignSectionsFeatureCentroidFilter::preflightImpl(const return MakePreflightErrorResult(k_OutOfRangeReferenceSliceValue, fmt::format("Reference Slice value ({}) must be ZERO or greater.", pReferenceSliceValue)); } + const auto* gridGeom = dataStructure.getDataAs(inputImageGeometry); + if(gridGeom == nullptr) + { + return MakePreflightErrorResult(-68070, fmt::format("The selected Image Geometry '{}' was not found or is not an Image Geometry.", inputImageGeometry.toString())); + } + const usize dims = gridGeom->getDimensions().getZ(); + if(filterArgs.value(k_UseReferenceSlice_Key) && static_cast(pReferenceSliceValue) >= dims) + { + return MakePreflightErrorResult(k_ReferenceSliceExceedsZDimension, fmt::format("Reference Slice value ({}) is too large. Image Geometry '{}' has Z dimension {}. Use a value from 0 to {}.", + pReferenceSliceValue, inputImageGeometry.toString(), dims, static_cast(dims) - 1)); + } + // Handle Array Creation if(pStoreAlignmentShifts) { - const auto* gridGeom = dataStructure.getDataAs(inputImageGeometry); - - if(gridGeom == nullptr) - { - return MakePreflightErrorResult(-68070, fmt::format("Store Alignment Shifts was selected, but an invalid image geometry was provided. Input Geometry Path :{}", inputImageGeometry.toString())); - } - - const usize dims = gridGeom->getDimensions().getZ(); auto pAlignmentAMName = filterArgs.value(k_AlignmentAMName_Key); const DataPath amPath = inputImageGeometry.createChildPath(pAlignmentAMName); diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ReadDREAM3DFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ReadDREAM3DFilter.cpp index 8f8de32e7d..3012a28bd9 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ReadDREAM3DFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ReadDREAM3DFilter.cpp @@ -98,6 +98,13 @@ IFilter::PreflightResult ReadDREAM3DFilter::preflightImpl(const DataStructure& d if(importData.ImportPolicy == Dream3dImportParameter::PathImportPolicy::IncludeList) { + for(const auto& dataPath : importData.DataPaths) + { + if(!importedDataStructure.containsData(dataPath)) + { + result.warnings().push_back(Warning{-12, fmt::format("The Include List path '{}' does not exist in the input file '{}'.", dataPath.toString(), importData.FilePath.string())}); + } + } auto allDataPaths = importedDataStructure.getAllDataPaths(); std::vector selectedDataPaths = DREAM3D::ExpandSelectedPathsToDescendants(importData.DataPaths, allDataPaths); selectedDataPaths = DREAM3D::ExpandSelectedPathsToAncestors(selectedDataPaths); diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ReadStlFileFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ReadStlFileFilter.cpp index 42d16fae17..877452b536 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ReadStlFileFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ReadStlFileFilter.cpp @@ -137,6 +137,14 @@ IFilter::PreflightResult ReadStlFileFilter::preflightImpl(const DataStructure& d return MakePreflightErrorResult(numTriangles, fmt::format("Error extracting the number of triangles from the STL file '{}'. Returned error code {}.", pStlFilePathValue.string(), numTriangles)); } + // Reject impossible triangle counts before creating actions that allocate the output arrays. + const StlConstants::StlFileCheck stlFileCheck = StlUtilities::SanityCheckFile(pStlFilePathValue); + if(stlFileCheck.error != 0) + { + return MakePreflightErrorResult(stlFileCheck.error, stlFileCheck.errorMessage); + } + numTriangles = stlFileCheck.numTriangles; + // This can happen in a LOT of STL files. Just means the writer didn't go back and update the header. if(numTriangles == 0) { diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/WriteVtkRectilinearGridFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/WriteVtkRectilinearGridFilter.cpp index 2c270592d7..7aaf6c2b67 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/WriteVtkRectilinearGridFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/WriteVtkRectilinearGridFilter.cpp @@ -91,6 +91,16 @@ IFilter::PreflightResult WriteVtkRectilinearGridFilter::preflightImpl(const Data return MakePreflightErrorResult(-2070, "No cell data arrays are selected. You must select at least one array."); } + for(const auto& arrayPath : pSelectedDataArrayPathsValue) + { + const usize components = dataStructure.getDataRefAs(arrayPath).getNumberOfComponents(); + if(components < 1 || components > 4) + { + return MakePreflightErrorResult( + -12073, fmt::format("Array '{}' has {} components. Legacy VTK SCALARS requires 1 to 4 components. Select an array with 1 to 4 components.", arrayPath.toString(), components)); + } + } + auto tupleValidityCheck = dataStructure.validateNumberOfTuples(pSelectedDataArrayPathsValue); if(!tupleValidityCheck) { diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/WriteVtkStructuredPointsFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/WriteVtkStructuredPointsFilter.cpp index 09d759fffb..e4654602df 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/WriteVtkStructuredPointsFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/WriteVtkStructuredPointsFilter.cpp @@ -90,6 +90,16 @@ IFilter::PreflightResult WriteVtkStructuredPointsFilter::preflightImpl(const Dat return MakePreflightErrorResult(-2070, "No cell data arrays are selected. You must select at least one array."); } + for(const auto& arrayPath : pSelectedDataArrayPathsValue) + { + const usize components = dataStructure.getDataRefAs(arrayPath).getNumberOfComponents(); + if(components < 1 || components > 4) + { + return MakePreflightErrorResult( + -12074, fmt::format("Array '{}' has {} components. Legacy VTK SCALARS requires 1 to 4 components. Select an array with 1 to 4 components.", arrayPath.toString(), components)); + } + } + auto tupleValidityCheck = dataStructure.validateNumberOfTuples(pSelectedDataArrayPathsValue); if(!tupleValidityCheck) { diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/utils/StlUtilities.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/utils/StlUtilities.cpp index f2daa02d64..02f7384b51 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/utils/StlUtilities.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/utils/StlUtilities.cpp @@ -168,20 +168,26 @@ StlConstants::StlFileCheck StlUtilities::SanityCheckFile(const fs::path& path) return stlFileCheck; } - stlFileCheck.header = header; - stlFileCheck.numTriangles = triCount; - stlFileCheck.fileSize = fileSize; - // The size this file would be if none of its triangles carry any attribute payload. The // arithmetic is done in uintmax_t because triCount * k_StlTriangleBytes overflows int32 // for meshes beyond roughly 42.9 million triangles. const uintmax_t sizeWithoutPayload = StlConstants::k_StlFixedHeaderBytes + (static_cast(triCount) * StlConstants::k_StlTriangleBytes); + if(fileSize < sizeWithoutPayload) + { + stlFileCheck.error = StlConstants::k_StlFileLengthError; + stlFileCheck.errorMessage = + fmt::format("STL file '{}' declares {} triangles and requires at least {} bytes, but the file is only {} bytes long.", path.string(), triCount, sizeWithoutPayload, fileSize); + return stlFileCheck; + } + + stlFileCheck.header = header; + stlFileCheck.numTriangles = triCount; + stlFileCheck.fileSize = fileSize; // Only a file with bytes left over beyond the fixed size records has anywhere to put // attribute payload. If the size matches exactly, then every attribute byte count in the file // is something other than a length and must be ignored, no matter what any individual - // triangle happens to contain. A file that is *smaller* than this is truncated; report no - // payload so the reader does not seek, and let the read loop report the truncation precisely. + // triangle happens to contain. Smaller files were rejected above before any mesh allocation. // Vendors known to misuse the field are excluded even when there are spare bytes, since those // bytes may just be trailing junk. stlFileCheck.attributePayloadPresent = (fileSize > sizeWithoutPayload) && !IsMagicsFile(header) && !IsVxElementsFile(header); diff --git a/src/Plugins/SimplnxCore/test/AddBadDataTest.cpp b/src/Plugins/SimplnxCore/test/AddBadDataTest.cpp index 7b3d138a77..07dfe68e46 100644 --- a/src/Plugins/SimplnxCore/test/AddBadDataTest.cpp +++ b/src/Plugins/SimplnxCore/test/AddBadDataTest.cpp @@ -6,6 +6,7 @@ #include "simplnx/DataStructure/DataArray.hpp" #include "simplnx/DataStructure/DataStore.hpp" #include "simplnx/DataStructure/Geometry/ImageGeom.hpp" +#include "simplnx/DataStructure/NeighborList.hpp" #include "simplnx/DataStructure/StringArray.hpp" #include "simplnx/Filter/IParameter.hpp" #include "simplnx/Utilities/AlgorithmDispatch.hpp" @@ -229,3 +230,53 @@ TEST_CASE("SimplnxCore::AddBadData: rejects nonnumeric input arrays", "[SimplnxC REQUIRE(preflight.outputActions.errors().front().message.find("Input") != std::string::npos); UnitTest::CheckArraysInheritTupleDims(dataStructure); } + +TEST_CASE("SimplnxCore::AddBadDataFilter: Reject nonnumeric cell arrays", "[SimplnxCore][AddBadDataFilter]") +{ + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + CAPTURE(scenario); + UnitTest::AlgorithmTestScope scope(scenario); + const bool useStrings = GENERATE(true, false); + CAPTURE(useStrings); + + DataStructure dataStructure; + auto* imageGeom = ImageGeom::Create(dataStructure, "Image"); + REQUIRE(imageGeom != nullptr); + imageGeom->setDimensions({1, 1, 1}); + auto* cellAM = AttributeMatrix::Create(dataStructure, "Cell Data", ShapeType{1, 1, 1}, imageGeom->getId()); + REQUIRE(cellAM != nullptr); + imageGeom->setCellData(*cellAM); + REQUIRE(Int32Array::Create(dataStructure, "Distances", std::make_shared>(ShapeType{1, 1, 1}, ShapeType{1}, 0), cellAM->getId()) != nullptr); + if(useStrings) + { + REQUIRE(StringArray::CreateWithValues(dataStructure, "Unsupported", cellAM->getShape(), {"label"}, cellAM->getId()) != nullptr); + } + else + { + REQUIRE(NeighborList::Create(dataStructure, "Unsupported", cellAM->getShape(), cellAM->getId()) != nullptr); + } + + AddBadDataFilter filter; + Arguments args; + args.insertOrAssign(AddBadDataFilter::k_ImageGeometryPath_Key, std::make_any(DataPath({"Image"}))); + args.insertOrAssign(AddBadDataFilter::k_GBEuclideanDistancesArrayPath_Key, std::make_any(DataPath({"Image", "Cell Data", "Distances"}))); + args.insertOrAssign(AddBadDataFilter::k_PoissonNoise_Key, std::make_any(true)); + args.insertOrAssign(AddBadDataFilter::k_PoissonVolFraction_Key, std::make_any(1.0F)); + args.insertOrAssign(AddBadDataFilter::k_UseSeed_Key, std::make_any(true)); + + // Type-contract oracle: zeroing numeric components is undefined for strings and variable-length lists. + // Reject either type in preflight, before execution can cast it to IDataArray. + const auto preflight = filter.preflight(dataStructure, args); + CHECK(preflight.outputActions.invalid()); + const auto execute = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_INVALID(execute.result); + REQUIRE(execute.result.errors().size() == 1); + CHECK(execute.result.errors()[0].code == -76239); + CHECK(execute.result.errors()[0].message.find("Image/Cell Data/Unsupported") != std::string::npos); + if(preflight.outputActions.invalid()) + { + REQUIRE(preflight.outputActions.errors().size() == 1); + CHECK(preflight.outputActions.errors()[0].code == -76239); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} diff --git a/src/Plugins/SimplnxCore/test/AlignSectionsFeatureCentroidTest.cpp b/src/Plugins/SimplnxCore/test/AlignSectionsFeatureCentroidTest.cpp index 29c1e534a0..0bdf7116fb 100644 --- a/src/Plugins/SimplnxCore/test/AlignSectionsFeatureCentroidTest.cpp +++ b/src/Plugins/SimplnxCore/test/AlignSectionsFeatureCentroidTest.cpp @@ -12,6 +12,7 @@ #include +#include #include #include @@ -182,3 +183,206 @@ TEST_CASE("SimplnxCore::AlignSectionsFeatureCentroidFilter: Geometry cell tuple REQUIRE(errorRef.message.find("'Image' has 12 cells") != std::string::npos); } } + +TEST_CASE("SimplnxCore::AlignSectionsFeatureCentroidFilter: Nonzero reference top slice", "[SimplnxCore][AlignSectionsFeatureCentroidFilter]") +{ + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + const bool storeShifts = GENERATE(false, true); + CAPTURE(scenario, storeShifts); + UnitTest::AlgorithmTestScope scope(scenario); + using FilterType = AlignSectionsFeatureCentroidFilter; + DataStructure dataStructure; + auto* geom = ImageGeom::Create(dataStructure, "Image"); + REQUIRE(geom != nullptr); + geom->setDimensions({3, 3, 4}); + geom->setSpacing({1.0F, 1.0F, 1.0F}); + auto* cells = AttributeMatrix::Create(dataStructure, "Cells", ShapeType{4, 3, 3}, geom->getId()); + REQUIRE(cells != nullptr); + geom->setCellData(*cells); + auto* mask = UInt8Array::Create(dataStructure, "Mask", std::make_shared>(ShapeType{4, 3, 3}, ShapeType{1}, 0), cells->getId()); + REQUIRE(mask != nullptr); + mask->fill(0); + // One marked cell per slice: (1,1), (0,0), (1,1), (2,2) in ascending Z. + // Reference index 1 counts from the top, so it is physical slice 2. + // Centroid differences in descending Z are (+1,+1), (0,0), (-1,-1), (0,0). + // Reading source at destination + shift must place all marks at (1,1). + (*mask)[4] = 1; + (*mask)[9] = 1; + (*mask)[22] = 1; + (*mask)[35] = 1; + + FilterType filter; + Arguments args; + args.insertOrAssign(FilterType::k_SelectedImageGeometryPath_Key, std::make_any(DataPath({"Image"}))); + args.insertOrAssign(FilterType::k_MaskArrayPath_Key, std::make_any(DataPath({"Image", "Cells", "Mask"}))); + args.insertOrAssign(FilterType::k_UseReferenceSlice_Key, std::make_any(true)); + args.insertOrAssign(FilterType::k_ReferenceSlice_Key, std::make_any(1)); + args.insertOrAssign(FilterType::k_StoreAlignmentShifts_Key, std::make_any(storeShifts)); + args.insertOrAssign(FilterType::k_AlignmentAMName_Key, std::make_any("Shifts")); + args.insertOrAssign(FilterType::k_CumulativeShiftsArrayName_Key, std::make_any("Cumulative")); + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + auto result = scope.executeFilter(filter, dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(result.result); + for(usize z = 0; z < 4; z++) + { + for(usize xy = 0; xy < 9; xy++) + { + CAPTURE(z, xy); + CHECK((*mask)[(z * 9) + xy] == (xy == 4 ? 1 : 0)); + } + } + if(storeShifts) + { + const auto& slices = dataStructure.getDataRefAs(DataPath({"Image", "Shifts", "Slice Indices"})); + // Row 0 pairs the top slice (physical 3) with the reference (physical 2). + CHECK(slices[0] == 3); + CHECK(slices[1] == 2); + const DataPath shiftsPath({"Image", "Shifts", "Cumulative"}); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(shiftsPath)); + const auto& shifts = dataStructure.getDataRefAs(shiftsPath); + const std::array expectedShifts = {1, 0, -1, 0}; + for(usize iter = 0; iter < 4; iter++) + { + CAPTURE(iter); + CHECK(shifts[iter * 2] == expectedShifts[iter]); + CHECK(shifts[(iter * 2) + 1] == expectedShifts[iter]); + } + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::AlignSectionsFeatureCentroidFilter: Empty slices", "[SimplnxCore][AlignSectionsFeatureCentroidFilter]") +{ + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + const bool storeShifts = GENERATE(false, true); + const int emptyCase = GENERATE(0, 1, 2); // Empty top, middle, or fixed reference. + const bool useReference = emptyCase != 1; + CAPTURE(scenario, storeShifts, emptyCase); + UnitTest::AlgorithmTestScope scope(scenario); + using FilterType = AlignSectionsFeatureCentroidFilter; + DataStructure dataStructure; + auto* geom = ImageGeom::Create(dataStructure, "Image"); + REQUIRE(geom != nullptr); + geom->setDimensions({3, 3, 4}); + geom->setSpacing({1.0F, 1.0F, 1.0F}); + auto* cells = AttributeMatrix::Create(dataStructure, "Cells", ShapeType{4, 3, 3}, geom->getId()); + REQUIRE(cells != nullptr); + geom->setCellData(*cells); + auto* mask = UInt8Array::Create(dataStructure, "Mask", std::make_shared>(ShapeType{4, 3, 3}, ShapeType{1}, 0), cells->getId()); + REQUIRE(mask != nullptr); + mask->fill(0); + // Descending-Z centroids are (2,2), (1,1), (0,0), (1,1) before removing a mark. + // Empty top with reference 1: offsets [0,0,-1,0]. Empty reference: all offsets 0. + // Empty middle (index 2), consecutive: relative [0,-1,0,0], cumulative [0,-1,-1,-1]. + // Both axes have the same values. A zero relative shift preserves the previous cumulative shift. + std::array expectedShifts = emptyCase == 0 ? std::array{0, 0, -1, 0} : emptyCase == 1 ? std::array{0, -1, -1, -1} : std::array{0, 0, 0, 0}; + std::array expectedRelative = emptyCase == 1 ? std::array{0, -1, 0, 0} : expectedShifts; + const usize emptyZ = emptyCase == 0 ? 3 : emptyCase == 1 ? 1 : 2; + const std::array markXY = {4, 0, 4, 8}; + for(usize z = 0; z < 4; z++) + { + if(z != emptyZ) + { + (*mask)[z * 9 + markXY[z]] = 1; + } + } + // A separate marker makes the empty slice's applied shift observable without diagnostics. + auto* values = Int32Array::Create(dataStructure, "Values", std::make_shared>(ShapeType{4, 3, 3}, ShapeType{1}, 0), cells->getId()); + REQUIRE(values != nullptr); + values->fill(0); + for(usize z = 0; z < 4; z++) + { + (*values)[z * 9 + 4] = 7; + } + + FilterType filter; + Arguments args; + args.insertOrAssign(FilterType::k_SelectedImageGeometryPath_Key, std::make_any(DataPath({"Image"}))); + args.insertOrAssign(FilterType::k_MaskArrayPath_Key, std::make_any(DataPath({"Image", "Cells", "Mask"}))); + args.insertOrAssign(FilterType::k_UseReferenceSlice_Key, std::make_any(useReference)); + args.insertOrAssign(FilterType::k_ReferenceSlice_Key, std::make_any(1)); + args.insertOrAssign(FilterType::k_StoreAlignmentShifts_Key, std::make_any(storeShifts)); + args.insertOrAssign(FilterType::k_AlignmentAMName_Key, std::make_any("Shifts")); + args.insertOrAssign(FilterType::k_CumulativeShiftsArrayName_Key, std::make_any("Cumulative")); + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + auto result = scope.executeFilter(filter, dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(result.result); + for(usize z = 0; z < 4; z++) + { + const int64 shift = expectedShifts[3 - z]; + for(usize xy = 0; xy < 9; xy++) + { + CAPTURE(z, xy); + const int64 sourceX = static_cast(xy % 3) + shift; + const int64 sourceY = static_cast(xy / 3) + shift; + CHECK((*values)[z * 9 + xy] == (sourceX == 1 && sourceY == 1 ? 7 : 0)); + } + } + if(storeShifts) + { + const DataPath shiftsPath({"Image", "Shifts", "Cumulative"}); + const DataPath relativePath({"Image", "Shifts", "Relative Shifts"}); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(shiftsPath)); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(relativePath)); + const auto& shifts = dataStructure.getDataRefAs(shiftsPath); + const auto& relative = dataStructure.getDataRefAs(relativePath); + for(usize iter = 0; iter < 4; iter++) + { + CAPTURE(iter); + for(usize axis = 0; axis < 2; axis++) + { + CHECK(shifts[iter * 2 + axis] == expectedShifts[iter]); + CHECK(relative[iter * 2 + axis] == expectedRelative[iter]); + } + } + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::AlignSectionsFeatureCentroidFilter: Reference slice bounds", "[SimplnxCore][AlignSectionsFeatureCentroidFilter]") +{ + using FilterType = AlignSectionsFeatureCentroidFilter; + const usize zDim = GENERATE(usize{1}, usize{3}); + const bool useReference = GENERATE(true, false); + const bool atUpperBound = GENERATE(true, false); + const bool storeShifts = GENERATE(false, true); + // Z slices have exactly the integer indices 0 through Z-1. Z itself is not an index. + const int32 reference = static_cast(atUpperBound ? zDim : zDim - 1); + CAPTURE(zDim, useReference, reference, storeShifts); + DataStructure dataStructure; + auto* geom = ImageGeom::Create(dataStructure, "Image"); + REQUIRE(geom != nullptr); + geom->setDimensions({3, 3, zDim}); + auto* cells = AttributeMatrix::Create(dataStructure, "Cells", ShapeType{zDim, 3, 3}, geom->getId()); + REQUIRE(cells != nullptr); + geom->setCellData(*cells); + REQUIRE(UInt8Array::Create(dataStructure, "Mask", std::make_shared>(ShapeType{zDim, 3, 3}, ShapeType{1}, 0), cells->getId()) != nullptr); + + FilterType filter; + Arguments args; + args.insertOrAssign(FilterType::k_SelectedImageGeometryPath_Key, std::make_any(DataPath({"Image"}))); + args.insertOrAssign(FilterType::k_MaskArrayPath_Key, std::make_any(DataPath({"Image", "Cells", "Mask"}))); + args.insertOrAssign(FilterType::k_UseReferenceSlice_Key, std::make_any(useReference)); + args.insertOrAssign(FilterType::k_ReferenceSlice_Key, std::make_any(reference)); + args.insertOrAssign(FilterType::k_StoreAlignmentShifts_Key, std::make_any(storeShifts)); + // Stop at preflight: invalid references must never reach the alignment algorithm. + IFilter::PreflightResult result; + REQUIRE_NOTHROW(result = filter.preflight(dataStructure, args)); + if(useReference && atUpperBound) + { + SIMPLNX_RESULT_REQUIRE_INVALID(result.outputActions); + REQUIRE(result.outputActions.errors().size() == 1); + const auto& error = result.outputActions.errors()[0]; + REQUIRE(error.code == -68073); + CHECK(error.message.find(fmt::format("Reference Slice value ({}) is too large.", reference)) != std::string::npos); + CHECK(error.message.find(fmt::format("Z dimension {}", zDim)) != std::string::npos); + CHECK(error.message.find(fmt::format("0 to {}", zDim - 1)) != std::string::npos); + } + else + { + SIMPLNX_RESULT_REQUIRE_VALID(result.outputActions); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} diff --git a/src/Plugins/SimplnxCore/test/ApplyTransformationToGeometryTest.cpp b/src/Plugins/SimplnxCore/test/ApplyTransformationToGeometryTest.cpp index cfc3b6479f..b6a0b46f58 100644 --- a/src/Plugins/SimplnxCore/test/ApplyTransformationToGeometryTest.cpp +++ b/src/Plugins/SimplnxCore/test/ApplyTransformationToGeometryTest.cpp @@ -1024,6 +1024,59 @@ TEST_CASE("SimplnxCore::ApplyTransformationToGeometryFilter:Manual_Image", "[Sim } } +TEST_CASE("SimplnxCore::ApplyTransformationToGeometryFilter: Uneven scaling keeps the classified source axis", "[SimplnxCore][ApplyTransformationToGeometryFilter]") +{ + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + CAPTURE(scenario); + + DataStructure dataStructure; + const DataPath inputGeometryPath({"Input"}); + const DataPath inputCellAMPath = inputGeometryPath.createChildPath(k_Cell_Data); + auto* image = ImageGeom::Create(dataStructure, "Input"); + REQUIRE(image != nullptr); + image->setDimensions({2, 2, 2}); + image->setSpacing({2.0F, 3.0F, 5.0F}); + image->setOrigin({0.0F, 0.0F, 0.0F}); + auto* cellAM = AttributeMatrix::Create(dataStructure, k_Cell_Data, {2, 2, 2}, image->getId()); + REQUIRE(cellAM != nullptr); + image->setCellData(*cellAM); + auto* values = Int32Array::CreateWithStore(dataStructure, "Data", {2, 2, 2}, {1}, cellAM->getId()); + REQUIRE(values != nullptr); + values->fill(1); + + const ApplyTransformationToGeometryFilter filter; + Arguments args; + args.insertOrAssign(ApplyTransformationToGeometryFilter::k_SelectedImageGeometryPath_Key, std::make_any(inputGeometryPath)); + args.insertOrAssign(ApplyTransformationToGeometryFilter::k_CellAttributeMatrixPath_Key, std::make_any(inputCellAMPath)); + args.insertOrAssign(ApplyTransformationToGeometryFilter::k_TransformationType_Key, std::make_any(apply_transformation_to_geometry::k_ManualTransformationMatrixIdx)); + args.insertOrAssign(ApplyTransformationToGeometryFilter::k_InterpolationType_Key, std::make_any(apply_transformation_to_geometry::k_NearestNeighborInterpolationIdx)); + const DynamicTableParameter::ValueType matrix{{1.0, 5.0, 0.0, 0.0}, {0.0, 100000.0, 0.0, 0.0}, {0.0, 0.0, 1.0, 0.0}, {0.0, 0.0, 0.0, 1.0}}; + args.insertOrAssign(ApplyTransformationToGeometryFilter::k_ManualTransformationMatrix_Key, std::make_any(matrix)); + args.insertOrAssign(ApplyTransformationToGeometryFilter::k_TranslateGeometryToGlobalOrigin_Key, std::make_any(false)); + + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions) + UnitTest::AlgorithmTestScope algorithmTestScope(scenario); + auto executeResult = algorithmTestScope.executeFilter(filter, dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result) + + // Hand calculation: (x,y,z) -> (x+5y,100000y,z). The normalized Y column's + // off-axis entry is about 5e-5, so the classified source axes remain X, Y, Z. + // Spacing is (2, 3*sqrt(100000^2+5^2), 5), or (2,300000,5) in float32. + // Bounds have extents (4+5*6,100000*6,10) = (34,600000,10). + // Extents divided by spacing give dimensions (17,2,2). + const auto& output = dataStructure.getDataRefAs(inputGeometryPath); + const FloatVec3 expectedSpacing = {2.0F, 300000.0F, 5.0F}; + const SizeVec3 expectedDims = {17, 2, 2}; + for(usize axis = 0; axis < 3; ++axis) + { + CAPTURE(axis); + CHECK(output.getSpacing()[axis] == Approx(expectedSpacing[axis])); + CHECK(output.getDimensions()[axis] == expectedDims[axis]); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::ApplyTransformationToGeometryFilter:Precomputed_Image", "[SimplnxCore][ApplyTransformationToGeometryFilter]") { UnitTest::LoadPlugins(); diff --git a/src/Plugins/SimplnxCore/test/ComputeFeatureClusteringTest.cpp b/src/Plugins/SimplnxCore/test/ComputeFeatureClusteringTest.cpp index 70f7c727a6..62fd0f49cd 100644 --- a/src/Plugins/SimplnxCore/test/ComputeFeatureClusteringTest.cpp +++ b/src/Plugins/SimplnxCore/test/ComputeFeatureClusteringTest.cpp @@ -113,7 +113,27 @@ TEST_CASE("SimplnxCore::ComputeFeatureClusteringFilter: Valid Filter Execution", SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result) CompareNeighborLists(dataStructure, k_CellFeatureDataPath.createChildPath(k_ExemplarClusteringList), k_CellFeatureDataPath.createChildPath(k_GeneratedClusteringList)); - CompareArrays(dataStructure, k_CellEnsembleAttributeMatrixPath.createChildPath(k_ExemplarRDF), k_CellEnsembleAttributeMatrixPath.createChildPath(k_GeneratedRDF)); + // The legacy RDF exemplar omits reference pairs involving sample zero (L11 / #1820). + // Correct RDF normalization is checked by the independent closed-form test below. + // The same seed gives the same samples, and the fix only adds reference pairs. So no bin can grow. + // In this data set some of the 999 distances from sample zero fall inside the RDF distance range, + // so at least one bin must shrink. + { + const auto& legacyRdf = dataStructure.getDataRefAs(k_CellEnsembleAttributeMatrixPath.createChildPath(k_ExemplarRDF)); + const auto& generatedRdf = dataStructure.getDataRefAs(k_CellEnsembleAttributeMatrixPath.createChildPath(k_GeneratedRDF)); + REQUIRE(generatedRdf.getSize() == legacyRdf.getSize()); + usize lowerBins = 0; + for(usize i = 0; i < generatedRdf.getSize(); i++) + { + INFO(fmt::format("RDF index {}: generated {} legacy {}", i, generatedRdf[i], legacyRdf[i])); + REQUIRE(generatedRdf[i] <= legacyRdf[i]); + if(generatedRdf[i] < legacyRdf[i]) + { + lowerBins++; + } + } + REQUIRE(lowerBins > 0); + } CompareArrays(dataStructure, k_CellEnsembleAttributeMatrixPath.createChildPath(k_ExemplarMinMaxDistances), k_CellEnsembleAttributeMatrixPath.createChildPath(k_GeneratedMinMaxDistances)); UnitTest::CheckArraysInheritTupleDims(dataStructure); @@ -244,7 +264,8 @@ TEST_CASE("SimplnxCore::ComputeFeatureClustering: four bins control", "[SimplnxC REQUIRE(maxMin.getSize() == 4); // Distances 1, 2, and 3 each occur twice. The last distance stays in the last bin. // Independent lattice-cell population enumeration gives these ordered reference-pair counts for seed 5489. - const std::array referencePairs = {175186, 53046, 292346, 46722}; + // The enumeration includes all 999000 ordered pairs of the 1000 sample points, including pairs with sample 0. + const std::array referencePairs = {175500, 53124, 292846, 46806}; const std::array observedPairs = {2, 0, 2, 2}; for(usize bin = 0; bin < 4; bin++) { @@ -317,3 +338,49 @@ TEST_CASE("SimplnxCore::ComputeFeatureClustering: positive bin count domain", "[ SIMPLNX_RESULT_REQUIRE_VALID(preflight.outputActions); UnitTest::CheckArraysInheritTupleDims(dataStructure); } + +TEST_CASE("SimplnxCore::ComputeFeatureClusteringFilter: Complete reference pair normalization", "[SimplnxCore][ComputeFeatureClusteringFilter]") +{ + DataStructure dataStructure; + auto* geometry = ImageGeom::Create(dataStructure, "Image"); + REQUIRE(geometry != nullptr); + geometry->setDimensions({1, 1, 1}); + geometry->setSpacing({1.0f, 1.0f, 1.0f}); + auto* features = AttributeMatrix::Create(dataStructure, "Features", {4}, geometry->getId()); + REQUIRE(features != nullptr); + auto* ensembles = AttributeMatrix::Create(dataStructure, "Ensembles", {2}, geometry->getId()); + REQUIRE(ensembles != nullptr); + auto* phases = Int32Array::CreateWithStore(dataStructure, "Phases", {4}, {1}, features->getId()); + auto* centroids = Float32Array::CreateWithStore(dataStructure, "Centroids", {4}, {3}, features->getId()); + REQUIRE(phases != nullptr); + REQUIRE(centroids != nullptr); + phases->fill(1); + (*phases)[0] = 0; + centroids->fill(0.0f); + (*centroids)[9] = 0.5f; + + ComputeFeatureClusteringFilter filter; + Arguments args = filter.getDefaultArguments(); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_SelectedImageGeometryPath_Key, std::make_any(DataPath({"Image"}))); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_NumberOfBins_Key, std::make_any(1)); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_PhaseNumber_Key, std::make_any(1)); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_RemoveBiasedFeatures_Key, std::make_any(false)); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_SetRandomSeed_Key, std::make_any(true)); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_SeedValue_Key, std::make_any(5489)); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_FeaturePhasesArrayPath_Key, std::make_any(DataPath({"Image", "Features", "Phases"}))); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_CentroidsArrayPath_Key, std::make_any(DataPath({"Image", "Features", "Centroids"}))); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_CellEnsembleAttributeMatrixPath_Key, std::make_any(DataPath({"Image", "Ensembles"}))); + args.insertOrAssign(ComputeFeatureClusteringFilter::k_RDFArrayName_Key, std::make_any("RDF")); + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions) + auto executeResult = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result) + + // Closed form: a one-voxel reference puts all 1000 * 999 ordered pairs at distance zero. + // The three feature distances are 0, 0.5, 0.5. One bin contains all six ordered pairs. + // Its reference probability is 1, so RDF = 6 / (1 * 3 * 2) = 1, independent of seed. + REQUIRE_NOTHROW(dataStructure.getDataRefAs(DataPath({"Image", "Ensembles", "RDF"}))); + const auto& rdf = dataStructure.getDataRefAs(DataPath({"Image", "Ensembles", "RDF"})); + REQUIRE(rdf[1] == Approx(1.0f).margin(1.0e-6f)); + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} diff --git a/src/Plugins/SimplnxCore/test/ComputeVectorColorsTest.cpp b/src/Plugins/SimplnxCore/test/ComputeVectorColorsTest.cpp index e76bc309b2..a49ad3903c 100644 --- a/src/Plugins/SimplnxCore/test/ComputeVectorColorsTest.cpp +++ b/src/Plugins/SimplnxCore/test/ComputeVectorColorsTest.cpp @@ -27,6 +27,39 @@ DataPath eulerAnglesPath = ebsdPath.createChildPath(Constants::k_EulerAngles); const std::string k_VecColorsNX = "Vector Colors"; } // namespace +TEST_CASE("SimplnxCore::ComputeVectorColorsFilter: Rounded full-turn trend stays red", "[SimplnxCore][ComputeVectorColorsFilter]") +{ + DataStructure dataStructure; + const DataPath vectorsPath({"Vectors"}); + auto vectorsStore = DataStoreUtilities::CreateDataStore(dataStructure, vectorsPath, {1}, {3}); + REQUIRE(Float32Array::Create(dataStructure, vectorsPath.getTargetName(), vectorsStore) != nullptr); + const std::array vectors = {1.0f, -1.0e-8f, 0.0f}; + const auto writeResult = vectorsStore->copyFromBuffer(0, nonstd::span(vectors.data(), vectors.size())); + SIMPLNX_RESULT_REQUIRE_VALID(writeResult); + + ComputeVectorColorsFilter filter; + Arguments args; + args.insertOrAssign(ComputeVectorColorsFilter::k_UseMask_Key, std::make_any(false)); + args.insertOrAssign(ComputeVectorColorsFilter::k_VectorsArrayPath_Key, std::make_any(vectorsPath)); + args.insertOrAssign(ComputeVectorColorsFilter::k_CellVectorColorsArrayName_Key, std::make_any(k_VecColorsNX)); + const auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + const auto executeResult = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + + // Hand calculation: atan2(-1e-8, 1) is about -5.73e-7 degrees. + // Adding 360 rounds to exactly 360 in float32, equivalent to trend 0. + // With Z=0, plunge is 90 degrees: the red endpoint is RGB (255, 0, 0). + const std::array expectedColors = {255, 0, 0}; + const auto* colors = dataStructure.getDataAs(DataPath({k_VecColorsNX})); + REQUIRE(colors != nullptr); + std::array actualColors = {}; + const auto readResult = colors->getDataStoreRef().copyIntoBuffer(0, nonstd::span(actualColors.data(), actualColors.size())); + SIMPLNX_RESULT_REQUIRE_VALID(readResult); + CHECK(actualColors == expectedColors); + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::ComputeVectorColorsFilter: Repeated unmasked execution creates only colors", "[SimplnxCore][ComputeVectorColorsFilter]") { UnitTest::LoadPlugins(); diff --git a/src/Plugins/SimplnxCore/test/DREAM3DFileTest.cpp b/src/Plugins/SimplnxCore/test/DREAM3DFileTest.cpp index effa28cc57..e2223cb418 100644 --- a/src/Plugins/SimplnxCore/test/DREAM3DFileTest.cpp +++ b/src/Plugins/SimplnxCore/test/DREAM3DFileTest.cpp @@ -1699,6 +1699,79 @@ TEST_CASE("DREAM3DFileTest: Path Import Policy Tests", "[ReadDREAM3DFilter]") UnitTest::CheckArraysInheritTupleDims(dataStructure, SmallIn100::k_TupleCheckIgnoredPaths); } +TEST_CASE("SimplnxCore::ReadDREAM3DFilter: Include List warns for missing paths", "[SimplnxCore][ReadDREAM3DFilter]") +{ + Application::GetOrCreateInstance()->loadPlugins(unit_test::k_BuildDir.view(), true); + + DataStructure writtenData; + auto* group = DataGroup::Create(writtenData, "Group"); + REQUIRE(group != nullptr); + auto store = std::make_shared>(ShapeType{1}, ShapeType{1}, 42); + REQUIRE(Int32Array::Create(writtenData, "Values", store, group->getId()) != nullptr); + const fs::path filePath = GetTestFilePath("read_dream3d_missing_include_paths.dream3d"); + fs::remove(filePath); + auto writeResult = DREAM3D::WriteFile(filePath, writtenData, Pipeline{}, false); + SIMPLNX_RESULT_REQUIRE_VALID(writeResult); + + const DataPath validPath({"Group", "Values"}); + std::vector selectedPaths; + std::vector missingPaths; + SECTION("Missing leaf with existing ancestor") + { + missingPaths = {DataPath({"Group", "Missing"})}; + selectedPaths = missingPaths; + } + SECTION("Mixed valid and missing paths") + { + missingPaths = {DataPath({"Group", "Missing"}), DataPath({"Absent"})}; + selectedPaths = {validPath, missingPaths[0], missingPaths[1]}; + } + SECTION("Existing path") + { + selectedPaths = {validPath}; + } + + // Oracle: the file contains exactly Group and Group/Values (one scalar, 42). + // Group/Missing and Absent do not exist: each must produce a warning naming that path. + // Selecting Group/Values alone must produce no warning and must import the scalar unchanged. + DataStructure dataStructure; + ReadDREAM3DFilter filter; + Arguments args; + Dream3dImportParameter::ImportData importData(filePath, Dream3dImportParameter::PathImportPolicy::IncludeList, selectedPaths); + args.insert(ReadDREAM3DFilter::k_ImportFileData, importData); + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + CHECK(preflightResult.outputActions.warnings().size() == missingPaths.size()); + for(const auto& missingPath : missingPaths) + { + bool foundWarning = false; + for(const auto& warning : preflightResult.outputActions.warnings()) + { + foundWarning = foundWarning || warning.message.find(missingPath.toString()) != std::string::npos; + } + CHECK(foundWarning); + } + + auto executeResult = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + REQUIRE(dataStructure.containsData(DataPath({"Group"}))); + for(const auto& missingPath : missingPaths) + { + REQUIRE_FALSE(dataStructure.containsData(missingPath)); + } + if(selectedPaths.front() == validPath) + { + const auto* values = dataStructure.getDataAs(validPath); + REQUIRE(values != nullptr); + REQUIRE((*values)[0] == 42); + } + else + { + REQUIRE(dataStructure.getAllDataPaths().size() == 1); + } + fs::remove(filePath); +} + TEST_CASE("SimplnxCore::ReadDREAM3DFilter: SIMPL Backwards Compatibility", "[SimplnxCore][ReadDREAM3DFilter][BackwardsCompatibility]") { auto app = Application::GetOrCreateInstance(); diff --git a/src/Plugins/SimplnxCore/test/ExecuteProcessTest.cpp b/src/Plugins/SimplnxCore/test/ExecuteProcessTest.cpp index 5c2e5639d4..25007c9336 100644 --- a/src/Plugins/SimplnxCore/test/ExecuteProcessTest.cpp +++ b/src/Plugins/SimplnxCore/test/ExecuteProcessTest.cpp @@ -12,6 +12,7 @@ #include "simplnx/UnitTest/UnitTestCommon.hpp" #include "simplnx/Utilities/StringUtilities.hpp" +#include "SimplnxCore/Filters/Algorithms/ExecuteProcess.hpp" #include "SimplnxCore/Filters/ExecuteProcessFilter.hpp" #include @@ -146,3 +147,21 @@ TEST_CASE("SimplnxCore::ExecuteProcessFilter: SIMPL Backwards Compatibility", "[ } } } + +TEST_CASE("SimplnxCore::ExecuteProcessFilter: Argument separator regression", "[SimplnxCore][ExecuteProcessFilter]") +{ + // Oracle: spaces outside quotes separate tokens; spaces inside quotes are literal. + // Each command below therefore has exactly the explicitly listed arguments. + const std::vector>> cases = { + {R"("/a b/prog" --help)", {"/a b/prog", "--help"}}, {"prog --help", {"prog", "--help"}}, {R"(prog "a b" --help)", {"prog", "a b", "--help"}}, + {R"( "a b" " c d " )", {"a b", " c d "}}, {"prog --help", {"prog", "--help"}}, + }; + for(const auto& [command, expected] : cases) + { + DYNAMIC_SECTION(command) + { + const auto actual = ExecuteProcess::splitArgumentsString(command); + REQUIRE(actual == expected); + } + } +} diff --git a/src/Plugins/SimplnxCore/test/FeatureFaceCurvatureTest.cpp b/src/Plugins/SimplnxCore/test/FeatureFaceCurvatureTest.cpp index 5c0cfbf636..3bffd87709 100644 --- a/src/Plugins/SimplnxCore/test/FeatureFaceCurvatureTest.cpp +++ b/src/Plugins/SimplnxCore/test/FeatureFaceCurvatureTest.cpp @@ -267,3 +267,85 @@ TEST_CASE("SimplnxCore::FeatureFaceCurvatureFilter: NaN patch data", "[FeatureFa } UnitTest::CheckArraysInheritTupleDims(dataStructure); } + +TEST_CASE("SimplnxCore::FeatureFaceCurvatureFilter: Single triangle face", "[SimplnxCore][FeatureFaceCurvatureFilter][SingleTriangle]") +{ + const bool useNormals = GENERATE(false, true); + const bool optionalOutputs = GENERATE(false, true); + CAPTURE(useNormals, optionalOutputs); + + DataStructure dataStructure; + auto* geom = TriangleGeom::Create(dataStructure, "Triangles"); + REQUIRE(geom != nullptr); + auto* faceData = AttributeMatrix::Create(dataStructure, "FaceData", {1}, geom->getId()); + REQUIRE(faceData != nullptr); + geom->setFaceAttributeMatrix(*faceData); + auto* vertices = Float32Array::Create(dataStructure, "Vertices", std::make_unique>(ShapeType{3}, ShapeType{3}, 0.0F), geom->getId()); + auto* faces = DataArray::Create(dataStructure, "Faces", std::make_unique>(ShapeType{1}, ShapeType{3}, 0), geom->getId()); + REQUIRE(vertices != nullptr); + REQUIRE(faces != nullptr); + geom->setVertices(*vertices); + geom->setFaceList(*faces); + auto* labels = Int32Array::Create(dataStructure, "Labels", std::make_unique>(ShapeType{1}, ShapeType{2}, 1), faceData->getId()); + auto* ids = Int32Array::Create(dataStructure, "Ids", std::make_unique>(ShapeType{1}, ShapeType{1}, 1), faceData->getId()); + auto* normals = Float64Array::Create(dataStructure, "Normals", std::make_unique>(ShapeType{1}, ShapeType{3}, 0.0), faceData->getId()); + auto* centroids = Float64Array::Create(dataStructure, "Centroids", std::make_unique>(ShapeType{1}, ShapeType{3}, 0.0), faceData->getId()); + REQUIRE(labels != nullptr); + REQUIRE(ids != nullptr); + REQUIRE(normals != nullptr); + REQUIRE(centroids != nullptr); + + // Triangle (0,0,0), (1,0,0), (0,1,0): centroid (1/3,1/3,0), normal +Z. + vertices->setComponent(1, 0, 1.0F); + vertices->setComponent(2, 1, 1.0F); + faces->setComponent(0, 1, 1); + faces->setComponent(0, 2, 2); + labels->setComponent(0, 1, 2); + normals->setComponent(0, 2, 1.0); + centroids->setComponent(0, 0, 1.0 / 3.0); + centroids->setComponent(0, 1, 1.0 / 3.0); + + const DataPath geomPath({"Triangles"}); + const DataPath facePath = geomPath.createChildPath("FaceData"); + Arguments args; + args.insertOrAssign(FeatureFaceCurvatureFilter::k_TriangleGeomPath_Key, std::make_any(geomPath)); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_FaceAttribMatrixPath_Key, std::make_any(facePath)); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_NeighborhoodRing_Key, std::make_any(1)); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_UseFaceNormals_Key, std::make_any(useNormals)); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_ComputePrincipalDirection_Key, std::make_any(optionalOutputs)); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_ComputeGaussianCurvature_Key, std::make_any(optionalOutputs)); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_ComputeMeanCurvaturePath_Key, std::make_any(optionalOutputs)); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_ComputeWeingartenMatrix_Key, std::make_any(optionalOutputs)); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_FaceLabelsPath_Key, std::make_any(facePath.createChildPath("Labels"))); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_FeatureFaceIdsPath_Key, std::make_any(facePath.createChildPath("Ids"))); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_FaceNormalsPath_Key, std::make_any(facePath.createChildPath("Normals"))); + args.insertOrAssign(FeatureFaceCurvatureFilter::k_FaceCentroidsPath_Key, std::make_any(facePath.createChildPath("Centroids"))); + const std::array outputKeys = {FeatureFaceCurvatureFilter::k_PrincipalCurvature1Path_Key, FeatureFaceCurvatureFilter::k_PrincipalCurvature2Path_Key, + FeatureFaceCurvatureFilter::k_GaussianCurvaturePath_Key, FeatureFaceCurvatureFilter::k_MeanCurvaturePath_Key, + FeatureFaceCurvatureFilter::k_WeingartenMatrixPath_Key, FeatureFaceCurvatureFilter::k_PrincipalDirection1Path_Key, + FeatureFaceCurvatureFilter::k_PrincipalDirection2Path_Key}; + for(usize i = 0; i < outputKeys.size(); ++i) + { + args.insertOrAssign(outputKeys[i], std::make_any(facePath.createChildPath("Output" + std::to_string(i)))); + } + FeatureFaceCurvatureFilter filter; + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + auto executeResult = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + + // Independent oracle: one centroid and normal cannot determine surface curvature. + // The insufficient-patch convention requires NaN in every requested component. + for(usize i = 0; i < (optionalOutputs ? outputKeys.size() : 2); ++i) + { + const auto* output = dataStructure.getDataAs(facePath.createChildPath("Output" + std::to_string(i))); + REQUIRE(output != nullptr); + REQUIRE(output->getNumberOfTuples() == 1); + for(usize c = 0; c < output->getNumberOfComponents(); ++c) + { + CAPTURE(i, c); + CHECK(std::isnan(output->getDataStoreRef().getComponentValue(0, c))); + } + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} diff --git a/src/Plugins/SimplnxCore/test/QuickSurfaceMeshFilterTest.cpp b/src/Plugins/SimplnxCore/test/QuickSurfaceMeshFilterTest.cpp index 899ff7c195..a8761c73d5 100644 --- a/src/Plugins/SimplnxCore/test/QuickSurfaceMeshFilterTest.cpp +++ b/src/Plugins/SimplnxCore/test/QuickSurfaceMeshFilterTest.cpp @@ -114,6 +114,52 @@ void RequirePreflightWithoutCellAttributeMatrix(UnitTest::AlgorithmTestScope* al } UnitTest::CheckArraysInheritTupleDims(dataStructure); } + +struct SwapExemplarSides +{ + template + void operator()(IDataArray& array, usize face) const + { + auto& typed = dynamic_cast&>(array); + const usize components = typed.getNumberOfComponents() / 2; + for(usize component = 0; component < components; component++) + { + const usize first = face * components * 2 + component; + const usize second = first + components; + const T value = typed[first]; + typed[first] = typed[second]; + typed[second] = value; + } + } +}; + +void CorrectLegacyExemplarTransferOrder(DataStructure& dataStructure, const DataPath& facePath) +{ + // The legacy exemplar transferred FeatureIds record the source side of every transferred tuple. + // Use those IDs, independently of the computed mesh, to align all exemplar tuples with Face Labels. + REQUIRE_NOTHROW(dataStructure.getDataRefAs(facePath.createChildPath(k_FeatureIds))); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(facePath.createChildPath(k_Face_Labels))); + const auto& ids = dataStructure.getDataRefAs(facePath.createChildPath(k_FeatureIds)); + const auto& labels = dataStructure.getDataRefAs(facePath.createChildPath(k_Face_Labels)); + auto& faces = dataStructure.getDataRefAs(facePath); + for(usize face = 0; face < labels.getNumberOfTuples(); face++) + { + if(labels[2 * face] >= 0 && labels[2 * face + 1] >= 0 && ids[2 * face] != labels[2 * face]) + { + REQUIRE(ids[2 * face] == labels[2 * face + 1]); + REQUIRE(ids[2 * face + 1] == labels[2 * face]); + for(const auto& [id, object] : faces) + { + if(object->getName() != k_Face_Labels) + { + auto* array = dynamic_cast(object.get()); + REQUIRE(array != nullptr); + ExecuteDataFunction(SwapExemplarSides{}, array->getDataType(), *array, face); + } + } + } + } +} } // namespace TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: Image Geometry without a Cell Attribute Matrix", "[SimplnxCore][QuickSurfaceMeshFilter]") @@ -129,6 +175,95 @@ TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: Rectilinear Grid without a Cell RequirePreflightWithoutCellAttributeMatrix(); } +TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: Transferred sides match face labels", "[SimplnxCore][QuickSurfaceMeshFilter]") +{ + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + const usize axis = GENERATE(0, 1, 2); + const bool ascending = GENERATE(false, true); + CAPTURE(scenario, axis, ascending); + UnitTest::AlgorithmTestScope scope(scenario); + + DataStructure dataStructure; + auto* image = ImageGeom::Create(dataStructure, "Image"); + REQUIRE(image != nullptr); + SizeVec3 dims = {1, 1, 1}; + dims[axis] = 2; + image->setDimensions(dims); + image->setOrigin({0, 0, 0}); + image->setSpacing({1, 1, 1}); + const ShapeType shape = {dims[2], dims[1], dims[0]}; + auto* cells = AttributeMatrix::Create(dataStructure, "Cells", shape, image->getId()); + REQUIRE(cells != nullptr); + image->setCellData(cells->getId()); + auto* ids = Int32Array::CreateWithStore>(dataStructure, "Ids", shape, {1}, cells->getId()); + auto* values = Int32Array::CreateWithStore>(dataStructure, "Cell Values", shape, {2}, cells->getId()); + auto* features = AttributeMatrix::Create(dataStructure, "Features", {3}, image->getId()); + REQUIRE(features != nullptr); + auto* featureValues = Int32Array::CreateWithStore>(dataStructure, "Feature Values", {3}, {2}, features->getId()); + REQUIRE(ids != nullptr); + REQUIRE(values != nullptr); + REQUIRE(featureValues != nullptr); + // Independent oracle: feature f owns cell tuple (10*f, 10*f+1) and feature tuple (100*f, 100*f+1). + // Two adjacent unit cubes have exactly two internal triangles, each with labels (1, 2). + for(usize cell = 0; cell < 2; cell++) + { + const int32 id = ascending ? static_cast(cell + 1) : static_cast(2 - cell); + (*ids)[cell] = id; + for(usize component = 0; component < 2; component++) + { + (*values)[2 * cell + component] = 10 * id + static_cast(component); + } + } + for(usize f = 0; f < 3; f++) + { + (*featureValues)[2 * f] = static_cast(100 * f); + (*featureValues)[2 * f + 1] = static_cast(100 * f + 1); + } + QuickSurfaceMeshFilter filter; + Arguments args; + args.insertOrAssign(QuickSurfaceMeshFilter::k_GridGeometryDataPath_Key, std::make_any(DataPath({"Image"}))); + args.insertOrAssign(QuickSurfaceMeshFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(DataPath({"Image", "Cells", "Ids"}))); + args.insertOrAssign(QuickSurfaceMeshFilter::k_SelectedDataArrayPaths_Key, + std::make_any(MultiArraySelectionParameter::ValueType{DataPath({"Image", "Cells", "Cell Values"})})); + args.insertOrAssign(QuickSurfaceMeshFilter::k_SelectedFeatureDataArrayPaths_Key, + std::make_any(MultiArraySelectionParameter::ValueType{DataPath({"Image", "Features", "Feature Values"})})); + args.insertOrAssign(QuickSurfaceMeshFilter::k_FixProblemVoxels_Key, std::make_any(false)); + args.insertOrAssign(QuickSurfaceMeshFilter::k_RepairTriangleWinding_Key, std::make_any(false)); + args.insertOrAssign(QuickSurfaceMeshFilter::k_CreatedTriangleGeometryPath_Key, std::make_any(DataPath({"Mesh"}))); + args.insertOrAssign(QuickSurfaceMeshFilter::k_FaceDataGroupName_Key, std::make_any("Faces")); + auto result = scope.executeFilter(filter, dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(result.result); + const DataPath facePath({"Mesh", "Faces"}); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(facePath.createChildPath("FaceLabels"))); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(facePath.createChildPath("Cell Values"))); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(facePath.createChildPath("Feature Values"))); + const auto& labels = dataStructure.getDataRefAs(facePath.createChildPath("FaceLabels")); + const auto& transferredCells = dataStructure.getDataRefAs(facePath.createChildPath("Cell Values")); + const auto& transferredFeatures = dataStructure.getDataRefAs(facePath.createChildPath("Feature Values")); + usize internalFaces = 0; + for(usize face = 0; face < labels.getNumberOfTuples(); face++) + { + if(labels[2 * face] >= 0 && labels[2 * face + 1] >= 0) + { + internalFaces++; + CHECK(labels[2 * face] == 1); + CHECK(labels[2 * face + 1] == 2); + } + for(usize side = 0; side < 2; side++) + { + const int32 label = labels[2 * face + side]; + for(usize component = 0; component < 2; component++) + { + const usize index = 4 * face + 2 * side + component; + CHECK(transferredCells[index] == (label == -1 ? 0 : 10 * label + static_cast(component))); + CHECK(transferredFeatures[index] == (label == -1 ? 0 : 100 * label + static_cast(component))); + } + } + } + CHECK(internalFaces == 2); + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: Preserve winding and pruning warnings", "[SimplnxCore][QuickSurfaceMeshFilter]") { UnitTest::LoadPlugins(); @@ -303,6 +438,7 @@ TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter", "[SimplnxCore][QuickSurfaceMesh CompareArrays(dataStructure, exemplarSharedVertexPath, computedTriangleGeomPath.createChildPath(TriangleGeom::k_SharedVertexListName)); DataPath exemplarFaceAttrMatPath = exemplarTriangleGeomPath.createChildPath(k_FaceDataGroupName); + CorrectLegacyExemplarTransferOrder(dataStructure, exemplarFaceAttrMatPath); CompareExemplarToGenerateAttributeMatrix(dataStructure, exemplarFaceAttrMatPath, dataStructure, faceGroupDataPath, true); DataPath exemplarVertexAttrMatPath = exemplarTriangleGeomPath.createChildPath(k_VertexDataGroupName); @@ -400,6 +536,7 @@ TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: Winding", "[SimplnxCore][QuickSu CompareArrays(dataStructure, computedTriangleGeomPath.createChildPath(TriangleGeom::k_SharedVertexListName), exemplarSharedVertexPath); DataPath exemplarFaceAttrMatPath = exemplarTriangleGeomPath.createChildPath(k_FaceDataGroupName); + CorrectLegacyExemplarTransferOrder(dataStructure, exemplarFaceAttrMatPath); CompareExemplarToGenerateAttributeMatrix(dataStructure, exemplarFaceAttrMatPath, dataStructure, faceGroupDataPath, true); DataPath exemplarVertexAttrMatPath = exemplarTriangleGeomPath.createChildPath(k_VertexDataGroupName); @@ -497,6 +634,7 @@ TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: Problem Voxels", "[SimplnxCore][ CompareArrays(dataStructure, computedTriangleGeomPath.createChildPath(TriangleGeom::k_SharedVertexListName), exemplarSharedVertexPath); DataPath exemplarFaceAttrMatPath = exemplarTriangleGeomPath.createChildPath(k_FaceDataGroupName); + CorrectLegacyExemplarTransferOrder(dataStructure, exemplarFaceAttrMatPath); CompareExemplarToGenerateAttributeMatrix(dataStructure, exemplarFaceAttrMatPath, dataStructure, faceGroupDataPath, true); DataPath exemplarVertexAttrMatPath = exemplarTriangleGeomPath.createChildPath(k_VertexDataGroupName); @@ -594,6 +732,7 @@ TEST_CASE("SimplnxCore::QuickSurfaceMeshFilter: Winding and Problem Voxels", "[S CompareArrays(dataStructure, computedTriangleGeomPath.createChildPath(TriangleGeom::k_SharedVertexListName), exemplarSharedVertexPath); DataPath exemplarFaceAttrMatPath = exemplarTriangleGeomPath.createChildPath(k_FaceDataGroupName); + CorrectLegacyExemplarTransferOrder(dataStructure, exemplarFaceAttrMatPath); CompareExemplarToGenerateAttributeMatrix(dataStructure, exemplarFaceAttrMatPath, dataStructure, faceGroupDataPath, true); DataPath exemplarVertexAttrMatPath = exemplarTriangleGeomPath.createChildPath(k_VertexDataGroupName); diff --git a/src/Plugins/SimplnxCore/test/ReadStlFileTest.cpp b/src/Plugins/SimplnxCore/test/ReadStlFileTest.cpp index 5db3c9f179..035f377458 100644 --- a/src/Plugins/SimplnxCore/test/ReadStlFileTest.cpp +++ b/src/Plugins/SimplnxCore/test/ReadStlFileTest.cpp @@ -165,6 +165,33 @@ TEST_CASE("SimplnxCore::ReadStlFileFilter:Valid_File", "[SimplnxCore][ReadStlFil UnitTest::CheckArraysInheritTupleDims(dataStructure); } +TEST_CASE("SimplnxCore::ReadStlFileFilter:CorruptTriangleCount", "[SimplnxCore][ReadStlFileFilter]") +{ + const fs::path inputFile = fs::path(unit_test::k_BinaryTestOutputDir.view()) / "ReadStlFileTest" / "corrupt_triangle_count.stl"; + WriteBinaryStlFile(inputFile, "Corrupt triangle count", 0x7FFFFFFF, std::vector(1)); + + // Binary STL needs 80 + 4 + 50 * N bytes: 107374182434 for N = 0x7FFFFFFF. + // This file has only 134 bytes (one triangle). Reject it before creating output actions + // so execution cannot allocate arrays for the corrupt count. + REQUIRE(fs::file_size(inputFile) == 134); + + DataStructure dataStructure; + Arguments args; + ReadStlFileFilter filter; + args.insertOrAssign(ReadStlFileFilter::k_StlFilePath_Key, std::make_any(inputFile)); + args.insertOrAssign(ReadStlFileFilter::k_CreatedTriangleGeometryPath_Key, std::make_any(DataPath({"[Triangle Geometry]"}))); + + IFilter::PreflightResult preflightResult; + REQUIRE_NOTHROW(preflightResult = filter.preflight(dataStructure, args)); + SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); + REQUIRE(preflightResult.outputActions.errors().front().code == StlConstants::k_StlFileLengthError); + + IFilter::ExecuteResult executeResult; + REQUIRE_NOTHROW(executeResult = filter.execute(dataStructure, args)); + SIMPLNX_RESULT_REQUIRE_INVALID(executeResult.result); + REQUIRE(executeResult.result.errors().front().code == StlConstants::k_StlFileLengthError); +} + TEST_CASE("SimplnxCore::ReadStlFileFilter:STLParseError", "[SimplnxCore][ReadStlFileFilter]") { UnitTest::LoadPlugins(); @@ -184,15 +211,16 @@ TEST_CASE("SimplnxCore::ReadStlFileFilter:STLParseError", "[SimplnxCore][ReadStl args.insertOrAssign(ReadStlFileFilter::k_StlFilePath_Key, std::make_any(fs::path(inputFile))); args.insertOrAssign(ReadStlFileFilter::k_CreatedTriangleGeometryPath_Key, std::make_any(triangleGeomDataPath)); - // Preflight the filter and check result + // Reject the truncated file before allocating output arrays auto preflightResult = filter.preflight(dataStructure, args); - SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); + REQUIRE(preflightResult.outputActions.errors().front().code == StlConstants::k_StlFileLengthError); // Execute the filter and check the result auto executeResult = filter.execute(dataStructure, args); SIMPLNX_RESULT_REQUIRE_INVALID(executeResult.result); - REQUIRE(executeResult.result.errors().front().code == -1108); + REQUIRE(executeResult.result.errors().front().code == StlConstants::k_StlFileLengthError); UnitTest::CheckArraysInheritTupleDims(dataStructure); } @@ -216,15 +244,16 @@ TEST_CASE("SimplnxCore::ReadStlFileFilter:TriangleParseError", "[SimplnxCore][Re args.insertOrAssign(ReadStlFileFilter::k_StlFilePath_Key, std::make_any(fs::path(inputFile))); args.insertOrAssign(ReadStlFileFilter::k_CreatedTriangleGeometryPath_Key, std::make_any(triangleGeomDataPath)); - // Preflight the filter and check result + // Reject the truncated file before allocating output arrays auto preflightResult = filter.preflight(dataStructure, args); - SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); + REQUIRE(preflightResult.outputActions.errors().front().code == StlConstants::k_StlFileLengthError); // Execute the filter and check the result auto executeResult = filter.execute(dataStructure, args); SIMPLNX_RESULT_REQUIRE_INVALID(executeResult.result); - REQUIRE(executeResult.result.errors().front().code == -1106); + REQUIRE(executeResult.result.errors().front().code == StlConstants::k_StlFileLengthError); UnitTest::CheckArraysInheritTupleDims(dataStructure); } @@ -248,15 +277,16 @@ TEST_CASE("SimplnxCore::ReadStlFileFilter:AttributeParseError", "[SimplnxCore][R args.insertOrAssign(ReadStlFileFilter::k_StlFilePath_Key, std::make_any(fs::path(inputFile))); args.insertOrAssign(ReadStlFileFilter::k_CreatedTriangleGeometryPath_Key, std::make_any(triangleGeomDataPath)); - // Preflight the filter and check result + // Reject the truncated file before allocating output arrays auto preflightResult = filter.preflight(dataStructure, args); - SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); + REQUIRE(preflightResult.outputActions.errors().front().code == StlConstants::k_StlFileLengthError); // Execute the filter and check the result auto executeResult = filter.execute(dataStructure, args); SIMPLNX_RESULT_REQUIRE_INVALID(executeResult.result); - REQUIRE(executeResult.result.errors().front().code == -1107); + REQUIRE(executeResult.result.errors().front().code == StlConstants::k_StlFileLengthError); UnitTest::CheckArraysInheritTupleDims(dataStructure); } @@ -388,6 +418,46 @@ TEST_CASE("SimplnxCore::ReadStlFileFilter:ShortFileWithAttributePayload", "[Simp REQUIRE(executeResult.result.errors().front().message.find(std::to_string(expectedFileSize)) != std::string::npos); } +TEST_CASE("SimplnxCore::ReadStlFileFilter:TruncatedRecordAfterPayload", "[SimplnxCore][ReadStlFileFilter]") +{ + const DataPath triangleGeomDataPath({"[Triangle Geometry]"}); + + // Triangle 0 contains 10 payload bytes. The file exceeds the 184-byte minimum for two records, + // so the reader honors attribute counts. Triangle 1 starts at byte + // 84 + 50 + 10 = 144 and is cut short, so the read loop itself must report where it stopped. + // - 44 trailing bytes (file size 188): only 11 of the 12 floats can be read. + // - 48 trailing bytes (file size 192): the 12 floats are read, the 2 byte attribute count is not. + const uint16_t k_PayloadBytes = 10; + std::vector triangleSpecs(1); + triangleSpecs[0].attributeByteCount = k_PayloadBytes; + triangleSpecs[0].payloadBytes = k_PayloadBytes; + + auto [trailingBytes, expectedError] = GENERATE(table({{44, StlConstants::k_TriangleParseError}, {48, StlConstants::k_AttributeParseError}})); + + DYNAMIC_SECTION("Trailing bytes: " << trailingBytes) + { + const fs::path inputFile = fs::path(unit_test::k_BinaryTestOutputDir.view()) / "ReadStlFileTest" / "truncated_record_after_payload.stl"; + WriteBinaryStlFile(inputFile, "Payload then a truncated record", 2, triangleSpecs); + { + std::ofstream outputFile(inputFile, std::ios::binary | std::ios::app); + REQUIRE(outputFile.is_open()); + const std::vector partialRecord(trailingBytes, 0x00); + outputFile.write(partialRecord.data(), static_cast(partialRecord.size())); + } + REQUIRE(fs::file_size(inputFile) == 144 + trailingBytes); + + const StlConstants::StlFileCheck stlFileCheck = StlUtilities::SanityCheckFile(inputFile); + REQUIRE(stlFileCheck.error == 0); + REQUIRE(stlFileCheck.numTriangles == 2); + REQUIRE(stlFileCheck.attributePayloadPresent == true); + + DataStructure dataStructure; + auto executeResult = RunReadStlFileFilter(dataStructure, inputFile, triangleGeomDataPath); + SIMPLNX_RESULT_REQUIRE_INVALID(executeResult.result); + REQUIRE(executeResult.result.errors().front().code == expectedError); + } +} + TEST_CASE("SimplnxCore::StlUtilities:SanityCheckFile", "[SimplnxCore][ReadStlFileFilter]") { const fs::path testDir = fs::path(unit_test::k_BinaryTestOutputDir.view()) / "ReadStlFileTest"; @@ -442,16 +512,15 @@ TEST_CASE("SimplnxCore::StlUtilities:SanityCheckFile", "[SimplnxCore][ReadStlFil REQUIRE(stlFileCheck.numTriangles == 0); } - SECTION("Truncated triangle data reports no payload so the reader does not seek") + SECTION("Truncated triangle data is rejected before allocating mesh arrays") { - // The header claims more triangles than the file can hold. The read loop is responsible for - // reporting exactly where it ran out, so SanityCheckFile must not seek past anything. + // 100 triangles require at least 84 + 50 * 100 = 5084 bytes; four records provide only 284. const fs::path inputFile = testDir / "truncated_triangle_data.stl"; WriteBinaryStlFile(inputFile, "Header count is too large", 100, std::vector(4)); const StlConstants::StlFileCheck stlFileCheck = StlUtilities::SanityCheckFile(inputFile); - REQUIRE(stlFileCheck.error == 0); - REQUIRE(stlFileCheck.numTriangles == 100); + REQUIRE(stlFileCheck.error == StlConstants::k_StlFileLengthError); + REQUIRE(stlFileCheck.numTriangles == 0); REQUIRE(stlFileCheck.attributePayloadPresent == false); } } diff --git a/src/Plugins/SimplnxCore/test/ReadVtkStructuredPointsTest.cpp b/src/Plugins/SimplnxCore/test/ReadVtkStructuredPointsTest.cpp index 9ed33e3ef8..828f1cf5c2 100644 --- a/src/Plugins/SimplnxCore/test/ReadVtkStructuredPointsTest.cpp +++ b/src/Plugins/SimplnxCore/test/ReadVtkStructuredPointsTest.cpp @@ -96,6 +96,200 @@ void WriteSectionValues(std::ostream& output, nonstd::span values, bool } } // namespace +TEST_CASE("SimplnxCore::ReadVtkStructuredPointsFilter: Singleton and Zero Dimensions", "[SimplnxCore][ReadVtkStructuredPointsFilter]") +{ + const fs::path filePath = fs::path(unit_test::k_BinaryDir.view()) / "ReadVtkDimensions.vtk"; + auto fileGuard = MakeScopeGuard([&filePath]() noexcept { + std::error_code errorCode; + fs::remove(filePath, errorCode); + }); + ReadVtkStructuredPointsFilter filter; + DataStructure ds; + Arguments args = createArgs(filePath); + + SECTION("Singleton axes retain one cell dimension") + { + const bool binary = GENERATE(false, true); + const usize grid = GENERATE(0, 1, 2, 3, 4, 5, 6, 7); + CAPTURE(binary, grid); + // Hand calculation: adjacent points define n-1 cells along an active axis. + // A singleton axis contributes a factor of one, including a single vertex cell. + // Thus 10x10x1 points have 9x9x1 = 81 cells, and 4x1x1 points have 3 cells. + const std::array pointDimensions = {SizeVec3{10, 10, 1}, SizeVec3{3, 1, 4}, SizeVec3{1, 3, 4}, SizeVec3{4, 1, 1}, + SizeVec3{1, 4, 1}, SizeVec3{1, 1, 4}, SizeVec3{1, 1, 1}, SizeVec3{3, 3, 3}}; + const std::array cellDimensions = {SizeVec3{9, 9, 1}, SizeVec3{2, 1, 3}, SizeVec3{1, 2, 3}, SizeVec3{3, 1, 1}, + SizeVec3{1, 3, 1}, SizeVec3{1, 1, 3}, SizeVec3{1, 1, 1}, SizeVec3{2, 2, 2}}; + const auto& pointDims = pointDimensions[grid]; + const auto& cellDims = cellDimensions[grid]; + std::vector pointValues(pointDims[0] * pointDims[1] * pointDims[2]); + std::vector cellValues(cellDims[0] * cellDims[1] * cellDims[2]); + for(usize index = 0; index < pointValues.size(); index++) + { + pointValues[index] = static_cast(index) + 1; + } + for(usize index = 0; index < cellValues.size(); index++) + { + cellValues[index] = static_cast(index) + 100; + } + { + std::ofstream output(filePath, std::ios::binary | std::ios::trunc); + REQUIRE(output.is_open()); + output << "# vtk DataFile Version 3.0\nDimension regression\n" << (binary ? "BINARY\n" : "ASCII\n"); + output << "DATASET STRUCTURED_POINTS\nDIMENSIONS " << pointDims[0] << ' ' << pointDims[1] << ' ' << pointDims[2] << "\nSPACING 1 1 1\nORIGIN 0 0 0\n"; + output << "POINT_DATA " << pointValues.size() << "\nSCALARS Values int 1\nLOOKUP_TABLE default\n"; + WriteSectionValues(output, pointValues, binary); + output << "CELL_DATA " << cellValues.size() << "\nSCALARS Values int 1\nLOOKUP_TABLE default\n"; + WriteSectionValues(output, cellValues, binary); + output.close(); + REQUIRE(output.good()); + } + + const auto preflightResult = filter.preflight(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + const auto executeResult = filter.execute(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + for(const bool points : {false, true}) + { + const DataPath geomPath({points ? "VTK Point Data" : "VTK Cell Data"}); + const auto& geom = ds.getDataRefAs(geomPath); + REQUIRE(geom.getDimensions() == (points ? pointDims : cellDims)); + const auto& values = ds.getDataRefAs(geomPath.createChildPath(points ? "Vertex Data" : "Cell Data").createChildPath("Values")); + REQUIRE(values.getNumberOfTuples() == (points ? pointValues.size() : cellValues.size())); + REQUIRE(values.getNumberOfComponents() == 1); + for(usize index = 0; index < values.getNumberOfTuples(); index++) + { + REQUIRE(values[index] == static_cast(index) + (points ? 1 : 100)); + } + } + UnitTest::CheckArraysInheritTupleDims(ds); + } + + SECTION("Zero dimensions are rejected before creating geometry") + { + const usize axis = GENERATE(0, 1, 2); + CAPTURE(axis); + SizeVec3 pointDims{1, 1, 1}; + pointDims[axis] = 0; + // A structured-points dimension must be at least one. A zero axis is invalid, + // even when CELL_DATA declares zero tuples; unsigned subtraction must not wrap. + { + std::ofstream output(filePath, std::ios::trunc); + REQUIRE(output.is_open()); + output << "# vtk DataFile Version 3.0\nZero dimension\nASCII\nDATASET STRUCTURED_POINTS\nDIMENSIONS "; + output << pointDims[0] << ' ' << pointDims[1] << ' ' << pointDims[2] << "\nSPACING 1 1 1\nORIGIN 0 0 0\nCELL_DATA 0\n"; + output.close(); + REQUIRE(output.good()); + } + args.insertOrAssign(ReadVtkStructuredPointsFilter::k_ReadPointData_Key, std::make_any(false)); + IFilter::PreflightResult preflightResult; + REQUIRE_NOTHROW(preflightResult = filter.preflight(ds, args)); + SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); + REQUIRE(preflightResult.outputActions.errors()[0].code == -10354); + // The message names the axis as X, Y or Z, not as a zero-based index. + const std::array axisNames = {"X axis", "Y axis", "Z axis"}; + REQUIRE(preflightResult.outputActions.errors()[0].message.find(axisNames[axis]) != std::string::npos); + IFilter::ExecuteResult executeResult; + REQUIRE_NOTHROW(executeResult = filter.execute(ds, args)); + SIMPLNX_RESULT_REQUIRE_INVALID(executeResult.result); + REQUIRE(executeResult.result.errors().size() == 1); + REQUIRE(executeResult.result.errors()[0].code == -10354); + REQUIRE(ds.getData(DataPath({"VTK Cell Data"})) == nullptr); + } +} + +TEST_CASE("SimplnxCore::ReadVtkStructuredPointsFilter: Singleton Axis Physical Position", "[SimplnxCore][ReadVtkStructuredPointsFilter]") +{ + const bool binary = GENERATE(false, true); + const usize grid = GENERATE(0, 1, 2, 3, 4); + CAPTURE(binary, grid); + // Hand calculation: The origin is (2, 6, 10), and spacing is (2, 6, 4). On a singleton + // axis, subtract half the spacing to center the voxel on the VTK point. + // The Z plane therefore has origin Z = 8 and center Z = 10. The next three + // cases are X, Y and Z lines; the last is a 3D control with unchanged origin. + // Active axes have n-1 cells and retain their original origin and spacing. + const std::array pointDimensions = {SizeVec3{3, 3, 1}, SizeVec3{3, 1, 1}, SizeVec3{1, 3, 1}, SizeVec3{1, 1, 3}, SizeVec3{3, 3, 3}}; + const std::array cellDimensions = {SizeVec3{2, 2, 1}, SizeVec3{2, 1, 1}, SizeVec3{1, 2, 1}, SizeVec3{1, 1, 2}, SizeVec3{2, 2, 2}}; + const std::array expectedOrigins = {FloatVec3{2, 6, 8}, FloatVec3{2, 3, 8}, FloatVec3{1, 6, 8}, FloatVec3{1, 3, 10}, FloatVec3{2, 6, 10}}; + const std::array expectedCenters = {FloatVec3{3, 9, 10}, FloatVec3{3, 6, 10}, FloatVec3{2, 9, 10}, FloatVec3{2, 6, 12}, FloatVec3{3, 9, 12}}; + const auto& pointDims = pointDimensions[grid]; + const auto& cellDims = cellDimensions[grid]; + std::vector pointValues(pointDims[0] * pointDims[1] * pointDims[2]); + std::vector cellValues(cellDims[0] * cellDims[1] * cellDims[2]); + for(usize index = 0; index < pointValues.size(); index++) + { + pointValues[index] = static_cast(index) + 1; + } + for(usize index = 0; index < cellValues.size(); index++) + { + cellValues[index] = static_cast(index) + 100; + } + + const fs::path filePath = fs::path(unit_test::k_BinaryDir.view()) / "ReadVtkPhysicalPosition.vtk"; + auto fileGuard = MakeScopeGuard([&filePath]() noexcept { + std::error_code errorCode; + fs::remove(filePath, errorCode); + }); + { + std::ofstream output(filePath, std::ios::binary | std::ios::trunc); + REQUIRE(output.is_open()); + output << "# vtk DataFile Version 3.0\nPhysical position regression\n" << (binary ? "BINARY\n" : "ASCII\n"); + output << "DATASET STRUCTURED_POINTS\nDIMENSIONS " << pointDims[0] << ' ' << pointDims[1] << ' ' << pointDims[2] << "\nSPACING 2 6 4\nORIGIN 2 6 10\n"; + output << "POINT_DATA " << pointValues.size() << "\nSCALARS Values int 1\nLOOKUP_TABLE default\n"; + WriteSectionValues(output, pointValues, binary); + output << "CELL_DATA " << cellValues.size() << "\nSCALARS Values int 1\nLOOKUP_TABLE default\n"; + WriteSectionValues(output, cellValues, binary); + output.close(); + REQUIRE(output.good()); + } + + ReadVtkStructuredPointsFilter filter; + Arguments args = createArgs(filePath); + for(const bool preflight : {true, false}) + { + CAPTURE(preflight); + DataStructure ds; + if(preflight) + { + auto preflightResult = filter.preflight(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + const auto applyResult = preflightResult.outputActions.value().applyRegular(ds, IDataAction::Mode::Preflight); + SIMPLNX_RESULT_REQUIRE_VALID(applyResult); + } + else + { + const auto executeResult = filter.execute(ds, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + } + for(const bool points : {false, true}) + { + CAPTURE(points); + const DataPath geomPath({points ? "VTK Point Data" : "VTK Cell Data"}); + const auto& geom = ds.getDataRefAs(geomPath); + REQUIRE(geom.getDimensions() == (points ? pointDims : cellDims)); + REQUIRE(geom.getSpacing() == FloatVec3{2, 6, 4}); + const auto center = geom.getCoordsf(0, 0, 0); + for(usize axis = 0; axis < 3; axis++) + { + CAPTURE(axis); + CHECK(geom.getOrigin()[axis] == expectedOrigins[grid][axis]); + CHECK(center[axis] == expectedCenters[grid][axis]); + } + const auto& values = ds.getDataRefAs(geomPath.createChildPath(points ? "Vertex Data" : "Cell Data").createChildPath("Values")); + REQUIRE(values.getNumberOfTuples() == (points ? pointValues.size() : cellValues.size())); + REQUIRE(values.getNumberOfComponents() == 1); + if(!preflight) + { + // Values must match the literal sequence written above, regardless of origin. + for(usize index = 0; index < values.getNumberOfTuples(); index++) + { + REQUIRE(values[index] == static_cast(index) + (points ? 1 : 100)); + } + } + } + UnitTest::CheckArraysInheritTupleDims(ds); + } +} + TEST_CASE("SimplnxCore::ReadVtkStructuredPointsFilter: Section Selection", "[SimplnxCore][ReadVtkStructuredPointsFilter]") { UnitTest::LoadPlugins(); diff --git a/src/Plugins/SimplnxCore/test/RotateSampleRefFrameTest.cpp b/src/Plugins/SimplnxCore/test/RotateSampleRefFrameTest.cpp index 575beaf8e4..3ed2c9fed6 100644 --- a/src/Plugins/SimplnxCore/test/RotateSampleRefFrameTest.cpp +++ b/src/Plugins/SimplnxCore/test/RotateSampleRefFrameTest.cpp @@ -810,3 +810,41 @@ TEST_CASE("SimplnxCore::RotateSampleRefFrame: Child names containing the geometr } UnitTest::CheckArraysInheritTupleDims(dataStructure); } + +TEST_CASE("SimplnxCore::RotateSampleRefFrame: anisotropic three-cycle preserves the voxel grid", "[SimplnxCore][RotateSampleRefFrameFilter]") +{ + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + CAPTURE(scenario); + UnitTest::AlgorithmTestScope algorithmTestScope(scenario); + DataStructure dataStructure; + CreateSequentialImageGeom(dataStructure, "Input", {2, 3, 4}, {2.0F, 3.0F, 5.0F}); + RotateSampleRefFrameFilter filter; + Arguments args = MakeAxisAngleArgs(k_InputPath, k_OutputPath, {1.0F, 1.0F, 1.0F, 120.0F}); + // Closed-form oracle: R(x,y,z)=(z,x,y). Thus dimensions (2,3,4) and + // spacing (2,3,5) become (4,2,3) and (5,2,3). Output (i,j,k) samples + // input (j,k,i), whose sequential value is 1 + j + 2*k + 6*i. + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + auto executeResult = algorithmTestScope.executeFilter(filter, dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + const ImageGeom* output = nullptr; + REQUIRE_NOTHROW(output = &dataStructure.getDataRefAs(k_OutputPath)); + CHECK(output->getDimensions() == SizeVec3{4, 2, 3}); + for(usize axis = 0; axis < 3; axis++) + { + CHECK(output->getSpacing()[axis] == Approx(FloatVec3{5.0F, 2.0F, 3.0F}[axis])); + } + const auto values = ReadOutputValues(dataStructure, k_OutputPath); + REQUIRE(values.size() == 24); + for(usize k = 0; k < 3; k++) + { + for(usize j = 0; j < 2; j++) + { + for(usize i = 0; i < 4; i++) + { + CHECK(values[i + 4 * j + 8 * k] == static_cast(1 + j + 2 * k + 6 * i)); + } + } + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} diff --git a/src/Plugins/SimplnxCore/test/WriteOnScaleTableFileTest.cpp b/src/Plugins/SimplnxCore/test/WriteOnScaleTableFileTest.cpp index f599aaed93..f1f62a4938 100644 --- a/src/Plugins/SimplnxCore/test/WriteOnScaleTableFileTest.cpp +++ b/src/Plugins/SimplnxCore/test/WriteOnScaleTableFileTest.cpp @@ -562,9 +562,12 @@ TEST_CASE("SimplnxCore::WriteOnScaleTableFileFilter: Reorders Z Then X", "[Simpl RequireReorderedExportMatchesRotation({2, 4, 3}, compositeRotation, outputPath, "OnScale_Reordered_ZX"); } -TEST_CASE("SimplnxCore::WriteOnScaleTableFileFilter: Anisotropic Two Axis Reorder Fails Loudly", "[SimplnxCore][WriteOnScaleTableFileFilter]") +TEST_CASE("SimplnxCore::WriteOnScaleTableFileFilter: Anisotropic Two Axis Reorder Preserves Cells", "[SimplnxCore][WriteOnScaleTableFileFilter]") { UnitTest::LoadPlugins(); + const auto scenario = GENERATE(from_range(UnitTest::SelectAlgorithmTestScenariosForInMemoryStores())); + CAPTURE(scenario); + UnitTest::AlgorithmTestScope algorithmTestScope(scenario); std::vector featureIds(24); std::iota(featureIds.begin(), featureIds.end(), 1); @@ -574,12 +577,14 @@ TEST_CASE("SimplnxCore::WriteOnScaleTableFileFilter: Anisotropic Two Axis Reorde fs::create_directories(outputPath); const WriteOnScaleTableFileFilter filter; - auto executeResult = filter.execute(dataStructure, CreateArguments(outputPath, "OnScale_Anisotropic_Two_Axis")); - REQUIRE(executeResult.result.invalid()); - REQUIRE(executeResult.result.errors().front().code == -12041); - REQUIRE(executeResult.result.errors().front().message.find("produced dimensions (Z=1, Y=9, X=3)") != std::string::npos); - REQUIRE(executeResult.result.errors().front().message.find("expected (Z=2, Y=3, X=4)") != std::string::npos); - REQUIRE_FALSE(fs::exists(outputPath / "OnScale_Anisotropic_Two_Axis.flxtbl")); + auto executeResult = algorithmTestScope.executeFilter(filter, dataStructure, CreateArguments(outputPath, "OnScale_Anisotropic_Two_Axis")); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + const std::string contents = ReadFile(outputPath / "OnScale_Anisotropic_Two_Axis.flxtbl"); + // Rx * Rz maps (x,y,z) to (-y,-z,x). Dimensions become (4,3,2). + // Output (i,j,k) samples input (k,3-i,2-j), giving 1+k+2*(3-i)+8*(2-j). + REQUIRE(contents.find("divisions\n4 3 2\n") != std::string::npos); + REQUIRE(ExtractMatrBlock(contents) == "23 21 19 17 15 13 11 9 7 5 3 1 24 22 20 18 16 14 12 10 8 6 4 2"); + RequireFeatureIdMultiset(ExtractMatrBlock(contents), featureIds); UnitTest::CheckArraysInheritTupleDims(dataStructure); } diff --git a/src/Plugins/SimplnxCore/test/WriteVtkRectilinearGridTest.cpp b/src/Plugins/SimplnxCore/test/WriteVtkRectilinearGridTest.cpp index 1a6eb8ed6c..e5b58dd9d9 100644 --- a/src/Plugins/SimplnxCore/test/WriteVtkRectilinearGridTest.cpp +++ b/src/Plugins/SimplnxCore/test/WriteVtkRectilinearGridTest.cpp @@ -182,7 +182,7 @@ void AppendCoordinatesAscii(std::string& expected, const std::string& axis, usiz expected += fmt::format("{} {} float\n", axis, count); for(usize idx = 0; idx < count; idx++) { - expected += fmt::format("{:.6f} ", static_cast(idx) - 0.5F); + expected += fmt::format("{:.9g} ", static_cast(idx) - 0.5F); if(idx % 20 == 0 && idx != 0) { expected += "\n"; @@ -298,6 +298,72 @@ TEST_CASE("SimplnxCore::WriteVtkRectilinearGridFilter: ASCII comparison preserve "SCALARS Values float 1\nversion\n3.583333\nSCALARS Ids int 1\nLOOKUP_TABLE default\n1.000000\n")); } +TEST_CASE("SimplnxCore::WriteVtkRectilinearGridFilter: ASCII small coordinates", "[SimplnxCore][WriteVtkRectilinearGridFilter][VtkAsciiCoordinates]") +{ + DataStructure dataStructure; + const DataPath imagePath({"SmallImage"}); + const DataPath cellPath = imagePath.createChildPath("CellData"); + const DataPath valuesPath = cellPath.createChildPath("Values"); + auto* image = ImageGeom::Create(dataStructure, imagePath.getTargetName()); + REQUIRE(image != nullptr); + image->setDimensions({3, 3, 3}); + image->setSpacing({1.0e-7F, 1.0e-7F, 1.0e-7F}); + image->setOrigin({0.0F, 0.0F, 0.0F}); + auto* cellData = AttributeMatrix::Create(dataStructure, cellPath.getTargetName(), ShapeType{3, 3, 3}, image->getId()); + REQUIRE(cellData != nullptr); + image->setCellData(*cellData); + auto store = DataStoreUtilities::CreateDataStore(dataStructure, valuesPath, {3, 3, 3}, {1}); + auto* values = Float32Array::Create(dataStructure, valuesPath.getTargetName(), store, cellData->getId()); + REQUIRE(values != nullptr); + values->fill(0.0F); + + const fs::path outputPath = fs::path(unit_test::k_BinaryTestOutputDir.view()) / "VtkAsciiSmallCoordinates.vtk"; + const WriteVtkRectilinearGridFilter filter; + Arguments args = filter.getDefaultArguments(); + args.insertOrAssign(WriteVtkRectilinearGridFilter::k_OutputFile_Key, std::make_any(outputPath)); + args.insertOrAssign(WriteVtkRectilinearGridFilter::k_WriteBinaryFile_Key, std::make_any(false)); + args.insertOrAssign(WriteVtkRectilinearGridFilter::k_ImageGeometryPath_Key, std::make_any(imagePath)); + args.insertOrAssign(WriteVtkRectilinearGridFilter::k_SelectedDataArrayPaths_Key, std::make_any(MultiArraySelectionParameter::ValueType{valuesPath})); + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + auto executeResult = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + + std::ifstream output(outputPath); + REQUIRE(output.is_open()); + std::string line; + while(std::getline(output, line) && line != "X_COORDINATES 4 float") + { + } + REQUIRE(line == "X_COORDINATES 4 float"); + // The writer uses origin + (i - 1/2) * spacing for its coordinates. + // The half-cell shift is tracked separately in #1878 (WriteVtkRectilinearGridFilter#1). + // For origin zero and spacing 1e-7, hand calculation gives these four values. + // The tolerance allows float arithmetic rounding, but not six-place truncation. + constexpr std::array k_Expected = {-5.0e-8, 5.0e-8, 1.5e-7, 2.5e-7}; + for(const std::string axis : {"X", "Y", "Z"}) + { + if(axis != "X") + { + std::string header; + usize count = 0; + std::string type; + REQUIRE(output >> header >> count >> type); + REQUIRE(header == axis + "_COORDINATES"); + REQUIRE(count == 4); + REQUIRE(type == "float"); + } + for(usize index = 0; index < k_Expected.size(); index++) + { + CAPTURE(axis, index); + double actual = 0.0; + REQUIRE(output >> actual); + CHECK(actual == Approx(k_Expected[index]).margin(2.0e-14)); + } + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + TEMPLATE_TEST_CASE("SimplnxCore::WriteVtkRectilinearGridFilter: ASCII floating point round trip", "[SimplnxCore][WriteVtkRectilinearGridFilter][VtkAsciiFloats]", float32, float64) { UnitTest::LoadPlugins(); @@ -553,3 +619,45 @@ TEST_CASE("SimplnxCore::WriteVtkRectilinearGridFilter: SIMPL Backwards Compatibi } } } + +TEST_CASE("SimplnxCore::WriteVtkRectilinearGridFilter: Legacy scalar component limits", "[SimplnxCore][WriteVtkRectilinearGridFilter]") +{ + const usize components = GENERATE(1, 2, 3, 4, 5, 6, 9); + const bool binary = GENERATE(false, true); + CAPTURE(components, binary); + DataStructure dataStructure; + auto* image = ImageGeom::Create(dataStructure, "Image"); + REQUIRE(image != nullptr); + image->setDimensions({1, 1, 1}); + auto* cells = AttributeMatrix::Create(dataStructure, "Cells", ShapeType{1, 1, 1}, image->getId()); + REQUIRE(cells != nullptr); + image->setCellData(*cells); + const DataPath arrayPath({"Image", "Cells", "Values"}); + auto store = DataStoreUtilities::CreateDataStore(dataStructure, arrayPath, ShapeType{1, 1, 1}, ShapeType{components}); + auto* values = Float32Array::Create(dataStructure, "Values", store, cells->getId()); + REQUIRE(values != nullptr); + + WriteVtkRectilinearGridFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(WriteVtkRectilinearGridFilter::k_OutputFile_Key, + std::make_any(fs::path(unit_test::k_BinaryTestOutputDir.view()) / "component_limits_RectilinearGrid.vtk")); + args.insertOrAssign(WriteVtkRectilinearGridFilter::k_WriteBinaryFile_Key, std::make_any(binary)); + args.insertOrAssign(WriteVtkRectilinearGridFilter::k_ImageGeometryPath_Key, std::make_any(DataPath({"Image"}))); + args.insertOrAssign(WriteVtkRectilinearGridFilter::k_SelectedDataArrayPaths_Key, std::make_any(MultiArraySelectionParameter::ValueType{arrayPath})); + + // Independent format oracle: legacy VTK SCALARS permits 1 through 4 components. + // Counts 5, 6 and 9 must be rejected before a file can be written, for either encoding. + IFilter::PreflightResult result; + REQUIRE_NOTHROW(result = filter.preflight(dataStructure, args)); + if(components <= 4) + { + SIMPLNX_RESULT_REQUIRE_VALID(result.outputActions); + } + else + { + SIMPLNX_RESULT_REQUIRE_INVALID(result.outputActions); + REQUIRE(result.outputActions.errors().size() == 1); + REQUIRE(result.outputActions.errors()[0].code == -12073); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} diff --git a/src/Plugins/SimplnxCore/test/WriteVtkStructuredPointsTest.cpp b/src/Plugins/SimplnxCore/test/WriteVtkStructuredPointsTest.cpp index f78261fe86..4bc36f9672 100644 --- a/src/Plugins/SimplnxCore/test/WriteVtkStructuredPointsTest.cpp +++ b/src/Plugins/SimplnxCore/test/WriteVtkStructuredPointsTest.cpp @@ -209,3 +209,45 @@ TEST_CASE("SimplnxCore::WriteVtkStructuredPointsFilter: ASCII correctness", "[Si UnitTest::AlgorithmTestScope scope(scenario); WriteVtkStructuredPointsTest::CheckAsciiOutput(scope, ""); } + +TEST_CASE("SimplnxCore::WriteVtkStructuredPointsFilter: Legacy scalar component limits", "[SimplnxCore][WriteVtkStructuredPointsFilter]") +{ + const usize components = GENERATE(1, 2, 3, 4, 5, 6, 9); + const bool binary = GENERATE(false, true); + CAPTURE(components, binary); + DataStructure dataStructure; + auto* image = ImageGeom::Create(dataStructure, "Image"); + REQUIRE(image != nullptr); + image->setDimensions({1, 1, 1}); + auto* cells = AttributeMatrix::Create(dataStructure, "Cells", ShapeType{1, 1, 1}, image->getId()); + REQUIRE(cells != nullptr); + image->setCellData(*cells); + const DataPath arrayPath({"Image", "Cells", "Values"}); + auto store = DataStoreUtilities::CreateDataStore(dataStructure, arrayPath, ShapeType{1, 1, 1}, ShapeType{components}); + auto* values = Float32Array::Create(dataStructure, "Values", store, cells->getId()); + REQUIRE(values != nullptr); + + WriteVtkStructuredPointsFilter filter; + auto args = filter.getDefaultArguments(); + args.insertOrAssign(WriteVtkStructuredPointsFilter::k_OutputFile_Key, + std::make_any(fs::path(unit_test::k_BinaryTestOutputDir.view()) / "component_limits_StructuredPoints.vtk")); + args.insertOrAssign(WriteVtkStructuredPointsFilter::k_WriteBinaryFile_Key, std::make_any(binary)); + args.insertOrAssign(WriteVtkStructuredPointsFilter::k_ImageGeometryPath_Key, std::make_any(DataPath({"Image"}))); + args.insertOrAssign(WriteVtkStructuredPointsFilter::k_SelectedDataArrayPaths_Key, std::make_any(MultiArraySelectionParameter::ValueType{arrayPath})); + + // Independent format oracle: legacy VTK SCALARS permits 1 through 4 components. + // Counts 5, 6 and 9 must be rejected before a file can be written, for either encoding. + IFilter::PreflightResult result; + REQUIRE_NOTHROW(result = filter.preflight(dataStructure, args)); + if(components <= 4) + { + SIMPLNX_RESULT_REQUIRE_VALID(result.outputActions); + } + else + { + SIMPLNX_RESULT_REQUIRE_INVALID(result.outputActions); + REQUIRE(result.outputActions.errors().size() == 1); + REQUIRE(result.outputActions.errors()[0].code == -12074); + } + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} diff --git a/src/Plugins/SimplnxCore/vv/RotateSampleRefFrameFilter.md b/src/Plugins/SimplnxCore/vv/RotateSampleRefFrameFilter.md index 4cbc017cdb..54bf58a6da 100644 --- a/src/Plugins/SimplnxCore/vv/RotateSampleRefFrameFilter.md +++ b/src/Plugins/SimplnxCore/vv/RotateSampleRefFrameFilter.md @@ -19,13 +19,13 @@ | Code paths enumerated | 11 (filter preflight guard + geometry setup, and the NN-resample execute). 10 of 12 exercised. | | Tests today | 12 registered cases plus one hidden HDF5 boundary test. Paired CTest selections pass 12/12. | | Exemplar archive | **`Rotate_Sample_Ref_Frame_Test_v2/v3.tar.gz` retired** — golden-file (regression) oracle replaced by an inlined Class 1 analytical oracle. No archive is downloaded. See `vv/provenance/RotateSampleRefFrameFilter.md`. | -| Legacy comparison | **Run** (SIMPLNX vs 6.5.171) on four principal-90 fixtures (90@Z, 180@Z, 90@X, 180@Y): **bit-identical** — same dims and same voxel values. 1 deviation, on the *unsupported* arbitrary-rotation domain only. | -| Bug flags | None. The single deviation (D1) is an intentional guard, not a bug. | +| Legacy comparison | **Run** (SIMPLNX vs 6.5.171) on four isotropic principal-90 fixtures (90@Z, 180@Z, 90@X, 180@Y): **bit-identical** dims and voxel values. Two documented deviations: D1 rejects unsupported rotations; D2 corrects anisotropic three-cycle spacing (analytical evidence, no new legacy binary comparison). | +| Bug flags | KI-008 / L28 corrected (D2). D1 is an intentional guard. | | V&V phase | Historical status and sign-off retained. Section 4.3 adds real-HDF5 boundary verification and paired CTest evidence. | ## Summary -`RotateSampleRefFrame` rotates the reference frame of an **Image Geometry** by nearest-neighbor resampling. V&V established that the operation is a *lossless voxel permutation* — the only correct meaning of a reference-frame rotation — **if and only if** the rotation maps the cubic voxel grid onto itself, i.e. it is a member of the octahedral rotation group (the 90/180/270-degree rotations about X/Y/Z, plus 180 about a face-diagonal and 120/240 about a body-diagonal); for any other rotation it silently produces a lossy, background-padded resample (empirically demonstrated: 45@Z turns 24 voxels into 50 with 24 introduced zeros). A preflight **guard** was added to enforce the supported domain (error `-6850`, plus `-6851` for a slice-reordering slice-by-slice combination), the golden-file exemplar was retired in favor of an inlined **Class 1 analytical-permutation** oracle (+ Class 4 invariants), and a legacy A/B on four principal-90 fixtures showed SIMPLNX is **bit-identical** to DREAM3D 6.5.171 on the standard EBSD sample transforms. +`RotateSampleRefFrame` rotates the reference frame of an **Image Geometry** by nearest-neighbor resampling. V&V established that the operation is a *lossless voxel permutation* — the only correct meaning of a reference-frame rotation — **if and only if** the rotation maps the cubic voxel grid onto itself, i.e. it is a member of the octahedral rotation group (the 90/180/270-degree rotations about X/Y/Z, plus 180 about a face-diagonal and 120/240 about a body-diagonal); for any other rotation it silently produces a lossy, background-padded resample (empirically demonstrated: 45@Z turns 24 voxels into 50 with 24 introduced zeros). A preflight **guard** was added to enforce the supported domain (error `-6850`, plus `-6851` for a slice-reordering slice-by-slice combination), the golden-file exemplar was retired in favor of an inlined **Class 1 analytical-permutation** oracle (+ Class 4 invariants), and a legacy A/B showed identical dimensions and voxel values in simplnx and DREAM3D 6.5.171 on the four tested isotropic principal-axis fixtures. ## Algorithm Relationship @@ -100,6 +100,17 @@ OOC recertification (2026-09-21): serial CTest passed 12/12 target entries in ea ## Deviations from DREAM3D 6.5.171 -Comparison run on four principal-90 fixtures (90@Z, 180@Z, 90@X, 180@Y) — see `vv/comparisons/RotateSampleRefFrameFilter/results.md`. On the supported domain, **no deviations** (bit-identical dims and voxel values). One deviation on the unsupported domain: +Comparison run on four principal-90 fixtures (90@Z, 180@Z, 90@X, 180@Y) — see `vv/comparisons/RotateSampleRefFrameFilter/results.md`. These four isotropic fixtures have bit-identical dims and voxel values. They do not cover anisotropic three-cycle rotations. The documented deviations are: - `RotateSampleRefFrameFilter-D1` — legacy silently runs arbitrary (non-principal-90) rotations as a lossy resample; SIMPLNX rejects them in preflight (`-6850`). Root cause: algorithmic choice (intentional; not a bug). See `vv/deviations/RotateSampleRefFrameFilter.md`. +- `RotateSampleRefFrameFilter-D2` — corrected spacing and dimensions for anisotropic three-cycle rotations (KI-008 / L28). Independent analytical tests cover both algorithms on in-memory stores. No new legacy binary comparison was run. See `vv/deviations/RotateSampleRefFrameFilter.md`. + +## Bugs found and fixed: errata KI-008 (L28) + +`SimplnxCore::RotateSampleRefFrame: anisotropic three-cycle preserves the voxel grid` uses an inline analytical oracle. The map R(x,y,z)=(z,x,y) sends dimensions (2,3,4) to (4,2,3) and spacing (2,3,5) to (5,2,3). Output (i,j,k) has value 1+j+2*k+6*i. The test checks every cell in Direct and Scanline paths on in-memory stores. No exemplar is used. + +Test sensitivity verification: before the fix, both paths returned dimensions (7,1,5), spacing (3,5,2), and 35 cells instead of 24. For transforms classified as grid rotations, the shared helper uses the classified source axis and its column scale for each output axis. Other transformations keep the earlier spacing rule. This intentionally corrects DREAM3D 6.5.171 behavior; see `RotateSampleRefFrameFilter-D2`. No new legacy binary comparison or disk-backed OOC validation is claimed. + +To repeat the L28 check, run the `SimplnxCore::RotateSampleRefFrame` CTest entries serially. The historical A/B results above are unchanged and apply only to their four fixtures. + +The R5 helper regression and Apply Transformation to Geometry regression check uneven scaling with a small shear. For (x,y,z) to (x+5y,100000y,z), a 2×2×2 image with spacing (2,3,5) has transformed extents (34,600000,10). The analytical spacing is (2,3*sqrt(100000^2+5^2),5), or (2,300000,5) in float32. The expected dimensions are (17,2,2). Both regressions failed before R5 and pass after it; the filter test covers Direct and Scanline on in-memory stores. This is shared-helper evidence, not a new DREAM3D 6.5.171 comparison. diff --git a/src/Plugins/SimplnxCore/vv/deviations/RotateSampleRefFrameFilter.md b/src/Plugins/SimplnxCore/vv/deviations/RotateSampleRefFrameFilter.md index 2e79c0da0f..1c814360ef 100644 --- a/src/Plugins/SimplnxCore/vv/deviations/RotateSampleRefFrameFilter.md +++ b/src/Plugins/SimplnxCore/vv/deviations/RotateSampleRefFrameFilter.md @@ -4,7 +4,7 @@ This file lists every documented behavioral difference between this SIMPLNX filt Entries are referenced by stable ID (`RotateSampleRefFrameFilter-D`) from the V&V report and from public migration guidance. The ID is stable across renames; the Filter UUID field is the permanent cross-reference anchor. -Legacy equivalent: `RotateSampleRefFrame` (SIMPL UUID `{e25d9b4c-2b37-578c-b1de-cf7032b5ef19}`) in the Sampling plugin. On the **supported domain** (a 90/180/270-degree rotation about the X, Y, or Z axis) SIMPLNX and 6.5.171 are **bit-identical** — same output dimensions and same voxel values on all four A/B fixtures (see `../comparisons/RotateSampleRefFrameFilter/results.md`). The single deviation below is about rotations *outside* that domain. +Legacy equivalent: `RotateSampleRefFrame` (SIMPL UUID `{e25d9b4c-2b37-578c-b1de-cf7032b5ef19}`) in the Sampling plugin. The four isotropic principal-axis fixtures produced identical dimensions and voxel values in simplnx and 6.5.171. D2 verifies the anisotropic three-cycle against an analytical result. See `../comparisons/RotateSampleRefFrameFilter/results.md` for the four A/B fixtures. --- @@ -20,8 +20,17 @@ Legacy equivalent: `RotateSampleRefFrame` (SIMPL UUID `{e25d9b4c-2b37-578c-b1de- **Root cause:** Algorithmic choice. A reference-frame rotation is only a lossless re-labeling of the voxels when the rotation maps the grid onto itself (a signed axis-permutation / octahedral-group rotation). For any other rotation the nearest-neighbor resample is lossy: it enlarges the bounding box, drops and duplicates voxels, and pads the result with background (0) values — 45° about Z of a 4×3×2 test volume, for example, turns 24 voxels into 50 with 24 introduced zeros. Legacy 6.5.171 performed this lossy resample silently (its documentation only *hedged* that the filter was "verified" for 90/180 about a principal axis). SIMPLNX makes the supported domain an enforced contract via a preflight guard (`RotateSampleRefFrameFilter.cpp`, `IsLosslessGridRotation`), so the arbitrary-rotation path can no longer be reached — arguably a latent-bug fix for a filter whose purpose is a reference-frame rotation. -**Affected users:** Anyone who previously fed a lossy rotation to the legacy filter (an undocumented/unsupported use). Pipelines that used the standard EBSD sample transforms (n×90 about a principal axis, e.g. 180° about Y) are unaffected — those remain bit-identical. +**Affected users:** Anyone who previously fed a lossy rotation to the legacy filter (an undocumented/unsupported use). The four tested isotropic principal-axis fixtures are unaffected. **Blast radius — internal callers.** `ReadH5Ebsd` (`ReadH5Ebsd.cpp`) forwards the sample-reference-frame transform stored in the `.h5ebsd` file directly into this filter with slice-by-slice enabled, and propagates preflight errors. If a file's *recommended* sample transform is not a lossless, Z-preserving rotation (for example a stored 90° about X, which trips `-6851`, or any non-group angle, which trips `-6850`), the entire H5EBSD import now fails preflight with an error naming a filter the user never placed. The prior behavior was a silently corrupted (slice-pinned) import, so failing loudly is arguably correct, but the message can be confusing in that context. The practical workaround is to uncheck *Use Recommended Transformations* in `ReadH5Ebsd` (or apply the transform separately). This is called out here so the failure mode is documented rather than surprising. **Recommendation:** Trust SIMPLNX. For an arbitrary rotation with interpolation, use the `Apply Transformation To Geometry` filter, which is designed for lossy resampling rotations (nearest-neighbor or trilinear). The legacy result for a lossy rotation was a silent lossy resample, not a faithful reference-frame rotation. For H5EBSD imports whose recommended transform is lossy, uncheck *Use Recommended Transformations* in `ReadH5Ebsd`. + +## RotateSampleRefFrameFilter-D2 + +- **Filter UUID:** `d2451dc1-a5a1-4ac2-a64d-7991669dcffc` +- **Status:** active +- **Symptom:** DREAM3D 6.5.171 errata KI-008 assigns spacing from the wrong source axis for three-cycle rotations. Anisotropic output spacing and dimensions are wrong. +- **Correction:** Use the source axis and column scale found when classifying the grid rotation. For R(x,y,z)=(z,x,y), spacing (2,3,5) becomes (5,2,3). +- **Evidence:** The inline analytical regression checks dimensions and every voxel in both Direct and Scanline paths. Both paths failed before the fix. No new legacy binary comparison was run. +- **Recommendation:** Use the corrected output. Affected anisotropic rotations intentionally differ from DREAM3D 6.5.171. diff --git a/src/Plugins/SimplnxCore/vv/provenance/RotateSampleRefFrameFilter.md b/src/Plugins/SimplnxCore/vv/provenance/RotateSampleRefFrameFilter.md index 841a78a682..096eea9398 100644 --- a/src/Plugins/SimplnxCore/vv/provenance/RotateSampleRefFrameFilter.md +++ b/src/Plugins/SimplnxCore/vv/provenance/RotateSampleRefFrameFilter.md @@ -42,3 +42,11 @@ These A/B artifacts are working files, not a test-consumed archive; they are kep ## Regenerated to fix a circular-oracle situation? Yes — this replaces the retired `Rotate_Sample_Ref_Frame_Test_v2/v3.tar.gz` golden-file archives with an independently-derived Class 1 analytical oracle. The new oracle is *not* derived from SIMPLNX (or legacy) output; it is hand-derived from the definition of a principal-90 reference-frame rotation. + +## Errata KI-008 regression (L28) + +`SimplnxCore::RotateSampleRefFrame: anisotropic three-cycle preserves the voxel grid` uses an inline analytical oracle. The map R(x,y,z)=(z,x,y) sends dimensions (2,3,4) to (4,2,3) and spacing (2,3,5) to (5,2,3). Output (i,j,k) has value 1+j+2*k+6*i. The test checks every cell in Direct and Scanline paths on in-memory stores. No exemplar is used. + +Test sensitivity verification: before the fix, both paths returned dimensions (7,1,5), spacing (3,5,2), and 35 cells instead of 24. For transforms classified as grid rotations, the shared helper uses the classified source axis and its column scale for each output axis. Other transformations keep the earlier spacing rule. This intentionally corrects DREAM3D 6.5.171 behavior; see `RotateSampleRefFrameFilter-D2`. No new legacy binary comparison or disk-backed OOC validation is claimed. + +The R5 helper regression and Apply Transformation to Geometry regression check uneven scaling with a small shear. For (x,y,z) to (x+5y,100000y,z), a 2×2×2 image with spacing (2,3,5) has transformed extents (34,600000,10). The analytical spacing is (2,3*sqrt(100000^2+5^2),5), or (2,300000,5) in float32. The expected dimensions are (17,2,2). Both regressions failed before R5 and pass after it; the filter test covers Direct and Scanline on in-memory stores. This is shared-helper evidence, not a new DREAM3D 6.5.171 comparison. diff --git a/src/simplnx/Utilities/AlignSections.cpp b/src/simplnx/Utilities/AlignSections.cpp index a7f50987cb..112b86c617 100644 --- a/src/simplnx/Utilities/AlignSections.cpp +++ b/src/simplnx/Utilities/AlignSections.cpp @@ -55,7 +55,7 @@ class AlignSectionsTransferDataImpl { T var = static_cast(0); - for(size_t i = 1; i < m_Dims[2]; i++) + for(size_t i = (!m_Xshifts.empty() && (m_Xshifts[0] != 0 || m_Yshifts[0] != 0)) ? 0 : 1; i < m_Dims[2]; i++) { m_Filter->sendThreadSafeProgressMessage(1); if(m_Filter->getCancel()) @@ -159,7 +159,7 @@ class AlignSectionsTransferDataOocImpl auto sliceBuffer = std::make_unique(sliceElements); auto outBuffer = std::make_unique(sliceElements); - for(usize i = 1; i < m_Dims[2]; i++) + for(usize i = (!m_Xshifts.empty() && (m_Xshifts[0] != 0 || m_Yshifts[0] != 0)) ? 0 : 1; i < m_Dims[2]; i++) { m_Filter->sendThreadSafeProgressMessage(1); if(m_Filter->getCancel()) @@ -311,7 +311,7 @@ Result<> AlignSections::execute(const SizeVec3& udims, const DataPath& imageGeom } // Every transfer reports through one shared throttle, so the denominator is aggregate: each array - // contributes the slices its loop visits, which is every slice after the first. Arrays that are not + // contributes the slices its loop visits, including the first when its shift is nonzero. Arrays that are not // IDataArray objects are skipped below, so count only the ones that will actually run. usize transferArrayCount = 0; for(const auto& cellArrayPath : selectedCellArrays) @@ -321,7 +321,8 @@ Result<> AlignSections::execute(const SizeVec3& udims, const DataPath& imageGeom transferArrayCount++; } } - const usize slicesPerArray = (udims[2] > 1) ? (udims[2] - 1) : 0; + const usize firstSlice = (!xShifts.empty() && (xShifts[0] != 0 || yShifts[0] != 0)) ? 0 : 1; + const usize slicesPerArray = udims[2] > firstSlice ? udims[2] - firstSlice : 0; m_Throttle.reset(transferArrayCount * slicesPerArray, "Updating cell data"); ParallelTaskAlgorithm taskRunner; diff --git a/src/simplnx/Utilities/ImageProcessing/ImageProcessingFilterUtilities.hpp b/src/simplnx/Utilities/ImageProcessing/ImageProcessingFilterUtilities.hpp index 52e35db44c..ba7bf4ea64 100644 --- a/src/simplnx/Utilities/ImageProcessing/ImageProcessingFilterUtilities.hpp +++ b/src/simplnx/Utilities/ImageProcessing/ImageProcessingFilterUtilities.hpp @@ -665,7 +665,8 @@ Result PreflightFullOverwriteImageFilter(const DataStructure& dat * is impossible because the collapsed output array cannot be inserted into the still-full cell * AttributeMatrix (tuple-count mismatch), and resizing the AttributeMatrix before executeImpl * would destroy the input the engine still has to read. Because that swap replaces the whole - * original geometry, any OTHER arrays in its cell data are discarded; a warning enumerates them. + * original geometry, any OTHER arrays in its cell data and other geometry children are + * discarded; a warning enumerates them. * * The output element type is the input type mapped through @p OutTypeMap (SameAsInput for type-preserving * reducers like Max; a fixed float map for Mean/StdDev/Sum). The execute façade MUST be instantiated with @@ -757,15 +758,15 @@ Result PreflightAxisProjection(const DataStructure& dataStructure result.value().appendDeferredAction(std::make_unique(renamedOriginalPath)); result.value().appendDeferredAction(std::make_unique(targetGeomPath, originalName)); - // In-place discards the whole original geometry, so any OTHER cell-data arrays are lost. They cannot - // survive the collapse (they have the original, non-collapsed tuple count), but say so rather than - // dropping them silently. The selected input (consumed by the projection) is not listed. + // In-place discards the original geometry. List its other children and cell-data arrays. The selected + // input is consumed by the projection, so it is not listed. The output array is created in the new + // geometry, so an old cell-data array with the output name is discarded and listed. const std::string inputArrayName = inputArrayPath.getTargetName(); std::string discarded; for(const auto& [childId, childObject] : *cellData) { const std::string childName = childObject->getName(); - if(childName == inputArrayName || childName == outputArrayName) + if(childName == inputArrayName) { continue; } @@ -773,13 +774,25 @@ Result PreflightAxisProjection(const DataStructure& dataStructure { discarded += ", "; } - discarded += childName; + discarded += imageGeom.getCellDataPath().createChildPath(childName).toString(); + } + for(const auto& [childId, childObject] : imageGeom) + { + if(childId == cellData->getId() || childId == inputArray.getId()) + { + continue; + } + if(!discarded.empty()) + { + discarded += ", "; + } + discarded += imageGeomPath.createChildPath(childObject->getName()).toString(); } if(!discarded.empty()) { - result.warnings().push_back(Warning{-8343, fmt::format("Perform In-Place replaces Image Geometry '{}' with the collapsed projection, so the following other array(s) in cell data '{}' will be " - "discarded (they cannot be projected and do not fit the collapsed shape): {}. Disable 'Perform In-Place' to keep the original geometry.", - imageGeomPath.toString(), cellDataName, discarded)}); + result.warnings().push_back(Warning{-8343, fmt::format("Perform In-Place replaces Image Geometry '{}' with the collapsed projection. The following other data object(s), including their " + "contents, will be discarded: {}. Disable 'Perform In-Place' to keep the original geometry.", + imageGeomPath.toString(), discarded)}); } } diff --git a/src/simplnx/Utilities/ImageRotationUtilities.cpp b/src/simplnx/Utilities/ImageRotationUtilities.cpp index 5fa1fee579..d4d8099263 100644 --- a/src/simplnx/Utilities/ImageRotationUtilities.cpp +++ b/src/simplnx/Utilities/ImageRotationUtilities.cpp @@ -74,13 +74,59 @@ RotateArgs CreateRotationArgs(const ImageGeom& imageGeom, const Matrix4fR& trans FloatVec6 minMaxCoords = DetermineMinMaxCoords(imageGeom, transformationMatrix); - const Eigen::Vector3f xAxisNew = rotationMatrix * k_XAxis; - const Eigen::Vector3f yAxisNew = rotationMatrix * k_YAxis; - const Eigen::Vector3f zAxisNew = rotationMatrix * k_ZAxis; + // Normalize each source column to distinguish grid rotations from general transforms. + // A signed permutation has exactly one unit entry in each column and each row. + // The filter checks raw entries with tolerance t. Accepted column norms are at least 1-t, + // so normalized off-axis entries can be as large as t/(1-t). + constexpr float32 k_RawGridTolerance = 1.0e-4F; + constexpr float32 k_GridTolerance = k_RawGridTolerance / (1.0F - k_RawGridTolerance); + bool isGridRotation = true; + std::array usedRows = {false, false, false}; + FloatVec3 gridSpacing; + for(int32 col = 0; col < 3; ++col) + { + const float32 scale = rotationMatrix.col(col).norm(); + if(!std::isfinite(scale) || scale <= 0.0F) + { + isGridRotation = false; + break; + } + const Eigen::Vector3f normalized = rotationMatrix.col(col) / scale; + Eigen::Index sourceRow = 0; + normalized.cwiseAbs().maxCoeff(&sourceRow); + if(usedRows[sourceRow] || std::abs(std::abs(normalized[sourceRow]) - 1.0F) > k_GridTolerance) + { + isGridRotation = false; + break; + } + usedRows[sourceRow] = true; + // Preserve the classified source axis and its scale for this output row. + gridSpacing[sourceRow] = spacing[col] * scale; + for(int32 row = 0; row < 3; ++row) + { + if(row != sourceRow && std::abs(normalized[row]) > k_GridTolerance) + { + isGridRotation = false; + } + } + } + + FloatVec3 outputSpacing; + if(isGridRotation) + { + outputSpacing = gridSpacing; + } + else + { + // Keep the column-based spacing rule for off-axis rotations and general transforms. + const Eigen::Vector3f xAxisNew = rotationMatrix * k_XAxis; + const Eigen::Vector3f yAxisNew = rotationMatrix * k_YAxis; + const Eigen::Vector3f zAxisNew = rotationMatrix * k_ZAxis; - FloatVec3 transformScale = {(rotationMatrix * k_XAxis).norm(), (rotationMatrix * k_YAxis).norm(), (rotationMatrix * k_ZAxis).norm()}; + FloatVec3 transformScale = {(rotationMatrix * k_XAxis).norm(), (rotationMatrix * k_YAxis).norm(), (rotationMatrix * k_ZAxis).norm()}; - FloatVec3 outputSpacing = {DetermineSpacing(spacing, xAxisNew) * transformScale[0], DetermineSpacing(spacing, yAxisNew) * transformScale[1], DetermineSpacing(spacing, zAxisNew) * transformScale[2]}; + outputSpacing = {DetermineSpacing(spacing, xAxisNew) * transformScale[0], DetermineSpacing(spacing, yAxisNew) * transformScale[1], DetermineSpacing(spacing, zAxisNew) * transformScale[2]}; + } USizeVec3 outputDims = {static_cast(std::nearbyint((minMaxCoords[1] - minMaxCoords[0]) / outputSpacing[0])), static_cast(std::nearbyint((minMaxCoords[3] - minMaxCoords[2]) / outputSpacing[1])), diff --git a/test/ImageRotationUtilitiesTest.cpp b/test/ImageRotationUtilitiesTest.cpp index b44a99559a..62457f5274 100644 --- a/test/ImageRotationUtilitiesTest.cpp +++ b/test/ImageRotationUtilitiesTest.cpp @@ -1,4 +1,7 @@ #include "simplnx/Utilities/ImageRotationUtilities.hpp" +#include "simplnx/DataStructure/DataStructure.hpp" +#include "simplnx/DataStructure/Geometry/ImageGeom.hpp" +#include "simplnx/UnitTest/UnitTestCommon.hpp" #include @@ -58,3 +61,149 @@ TEST_CASE("ImageRotationUtilities: Octant offsets match the trilinear corner ord } } } + +TEST_CASE("ImageRotationUtilities: Scaled three-cycle takes spacing from the scaled source axis", "[simplnx][ImageRotationUtilities][CreateRotationArgs]") +{ + DataStructure dataStructure; + ImageGeom* image = ImageGeom::Create(dataStructure, "Input"); + REQUIRE(image != nullptr); + image->setDimensions({2, 3, 4}); + image->setSpacing({2.0F, 3.0F, 5.0F}); + image->setOrigin({1.0F, 2.0F, 3.0F}); + + // The matrix maps (x,y,z) to (-4z, 2x, 3y). It is a three-cycle with a reflection and a different scale on each source axis. + ImageRotationUtilities::Matrix4fR transform = ImageRotationUtilities::Matrix4fR::Identity(); + transform.block<3, 3>(0, 0) << 0.0F, 0.0F, -4.0F, 2.0F, 0.0F, 0.0F, 0.0F, 3.0F, 0.0F; + + // Hand calculation: + // Output X comes from source Z: spacing 4 * 5 = 20, extent 4 * (4 * 5) = 80, so 4 cells. + // Output Y comes from source X: spacing 2 * 2 = 4, extent 2 * (2 * 2) = 8, so 2 cells. + // Output Z comes from source Y: spacing 3 * 3 = 9, extent 3 * (3 * 3) = 27, so 3 cells. + // Minimum corner: X = -4 * (3 + 20) = -92, Y = 2 * 1 = 2, Z = 3 * 2 = 6. + const ImageRotationUtilities::RotateArgs args = ImageRotationUtilities::CreateRotationArgs(*image, transform); + const std::array expectedDims = {4, 2, 3}; + const std::array expectedSpacing = {20.0F, 4.0F, 9.0F}; + const std::array expectedOrigin = {-92.0F, 2.0F, 6.0F}; + for(usize axis = 0; axis < 3; ++axis) + { + INFO("axis " + std::to_string(axis)); + CHECK(args.outputDims[axis] == expectedDims[axis]); + CHECK(args.outputSpacing[axis] == Approx(expectedSpacing[axis])); + CHECK(args.TransformedSpacing[axis] == Approx(expectedSpacing[axis])); + CHECK(args.TransformedOrigin[axis] == Approx(expectedOrigin[axis])); + } +} + +TEST_CASE("ImageRotationUtilities: Uneven scaling keeps the classified source axis", "[simplnx][ImageRotationUtilities][CreateRotationArgs]") +{ + DataStructure dataStructure; + ImageGeom* image = ImageGeom::Create(dataStructure, "Input"); + REQUIRE(image != nullptr); + image->setDimensions({2, 2, 2}); + image->setSpacing({2.0F, 3.0F, 5.0F}); + image->setOrigin({0.0F, 0.0F, 0.0F}); + + ImageRotationUtilities::Matrix4fR transform = ImageRotationUtilities::Matrix4fR::Identity(); + transform(0, 1) = 5.0F; + transform(1, 1) = 100000.0F; + + // Hand calculation: (x,y,z) -> (x+5y,100000y,z). The normalized Y column's + // off-axis entry is about 5e-5, so the classified source axes remain X, Y, Z. + // Spacing is (2, 3*sqrt(100000^2+5^2), 5), which rounds to (2,300000,5) in float32. + // Input extents (4,6,10) become (4+5*6,100000*6,10) = (34,600000,10). + // Dividing by spacing and rounding gives dimensions (17,2,2). + const auto args = ImageRotationUtilities::CreateRotationArgs(*image, transform); + const std::array expectedSpacing = {2.0F, 300000.0F, 5.0F}; + const std::array expectedDims = {17, 2, 2}; + for(usize axis = 0; axis < 3; ++axis) + { + CAPTURE(axis); + CHECK(args.outputSpacing[axis] == Approx(expectedSpacing[axis])); + CHECK(args.TransformedSpacing[axis] == Approx(expectedSpacing[axis])); + CHECK(args.outputDims[axis] == expectedDims[axis]); + CHECK(args.TransformedDims[axis] == expectedDims[axis]); + } +} + +TEST_CASE("ImageRotationUtilities: Off-axis scaling keeps the column-based spacing", "[simplnx][ImageRotationUtilities][CreateRotationArgs]") +{ + DataStructure dataStructure; + ImageGeom* image = ImageGeom::Create(dataStructure, "Input"); + REQUIRE(image != nullptr); + image->setDimensions({2, 3, 4}); + image->setSpacing({2.0F, 3.0F, 5.0F}); + image->setOrigin({0.0F, 0.0F, 0.0F}); + + const float32 c = std::sqrt(0.5F); + ImageRotationUtilities::Matrix4fR transform = ImageRotationUtilities::Matrix4fR::Identity(); + transform.block<3, 3>(0, 0) << 2 * c, -c, 0, 2 * c, c, 0, 0, 0, 1; + + // Each column selects spacing from its largest absolute direction cosine. A 45-degree tie + // selects X. The result is multiplied by the column norm. + // Column norms (2,1,1) therefore give spacing (2*2,2*1,5*1) = (4,2,5). + // Source extents are (4,9,20). Rotated XY extents are (2*4+9)/sqrt(2). + // nearbyint gives dimensions (round(17/(4*sqrt(2))),round(17/(2*sqrt(2))),4) = (3,6,4). + const auto args = ImageRotationUtilities::CreateRotationArgs(*image, transform); + const std::array expectedSpacing = {4.0F, 2.0F, 5.0F}; + const std::array expectedDims = {3, 6, 4}; + for(usize axis = 0; axis < 3; ++axis) + { + CHECK(args.outputSpacing[axis] == Approx(expectedSpacing[axis])); + CHECK(args.outputDims[axis] == expectedDims[axis]); + } +} + +TEST_CASE("ImageRotationUtilities: Near-grid three-cycle accepted by Rotate Sample Reference Frame uses source-axis spacing", "[simplnx][ImageRotationUtilities][CreateRotationArgs]") +{ + DataStructure dataStructure; + ImageGeom* image = ImageGeom::Create(dataStructure, "Input"); + REQUIRE(image != nullptr); + image->setDimensions({2, 3, 4}); + image->setSpacing({2.0F, 3.0F, 5.0F}); + image->setOrigin({0.0F, 0.0F, 0.0F}); + + // 119.995 degrees about (111) differs from the exact three-cycle by about 5e-5 per entry. + // Rotate Sample Reference Frame accepts it as a grid rotation (tolerance 1e-4). + // The exact map is R(x,y,z) = (z,x,y). Output axes X, Y and Z take source axes Z, X and Y. + // Spacing (2,3,5) becomes (5,2,3) and dimensions (2,3,4) become (4,2,3). + const ImageRotationUtilities::Matrix4fR transform = ImageRotationUtilities::GenerateRotationTransformationMatrix({1.0F, 1.0F, 1.0F, 119.995F}); + const auto args = ImageRotationUtilities::CreateRotationArgs(*image, transform); + const std::array expectedSpacing = {5.0F, 2.0F, 3.0F}; + const std::array expectedDims = {4, 2, 3}; + for(usize axis = 0; axis < 3; ++axis) + { + CHECK(args.outputSpacing[axis] == Approx(expectedSpacing[axis])); + CHECK(args.outputDims[axis] == expectedDims[axis]); + } +} + +TEST_CASE("ImageRotationUtilities: Raw grid tolerance accounts for column normalization", "[simplnx][ImageRotationUtilities][CreateRotationArgs]") +{ + DataStructure dataStructure; + ImageGeom* imageGeomPtr = ImageGeom::Create(dataStructure, "Input"); + REQUIRE(imageGeomPtr != nullptr); + imageGeomPtr->setDimensions({2, 3, 4}); + imageGeomPtr->setSpacing({2.0F, 3.0F, 5.0F}); + imageGeomPtr->setOrigin({0.0F, 0.0F, 0.0F}); + + ImageRotationUtilities::Matrix4fR transform = ImageRotationUtilities::Matrix4fR::Identity(); + transform.block<3, 3>(0, 0) << 0.000099998F, 0.0F, 1.0F, 0.99995F, 0.0F, 0.0F, 0.0F, 1.00005F, 0.0F; + + // Raw entries and determinant pass the filter's grid guard. Normalizing column 0 raises its small entry above 1e-4. + REQUIRE(std::abs(transform(0, 0)) < 1.0e-4F); + REQUIRE(std::abs(transform(1, 0) - 1.0F) < 1.0e-4F); + REQUIRE(std::abs(transform(2, 1) - 1.0F) < 1.0e-4F); + REQUIRE(std::abs(transform.block<3, 3>(0, 0).determinant() - 1.0F) < 1.0e-4F); + REQUIRE(transform(0, 0) / transform.block<3, 1>(0, 0).norm() > 1.0e-4F); + + // Source axes Z, X, Y supply output spacing. Their scales are 1, sqrt(0.99995^2+0.000099998^2), 1.00005. + // Output extents are 20+4*0.000099998, 4*0.99995, 9*1.00005. Rounding extent/spacing gives (4,2,3). + const std::array expectedSpacing = {5.0, 2.0 * std::sqrt(0.99995 * 0.99995 + 0.000099998 * 0.000099998), 3.0 * 1.00005}; + const std::array expectedDims = {4, 2, 3}; + const auto args = ImageRotationUtilities::CreateRotationArgs(*imageGeomPtr, transform); + for(usize axis = 0; axis < 3; ++axis) + { + CHECK(args.outputSpacing[axis] == Approx(expectedSpacing[axis])); + CHECK(args.outputDims[axis] == expectedDims[axis]); + } +}