Repository navigation
BUG: Fix store proxy swaps, binary write results and root comparison - #1986
Open
joeykleingers wants to merge 1 commit into
Open
joeykleingers wants to merge 1 commit into
joeykleingers wants to merge 1 commit into
Conversation
* DataStore::writeBinaryFile reports a stream that is already in a failed state, and a file that could not be written completely when it is closed (new error -10176). Both were reported as success, which left a truncated file reported as written. * The AbstractDataStore value proxy swap exchanges values between two different stores. It swapped an index with itself in the first store, so std::iter_swap across stores changed nothing. * The AbstractListStore list proxy keeps its own store and tuple in swap and move assignment. Swapping two tuples changed nothing, a swap across stores wrote to the wrong tuple, and assigning through iterators (std::copy) wrote nothing. With Mirror Geometry on, Append Image Geometry therefore left neighbor lists unmirrored; it now mirrors them like the other arrays. The Small_IN100 exemplar holds the unmirrored lists, so the test checks the mirrored lists against the lists it creates. * UnitTest::CompareDataStructures compares the children of the root when it is called without a path. It compared only the total number of objects, so any two structures of the same size compared equal. * Remove WriteVtkDataFunctor from VtkUtilities.hpp. Nothing calls it, and it ignored the result of writeBinaryFile. * Add regression tests for each fix. Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
Author
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes four defects in simplnx data stores and in the test helper
CompareDataStructures. Each one reported success or changed nothing where it should have failed or written data. The list-store fix also changes the output of Append Image Geometry, which now mirrors neighbor lists. It also removes an unused VTK writer that ignored the binary-write result. The paired SimplnxOoc pull request fixes the same binary-write defect in its out-of-core store. 8 files changed, +517 / −198 lines (most of the additions are tests).1. Binary writes report failures
DataStore::writeBinaryFile(std::ostream&)checksfail()instead ofbad(). A stream that is already in a failed state writes nothing; it was reported as success and is now error -10175.DataStore::writeBinaryFile(const std::string&)closes the file and checks the result. The values are buffered until the file is closed, so a failure at that point (for example a full disk) left a truncated file that was reported as written. It is now error -10176, with the file path and what to check. A partly written file stays in place, as before.2. Value proxy swap across two stores
std::iter_swapon iterators of two different numeric stores swapped an index with itself in the first store, so neither store changed. TheAbstractDataStorevalue proxy now exchanges the two values when the stores differ. A swap inside one store still calls the store's ownswap.3. List proxy keeps its own tuple
The proxy that
AbstractListStoreiterators return writes its list back to its tuple when it goes away. Itsswapexchanged the tuple indices together with the lists, and its move assignment took over the other proxy's tuple. As a result:std::copydoes, wrote nothing.The proxy now keeps its store and tuple, and only the list values are exchanged or assigned.
Behavior change. With Mirror Geometry on, Append Image Geometry mirrors each appended array with
std::swap_ranges. Because the list swap did nothing, neighbor lists were left in their unmirrored order while the other arrays were mirrored. They are now mirrored too. TheSmall_IN100_dream3d_v3exemplar holds the unmirrored lists in its sixMirrored*containers. The mirrored test cases therefore check the appended neighbor lists against the lists that the test creates, and no longer compare the exemplar's neighbor lists. The test-data archive does not change.4.
CompareDataStructurescompares the whole treeCalled without a path,
UnitTest::CompareDataStructurescompared only the total number of objects, because the loop over the children ran only for a group path. Any two structures of the same size compared equal. The loop now runs for the root too, after a check that both sides have the same number of children. Open PR #1983 moves this function, unchanged, toUnitTestCommon.cpp; whichever of the two merges second takes over the change.5. Unused VTK writer removed
WriteVtkDataFunctorinSimplnxCore/utils/VtkUtilities.hppignored the result ofwriteBinaryFile. Nothing calls it: PR #1568 replaced its last use in Write Vtk Structured Points with writers that report stream errors, and Write Vtk Rectilinear Grid usesWriteVtkDataArrayFunctor. It is removed, with the include that only it used.Test plan
Each new test fails on
developand passes with the fix:std::copyCompareDataStructureswith a changed value and with different root children (both orders)The write test of a successful file and the swap inside one store pass before and after.
macOS in-core Release build (DREAM3D-NX with this simplnx): all 4,381 tests; 1 is skipped (it needs native OpenGL).
macOS out-of-core Release build with SimplnxOoc: all 1,056 tests with the
ooclabel (without benchmarks).After the last test changes, both builds again: the Append Image Geometry tests and every new test pass (16 in-core, 21 out-of-core, including the Python Append Image Geometry test).
After removing the VTK writer, both builds again: the Write Vtk Structured Points and Write Vtk Rectilinear Grid tests pass.
CI passes on Linux, macOS and Windows.
Related Issue