Skip to content

BUG: Replace files on Windows while another program has them open - #1990

Open
imikejackson wants to merge 4 commits into
BlueQuartzSoftware:developfrom
imikejackson:fix/windows-file-replace
Open

imikejackson wants to merge 4 commits into
BlueQuartzSoftware:developfrom
imikejackson:fix/windows-file-replace

Conversation

@imikejackson

@imikejackson imikejackson commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

On Windows, a replacing rename fails with "Access is denied" while a handle to the destination file is open in a process, also when that handle shares read, write and delete access. The Windows Search indexer opens new files in that way for some microseconds. InstallReplacementFile() is a new helper that replaces a file with a rename that such a handle does not block. AtomicFile::commit() and the .xdmf file of WriteDREAM3D now use it. On Linux and macOS the helper renames one time, as before. 13 files changed, +948 / −39 lines.

Companion pull request: BlueQuartzSoftware/SimplnxOoc#48 uses this helper in the out-of-core storage plugin. That pull request cannot merge before this one.

1. Cause

The defect was found in the out-of-core storage plugin (BlueQuartzSoftware/SimplnxOoc#39), where one filter replaces a file several thousand times. The measurements were made on Windows 11, on NTFS.

  • A kernel file trace showed SearchProtocolHost.exe (Windows Search) opening the destination file 6 microseconds before a failed replace.
  • With the Windows Search service stopped, a loop of 1.33 million replaces had 0 failures. With the service started again, 1.25 million replaces had 27 failures.
  • Of two CI machines, the one where the Windows Search service was disabled never failed (0 of 8 runs). The one where the service runs failed in 11 of 17 runs. These two machines also have different Windows builds, so this comparison alone does not exclude the build as a second condition.
  • With Windows Search running, a loop did one MoveFileExW replace and one replace with POSIX semantics in the same folders in each iteration, for 30 minutes and 1.26 million iterations. The MoveFileExW replace failed 12 times. The replace with POSIX semantics failed 0 times.
  • With a GENERIC_READ handle held open on the destination, the two types of replace gave these results:
Share mode of the held handle MoveFileExW replace Replace with POSIX semantics
Read, write and delete ERROR_ACCESS_DENIED (5) Succeeds
Read and write, no delete ERROR_ACCESS_DENIED (5) ERROR_SHARING_VIOLATION (32)

AtomicFile::commit() used one MoveFileExW() call, so each filter that writes over an existing output file can fail in the same way, with a lower probability.

2. File replacement helper

  • src/simplnx/Common/FileReplacement.hpp, src/simplnx/Common/FileReplacement.cpp — InstallReplacementFile(), DescribeFileReplacementFailure() and DescribeFileReplacementSystemError().
  • src/simplnx/Common/FileReplacementDetails.hpp — the retry steps, the retry limits and RenameWithFallback() in the namespace nx::core::detail. The tests use them to script each step.

On Windows, one attempt does these steps:

  1. A rename with SetFileInformationByHandle() and FileRenameInfoEx with POSIX semantics.
  2. If step 1 fails for any reason, one MoveFileExW(MOVEFILE_REPLACE_EXISTING). On exFAT, FAT32 and an SMB share, step 1 fails with ERROR_INVALID_PARAMETER and step 2 does the rename (measured). No list of "not supported" error codes is used, because other file systems were not measured.
  3. If the two steps fail, the attempt reports the error of step 1 when that error permits a retry, and the error of step 2 when it does not.

The helper retries ERROR_ACCESS_DENIED, ERROR_SHARING_VIOLATION and ERROR_LOCK_VIOLATION with waits that double from 1 ms to 64 ms, and it stops when the waits reach 2,000 ms. It does not retry ERROR_ACCESS_DENIED when the destination is read-only or a directory. Each other error returns after one attempt.

FileReplacementDurability::WriteThrough flushes the data of the replacement file to the device (FlushFileBuffers()) before the rename. If the flush fails, the destination is unchanged. The rename itself is not flushed. The default does not flush, because a caller can replace a file thousands of times. On Linux and macOS this option has no effect (#1992).

DescribeFileReplacementFailure() gives the destination, says that it is unchanged, gives one action for the user, and gives the operating-system message last.

3. AtomicFile

  • src/simplnx/Common/AtomicFile.cpp, src/simplnx/Common/AtomicFile.hpp — commit() calls the helper with WriteThrough.

Error code -15782 does not change. The message text changes: it now comes from DescribeFileReplacementFailure().

Behavior change: before, commit() used MOVEFILE_WRITE_THROUGH. Now it flushes the complete temporary file before the rename. This is a stronger guarantee, and it can make the save of a large file slower. The time was not measured.

4. WriteDREAM3D

  • src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/WriteDREAM3D.cpp — the .xdmf file is installed with the helper. A failure returns the new error code -78500. The message says that the .dream3d file was saved and that an existing .xdmf file can be out of date. The two error returns keep the warnings of the writer.
  • src/Plugins/SimplnxCore/docs/WriteDREAM3DFilter.md — describes error -78500.

5. Tests and knowledge base

  • test/FileReplacementTest.cpp — the retry limits with scripted steps, the order and the reported error of the two rename steps, the messages, real files, and Windows cases in which a second handle holds the destination or the replacement.
  • test/AtomicFileTest.cpp — a failed commit keeps the destination and the temporary file (all platforms), and a Windows case in which commit() replaces a file while a handle with full sharing is open on it.
  • src/Plugins/SimplnxCore/test/DREAM3DFileTest.cpp — error -78500 when the .xdmf path cannot be replaced.
  • docs/Coding_Knowledge_Base.md — a new entry: replace files through InstallReplacementFile().

Test plan

  • Windows 11, MSVC, Release, with the companion branch: ctest -R "InstallReplacementFile|FileReplacement|AtomicFile|StoreFileReplacement|NumericPrepared|WriteDREAM3D|DREAM3DFile|MergeTwins|Resize|HDF5Chunked" passes, 199 of 199 tests.
  • Windows, mutation check: with the POSIX-semantics flag removed from the rename, AtomicFile::CommitReplacesFileWithFullSharingHandle and the Windows cases of the companion pull request fail. With the flag, they pass.
  • macOS arm64, in-core Release build: ctest -R "InstallReplacementFile|FileReplacement|AtomicFile|WriteDREAM3D|DREAM3DFileTest|MergeTwins" passes, 34 of 34 tests.
  • macOS arm64, out-of-core Release build: ctest -L "^ooc$" -LE "benchmark" passes, 1061 of 1061 tests.
  • The MoveFileExW() step has a scripted test only. The helper was not run on exFAT, FAT32, an SMB share or ReFS. The behavior of the two Win32 calls on exFAT, FAT32 and SMB was measured with a separate script.
  • No test makes FlushFileBuffers() fail, and that the data is on the disk after a power failure was not tested.
  • No test covers the warnings that WriteDREAM3D keeps on its error returns. No existing fixture makes the writer succeed with a warning.
  • Full suites on the final commit, rebased on develop, with the companion branch and DREAM3DNX develop:
    • Linux (Ubuntu 24.04), simplnx only: gcc-13 passes 4162 of 4162 tests, clang-16 passes 4162 of 4162 tests.
    • Linux, complete DREAM3D-NX out-of-core build with the Azure CI scripts: 5570 of 5570 tests pass.
    • Windows 11, MSVC, complete DREAM3D-NX out-of-core build, Windows Search running: 5554 of 5572 tests pass.
    • Windows 11, MSVC, complete DREAM3D-NX in-core build: 4425 of 4443 tests pass.
    • The 18 Windows failures are the same in the two builds: 17 VizTest rendering tests and Perf: Offscreen render (empty scene). The run was started through ssh without a desktop session, and the log shows that the render window got no OpenGL pixel format. They were not run on develop for comparison.

Related Issue

Related to BlueQuartzSoftware/SimplnxOoc#39

@imikejackson imikejackson self-assigned this Nov 18, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

On Windows, a replacing rename fails with "Access is denied" while a
handle to the destination file is open in a process, also when that
handle shares read, write and delete access. The Windows Search
indexer opens new files in that way for some microseconds. An
operation that replaces a file many times then fails on a machine
where the Windows Search service runs.

* Add InstallReplacementFile(). On Windows it uses a rename with POSIX
  semantics, which an open handle with delete sharing does not block.
  It uses MoveFileExW() when the file system does not support that
  rename, and it retries a sharing error for a maximum of 2 seconds.
  On Linux and macOS it renames one time.
* Use it in AtomicFile::commit() with the write-through option. Error
  code -15782 does not change.
* Add tests for the retry limits, for real files, and for Windows
  handles that hold the destination.
* Add the rule to the coding knowledge base.

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
* Use MoveFileExW() also when the open of the replacement file reports
  an operation that the file system does not support.
* Give the public members of the helper types CamelCase names, as the
  clang-tidy configuration requires.
* Return the new error code -15785 from AtomicFile::commit() when the
  rename succeeded and only the flush to disk failed. WriteDREAM3D
  then installs the XDMF file before it returns the error.
* Install the XDMF file of WriteDREAM3D with InstallReplacementFile()
  and report a failure with the new error code -78500.
* Stop after one attempt when the destination is read-only or a
  directory. The other access-denied errors keep the limited retry.
* Open the Windows test handles with read access, because Windows does
  not apply the share mode to a handle with attribute access only.
* Make each failure description say whether the destination changed.
* Add tests for a relative destination, an absent destination, a
  read-only destination and a held replacement file.

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
A design review of the file replacement helper required these changes.

* Try MoveFileExW() in the same attempt after each failure of the
  rename with POSIX semantics. The list of "not supported" error codes
  is removed, because it was measured on three file systems only. When
  the two renames fail, report the first error if it permits a retry,
  and the error of MoveFileExW() if it does not.
* Flush the replacement file before the rename when the write-through
  option is set. A failed flush then leaves the destination unchanged.
  Remove error code -15785, AtomicFile::commitResult() and the special
  case in WriteDREAM3D, which were necessary only for a flush after
  the rename.
* Move the retry steps, the retry limits and the fallback function to
  FileReplacementDetails.hpp in the namespace nx::core::detail. The
  public header keeps only the functions that callers use.
* Say "on NTFS" where the documentation describes the rename with
  POSIX semantics.
* Add tests for the fallback order and for a failed commit that keeps
  the destination and the temporary file.

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
An independent review found no Critical or Important problem. These
are its Minor and Nit findings.

* Give the read-only and permission advice when the replacement stops
  after one attempt, and the "open in another program" advice when
  the retries are used up. Before, a read-only destination got the
  least specific text.
* Make the -78500 message of WriteDREAM3D say that the .dream3d file
  was saved, that an existing .xdmf file can be out of date, and what
  to do. Use the same words in the filter documentation.
* Keep the warnings of the writer when WriteDREAM3D returns the commit
  error or the -78500 error.
* Change the retry detail to "(tried N times over M ms)".
* Name the WriteDREAM3D test after its plugin and write its files in
  the unit test output folder.
* Correct the clang-tidy findings in the new code.

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
@imikejackson

Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant