Repository navigation
The Windows CI jobs install a slim conda environment and cache none of it, and link with Visual Studio's tools - #527
Merged
ofloveandhate merged 14 commits intoNov 7, 2026
Conversation
…ead of compilers, whose Fortran compiler brought clang along with ~600 MB of LLVM tools and flang; clang is now 22 instead of 19 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…runners have git, and nothing asked for conda's copy Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ller; b2 links no BLAS, and the Linux jobs already test against OpenBLAS (numpy's pip wheels) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the environment file and the month, not the workflow: ~0.5 GB in place of six unpacked folders of ~2.6 GB each, which evicted every other platform's caches; the wheel tests restore it too Of a ~320 MB solve, ~40 MB depends on the Python version, so whichever job saves the cache first leaves the others a few packages to download. Unpacked copies are left out of the cache: conda unpacks an archive it finds. Leaving the workflow file out of the key means editing the workflow reuses the cache; the month in it brings conda-forge's updates in. The wheel builds' pip cache held only build and delvewheel and is dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he environment asks for clang >=20, which Visual Studio 2026's STL accepts, the proper fix the override's comment named Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kage archives, keyed by the environment file and the month Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…month: each job restores the newest cache for its Python, installs, records conda list --explicit, prunes to those archives, and saves under the record's hash when it is new A cache that hits saves nothing, so a key that stays the same would make every run download each conda-forge release since the save; the month only bounded that. The record changes exactly when the contents do. One cache per Python version (~320 MB each) stores the shared packages six times, but needs no job ahead of the Windows jobs to solve all six. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ut Boost's auto-linking on Windows asked for boost_json.lib by name, and the link failed once the linker no longer searched conda's library folder Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the SDK's rc.exe and mt.exe, by full path, instead of whatever LLVM the runner image ships, which CMake took once conda's lld was gone Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s/msvc_link_tools.ps1), and log what they were configured with, how the conda cache was used, and fail where the build fails The conda activation does not run Visual Studio's setup script in PowerShell, so VCToolsInstallDir was unset and the previous run stopped before configuring. tools/msvc_link_tools.ps1 takes link.exe and lib.exe from the default toolset of the newest Visual Studio, the one clang-cl finds, and rc.exe and mt.exe from the newest Windows SDK. tools/conda_archive_cache.ps1 takes over the cache bookkeeping and reports, in every Windows job including the wheel tests, the archives reused from the cache, downloaded, and restored but unused, with the main package versions. The build steps print clang-cl's version and the compiler and link tools in CMakeCache.txt, and now fail when the build does: a native command's exit code does not stop a pwsh step, so the first run's link failure surfaced only as ten tests Not Run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cache of the package archives saved no time, so it is gone, and the builds enter Visual Studio's developer environment so link.exe finds the Windows libraries Over three runs, conda installs took 178 s and 211 s in the two jobs that restored a 326.7 MB cache and 116-252 s in the 16 that had none; conda rewrote all 81 restored archives, reusing none, and the most a working cache could save is a download about as fast as its own 5 s restore. ADR-0078 records the numbers and the decision. The previous run's link failed with LNK1104 kernel32.lib: link.exe needs LIB, which only Visual Studio's developer environment sets. tools/msvc_link_tools.ps1 enters it, puts PATH back so the build keeps conda's cmake, ninja, ccache and clang-cl, and takes the four tools from the variables it sets. The build steps print which cmake, ninja, clang-cl and ccache they run, and every Windows job prints the main conda package versions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ofloveandhate
changed the title
…e stack: test_nag_algorithms died with heap corruption (0xc0000374) under clang 22 and link.exe where it passed under clang 19 and lld-link The first run of bertiniteam#527 that linked built everything and passed nine suites, and test_nag_algorithms exited with 0xc0000374 two seconds in, during the zero_dim tests, with no stack and its buffered stdout lost. The suites run in the C++ test job without the mimalloc hook (only the CLI and the Python module install it), so the corruption is in the C runtime heap itself. The C++ test build now carries debug information (/Z7, which ccache can cache, and /DEBUG) and, when a suite fails, a step reruns each failed suite under cdb with Boost.Test naming every test case on stderr as it starts, under both thread settings, and lists the DLLs the binary imports. The package listing adds gmp, mpfr, mpc, libboost-headers and the two runtime packages. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s a scoped variable, it adds a run under full page heap, and the C++ job saves ccache before the tests so a failed test does not cost a cold build The previous run's crash step stopped at a parse error, $failedLog: in a double-quoted string, so cdb never ran. test_nag_algorithms crashed the same way again, 0xc0000374 about two seconds in, so the crash is deterministic. The failed job skipped actions/cache's post step and saved nothing, so restore and save are now separate steps and the save comes right after the build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cleanup ran after the thread's MPFR state was freed, which corrupted the heap when a worker exited (ADR-0079) Run under cdb, test_nag_algorithms faulted in mpfr_bernoulli_freecache reading 0xfeeefeee, called from mpfr_free_cache2 in the executable's __dyn_tls_dtor during thread shutdown, and under full page heap in libgcc's _emutls_get_address. conda-forge's Windows MPFR is built by gcc and keeps its thread-local state in emulated TLS, which was already freed. On Windows, mpfr_extensions.hpp now specializes mpfr_cleanup<true>::force_instantiate to skip Boost's thread_local object, and WorkerThreadPool, the only place b2 starts threads, calls ReleaseThreadMpfrCaches last on every platform. thread_pool_workers_that_filled_mpfr_caches_exit_cleanly is the regression test. The serial rerun found a separate bug, in the test the_accuracy_estimates_are_recorded_under_names_that_say_their_coordinates: earlier["accuracy_estimate"] = earlier.at(...) read a value through a reference the insert had invalidated. AddressSanitizer reports it as heap-use-after-free on Linux too. The test now copies the value first. The C++ test build drops the /Z7 and /DEBUG flags that found this, and keeps the step that reruns a failed suite under cdb. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ofloveandhate
added a commit
that referenced
this pull request
Nov 8, 2026
ofloveandhate
added a commit
that referenced
this pull request
Nov 8, 2026
…MPFR thread-exit fix, Windows ccache, the random state and solution metadata JSON (#531) Brings `develop` into `integration/intervals`, so the interval work builds on what has landed there since the branch was cut: - #527: the slim Windows conda environment, clang 22 with Visual Studio's `link.exe`, and the fix for heap corruption when a worker thread exits on Windows (ADR-0078, ADR-0079). - #524: ccache caches the Windows builds (ADR-0076). - #520: the random state can be got and set (ADR-0077). - #522: a solution's metadata round-trips through exact JSON. It is a merge, not a rebase, as an integration branch takes its trunk. Two files conflicted, both lists where each side added entries: the 4.1.0 Added block of the changelog (FLINT here; the random state and solution metadata on develop) and the ADR index (0075 here; 0076 to 0079 on develop). Both sides are kept, in order. The workflow, `core/CMakeLists.txt` and `environment-win.yml` merged without conflict, and `libflint` stays in the slim Windows environment. One commit beyond the merge: the Windows jobs print FLINT's, GMP's, MPFR's and MPC's versions with the other main conda packages. What this run shows for the first time: FLINT linked by `link.exe` on Windows (#526 linked it with `lld-link`), and `test_flint` under clang 22. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The repository's Actions caches were full: 8.9 GB of GitHub's 10 GB, all on develop, and not one Linux cache. The Windows jobs each cached their whole conda package folder, about 2.6 GB apiece, one for the C++ tests and one per Python version. Six of those don't fit, and while they evicted each other they evicted every other platform's caches and every pull request's, so PRs compiled cold.
This PR set out to cache the Windows environment better, measured whether that pays, and found it doesn't: the Windows jobs now install a slim environment and cache none of it. ADR-0078 records the numbers.
Moving the toolchain also exposed a crash on Windows that the old one had hidden, which this PR fixes. ADR-0079 records it.
What changes
environment-win.ymlasks forclang >=20, which provides clang-cl, instead of thecompilersmetapackage: its Fortran compiler (flang) brought clang along with llvm-tools and lld, 830 MB of a 1153 MB download. It drops conda's git (115 MB; the runners have git) and takes OpenBLAS instead of MKL for numpy and scipy (109 MB; b2 links no BLAS). An install downloads ~325 MB._ALLOW_COMPILER_AND_STL_VERSION_MISMATCH. The pin isclang >=20(22.1.8 today), which compiled everything without it.lld-link,llvm-rcandllvm-mtfrom the runner's own LLVM, which nothing pins.tools/msvc_link_tools.ps1enters the developer environment of the newest Visual Studio (vswhere; the one clang-cl uses) forLIB,LIBPATHandINCLUDE, puts PATH back so conda's cmake, ninja, ccache and clang-cl stay in use, and returnslink.exe,lib.exe,rc.exeandmt.exe, which both builds pass to CMake by full path.bertini2definesBOOST_JSON_NO_LIB. b2 uses Boost.JSON header-only, but on Windows Boost's auto-linking asked forboost_json.libby name, and only the old linker happened to find one in conda's library folder.clang-cl --version, and the compiler and link tools inCMakeCache.txt. A native command's exit code does not stop a pwsh step, so a failed link had surfaced only as ten tests "Not Run".test_nag_algorithmsdied with heap corruption (0xc0000374). Boost.Multiprecision frees MPFR's per-thread caches from athread_localdestructor; conda-forge's Windows MPFR keeps that state in libgcc's emulated TLS, and in an executable the destructor ran after the state was freed, so a worker's exit read freed memory. The read does not depend on the compiler; why it became fatal here is not established. On Windows,mpfr_extensions.hppnow skips Boost'sthread_local, andWorkerThreadPoolcallsReleaseThreadMpfrCaches()last on every platform.thread_pool_workers_that_filled_mpfr_caches_exit_cleanlyis the regression test.earlier["accuracy_estimate"] = earlier.at(...)inzero_dim_records.cppread through a reference the insert had invalidated. The debugger's heap made it fail, and AddressSanitizer reports it on Linux too.actions/cache's post step is skipped when a job fails.The wheel jobs' pip cache, which held only
buildanddelvewheel, is dropped. The branch name records where this started.After merging
The old
windows-conda-*caches on develop (~2.6 GB each) can be deleted by hand or left to age out after 7 days unused.🤖 Generated with Claude Code