Repository navigation
TST: Do not compare separate calls bit for bit on macOS x86-64 - #959
Merged
Merged
Conversation
Separate calls are not bit-identical on macOS on x86-64: ZivotAndrews with trim=np.float32(0.25) and trim=0.25 differed in the last place. The float32 data type test of arch_model fitted a GARCH(1,1) to i.i.d. data, whose estimate is on the boundary (alpha=0, beta=1), so the optimizer turned a difference in the last bits into a relative difference of 3e-8 in the mean and omega, against a tolerance of 1e-12. Compare the results of separate calls with a tolerance of 1e-12, and evaluate the GARCH targets of the data type tests at fixed parameters instead of fitting them, so that the test still detects a calculation in float32.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #959 +/- ##
=======================================
Coverage 98.97% 98.97%
=======================================
Files 136 136
Lines 30021 30028 +7
Branches 2327 2327
=======================================
+ Hits 29712 29719 +7
Misses 255 255
Partials 54 54
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
fit and fix store the sample on the model, and forecast checks start against it. The tests shared one model, so when the result fixture was created before a test that fit with first_obs=-20, the forecasts with start=-25 or start=250 raised a ValueError. This failed on the Windows legs, where the tests ran in that order.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Branch
a90-fix-macos-intel-tests, two commits on main e677722.The macos-15-intel wheel job fails on main, with a different test in each run:
test_valid_results_are_unchanged_by_numpy_scalarsfails. It asserts thatZivotAndrews(Y, trim=np.float32(0.25)).stat==ZivotAndrews(Y, trim=0.25).stat, and the two differ in the last place(-4.3285219553671475 vs -4.328521955367147).
test_float32_is_computed_in_float64[arch_model]. That second test passed inthe later run.
Cause. Not float32 itself: float32 to float64 is exact, and both trims give
the same trim index. On macOS on x86-64, two calls with the same inputs are not
always bit-identical. This is most likely because the BLAS kernels' summation
order depends on memory alignment, which changes from call to call.
==.arch_modelfits a GARCH(1,1) to i.i.d. data. Theestimate is on the boundary (alpha = 0, beta = 1), where the likelihood is
flat, so the optimizer turned a difference in the last bits into a relative
difference of 3e-8 in mu and omega, against a tolerance of 1e-12.
Fix (tests only):
assert_allclose(..., rtol=1e-12): theADF, Zivot-Andrews,
mackinnonpandmackinnoncritcomparisons, and thefloat32 against float64 long-run covariance in
test_covariance_dtypes.py.arch_modelandarch_model(mean="ARX")) evaluate the model at fixed parameters (thelog-likelihood and the conditional volatility) instead of fitting it. They
still exercise the float64 conversion and still detect a calculation in
float32 (an error of about 1e-7), without the optimizer.
Fast suite 26,955 passed. No library code changes.
Also fixed: an order-dependent test on the Windows legs. The first CI run
of this PR failed on the two Windows legs:
test_model_forecast_negative_startandtest_align_horizon_is_target, from ENH: Accept 0 and 1 as flags, negative positions and the horizon alias of align #956.Cause:
modelfixture for the whole module, andfitandfixstore the sample on the model.
resultfixture had been created before the test that fits withfirst_obs=-20, the model kept that sample. Thenforecast(start=-25)failed the check that
startis not before the first fitted observation.The tests now use a new model each, and
resultfits its own. Reproducedlocally with that order and fixed. The file passes in 8 random orders, and the
full fast suite passes in 2 random orders (26,955 passed). The other modules
with shared models passed in 5 random orders.
🤖 Generated with Claude Code