Repository navigation
Conversation
Add arch.univariate.backtest with standalone Value-at-Risk backtests, exported from arch.univariate and added to the utility page of the documentation (xref #904). VaR is a positive loss and an exception is -returns > var. - var_exceptions returns the bool array of exceptions. - kupiec_pof is the proportion-of-failures test of Kupiec (1995), chi2(1). - christoffersen_independence is the Markov independence test of Christoffersen (1998), chi2(1), and christoffersen_conditional_coverage is the sum of the two statistics, chi2(2). The tests return a WaldTestStatistic. - traffic_light is the Basel Committee (1996) zone over the last window observations (250 by default) with the exception count and the cumulative binomial probability, returned as a TrafficLightResult. For 250 observations and p=0.01 the zones are those of Table 2: green 0-4, yellow 5-9, red 10 or more. The inputs must be finite. Series are aligned on the index when both are Series, and must otherwise have the same length. A series with no exceptions, only exceptions or no 1 -> 1 transition is handled with the limits of the likelihood (0 log 0 = 0). A series that is shorter than the window raises. The tests compare the statistics and p-values with rugarch::VaRTest (1.5.6) and with an independent likelihood implementation in R, the zones with the binomial distribution and the published Tables 1 and 2 of the Basel framework, and the non-rejection regions with Kupiec (1995).
|
|
||
| def test_p_must_be_given(): | ||
| with pytest.raises(TypeError, match="p"): | ||
| kupiec_pof(RETURNS, VAR) # type: ignore[call-arg] |
| with pytest.raises(TypeError, match="p"): | ||
| kupiec_pof(RETURNS, VAR) # type: ignore[call-arg] | ||
| with pytest.raises(TypeError, match="p"): | ||
| christoffersen_conditional_coverage(RETURNS, VAR) # type: ignore[call-arg] |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #965 +/- ##
==========================================
+ Coverage 98.95% 98.97% +0.01%
==========================================
Files 136 139 +3
Lines 30043 30596 +553
Branches 2331 2360 +29
==========================================
+ Hits 29729 30282 +553
Misses 256 256
Partials 58 58
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Target 9.1: merge after v9.0.0 is tagged. xref #904. This implements the five standard
backtests from the proposal.
max_undetectable_understatementis left out, pending adecision on the issue.
Adds the new module
arch/univariate/backtest.py, exported fromarch.univariate:var_exceptions(returns, var): a bool array of-returns > var. VaR is a positive loss,and a loss equal to the VaR is not an exception.
kupiec_pof(returns, var, p): the Kupiec (1995) proportion-of-failures likelihood-ratiotest, chi2(1).
christoffersen_independence(returns, var): Christoffersen's (1998) Markov independencetest, chi2(1).
christoffersen_conditional_coverage(returns, var, p): LR_uc + LR_ind, chi2(2).traffic_light(returns, var, p=0.01, window=250): the Basel Committee (1996) zone,returned as a frozen
TrafficLightResultwith the zone, the count, the cumulativeprobability,
nobsandp.The three tests return
WaldTestStatistic.Inputs:
0·log 0 = 0throughxlogy.Validation:
VaRTest, on 11 cases: largest difference in the statistics1.2e-14.
cannot go through rugarch, which indexes a 2x2 table. They are checked against an
independent R likelihood implementation, cross-checked with rugarch.
pbinomto 1e-12, and the publishedTable 2 of the BIS 1996 backtesting framework, including the zone boundaries at 5 and
10.
values.
Tests:
test_backtest.py(198) andtest_validation_backtest.py(299 table rows).docs build has no new warnings (a new "Backtesting Value-at-Risk" section with an
example).
For review:
traffic_lightraises for a series shorter thanwindow, rather than using what isthere.
pis required except intraffic_light.TrafficLightResultis public.var_exceptionsreturns an ndarray, not a Series.WaldTestStatistic.pvalis1 - cdf, which is imprecise below about 1e-15. A survivalfunction would fix it, as a separate change.
Thanks to @sarthakguptaquant for the proposal.
🤖 Generated with Claude Code