Repository navigation
fix(backtests): print the deadline-hold notice on a late submit too - #650
Merged
FlyM1ss merged 2 commits intoOct 22, 2026
Merged
Conversation
Contributor
|
@subidhkhanal is attempting to deploy a commit to the allan-feng's projects Team on Vercel. A member of the Team first needs to authorize it. |
Review follow-up on the late-submit hold notice. - submit_decisions refuses a decision built for a step that has already closed instead of applying it to the next step. It catches this in two ways: an explicit expected_step (v2 _submit_for, v1 protocol submit), and an inference from the last step served whose step was auto-held (the legacy route, and a v2 POST arriving after the reaper's hold). - All four hold sites go through _apply_expired_holds, the only caller of _maybe_apply_timeout, so no hold can be silent and the count is no longer hard-coded. - A late hold of the final step returns run_id/metrics/compare_url, and v2 marks protocol_runs completed and closes the trace right away. - v2 audits the idempotency row and trace under the step that actually executed or was refused. - The accepted ack reports decision_source again: _open_current_step no longer resets it. - Tests: submit and route-level coverage for each case. The load-test README and test docstring no longer describe Open-Finance-Lab#375 as open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #375
Problem. A decision step auto-holds (no trade, the agent's decision skipped) at one of four places when its deadline has passed. #374 added a printed warning at three of them —
get_current_step,drain_expired,get_status— all of which go through the shared_maybe_apply_timeout.submit_decisionshad its own, independent deadline check that bypassed that helper entirely, so a hold created by a late POST printed nothing. Two real routes reach this: the legacy/api/v1/backtest/*surface (Discord bot, built-in agents) and the canonical/api/v2contract. Both are ordinary GET-then-POST flows where the deadline can elapse in the gap, which is exactly the case this feature exists to surface. The periodic reaper doesn't catch it either, sincesubmit_decisionshas already advanced the step by the time it runs.Fix.
submit_decisions's late-submit branch now goes through the same shared_maybe_apply_timeoutthe other three sites use, instead of its own inline check, and prints the warning via the existing_emit_deadline_hold_noticehelper on a hold. Routing through the shared function means there is only one place in the code that decides "this step is late" — a future change to that rule can't miss this spot the way #374 did.Tests. Two new cases in
test_deadline_hold_visibility.py:I verified the new late-submit test actually catches the bug: run against the pre-fix code it fails (0 warnings printed instead of 1).
Verification run locally:
84 passed. Two pre-existing failures in that run also fail on unmodified
mainon Windows and are unrelated (Windows clock-resolution flakiness in deadline-timing tests — real-clock comparisons at a 0-second deadline can land exactly on the boundary; both pass reliably on Linux/CI).Review follow-up (dd54d9f). Both items from the scope note are fixed here, plus three related ones from review:
_apply_expired_holds, now handles hold-and-print for all four sites. It is the only caller of_maybe_apply_timeout.run_id/metrics/compare_url. On v2 it also marks theprotocol_runsrow completed right away.step_already_closed) instead of being applied to the next step. That covers the reaper or a status poll holding the step first, and a hold landing between v2's step read and apply. Mechanism: anexpected_steppin plus served-step inference.decision_sourceagain, instead ofNone.🤖 Generated with Claude Code