Repository navigation
test(sdkharness): refuse a resilience run on a simulator that already served requests - #624
Merged
sophiecarreras merged 1 commit intoOct 4, 2026
Conversation
… served requests The resilience leaves count the simulator's whole request journal, which has no reset. On a simulator shared by several scenarios, upload.cap_exceeded_403 read earlier scenarios' b2_upload_file entries as its own and failed with N calls, and api.retry_after_503 counted earlier injected 503s. The dispatcher now FAILs with a configuration reason when the journal is not empty. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
sophiecarreras
deleted the
sdkharness/resilience-require-fresh-simulator
branch
October 4, 2026 01:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Local runs of the repository-owned resilience scenarios on ONE shared simulator failed where the harness runner (fresh simulator per scenario) passes:
upload.cap_exceeded_403("N b2_upload_file calls", fleet run 37096583023 passed it) andapi.retry_after_503("2 injected 503 ..., expected 1").Root cause: the leaves read the simulator whole request journal (
GET /journal) and count its entries, and the simulator has no journal reset. On a simulator an earlier scenario used, a leaf reads that scenario requests as its own. This is not an SDK defect and the simulator behaves as documented.The dispatcher now FAILs with a
configurationreason when the journal is not empty, so a shared simulator produces an honest setup error, not a false SDK verdict. The harness runner starts a fresh simulator per scenario, so its results are unchanged (15 PASS +upload.stallno-client-option).Test plan
test_sdkharness_conformance_resilience_contract.py(98 passed acrosstest_sdkharness*.py).api.retry_after_503andupload.cap_exceeded_403FAIL; after: refused with the configuration reason. Fresh simulator per scenario: 15 PASS + 1 SKIP. Harnessbin/run-resilience.sh b2-sdk-pythonagainst this commit: 15 PASS + 1 SKIP.Touches
.sdkharness/tests/lib/contract.py, which #623 also edits; trivial rebase if #623 merges first.🤖 Generated with Claude Code