test: add opt-in live tests against the Anthropic API and gateways

[hakula139]

Summary

The runtime features that depend on a real endpoint had no repeatable check, including streaming, prompt caching, signed thinking replay, compaction, resume, and the billing header. This adds opt-in live tests that run ox against whatever endpoint the environment selects, plus a checklist for the TUI-only paths. Both were run against the Anthropic API and an Anthropic-compatible gateway, and the per-feature results are in the comment on #123.

• Live tests. crates/oxide-code/tests/live.rs runs ox headless and in the bare REPL, then checks the session file it writes:
  • Streamed text arrives in several chunks and matches the recorded reply.
  • A tool round trip reads the first request from the prompt cache.
  • Signed thinking after tool results is replayed across three turns on Opus, and each request reads the previous one's cached prefix.
  • Auto-compaction summarizes from the cached conversation and leaves the next prompt under half its size.
  • --continue answers from the resumed transcript and reads its cache.
• Opt-in and isolated. The tests are #[ignore]d and also return early unless OX_LIVE=1, so CI and --include-ignored send no requests. Each runs with a scratch home, config, data, and state directory, so no global instructions, config file, or stored login apply, and the endpoint and credentials come from the environment alone. The harness drops inherited OX_* settings other than OX_EXTRA_CA_CERTS, along with ANTHROPIC_EFFORT and ANTHROPIC_MAX_TOKENS. OX_LIVE_MODEL and OX_LIVE_THINKING_MODEL pick the models.
• Leak check. After each run, before any assertion formats its output, the harness fails the test when the value of ANTHROPIC_AUTH_TOKEN, ANTHROPIC_API_KEY, or CLAUDE_CODE_OAUTH_TOKEN, or an ANTHROPIC_CUSTOM_HEADERS value of 8 or more characters, appears in the captured stdout, stderr, or any file under the scratch directory. The failure names the source and the variable without printing the value.
• Docs. The testing conventions gain a Live Tests section with the run command and a checklist for /compact, the /model swap, Esc cancellation, and a leak check that tests grep's exit status. The API research note records the billing header recheck against Claude Code 2.1.301. The salt and indices match, the fingerprint's message selection changed (Fingerprint the billing header from the first user prompt #257), and a scan of the binary did not find the xxHash64 seed.

The runs found four bugs, filed as #254, #255, #256, and #257. This PR leaves #123 open because the seed recheck is incomplete.

Test plan

• OX_LIVE=1 cargo test --test live -- --ignored passed all 5 tests against the Anthropic API with an OAuth token, where requests carry the billing header with a live cch.
• The same command passed all 5 tests through an Anthropic-compatible gateway. Those runs went through a localhost proxy that maps model ids to the ones the gateway serves (Load the model catalog from data #152), since ox on macOS rejects the gateway's certificate (Trust long-lived certificates from an extra CA on macOS #254).
• With OX_LIVE=1, temporary probe tests confirmed that a fake ANTHROPIC_API_KEY echoed to stdout and a fake custom header value written to a scratch file each fail with the source and variable name and without the value, and that a 2-character header value is skipped.
• Without OX_LIVE=1, cargo test reports the 5 live tests as ignored, and --include-ignored passes them without sending requests.
• On ba9d130, I ran the TUI checklist through the gateway on Haiku and Opus. /compact and the /model swap in both directions passed, and the leak check found no token in the logs or session files. Esc stops the stream, but the next reply also answers the cancelled prompt (Tell the model when the user interrupted the previous prompt #256).
• cargo fmt --all --check, cargo build, cargo clippy --all-targets -- -D warnings, cargo test, cargo llvm-cov, pnpm format, pnpm lint, pnpm spellcheck, and the pre-commit hooks passed. Files under tests/ are outside the coverage report.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!