Verify runtime features live through a gateway #123

Goal

Unit and WireMock tests pin down request shapes, but only a real endpoint can show whether the betas, block_binding, display: "updates", cache breakpoints, and compaction fork behave as designed. Run a scripted live session through a gateway with a real key, and record the result of each feature.

Acceptance criteria

• A repeatable checklist, or a test disabled by default and gated on an environment variable, covers:
  • streaming text and tool use
  • thinking interleaved across three turns and preserved
  • a cache read on the second request
  • /compact and auto-compaction
  • a /model swap
  • cancellation mid-stream
  • a resumed session
  • the billing header's cch hash, rechecked against the current Claude Code release (xxHash64 seed included)
• Record each result and create a separate bug issue for each failure.
• Logs, issues, and PRs contain no token, internal hostname, or header value.

Results

The latest run was the alpha.3 gate on 2026-10-17. It used ox at 01b4823, with a delta rerun at 5b42851 (#266). The thinking test and the /model swap ran on Opus 5.5, and everything else ran on Haiku 5.5. The Anthropic API runs used an OAuth token. The gateway runs went to an Anthropic-compatible gateway through a localhost proxy (#152, #254). The live tests and the TUI checklist come from #258.

• Cache: In every inspected session, each request read exactly what the previous one wrote to and read from the cache. Replayed thinking blocks, including blocks written by another model after a /model swap, kept the cached prefix intact.
• Cancellation: After a cancel before output and a cancel mid-stream, the next prompt got only its own answer. The session recorded [Request interrupted by user] after the cancelled prompt.
• Compaction: With a 50K threshold, a transcript dominated by two 60 KB tool results compacted on the next prompt, and the summary request read all prior tokens from the cache. /compact below 1000 tokens reports the minimum size.
• cch recheck against Claude Code 2.1.301: The version fingerprint in its bundled JavaScript still uses the salt 59cf53e54c78 and characters 4, 7, and 20 of the first non-meta user message, which matches ox after Fingerprint the billing header from the first user prompt #257. The body hash itself is computed in the runtime's native request layer, where the xxHash64 seed 0x6E52736AC806831E can't be read out of the binary, so the seed is confirmed only by the API accepting every request hashed with it. Checking it further would require capturing Claude Code's own TLS traffic, which this issue does not cover.

Bugs found by earlier runs: #254, #255, #256, and #257, all fixed. Not covered yet: the TUI checks against the Anthropic API, and a failed auto-compaction holding the prompt (#265).