Skip to content

fix(ai): capture error event when streaming request fails to open - #1213

Closed
pcbeingused333 wants to merge 1 commit into
PostHog:mainfrom
pcbeingused333:fix/stream-open-error
Closed

pcbeingused333 wants to merge 1 commit into
PostHog:mainfrom
pcbeingused333:fix/stream-open-error

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Related to #1188 and #1110 (same family: provider errors not reaching $ai_generation).

💡 Motivation and Context

create(stream=True) in the OpenAI (chat completions, responses; sync and async) and Anthropic (sync and async) wrappers called the provider outside the tracked generator. A request that fails while opening the stream (401, 429, 5xx) raised straight to the caller and no event was captured; only mid-stream errors were reported, and the non-streaming path (call_llm_and_track_usage) already captures open-time errors.

Changes:

  • Wrap the provider call in each _create_streaming; on failure, capture the existing streaming event with the error (so $ai_is_error, $ai_error and $ai_http_status are set) and re-raise.
  • Sampo changeset (patch).

💚 How did you test it?

Added six tests to test_stream_midstream_error.py (chat, responses, Anthropic; sync and async) where the provider create raises a 429-style error: they fail on main (no event) and pass with the change. pytest posthog/test/ai/openai posthog/test/ai/anthropic posthog/test/ai/test_stream_midstream_error.py posthog/test/ai/test_async_stream_wrapper.py passes (196 passed, 2 skipped). ruff format --check passes on the touched files.

📝 Checklist

  • All commits are signed (see Commit signing)
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran sampo add to generate a changeset file (.sampo/changesets/ai-stream-open-error.md, patch)

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Prepared with Claude Code. Scope is create(stream=True) only. Two related paths are left out to keep the diff small: Anthropic messages.stream() still captures nothing when the request fails to open (checked against a mocked 429), and from reading the code the async Gemini wrapper awaits the request outside its generator, so it likely has the same gap. Sync Gemini was not touched (its stream is a lazy generator).

OpenAI chat/responses and Anthropic create(stream=True) called the provider outside the tracked generator, so a 401/429/5xx raised while opening the stream was never reported. Capture an error event (with the HTTP status) before re-raising.

Signed-off-by: Alex Castillo <alex.castillog33@gmail.com>
@pcbeingused333
pcbeingused333 requested a review from a team as a code owner November 9, 2026 09:45
@pcbeingused333

Copy link
Copy Markdown
Contributor Author

Closing: main already includes the same fix via #1146, so this PR is now a duplicate. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant