Repository navigation
fix(ai): capture an error event when a streaming call fails to connect - #1146
Conversation
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Alex Castillo <alex.castillog33@gmail.com>
marandaneto
left a comment
There was a problem hiding this comment.
Automated advisory code review.
marandaneto
left a comment
There was a problem hiding this comment.
Automated advisory code review.
…ror-capture # Conflicts: # posthog/ai/openai/openai.py # posthog/ai/openai/openai_async.py
|
@pcbeingused333 ping here, otherwise i can fix comments as well |
Signed-off-by: Alex Castillo <alex.castillog33@gmail.com> # Conflicts: # posthog/test/ai/test_stream_midstream_error.py
…nnect If building the telemetry event raises (for example while formatting input the provider just rejected), the capture error replaced the original exception. Log it instead and re-raise the provider error. Signed-off-by: Alex Castillo <alex.castillog33@gmail.com>
|
Thanks for the ping. Pushed two things:
The Gemini mypy annotation from the second review was already fixed in 29325a4. This comment was drafted with Claude Code. |
dustinbyrne
left a comment
There was a problem hiding this comment.
The Anthropic exception-preservation correction is verified. I still found an OpenAI Chat exception-masking case and privacy-sensitive fallback logging, plus two small repository-check failures. Details inline.
AI-assisted review.
If capturing the connect-error event itself raised (e.g. unformattable input), the capture exception replaced the provider error. Wrap the capture in each OpenAI/Gemini wrapper, log a fixed message, and re-raise the original error. The Anthropic fallback log no longer includes the exception text or traceback. Signed-off-by: Alex Castillo <alex.castillog33@gmail.com>
|
Thanks, all three addressed in 8c738e1:
This change was prepared with AI assistance (Claude). |
marandaneto
left a comment
There was a problem hiding this comment.
The code looks good; no qualifying findings. Verdict: correct.
💡 Motivation and Context
For streaming calls the OpenAI chat, OpenAI Responses, Anthropic and Gemini wrappers call the underlying
create/generate_content_streamoutside the try/finally that captures the event, and only wrap the iteration of the returned stream. A connection-time failure (401 bad key, 429 rate limit, invalid model) is raised before any generator exists, so no$ai_generationevent is sent; only errors raised mid-stream are recorded. Non-streaming calls already record these errors throughcall_llm_and_track_usage.The streaming call is now wrapped in
try/except Exception: on failure the existing_capture_streaming_eventis called with an empty stream state anderror=exc(which sets$ai_is_errorand$ai_http_status), and the exception is re-raised unchanged. Sync and async, all four providers. No issue is linked; the gap is visible from the code path and reproduced by the tests below.💚 How did you test it?
Eight tests in
posthog/test/ai/test_stream_midstream_error.py(OpenAI chat, OpenAI Responses, Anthropic and Gemini, sync and async) make the underlying call raise a 401 error and assert the captured event has$ai_is_errorand$ai_http_status == 401; all fail onmain(no event captured) and pass with the change.pytest posthog/test/ai: full suite green;ruff checkandruff format --checkclean. Not run against live provider APIs.📝 Checklist
If releasing new changes
.sampo/changesets/stream-connect-error.md(pypi/posthog: patch).🤖 Agent context
Autonomy: Human-driven (agent-assisted)
The change and the tests were written by Claude Code (Anthropic's agent) working for the PR author, who directs this contribution work and is the DRI. The agent compared the streaming and non-streaming paths of each wrapper, wrote the failing tests first, then wrapped only the initial call so the per-chunk handling is untouched. No manual testing against the providers was done.