Skip to content

fix: Log output and metrics for incomplete OpenAI Responses API streams - #2540

Merged
Luca Forstner (lforst) merged 8 commits into
braintrustdata:mainfrom
RaphaelFakhri:fix/openai-responses-stream-incomplete
Oct 2, 2026
Merged

Luca Forstner (lforst) merged 8 commits into
braintrustdata:mainfrom
RaphaelFakhri:fix/openai-responses-stream-incomplete

Conversation

@RaphaelFakhri

@RaphaelFakhri Raphael Fakhri (RaphaelFakhri) commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Streaming Responses API calls that end with response.incomplete or response.failed now get their output, metadata and token metrics logged. For response.failed, the response's error is also logged as the span error.

The OpenAI plugin only read the final response object from response.completed events. A stream that stops at max_output_tokens or because of a content filter ends with response.incomplete instead, and a stream that errors server-side ends with response.failed. Both events carry the same final response object with the partial output and usage, but we ignored them, so the span had no output and no token metrics even though the request was billed.

aggregateResponseStreamEvents (responses.create({ stream: true })) and the responses.stream() event handler now both treat these three event types as terminal. The OpenRouter plugins already do the same. To log the response.failed error, traceStreamingChannel's aggregateChunks result can now return an optional error.

Tests

  • Unit tests in openai-plugin.test.ts cover output, metadata and metrics for all three terminal events, the response.failed error, and ignoring non-terminal events that carry a response.
  • New responses-create-stream-incomplete and responses-stream-incomplete e2e operations in openai-instrumentation hit max_output_tokens against gpt-4o-mini. Cassettes were recorded for all v4/v5/v6 pinned and latest variants.

@lforst
Luca Forstner (lforst) merged commit c1138e7 into braintrustdata:main Oct 2, 2026
56 checks passed
@AbhiPrasad

Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants