Skip to content

fix(aiclient): surface Responses API stream and status errors - #3285

Open
Abhiram Shaji (abhiramshaji-cyber) wants to merge 3 commits into
microsoft:mainfrom
abhiramshaji-cyber:fix/responses-stream-errors
Open

Abhiram Shaji (abhiramshaji-cyber) wants to merge 3 commits into
microsoft:mainfrom
abhiramshaji-cyber:fix/responses-stream-errors

Conversation

@abhiramshaji-cyber

@abhiramshaji-cyber Abhiram Shaji (abhiramshaji-cyber) commented Nov 14, 2026 •

Copy link
Copy Markdown

When a Responses API request fails partway through, callers got the half finished text back as a normal answer, and now they get the text so far followed by the server's error.

What this code is for

ResponsesWireApiProvider in ts/packages/aiclient turns Responses API events into StreamPiece objects. The completeStream loops in openai.ts and copilotModels.ts read events with readServerEventStream, call decoder.push on each one, yield piece.text, and throw piece.error. parseResponse does the same job for non streaming calls.

Before this PR

An app streams a completion from a Responses endpoint (Azure/OpenAI, or Copilot with wireApi === "responses").

  1. A few response.output_text.delta events arrive and decoder.push returns them as piece.text, which the loop yields.
  2. The server sends {"type":"error","message":"..."} or response.failed with response.error.message.
  3. decoder.push only checks response.incomplete, deltas and usage, so it returns {}.
  4. The loop has no piece.error to throw, Responses sends no [DONE], so the generator just ends.
  5. The app receives the partial text as a finished completion.

Without streaming, a body with status: "failed" or "cancelled" and HTTP 200 went through parseResponse as success("") or partial text. On the Copilot loop, try { decoder.push } catch { continue } also dropped the error decoder.push already throws for response.incomplete.

After this PR

Same app, same request, same as above through step 2.

  1. decoder.push sees error or response.failed and returns { error: new Error(<server message>) }, reading message or error.message on error events.
  2. The loop has already yielded the earlier text, and now throws piece.error.
  3. The app gets the partial text, then an error with the server's message.

Without streaming, parseResponse returns error(<server message>) for failed and cancelled. On the Copilot loop, a line that is not JSON is still skipped, and any other error from decoder.push is thrown.

flowchart LR

classDef box fill:#eef1ff,stroke:#4c5bd4,color:#111827
classDef bad fill:#ffd6d6,stroke:#cc0000,color:#111827,stroke-width:3px
classDef good fill:#c8f0c8,stroke:#008800,color:#111827,stroke-width:3px

subgraph BEFORE["BEFORE: partial text returned as success"]
direction TB
  B1["app calls completeStream"] --> B2["readServerEventStream"]
  B2 --> B3["decoder.push"]
  B3 --> B4{"error or response.failed?"}
  B4 -->|"returns empty piece"| B5["loop ends, partial text looks finished"]
  B4 -->|"output_text.delta"| B6["piece.text yielded"]
end

subgraph AFTER["AFTER: partial text, then the server error"]
direction TB
  A1["app calls completeStream"] --> A2["readServerEventStream"]
  A2 --> A3["decoder.push"]
  A3 --> A4{"THE CHANGE: error or response.failed sets piece.error"}
  A4 -->|"piece.error"| A5["earlier text yielded, then throws"]
  A4 -->|"output_text.delta"| A6["piece.text yielded as before"]
end

B1 ~~~ A1

class B1,B2,B3,B6 box
class A1,A2,A3,A6 box
class B4,B5 bad
class A4,A5 good

linkStyle 3 stroke:#cc0000,stroke-width:5px
linkStyle 8 stroke:#008800,stroke-width:5px

style BEFORE fill:#fff5f5,stroke:#cc0000,color:#111827
style AFTER fill:#f3fff3,stroke:#008800,color:#111827
Loading

Both completeStream loops read Responses events through readServerEventStream
The two completeStream loops (copilotModels.ts:1284, openai.ts:755) that read these events.

The change

  • The fix, in responsesWireApiProvider.ts: the decoder sets piece.error for error (message, or error.message when nested) and response.failed (response.error.message), and parseResponse returns an error for status failed or cancelled. ResponsesResult gets an error field. This follows the StreamPiece.error pattern from fix(aiclient): propagate messages streaming provider errors #3281 by alexcastgonz (@pcbeingused333), which does the same for the Messages provider. Thank you for that, it made this one easy to line up.
  • The Copilot catch in copilotModels.ts now continues only when decoder.push throws a SyntaxError from JSON.parse, and rethrows anything else. So valid JSON the old catch used to skip doesn't start throwing, both decoders now return an empty piece for a null or non object event, and the chat completions decoder treats a missing choices (a usage only chunk) as no choices.
  • Tests: providerAdapter.spec.ts covers error (flat and nested message) and response.failed, each before and after a text delta, a null or non object event, plus parseResponse for failed and cancelled. copilotTransport.spec.ts streams a non JSON line, a delta, then error or response.incomplete through the Copilot loop, plus a Copilot chat completions stream with a null line and a usage only chunk, and one where a content filter error now surfaces.
  • Checked locally: aiclient build, 660 tests passing, prettier, and code-lint --ratchet (no new violations).

What changes for a user

  • Normal Responses streams and non streaming calls return exactly what they did before.
  • Text that streamed before a failure still reaches the caller first, then the error is thrown, the same order chat completions and fix(aiclient): propagate messages streaming provider errors #3281 use.
  • A failed or cancelled non streaming response is an error now, instead of an empty or partial success.
  • On Copilot endpoints, response.incomplete and content filter errors from the decoder now surface instead of being skipped; lines that are not JSON, null events and usage only chunks are still ignored or read as before.
  • If you'd rather treat cancelled differently, or prefer another shape for any of this, I'm very happy to adjust. Thanks a lot for reviewing!

Fixes #3284

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the Responses API error-propagation fix.

No open issues 🎉

What changed in this PR

Fixes Responses API error propagation in aiclient so failed requests no longer appear as successful partial completions. This addresses #3284 and follows the shared streaming error pattern used for Messages in #3281.

Changes:

  • Surface Responses stream errors and reject failed or cancelled non-streaming responses.
  • Let Copilot decoder errors propagate while preserving handling of non-JSON lines, non-object events, and usage-only chunks.
  • Add regression coverage for error propagation, partial-text delivery, and ignored events.
File Description
ts/​packages/​aiclient/​test/​providerAdapter.spec.ts Tests Responses stream errors, failed statuses, and non-object events.
ts/​packages/​aiclient/​test/​copilotTransport.spec.ts Tests transport error propagation and tolerated stream chunks.
ts/​packages/​aiclient/​src/​wireApiProvider/​responsesWireApiProvider.ts Reports stream failures and failed or cancelled response statuses.
ts/​packages/​aiclient/​src/​wireApiProvider/​chatCompletionsWireApiProvider.ts Handles non-object events and chunks without choices.
ts/​packages/​aiclient/​src/​copilotModels.ts Rethrows decoder errors other than JSON syntax errors.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aiclient: Responses API stream and status errors are returned as successful completions

2 participants