Repository navigation
fix(aiclient): surface Responses API stream and status errors - #3285
Open
Abhiram Shaji (abhiramshaji-cyber) wants to merge 3 commits into
Open
Abhiram Shaji (abhiramshaji-cyber) wants to merge 3 commits into
Abhiram Shaji (abhiramshaji-cyber) wants to merge 3 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified for the Responses API error-propagation fix.
No open issues 🎉
What changed in this PR
Fixes Responses API error propagation in aiclient so failed requests no longer appear as successful partial completions. This addresses #3284 and follows the shared streaming error pattern used for Messages in #3281.
Changes:
- Surface Responses stream errors and reject failed or cancelled non-streaming responses.
- Let Copilot decoder errors propagate while preserving handling of non-JSON lines, non-object events, and usage-only chunks.
- Add regression coverage for error propagation, partial-text delivery, and ignored events.
| File | Description |
|---|---|
ts/packages/aiclient/test/providerAdapter.spec.ts |
Tests Responses stream errors, failed statuses, and non-object events. |
ts/packages/aiclient/test/copilotTransport.spec.ts |
Tests transport error propagation and tolerated stream chunks. |
ts/packages/aiclient/src/wireApiProvider/responsesWireApiProvider.ts |
Reports stream failures and failed or cancelled response statuses. |
ts/packages/aiclient/src/wireApiProvider/chatCompletionsWireApiProvider.ts |
Handles non-object events and chunks without choices. |
ts/packages/aiclient/src/copilotModels.ts |
Rethrows decoder errors other than JSON syntax errors. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a Responses API request fails partway through, callers got the half finished text back as a normal answer, and now they get the text so far followed by the server's error.
What this code is for
ResponsesWireApiProviderints/packages/aiclientturns Responses API events intoStreamPieceobjects. ThecompleteStreamloops inopenai.tsandcopilotModels.tsread events withreadServerEventStream, calldecoder.pushon each one, yieldpiece.text, and throwpiece.error.parseResponsedoes the same job for non streaming calls.Before this PR
An app streams a completion from a Responses endpoint (Azure/OpenAI, or Copilot with
wireApi === "responses").response.output_text.deltaevents arrive anddecoder.pushreturns them aspiece.text, which the loop yields.{"type":"error","message":"..."}orresponse.failedwithresponse.error.message.decoder.pushonly checksresponse.incomplete, deltas and usage, so it returns{}.piece.errorto throw, Responses sends no[DONE], so the generator just ends.Without streaming, a body with
status: "failed"or"cancelled"and HTTP 200 went throughparseResponseassuccess("")or partial text. On the Copilot loop,try { decoder.push } catch { continue }also dropped the errordecoder.pushalready throws forresponse.incomplete.After this PR
Same app, same request, same as above through step 2.
decoder.pushseeserrororresponse.failedand returns{ error: new Error(<server message>) }, readingmessageorerror.messageonerrorevents.piece.error.Without streaming,
parseResponsereturnserror(<server message>)forfailedandcancelled. On the Copilot loop, a line that is not JSON is still skipped, and any other error fromdecoder.pushis thrown.flowchart LR classDef box fill:#eef1ff,stroke:#4c5bd4,color:#111827 classDef bad fill:#ffd6d6,stroke:#cc0000,color:#111827,stroke-width:3px classDef good fill:#c8f0c8,stroke:#008800,color:#111827,stroke-width:3px subgraph BEFORE["BEFORE: partial text returned as success"] direction TB B1["app calls completeStream"] --> B2["readServerEventStream"] B2 --> B3["decoder.push"] B3 --> B4{"error or response.failed?"} B4 -->|"returns empty piece"| B5["loop ends, partial text looks finished"] B4 -->|"output_text.delta"| B6["piece.text yielded"] end subgraph AFTER["AFTER: partial text, then the server error"] direction TB A1["app calls completeStream"] --> A2["readServerEventStream"] A2 --> A3["decoder.push"] A3 --> A4{"THE CHANGE: error or response.failed sets piece.error"} A4 -->|"piece.error"| A5["earlier text yielded, then throws"] A4 -->|"output_text.delta"| A6["piece.text yielded as before"] end B1 ~~~ A1 class B1,B2,B3,B6 box class A1,A2,A3,A6 box class B4,B5 bad class A4,A5 good linkStyle 3 stroke:#cc0000,stroke-width:5px linkStyle 8 stroke:#008800,stroke-width:5px style BEFORE fill:#fff5f5,stroke:#cc0000,color:#111827 style AFTER fill:#f3fff3,stroke:#008800,color:#111827The two
completeStreamloops (copilotModels.ts:1284, openai.ts:755) that read these events.The change
responsesWireApiProvider.ts: the decoder setspiece.errorforerror(message, orerror.messagewhen nested) andresponse.failed(response.error.message), andparseResponsereturns an error forstatusfailedorcancelled.ResponsesResultgets anerrorfield. This follows theStreamPiece.errorpattern from fix(aiclient): propagate messages streaming provider errors #3281 by alexcastgonz (@pcbeingused333), which does the same for the Messages provider. Thank you for that, it made this one easy to line up.copilotModels.tsnow continues only whendecoder.pushthrows aSyntaxErrorfromJSON.parse, and rethrows anything else. So valid JSON the old catch used to skip doesn't start throwing, both decoders now return an empty piece for anullor non object event, and the chat completions decoder treats a missingchoices(a usage only chunk) as no choices.providerAdapter.spec.tscoverserror(flat and nested message) andresponse.failed, each before and after a text delta, anullor non object event, plusparseResponseforfailedandcancelled.copilotTransport.spec.tsstreams a non JSON line, a delta, thenerrororresponse.incompletethrough the Copilot loop, plus a Copilot chat completions stream with anullline and a usage only chunk, and one where a content filter error now surfaces.aiclientbuild, 660 tests passing, prettier, andcode-lint --ratchet(no new violations).What changes for a user
response.incompleteand content filter errors from the decoder now surface instead of being skipped; lines that are not JSON,nullevents and usage only chunks are still ignored or read as before.cancelleddifferently, or prefer another shape for any of this, I'm very happy to adjust. Thanks a lot for reviewing!Fixes #3284