Repository navigation
Description
Hi everyone, hope you're doing well!
When a Responses API stream fails partway through (an error or response.failed event), aiclient hands the caller whatever text arrived so far as if the completion finished normally.
What this code is for
ResponsesWireApiProvider turns OpenAI/Azure Responses API traffic into the shared StreamPiece shape, and the stream loops in openai.ts and copilotModels.ts turn those pieces into the text iterator callers read. Copilot endpoints use this path too when wireApi === "responses".
Current behavior
- A streamed request goes to
/responsesand a fewresponse.output_text.deltaevents arrive. - The server then sends
{"type":"error","message":"..."}or{"type":"response.failed","response":{"status":"failed","error":{"message":"..."}}}. - The decoder only looks at
response.incomplete, text deltas and usage, so it returns an empty piece with noerror:
TypeAgent/ts/packages/aiclient/src/wireApiProvider/responsesWireApiProvider.ts
Lines 128 to 159 in 9d0c0c6
createStreamDecoder(): StreamDecoder { return { push: (raw: string): StreamPiece => { const evt = JSON.parse(raw) as { type?: string; delta?: string; response?: { usage?: ResponsesUsage; }; }; const piece: StreamPiece = {}; if (evt.type === "response.incomplete") { throw new Error(incompleteOutputMessage); } if ( evt.type === "response.output_text.delta" && typeof evt.delta === "string" ) { piece.text = evt.delta; } const usage = evt.response?.usage; if (usage) { piece.usage = usageFromInputOutput( usage.input_tokens, usage.output_tokens, usage.input_tokens_details?.cached_tokens, ); } return piece; }, finish: () => undefined, }; - The stream loop only throws on
piece.error, and Responses sends no[DONE], so the generator just ends:
TypeAgent/ts/packages/aiclient/src/openai.ts
Lines 830 to 846 in 9d0c0c6
const piece = decoder.push(evt.data); // Function-calling may emit multiple yields per SSE event // (including undefined argument tails — legacy coercion). // Yield texts, then throw piece.error before usage — matches // legacy mid-loop yield-then-throw (usage skipped on throw). if (piece.texts !== undefined) { for (const chunk of piece.texts) { fullResponseText += chunk as string; yield chunk as string; } } else if (piece.text) { fullResponseText += piece.text; yield piece.text; } if (piece.error) { throw piece.error; } - The caller gets the partial text as a successful answer.
The non streaming path does the same: parseResponse only rejects status === "incomplete", so a failed or cancelled body with HTTP 200 comes back as success with "" or partial text (and ResponsesResult has no error field):
TypeAgent/ts/packages/aiclient/src/wireApiProvider/responsesWireApiProvider.ts
Lines 100 to 117 in 9d0c0c6
| parseResponse(data: unknown, _request: ModelRequest): Result<string> { | |
| const res = data as ResponsesResult; | |
| if (res.status === "incomplete") { | |
| return error(incompleteOutputMessage); | |
| } | |
| if (typeof res.output_text === "string") { | |
| return success(res.output_text); | |
| } | |
| const parts: string[] = []; | |
| for (const item of res.output ?? []) { | |
| for (const c of item.content ?? []) { | |
| if (c.type === "output_text" && typeof c.text === "string") { | |
| parts.push(c.text); | |
| } | |
| } | |
| } | |
| return success(parts.join("")); | |
| } |
On the Copilot path, the try/catch { continue } around decoder.push also swallows errors the decoder does throw today, like the response.incomplete one:
TypeAgent/ts/packages/aiclient/src/copilotModels.ts
Lines 1357 to 1363 in 9d0c0c6
| let piece; | |
| try { | |
| piece = decoder.push(evt.data); | |
| } catch { | |
| // Ignore non-JSON keep-alive/comment lines. | |
| continue; | |
| } |
Expected behavior
The text that already streamed still reaches the caller, and then the iterator throws with the server's message. A failed or cancelled non streaming response returns an error Result.
Minimal repro
import { adapterFor } from "./src/wireApiProvider/index.js";
const decoder = adapterFor("responses").createStreamDecoder!({} as any);
decoder.push(JSON.stringify({ type: "response.output_text.delta", delta: "Partial" }));
const piece = decoder.push(
JSON.stringify({ type: "error", code: "server_error", message: "Boom" }),
);
console.log(piece.error); // undefined, so the loop ends quietly with "Partial"
const r = adapterFor("responses").parseResponse(
{ status: "failed", error: { message: "Boom" }, output_text: "" },
{} as any,
);
console.log(r.success); // trueflowchart LR
A["SSE: output_text.delta"] --> B["decoder.push"]
E["SSE: error / response.failed"] --> B
B -->|"piece.text"| C["stream loop yields text"]
B -->|"empty piece, no error"| D["no DONE, generator ends"]
D --> F["caller sees partial text as success"]

The two completeStream loops (copilotModels.ts:1284, openai.ts:755) that read these events.
I noticed #3281 by alexcastgonz (@pcbeingused333) handles this same kind of thing really nicely for the Messages provider by setting StreamPiece.error, so yielded text still flows before the throw. Would you be open to the same approach here, setting piece.error for error and response.failed, returning an error from parseResponse for failed/cancelled, and letting the Copilot loop skip only lines that aren't JSON? Or is there a different shape you'd prefer? I'm happy to put up a small PR with that approach so there's something concrete to look at, and I'll gladly change it to whatever fits best. Thanks so much for taking a look!
Activity