Skip to content

aiclient: Responses API stream and status errors are returned as successful completions #3284

Open

Description

Hi everyone, hope you're doing well!

When a Responses API stream fails partway through (an error or response.failed event), aiclient hands the caller whatever text arrived so far as if the completion finished normally.

What this code is for

ResponsesWireApiProvider turns OpenAI/Azure Responses API traffic into the shared StreamPiece shape, and the stream loops in openai.ts and copilotModels.ts turn those pieces into the text iterator callers read. Copilot endpoints use this path too when wireApi === "responses".

Current behavior

  1. A streamed request goes to /responses and a few response.output_text.delta events arrive.
  2. The server then sends {"type":"error","message":"..."} or {"type":"response.failed","response":{"status":"failed","error":{"message":"..."}}}.
  3. The decoder only looks at response.incomplete, text deltas and usage, so it returns an empty piece with no error:
    createStreamDecoder(): StreamDecoder {
    return {
    push: (raw: string): StreamPiece => {
    const evt = JSON.parse(raw) as {
    type?: string;
    delta?: string;
    response?: {
    usage?: ResponsesUsage;
    };
    };
    const piece: StreamPiece = {};
    if (evt.type === "response.incomplete") {
    throw new Error(incompleteOutputMessage);
    }
    if (
    evt.type === "response.output_text.delta" &&
    typeof evt.delta === "string"
    ) {
    piece.text = evt.delta;
    }
    const usage = evt.response?.usage;
    if (usage) {
    piece.usage = usageFromInputOutput(
    usage.input_tokens,
    usage.output_tokens,
    usage.input_tokens_details?.cached_tokens,
    );
    }
    return piece;
    },
    finish: () => undefined,
    };
  4. The stream loop only throws on piece.error, and Responses sends no [DONE], so the generator just ends:
    const piece = decoder.push(evt.data);
    // Function-calling may emit multiple yields per SSE event
    // (including undefined argument tails — legacy coercion).
    // Yield texts, then throw piece.error before usage — matches
    // legacy mid-loop yield-then-throw (usage skipped on throw).
    if (piece.texts !== undefined) {
    for (const chunk of piece.texts) {
    fullResponseText += chunk as string;
    yield chunk as string;
    }
    } else if (piece.text) {
    fullResponseText += piece.text;
    yield piece.text;
    }
    if (piece.error) {
    throw piece.error;
    }
  5. The caller gets the partial text as a successful answer.

The non streaming path does the same: parseResponse only rejects status === "incomplete", so a failed or cancelled body with HTTP 200 comes back as success with "" or partial text (and ResponsesResult has no error field):

parseResponse(data: unknown, _request: ModelRequest): Result<string> {
const res = data as ResponsesResult;
if (res.status === "incomplete") {
return error(incompleteOutputMessage);
}
if (typeof res.output_text === "string") {
return success(res.output_text);
}
const parts: string[] = [];
for (const item of res.output ?? []) {
for (const c of item.content ?? []) {
if (c.type === "output_text" && typeof c.text === "string") {
parts.push(c.text);
}
}
}
return success(parts.join(""));
}

On the Copilot path, the try/catch { continue } around decoder.push also swallows errors the decoder does throw today, like the response.incomplete one:

let piece;
try {
piece = decoder.push(evt.data);
} catch {
// Ignore non-JSON keep-alive/comment lines.
continue;
}

Expected behavior

The text that already streamed still reaches the caller, and then the iterator throws with the server's message. A failed or cancelled non streaming response returns an error Result.

Minimal repro

import { adapterFor } from "./src/wireApiProvider/index.js";

const decoder = adapterFor("responses").createStreamDecoder!({} as any);
decoder.push(JSON.stringify({ type: "response.output_text.delta", delta: "Partial" }));
const piece = decoder.push(
    JSON.stringify({ type: "error", code: "server_error", message: "Boom" }),
);
console.log(piece.error); // undefined, so the loop ends quietly with "Partial"

const r = adapterFor("responses").parseResponse(
    { status: "failed", error: { message: "Boom" }, output_text: "" },
    {} as any,
);
console.log(r.success); // true
flowchart LR
    A["SSE: output_text.delta"] --> B["decoder.push"]
    E["SSE: error / response.failed"] --> B
    B -->|"piece.text"| C["stream loop yields text"]
    B -->|"empty piece, no error"| D["no DONE, generator ends"]
    D --> F["caller sees partial text as success"]
Loading

Both completeStream loops read Responses events through readServerEventStream
The two completeStream loops (copilotModels.ts:1284, openai.ts:755) that read these events.

I noticed #3281 by alexcastgonz (@pcbeingused333) handles this same kind of thing really nicely for the Messages provider by setting StreamPiece.error, so yielded text still flows before the throw. Would you be open to the same approach here, setting piece.error for error and response.failed, returning an error from parseResponse for failed/cancelled, and letting the Copilot loop skip only lines that aren't JSON? Or is there a different shape you'd prefer? I'm happy to put up a small PR with that approach so there's something concrete to look at, and I'll gladly change it to whatever fits best. Thanks so much for taking a look!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions