Repository navigation
Description
Problem
HAR tool-call extraction tolerates malformed input by skipping it, but does not report which entries were rejected. Consumers cannot distinguish a complete extraction from a partial one.
PR #1488 demonstrates the gap: a non-object entry in a Chat Completions response's choices array is skipped so that valid calls can still be extracted. This prevents a crash, but produces no warning or rejected-entry record. Other extraction paths also skip malformed payloads.
We already preserve invalid JSON arguments on recognizable calls as arguments: { _raw: originalString }. That fallback does not capture malformed choices or call structures that cannot be represented as a ToolCall. The original HAR remains available, but investigating omissions requires manually inspecting it.
Desired behavior
Capture malformed tool-call-related entries as structured extraction diagnostics, without aborting extraction of valid calls or fabricating valid ToolCall objects.
- Record the rejection reason and source location (HAR entry index, relevant choice/call/event index or JSON path, and transport/format where identifiable).
- Preserve the rejected payload in a diagnostic artifact, or retain a reliable reference to it in the source HAR. Apply existing artifact access controls and sanitization; do not dump response bodies, arguments, or secrets into ordinary logs.
- Expose and persist diagnostics alongside extraction results so downstream consumers can distinguish complete, partial, and failed extraction. Surface a concise warning/count when data is rejected.
- Preserve the existing
_rawargument fallback and successful tool-call extraction behavior. - Do not classify unrelated HTTP traffic, a legitimately empty response, or a valid response without tool calls as malformed.
Acceptance criteria
- A response with
choices: [null, validChoice]still yields the valid call and records a diagnostic for the null choice. - Malformed tool-call structures and malformed payloads in supported extraction paths produce actionable diagnostics rather than disappearing silently.
- A valid call with invalid JSON arguments retains its raw arguments under the existing
_rawbehavior. - Valid responses without tool calls do not produce spurious malformed-call diagnostics.
- Diagnostics remain accessible after extraction and identify the source data without leaking sensitive payloads into logs.
- Focused regression tests cover mixed valid/malformed input and diagnostic persistence.
Related
- fix(shared/har): guard against a null entry in the choices array #1488 — fixes the null-choice crash; this issue tracks observability and capture of rejected data.
- HAR tool-call extraction misses OpenAI Responses API (
function_call/custom_tool_call) → gpt-5.x ACP-copilot runs get an empty tool-call history, silently failing tool-output criteria #1250 — addressed unsupported Responses API extraction and proposed zero-extraction warnings; per-entry malformed-data diagnostics are a separate gap.
Relevant code
packages/shared/src/har/har-parser.ts:extractToolCalls,processChoices, streaming and Anthropic extraction paths.packages/shared/src/har/parsed-body.ts: body/SSE parsing currently discards malformed payloads.packages/shared/src/har/responses-api-parser.ts: Responses API extraction.packages/shared/src/har/types.ts:ToolCallcurrently has no associated extraction-diagnostics result.
Activity