Skip to content

bug: diagnose whitespace-only LLM responses from third-party relays #815

Description

@ZhulongNT

Summary

GenericAgent can receive a syntactically valid but semantically empty completion: for example, the response payload contains one text block whose value is a single whitespace character (" "). The TUI then shows:

· LLM returned an empty response. Retrying...

The retry succeeds in the observed case, so this is recoverable, but it is confusing to diagnose.

Observation

  • Session trace: model_responses_858881.txt, 2026-09-21 16:27:46.
  • The response was represented as a Python object equivalent to:
    [{"type": "text", "text": " "}]
  • There was no meaningful content or thinking; GA correctly treated it as blank and regenerated.
  • The relevant guard is ga.py / do_no_tool() (currently around lines 459–467). It emits [Warn] LLM returned an empty response. Retrying... and routes through _retry_or_exit().
  • The TUI replaces [Warn] with ·, which makes the displayed line look less like an actionable warning.

Important context: third-party relays / gateways

This has been observed when using a non-official API relay/gateway rather than the model provider's official endpoint. The relay may return an HTTP-successful response while dropping, truncating, or normalizing streamed content to whitespace. This is an observation, not a claim that every relay is faulty; the same symptom could also originate upstream. However, intermediaries make the failure mode substantially harder to attribute because the request technically succeeds.

There is a related lower-level path in llmcore.py where a stream produces no output and is retried as ConnectionError("empty response"), but the whitespace-only case reaches the agent-level blank-response guard instead.

Why this deserves a separate issue

#201 added the useful retry limit (retry up to three times, then stop). It does not expose enough context to distinguish:

  1. a provider-side empty completion,
  2. a relay/gateway that returned a malformed or whitespace-only successful response, and
  3. a stream that ended before producing any content.

Suggested improvements

  1. Keep the current safe retry behavior.
  2. Log a compact, redacted diagnostic for blank responses: session/model name, configured base URL host (no credentials), response block types, text lengths / whitespace-only flag, whether any thinking was present, and retry count.
  3. Make the TUI warning retain a visible severity label, e.g. · [Warn] Empty LLM response; retrying (1/3).
  4. Optionally classify whitespace_only, no_stream_output, and empty_payload separately in the trace/logs.
  5. Documentation: recommend reproducing against the official endpoint before filing provider-specific issues when a third-party relay is in the path.

This should make relay-related incidents diagnosable without weakening compatibility with OpenAI-compatible gateways.

Activity

  1. louisss1016 commented on Oct 1, 2026

    @louisss1016

    I'd like to take this one. Plan, following the issue's suggested improvements:

    1. Diagnostic at the guard — in ga.py do_no_tool(), keep the safe retry path unchanged, and emit a compact redacted diagnostic on blank responses: classification (whitespace_only vs empty_payload), response block types, content/thinking length and whitespace flag, retry count, model name, and the gateway host only (no path, no credentials).
    2. TUI severity label — in frontends/tui_v3.py, stop stripping [Warn]/[Warning]/[Error] into a bare · ; render them as · [Warn] ... so the severity survives.
    3. Tests — pytest coverage in frontends/tests/ for the classifier and the TUI prefix rule.

    The retry behavior itself (up to 3, then stop) stays exactly as it is — this is diagnosability only, no compatibility change for OpenAI-compatible gateways.

  2. louisss1016 commented on Oct 1, 2026

    @louisss1016

    PR up: #819

    Summary of the implementation:

    • ga.py gains describe_blank_response() (kind / sizes / block types / model / gateway host — hostname only, never credentials) and _url_host(); do_no_tool()'s retry semantics are untouched (3 attempts then exit), only the warning line gains the attempt counter + diagnostic.
    • tui_v3.py splits _ACTION_RE so [Warn]/[Warning]/[Error] now render as · [Warn] … instead of a neutral bullet.
    • 15 new tests via the repo's usual ast/exec extraction pattern; full suite 287 passed (the 3 failures are pre-existing Windows-env issues: symlink perms ×2 and the release-qualification test that needs a built package).

    Left out on purpose: the llmcore.py no_stream_output path (already a distinct ConnectionError), and suggestion #5's docs note — that one feels like a maintainer call, happy to add it if you want it in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions