ShellTool: empty ShellResult(output=[]) fabricates stdout "(no output)" in the model-visible shell_call_output and persisted shell_output #5348

Activity

1. BlueX888 commented on Oct 11, 2026

   

   BlueX888

   on Oct 11, 2026

   Author

   More actions

   Reproduced this. Root cause: src/agents/run_internal/tool_actions.py:608 — the truthiness check if shell_output_payload: treats the valid empty payload computed at tool_actions.py:578 as absent, so control falls into the elif output_text: branch (tool_actions.py:610-618), which wraps render_shell_outputs()'s "(no output)" placeholder into the structured channel as if it were executor stdout and persists it at tool_actions.py:632.

   Minimal reproduction:

   python3 - <<'EOF'
   import asyncio
   import json
   
   from openai.types.responses import ResponseFunctionShellToolCall, ResponseOutputMessage, ResponseOutputText
   
   from agents import Agent, Runner
   from agents.testing import ScriptedModel
   from agents.tool import ShellResult, ShellTool
   
   
   def empty_output_executor(request):
       # Executor for a command that succeeded and printed nothing.
       return ShellResult(output=[])
   
   
   def shell_call(call_id="call_shell"):
       return ResponseFunctionShellToolCall(
           id="sc1",
           call_id=call_id,
           type="shell_call",
           action={"type": "shell_command", "commands": ["true"]},
           status="completed",
       )
   
   
   async def main():
       model = ScriptedModel(
           [
               [shell_call()],
               [
                   ResponseOutputMessage(
                       id="m",
                       type="message",
                       role="assistant",
                       content=[
                           ResponseOutputText(text="done", type="output_text", annotations=[], logprobs=[])
                       ],
                       status="completed",
                   )
               ],
           ]
       )
       agent = Agent(name="t", model=model, tools=[ShellTool(executor=empty_output_executor)])
       result = await Runner.run(agent, input="run true", max_turns=5)
   
       for item in result.new_items:
           if type(item).__name__ == "ToolCallOutputItem":
               print("MODEL-VISIBLE structured output:", json.dumps(item.raw_item["output"]))
               print("PERSISTED shell_output:", json.dumps(item.raw_item.get("shell_output")))
       print("final_output:", repr(result.final_output))
   
   
   asyncio.run(main())
   EOF
   

   Observed output:

   MODEL-VISIBLE structured output: [{"stdout": "(no output)", "stderr": "", "outcome": {"type": "exit", "exit_code": 0}}]
   PERSISTED shell_output: [{"stdout": "(no output)", "stderr": "", "status": "completed", "outcome": "success"}]
   final_output: 'done'
   

   Happy to open a PR with this fix, or to be assigned if you prefer to take it.
2. jbeckwith-oai commented on Oct 13, 2026

   

   jbeckwith-oai

   on Oct 13, 2026

   Collaborator

   More actions

   Thanks for the detailed report and for checking the neighboring cases! The code path does promote the display placeholder into structured stdout when ShellResult.output is empty.

   For a command that succeeds without printing anything, you can currently return a ShellCommandOutput with empty stdout/stderr and ShellCallOutcome(type="exit", exit_code=0) inside the result list. That retains the command's outcome without adding placeholder text to its stdout.

   Could you share a little more about the workflow where your executor returns no entries, and whether the placeholder has affected a downstream consumer or a model response? A small, sanitized example would help us understand the practical impact and prioritize a focused fix. Thanks again for taking the time to investigate this.
3. BlueX888 commented on Oct 14, 2026

   

   BlueX888

   on Oct 14, 2026

   Author

   More actions

   Thanks for taking a look, and for confirming the workaround. To answer both questions honestly:

   Where an executor returns no entries. Our evidence is code reading plus the scripted repro in the report — we don't have a production incident to cite, so please treat this as a design sharp edge rather than a live failure we hit. The scenario we're concerned about is user-supplied executors for local environments: ShellExecutor returns str | ShellResult, and nothing requires ShellResult.output to be non-empty, so two natural executor shapes land on output=[] for a successful quiet command (mkdir, cp, git commit -q, …):

   • chunk-per-entry executors that emit one ShellCommandOutput per captured output segment — a command that printed nothing produces zero segments;
   • result-filtering executors that only record commands which printed something or failed — a fully quiet batch returns an empty list.

   Both read as a faithful "the command ran and printed nothing", and the type system gives no hint that this is unsupported.

   Practical impact (sanitized). The fabricated entry is presented as real stdout in two places, and nothing marks the string as a placeholder in either:

   • the model-visible shell_call_output for the next turn ({"stdout": "(no output)", ...}), so the model can reasonably quote it back as the command's output;
   • the persisted raw_item["shell_output"], so a trace or log consumer that parses these entries records stdout the command never produced.

   The plain-text channel carries the same placeholder (ToolCallOutputItem.output), which is fine for human display but is exactly what leaks into the structured channel here.

   The explicit-ShellCommandOutput workaround works for us, and we're happy to leave the fix itself to you. If a focused change is on the table, the seam we'd point at: when the executor returned a structured ShellResult, an empty output list means "structured result: nothing to report", so skipping the elif output_text: fallback in that case (keeping it for plain str returns) would stop the fabrication without changing what render_shell_outputs() shows humans. Happy to send that as a PR if you'd like — otherwise we'll defer to whatever you prefer.
4. zhgao commented on Oct 17, 2026

   

   zhgao

   on Oct 17, 2026

   More actions

   Hi, I've implemented a fix for this issue and it's ready on my fork: zhgao/openai-agents-python branch fix/issue-5348.

   Compare view: main...zhgao:fix/issue-5348

   The branch has two commits ("Fix ShellTool fabricating stdout for empty structured results") which:

   • In ShellAction.execute (src/agents/run_internal/tool_actions.py), change elif output_text: to elif output_text and shell_output_payload is None: so the "(no output)" placeholder path from render_shell_outputs() only applies when there is genuinely no structured result (plain string or exception). A structured ShellResult(output=[]) now keeps output == [] with no fabricated stdout and no persisted shell_output entry.
   • Add test_shell_tool_empty_structured_result_does_not_fabricate_stdout in tests/test_shell_tool.py, asserting output == [] and no shell_output key for ShellResult(output=[]).

   However, this repository restricts pull request creation to collaborators, so I'm unable to open the PR myself. Could a maintainer please open a PR from the fix/issue-5348 branch on my fork (zhgao/openai-agents-python), or grant me collaborator access so I can submit it directly? I'm happy to address any review feedback. Thank you!
5. EngineeringWithJay commented on Oct 17, 2026

   

   EngineeringWithJay

   on Oct 17, 2026

   More actions

   Complementary offline coverage for #5348: on 9abc9182d06dbd28330af888306031269d57a175 (openai-agents 0.24.0, Python 3.12.13, openai 3.34.0), Runner + ScriptedModel reproduces (no output) as stdout in both the actual second model call and persisted raw_item["shell_output"] when the executor returns ShellResult(output=[]).

   Two fixed-behavior assertions fail; eight checks pass, consistently across two runs. Controls cover empty/nonempty strings, quiet structured entries with unspecified or successful exit outcomes, real stdout/stderr with newlines and ordering, and literal executor-provided (no output) that must not be stripped. The human-readable display placeholder is allowed to remain. A temporary one-line fallback guard made all ten checks pass twice; it was then reverted. No source fix is included, and this is synthetic evidence, not a production incident or live-provider test.

   I see a fix branch has already been offered. If complementary coverage would help, I can share these cases for a collaborator-led test-only PR, or open a test-only PR if permitted under the collaborator-only contribution policy. Otherwise, no duplicate implementation or PR is intended.

   AI disclosure: this repro, test coverage, and summary were prepared with Astra Coder (OpenAI GPT-6 Astra, via OpenCode). The reported checks were executed locally.
6. AbarnaaSree commented on Oct 18, 2026

   

   AbarnaaSree

   on Oct 18, 2026

   More actions

   Hi, I’d like to contribute to the validation of this fix. Since an implementation and complementary test coverage have already been shared, would it be okay if I review the proposed changes and check for any remaining edge cases around empty ShellResult outputs, plain-string returns, and persisted shell_output? I’ll coordinate with the existing contributors to avoid duplicating work and can help strengthen the regression tests if needed.
7. MuneebMalik244535 commented on Oct 18, 2026

   

   MuneebMalik244535

   on Oct 18, 2026

   More actions

   I have investigated this issue and confirmed the root cause in src/agents/run_internal/tool_actions.py.

   Root Cause

   In ShellAction.execute:

   raw_entries: list[dict[str, Any]] | None = None
   if shell_output_payload:
       raw_entries = shell_output_payload
   elif output_text:
       raw_entries = [{"stdout": output_text, ...}]