[Roydon Sequeira (roydonsequeira)]

Description

Fixes #3668.

With fail_on_missing_function=False, _execute_call_section_async returns a malformed_arguments function output for arguments that are not valid JSON or not a JSON object, so the model can correct the call. Arguments that are a JSON object but don't match a FunctionTool's signature (a wrong type, a missing required argument or an unexpected key) were only checked inside FunctionTool.execute_async, and the ValidationError failed the whole send.

Tool now has a validate_arguments method, which _execute_call_section_async calls before execute_async. It accepts any arguments by default, since MCP servers report input errors in the tool result, and FunctionTool implements it with its argument model. When it raises, the dispatcher returns the same malformed_arguments output with invoked=False, or re-raises when fail_on_missing_function=True, which is what that mode already did. Only the validation call is guarded, so an exception raised by the function itself is not reported as an argument error. MessageToolCallScorer keeps counting these calls as not executed.

Tests and Documentation

Added tests in tests/unit/prompt_target/target/test_tool.py:

• FunctionTool.validate_arguments raises ValidationError for a wrong type, a missing required argument and an unexpected key.
• Through send_prompt_async, the same three cases reach the model as a malformed_arguments output in the next request. The function is never called, the output is recorded with invoked=False, and the stored history still validates for replay. With fail_on_missing_function=True the send raises ValidationError after one request.
• A ValueError raised by the tool function itself still propagates instead of being returned as malformed_arguments.

No docs or notebooks changed, so JupyText was not needed.

[Roydon Sequeira (roydonsequeira)]

@microsoft-github-policy-service agree

[Copilot]

🟢 Approval recommended

No active review issues are reported for the change that makes invalid tool arguments recoverable.

No open issues 🎉

What changed in this PR

Addresses #3668 by letting the OpenAI Responses tool loop return recoverable errors for arguments that do not match a FunctionTool signature, rather than failing the entire send.

Changes:

• Adds a default argument-validation hook to Tool and schema validation to FunctionTool.
• Returns malformed_arguments with invoked=False for rejected arguments, while preserving strict-mode failures and exceptions raised during execution.
• Adds regression tests for invalid arguments, execution metadata, replay validation, and function-error propagation.

File	Description
tests/​unit/​prompt_target/​target/​test_tool.py	Tests argument rejection, recoverable tool errors, and exception propagation.
pyrit/​prompt_target/​openai/​openai_response_target.py	Validates tool arguments before execution and handles rejected calls.
pyrit/​prompt_target/​common/​tool_provider.py	Adds the validation hook and implements function-signature validation.

_{Review effort: Balanced}

[hannahwestra25]

thanks for contributing!

[Roydon Sequeira (roydonsequeira)]

The failing checks aren't from this change. main-job (ubuntu-latest, 3.11, pyrit, dev_all) had one failure, test_xss_javascript_uri_near_miss_does_not_backtrack (1.140s against its 0.5s limit). That timing test came in with #3640 and has failed the same way on main (job). The other three red jobs were cancelled by fail-fast. Could you re-run the failed jobs?