Repository navigation
fix(mcp): don't read the advertised schema on FastMCP 2.x - #1203
Conversation
What: _listed_input_schema now falls back to FastMCP 2.x's _list_tools_middleware when the server has no list_tools. Why: 2.x has no list_tools, so the schema lookup raised, was logged and dropped, and every argument looked undeclared. $mcp_input_keys came back as ['[redacted]'] on 2.x servers with middleware. Tested: pytest posthog/test/mcp on fastmcp 2.14.7 (661 passed, 14 skipped) and 3.2.0 (665 passed, 10 skipped); ruff format/check and mypy-baseline clean.
There was a problem hiding this comment.
PostHog Review · Standard · reviewed 7689e2d: 1 Should fix.
See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.
posthog-python Compliance ReportDate: 2026-11-06T21:37:46.070885+00:00 ✅ All Tests Passed!121/121 tests passed Capture_V1 Tests✅ 95/95 tests passed View Details
Capture_Ai Tests✅ 5/5 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
Feature_Flags_Local_Evaluation Tests✅ 4/4 tests passed View Details
|
There was a problem hiding this comment.
Not approved — this change needs a human reviewer.
Re-add the stamphog label to request another review once you have addressed this.
The current-head review has an unresolved substantive middleware concern and therefore provides no independent assurance. Address @PostHog's comment in posthog/mcp/_instrument_lowlevel.py by avoiding additional request-middleware execution during schema lookup, then request review from Georgis Andonis or another teammate before re-requesting.
- Author wrote 0% of the modified lines and has 5 merged PRs in these paths (familiarity MODERATE).
- PostHog Review (Standard) reviewed the current head: 1 finding, posted on the PR.
- posthog[bot] reviewed the current head.
- @PostHog, posthog/mcp/_instrument_lowlevel.py: the concern about consuming request quota during schema lookup remains unaddressed. _standalone_ownership calls _listed_input_schema before dispatch, and the new fallback explicitly invokes the middleware-backed listing without preventing middleware side effects. The specific rate-limit reproduction cannot be independently run with the available tools, but the current code retains the additional middleware invocation identified by the reviewer.
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 8L, 1F substantive, 13L/2F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1b-small (13L, 2F, two-areas, fix) |
| stamphog 2.7.0 | .stamphog/policy.yml @ 7689e2d · reviewed head 7689e2d |
SDK performance: posthog-python2 changes worth reviewing: 2 observed improvements. Flags require at least a 5% change in the median and all six pairs agreeing on direction. This is a noise filter, not proof of a performance change or a CI gate.
View all results
Values are medians of paired measurements. Not flagged does not mean unchanged. Unavailable means a measurement could not be provided, not zero. Estimated allocation measurements retain their instrumentation uncertainty. RSS measures the whole process, not just SDK allocations; observed peaks are sampled checkpoints. Paired ranges in the diagnostics describe variability, not confidence intervals. Each shard uses its own runner; compare Before PR against With PR within a row, not across shards. Compared 102 metric rows; 102 rows shown in all results. Omitted or shortened data remain in |
Review: the 2.x fallback ran _list_tools_middleware, so a request middleware such as SlidingWindowRateLimitingMiddleware spent a quota slot on the schema lookup and could fail a call that succeeded before. 2.x now reads get_tools(), which runs no middleware; 3.x keeps list_tools(). Tested: new test_schema_lookup_on_fastmcp_2_runs_no_request_middleware fails without the fix and passes with it; pytest posthog/test/mcp on fastmcp 2.14.7 (661 passed; the 1 failure is test_stdio_handshake_then_calls, which fails the same way locally without this change) and fastmcp 3.2.0 (665 passed); ruff clean.
|
No review started. stamphog started a review of this pull request less than 10 minutes ago, and a label can request at most one review per 10 minutes. The |
There was a problem hiding this comment.
Not approved — escalated to a human reviewer.
Re-add the stamphog label to request another review once you have addressed this.
The change affects privacy-sensitive analytics capture without current-head independent assurance; the older bot review provides no specific concern that can be checked. Request review from Georgis Andonis, a teammate, or an agent reviewer covering the registry fallback and redaction behavior before re-requesting.
- Author wrote 0% of the modified lines and has 5 merged PRs in these paths (familiarity MODERATE).
- Risky territory without independent assurance: Security-sensitive surface: posthog/mcp/_instrument_lowlevel.py:715 now trusts registry schemas when the public listing method is unavailable; these schemas determine which argument names may bypass analytics redaction in posthog/mcp/_tool_input.py.; Data ingestion or write paths: posthog/mcp/_instrument_lowlevel.py:709-716 changes the schema used to populate captured tool-input analytics, replacing previously redacted names with registry-declared names.
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 11L, 1F substantive, 40L/3F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1b-small (40L, 3F, two-areas, fix) |
| stamphog 2.7.0 | .stamphog/policy.yml @ 9b9eed4 · reviewed head 9b9eed4 |
What: removed the public PostHogMCP.capture_unknown_tool() and the public PostHogMCPAnalyticsEvent.UNKNOWN_TOOL constant; the event name now lives in the private event-name map. The registry helpers stay in _instrument_lowlevel next to _dispatch_can_differ instead of moving to _instrumentation. Dropped the slow-listing and hidden-tool test cases (they guarded a listing probe this branch no longer has and pass on main), the lowlevel test fixture edits, and README text that documented #1204's annotations. The manual-session test moved to test_posthog_mcp.py. Why: stamphog refused #1199 for a public API change and its size. The spec makes manual capture methods optional, so capture_unknown_tool can return in its own PR. Tested with: pytest posthog/test/mcp on .venv, v2, f2, v2f4 (689, 526, 681, 570 passed; f2's one failure is test_jlowin_middleware_defaults_work_across_fresh_instances, failing on main until #1203). ruff format/check, mypy-baseline filter clean. make public_api_check: snapshot up to date, identical to origin/main.
|
@PostHog review |
|
@lucasheriques PostHog Review is reviewing this pull request. |
Review on the previous commit: reading the registry with get_tools() can disagree with what middleware advertises (a replaced tool, a prefixed mount sharing a root name), so trusting it records undeclared argument names. Running the middleware listing instead spends rate limits. FastMCP 2.x has no safe source for the advertised schema, so the lookup now returns None there, which keeps argument names redacted, runs no middleware, and stops the error logged on every call. The 2.x middleware test now expects the redaction. Tested: pytest posthog/test/mcp on fastmcp 2.14.7 (661 passed; the only failure is test_stdio_handshake_then_calls, which fails locally without this change) and fastmcp 3.2.0 (665 passed); ruff clean.
There was a problem hiding this comment.
Not approved — escalated to a human reviewer.
Re-add the stamphog label to request another review once you have addressed this.
The change touches privacy-sensitive analytics filtering without current-head independent assurance; the older bot summaries provide no specific concern to verify. Request review from Georgis Andonis, another teammate, or an agent reviewer before re-requesting.
- Author wrote 0% of the modified lines and has 5 merged PRs in these paths (familiarity MODERATE).
- Risky territory without independent assurance: Security-sensitive surface: posthog/mcp/_instrument_lowlevel.py:709–715 changes schema discovery used to decide which tool argument names may appear in analytics; returning no schema preserves conservative redaction when the listing method is absent.
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 10L, 1F substantive, 44L/3F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1b-small (44L, 3F, two-areas, fix) |
| stamphog 2.7.0 | .stamphog/policy.yml @ 4f115d9 · reviewed head 4f115d9 |
|
@PostHog review |
|
@lucasheriques PostHog Review is reviewing this pull request. |
* feat(mcp): align posthog.mcp with the sdk-specs MCP spec before v1 What: - get_more_tools and send_feedback annotate readOnlyHint, destructiveHint, idempotentHint and openWorldHint as false, with no title, so the spec's exact annotations JSON matches. Hosts cannot override them. - The injected `context` argument uses the spec's shared description string. - `llm_model` is never added to a schema's `required` list. - A tools/call naming a tool the server never registered sends $mcp_unknown_tool (tool name, identity, session) instead of $mcp_tool_call and $exception. The client gets the same response. A disabled tool, invalid arguments, a raw server with no tools/list handler or a name missing from a paginated first page, and standalone FastMCP with dispatch-overriding middleware stay failed tool calls. PostHogMCP.capture_unknown_tool() covers custom dispatchers. - Every adapter's raise path and isError-result path calls one ToolCallLifecycle.record_if_unknown_tool(). The standalone FastMCP registry lookup and its middleware check moved to _instrumentation.py, so _instrument_v2 no longer imports from _instrument_lowlevel. - Minor sampo changeset. Why: posthog.mcp is about to be declared stable. Customers' insights over $mcp_* events must not shift after that, so every SDK matches the spec now. Tested: - pytest posthog/test/mcp: 678 passed, 10 skipped - annotations test compares the exact JSON; fails 2/4 with a title present - ruff format --check . and ruff check .: clean - mypy --no-site-packages --config-file mypy.ini . | mypy-baseline filter: clean - python -W error -c "import posthog": ok; make public_api_check: up to date - full pytest without provider API keys: 6142 passed, 3 failed in test_metrics_autocapture.py fork tests (macOS objc fork-safety abort, unrelated to this change) Reviewer notes: _standalone_tool_exists catches the cached fastmcp not-found errors, which include DisabledError; get_tool raises only NotFoundError in fastmcp 2.14 and 3.2, so a disabled tool still reads as existing. Client conversation ids are untouched (posthog-python#1193). * fix(mcp): close unknown-tool gaps found in review of the spec alignment What: - A disabled tool on a mounted FastMCP 3+ server stays a failed $mcp_tool_call. A server serving other providers (mounts, proxies) can't prove a name absent, so existence answers "can't tell". - FastMCP's stock middleware (logging, timing, error handling, retry, rate limiting, ping, auth, caching, response limiting) no longer hides the registry. Tool-injection middleware and app middleware still do. - A call naming no tool emits nothing, automatic or manual. - Truncated strings fit their limit with the "..." marker included, so $mcp_tool_name stops at 256 characters (the spec's Property reference). - first_listing_page reads nextCursor only when next_cursor is absent, so fastmcp 4 on MCP 2 logs no deprecation warning. - A raw server's unknown-tool error result carries the conversation handle again, and $mcp_unknown_tool carries it once delivered, as origin/main did and as posthog-js does. - Unknown-tool tests skip only the FastMCP.disable cases on fastmcp 2. - Changeset names the listing probe and hidden-tool upgrade differences. Why: posthog.mcp must match the sdk-specs MCP spec before it is declared stable, so customers' insights don't shift afterwards. Tested: - pytest posthog/test/mcp (mcp 1.30 + fastmcp 3.2): 694 passed, 11 skipped - test_unknown_tool.py on mcp 1.30 + fastmcp 2.14.7: 21 passed, 4 skipped; mcp 2.3: 15 passed, 10 skipped; mcp 2.3 + fastmcp 4.1.6: 25 passed - ruff format --check, ruff check, mypy | mypy-baseline filter, make public_api_check: clean Reviewer notes: posthog-js truncates with the marker past the limit too and needs the same fix. test_fastmcp_v2.py:: test_jlowin_middleware_defaults_work_across_fresh_instances fails on fastmcp 2.14.7 on origin/main as well ($mcp_input_keys is ["[redacted]"]); not touched here. * fix(mcp): bound the listing probe and give manual capture a session What: raw_listing_owns_tool_name waits at most LISTING_PROBE_DEADLINE_SECONDS (1s) for the host's tools/list and treats a timeout as None (can't tell), so a failed call records a failed $mcp_tool_call and the client gets its error promptly. PostHogMCP._base_event mints a fresh ses_<UUIDv7> when no session_id is passed, so capture_unknown_tool (and capture_tool_call and the other manual captures) no longer send no $session_id and an anonymous distinct_id. Why: PostHog Review on #1199. A stalled tools/list handler held every failed call forever; the spec's Session identity and Manual capture require a non-empty $session_id, fresh per call when nothing groups it. Tested: pytest posthog/test/mcp (697 passed, 11 skipped); ruff format/check, mypy | mypy-baseline filter, make public_api_check all clean. New tests failed first: stalled listing (timed out), manual capture without session (KeyError). Reviewer: a manual capture passing conversation_id but no session_id still gets a fresh session, not one derived from the handle. * fix(mcp): make the stalled-listing test stall on mcp 2.x What: `_raw(stall_listing=True)` now stalls every tools/list on mcp 2.x; on 1.x it still spares the server's own cache refresh. Why: the v2 adapter probes the host listing as `original(ctx, None)`, and the helper stalled only when a request was present, so the probe was answered at once, the call was recorded as `$mcp_unknown_tool`, and the test found no `$mcp_tool_call` (CI: MCP SDK v2 and v2-bare, Python 3.10). The product code was right: the deadline path falls back to a failed call on mcp 2. Tested: `pytest posthog/test/mcp` on mcp 1.30 + fastmcp 3.2 (697 passed), mcp 2.3 (536), mcp 2.3 + fastmcp 4.1.6 (579), and Python 3.10 with mcp 2.x bare (536) and with fastmcp 4 (579). Raising the deadline to 999s makes the test fail on mcp 2. ruff format/check and mypy baseline clean. * fix(mcp): stop treating a tools/list listing as proof a tool is unknown A call is now unknown only on authoritative evidence: jlowin FastMCP's registry when nothing overrides dispatch, or the official high-level server's registry. A raw low-level server's own dispatcher decides which names exist and its listing may hide tools it serves, so there the SDK answers "can't tell" and records a failed $mcp_tool_call (spec, "Tool call event"). This removes the failure-path listing probe, its one-second deadline, the pagination flag and FirstListingPage. The deadline also sat in the check that decides whether get_more_tools and send_feedback are virtual-tool calls, so a tools/list slower than one second sent those calls to the application. That check is back to origin/main's. Tested with: - new tests failed on the previous head: a slow tools/list still answers get_more_tools and send_feedback; a raw low-level server records a failed $mcp_tool_call for an unknown name and for a hidden tool - pytest posthog/test/mcp: .venv (mcp 1.30, fastmcp 3.2) 696 passed; v2 (mcp 2.3) 533 passed; v2f4 (mcp 2.3, fastmcp 4.1) 577 passed; f2 (mcp 1.30, fastmcp 2.14) 688 passed, 1 failed - ruff format --check, ruff check, mypy | mypy-baseline filter, make public_api_check: clean Reviewer notes: the f2 failure, test_jlowin_middleware_defaults_work_across_fresh_instances, also fails on origin/main in that venv. The unknown-tool handle-delivery test now covers the error-result branch through standalone FastMCP, since a raw low-level server no longer records unknown tools. * fix(mcp): drop the manual unknown-tool API and shrink the spec alignment What: removed the public PostHogMCP.capture_unknown_tool() and the public PostHogMCPAnalyticsEvent.UNKNOWN_TOOL constant; the event name now lives in the private event-name map. The registry helpers stay in _instrument_lowlevel next to _dispatch_can_differ instead of moving to _instrumentation. Dropped the slow-listing and hidden-tool test cases (they guarded a listing probe this branch no longer has and pass on main), the lowlevel test fixture edits, and README text that documented #1204's annotations. The manual-session test moved to test_posthog_mcp.py. Why: stamphog refused #1199 for a public API change and its size. The spec makes manual capture methods optional, so capture_unknown_tool can return in its own PR. Tested with: pytest posthog/test/mcp on .venv, v2, f2, v2f4 (689, 526, 681, 570 passed; f2's one failure is test_jlowin_middleware_defaults_work_across_fresh_instances, failing on main until #1203). ruff format/check, mypy-baseline filter clean. make public_api_check: snapshot up to date, identical to origin/main. * fix(mcp): treat FastMCP 2.x mounts as unable to rule out a tool Review: _serves_other_providers only checked FastMCP 3.x providers, so on 2.x a mounted server (or one whose mount failed to load) made a missing name look unregistered and emitted $mcp_unknown_tool instead of a failed tool call. It now also checks 2.x's _mounted_servers. Tested: new server-with-mounted-server case fails on fastmcp 2.14.7 before the fix and passes after; pytest posthog/test/mcp on fastmcp 3.2 (690 passed), 2.14.7 and mcp 2.3+fastmcp 4.1 (only failures are the stdio handshake tests, which fail locally without this change, and the 2.x input-keys test that posthog-python#1203 fixes); ruff clean.
There was a problem hiding this comment.
Approved.
The privacy-sensitive portion has independent assurance from a current-head agent review. No concrete unresolved review concerns or other blocking grounds are present.
- Author wrote 0% of the modified lines and has 5 merged PRs in these paths (familiarity MODERATE).
- PostHog Review (Standard) reviewed the current head: no findings.
- posthog[bot] reviewed the current head.
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 10L, 1F substantive, 44L/3F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1b-small (44L, 3F, two-areas, fix) |
| stamphog 2.7.0 | .stamphog/policy.yml @ 4f115d9 · reviewed head 4f115d9 |
Problem
On FastMCP 2.x servers with middleware,
_listed_input_schemacallslist_tools(), which only exists on FastMCP 3.x. TheAttributeErrorwas caught and logged on every tool call, andtest_fastmcp_v2.py::test_jlowin_middleware_defaults_work_across_fresh_instancesexpected argument names that 2.x can't provide. Review ruled out the two alternatives:_list_tools_middleware). It spends request rate limits, so it can fail calls.get_tools()). It can disagree with what middleware advertises, for example a replaced tool or a prefixed mount sharing a root name.Changes
None. Argument names stay redacted in$mcp_input_keys, as on main, no middleware runs, and nothing is logged per call.["[redacted]"]. 3.x still expects["a", "b"].How I tested
pytest posthog/test/mcpon fastmcp 2.14.7: 661 passed. The only failure istest_stdio_handshake_then_calls, which fails locally without this change.pytest posthog/test/mcpon fastmcp 3.2.0: 665 passed.ruff format --check .andruff check .are clean.