Skip to content
10 changes: 10 additions & 0 deletions .sampo/changesets/mcp-v1-spec-align.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
pypi/posthog: minor
---

MCP analytics now matches the cross-SDK MCP spec ahead of 1.0:

- `instrument()` sends `$mcp_unknown_tool` instead of a failed `$mcp_tool_call` and `$exception` when a `tools/call` names a tool your server never registered. Insights that count failed tool calls stop counting these. A call naming no tool sends nothing. A raw low-level server has no registry the SDK can read, so a failed call there stays a failed tool call.
- FastMCP's stock logging, timing, error-handling, and similar middleware no longer turns off `llm_model` injection on standalone `fastmcp.FastMCP`.
- Manual capture without a `session_id` now sends a fresh `ses_` session id and uses it as the `distinct_id`, instead of no `$session_id` and an `anonymous` person.
- Truncated string properties now fit their limit with the `...` marker included, so `$mcp_tool_name` and other names stop at 256 characters instead of 259.
23 changes: 21 additions & 2 deletions posthog/mcp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,8 +191,10 @@ On MCP SDK 1.x, standalone FastMCP application middleware can replace or reroute
When it overrides a listing or dispatch hook, PostHog does not inject `llm_model`: a fresh
replica cannot safely distinguish that field from a replacement tool's own argument.
Client metadata capture remains enabled. This also applies to pass-through logging or
authorization middleware with those hooks; argument-based model capture is skipped so an
application-owned value cannot be mistaken for analytics. No additional catalog lookup runs
authorization middleware of your own with those hooks; argument-based model capture is skipped
so an application-owned value cannot be mistaken for analytics. FastMCP's stock middleware,
such as `LoggingMiddleware`, `TimingMiddleware`, and `ErrorHandlingMiddleware`, never reroutes a
tool and does not count; its tool-injection middleware does. No additional catalog lookup runs
during a tool call.

For a custom dispatcher, `PostHogMCP` enables model capture and conversation correlation
Expand Down Expand Up @@ -339,6 +341,23 @@ call = posthog.prepare_tool_call(
)
```

## Calls to tools your server does not have

A `tools/call` naming a tool your server never registered is not a tool call.
`instrument()` sends `$mcp_unknown_tool` with the requested name as
`$mcp_tool_name`, and no `$mcp_tool_call` or `$exception`. The agent gets the
same error as before. A call naming no tool sends nothing.

The SDK asks the tool registry only after the call fails, and records a failed
tool call whenever the registry can't rule the tool out:

- A disabled tool, or a tool called with invalid arguments, still exists.
- A raw low-level server has no registry the SDK can read.
- On standalone `fastmcp.FastMCP`, middleware of your own that overrides a
listing or dispatch hook can serve tools the registry lacks.
- On standalone `fastmcp.FastMCP` 3 and later, a mounted server, proxy, or
other provider hides its disabled tools.

## Virtual tools on a paginated `tools/list`

`instrument()` advertises up to two tools of its own: `get_more_tools`
Expand Down
1 change: 1 addition & 0 deletions posthog/mcp/_event_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,4 @@ class MCPAnalyticsEventType:
MCP_RESOURCES_READ = "mcp:resources/read"
MCP_TOOLS_CALL = "mcp:tools/call"
MCP_TOOLS_LIST = "mcp:tools/list"
MCP_UNKNOWN_TOOL = "mcp:unknown_tool"
4 changes: 4 additions & 0 deletions posthog/mcp/_instrument_fastmcp.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,10 @@ async def wrapped(
name, call_arguments, context=context, convert_result=convert_result
)
except Exception as error:
if await lifecycle.record_if_unknown_tool(
_name_owned_by_real_tool(server, name)
):
raise
# The minted prompt-back was never delivered to the agent — don't stamp
# an orphan conversation_id it can't echo (an agent-supplied id is kept).
await lifecycle.record_error(error, (time.monotonic() - start) * 1000)
Expand Down
89 changes: 85 additions & 4 deletions posthog/mcp/_instrument_lowlevel.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,11 @@
from __future__ import annotations

import functools
import importlib
import inspect
import time
from functools import lru_cache
from typing import Any, Dict, List, Optional, Set, Tuple
from typing import Any, Dict, List, Optional, Set, Tuple, Type

import mcp.types as mcp_types

Expand All @@ -36,6 +37,7 @@
copy_tools_list_result,
extract_tools,
is_first_listing_page,
is_tool_result_error,
mutate_tool_schema,
prepare_request,
raw_listing_owns_tool_name,
Expand Down Expand Up @@ -282,6 +284,10 @@ async def handler(req: Any) -> Any:
try:
result = await original(req)
except Exception as error:
if await lifecycle.record_if_unknown_tool(
await registered_tool_exists(high_level, name)
):
raise
# The @server.call_tool() decorator converts raises into
# CallToolResult(isError=True), but a handler wired straight into
# request_handlers can raise — capture before re-raising so the failed
Expand Down Expand Up @@ -333,6 +339,12 @@ async def handler(req: Any) -> Any:
else call_result
)

if is_tool_result_error(call_result) and await lifecycle.record_if_unknown_tool(
await registered_tool_exists(high_level, name),
conversation_id_delivered=delivered,
):
return result

await lifecycle.record_result(
call_result,
duration_ms,
Expand Down Expand Up @@ -573,7 +585,7 @@ async def _name_owned_by_real_tool(


@lru_cache(maxsize=1)
def _tool_lookup_not_found_errors() -> Tuple[type, ...]:
def _tool_lookup_not_found_errors() -> Tuple[Type[BaseException], ...]:
"""The fastmcp exceptions that mean "no live tool by that name" -- an answer.
Anything else out of ``get_tool`` is the lookup itself failing. Empty when
fastmcp is absent or has moved them, which makes every failure delegate:
Expand Down Expand Up @@ -793,13 +805,82 @@ def _dispatch_can_differ(server: Any) -> bool:
)


# Stock fastmcp middleware that observes, times, retries, throttles, denies,
# caches, or trims a call to the tool it names, and never provides, renames, or
# reroutes one. The tool-injection family is left out on purpose.
_STOCK_DISPATCH_PRESERVING_MIDDLEWARE = (
("logging", "LoggingMiddleware"),
("logging", "StructuredLoggingMiddleware"),
("timing", "TimingMiddleware"),
("timing", "DetailedTimingMiddleware"),
("error_handling", "ErrorHandlingMiddleware"),
("error_handling", "RetryMiddleware"),
("rate_limiting", "RateLimitingMiddleware"),
("rate_limiting", "SlidingWindowRateLimitingMiddleware"),
("ping", "PingMiddleware"),
("authorization", "AuthMiddleware"),
("caching", "ResponseCachingMiddleware"),
("response_limiting", "ResponseLimitingMiddleware"),
)


@functools.lru_cache(maxsize=1)
def _builtin_middleware_types() -> tuple:
"""The middleware a bare ``FastMCP()`` installs on its own, probed rather
than named so a new built-in in a later release is still recognised."""
than named so a new built-in in a later release is still recognised, plus
the stock middleware above that this fastmcp release ships."""
from fastmcp import FastMCP

return tuple(type(middleware) for middleware in FastMCP("posthog-probe").middleware)
stock = []
for module, name in _STOCK_DISPATCH_PRESERVING_MIDDLEWARE:
try:
stock.append(
getattr(
importlib.import_module(f"fastmcp.server.middleware.{module}"), name
)
)
except (ImportError, AttributeError):
continue
return tuple(
type(middleware) for middleware in FastMCP("posthog-probe").middleware
) + tuple(stock)


async def registered_tool_exists(server: Any, name: str) -> Optional[bool]:
"""Whether jlowin's FastMCP registers ``name``, enabled or not, asked after a
failed call to tell an unknown tool from a failed one. A disabled tool still
exists: calling it is a failed call. ``None`` when nothing authoritative can
say: no registry (a raw low-level server, whose listing may hide tools its
dispatcher serves), middleware that can serve tools the registry lacks, or a
mounted server or other provider, which hides its disabled tools."""
if server is None or _dispatch_can_differ(server):
return None
try:
from fastmcp import FastMCP

# From 3.x, get_tool hides a disabled tool; the providers still hold it.
found = (
await server.get_tool(name) is not None
or await super(FastMCP, server).get_tool(name) is not None
)
except _tool_lookup_not_found_errors():
found = False
except Exception as err: # noqa: BLE001 - analytics must not break the call
log(f"PostHog MCP: could not look up tool {name!r} - {err}")
return None
if found:
return True
return None if _serves_other_providers(server) else False


def _serves_other_providers(server: Any) -> bool:
"""Whether a FastMCP server serves tools from anywhere but its own
registry: a mounted server, a proxy, or a custom provider. FastMCP 2.x
keeps mounts in ``_mounted_servers``; 3.x lists them in ``providers``."""
if getattr(server, "_mounted_servers", None):
return True
local = getattr(server, "_local_provider", None)
return any(provider is not local for provider in getattr(server, "providers", ()))
Comment thread
lucasheriques marked this conversation as resolved.
Resolved


def _reference_chain(schema: Dict[str, Any]) -> Optional[List[Dict[str, Any]]]:
Expand Down
16 changes: 16 additions & 0 deletions posthog/mcp/_instrument_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
_to_jsonable,
copy_tools_list_result,
is_first_listing_page,
is_tool_result_error,
mutate_tool_schema,
params_to_request_dict,
prepare_request,
Expand All @@ -66,6 +67,7 @@
start_tools_list_lifecycle,
warn_ownership_lookup_failed,
)
from ._instrument_lowlevel import registered_tool_exists
from ._internal import MCPAnalyticsData
from ._model_parameters import (
can_inject_model_parameter,
Expand Down Expand Up @@ -450,6 +452,10 @@ async def wrapped(
name, call_arguments, context=context, convert_result=convert_result
)
except Exception as error:
if await lifecycle.record_if_unknown_tool(
_name_owned_by_real_tool_v2(server, name)
):
raise
# The outer MCPServer layer converts the raise after this seam; no
# freshly minted handle could have been delivered yet.
await lifecycle.record_error(error, (time.monotonic() - start) * 1000)
Expand Down Expand Up @@ -669,6 +675,10 @@ async def handler(ctx: Any, params: Any) -> Any:
try:
result = await original(ctx, params)
except Exception as error:
if await lifecycle.record_if_unknown_tool(
await registered_tool_exists(standalone, name)
):
raise
# Raw v2 handlers raise through to JSON-RPC errors; no freshly minted
# handle could have been delivered yet.
await lifecycle.record_error(error, (time.monotonic() - start) * 1000)
Expand All @@ -685,6 +695,12 @@ async def handler(ctx: Any, params: Any) -> Any:
lifecycle.minted_conversation_id,
)

if is_tool_result_error(result) and await lifecycle.record_if_unknown_tool(
await registered_tool_exists(standalone, name),
conversation_id_delivered=delivered,
):
return result

await lifecycle.record_result(
result, duration_ms, conversation_id_delivered=delivered
)
Expand Down
59 changes: 59 additions & 0 deletions posthog/mcp/_instrumentation.py
Original file line number Diff line number Diff line change
Expand Up @@ -615,6 +615,32 @@ async def record_feedback(self, *, conversation_id_delivered: bool = False) -> s
)
return await handle_feedback(report, self.feedback_options)

async def record_if_unknown_tool(
self, tool_exists: Optional[bool], *, conversation_id_delivered: bool = False
) -> bool:
"""Record the failed call as ``$mcp_unknown_tool`` when ``tool_exists``
proves the server has no tool by that name, and say whether the call is
settled that way. A call naming no tool is settled with no event. The
adapter then re-raises or returns its response."""
if not self.name:
return True
if tool_exists is not False:
return False
conversation_id = self._anchored_conversation_id(conversation_id_delivered)
session_id = await self.prepare_session(conversation_id)
await record_unknown_tool(
self.data,
session_id,
conversation_id=conversation_id,
name=self.name,
arguments=self.arguments,
client_name=self.client_name,
client_version=self.client_version,
protocol_version=self.protocol_version,
extra=self.extra,
)
return True

async def record_error(self, error: Any, duration_ms: float) -> None:
# A freshly minted handle cannot anchor or be captured when dispatch
# raised: no adapter had an opportunity to deliver it to the agent.
Expand Down Expand Up @@ -1374,6 +1400,39 @@ async def record_missing_capability(
log(f"record_missing_capability failed (event dropped): {err}")


async def record_unknown_tool(
data: MCPAnalyticsData,
session_id: str,
*,
conversation_id: Optional[str],
name: str,
arguments: Optional[Dict[str, Any]],
client_name: Optional[str],
client_version: Optional[str],
protocol_version: Optional[str],
extra: Optional[Dict[str, Any]],
) -> None:
"""Record a call naming a tool the server never registered as
``$mcp_unknown_tool``. It is not a tool call, so it carries no outcome."""
try:
event: Dict[str, Any] = {
"event_type": MCPAnalyticsEventType.MCP_UNKNOWN_TOOL,
"session_id": session_id,
"conversation_id": conversation_id,
"resource_name": name,
"client_name": client_name,
"client_version": client_version,
"protocol_version": protocol_version,
}
await _apply_event_properties(
data, event, build_tool_call_request(name, arguments), extra
)
stamp_transport_identity(event, extra)
fire_and_forget(capture_event(data, event), data)
except Exception as err: # noqa: BLE001 - isolate analytics from the tool path
log(f"record_unknown_tool failed (event dropped): {err}")


async def record_feedback(
data: MCPAnalyticsData,
session_id: str,
Expand Down
9 changes: 7 additions & 2 deletions posthog/mcp/_posthog_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
MCPAnalyticsEventType.MCP_RESOURCES_READ: PostHogMCPAnalyticsEvent.RESOURCE_READ,
MCPAnalyticsEventType.MCP_TOOLS_CALL: PostHogMCPAnalyticsEvent.TOOL_CALL,
MCPAnalyticsEventType.MCP_TOOLS_LIST: PostHogMCPAnalyticsEvent.TOOLS_LIST,
MCPAnalyticsEventType.MCP_UNKNOWN_TOOL: "$mcp_unknown_tool",
}

_P = PostHogMCPAnalyticsProperty
Expand Down Expand Up @@ -112,8 +113,12 @@ def _is_tool_call(event: Event) -> bool:

def _add_common_properties(event: Event, properties: Dict[str, Any]) -> None:
if event.get("resource_name"):
properties[_P.RESOURCE_NAME] = event["resource_name"]
if _is_tool_call(event):
is_unknown_tool = (
event.get("event_type") == MCPAnalyticsEventType.MCP_UNKNOWN_TOOL
)
if not is_unknown_tool:
properties[_P.RESOURCE_NAME] = event["resource_name"]
if is_unknown_tool or _is_tool_call(event):
properties[_P.TOOL_NAME] = event["resource_name"]
if event.get("tool_description") and _is_tool_call(event):
properties[_P.TOOL_DESCRIPTION] = event["tool_description"]
Expand Down
11 changes: 10 additions & 1 deletion posthog/mcp/_sanitization.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
from typing import Any, Dict, List, Optional, Set, Tuple
from urllib.parse import SplitResult, parse_qsl, urlencode, urlsplit, urlunsplit

from ._event_types import MCPAnalyticsEventType

# SDK-injected arguments stripped from captured $mcp_parameters (they surface as
# dedicated properties: $mcp_intent and $mcp_conversation_id).
_INJECTED_ARGUMENT_NAMES = ("context", "conversation_id")
Expand Down Expand Up @@ -628,8 +630,15 @@ def sanitize_event(event: Dict[str, Any]) -> Dict[str, Any]:
if result.get("parameters") is not None:
result["parameters"] = sanitize_captured_value(result["parameters"])

# An unknown tool's name is whatever the agent sent, not one of the
# server's identifiers, so it gets the free-text pass.
if result.get("resource_name") is not None:
result["resource_name"] = _sanitize_resource_name(result["resource_name"])
sanitize_name = (
sanitize_free_text
if result.get("event_type") == MCPAnalyticsEventType.MCP_UNKNOWN_TOOL
else _sanitize_resource_name
)
result["resource_name"] = sanitize_name(result["resource_name"])

# The intent comes straight from an agent-narrated `context` string, so it
# can contain a secret the LLM read aloud or personal data it narrated about
Expand Down
Loading
Loading