Repository navigation
feat(mcp): group sessions by ChatGPT, Codex and VS Code conversation ids #1193
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
c81277c
1688eeb
208707d
02a4815
3cbcd29
08774e9
6571788
5417daa
c8df7b0
4b103b5
be1d543
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| pypi/posthog: minor | ||
| --- | ||
|
|
||
| MCP analytics now groups tool calls into sessions by the client's own conversation id in `_meta`: ChatGPT's `openai/session`, Codex's `sessionId`, and VS Code's `vscode.conversationId`, each read only from the client its `User-Agent` or client name identifies. The client's id wins over a conversation handle and a transport session. ChatGPT gets no `conversation_id` argument and no handle, so its tool schemas and results stay unchanged. Events from `instrument()` now carry `$mcp_session_source` (`client_metadata`, `conversation`, `transport` or `generated`), so session fragmentation can be measured. `PostHogMCP.prepare_tool_list` and `prepare_tool_call` take the request's `headers` (and `client_info`), and `capture_tool_call` takes `request_meta`. |
| Original file line number | Diff line number | Diff line change | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -59,6 +59,8 @@ | ||||||||||
| from .session import ( | |||||||||||
| derive_session_id_from_conversation, | |||||||||||
| derive_session_id_from_mcp_session, | |||||||||||
| current_session, | |||||||||||
| unsettled_custom_events, | |||||||||||
| new_session_id, | |||||||||||
| ) | |||||||||||
| from .session_token import ( | |||||||||||
|
|
@@ -161,16 +163,21 @@ async def capture(self, event: str, properties: Optional[dict] = None) -> None: | ||||||||||
| data = get_server_tracking_data(self._key) | |||||||||||
| if data is None: | |||||||||||
| return | |||||||||||
| coro = capture_event( | |||||||||||
| data, | |||||||||||
| { | |||||||||||
| "session_id": data.session_id, | |||||||||||
| "event_type": MCPAnalyticsEventType.CUSTOM, | |||||||||||
| "event_name": event, | |||||||||||
| "timestamp": datetime.now(timezone.utc), | |||||||||||
| "properties": properties, | |||||||||||
| }, | |||||||||||
| ) | |||||||||||
| session_id, session_source = current_session(data) | |||||||||||
| event_input = { | |||||||||||
| "session_id": session_id, | |||||||||||
| "session_source": session_source, | |||||||||||
| "event_type": MCPAnalyticsEventType.CUSTOM, | |||||||||||
| "event_name": event, | |||||||||||
| "timestamp": datetime.now(timezone.utc), | |||||||||||
| "properties": properties, | |||||||||||
| } | |||||||||||
| pending = unsettled_custom_events(data) | |||||||||||
| if pending is not None: | |||||||||||
| # Sent once the call settles; its duration is still the moment it was captured. | |||||||||||
| pending.events.append({**event_input, "duration": 0}) | |||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2 · Snapshot custom properties before queuing events When a tool reuses a properties dictionary across awaited
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2 · Bound the deferred custom-event buffer During a long-running tool call that mints a handle, each awaited There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Medium: Unbounded deferred-event buffer Calls that mint a conversation handle now accumulate every full custom event until the tool finishes, before the sink or the client's bounded queue can drop anything. For bulk or streaming tools that capture per client-supplied item, this allows unbounded SDK memory growth despite
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P2] Bound deferred payloads before downstream queue limitsCompared base MINOR · OPTIONAL · Introduced by deferred capture. A call minting a handle retains every full custom payload until completion: for index in range(64):
await analytics.capture(
"progress", {"blob": str(index) + ":" + "x" * (256 * 1024)}
)
The executed harness below uses a counting/discarding client that retains no payloads:
Anchoring-off controls remain approximately 24–29 KB on head. Head forwards zero captures before the tool returns and all 64 afterwards. Downstream blobs are approximately 101 KB, while the list retains their original 256 KiB inputs. Consider an internal count-and-size budget with explicit overflow handling and bounded settlement work, preserving outcome-dependent attribution. Why optional: The mechanism is confirmed, but this deliberately large fixture does not establish typical residency or likely exhaustion. It measures wrapper-stage Python allocations, not a real host queue’s total memory, RSS, OOM or production harm. All 64 events eventually reach the sink. The observer-only requirement bounds per-session state/pending tasks, but does not specify an exact per-invocation raw-byte budget. Downstream limits cannot constrain this new list; their existing implementation is not alleged to be broken. Executed measurement: Run the complete harness below with Executed shared scalar/retention harnessRun with the same selected MCP version—1.28.1 or 2.3.0—on both revisions: pytest -q -s --asyncio-mode=auto test_deferred_capture_standalone.pyUse import importlib.metadata
import json
import tracemalloc
import httpx
import pytest
from posthog.mcp import instrument, MCPAnalyticsOptions
from posthog.test.mcp._helpers import FakeClient
MAJOR = int(importlib.metadata.version("mcp").split(".")[0])
if MAJOR < 2:
from mcp.server.fastmcp import FastMCP
from mcp.server.transport_security import TransportSecuritySettings
else:
from mcp.server.mcpserver import MCPServer
from posthog.test.mcp._helpers_v2 import modern_headers, modern_meta
async def call_tool(body, client, anchored):
server = (
FastMCP(
"deferred-controls", stateless_http=True, json_response=True,
transport_security=TransportSecuritySettings(
enable_dns_rebinding_protection=False
),
)
if MAJOR < 2 else MCPServer("deferred-controls")
)
@server.tool()
async def probe() -> str:
await body(analytics)
return "ok"
analytics = instrument(
server, client,
MCPAnalyticsOptions(
enable_conversation_id=anchored,
context=False, capture_model=False,
send_initialize_event=False,
),
)
app = (
server.streamable_http_app() if MAJOR < 2
else server.streamable_http_app(
json_response=True, stateless_http=True
)
)
async with server.session_manager.run():
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app),
base_url="http://127.0.0.1:8000",
) as http:
headers = (
{
"content-type": "application/json",
"accept": "application/json, text/event-stream",
"mcp-protocol-version": "2025-06-18",
}
if MAJOR < 2 else modern_headers("tools/call", "probe")
)
response = await http.post(
"/mcp",
headers={**headers, "user-agent": "example-client/1.0"},
json={
"jsonrpc": "2.0", "id": 1, "method": "tools/call",
"params": {
"name": "probe", "arguments": {},
"_meta": {} if MAJOR < 2 else modern_meta(),
},
},
)
assert response.status_code == 200, response.text
result = response.json()
assert "error" not in result
assert not result["result"].get("isError"), result
if anchored:
assert "conversation_id" in json.dumps(result)
await analytics.flush()
@pytest.mark.parametrize("anchored", [False, True])
async def test_capture_snapshots_properties_before_returning(anchored):
client = FakeClient()
async def body(analytics):
props = {"step": 0}
await analytics.capture("progress", props)
props["step"] = 1
await analytics.capture("progress", props)
props.clear()
await call_tool(body, client, anchored)
events = [e for e in client.events if e["event"] == "progress"]
assert len(events) == 2
actual = [e["properties"].get("step") for e in events]
print(json.dumps({"anchored": anchored, "steps": actual}))
assert actual == [0, 1]
class CountingClient:
"""Never retains event payloads itself."""
def __init__(self):
self.count = 0
self.largest_blob = 0
def capture(self, event, **kwargs):
if event == "progress":
self.count += 1
self.largest_blob = max(
self.largest_blob,
len(kwargs["properties"].get("blob", "")),
)
@pytest.mark.parametrize("anchored", [False, True])
async def test_large_progress_events_do_not_accumulate_before_sink_limits(
anchored,
):
client = CountingClient()
observed = {}
async def body(analytics):
tracemalloc.start()
before = tracemalloc.get_traced_memory()[0]
try:
for index in range(64):
await analytics.capture(
"progress",
{"blob": str(index) + ":" + "x" * (256 * 1024)},
)
current, peak = tracemalloc.get_traced_memory()
observed.update(
retained_bytes=current-before,
peak_bytes=peak-before,
captures_before_tool_returns=client.count,
)
finally:
tracemalloc.stop()
await call_tool(body, client, anchored)
print(json.dumps({
"anchored": anchored, **observed,
"total_captures": client.count,
"largest_sink_blob": client.largest_blob,
}))
assert client.count == 64
assert observed["retained_bytes"] < 2 * 1024 * 1024The harness is formatting-equivalent to the executed source. The memory threshold detects the change, not an agreed future cap/drop contract. |
|||||||||||
| return | |||||||||||
| coro = capture_event(data, event_input) | |||||||||||
| if coro is not None: | |||||||||||
| await coro | |||||||||||
|
|
|||||||||||
|
|
|||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,173 @@ | ||
| """Client conversation ids: a value an MCP client puts in a ``tools/call`` | ||
| request's ``_meta`` to name the conversation the call belongs to. | ||
|
|
||
| ``CLIENT_CONVERSATION_ROWS`` is the table of the spec's "Session identity" | ||
| requirement, row for row. A row's key is read only from a request its recognition | ||
| rule matches, so a generic key such as Codex's ``sessionId`` is never read from a | ||
| client that means something else by it. posthog-js keeps the same table. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import re | ||
| from typing import Any, Callable, NamedTuple, Optional, Sequence | ||
|
|
||
| from ._transport_identity import CLIENT_USER_AGENT_HEADER | ||
| from .request_headers import _to_header_bag, get_request_headers | ||
| from .session_token import decode_session_id, read_mcp_session_header | ||
|
|
||
| __all__ = [ | ||
| "CLIENT_CONVERSATION_ROWS", | ||
| "ClientConversation", | ||
| "ClientConversationRow", | ||
| "meta_client_name", | ||
| "read_client_conversation", | ||
| "recognize_client", | ||
| "recognize_manual_client", | ||
| "request_user_agent", | ||
| ] | ||
|
|
||
|
|
||
| class ClientConversation(NamedTuple): | ||
| """A recognized client's conversation id, and the ``_meta`` key it came from.""" | ||
|
|
||
| key: str | ||
| id: str | ||
|
|
||
|
|
||
| class ClientConversationRow(NamedTuple): | ||
| key: str | ||
| read_value: Callable[[Any], Optional[str]] | ||
| recognizes: Callable[[Optional[str], Optional[str]], bool] | ||
| # The client sends its id on every call, so the SDK advertises no | ||
| # ``conversation_id`` to it and mints it no handle. Rows whose clients have | ||
| # versions or kinds of conversation that send no id keep the handle. | ||
| handle_free: bool | ||
|
|
||
|
|
||
| _PRINTABLE_ASCII_UP_TO_128 = re.compile(r"[\x20-\x7e]{1,128}") | ||
| _UUID = re.compile(r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}") | ||
| _NIL_UUID = "00000000-0000-0000-0000-000000000000" | ||
| _CHATGPT_USER_AGENT = re.compile(r"openai-mcp/[^\s(]*\s*\(\s*chatgpt\s*[;,)]", re.I) | ||
| _VSCODE_PRODUCT_NAMES = ("Visual Studio Code", "Visual Studio Code - Insiders") | ||
|
|
||
|
|
||
| def _read_printable_ascii(value: Any) -> Optional[str]: | ||
| if isinstance(value, str) and _PRINTABLE_ASCII_UP_TO_128.fullmatch(value): | ||
| return value | ||
| return None | ||
|
|
||
|
|
||
| def _read_uuid(value: Any) -> Optional[str]: | ||
| if not isinstance(value, str): | ||
| return None | ||
| lowered = value.lower() | ||
| return lowered if _UUID.fullmatch(lowered) and lowered != _NIL_UUID else None | ||
|
|
||
|
|
||
| def _user_agent_starts_with(user_agent: Optional[str], prefixes: Sequence[str]) -> bool: | ||
| lowered = (user_agent or "").lower() | ||
| return bool(lowered) and any(lowered.startswith(p.lower()) for p in prefixes) | ||
|
|
||
|
|
||
| def _client_name_is(client_name: Optional[str], names: Sequence[str]) -> bool: | ||
| lowered = (client_name or "").lower() | ||
| return bool(lowered) and any(lowered == name.lower() for name in names) | ||
|
|
||
|
|
||
| # Checked in this order; the first row that recognizes a request is its client's row. | ||
| CLIENT_CONVERSATION_ROWS = ( | ||
| ClientConversationRow( | ||
| key="openai/session", | ||
| read_value=_read_printable_ascii, | ||
| recognizes=lambda user_agent, _: bool( | ||
| user_agent and _CHATGPT_USER_AGENT.match(user_agent) | ||
| ), | ||
| handle_free=True, | ||
| ), | ||
| ClientConversationRow( | ||
| key="sessionId", | ||
| read_value=_read_uuid, | ||
| recognizes=lambda user_agent, client_name: ( | ||
| _user_agent_starts_with(user_agent, ["codex-mcp-client/"]) | ||
| or _client_name_is(client_name, ["codex-mcp-client"]) | ||
| ), | ||
| handle_free=False, | ||
| ), | ||
| ClientConversationRow( | ||
| key="vscode.conversationId", | ||
| read_value=_read_uuid, | ||
| recognizes=lambda user_agent, client_name: ( | ||
| _user_agent_starts_with( | ||
| user_agent, [f"{name}/" for name in _VSCODE_PRODUCT_NAMES] | ||
| ) | ||
| or _client_name_is(client_name, _VSCODE_PRODUCT_NAMES) | ||
| ), | ||
| handle_free=False, | ||
| ), | ||
| ) | ||
|
|
||
|
|
||
| def recognize_client( | ||
| user_agent: Optional[str], client_name: Optional[str] | ||
| ) -> Optional[ClientConversationRow]: | ||
| """The row of the client a request's ``User-Agent`` or client name identifies. | ||
|
|
||
| The client name is ``$mcp_client_name`` as the adapters resolve it for the | ||
| request: its own ``_meta``, then a replayed session token, then a handshake.""" | ||
| user_agent = user_agent if isinstance(user_agent, str) else None | ||
| client_name = client_name if isinstance(client_name, str) else None | ||
| for row in CLIENT_CONVERSATION_ROWS: | ||
| if row.recognizes(user_agent, client_name): | ||
| return row | ||
| return None | ||
|
|
||
|
|
||
| def read_client_conversation( | ||
| row: Optional[ClientConversationRow], meta: Any | ||
| ) -> Optional[ClientConversation]: | ||
| """The recognized client's conversation id in ``meta``, or ``None`` when there | ||
| is none or it is not the row's shape.""" | ||
| if row is None or not isinstance(meta, dict): | ||
| return None | ||
| value = row.read_value(meta.get(row.key)) | ||
| return ClientConversation(row.key, value) if value is not None else None | ||
|
|
||
|
|
||
| def request_user_agent(extra: Any) -> Optional[str]: | ||
| """The request's ``User-Agent``, or ``None`` off HTTP. Never raises.""" | ||
| try: | ||
| headers = get_request_headers(extra) | ||
| except Exception: # noqa: BLE001 - a header read must not break a request | ||
| return None | ||
| value = (headers or {}).get(CLIENT_USER_AGENT_HEADER) | ||
| return value if isinstance(value, str) else None | ||
|
|
||
|
|
||
| def recognize_manual_client( | ||
| headers: Any, request_meta: Any, handshake_client_name: Optional[str] | ||
| ) -> Optional[ClientConversationRow]: | ||
| """:func:`recognize_client` for a custom dispatcher, from what it passes in: | ||
| the request's headers, its ``_meta``, and a handshake ``clientInfo`` name | ||
| bound to the request's transport session. The client name follows the | ||
| request's ``_meta``, then a replayed session token, then the handshake.""" | ||
| bag = _to_header_bag(headers) if headers is not None else None | ||
| user_agent = (bag or {}).get(CLIENT_USER_AGENT_HEADER) | ||
| token = decode_session_id(read_mcp_session_header(bag)) if bag else None | ||
| return recognize_client( | ||
| user_agent if isinstance(user_agent, str) else None, | ||
| meta_client_name(request_meta) | ||
| or (token.client_name if token is not None else None) | ||
| or handshake_client_name, | ||
| ) | ||
|
|
||
|
|
||
| def meta_client_name(meta: Any) -> Optional[str]: | ||
| """The client name a 2026-07-28 request carries in its own ``_meta``.""" | ||
| info = ( | ||
| meta.get("io.modelcontextprotocol/clientInfo") | ||
| if isinstance(meta, dict) | ||
| else None | ||
| ) | ||
| name = info.get("name") if isinstance(info, dict) else None | ||
| return name if isinstance(name, str) and name else None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P1] Snapshot scalar properties before returning from deferred capture
Compared base
f22511891225fb15a4e6a0f889e723e9b5d0ebe4with headbe1d543c1909f6b9d21e1aa449258cbc60dc8af1.IMPORTANT · REQUIRED · Introduced by deferred capture.
Related:
posthog/mcp/__init__.py:175–179.Reusing a dictionary after awaited capture now changes the contents of earlier events:
Expected: the events retain
step=0andstep=1. Actual: both are emitted withoutstep.The shallow event-input copy retains the caller-owned
propertiesmapping until settlement. Base awaited materialization, which copied its top-level entries into the outgoing mapping before returning.Executed public HTTP/ASGI comparison, both official majors:
[0, 1][0, 1][0, 1]No concurrency, malformed input or private-state mutation is required. This regresses established observable capture behavior; the timestamp represents capture time while its payload instead reflects later application state.
Consider snapshotting the top-level mapping when accepting the deferred event, preserving fail-closed handling and final session selection.
Scope: This establishes top-level scalar preservation only. Nested mutation fails on base too; no deep-copy contract is inferred. Automatic events and identities are not shown to be lost.
Each major’s scalar cases produce base 2 passed, head 1 failed / 1 passed. The complete shared harness below produces base 4 passed, head 2 failed / 2 passed, including the separate optional allocation measurement.
Executed shared scalar/retention harness
Run with the same selected MCP version—1.28.1 or 2.3.0—on both revisions:
Use
-k snapshotsfor the scalar cases and-k large_progressfor retention.The harness is formatting-equivalent to the executed source. The memory threshold detects the change, not an agreed future cap/drop contract.