Skip to content

feat: send first-party analytics headers on Kosli API requests - #106

Merged
dangrondahl merged 1 commit into
mainfrom
analytics-headers
Oct 29, 2026
Merged

dangrondahl merged 1 commit into
mainfrom
analytics-headers

Conversation

@tooky

@tooky tooky commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Steve Tooke · Slack thread

Before: the Kosli server can tell a request came from the MCP server (via the User-Agent), but not which tool or catalog action made it, or which requests belong to the same MCP session.

After: every Kosli API request carries first-party analytics headers naming the client, and requests from the execute tools also name the tool, the action and the session, so product analytics can count MCP usage per operation and per session.

Header Value Sent on
X-Kosli-Client mcp/<version> (from src/version.ts, the same source as the User-Agent) every Kosli API request
X-Kosli-Operation <tool> <action id>, e.g. execute_read_action get_trail, execute_write_action create_control requests from execute_read_action / execute_write_action
X-Kosli-Invocation-Id UUID v4, generated once per createServer, so one per MCP session requests from execute_read_action / execute_write_action

User-Agent: kosli-mcp-server/<version> is unchanged. search_actions searches the bundled catalog and makes no request, so it sends nothing. The redirect download for evidence files goes to a pre-signed object-store URL, not Kosli, so it keeps only the User-Agent as before.

Why: Kosli's product analytics (Segment to Mixpanel, grouped by org) only see write events today, and can only tell "CLI" from "other" by the User-Agent. These headers let the server attribute each API call to a client, its version, the operation and one run or session, so we learn how customers actually use Kosli with the data they put in. A follow-up server change reads the headers and adds them to the events. Until then the server ignores them, and they are all optional, so older clients keep working and the server falls back to the User-Agent.

What the MCP headers will let us see:

  • Which catalog actions agents call, for example execute_read_action get_trail.
  • The mix of read and write actions agents make.
  • Activity per MCP session, grouped by the invocation id.
  • Which MCP server versions are in use. Today the server's User-Agent matching counts versioned MCP traffic as "other"; the server follow-up fixes that.

Tool and action names only, never arguments or param values, and the headers go only to the Kosli API.

Privacy: only names go in these headers: the tool name and the catalog action id, which is a public OpenAPI operationId. No tool argument, param value, org, search query or prompt text is ever put in a header. Tests send arguments containing a marker string through both execute tools and assert it appears in no header other than Authorization.

Compatibility: the server ignores these headers until it reads them, and treats each as optional with a fallback to the User-Agent, so this can ship first and older releases keep working. Related: kosli-dev/server#7728 (server-side query events). There is no opt-out, by decision.

Changes

  • src/client/kosli-client.ts: X-Kosli-Client on every request; execute() takes an optional RequestOrigin (tool, invocationId) that adds the other two.
  • src/tools/execute-action.ts: ExecuteOptions.origin passed through to the client.
  • src/server.ts: one randomUUID() per createServer; each execute tool passes its own name.
  • CLAUDE.md: documents the headers and the names-only rule next to the User-Agent convention.
  • Tests in test/client/kosli-client.test.ts and test/server.test.ts (over the in-memory transport): header values, operation per tool, one invocation id per session and a new one per server, no arguments in any header, no request from search_actions.

npm run build, npm run typecheck, npm test (147 passed) and the offline npm run smoke all pass locally.

Related PRs

The header names are shared across clients: X-Kosli-Client, X-Kosli-Operation and X-Kosli-Invocation-Id. The CLI also sends X-Kosli-Cli-Flags and X-Kosli-Ci.


Generated by Claude Code

The Kosli server can see that a request came from the MCP server, through
the User-Agent, but not which tool or catalog action made it, or which
requests belong to the same session. Steve Tooke asked for first-party
headers so product analytics can count MCP usage per operation and per
session, alongside the query events the server now records
(kosli-dev/server#7728).

Every Kosli API request now sends X-Kosli-Client: mcp/<version>.
Requests from execute_read_action and execute_write_action also send
X-Kosli-Operation: "<tool> <action id>" and X-Kosli-Invocation-Id, a
UUID made once per createServer, so one per MCP session. The User-Agent
is unchanged. Only names are sent: the action id is the catalog's public
OpenAPI operationId, and no argument, param value, search query or prompt
text goes into a header. search_actions makes no request, and the
redirect download goes to an object store rather than Kosli, so neither
carries them.

The server treats every header as optional and falls back to the
User-Agent, so this can ship before the server reads them and older
releases keep working.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N9LH8PXUvMmSKYK9Mr65we
@tooky
tooky requested a review from dangrondahl October 28, 2026 16:30
@tooky
tooky marked this pull request as ready for review October 28, 2026 16:39
@claude

claude Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Claude finished @tooky's task in 2m 35s —— View job


Review: analytics headers

  • Read CLAUDE.md invariants and the PR diff
  • Review src/client/kosli-client.ts (headers, redirect path, multipart)
  • Review src/server.ts / src/tools/execute-action.ts wiring
  • Review tests
  • Check CI status (CI ✅, CodeQL ✅ on the latest commit)
  • Post findings

No blocking findings. The change is tight and holds the repo's invariants:

  • The three-tool, catalog-driven design is untouched; no new tool, no new dependency, no change to searchActions.
  • Header values are derived only from VERSION, the hard-coded tool name and entry.id from the generated catalog — nothing from params, org, fields or the query reaches a header. I checked every id in src/catalog.json against ^[A-Za-z0-9_.-]+$: all match, so no action id can inject into a header value or make undici reject one today.
  • The tool-name literals in run("execute_read_action", …) / run("execute_write_action", …) match the two registerTool names exactly (src/server.ts:93, src/server.ts:112), so the operation label can't mislabel the read/write split. Annotations are unchanged.
  • X-Kosli-Client is added to baseHeaders before the multipart branch, so it rides the Content-Type-less multipart path too (src/client/kosli-client.ts:119) — and the test at test/client/kosli-client.test.ts covers that.
  • The pre-signed download in download() still sends only User-Agent (src/client/kosli-client.ts:307), matching what the PR body and the new CLAUDE.md line claim.
  • origin is threaded as an optional parameter, so existing executeAction tests keep working unchanged, and search_actions makes no request at all.
  • Tests cover the real paths: per-tool operation, one invocation id per createServer and a different one for a second server, and the no-arguments-in-headers assertion over the in-memory transport. No test hits the network. Commit message and PR title follow Conventional Commits.

Two minor notes, neither worth blocking on: a write or DELETE is still handed to fetch with redirects followed automatically (src/client/kosli-client.ts:225), and per the Fetch spec only Authorization/Cookie/Proxy-Authorization are stripped across origins — so on a cross-origin redirect of a write the new X-Kosli-* headers would be replayed to the target, which the CLAUDE.md sentence about redirects doesn't quite cover (the payload is only a UUID and a public operationId, so the impact is slight). And since X-Kosli-Invocation-Id is a new per-session identifier with no opt-out, a line in README.md would be the natural place to disclose it — though the existing User-Agent tracking isn't documented there either, so that's a gap this PR inherits rather than creates.
· branch analytics-headers

tooky commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review. On the two minor notes:

  1. Cross-origin redirect on writes replaying X-Kosli-*: agreed that it can happen in principle. The headers carry only a random per-session UUID and a public operationId, and switching writes to manual redirect handling would change request behaviour for little gain, so this stays as is. I checked the 56 write actions in src/catalog.json: none documents a redirect. The closest is create_attestation_upload, which returns the upload location in the response body rather than redirecting, and this server never follows that location.
  2. README disclosure: agreed, this is a gap the PR inherits, since the User-Agent tracking isn't documented there either. I'll leave both for a docs follow-up rather than widen this PR.

Generated by Claude Code

@dangrondahl
dangrondahl merged commit 1ad9e19 into main Oct 29, 2026
11 checks passed
@dangrondahl
dangrondahl deleted the analytics-headers branch October 29, 2026 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants