Repository navigation
refactor: reduce SDK complexity and improve CRAP analysis - #1242
Conversation
posthog-python Compliance ReportDate: 2026-11-13T08:57:41.221562+00:00 ✅ All Tests Passed!270/270 tests passed Capture_V1 Tests✅ 95/95 tests passed View Details
Capture_Ai Tests✅ 5/5 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
Feature_Flags_V4 Tests✅ 149/149 tests passed View Details
Feature_Flags_Local_Evaluation Tests✅ 4/4 tests passed View Details
|
|
Test changes are mostly additive, rather than changing expectations to accommodate the refactor:
No existing SDK tests were deleted or weakened to make the refactoring pass. The new SDK contract tests were also checked against the corresponding pre-refactor implementations to verify that they protect existing behavior. Separately, tooling tests reproduce the CRAP-scoring and API-discovery bugs before their fixes. The API-checker test entrypoint now runs the complete pytest suite instead of manually calling only its original two tests. |
# Conflicts: # posthog/ai/evaluations/_client.py # posthog/ai/evaluations/_types.py # posthog/emails.py # posthog/mcp/__init__.py # posthog/mcp/_instrumentation.py # posthog/test/test_emails.py
SDK performance: posthog-python40 changes worth reviewing: 4 observed improvements, 36 observed regressions. Flags require at least a 5% change in the median and all six pairs agreeing on direction. This is a noise filter, not proof of a performance change or a CI gate.
Showing the 12 largest flagged changes; 28 more are in all results. View all results
Values are medians of paired measurements. Not flagged does not mean unchanged. Unavailable means a measurement could not be provided, not zero. Estimated allocation measurements retain their instrumentation uncertainty. RSS measures the whole process, not just SDK allocations; observed peaks are sampled checkpoints. Paired ranges in the diagnostics describe variability, not confidence intervals. Each shard uses its own runner; compare Before PR against With PR within a row, not across shards. Compared 102 metric rows; 102 rows shown in all results. Omitted or shortened data remain in |
💡 Motivation and Context
This PR records the SDK-wide CRAP cleanup experiment. CRAP combines function complexity with test coverage. We wanted to measure the change required to keep every production function below 10, rather than checking only
utils.py.The existing calculator counted comments, docstrings, and blank lines as uncovered code. This fixes the checker to use executable statements, then refactors complex functions into private helpers and adds contract tests for coverage gaps. SDK behavior and public signatures are intended to remain unchanged. The relevant SDK specifications were checked, with no intentional behavior divergence.
Originally stacked on #1239, which has now merged. This PR now targets
main; mutation-test removal is excluded from its diff.Changes
Client.before_sendrecord. This does not introduce a runtime API.Measured result and rollout boundary
The initial combined-coverage measurement, before the latest main sync, covered all 143 production Python files. It reduced 226 CRAP failures to zero across 2,435 functions, with a maximum CRAP score of 9.79. Helper extraction added 378 functions; the initial refactor changed 69 production files and 25 test files.
After merging main at
f6f61834, production contains 144 files and 2,512 functions. Two unchanged upstream email helpers,_checked_attachmentand_pending_upload, have complexity 11 and 10 respectively. Full combined coverage has not been recollected after this sync, so the earlier zero-failure result is historical, not a claim about the current head.CI still enforces the corrected utils-only gate. This PR does not enable SDK-wide enforcement. Reproducing the SDK-wide result in CI requires merging coverage from the full suite and the MCP variants. That collection and merge step remains a follow-up.
💚 How did you test it?
Initial cleanup validation, before the latest main sync
make public_api_check, warning-as-error import, actionlint, andgit diff --checkpass.925fc71268364e89785c21e569528d71a810029dagainstorigin/chore/remove-mutation-testsand reported no actionable findings. An earlier cache-TTL finding was rejected after checking the unchanged final guard, comparing 10 inputs againstmain, and passing seven related LangChain tests. Missing TTL counts still returnNone.Latest main integration:
713be1f128911ae4741e484c5b1c5939286e48eaa8347842d4d2c60e199d02b982eb3ec5e4f487daincorporatesorigin/mainatf6f61834c8782e2b8c7ec6136af51c077d8bb22c, preserving composite evaluation uploads, email CC/BCC/attachments/tracking, and MCP raised-exception/cause-chain reporting. Both the merge and follow-up are signed; history was not rewritten.BaseExceptioninstead ofExceptionmakes it fail for the intended reason. Follow-up713be1f1changes only this test, not SDK behavior.origin/main: two review passes, no actionable findings; secret scanning was clean.713be1f128911ae4741e484c5b1c5939286e48ea: 51 passed, one neutral (Wiz Secret Scanner), no pending or failing checks. Main CI run, including every MCP/Python matrix entry, passed. A superseded PR-title lint run was cancelled; its replacement passed.📝 Checklist
If releasing new changes
sampo addto generate a changeset file🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Pi used Git, uv, pytest, Docker, GitHub CLI, and autoreview. Independent workers used separate worktrees, followed by read-only preservation reviews. The Pi session is local and has no published transcript link. Human review is required.