Repository navigation
fix: keep frozenset and deque values when cleaning event properties - #1230
Conversation
clean() turned set, list and tuple values into lists but sent frozenset and deque values as None, so those properties were silently lost.
|
@PostHog/team-client-libraries wonder if it's worth updating sdk-specs for container values? |
SDK performance: posthog-pythonBenchmarks completed. No changes flagged for review. Flags require at least a 5% change in the median and all six pairs agreeing on direction. This is a noise filter, not proof of a performance change or a CI gate. View all results
Values are medians of paired measurements. Not flagged does not mean unchanged. Unavailable means a measurement could not be provided, not zero. Estimated allocation measurements retain their instrumentation uncertainty. RSS measures the whole process, not just SDK allocations; observed peaks are sampled checkpoints. Paired ranges in the diagnostics describe variability, not confidence intervals. Each shard uses its own runner; compare Before PR against With PR within a row, not across shards. Compared 102 metric rows; 102 rows shown in all results. Omitted or shortened data remain in |
💡 Motivation and Context
utils.clean()runs on every captured event. It turnsset,listandtuplevalues into lists, butfrozensetandcollections.dequedon't match that check, fall through to_coerce_unicode(), and come back asNone. So a property sent as a frozenset or a deque is silently dropped tonull, while the same data sent as a set or a list arrives intact:The fix adds
frozensetanddequeto the container check inclean(), so they go through_clean_list()like the other containers and their items are cleaned the same way (Decimal to float, UUID to str, and so on). Nothing else changes: other unknown types still becomeNone.💚 How did you test it?
test_clean_containers_and_invalid_dict_valuesinposthog/test/test_utils.pywith a frozenset and a deque holding aDecimaland aUUID. It fails onmainand passes with the fix.Clientinsync_modewith onlybatch_postmocked: onmainboth properties arrive asNone; with the fix they arrive as lists.pytest posthog/test/test_utils.py --timeout=30: 106 passed, 2 skipped, 1 failed. The failure isTestFlagCache::test_lru_eviction, which fails the same way on an unchangedmainon my Windows machine (the test relies on two consecutivetime.time()calls returning different values, and Windows' clock resolution is coarser than that), so it isn't related to this change.pytest --timeout=30(Python 3.12, Windows 11): 6726 passed, 76 skipped, 6 failed. The same 6 tests fail on an unchangedmaincheckout in the same environment (test_client_forkx2 needos.register_at_fork, plustest_lru_eviction, the two Gemini basic-generation tests andtest_basic_capture_exception_with_no_exception_given), so none of them come from this change.ruff format --check,ruff check,mypy --no-site-packages --config-file mypy.ini posthog/utils.py | mypy-baseline filterandpython -W error -c "import posthog"all pass.📝 Checklist
If releasing new changes
.sampo/changesets/frozen-deque-clean.md, written in the formatsampo addproduces)🤖 Agent context
Autonomy: Human-driven (agent-assisted)
I directed the work and reviewed the diff; a Claude Code agent (Claude Opus) read
posthog/utils.py, reproduced the dropped values through the real client, and wrote the fix, test and changeset. I kept the change to the two container types that sit next toset/list/tupleinstead of widening the check to every iterable, because that would also catchbytes,bytearray,rangeand generators and change how they are sent today.