Repository navigation
fix: preserve sync delivery results in compliance tests - #362
Conversation
posthog-ruby-sync Compliance ReportDate: 2026-11-17T08:59:48.112529+00:00 ✅ All Tests Passed!47/47 tests passed Capture Tests✅ 30/30 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
|
posthog-ruby-async Compliance ReportDate: 2026-11-17T09:00:01.025545+00:00 ✅ All Tests Passed!47/47 tests passed Capture Tests✅ 30/30 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
|
SDK performance: posthog-rubyBenchmarks completed. No changes flagged for review. Flags require at least a 5% change in the median and all six pairs agreeing on direction. This is a noise filter, not proof of a performance change or a CI gate. View all results
Values are medians of paired measurements. Not flagged does not mean unchanged. Unavailable means a measurement could not be provided, not zero. Estimated allocation measurements retain their instrumentation uncertainty. RSS measures the whole process, not just SDK allocations; observed peaks are sampled checkpoints. Paired ranges in the diagnostics describe variability, not confidence intervals. Each shard uses its own runner; compare Before PR against With PR within a row, not across shards. Compared 102 metric rows; 102 rows shown in all results. Omitted or shortened data remain in |
PR overviewNo security issues were found in this pull request. Reviews
|
💡 Motivation and Context
Native synchronous capture returns delivery success, not queue acceptance. The compliance adapter treated a false delivery result as an adapter failure, so six retry/error cases stopped before checking SDK traffic. This PR acknowledges completed sync invocations while returning the native boolean in
delivered. Failed delivery stays visible in request telemetry andlast_error. Exceptions still return HTTP 500, and async enqueue failures remain errors.Remote flag requests now explicitly serialize omitted/nil
disable_geoipasgeoip_disable: false. The backend already treats omitted/null as false. Explicit true and false remain unchanged, and the adapter does not inject defaults. This implements effective-value serialization in the canonical GeoIP contract, without adding client-level GeoIP configuration or changing Ruby's effective privacy default. A core SDK patch changeset is included. Broader client-level GeoIP configuration is not implemented by this PR.The adapter advertises
synchronous_captureonly in sync mode. Native sync still sends one event per call immediately. It does not emulate queued batching or hide delivery errors. CI and Compose now use published harness 1.32.0, with the reusable workflow pinned to immutable release commitdde638562d33572b5c8ac3a2eafc98b92ffa2261. The successful release pins specs commit53e9bc16d14f973809f80049864866d5b6d9bdff. The V1 image digest issha256:7eb6c62e2241ddd48a2dfb326652e7933cd2fe26e0ffcad98cd385d603b4eb82.Published 1.32.0 checks immediate delivery for sync with
capture.batch_format.synchronous_capture_delivers_each_event_immediately. Async retainscapture.batch_format.multiple_events_batched_together. The published harness selects between them using the existingsynchronous_capturecapability. This SDK update adds no filtering or capability changes. Each profile still selects 47 definitions (30 capture and 17 flags). Assertion failures remain advisory, and the inventory check only verifies completeness. Actual reports and harness step outcomes must be inspected before claiming compliance.💚 How did you test it?
Fresh local validation with Ruby 4.0.7 and Bundler 4.0.13:
The two pending focused cases require the optional OpenTelemetry bundle. Hosted CI covers the multi-Ruby and OpenTelemetry matrix. As a fresh regression check, running the adapter and GeoIP tests against pre-fix source produced 12 intended failures in 35 examples. Restoring the source fixes produced 35 examples with no failures.
Full-branch isolated autoreview passed for exact HEAD
7c75168eb836e60f4cc86b427e3cf32515061fd9against latestorigin/mainat7189e2a16d14eb99fc4d84b1ba7d2bacefb102f9. TruffleHog was clean. The isolated Claude Opus review reported no blocking/actionable findings. The existing source and test changes were reviewed anew, not just the pin update.Hosted checks passed for exact pushed SHA
7c75168eb836e60f4cc86b427e3cf32515061fd9:Every report row matched the case IDs selected from the immutable release contracts. Logs confirmed async capabilities
capture_v0, encoding_gzipand sync capabilitiescapture_v0, encoding_gzip, synchronous_capture. Both pulled the published V1 digest above. Both actualRun testssteps succeeded and skipped the failure-warning step, confirming successful harness exits despite advisory configuration. The sync-only immediate-delivery case passed, while async retained the queued-batching case. No SDK capability or test inventory was reduced to get these results.To reproduce the published harness locally, run
docker compose -f sdk_compliance_adapter/docker-compose.yml up --build --abort-on-container-exit. SetSDK_MODE=syncto exercise native sync mode.📝 Checklist
If releasing new changes
pnpm changesetto generate a changeset fileThe patch changeset was authored directly in the repository's existing format.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
OpenAI Codex via Pi prepared the changes using local shell, Git, Ruby/Bundler and GitHub CLI. Pi's isolated autoreview helper reviewed the full committed diff with Claude Opus, with tools disabled and secret scanning enabled. No shareable session link is available. The human-directed scope preserves native delivery and privacy behavior rather than rebatching synchronous events or converting delivery failures into claimed success. Human review is required before merging.
Related PRs
The specs and harness dependencies are resolved by published 1.32.0. These SDK pins consume that real release, not a locally patched harness.