Skip to content

fix: preserve sync delivery results in compliance tests - #362

Merged
marandaneto merged 3 commits into
mainfrom
fix/local-harness-1.31.1-20261115
Nov 18, 2026
Merged

marandaneto merged 3 commits into
mainfrom
fix/local-harness-1.31.1-20261115

Conversation

@marandaneto

@marandaneto marandaneto commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

💡 Motivation and Context

Native synchronous capture returns delivery success, not queue acceptance. The compliance adapter treated a false delivery result as an adapter failure, so six retry/error cases stopped before checking SDK traffic. This PR acknowledges completed sync invocations while returning the native boolean in delivered. Failed delivery stays visible in request telemetry and last_error. Exceptions still return HTTP 500, and async enqueue failures remain errors.

Remote flag requests now explicitly serialize omitted/nil disable_geoip as geoip_disable: false. The backend already treats omitted/null as false. Explicit true and false remain unchanged, and the adapter does not inject defaults. This implements effective-value serialization in the canonical GeoIP contract, without adding client-level GeoIP configuration or changing Ruby's effective privacy default. A core SDK patch changeset is included. Broader client-level GeoIP configuration is not implemented by this PR.

The adapter advertises synchronous_capture only in sync mode. Native sync still sends one event per call immediately. It does not emulate queued batching or hide delivery errors. CI and Compose now use published harness 1.32.0, with the reusable workflow pinned to immutable release commit dde638562d33572b5c8ac3a2eafc98b92ffa2261. The successful release pins specs commit 53e9bc16d14f973809f80049864866d5b6d9bdff. The V1 image digest is sha256:7eb6c62e2241ddd48a2dfb326652e7933cd2fe26e0ffcad98cd385d603b4eb82.

Published 1.32.0 checks immediate delivery for sync with capture.batch_format.synchronous_capture_delivers_each_event_immediately. Async retains capture.batch_format.multiple_events_batched_together. The published harness selects between them using the existing synchronous_capture capability. This SDK update adds no filtering or capability changes. Each profile still selects 47 definitions (30 capture and 17 flags). Assertion failures remain advisory, and the inventory check only verifies completeness. Actual reports and harness step outcomes must be inspected before claiming compliance.

💚 How did you test it?

Fresh local validation with Ruby 4.0.7 and Bundler 4.0.13:

bundle exec rspec spec/sdk_compliance_adapter_spec.rb spec/posthog/feature_flag_evaluations_spec.rb spec/posthog/client_spec.rb spec/posthog/feature_flag_spec.rb
# 620 examples, 0 failures, 2 pending
bundle exec rubocop
# 168 files, no offenses
bundle exec rake public_api:check
# unchanged public API
actionlint .github/workflows/sdk-compliance.yml
docker compose -f sdk_compliance_adapter/docker-compose.yml config --quiet
git diff --check

The two pending focused cases require the optional OpenTelemetry bundle. Hosted CI covers the multi-Ruby and OpenTelemetry matrix. As a fresh regression check, running the adapter and GeoIP tests against pre-fix source produced 12 intended failures in 35 examples. Restoring the source fixes produced 35 examples with no failures.

Full-branch isolated autoreview passed for exact HEAD 7c75168eb836e60f4cc86b427e3cf32515061fd9 against latest origin/main at 7189e2a16d14eb99fc4d84b1ba7d2bacefb102f9. TruffleHog was clean. The isolated Claude Opus review reported no blocking/actionable findings. The existing source and test changes were reviewed anew, not just the pin update.

Hosted checks passed for exact pushed SHA 7c75168eb836e60f4cc86b427e3cf32515061fd9:

Profile Actual report Actual harness exit Cases
Async 47/47 passed 0 30 capture + 17 flags
Sync 47/47 passed 0 30 capture + 17 flags

Every report row matched the case IDs selected from the immutable release contracts. Logs confirmed async capabilities capture_v0, encoding_gzip and sync capabilities capture_v0, encoding_gzip, synchronous_capture. Both pulled the published V1 digest above. Both actual Run tests steps succeeded and skipped the failure-warning step, confirming successful harness exits despite advisory configuration. The sync-only immediate-delivery case passed, while async retained the queued-batching case. No SDK capability or test inventory was reduced to get these results.

To reproduce the published harness locally, run docker compose -f sdk_compliance_adapter/docker-compose.yml up --build --abort-on-container-exit. Set SDK_MODE=sync to exercise native sync mode.

📝 Checklist

  • All commits are signed (see Commit signing)
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

The patch changeset was authored directly in the repository's existing format.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

OpenAI Codex via Pi prepared the changes using local shell, Git, Ruby/Bundler and GitHub CLI. Pi's isolated autoreview helper reviewed the full committed diff with Claude Opus, with tools disabled and secret scanning enabled. No shareable session link is available. The human-directed scope preserves native delivery and privacy behavior rather than rebatching synchronous events or converting delivery failures into claimed success. Human review is required before merging.

Related PRs

The specs and harness dependencies are resolved by published 1.32.0. These SDK pins consume that real release, not a locally patched harness.

@marandaneto marandaneto self-assigned this Nov 15, 2026
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

posthog-ruby-sync Compliance Report

Date: 2026-11-17T08:59:48.112529+00:00
Duration: 94356ms

✅ All Tests Passed!

47/47 tests passed


Capture Tests

✅ 30/30 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 10ms
Format Validation.Event Has Uuid ✅ 7ms
Format Validation.Event Has Lib Properties ✅ 6ms
Format Validation.Distinct Id Is String ✅ 9ms
Format Validation.Token Is Present ✅ 7ms
Format Validation.Custom Properties Preserved ✅ 7ms
Format Validation.Event Has Timestamp ✅ 8ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 8ms
Retry Behavior.Retries On 503 ✅ 5314ms
Retry Behavior.Does Not Retry On 400 ✅ 2011ms
Retry Behavior.Does Not Retry On 401 ✅ 2010ms
Retry Behavior.Respects Retry After Header ✅ 8017ms
Retry Behavior.Implements Backoff ✅ 15318ms
Retry Behavior.Retries On 500 ✅ 5116ms
Retry Behavior.Retries On 502 ✅ 5145ms
Retry Behavior.Retries On 504 ✅ 5119ms
Retry Behavior.Max Retries Respected ✅ 15469ms
Deduplication.Generates Unique Uuids ✅ 20ms
Deduplication.Preserves Uuid On Retry ✅ 5164ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10286ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5159ms
Deduplication.No Duplicate Events In Batch ✅ 20ms
Deduplication.Different Events Have Different Uuids ✅ 11ms
Compression.Sends Gzip When Enabled ✅ 6ms
Batch Format.Uses Proper Batch Structure ✅ 6ms
Batch Format.Flush With No Events Sends Nothing ✅ 3ms
Batch Format.Synchronous Capture Delivers Each Event Immediately ✅ 17ms
Error Handling.Does Not Retry On 403 ✅ 2008ms
Error Handling.Does Not Retry On 413 ✅ 2010ms
Error Handling.Retries On 408 ✅ 5115ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 11ms
Request Payload.Flags Request Uses V2 Query Param ✅ 8ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 9ms
Request Payload.Flags Request Omits Authorization Header ✅ 9ms
Request Payload.Token In Flags Body Matches Init ✅ 7ms
Request Payload.Groups Round Trip ✅ 6ms
Request Payload.Groups Default To Empty Object ✅ 7ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 7ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 6ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 7ms
Request Lifecycle.No Flags Request On Init Alone ✅ 3ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 6ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 9ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 7ms
Retry Behavior.Retries Flags On 502 ✅ 148ms
Retry Behavior.Retries Flags On 504 ✅ 112ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 8ms

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

posthog-ruby-async Compliance Report

Date: 2026-11-17T09:00:01.025545+00:00
Duration: 98341ms

✅ All Tests Passed!

47/47 tests passed


Capture Tests

✅ 30/30 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 106ms
Format Validation.Event Has Uuid ✅ 105ms
Format Validation.Event Has Lib Properties ✅ 106ms
Format Validation.Distinct Id Is String ✅ 104ms
Format Validation.Token Is Present ✅ 105ms
Format Validation.Custom Properties Preserved ✅ 105ms
Format Validation.Event Has Timestamp ✅ 105ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 4ms
Retry Behavior.Retries On 503 ✅ 5410ms
Retry Behavior.Does Not Retry On 400 ✅ 2109ms
Retry Behavior.Does Not Retry On 401 ✅ 2108ms
Retry Behavior.Respects Retry After Header ✅ 8114ms
Retry Behavior.Implements Backoff ✅ 15521ms
Retry Behavior.Retries On 500 ✅ 5213ms
Retry Behavior.Retries On 502 ✅ 5211ms
Retry Behavior.Retries On 504 ✅ 5212ms
Retry Behavior.Max Retries Respected ✅ 15522ms
Deduplication.Generates Unique Uuids ✅ 109ms
Deduplication.Preserves Uuid On Retry ✅ 5210ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10316ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5213ms
Deduplication.No Duplicate Events In Batch ✅ 109ms
Deduplication.Different Events Have Different Uuids ✅ 105ms
Compression.Sends Gzip When Enabled ✅ 105ms
Batch Format.Uses Proper Batch Structure ✅ 104ms
Batch Format.Flush With No Events Sends Nothing ✅ 2ms
Batch Format.Multiple Events Batched Together ✅ 109ms
Error Handling.Does Not Retry On 403 ✅ 2107ms
Error Handling.Does Not Retry On 413 ✅ 2108ms
Error Handling.Retries On 408 ✅ 5212ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 106ms
Request Payload.Flags Request Uses V2 Query Param ✅ 105ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 105ms
Request Payload.Flags Request Omits Authorization Header ✅ 105ms
Request Payload.Token In Flags Body Matches Init ✅ 104ms
Request Payload.Groups Round Trip ✅ 105ms
Request Payload.Groups Default To Empty Object ✅ 104ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 105ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 104ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 105ms
Request Lifecycle.No Flags Request On Init Alone ✅ 2ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 105ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 110ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 105ms
Retry Behavior.Retries Flags On 502 ✅ 208ms
Retry Behavior.Retries Flags On 504 ✅ 243ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 109ms

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

SDK performance: posthog-ruby

Benchmarks completed. No changes flagged for review.

Flags require at least a 5% change in the median and all six pairs agreeing on direction. This is a noise filter, not proof of a performance change or a CI gate.

View all results
Test Measurement Before PR With PR Change Assessment
Capture and deliver events (32-byte payload, burst)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Capture and deliver events (32-byte payload, steady)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Queue an event (32-byte payload)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background serialization has no public enqueue completion boundary; use completed-delivery workloads for process-wide allocation counts.​; head:​ Background serialization has no public enqueue completion boundary; use completed-delivery workloads for process-wide allocation counts.​
Queue an event (32-byte payload)
Shard: capture
Allocations per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background serialization has no public enqueue completion boundary; use completed-delivery workloads for process-wide allocation counts.​; head:​ Background serialization has no public enqueue completion boundary; use completed-delivery workloads for process-wide allocation counts.​
Queue an event (4,096-byte payload)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background serialization has no public enqueue completion boundary; use completed-delivery workloads for process-wide allocation counts.​; head:​ Background serialization has no public enqueue completion boundary; use completed-delivery workloads for process-wide allocation counts.​
Queue an event (4,096-byte payload)
Shard: capture
Allocations per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background serialization has no public enqueue completion boundary; use completed-delivery workloads for process-wide allocation counts.​; head:​ Background serialization has no public enqueue completion boundary; use completed-delivery workloads for process-wide allocation counts.​
Evaluate all 10 flags
Shard: flags-scaling
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate all 100 flags
Shard: flags-scaling
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate all 1,000 flags
Shard: flags-scaling
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate one boolean flag
Shard: flags-single
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate a cohort-filtered flag
Shard: flags-single
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate a flag (early match)
Shard: flags-single
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate a flag (late match)
Shard: flags-single
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate a flag (no match)
Shard: flags-single
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate a property-filtered flag
Shard: flags-single
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate a regex-filtered flag
Shard: flags-single
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate a rollout flag
Shard: flags-single
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Evaluate a multivariate flag
Shard: flags-single
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​; head:​ Ruby has no cumulative allocated-byte counter here; retained ObjectSpace bytes are not allocation volume.​
Capture and deliver events (32-byte payload, burst)
Shard: capture
Allocations per operation 56.27 count/op 56.27 count/op No observed change Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Time per operation 108.81 µs/op 108.78 µs/op <0.1% less time Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Idle process memory (RSS) 28.38 MiB 28.34 MiB 0.1% less memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Observed peak process memory (RSS) 31.15 MiB 31.12 MiB 0.1% less memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Managed heap after settling 5.3 MiB 5.3 MiB <0.1% less memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Process memory after settling (RSS) 31.15 MiB 31.12 MiB 0.1% less memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Process CPU time per operation 19.53 µs/op 19.54 µs/op <0.1% more time Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Lifetime peak process memory (RSS) 31.27 MiB 31.25 MiB <0.1% less memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Allocations per operation 58.65 count/op 58.65 count/op No observed change Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Time per operation 108.68 µs/op 108.68 µs/op <0.1% more time Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Idle process memory (RSS) 33.17 MiB 33.21 MiB 0.1% more memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Observed peak process memory (RSS) 42.25 MiB 42.07 MiB 0.4% less memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Managed heap after settling 9.17 MiB 9.17 MiB <0.1% more memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Process memory after settling (RSS) 39.2 MiB 39.43 MiB 0.6% more memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Process CPU time per operation 27.31 µs/op 27.12 µs/op 0.7% less time Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Lifetime peak process memory (RSS) 41.98 MiB 41.88 MiB 0.2% less memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Allocations per operation 58.68 count/op 58.68 count/op No observed change Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Time per operation 1.01 ms/op 1.02 ms/op <0.1% more time Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Idle process memory (RSS) 28.3 MiB 28.28 MiB <0.1% less memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Observed peak process memory (RSS) 30.44 MiB 30.39 MiB 0.2% less memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Managed heap after settling 5.3 MiB 5.3 MiB <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Process memory after settling (RSS) 30.44 MiB 30.39 MiB 0.2% less memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Process CPU time per operation 30.88 µs/op 31.03 µs/op 0.5% more time Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Lifetime peak process memory (RSS) 30.44 MiB 30.39 MiB 0.2% less memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Allocations per operation 58.68 count/op 58.68 count/op <0.1% more Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Time per operation 1.01 ms/op 1.01 ms/op <0.1% less time Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Idle process memory (RSS) 33.24 MiB 33.23 MiB <0.1% less memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Observed peak process memory (RSS) 40.94 MiB 40.88 MiB 0.2% less memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Managed heap after settling 9.17 MiB 9.17 MiB <0.1% less memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Process memory after settling (RSS) 40.94 MiB 39.2 MiB 4.3% less memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Process CPU time per operation 36.43 µs/op 35.57 µs/op 2.4% less time Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Lifetime peak process memory (RSS) 40.94 MiB 41.06 MiB 0.3% more memory Not flagged
Queue an event (32-byte payload)
Shard: capture
Caller-thread CPU time per operation 8.57 µs/op 8.54 µs/op 0.3% less time Not flagged
Queue an event (32-byte payload)
Shard: capture
Time per operation 8.57 µs/op 8.54 µs/op 0.3% less time Not flagged
Queue an event (4,096-byte payload)
Shard: capture
Caller-thread CPU time per operation 8.52 µs/op 8.51 µs/op 0.1% less time Not flagged
Queue an event (4,096-byte payload)
Shard: capture
Time per operation 8.52 µs/op 8.51 µs/op 0.1% less time Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Allocations per operation 343.02 count/op 343.02 count/op No observed change Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Time per operation 123.88 µs/op 124.37 µs/op 0.4% more time Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Process CPU time per operation 123.88 µs/op 124.33 µs/op 0.4% more time Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Allocations per operation 3,324 count/op 3,324 count/op No observed change Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Time per operation 1.2 ms/op 1.19 ms/op 0.6% less time Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Process CPU time per operation 1.2 ms/op 1.19 ms/op 0.6% less time Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Allocations per operation 33,113.56 count/op 33,113.62 count/op <0.1% more Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Time per operation 12.17 ms/op 11.99 ms/op 1.5% less time Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Process CPU time per operation 12.17 ms/op 11.99 ms/op 1.5% less time Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Idle process memory (RSS) 38.91 MiB 38.9 MiB <0.1% less memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Observed peak process memory (RSS) 40.39 MiB 40.27 MiB 0.3% less memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Managed heap after settling 7.99 MiB 7.99 MiB <0.1% less memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Process memory after settling (RSS) 39.26 MiB 39.24 MiB <0.1% less memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Lifetime peak process memory (RSS) 40.18 MiB 40 MiB 0.4% less memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Idle process memory (RSS) 37.39 MiB 38.05 MiB 1.8% more memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Observed peak process memory (RSS) 42.29 MiB 42.27 MiB <0.1% less memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Managed heap after settling 9.5 MiB 9.5 MiB <0.1% less memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Process memory after settling (RSS) 41.4 MiB 41.37 MiB <0.1% less memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Lifetime peak process memory (RSS) 42.13 MiB 42.13 MiB <0.1% less memory Not flagged
Evaluate one boolean flag
Shard: flags-single
Allocations per operation 38 count/op 38 count/op No observed change Not flagged
Evaluate one boolean flag
Shard: flags-single
Time per operation 10.14 µs/op 10.11 µs/op 0.3% less time Not flagged
Evaluate one boolean flag
Shard: flags-single
Process CPU time per operation 10.14 µs/op 10.11 µs/op 0.3% less time Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Allocations per operation 38 count/op 38 count/op No observed change Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Time per operation 13.8 µs/op 13.83 µs/op 0.2% more time Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Process CPU time per operation 13.8 µs/op 13.83 µs/op 0.2% more time Not flagged
Evaluate a flag (early match)
Shard: flags-single
Allocations per operation 55 count/op 55 count/op No observed change Not flagged
Evaluate a flag (early match)
Shard: flags-single
Time per operation 17.55 µs/op 17.5 µs/op 0.3% less time Not flagged
Evaluate a flag (early match)
Shard: flags-single
Process CPU time per operation 17.54 µs/op 17.5 µs/op 0.3% less time Not flagged
Evaluate a flag (late match)
Shard: flags-single
Allocations per operation 76 count/op 76 count/op No observed change Not flagged
Evaluate a flag (late match)
Shard: flags-single
Time per operation 24.84 µs/op 24.74 µs/op 0.4% less time Not flagged
Evaluate a flag (late match)
Shard: flags-single
Process CPU time per operation 24.84 µs/op 24.74 µs/op 0.4% less time Not flagged
Evaluate a flag (no match)
Shard: flags-single
Allocations per operation 67 count/op 67 count/op No observed change Not flagged
Evaluate a flag (no match)
Shard: flags-single
Time per operation 22.19 µs/op 22.24 µs/op 0.2% more time Not flagged
Evaluate a flag (no match)
Shard: flags-single
Process CPU time per operation 22.19 µs/op 22.24 µs/op 0.2% more time Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Allocations per operation 34 count/op 34 count/op No observed change Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Time per operation 10.4 µs/op 10.36 µs/op 0.4% less time Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Process CPU time per operation 10.4 µs/op 10.36 µs/op 0.4% less time Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Allocations per operation 99 count/op 99 count/op <0.1% fewer Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Time per operation 30.36 µs/op 30.19 µs/op 0.6% less time Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Process CPU time per operation 30.36 µs/op 30.19 µs/op 0.6% less time Not flagged
Evaluate a rollout flag
Shard: flags-single
Allocations per operation 31 count/op 31 count/op No observed change Not flagged
Evaluate a rollout flag
Shard: flags-single
Time per operation 9.11 µs/op 9.13 µs/op 0.2% more time Not flagged
Evaluate a rollout flag
Shard: flags-single
Process CPU time per operation 9.11 µs/op 9.13 µs/op 0.2% more time Not flagged
Evaluate a multivariate flag
Shard: flags-single
Allocations per operation 37 count/op 37 count/op No observed change Not flagged
Evaluate a multivariate flag
Shard: flags-single
Time per operation 11.34 µs/op 11.35 µs/op <0.1% more time Not flagged
Evaluate a multivariate flag
Shard: flags-single
Process CPU time per operation 11.33 µs/op 11.34 µs/op <0.1% more time Not flagged

Values are medians of paired measurements. Not flagged does not mean unchanged. Unavailable means a measurement could not be provided, not zero. Estimated allocation measurements retain their instrumentation uncertainty. RSS measures the whole process, not just SDK allocations; observed peaks are sampled checkpoints. Paired ranges in the diagnostics describe variability, not confidence intervals.

Each shard uses its own runner; compare Before PR against With PR within a row, not across shards.

Compared 7189e2a16d14eb99fc4d84b1ba7d2bacefb102f9 → 7c75168eb836e60f4cc86b427e3cf32515061fd9.

Full report and diagnostics

102 metric rows; 102 rows shown in all results. Omitted or shortened data remain in comparison.json, including per-runner environments, resource limits, measurement methods, samples and checkpoints.

@marandaneto
marandaneto marked this pull request as ready for review November 17, 2026 09:39
@marandaneto
marandaneto requested a review from a team as a code owner November 17, 2026 09:39
@veria-ai

veria-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

PR overview

No security issues were found in this pull request.

Reviews

@marandaneto
marandaneto merged commit 53cdca9 into main Nov 18, 2026
33 of 34 checks passed
@marandaneto
marandaneto deleted the fix/local-harness-1.31.1-20261115 branch November 18, 2026 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants