Skip to content

bench: add async pipeline research instrumentation - #151

Merged
LimiNode merged 3 commits into
mainfrom
research/async-pipeline
Oct 3, 2026
Merged

LimiNode merged 3 commits into
mainfrom
research/async-pipeline

Conversation

@LimiNode

@LimiNode LimiNode commented Oct 2, 2026

Copy link
Copy Markdown
Owner

Summary\n\n- add a benchmark-only contract-matched async/null pipeline research target\n- record producer-call, sink-entry, producer-phase, drain-tail, wall-time, throughput, and common backlog telemetry\n- add deterministic alternating library order and rate-controlled workloads\n- retain raw CSV/JSONL receipts and aggregate output; document interpretation and limitations\n\nNo production LogIt++ implementation is changed.\n\nValidation:\n- Release/MSVC build of logit_bench_pipeline_research\n- default benchmark validation and async payload contract tests\n- publication-sized matrix: 160 rows (4 producers x 4 queues x 2 libraries x 5 repeats)\n- rate-controlled run: 50 rows (5 rates x 2 libraries x 5 repeats)\n- all submitted and sink-completed counters matched for every row\n

@LimiNode
LimiNode force-pushed the research/async-pipeline branch from 2ab17ae to 21f3eee Compare October 2, 2026 23:10
@LimiNode

LimiNode commented Oct 2, 2026

Copy link
Copy Markdown
Owner Author

Publication-sized local validation on Intel Xeon E5-2696 v3 / MSVC 19.44 / Release C++17 (commit 21f3eee, comparable metadata enabled):\n\n- Matrix: 4 producers, 200-byte payload, 200000 messages, warmup 4096, queue 1024/8192/65536/400000, 5 repeats per point, 160 rows total. At queue=400000: LogIt++ median sink p50 2.2 us, producer p50 1.5 us, throughput 0.872 M/s; spdlog median sink p50 51.9 ms, producer p50 0.7 us, throughput 1.036 M/s.\n- Rate mode: 100k/250k/500k/750k/1M msg/s, 5 repeats each, 50 rows. At 750k offered load spdlog median sink p50 27.4 ms and backlog high-water is tens of thousands; LogIt++ sink p50 1.6 us and throughput tracks ~750k. At 1M spdlog sink p50 67.4 ms vs LogIt++ 2.1 us, with LogIt++ throughput ~0.894 M/s.\n\nRaw receipts remain local under tmp/pipeline-publication-{matrix,rate}*.csv/.jsonl; every row had submitted == sink_completed == total. These are saturated workload observations, not intrinsic latency or universal speed claims.

Add a library-neutral async pipeline research target with producer-call latency, sink-entry timing, drain phases, backlog telemetry, deterministic alternating order, and rate-controlled workloads. Preserve raw CSV/JSONL receipts and document the saturated-workload interpretation.
@LimiNode
LimiNode force-pushed the research/async-pipeline branch from 21f3eee to 472fc1f Compare October 2, 2026 23:13
Cover the research target with CI smoke runs, preserve the original sink-entry timestamp with a shared call-start sample, expose target versus realized pacing and schedule lag, use benchmark-issued terminology, and aggregate the full stage/backlog metrics. Alternate libraries within each matched key point.
@LimiNode

LimiNode commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Corrective pass pushed as d892082. Key methodology fixes:\n\n- Linux C++17 CI now builds and runs both matrix/rate research smoke modes, checks receipt creation, and asserts issued == sink_completed == total.\n- offered_rate is now arget_rate; receipts include realized submission rate and schedule lag p50/p99/max. Rate mode is documented as finite-producer target pacing, not guaranteed open-loop arrival.\n- Producer and sink recorders share one call-start timestamp, preserving the original enqueue-to-sink-entry semantics without hidden recorder/telemetry skew.\n- submitted is now issued; outstanding is explicitly benchmark-issued minus sink-completed, not private queue depth.\n- Library order alternates inside each matched key point, so LogIt++/spdlog runs are adjacent.\n- Aggregate receipt now includes producer/sink p50/p99, producer phase, drain tail, wall time, realized rate, schedule lag, and outstanding summaries.\n\nExact-head Linux C++17 already passed, including the new research smoke steps; remaining matrix jobs are still running.

Clarify that research sink latency shares a common call-start timestamp but includes the small recorder reservation and telemetry overhead before adapter.log(). Keep it distinct from older matched-benchmark absolute latency.
@LimiNode

LimiNode commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Final methodology wording fix pushed as 76a0478 (docs-only). The research sink p50 is now explicitly documented as an instrumented sink-entry metric: producer/sink share one call-start timestamp, but reservation and telemetry work between that timestamp and adapter.log() remain part of the measured interval. It is not presented as identical to older matched-benchmark absolute latency. No code or CI semantics changed in this final corrective commit.

@LimiNode
LimiNode merged commit b629202 into main Oct 3, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant