bench: add async pipeline research instrumentation - #151
Conversation
2ab17ae to
21f3eee
Compare
|
Publication-sized local validation on Intel Xeon E5-2696 v3 / MSVC 19.44 / Release C++17 (commit 21f3eee, comparable metadata enabled):\n\n- Matrix: 4 producers, 200-byte payload, 200000 messages, warmup 4096, queue 1024/8192/65536/400000, 5 repeats per point, 160 rows total. At queue=400000: LogIt++ median sink p50 2.2 us, producer p50 1.5 us, throughput 0.872 M/s; spdlog median sink p50 51.9 ms, producer p50 0.7 us, throughput 1.036 M/s.\n- Rate mode: 100k/250k/500k/750k/1M msg/s, 5 repeats each, 50 rows. At 750k offered load spdlog median sink p50 27.4 ms and backlog high-water is tens of thousands; LogIt++ sink p50 1.6 us and throughput tracks ~750k. At 1M spdlog sink p50 67.4 ms vs LogIt++ 2.1 us, with LogIt++ throughput ~0.894 M/s.\n\nRaw receipts remain local under tmp/pipeline-publication-{matrix,rate}*.csv/.jsonl; every row had submitted == sink_completed == total. These are saturated workload observations, not intrinsic latency or universal speed claims. |
Add a library-neutral async pipeline research target with producer-call latency, sink-entry timing, drain phases, backlog telemetry, deterministic alternating order, and rate-controlled workloads. Preserve raw CSV/JSONL receipts and document the saturated-workload interpretation.
21f3eee to
472fc1f
Compare
Cover the research target with CI smoke runs, preserve the original sink-entry timestamp with a shared call-start sample, expose target versus realized pacing and schedule lag, use benchmark-issued terminology, and aggregate the full stage/backlog metrics. Alternate libraries within each matched key point.
|
Corrective pass pushed as d892082. Key methodology fixes:\n\n- Linux C++17 CI now builds and runs both matrix/rate research smoke modes, checks receipt creation, and asserts issued == sink_completed == total.\n- offered_rate is now arget_rate; receipts include realized submission rate and schedule lag p50/p99/max. Rate mode is documented as finite-producer target pacing, not guaranteed open-loop arrival.\n- Producer and sink recorders share one call-start timestamp, preserving the original enqueue-to-sink-entry semantics without hidden recorder/telemetry skew.\n- submitted is now issued; outstanding is explicitly benchmark-issued minus sink-completed, not private queue depth.\n- Library order alternates inside each matched key point, so LogIt++/spdlog runs are adjacent.\n- Aggregate receipt now includes producer/sink p50/p99, producer phase, drain tail, wall time, realized rate, schedule lag, and outstanding summaries.\n\nExact-head Linux C++17 already passed, including the new research smoke steps; remaining matrix jobs are still running. |
Clarify that research sink latency shares a common call-start timestamp but includes the small recorder reservation and telemetry overhead before adapter.log(). Keep it distinct from older matched-benchmark absolute latency.
|
Final methodology wording fix pushed as 76a0478 (docs-only). The research sink p50 is now explicitly documented as an instrumented sink-entry metric: producer/sink share one call-start timestamp, but reservation and telemetry work between that timestamp and adapter.log() remain part of the measured interval. It is not presented as identical to older matched-benchmark absolute latency. No code or CI semantics changed in this final corrective commit. |
Summary\n\n- add a benchmark-only contract-matched async/null pipeline research target\n- record producer-call, sink-entry, producer-phase, drain-tail, wall-time, throughput, and common backlog telemetry\n- add deterministic alternating library order and rate-controlled workloads\n- retain raw CSV/JSONL receipts and aggregate output; document interpretation and limitations\n\nNo production LogIt++ implementation is changed.\n\nValidation:\n- Release/MSVC build of logit_bench_pipeline_research\n- default benchmark validation and async payload contract tests\n- publication-sized matrix: 160 rows (4 producers x 4 queues x 2 libraries x 5 repeats)\n- rate-controlled run: 50 rows (5 rates x 2 libraries x 5 repeats)\n- all submitted and sink-completed counters matched for every row\n