Skip to content

kv-compare: cold-cache read benchmark, and a Graviton4 + gp3 run - #172

Merged
qdequele merged 1 commit into
review-fixesfrom
perf/cold-bench-and-hook
Nov 12, 2026
Merged

qdequele merged 1 commit into
review-fixesfrom
perf/cold-bench-and-hook

Conversation

@qdequele

Copy link
Copy Markdown
Owner

Stacked on #171 (base review-fixes). Benchmark tooling and results only: no engine code changes.

What this adds

  • benches/kv-compare/cold.sh measures the first reads of a process with nothing cached, on a data set several times larger than its memory limit. It runs LMDB and ZeroDB, with two settings as variables: ZeroDB's page size (--page-kib) and kernel read-ahead (--read-ahead). Each measurement runs in its own process and memory-limited cgroup, right after a page-cache drop, with no warm-up.
  • Disk-read counting. Each result records the block device's read requests and bytes, so the cost of one lookup is visible in disk reads. cold-report.py prints the table.
  • Readers read every value byte. The LMDB and ZeroDB readers used to take only the value's length. A value stored in an overflow page then cost no disk read, so large-value reads measured only the tree descent. This also changes the existing warm kv-compare phases: earlier kv-compare numbers for large values are not comparable with new ones.
  • A report: benches/results/2026-11-12-cold-cache-reads-graviton-gp3.md.
  • A perf-ledger entry for an experiment that did not work (below).

Cold-cache results

Graviton4 c8g.xlarge, EBS gp3 at 3,000 IOPS, 1 GiB memory limit, about 5.5 GiB of data. One 60-second run per cell.

Point reads without read-ahead, reads/s:

Variant 100 B, 1 thread 100 B, 8 threads 3,000 B, 1 thread 3,000 B, 8 threads Disk reads per get
LMDB, 4 KiB pages 1,622 3,011 1,519 2,912 1.0–1.2
ZeroDB, 4 KiB pages 1,566 2,986 1,515 2,910 1.0–1.2
ZeroDB, 16 KiB pages 586 1,058 578 1,039 2.9–3.1

Range scans of 100 keys, one thread, scans/s:

Variant 100 B, read-ahead off 100 B, read-ahead on 3,000 B, read-ahead off 3,000 B, read-ahead on
LMDB, 4 KiB pages 306 191 18 12
ZeroDB, 4 KiB pages 304 190 18 12
ZeroDB, 16 KiB pages 238 563 17 42
  • At the same page size ZeroDB and LMDB are equal: within 3.5 % on every pair of rows. The eight-thread rows sit at the volume's IOPS limit.
  • 16 KiB pages without read-ahead cost about three disk reads per get, because the kernel reads a 16 KiB database page as separate 4 KiB pages.
  • 16 KiB pages with read-ahead give the fastest range scans (1.8× and 2.3× the best 4 KiB setting).
  • Read-ahead halves random gets on 4 KiB pages, for both engines.

The experiment that did not work

I tried skipping the commit-hook lock when no hook is installed, expecting it to narrow the per-commit gap. just bench-ab on Graviton4 came back flat on every rung (speedup 0.995 to 1.004, LMDB drift at most 1.1 %), so the change is reverted and recorded in perf-ledger.jsonl. An empty write transaction returns before the first hook site, and five uncontended locks are under 1 % of a one-put commit.

Checks

  • cargo fmt --check and cargo clippy --release in benches/kv-compare (macOS): clean.
  • The harness ran end to end on Linux aarch64: 42 measurements, no failures.
  • Not tested: the last cold.sh change, which raises the kernel's dirty-page limits during the loads and restores them on exit. For the recorded run the limits were raised by hand; the script change is only syntax-checked.

Unrelated finding

just perf-ledger show fails on main with KeyError: 'ratio_vs_lmdb_before' on an older entry. Not touched here.

🤖 Generated with Claude Code

Add `cold.sh`: first reads of a process with nothing cached, on a data set
larger than its memory limit, for LMDB and ZeroDB, with ZeroDB's page size
and kernel read-ahead as variables. Each result records the block device's
read requests and bytes.

The LMDB and ZeroDB readers now read every byte of a value. Before, a value
stored in an overflow page cost no disk read, so large-value reads measured
only the tree descent.

Record the run on Graviton4 with EBS gp3, and the hook-cell experiment
(flat, reverted) in the perf ledger.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 695f8c75-3a8a-459d-9981-51ac7a2bac3b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@qdequele
qdequele merged commit f6fa243 into review-fixes Nov 12, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant