Skip to content

ring: the completion queue takes its own size, ServerConfig.CqEntries, four times RingEntries by default - #386

Open
MDA2AV wants to merge 1 commit into
mainfrom
fix/95-cq-size
Open

MDA2AV wants to merge 1 commit into
mainfrom
fix/95-cq-size

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Oct 22, 2026

Copy link
Copy Markdown
Owner

Fixes #95.

The CQ was always the kernel's default, twice the SQ: 16384 entries at the default RingEntries of 8192. Nothing ties a reactor's completions to its submissions. A multishot recv or accept posts many from one submission. And the CQ keeps the batch being dispatched until the pass ends, so whatever the pass itself submits, and the kernel completes on the spot, lands behind it. A completion with no room waits in an allocated kernel list until the next enter, and a multishot recv or accept whose completion goes there ends and has to be re-armed.

ServerConfig.CqEntries now sets the CQ (IORING_SETUP_CQSIZE). The default, 0, is four times RingEntries, capped at the kernel's 65536, so no RingEntries gets a smaller CQ than before. An explicit value below RingEntries or above 65536 is refused at construction, by name, like the other sizes.

What can overflow depends on the kernel:

  • Since 6.13, one io_uring_enter runs at most 20 deferred completions. Measured here on 6.14: 2,000 and 20,000 armed recvs, all made ready at once, came out 20 per call. A burst of network events can't fill the CQ there. What can is a pass whose own submissions complete on the spot (sends with room in the socket buffer, page-cache reads) in greater numbers than the CQ has left behind the batch.
  • Before 6.13, every deferred completion runs in the next enter, so a burst across more connections than the CQ holds overflows it. That comes from the kernel source; it was not measured here.

Cost: 16 bytes an entry, charged like the rest of the ring, which on 6.14 means RLIMIT_MEMLOCK. At the defaults the ring goes from 772 KB to 1028 KB (measured by counting the rings that fit under two RLIMIT_MEMLOCK values: 771 KiB a ring before, 1028 KiB now, and 1542 KiB at a 65536-entry CQ). With its buffer ring a reactor is about 1.1 MB, so an 8 MB ulimit -l fits seven instead of ten. The README says so.

Not done: the issue's second suggestion, surfacing the mapped cq_off.overflow counter. It stayed 0 on 6.14 while 16 completions sat in the overflow list, because it counts only completions dropped for lack of memory, so it would always read 0. The live signal is IORING_SQ_CQ_OVERFLOW, already read as Reactor.CqOverflowed by the kTLS handoff.

Tests

New, in CompletionQueueTests (Unit, where Reactor.CqOverflowed is visible). A handler on a 16-entry SQ sends 48 single bytes in one pass. Every send completes as it is submitted, each time the SQ fills, behind the batch being dispatched.

  • ring: by default a pass that completes three times RingEntries fits the CQ
  • ring: control: the kernel's CQ, twice the SQ, overflows on the same pass, which proves the first one can fail.
reactor ignores CqEntries (the old CQ) this PR
by default FAIL: overflowed PASS
control, CqEntries = 32 PASS: overflows PASS: overflows

Changed:

  • HardeningTests' "an incremental recv ended by a full completion queue still hands over the peer's bytes" now sets CqEntries = 8. It needs an 8-entry CQ, which RingEntries = 4 used to imply and now would make 16.
  • RingSetupMessageTests refuses CqEntries below RingEntries and above 65536, and its control constructs at 65536.

API: adds ServerConfig.CqEntries and a Ring.Create(uint, uint) overload. Ring.Create(uint) keeps the kernel's default CQ. Nothing is removed or changed.

All suites pass: Unit 106, E2E 311, Chaos 80, Http 88, Tls 169 (+14 skipped), File 5, Pg 7 (+11 skipped), Redis (6 skipped, 1 pending). The skips need servers this box doesn't run.

Bench

Flat. Setup:

  • 2 reactors on two P-cores, wrk -t4 -c64 on other cores.
  • Every round runs each build back to back, in rotated order, against origin/main.
  • A second worktree of main is the noise control.
  • Figures are the median of the per-round ratios.
rounds req/s vs main CPU/req vs main control, main vs main
Tcp/Raw 10 × 10 s −0.1% +0.1% −0.4% / +0.7%
Tcp/Raw's handler, resumed on the thread pool after each read 5 × 10 s −0.5% +0.1% −1.8% / +0.1%

…, four times RingEntries by default

Ring.Create set no IORING_SETUP_CQSIZE, so the CQ was the kernel's default, twice the SQ, though a
reactor's completions are not bounded by its submissions: a multishot recv or accept posts many from
one, and the batch being dispatched holds its CQ slots until the pass ends, so what the pass itself
submits and the kernel completes on the spot lands behind it. A completion with no room waits in an
allocated kernel list, and a multishot recv or accept whose completion goes there ends.

CqEntries 0 (the default) is four times RingEntries, capped at the kernel's 65536, so no RingEntries
gets a smaller CQ than before; an explicit value below RingEntries or above 65536 is refused at
construction, by name. It costs 16 bytes an entry, charged like the rest of the ring: on 6.14 the ring
at the defaults takes 1028 KB of RLIMIT_MEMLOCK instead of 772 KB (README updated).

Tests (Unit, CompletionQueueTests): a handler on a 16-entry SQ sends 48 single bytes in one pass; with
the default CQ nothing overflows, with CqEntries 32, the old CQ, it does. Confirmed both ways: with the
reactor ignoring CqEntries the default test fails. HardeningTests' full-CQ test sets CqEntries 8, the
CQ RingEntries 4 used to give it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ring: set IORING_SETUP_CQSIZE - multishot decouples CQ pressure from SQ depth

1 participant