Repository navigation
fix(moq-uring): keep the rings region within order 3 so setup survives fragmentation - #8718
Conversation
Quest-Claim: 9a6873536af6b893
…s fragmentation Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
ℹ️ No critical issues — one stale doc outside the diff.
Reviewed changes
Reviewed the full diff: shrinking the production io_uring CQ so the rings region stays within order 3, the matching tx pool ceiling, and the reworded ENOMEM message.
- CQ 2048 → 1024:
PRODUCTION_CQ_ENTRIEShalves, so the rings region drops from 33,856 B (order 4) to 17,472 B (order 3, thePAGE_ALLOC_COSTLY_ORDERbound the allocator still reclaims for). The arithmetic inthe_rings_region_stays_within_order_3matches v7.3rings_sizewith the 64 B header the author measured. tx_buffers_max1024 → 512: keepscq_covers_the_default_pool_ceilingstrue (512 + 256 < 1024).PRODUCTION_SQ_ENTRIES: split out so the new test reads the production SQ depth undercfg(test).Error::ringmessage: lists the memory cgroup and fragmented contiguous pages beforeRLIMIT_MEMLOCK, and the test checks all three.- Quest bookkeeping: removes the finished quest and updates the epic README.
ℹ️ rs/moq-uring/README.md still describes the 2048-entry geometry
The "Parallel test runs" section says unit tests shrink from "the production 256/2048 geometry" (line 159). It also says "A production-geometry worker takes 52 KiB" (line 167). Both now overstate the production ring. With this PR it is 256/1024, and the locked-memory footprint is about 36 KiB: 5 pages of rings plus 4 pages of SQEs, instead of 9 + 4. Anyone working out how many workers fit under RLIMIT_MEMLOCK would get it wrong.
Technical details
# Stale production ring geometry in moq-uring README
## Affected sites
- rs/moq-uring/README.md:159 — "256/2048 geometry" should be 256/1024
- rs/moq-uring/README.md:167 — "A production-geometry worker takes 52 KiB" should be ~36 KiB (rings 17,472 B → 5 pages, SQEs 256×64 = 16 KiB → 4 pages)
## Required outcome
- README numbers match `PRODUCTION_CQ_ENTRIES = 1024` / `PRODUCTION_SQ_ENTRIES = 256`.claude-opus-5-5 | 𝕏
|
No actionable comments were generated in the recent review. WalkthroughThe production completion queue limit decreases from 2048 to 1024, and the default transmit-buffer ceiling decreases from 1024 to 512. A new test checks that the estimated ring region is at most 32 KiB. The ENOMEM documentation and error message now describe memory-cgroup exhaustion, fragmented-host allocation, and the Linux 6.14-to-7.3 RLIMIT_MEMLOCK case. The quest notes now report the queue-size change and remove the prior investigation document. Severity of issue fixed: Low Merge Risk:
|
…ts 36 KiB Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@pullfrog review this pull request and submit an approving review if it is ready to merge Your note on fe6e2db is fixed in 2a16a28: (Written by Claude Opus 5.5) |
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — the delta since the prior Pullfrog review (fe6e2db → 2a16a28).
- Corrected README geometry —
rs/moq-uring/README.mdnow states the 256/1024 production geometry and 36 KiB per production worker, which resolves the stale 256/2048 and 52 KiB figures flagged earlier.
claude-sonnet-5-5 | 𝕏
There was a problem hiding this comment.
✅ No new issues found. The stale README note from the prior review is fixed.
Reviewed changes
I reviewed what changed since the prior Pullfrog review at fe6e2db: one docs commit.
- README geometry corrected:
rs/moq-uring/README.mdnow gives the production geometry as 256/1024 and says a production worker takes 36 KiB. That matchesPRODUCTION_SQ_ENTRIES = 256andPRODUCTION_CQ_ENTRIES = 1024. The rings come to 17,472 B, which rounds up to 5 pages, and the SQEs take 256 × 64 B = 16 KiB, which is 4 pages. Together that is 9 pages, or 36 KiB. The 20 KiB unit-test figure still works out the same way: 8,768 B of rings in 3 pages plus 8 KiB of SQEs in 2 pages.
A grep over rs/moq-uring, rs/moq-relay, and doc/ found no other mention of the old 2048-entry CQ or the 52 KiB figure.
claude-opus-5-5 | 𝕏

Summary
On a loaded host, moq-relay's io_uring tests sometimes failed at
moq_uring::Worker::newbecauseio_uring_setupreturnedENOMEM. In v7.3,io_region_allocate_pagesfirst tries one compound allocation ofget_order(size). If that fails, it falls back toalloc_pages_bulk_node, and that call can return fewer pages than asked. With 2048 CQ entries the rings region came to 33,856 B, just past 32 KiB, so it needed an order-4 allocation. Order 4 is abovePAGE_ALLOC_COSTLY_ORDER, so the allocator gives up on it quickly when memory is fragmented.udp::Config::default().tx_buffers_maxgoes from 1024 to 512. With 256 rx buffers, the default pool ceilings still fit in the CQ, whichcq_covers_the_default_pool_ceilingschecks.Error::ring'sENOMEMmessage now lists the memory cgroup and fragmented contiguous pages beforeRLIMIT_MEMLOCK, which only Linux 6.14 up to the 7.3 fix charges.Decisions
All decisions here were taken by the agent under /quest-afk, for the maintainer's review.
IORING_SETUP_NO_SQARRAYalone: it still leaves 32,832 B, which is order 4.IORING_SETUP_NO_MMAPwith user-allocated rings: io-uring 0.7.15 has no builder for it.rs/moq-uring/README.mdnow says 256/1024 and 36 KiB per production worker (17,472 B of rings in 5 pages plus 16 KiB of SQEs in 4), which is consistent with its 20 KiB unit-test figure.Validation
io_uring_setupdirectly on Linux 7.3.3 (fedora) with these flags: the kernel reportscq_off.cqes = 64. Following v7.3'srings_size, the region is 64 + 2048·16 = 32,832 B before the SQ array, so order 4 even withNO_SQARRAY. With 1024 entries it is 17,472 B with the SQ array, so order 3. This confirms More tests hold up under load #8670's CodeRabbit correction.io_uring/memmap.c: the compound attempt, then the bulk fallback that returnsENOMEMon a short count.cargo test -p moq-uring --lib: 79 passed.cargo test -p moq-relay --features io-uring --test integration uring_: 17 passed, includingbitrate_caps::uring_session_is_held_to_its_caps.just checkpassed.tx_buffers_max.Impact
moq_uring::udp::Config::tx_buffers_maxdrops from 1024 to 512, and each worker's CQ halves.Caused by
Models
Size: S
Implementer: Claude, claude-opus-5-5, unknown effort
Implementer: Claude Code, claude-opus-5-5, unknown effort (iterate)
🤖 Generated with Claude Code
Review
Rewritten by
quest reviewafter each round.Head: 2a16a28
Base: origin/main at 6f8b127
Risk: medium,
the base has no RISK.md to name what is always high or customer-facing, so every change is mediumClassified: medium by default
Reviewer: Codex, gpt-6.1-sol, medium effort
Session:
01a1d85c-bebe-7b81-893c-e1a062ef629c, round 2Score: 9/10,
The incremental changes only correct README ring geometry and memory figures to match the implementation, with no actionable findings; builds and tests were not run as instructed.Rounds: 2; findings 0: fixed 0, refuted 0, open 0
Findings: none