Repository navigation
ring: the ring's descriptor is registered, so an enter skips the fget/fput of it - #391
Merged
Merged
Conversation
…/fput of it Every io_uring_enter looked the ring's descriptor up and took a reference to it, then dropped it, before doing any work: once per loop pass, on the busiest syscall in the process. IORING_REGISTER_RING_FDS (5.18) puts the ring in the calling thread's table and hands back a slot; an enter that passes the slot with IORING_ENTER_REGISTERED_RING reads the ring from there instead. liburing does this for every ring it creates. Ring.Create registers the descriptor last, and SubmitAndWait passes the slot and the flag. The slot is in the creating thread's table, which restricts nothing: only the thread that created a SINGLE_ISSUER | DEFER_TASKRUN ring can enter it at all. A thread has 16 slots; a ring created without one enters by descriptor as before. Dispose frees the slot before closing the descriptor - the slot holds a reference to the ring, so a ring closed while registered would stay alive, memlock charge included, until its thread exited. Ring.Fd and the register calls are unchanged. Measured with a C probe on a ring set up like ioxide's, pinned to one P-core: an empty GETEVENTS enter 71 -> 64 ns, a NOP round trip 90 -> 82 ns, the same with a second thread alive. wrk Tcp/Raw, two runs of paired rounds against main: -1.5% and +0.8% req/s, with the main-vs-main control at -0.2% and +0.6% - flat, as it should be when one enter carries a batch of requests. Tests (Unit, RegisteredRingTests): an enter goes through the slot (emptied behind the ring's back, the next enter fails with EBADF; by descriptor it would not notice); Dispose frees the slot (an enter on it afterwards answers EBADF); a ring created with all 16 slots taken works through its descriptor. Red-checked one break per test: entering by descriptor, Dispose leaving the slot, and no fallback each fail exactly their own test. Docs: the ring setup list in architecture and the Ring section of dev-core.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #98.
Every
io_uring_enterlooked up the ring's own descriptor and took a reference to it (fget), then dropped it again (fput), before doing any work. That happens once per loop pass, on the process's busiest syscall.IORING_REGISTER_RING_FDS(kernel 5.18) puts the ring in the calling thread's table and hands back a slot. An enter that passes that slot withIORING_ENTER_REGISTERED_RINGreads the ring from the table instead. liburing does this for every ring it creates.What changes:
Ring.Createregisters the descriptor as its last step, andSubmitAndWaitpasses the slot and the flag.SINGLE_ISSUER | DEFER_TASKRUNring can enter it at all.Ring.Fdis unchanged, and the register calls keep using it.Measured
Per enter, with a C probe on a ring set up like ioxide's, pinned to one P-core, 5M iterations, 3 alternating rounds each:
GETEVENTSenterThe same holds with a second thread alive, as in any .NET process.
wrkonTcp/Raw: flat, as it should be when one enter carries a batch of requests.wrk -t4 -c64on other cores.The two runs disagree in sign, and single rounds varied by up to ±10% on this box today.
Tests
New, in
RegisteredRingTests(Unit):ring: enters go through the registered slot, not the descriptor: a NOP round trip works, then the slot is emptied behind the ring's back and the next enter must fail withEBADF. An enter by descriptor would not notice.ring: disposing a ring frees its registered slot: afterDispose, an enter on the old slot must answerEBADF.ring: a ring created with every slot taken still works, through its descriptor: 17 rings on one thread. The first takes a slot; the 17th falls back to its descriptor and works.Red-checked, one break per test:
Disposeleaves the slot registeredDocs: the ring setup list in
architectureand theRingsection ofdev-core.All suites pass, every reactor in them on a registered ring: Unit 105, E2E 322, Chaos 80, Http 89, Tls 169 (+14 skipped), File 5, Pg 7 (+11 skipped), Redis (6 skipped, 1 pending). The Tls skips are the kTLS cases (no
tlsmodule loaded here), and the Pg and Redis ones need servers this box doesn't run.It merges cleanly with #386, which also changes
Ring.Create.API:
NativegainsIORING_ENTER_REGISTERED_RING,IORING_REGISTER_RING_FDS,IORING_UNREGISTER_RING_FDSandio_uring_rsrc_update. Nothing is removed or changed.