[MDA2AV]

Fixes #402.

With WriteOverflow = Segmented, a response that outgrows the write slab spills into extra fixed-size slabs. Those came from one pool per reactor: a bare Stack<nint> that every connection on the reactor shared, commented "Reactor-thread-only, so no locking". Two kinds of thread used it:

• Renting happened on the writer's thread (RentOverflow), wherever the handler runs.
• Returning happened on the reactor thread when a send completed (CompleteFlush) or a connection was recycled (Clear). It also happened on the writer's thread, for a flush on a closed connection (DropStaged).

A handler that resumes on the thread pool writes from there, and one await … ConfigureAwait(false) on a task the reactor completed is enough to put it there. One connection's rent then overlaps another connection's return. A push and a pop that overlap hand one slab to two connections, or keep it in the pool twice. Each connection then writes into the other's response, and the shutdown drain frees the duplicates twice.

The fix: each connection keeps its own spill segments, so nothing is shared and nothing needs a lock.

• Kept per connection: after a send completes, its segments stay in _ov, the connection's list of segments, and its next response reuses them in order, segment k for segment k. One connection's rent and return can't overlap: the writer rents while it builds a response, the reactor returns only after that send has completed, and writes are refused in between. That flush handoff is what already protects the primary slab.
• Doubling sizes: each further segment of a response is twice the last, from twice the slab up to 64 slabs: 32 KiB to 1 MiB with the default 16 KiB slab. A response that spills a lot spills into few segments, so a 64 KiB response takes 2 instead of 4. A single write bigger than that gets a segment of its own size.
• Recycle: Clear frees the kept segments, the same way it shrinks Grow's overgrown slab back, so a pooled connection keeps only its base slab.
• Removed: the reactor's pool, its cap and its shutdown drain.

What changes for memory: a connection keeps the segments of its biggest response until it's recycled, which with doubling can be up to about twice that response. Grow, the default, already keeps its grown slab the same way. The shared pool took slabs back after every send and kept up to 4,096 per reactor. Segments also aren't shared across connections any more, so a new connection's first spill allocates.

Earlier revisions are still in this PR's history. The first put a lock on the shared pool, which let the reactor wait on a thread-pool thread holding it. The second kept a per-connection stack of slab-size spares, which lost more locality as connections grew (see below).

Tests

• core: two connections writing past their slabs from the thread pool each get only their own bytes (Segmented) (E2E, CoreTests), unchanged from the first revision.
  • What it does: two clients send 60 requests each, of 20–200 KB, against a 4 KB slab. The handler writes from the thread pool. Every byte is checked against a pattern that differs per request and per 4 KB.
  • Vacuity guard: at least 10 requests must overlap the other connection's.
  • On main: a wrong byte at offsets 28,672 and 8,192, both slab boundaries, then the process aborts with corrupted size vs. prev_size while consolidating.
• tcp: a Segmented connection reuses its spill segments for its next response, and frees them when it is recycled (Unit, SpillSegmentTests): spills 3 segments, completes the send and spills 3 again. The second response must get the same 3 in the same order, the connection must keep exactly 3, and none after Clear.
• tcp: each spill segment of a Segmented response is twice the last, from twice the slab up to 64 slabs (Unit, SpillSegmentTests): spills 8 segments from a 4 KB slab and checks their sizes, 8 KB doubling to 256 KB, then 256 KB.
• Red-check: each unit assertion flipped to the defect's value fails against the real one: new segments for the second response, none kept, segments surviving the recycle, and slab-size segments.
• Stop tests: the two reactor: a stop frees its pooled connections and its spare write slabs tests read the reactor's pool by name, so they now count the connection's kept segments. Flipped to the defect's value (they outlive the reactor), the segmented one fails with 0 left.

All suites pass: Unit 123, E2E 329, Http 89, Tls 194, Chaos 80, File 5, Pg 7 (+11 skipped), Redis (6 skipped, 1 pending).

• Skips: the tls module was loaded, so no Tls case skipped. The Pg and Redis skips need servers this box doesn't run.

Benchmark

A small HTTP/1.1 server, not in the repo: Segmented, a 16 KB slab, and 64 KB bodies written in 4 KB spans, so every response spills. 2 reactors on CPUs 2–3, wrk -t4 on CPUs 16–19, a 2 s warm-up then 5 s, builds in rotated order. Medians of server CPU per request:

	main	slab-size stack (previous revision)	doubling (this PR)
Keep-alive, 64 connections (4 rounds)	13.15 µs	13.25 µs	13.19 µs
Keep-alive, 256 connections (6 rounds)	13.33 µs	13.91 µs	13.65 µs
Connection: close, 64 connections (4 rounds)	21.17 µs	21.93 µs	21.82 µs

• Keep-alive: level with main at 64 connections. At 256 it's 2.4% above main, against 4.3% for the slab-size stack, with no overlap between the three builds' rounds. The gap is locality: the shared pool hands each response the slabs that were just returned and are still in cache, while a connection's own segments were last touched a full round of the other connections ago. Fewer, longer segments recover part of that.
• Connection: close: 3% above main. Each new connection allocates its segments and frees them on recycle, where the pool reused slabs it already had.
• Writer on the thread pool: this is the tcp: Segmented overflow slabs come from a reactor pool that writers and the reactor share without a lock, so one slab ends up in two connections #402 case, at 64 connections, in a separate run, with main left out because it corrupts responses there. Locked pool 14.59 µs, slab-size stack 14.68, doubling 14.56.
• Load: the box was also running a browser, so absolute numbers sit about 0.2 µs above quieter runs. The builds are interleaved, so the comparison holds.

Merges cleanly with #363, #384, #386, #393, #407 and #408. Against #307 the only conflict is in HardeningTests.cs, which #307 already has with main.

API: none.

🤖 Generated with Claude Code