Skip to content

perf(net): resolve the paired network microbenchmark regressions - #7139

Merged
kixelated merged 3 commits into
mainfrom
quest/m0/nightly/paired-net
Oct 29, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/m0/nightly/paired-net

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The 2026-10-28 Nightly Rust bench targets job compared 270395a337 against 22808bbb2f and confirmed 14 regressions in moq-net. This PR finishes quest/m0/nightly/paired-net.md: work out what caused each one, and fix the ones that are real extra work.

Method

This host is shared with about 25 agents, so wall-clock comparisons here are noisy. Every claim below rests on one of two measures that load doesn't skew much:

  • Instruction counts. Callgrind on symbol builds of each side, counting Ir per call of the production function the bench iterates (write_frames, poll_read_frame, poll_requested_broadcast, requests_inner plus its drop, and BudgetWidening::widen with collection toggled to it).
  • Interleaved wall-clock A/B runs. All sides run on the same host, alternating order each round, with 2 to 3 rounds.

To find which commit caused each step, I built all 14 moq-net commits in the range and ran every flagged case across that chain.

Attribution

Case (ARM delta) Cause Evidence Action
codec_messages/decode/lite (+31%), encode/lite (+68%) Bench binary composition. moq-dev/moq#6976 added a setup_parameters group to benches/codec.rs. It links the whole lite_wire fuzz target into the binary, so LTO stops inlining the message codecs into Messages::encode_lite and decode_lite. Built 197bdae8ee with its parent's codec.rs: decode +3.6% and encode −0.4% against the parent. With the new group: +42% and +45%. None. No production code changed, and the outlined form is what session code calls.
group_relay_frames/plain/512/1, 8192/64 (+23%) Code or heap layout, not work. 237.3 Ir per poll_read_frame on base, 239.1 on head. On x86 the chain jumps in +25% steps at commits that don't touch the read path, e.g. 38bc872f1d +25%, then fbeac8a9fd +1%. None.
group_write_frames/batch32/{4,512,2048,8192} (+17% to +45%) Real. moq-dev/moq#6922 made write_frames clone each payload into a prepared entry and then drop the original. That is two atomic refcount RMWs per frame, plus a two-phase prepare/commit that only canonical content needs. Inside write_frames, Ir per frame: base 380, main 536 (plus 2 refcount RMWs), this PR 459 (no refcount ops). Wall-clock A/B against base, main → this PR: 2048 +55.5% → +11.7%, 512 +52.1% → +11.4%, 8192 +48.4% → +6.3%, 4 +11.0% → +7.7%. Fixed (see Changes).
origin/route_churn/10000b_1s (+14%) Real. moq-dev/moq#6648 turned Path::parts into a Flatten<Option<Split>>, which the route table walks on every lookup. Ir per iteration (requests_inner plus drop): base 19,293, main 20,658, this PR 19,848. Wall clock: +22.1% → +6.0%. Fixed (see Changes).
origin/serve_idle/3000r (+46%) Layout, not work. 201.0 Ir per poll_requested_broadcast on both base and head. On x86 the wall-clock delta is within noise (+3.1% on main). None.
track_budget_widening/{1r/16384c,64r/1024c,64r/16384c,64r/64c}/1 (+13% to +25%) Intended correctness work. A widen re-judges the reader's misses (moq-dev/moq#6921: Misses::judge, its range sets, and 3 more clock reads). moq-dev/moq#6922 added serves_end and arrival_grace. Ir per widen: 10,218 → 12,918 at 64r/64c/1, and the same +2.7k in all four cases. The extra cost doesn't grow with cache depth (64 to 16,384) or with readers (1 to 64), so there is no new slope. None. This is a fixed cost on a rare control operation.

Changes

  • group::Producer::write_frames now moves each frame out of the drained buffer instead of cloning its payload. This keeps the documented contract: the buffer survives a refusal unless the refusal fails the group, and a canonical conflict does fail it. Only a group that shares canonical content prepares the whole batch before storing any of it. Any other group stores each frame as it goes, which skips the per-frame canonicalize call and the extra move through the prepared array.
  • GroupState::commit is a new helper that holds the store-and-charge steps write_frame and write_frames used to repeat inline.
  • Path::parts is now split_terminator('/'). A valid path has no empty segments, so only the empty path produces an empty substring, and a terminator split skips it.
  • Quests. This deletes the finished quest and its link. It also adds the layout-only and inlining-only evidence to quest/m1/bench-ci.md, because those cases set its calibration problem: a paired round reruns the same two binaries, so a layout shift reads as a stable regression that extra rounds can't filter out.

Public API: none. Wire: none.

Validation

  • just check origin/main passed: 9020 tests, plus clippy, fmt, doc, shear, and sort.
  • quest check --base origin/main passed.
  • The swept benches already cover every affected scaling dimension, so no bench was added: batch size (group_write_frames), table size × cursors (route_churn), routes (serve_idle), readers × cached × runs (track_budget_widening), and frames × subscribers (group_relay_frames).

Choices made without asking

  • I kept the track_budget_widening cost. It is new correctness work with no slope, and a budget widen is a rare control-path operation.
  • I left the codec, relay, and serve_idle deltas without code changes. Instruction counts show no added work, and the paired gate compares night to night, so each flags only once.
  • write_frames stays about 20% above the old base in instructions per frame (+6% to +12% wall clock). That comes from Entry's canonical owner slot and helpers LLVM no longer inlines into the bigger function. Both are part of moq-dev/moq#6922's design.

Follow-ups

  • quest/m1/bench-ci.md should weigh instruction counts (CodSpeed, or callgrind in the paired gate) for the cases above, because their wall-clock rounds agree on a layout shift.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits October 28, 2026 15:03
write_frames cloned every payload into its prepared entries and then dropped
the originals, two atomic refcount writes per frame that only a canonical
conflict needed, and only shared canonical content can refuse a frame. Path::parts
flattened an optional split, which the route table walks on every lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Resolve the paired network microbenchmark regressions perf(net): resolve the paired network microbenchmark regressions Oct 29, 2026
@kixelated
kixelated marked this pull request as ready for review October 29, 2026 00:36
@coderabbitai

coderabbitai Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 35 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fc28750b-c109-4220-ae79-40993c1ab3e4
Commits

Reviewing files that changed from the base of the PR and between 53c0dcd and d817d7d.

Files selected for processing: 5
  • quest/m0/nightly/README.md
  • quest/m0/nightly/paired-net.md
  • quest/m1/bench-ci.md
  • rs/moq-net/src/model/group.rs
  • rs/moq-net/src/path/mod.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T00:38:04.549779Z d817d7d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: d817d7d

No actionable bugs found. Batch validation still precedes draining, shared canonical content still prepares the full batch before storing it, and conflict-driven consumption remains within the documented failed-group exception. split_terminator preserves empty-root and segment behavior under Path's validation invariant.

Direction: targeted removal of payload refcount churn and iterator overhead is preferable to weakening canonical consistency or budget correctness for benchmark numbers. Keeping layout/inlining evidence separate from production-work changes is sensible.

Verification: static review of both changed code paths, surrounding invariants and existing tests, including an independent review of the batch-write lifecycle. No tests or benchmarks run; the reported performance measurements were not independently reproduced.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary: head d817d7ddd17401c0aef8e9061fa1ea06fbc1d9ab lands unchanged.

  • CI: 13 checks passed, merge state clean against main.
  • Reviews: Codex (no findings, thumbs up) and the automated review from the maintainer account (no actionable bugs). No unresolved threads; CodeRabbit was rate-limited and does not count.
  • quest check --base origin/main --branch quest/m0/nightly/paired-net passed: the finished quest and its link are deleted.
  • Decisions kept as listed in the body: the track_budget_widening cost stays (fixed-cost correctness work, no slope), and the codec, relay, and serve_idle deltas get no code change (layout or inlining only).
  • Follow-up already recorded in quest/m1/bench-ci.md: weigh instruction counts for layout-only deltas in the paired gate.
  • Maintainer approved merging.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 29, 2026
Merged via the queue into main with commit db83154 Oct 29, 2026
13 checks passed
@kixelated
kixelated deleted the quest/m0/nightly/paired-net branch October 29, 2026 05:43
kixelated added a commit that referenced this pull request Oct 29, 2026
Main's #7139 already moves batched payloads into group storage and
skips canonicalization for groups without shared content, so keep its
write_frames. This branch keeps its tests and 32/33-frame bench sweep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant