fix(mux): pin glibc's heap in the ts_schedule bench by kixelated · Pull Request #6727 · moq-dev/moq

kixelated · 2026-10-25T05:41:45Z

Finishes quest/m0/nightly/ts-schedule.md.

What happened

Nightly's first paired run (run 40090959108) read ts_schedule/{1/1, 1/16, 4/1} 400 to 460% slower than 4381af6. The schedule did not get slower:

The only change to the measured code in the window is the --sync removal (moq-dev/moq#6554), dead in the bench. A counting allocator shows a byte-identical allocation sequence at both commits.
On the same ubuntu-24.04-arm runner type, each commit's bench built by itself measures within 2% of the other (92.5 vs 94.1 µs for 1/1).
Rebuilt the way Nightly builds it (nix dev shell, unified rust-shared build), the slowdown reproduces on today's main: 564 µs vs 92 µs.
Forcing glibc to mmap every slot buffer (GLIBC_TUNABLES=glibc.malloc.mmap_threshold=131072) gives 660 µs on that runner for both commits, the same as the slow reading.

The cause: at the bench's 100 Mb/s, a padded slot lays out about 300 KB, right where glibc switches between mmap and the heap and trims the heap's top. The mode a process lands in depends on its heap history, not on the schedule. In the slow mode, every slot faults in fresh pages. The cases with the fewest small allocations per slot (1/1, 1/16, 4/1) are the ones that flipped.

Fix

The bench calls mallopt before measuring to pin M_MMAP_THRESHOLD (32 MiB) and M_TRIM_THRESHOLD (max), so every slot reuses the heap. This only applies on Linux glibc (libc is a dev-dependency gated to that target). Locally, forcing the mmap mode makes the old bench 755 µs and the fixed one 106 µs, about its normal cost.

Choices made

I fixed the measurement in this bench only, not in the harness. Exporting the same tunables from bench/run.sh would protect every Criterion target, but it would also shift every trend line. That is a call for bench CI calibration.
I left Slot::layout as it is. Writing into a caller's buffer would avoid the allocation, but the exporter needs an owned payload per frame anyway.

Checks

just rs lint-changed / test-changed for moq-mux: pass, 10328 tests.
clippy on the bench: pass.
quest check: pass.
taplo: pass.
Not passing locally: just check's flake module fails building the github-runner image because the local sandbox refuses unshare. That is unrelated to this diff.
Not run: Nightly itself (agents don't dispatch it). The next scheduled paired run is the confirmation.

I added a temporary ARM diagnostic workflow to get the runner numbers above and removed it in the final commit. The squashed diff does not include it.

Public API: none. Wire: none.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

Quest-Claim: 399f71e16f2cca7b

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A padded slot at the bench's 100 Mb/s lays out about 300 KB, where glibc adapts between mmap and the heap and trims the heap's top. Which mode a process settles on depends on its heap history, not the schedule: on the ARM runners Nightly's build of an unchanged schedule faulted fresh pages every slot, reading 5x slower than the previous night. The bench now holds both thresholds so every slot reuses the heap. Drop the temporary ARM diagnostic workflow and the finished quest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

coderabbitai · 2026-10-25T09:05:07Z

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 18 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: CHILL
Plan: Advanced
Run ID: 7c3ee298-ba07-4a3e-be8b-56f7fb662d0e

Commits

Reviewing files that changed from the base of the PR and between 6515b58 and ca0e350.

Files ignored due to path filters: 1

Cargo.lock is excluded by !**/*.lock

Files selected for processing: 4

quest/m0/nightly/README.md
quest/m0/nightly/ts-schedule.md
rs/moq-mux/Cargo.toml
rs/moq-mux/benches/ts_schedule.rs

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

chatgpt-codex-connector · 2026-10-25T09:05:12Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-25T17:21:15.150237Z	`ca0e350`	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

kixelated

Automated review by review (OpenAI)

Reviewed commit: ae16f42

No actionable bugs found. Pinning allocator policy within this benchmark is a focused way to remove heap-history sensitivity without changing production scheduling or every benchmark's baseline. Both benchmark groups apply the policy before measurement, and the libc dependency matches the Linux/glibc compile guard.

Verification: static review of the patch, benchmark and surrounding harness configuration. I did not run benchmarks or independently reproduce the reported allocation-mode slowdown. Confirmation on the scheduled paired Nightly run remains outstanding.

kixelated · 2026-10-25T16:01:09Z

Landing: CI green, OpenAI review of ae16f42 found nothing actionable, no open threads, and quest check --base origin/main passes. The squashed diff carries no diagnostic workflow. No changes made during merge.

(Written by Claude Opus 5.5)

# Conflicts: # quest/m0/nightly/README.md

kixelated · 2026-10-25T17:20:07Z

Dropped from the queue for a quest/m0/nightly/README.md conflict (the qmux-fallback entry left main beside this one). ca0e350 merges main, dropping both finished entries; a trivial change since the OpenAI review of ae16f42. quest check and cargo check --locked -p moq-mux --benches pass. Re-landing.

(Written by Claude Opus 5.5)

kixelated and others added 4 commits October 24, 2026 22:41

chore(quest): claim quest/m0/nightly/ts-schedule.md

1095e39

Quest-Claim: 399f71e16f2cca7b

ci: temporary ARM diagnostic for the ts_schedule bench delta

183ff97

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ci: rerun the ts_schedule diagnostic the way Nightly runs it

427b179

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

kixelated changed the title ~~ts_schedule regression~~ fix(mux): pin glibc's heap in the ts_schedule bench Oct 25, 2026

kixelated marked this pull request as ready for review October 25, 2026 09:05

kixelated commented Oct 25, 2026

View reviewed changes

kixelated mentioned this pull request Oct 25, 2026

quest: record pre-cut follow-ups from the quest-spawn run #6790

Merged

kixelated added the quest:merge label Oct 25, 2026

kixelated added this pull request to the merge queue Oct 25, 2026

github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 25, 2026

kixelated added quest:merge and removed quest:merge labels Oct 25, 2026

Merge remote-tracking branch 'origin/main' into HEAD

ca0e350

# Conflicts: # quest/m0/nightly/README.md

kixelated added this pull request to the merge queue Oct 25, 2026

Merged via the queue into main with commit bd3bde0 Oct 25, 2026
11 checks passed

kixelated deleted the quest/m0/nightly/ts-schedule branch October 25, 2026 19:14

Navigation Menu

Repository navigation

fix(mux): pin glibc's heap in the ts_schedule bench - #6727

fix(mux): pin glibc's heap in the ts_schedule bench#6727
kixelated merged 5 commits into
mainfrom
quest/m0/nightly/ts-schedule

kixelated commented Oct 25, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 25, 2026 •

edited

Loading

Review limit reached

Uh oh!

chatgpt-codex-connector Bot commented Oct 25, 2026 •

edited

Loading

Uh oh!

kixelated left a comment

Uh oh!

kixelated commented Oct 25, 2026

Uh oh!

kixelated commented Oct 25, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

kixelated commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What happened

Fix

Choices made

Checks

Uh oh!

coderabbitai Bot commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

chatgpt-codex-connector Bot commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

kixelated left a comment

Choose a reason for hiding this comment

Uh oh!

kixelated commented Oct 25, 2026

Uh oh!

kixelated commented Oct 25, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

kixelated commented Oct 25, 2026 •

edited

Loading

coderabbitai Bot commented Oct 25, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 25, 2026 •

edited

Loading