Skip to content

fix(size): the failing report ends with its regressions, and the sdist drops vendored RTP captures - #8654

Merged
kixelated merged 4 commits into
mainfrom
fix/size-report-names-regressions
Nov 14, 2026
Merged

kixelated merged 4 commits into
mainfrom
fix/size-report-names-regressions

Conversation

@kixelated

@kixelated kixelated commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Nightly's Shipping sizes job (run) fails with exit 1, and its log ends with the release build-time table, so the reason is easy to miss. The reason is a real size regression against the 2026-11-07 report (the job was skipped 11-08 through 11-12 because a release job it needs failed, so six days of growth landed in one comparison):

  • Every JS bundle grew 11 to 15% (js/net brotli 136,880 to 157,282 bytes). The growth is spread across about 96 js/net, js/qmux, and js/clock commits (track, lite subscriber, the byte-pool cache, frame, drops, floors), not one change.
  • release/python-sdist/moq_ffi.tar.gz grew 35% (8,840,141 to 11,939,943 bytes). The sdist includes all of rs/**, so it picked up the vendored WebRTC stack, including str0m's RTP captures, which moq-webrtc's own Cargo exclude already drops because only its tests read them.
  • moq-c and moq-relay aarch64 archives and the Windows Go import library grew 5 to 6%.

The mbx remote cache download budget exceeded warnings in the same step are cache misses, not the failure.

Approach

  • sh/size/report.py repeats the regressions on stderr as it exits, so the failing step's log ends with them. The summary is unchanged.
  • The sdist excludes **/tests/data/**, as the crate does. This removes 1.38 MB (11.5%) of the sdist. The JS growth is accepted under Decisions, with an audit quest; the binary growth is left alone.

Impact

  • No public API or wire change. The Python sdist no longer ships str0m's test captures.

Validation

  • sh/size/report.py on tonight's sizes.json with the 11-07 history exits 1, and stderr ends with the 21 regression lines; report.test.py asserts the stderr ending.
  • With maturin 1.16.0 (the version CI's sdist job used), the sdist from main is 11,939,914 bytes (CI measured 11,939,943), and this branch's is 10,562,377 bytes, with only the 11 str0m/tests/data files gone. pip wheel from the trimmed sdist builds, installs, and import moq_ffi succeeds.
  • report.test.py also runs the report through one pipe (2>&1) and asserts the log ends with the regression, so the ordering holds in a job log.
  • Built main at a026b71b1 as Nightly does (just js build, bun sh/size/bundles.ts): js/net brotli is 153,149 bytes (11-13 report 157,282; 11-07 report 136,880), the figures the audit quest records.
  • just check passes.

Caused by

  • moq-dev/moq#5275 - exits after the build-time table without naming the regression.
  • moq-dev/moq#8163 - vendored str0m's RTP captures, excluded from the crate but not from the sdist.

Decisions

  • 2026-11-13 (maintainer, /quest-complete): ✅ accept the JS bundle growth as feature growth (lite-07 routes, the byte pool, Clock types) and plan a bundle audit quest (rejected: accepting it without an audit; holding until the bundles shrink). moq-dev/moq#8671 has since dropped sorted-btree and the test-only Manual clock from every bundle; on main at a026b71b1, brotli is 2% to 3% below the 11-13 report and still 9% to 12% above the 11-07 one. The 30-day comparison fails from 11-21 until a post-growth report is the anchor (12-13 for the 11-13 report), as the audit quest details.

Alternatives

  • Excluding the whole vendored test tree, or every crate's exclude, from the sdist: the RTP captures are 1.37 of the 1.38 MB, so the narrow pattern earns nearly all of it.

Follow-ups

  • Audit what the JS bundles carry (quest/m1/js-bundle-audit.md, M, m1): what each bundle carries, what can be split or lazy-loaded, and the 30-day anchor. No open decisions.
  • Shipping sizes is skipped whenever any release job it needs fails (11-08 to 11-12: the moq-cli armv7 and macOS builds, Release FFI Go bindings, aarch64 FFI builds), so several nights of growth arrive as one jump.

Models

Size: S
Implementer: Claude Code, claude-opus-5-5
Implementer: Claude Code, claude-sonnet-5-5, unknown effort (iterate, S)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

(Written by Claude Sonnet 5.5)

Review

Rewritten by quest review after each round.

  • Head: 80d805b

  • Base: origin/main at a026b71

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1d80f-22ee-7ab0-adc0-483c08aafd42, round 2

  • Score: 10/10, The incremental change verifies combined-stream log ordering, and the author's evidence refutes the earlier buffering concern.

  • Rounds: 2; findings 1 (medium 1): fixed 0, refuted 1, open 0

  • Findings: none

  • The last round's findings, at f4a319b:

    • Refuted: medium, sh/size/report.py:100: Flush the summary before emitting the final regression; The reported command output demonstrates CPython's ordering, and sh/size/report.test.py:140–146 now captures stdout and stderr through one pipe and asserts that the regression appears last.

kixelated and others added 2 commits November 13, 2026 09:10
…t drops vendored RTP captures

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 2 commits November 13, 2026 19:17
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 14, 2026 03:21
@coderabbitai

coderabbitai Bot commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 9 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 63052c5a-7a9b-4040-8cc9-6a949dd77257
Commits

Reviewing files that changed from the base of the PR and between 8206d7f and 80d805b.

Files selected for processing: 4

  • pyproject.toml
  • quest/m1/js-bundle-audit.md
  • sh/size/report.py
  • sh/size/report.test.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the full diff of the 4 changed files.

  • Regression summary on stderr — sh/size/report.py now exits with SystemExit("size regression:\n" + ...), so the failing step's log ends with the regressions. The exit code stays 1.
  • sdist exclude — pyproject.toml adds **/tests/data/** for the sdist, matching what moq-webrtc already excludes from its crate.
  • Tests — sh/size/report.test.py asserts the stderr ending. It also runs the report through one pipe and asserts the log ends with the regression.
  • Audit quest — quest/m1/js-bundle-audit.md records the accepted JS growth and the follow-up audit.

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 14, 2026
Merged via the queue into main with commit d800547 Nov 14, 2026
7 checks passed
@kixelated
kixelated deleted the fix/size-report-names-regressions branch November 14, 2026 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant