Skip to content

fix(hang,moq-audio): the join's backlog no longer holds the playout target - #8730

Merged
kixelated merged 4 commits into
mainfrom
quest/m2/audio-playout/join-hold
Nov 14, 2026
Merged

kixelated merged 4 commits into
mainfrom
quest/m2/audio-playout/join-hold

Conversation

@kixelated

@kixelated kixelated commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Finishes quest/m2/audio-playout/join-hold.md: after a subscription's join, the playout target (and video's delay with it) no longer holds 100 to 460 ms for a 20 s half-life before falling to the path's own.

The rule (doc/concept/audio-jitter.md, new "The join's backlog" section):

  • fastest, the window's smallest arrival - media, says when a frame was due.
  • A frame due before start, the first arrival, was produced before the subscription began: it is backlog.
  • joined is the latest arrival of a backlog frame.
  • An interval holding a frame due before joined measured the join, not the path. Its frames are the backlog or queued behind it, so it folds nothing, as if it held no arrival.

The test runs when the interval closes, because a join reveals itself late. The fastest path moves 1.3 to 2 s earlier over the first 400 to 800 ms as the backlog drains, so a frame that looks punctual against the backlog before it can turn out to be due before start. Only backlog moves joined, so lateness after the join (an outage, a publisher flushing seconds at once) never extends it.

Implemented in the reference, js/hang Jitter (including fork(), judged by the outgoing track's join), and moq-audio jitter.rs, bit-identical on all 15 corpus cases. The new join case replays the first 30 s of the checked-in relay-mic-rtt40 recording through the public relay: today's estimator holds 780 ms there for 20 s, and this one goes from the prior to 60 to 100 ms. backlog, reorder, tunein and batches each converge one interval later, because each opens with frames due before its first arrival. batches still reaches its 1100 ms, so 1 s flushes don't chain the join.

How it was picked

I replayed the 42 browser arrival captures of the 11-11, 11-12 and 11-13 Nightlies and the 6 replay traces through the reference, comparing variants. The quest's recommended candidate (measure a reordered frame against the window's fastest frame when its own reference lags it, and skip it when far behind) cleared the joins whose reference was itself stale. It left every wide row and the 11-12 Opus congestion row holding: there the chunk's reference is the fastest frame so far, and only the live frames that follow show the chunk was backlog. Per-frame "due before start" had the same blind spot. Measuring at interval close, and extending through the backlog's last arrival, cleared them all.

Then, through the player's real consumer (each capture converted to a replay trace and run through replay.ts, main's estimator against this one), mean target over 3 to 25 s:

rows main this
Opus congestion (3 nights) 178 to 195 ms, 20 s hold 63
Opus wide 303 to 382, 20 s hold 180, its steady state from the start
Opus near-zero, mild, step, fixed-250 77 to 126 63
Opus bursty 140 to 193 120 to 160
AAC congestion 11-13 310 238
other AAC rows unchanged unchanged

Player underruns stayed at 29 in total across the 42 rows, and trimmed audio fell slightly (52975 → 52782 ms/min summed). On the congestion rows the join's residue no longer lets the 25 s outage tip the target back up (11-12 Opus: 340 at 40 s on main, 60 here).

Decisions

Each was taken by the agent under /quest-afk, for the maintainer's review.

  • Backlog is "due before the first arrival", measured when the interval closes, through the backlog's last arrival. ✅ Rejected: the quest's recommended per-frame test against a stale reference (misses wide and 11-12 congestion, above); the same per-frame without hindsight; a per-delay-bucket record that dropped only backlog frames from an interval (more state and spec for the same Nightly result, and it still left the live frames queued behind the backlog); extending joined by every skipped interval (a publisher whose lateness exceeds an interval would chain it forever and never be measured).
  • A join interval folds nothing, like an empty interval, so it relaxes toward the prior as an idle one does. ✅ Rejected: dropping it without any fold, which would make the join's wall time not count toward the time constant.
  • relay-mic-rtt40 stays at 320 ms in the player, filed as a follow-up. ✅ The browser consumer closes groups below its cursor unread, so its 267 stale backlog groups arriving 1.0 to 1.3 s into the join never reach the estimator, and the 300 ms they delay the live frames reads as an ordinary stall. moq-mux's consumer does the same. Rejected: fixing it here, since it changes both consumers' read loops and public API.
  • The 1% fast clock test (replay.test.ts) now grades until its last arrival. ✅ A fast publisher clock makes the first frame slower than the path after it, so that first interval folds nothing and the target settles 500 ms later. That shifts the phase of the drift's splice sawtooth. Through the steady state both estimators show the same 5 to 11 ms; only the last 25 ms, after the final arrival, where the bridge slows audio as it runs out, differ: 29.9 ms here against 13.9 on main. The test's intent is the steady state. Rejected: keeping the run-out in the window and loosening the bound.

Validation

  • Corpus: the reference, js/hang (bun test src/container/jitter.test.ts) and moq-audio (cargo test -p moq-audio --lib jitter) pass all 15 cases; corpus.test.ts confirms the files match the reference.
  • Replays: just test audio-quality --replays --enforce passes on the merge with main after Audio follows the clock by its lag alone #8722 (77c9ca5), with its exact channel start. Against main's estimator on the same tree, only relay-mic (A/V lag p50 1.8 → -0.3 ms, p95 6.9 → 9) and relay-mic-stall (p50 3.1 → 1.7) moved, each within the half hop the lag rule leaves alone, re-measured in budgets.json with a note.
  • Nightly captures through the player's consumer: the table above, main and this branch on the same tree.
  • Drift: traced the 1% fast clock test's lag per quantum on both estimators; the difference is only past the last arrival.
  • just check passes.

Public API and wire

No API or wire change. Behavior change: the audio playout target in @moq/hang and moq-audio.

Caused by

No single PR: the cold start's FRESH_MAX weight on the join's intervals is as old as the estimator (#4162).

Follow-ups

  • quest/m2/audio-playout/join-backlog.md (M): both consumers report the arrival of a join's backlog group they close unread, so the estimator sees when that backlog ended; relay-mic-rtt40's replay target p95 falls from 320 ms. Recommended: a Jitter.backlog(arrival) and Rust counterpart, called for groups dropped below where delivery started, not below a cursor a later gap moved. No open decisions.
    Decided 2026-11-14 (maintainer, /quest-complete): ✅ keep in m2 (rejected: drop it, move to m3).

Models

Size: M
Implementer: Claude Code, claude-opus-5-5, unknown effort

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Review

Rewritten by quest review after each round.

  • Head: 03ef20e

  • Base: origin/main at 4fec611

  • Reviewer: Codex, gpt-6.1-sol, high effort

  • Session: 01a1da71-12c1-77a1-80f9-ce8188dd3a6c, round 1

  • Score: 8/10, No actionable regressions were found and the implementations agree, with the skipped-backlog limitation documented, but builds and tests were not run as instructed.

  • Rounds: 1; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

kixelated and others added 4 commits November 14, 2026 03:12
…t target

An interval holding a frame due before the subscription's backlog finished
arriving measured the join, not the path, so it folds nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…klog as a follow-up

The two replay rows whose first interval now folds nothing move within half a hop.
The 1% fast clock test grades until its last arrival, not the run-out after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title The join's backlog does not hold the target for 20 s fix(hang,moq-audio): the join's backlog no longer holds the playout target Nov 14, 2026
@kixelated
kixelated marked this pull request as ready for review November 14, 2026 14:27
@coderabbitai

coderabbitai Bot commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 2 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 77b8d332-4bfe-4520-ac4b-10af064200a6
Commits

Reviewing files that changed from the base of the PR and between 77c9ca5 and 03ef20e.

Files selected for processing: 16

  • doc/concept/audio-jitter.md
  • doc/concept/audio-jitter/backlog.json
  • doc/concept/audio-jitter/batches.json
  • doc/concept/audio-jitter/corpus.ts
  • doc/concept/audio-jitter/join.json
  • doc/concept/audio-jitter/reference.ts
  • doc/concept/audio-jitter/reorder.json
  • doc/concept/audio-jitter/tunein.json
  • js/hang/src/container/jitter.test.ts
  • js/hang/src/container/jitter.ts
  • js/watch/src/audio/replay.test.ts
  • quest/m2/audio-playout/README.md
  • quest/m2/audio-playout/join-backlog.md
  • quest/m2/audio-playout/join-hold.md
  • rs/moq-audio/src/jitter.rs
  • test/audio-quality/budgets.json
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

I reviewed the join-backlog rule in all three jitter estimators, the new join corpus case, the regenerated corpus files, and the replay test and budget updates.

  • Join detection: start is the first arrival. joined advances only for frames where media + fastest < start, and it is updated before the unmeasured early return, so unmeasured backlog still extends the join. The reference, js/hang Jitter, and moq-audio Estimator do this identically.
  • Interval skip: at close, an interval with oldest + fastest < joined folds nothing and adds 1 to pending. pending = … changed to pending += … because close() no longer always runs first to zero it. A skipped interval on the declared-silence path keeps its pending past settle() and folds later as idle, which matches "like an interval with no arrival".
  • fork(): the interval in progress is judged by the outgoing track's join, and the fork starts a fresh start and joined.
  • History.fastest(): this is reference(+∞). The Rust expect cannot fire. The first observation always pushes, because newest starts at −∞ and media is finite since Timescale is NonZero. That first observation also takes the start branch, so it never calls fastest().
  • Corpus: the new join case comes from relay-mic-rtt40's first 30 s. backlog, batches, reorder, and tunein were regenerated.
  • Tests and budgets: the 1% fast-clock replay test now grades only until the last arrival, and the relay-mic and relay-mic-stall A/V lag budgets were re-measured.
  • Quest: join-hold.md is removed, and the follow-up join-backlog.md covers backlog that the consumers skip without reading.

Edge cases I worked through:

  • Fast publisher clock: only the first interval is skipped.
  • Forward timestamp jump: one interval is skipped.
  • Outages: they never extend joined.
  • Publisher that keeps sending stale frames: it can keep skipping its own intervals, but the work stays bounded and other broadcasts are unaffected. That fits the rule in AGENTS.md.

Tests run locally: bun test doc/concept/audio-jitter/corpus.test.ts js/hang/src/container/jitter.test.ts (99 pass) and cargo test -p moq-audio --lib jitter (7 pass, including conforms_to_the_corpus).

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 14, 2026
Merged via the queue into main with commit 9b80cfc Nov 14, 2026
12 checks passed
@kixelated
kixelated deleted the quest/m2/audio-playout/join-hold branch November 14, 2026 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant