Skip to content

fix(audio-quality): replay recorded codec durations - #6282

Merged
kixelated merged 4 commits into
mainfrom
quest/m1/audio-jitter-target/watch
Oct 22, 2026
Merged

kixelated merged 4 commits into
mainfrom
quest/m1/audio-jitter-target/watch

Conversation

@kixelated

@kixelated kixelated commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Audio replay inferred decoded frame length from median timestamp spacing. Losing most packets stretched the remaining audio and inflated a 20 ms codec term to 100 ms. A single-packet trace could not play. This advances the browser playout quest; its manual proof remains unfinished.

Approach

Capture encoded frame durations using the same container/Opus/fixed-codec readers as playback. Version 3 traces retain that duration per arrival; replay sizes PCM and updates the target from delivered packets. AAC fixtures retain their arrival clocks and use their fixed 1024-sample codec duration. Re-recorded the existing explicitly fake Chromium microphone lane because its old trace retained no Opus payloads.

Impact

  • Public API and wire: none. Production playback and ring refill behavior are unchanged.
  • Internal trace/capture format: arrivals become [at, timestamp, group, duration], in milliseconds; trace version 3 refuses missing duration evidence.
  • Consumer: deterministic audio-quality replay rows on both player rings. No browser or native quality ceiling changes.

Alternatives

Keeping timestamp inference cannot distinguish packet loss from long packets. Fabricating old Opus durations would erase missing evidence. Publishing a new codec API is unnecessary: the harness reuses private helpers.

Measurements

Before the fix, a 20 ms packet retained every 100 ms settled at a 120 ms target instead of 40 ms on both rings. Single-packet replay threw. The corrected replay covers those cases, variable Opus durations, and malformed durations.

All eight corrected replay rows measure zero underruns. The newly recorded fake-mic trace has 1,751 packets over 35 s, TOC durations of 20 ms, WebTransport and a 2 ms minimum RTT. Both rings have zero stalls, silence or skips, a 60 ms p95 target and 3.5 s convergence. This new recording proves neither improvement over the former recording nor the manual goal.

Exact AAC sizing changes local-aac/plain stalled share from 0.008 to 0.017 and silence from 0.015 to 0.020 because the old replay stretched frames to nearby timestamp gaps. Its recalibrated deterministic ceilings expose the actual sample lengths. AAC otherwise remains at zero underruns, with up to 2.5% stalls and about two skips per minute on local-aac while converging.

Validation

  • nix develop --command just ci: passed, including JS checks/builds/tests, quest, Markdown, shell and configuration checks. Player unit tests: 394 passed. Audio-quality harness: 36 passed.
  • just test audio-quality --replays --enforce: all eight deterministic rows pass their exact ceilings, re-run after merging main.
  • After merging main: @moq/watch 400 and @moq/audio-quality 36 tests pass. just check fails only on js/hang's libav test-order bug, which this PR does not touch and moq-dev/moq#6382 fixes.
  • quest check --base origin/main: 575 quest documents valid after merging main.
  • just test audio-quality-record --lanes relay-mic: successfully re-recorded the explicitly fake Chromium device through the public relay over WebTransport.
  • Real-microphone Chrome/Safari playback with added RTT and encoder lag/group-duration comparisons were not run; they remain in the quest as its manual remainder.

Follow-ups

The existing quest keeps the human-run remainder, settled 2026-10-22, so no new quest is needed:

  • A person on Apple hardware proves Chrome and Safari with a real microphone, both rings, the public relay and at least 40 ms added RTT. The fake-device trace here is explicitly insufficient.
  • The production refill policy stays until that run measures a cause and a justified alternative.
  • The same run measures encoder input-to-output lag/drift and A/Bs the 20 ms audio group default against roughly 100 ms.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

@kixelated kixelated changed the title js/watch: prove the measured playout target in a real browser fix(audio-quality): replay recorded codec durations Oct 22, 2026
Record encoded frame durations instead of inferring them from timestamp gaps. Migrate AAC traces from their fixed codec sample count and re-record the fake microphone trace with Opus TOC durations.

Co-Authored-By: GPT-6 <noreply@openai.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

The deterministic trace work is complete at 559984b: capture records codec durations, all fixtures use version 3, and replay no longer infers missing audio from timestamp spacing. The fake-microphone trace was re-recorded rather than assigning guessed durations to the former payload-free recording.

just ci, all eight enforced replay rows, and quest check --base origin/main pass. Production playback and browser/native budgets are unchanged. The draft remains open for the real-microphone Chrome/Safari proof, refill-policy assessment, and encoder lag/group-duration measurements listed under Decisions needed. No merge is requested.

(Written by GPT-6)

kixelated and others added 2 commits October 22, 2026 14:25
Keep the production refill policy until the real-microphone run measures it,
and fold the encoder lag and group duration A/B into that same run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 22, 2026 21:28
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@coderabbitai

coderabbitai Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2e0ab179-9613-4796-ab72-a2d8e0bef383
📥 Commits

Reviewing files that changed from the base of the PR and between b8bb673 and 402b484.

📒 Files selected for processing: 15
  • js/watch/src/audio/replay.test.ts
  • js/watch/src/audio/replay.ts
  • quest/m1/audio-jitter-target/watch.md
  • test/audio-quality/README.md
  • test/audio-quality/budgets.json
  • test/audio-quality/clients/js/record.ts
  • test/audio-quality/clients/js/replay.ts
  • test/audio-quality/clients/js/src/capture.test.ts
  • test/audio-quality/clients/js/src/capture.ts
  • test/audio-quality/clients/js/src/recorder.ts
  • test/audio-quality/clients/js/src/schema.ts
  • test/audio-quality/traces/bursty-aac.json
  • test/audio-quality/traces/local-aac.json
  • test/audio-quality/traces/relay-bbb.json
  • test/audio-quality/traces/relay-mic.json

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T21:31:12.172708Z 402b484 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 402b4846cc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated
kixelated added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit 6631674 Oct 22, 2026
12 checks passed
@kixelated
kixelated deleted the quest/m1/audio-jitter-target/watch branch October 22, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant