diff --git a/quest/m1/README.md b/quest/m1/README.md index 31aea28603..1c7a0003af 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -153,7 +153,9 @@ removed). - [Rust FETCH parity](/quest/m1/ietf-fetch-parity.md) - Rust finishes an answered joining FETCH's head, refuses a lost head with DOES_NOT_EXIST, and sends no timestamps without TIMESCALE, as the JS publisher does - [SUBSCRIBE honors the properties opt-out](/quest/m1/subscribe-timestamps.md) - the Rust publisher sends untimed objects to a SUBSCRIBE that declines Track Properties, and the Rust subscriber asks for them when it needs timestamps - [JS FETCH timeout test](/quest/m1/js-fetch-timeout-test.md) - a JS joining FETCH still waiting on its SUBSCRIBE at the deadline is refused with TIMEOUT +- [JS joins at the current group](/quest/m1/ietf-js-join.md) - a late JS moq-transport subscriber gets the group in progress whole through a joining FETCH or fill, as Rust does - [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - Rust honors native FILL_TIMEOUT to bound upstream waiting on drafts 18 and 22, alongside local content-age enforcement +- [Catalog estimates settle](/quest/m1/catalog-estimates.md) - hang quantizes jitter and delay rises, and MSF publishes once its estimates are made and then only for track changes - [IETF takeover](/quest/m1/ietf-takeover.md) - carry the shared takeover contract through the negotiated IETF epoch extension - [Identity and recovery guide](/quest/m1/identity-guide.md) - explain epochs, pinned catalogs, lazy jobs, restart and version compatibility as they exist; takeover adds its own section - [Native rung disable](/quest/m1/simulcast-rung-disable.md) - OBS stops sending the top renditions its grant cannot fund, as the Rust/FFI publishers do, and a shaped uplink through moq-ffi proves the resume @@ -161,6 +163,8 @@ removed). - [A/V clock](/quest/m1/av-clock.md) - the audio playhead drives Sync.reference while audio plays, through per-track sync handles - [Container reads drop backfill](/quest/m1/read-backfill.md) - the JS container consumer drops groups below its read cursor like Rust - [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - a buffered `` asks for the oldest group first, so a voice reply's opening arrives first, and audio's head hold shrinks to the floor +- [Push-to-talk loses no speech](/quest/m1/push-to-talk/README.md) - consumers hold a disabled rendition, and an unmute encodes at once with the mic still open +- [Always-on encoding](/quest/m1/eager-encode/README.md) - publishers encode while enabled whatever the demand, so the past exists to fetch; on-demand is opt-in, and audio returns to it through a raw backbuffer - [TS fidelity and recovery](/quest/m1/ts-reliability/README.md) - both modes have explicit profiles, automated validation, loss tests, and a dedicated guide - [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS - [CI housekeeping](/quest/m1/ci-housekeeping.md) - remove the actionlint queue ignore once a release after 1.7.12 reaches the dev shell diff --git a/quest/m1/catalog-estimates.md b/quest/m1/catalog-estimates.md new file mode 100644 index 0000000000..ecd245bbc1 --- /dev/null +++ b/quest/m1/catalog-estimates.md @@ -0,0 +1,59 @@ +# [M] Catalog estimates settle before they churn + +## Goal + +An unhinted moq-mux publisher's catalogs stop changing for small estimate +creep. Hang republishes `jitter` and `delay` only when they cross a coarse +step. The MSF catalog publishes once its estimates are made and then changes +only when the track list or a track's identity does. MSF-01 §5.3 fixes a +declared track's attributes. + +## Plan + +From [#7067](https://github.com/moq-dev/moq/issues/7067): an SDI encoder +through moq-c published 68 catalog versions in 32 s, and 16 even with every +hint from [#7068](https://github.com/moq-dev/moq/pull/7068). Of the ten that +weren't structural, eight were `delay` rising in ~8 ms steps. +[#6595](https://github.com/moq-dev/moq/pull/6595) already measures from +timestamps. `delay` still follows the cross-track write order, which tracks +runtime encode latency. It only rises, at most once per `ESTIMATE_WINDOW` +(`rs/moq-mux/src/catalog/tracks.rs:655-700`). + +Decided 2026-10-28 (maintainer): + +- **`jitter` and `delay` stay in hang.** @moq/watch, moq-audio and the TS + export use them as latency floors, and a receiver can't measure cross-track + skew itself. Hang keeps updating them in place; only a decoder identity + change publishes a new track + ([#7110](https://github.com/moq-dev/moq/pull/7110)). +- **Hang quantizes the rise.** A new `jitter` or `delay` republishes only when + it crosses the next multiple of the track's frame duration. Measurement is + unchanged. +- **MSF deduplicates.** `emit` (`rs/moq-msf` producer, `catalog/producer.rs:221-243`) + skips a group whose serialized catalog matches the previous one. +- **MSF waits for estimates, then freezes them.** MSF publishes once every + track with an unhinted estimate has closed its first group and carried + `RATE_WINDOW` (1 s, `catalog/estimate.rs:7`) of media, whichever is later. + The minimum duration matters because bitrate is measured over that window: + 20 ms audio groups would otherwise freeze it absent. Its estimated fields + then stay as first declared, and MSF republishes only for track-list or + identity changes. A track whose every estimated field is hinted publishes + at once. moq-c can hint bitrate and framerate but not jitter or delay + ([#7068](https://github.com/moq-dev/moq/pull/7068)), so its tracks wait. +- **No opt-out.** Quantizing and freezing remove the churn without new config + surface on fields players rely on. +- The HLS pointer goes away with + [time-addressed consumers](/quest/m1/time-addressing/consumers.md), so it is + out of scope here. + +Tests: a measured `delay` creeping in 1 ms steps republishes hang once per +frame-duration step. Identical MSF content publishes once. An unhinted MSF +catalog appears once the first groups close and a second of media has +passed, carries a bitrate for 20 ms audio groups, and keeps its estimates. + +Public API: none. Wire: fewer catalog groups; MSF estimate fields frozen at +first declaration. + +## Closes + +- [#7067](https://github.com/moq-dev/moq/issues/7067) - a catalog describes what a track is: keep channel measurements from churning it diff --git a/quest/m1/eager-encode/README.md b/quest/m1/eager-encode/README.md new file mode 100644 index 0000000000..db1025c011 --- /dev/null +++ b/quest/m1/eager-encode/README.md @@ -0,0 +1,61 @@ +# Always-on encoding + +## Goal + +First-party publishers capture and encode while a source is enabled, +whatever the subscriber demand, so the recent past exists for FETCH and a +subscription's `maxDelay` to serve. On-demand encoding stays available as an +explicit opt-in. Video stays always-on. Audio becomes on-demand again once it +can keep raw frames up to `maxAge` while nobody listens. + +## Plan + +Decided 2026-10-28 (maintainer), from [#7101](https://github.com/moq-dev/moq/issues/7101): + +- **Default always-on, in every language.** A demand-gated encoder leaves + nothing for a late subscriber to fetch, and FETCH and `maxDelay` make the + past part of the model. Nobody relies on on-demand encoding; the value is + in the networking. On-demand becomes an opt-in on each publisher that has + it today. +- **Rejected for video:** keeping a backbuffer of raw frames, too costly. + Audio does keep one, in [raw audio on demand](/quest/m1/eager-encode/raw-audio.md), + where it is cheap. +- **Stay on-demand by default:** moq-transcode rungs (a server-side ladder per + broadcast, and a past FETCH is already served by transcoding from the + source), the moq-boy demo, and the CARLA demo. Each has its own gating, + which is unchanged; their docs present on-demand as their choice. +- Bandwidth reservations stay keyed on demand: nothing is sent without a + subscriber. +- An unwatched rendition encodes at its configured bitrate (decided + 2026-10-28). It needs no remembered state, a past viewer's grant is stale + anyway, and the backlog is served from cache rather than paced live. + Rejected: the last granted target, and the lowest rung. + +Already opt-in, with no built-in gate: moq-c (`moq_encode_*_demand`), moq-ffi, +the wrappers, OBS and moq-gst. Only their docs change. + +The epic's own work, once the children land: + +- Docs: `doc/lib/rs/moq-net.md:25` ("publish only while someone is + watching"), `doc/concept/use-case/contribution.md` (on-demand becomes an + option), the moq-c doc comment at `rs/moq-c/src/audio.rs:433`, and the + Swift/Python/Go/Dart wrapper docs. +- Rewrite the quests that assume demand-gated capture: + [studio replay](/quest/m3/studio/replay.md) and its compose sources ("arm + replay = local subscriber"), [iOS capture](/quest/m2/mobile-capture-ios.md), + [video hardware](/quest/m3/video-hardware.md), and + [time-addressed consumers](/quest/m1/time-addressing/consumers.md)'s + HLS-only-viewer clause. [An HLS viewer starts capture](/quest/m1/hls-record-demand.md) + narrows to the opt-in mode. + +## Required + +- [@moq/publish encodes while enabled](/quest/m1/eager-encode/js.md) - audio and video encoders drop the demand gate, with an on-demand opt-in +- [moq-video captures while enabled](/quest/m1/eager-encode/video.md) - the camera stays open and encodes while enabled, with an on-demand opt-in +- [moq-audio captures while enabled](/quest/m1/eager-encode/audio.md) - the microphone stays open and encodes while enabled, with an on-demand opt-in +- [moq-cli on-demand flag](/quest/m1/eager-encode/cli.md) - `moq publish` capture encodes always, unless `--on-demand` +- [Raw audio on demand](/quest/m1/eager-encode/raw-audio.md) - audio encodes on demand again, from raw frames kept up to `maxAge` + +## Related + +- [Push-to-talk loses no speech](/quest/m1/push-to-talk/README.md) - the late-subscriber half of an unmute this fixes diff --git a/quest/m1/eager-encode/audio.md b/quest/m1/eager-encode/audio.md new file mode 100644 index 0000000000..26510e6f68 --- /dev/null +++ b/quest/m1/eager-encode/audio.md @@ -0,0 +1,20 @@ +# [S] moq-audio captures and encodes while enabled + +## Goal + +`moq-audio`'s capture supervisor keeps the microphone open and encodes while +enabled, whatever the demand. An on-demand option restores today's behavior. + +## Plan + +Decided 2026-10-28 (maintainer); see the [epic](/quest/m1/eager-encode/README.md). + +- The gate: the supervisor at `rs/moq-audio/src/encode/capture.rs:1035-1142` + (idle until used, closes the device when unused), the demand trait at + `:755-770`, and `Control::new` (`:455-485`) and `publish_capture` (`:675`). +- Update `lib.rs:22-25`, `aec.rs:179` and `doc/lib/rs/moq-audio.md:24`. + +Tests: with no subscriber, the capture publisher writes groups. The opt-in +keeps the device closed until demand arrives. + +Public API: default behavior change, plus an additive option. Wire: none. diff --git a/quest/m1/eager-encode/cli.md b/quest/m1/eager-encode/cli.md new file mode 100644 index 0000000000..3553e303f4 --- /dev/null +++ b/quest/m1/eager-encode/cli.md @@ -0,0 +1,25 @@ +# [S] moq-cli capture encodes always unless --on-demand + +## Goal + +`moq publish` capture of a camera, display or microphone encodes from start +to finish by default. `--on-demand` opens and encodes only while subscribed, +as today. + +## Plan + +Decided 2026-10-28 (maintainer); see the [epic](/quest/m1/eager-encode/README.md). +`rs/moq-cli/src/publish.rs:592` calls both `publish_capture`s, so the default +follows the libraries. The flag selects their on-demand option. Update the +comment there and `doc/bin/cli.md:250`. Name the flag under the repository's +CLI rule. + +Tests: the CLI's capture args parse `--on-demand`, and the default publishes +without a subscriber. + +Public API: CLI default change, plus a new flag. Wire: none. + +## Required + +- [moq-video captures while enabled](/quest/m1/eager-encode/video.md) - the video option this selects +- [moq-audio captures while enabled](/quest/m1/eager-encode/audio.md) - the audio option this selects diff --git a/quest/m1/eager-encode/js.md b/quest/m1/eager-encode/js.md new file mode 100644 index 0000000000..7cf989ef28 --- /dev/null +++ b/quest/m1/eager-encode/js.md @@ -0,0 +1,29 @@ +# [M] @moq/publish encodes while enabled + +## Goal + +`@moq/publish` audio and video encoders run whenever their rendition is +enabled, with or without a subscriber. Their groups sit in the track's cache +for a late SUBSCRIBE or FETCH to reach within its `maxDelay`. An `onDemand` +option restores today's gate. + +## Plan + +Decided 2026-10-28 (maintainer); see the [epic](/quest/m1/eager-encode/README.md). + +- The root gate: `js/publish/src/broadcast.ts:413-462` sets `rendition.track` + only while `served.demand().wanted` is defined. Both encoders key off it + (`video/encoder.ts:341-349`, `audio/encoder.ts:401-416` and `:686-688`). +- Losing demand no longer ends an epoch (`#cut()` on demand loss). Disabling + still does. +- Video still answers a join that wants a newer group with a fresh keyframe + (`demand.wanted`). +- `onDemand` is one option on the broadcast, applying to every rendition. + `doc/lib/js/publish.md:146` describes both modes. + +Tests: an enabled encoder with no subscriber writes groups. A subscriber that +arrives later with a covering `maxDelay` receives the earlier groups. +`onDemand` keeps today's behavior. + +Public API: default behavior change, plus an additive `onDemand` option. +Wire: none. diff --git a/quest/m1/eager-encode/raw-audio.md b/quest/m1/eager-encode/raw-audio.md new file mode 100644 index 0000000000..806c855341 --- /dev/null +++ b/quest/m1/eager-encode/raw-audio.md @@ -0,0 +1,37 @@ +# [M] Audio encodes on demand from a raw backbuffer + +## Goal + +With no subscriber, the audio publishers in JS and Rust keep captured raw +audio up to the track's `maxAge` instead of encoding it. When demand arrives, +they encode that backlog and continue live, so a late subscriber can still +fetch the recent past within its `maxDelay`. This becomes audio's default. +Video stays always-on. + +## Plan + +Decided 2026-10-28 (maintainer): audio is cheap to keep raw (48 kHz stereo +f32 is ~384 KB per second retained), unlike video. So audio gets back the CPU +on-demand saved, without losing the past. + +- The capture keeps running, so the microphone stays open as under always-on. +- The backlog encodes as live would have: the same frame size, the same + timestamps, one group per frame as today. Encoded groups land in the cache + in order, ahead of the live edge. +- A track with no `maxAge` keeps a fixed cap, chosen and justified in the PR. +- Leaving demand stops encoding and resumes buffering. The epoch continues, so + the past stays addressable. +- JS (`js/publish/src/audio/encoder.ts`) and Rust (`rs/moq-audio` capture) + both switch. The always-on default from their sibling quests becomes the + opt-out. + +Tests: no subscriber for a few seconds, then a subscriber with `maxDelay` +covering the last second receives that second, timestamps matching capture +time. Memory stays bounded by `maxAge` or the cap. + +Public API: default behavior change for audio. Wire: none. + +## Required + +- [@moq/publish encodes while enabled](/quest/m1/eager-encode/js.md) - the JS always-on path this switches from +- [moq-audio captures while enabled](/quest/m1/eager-encode/audio.md) - the Rust always-on path this switches from diff --git a/quest/m1/eager-encode/video.md b/quest/m1/eager-encode/video.md new file mode 100644 index 0000000000..c0a9f713a3 --- /dev/null +++ b/quest/m1/eager-encode/video.md @@ -0,0 +1,24 @@ +# [M] moq-video captures and encodes while enabled + +## Goal + +`moq-video`'s capture publisher keeps the camera open and encodes while +enabled, whatever the demand. An on-demand option on the capture config +restores today's close-between-viewers behavior. + +## Plan + +Decided 2026-10-28 (maintainer); see the [epic](/quest/m1/eager-encode/README.md). + +- The gate: `rs/moq-video/src/encode/producer.rs:3-13` (the module doc's + "strictly on demand"), `:525` (`publish_capture`) and `:660-700` + (`until_wanted`/`wait_capture`). +- The demand-driven PipeWire reopen (`capture/pipewire.rs:14,157,304,866`) + serves only the opt-in. +- Update the notes at `capture/mod.rs:116`, `lib.rs:29` and `encode/mod.rs:19`, + and `doc/lib/rs/moq-video.md:30`. + +Tests: with no subscriber, the capture publisher writes groups. The opt-in +keeps the camera closed until demand arrives. + +Public API: default behavior change, plus an additive option. Wire: none. diff --git a/quest/m1/ietf-js-join.md b/quest/m1/ietf-js-join.md new file mode 100644 index 0000000000..abdf448040 --- /dev/null +++ b/quest/m1/ietf-js-join.md @@ -0,0 +1,63 @@ +# [M] The JS moq-transport subscriber joins at the current group + +## Goal + +A late JS subscriber behind a spec-strict relay (such as moqx) receives the +group in progress whole, as the Rust subscriber does. A catalog or snapshot +written once arrives, and the first media group plays from its head. It works +on draft 18 and on draft 22. + +## Plan + +Decided 2026-10-28 (maintainer), from [#7088](https://github.com/moq-dev/moq/issues/7088): + +- **Mirror Rust exactly.** On draft 18, SUBSCRIBE with Largest Object. After + a SUBSCRIBE_OK with Largest `{G, k}`, send a Relative Joining FETCH + (offset 0) as its own request, outside PUBLISH_DONE's Stream Count, and + stitch the head and the live tail into one group producer. On draft 22, + `relative(1)` carries the matching fill. Today it sends none, so the same + gap exists there. +- **No join knob.** Every subscription joins at the current group. Rejected: + `join: "current" | "next"` and @moq/watch joining media at the next group. + [Fill refill](/quest/m2/ietf-fill-refill.md) already rejected the next group + ("costs a GOP of startup"). Lite and Rust have no such anchor, and how far + behind live to play is the player's choice through `maxDelay`. A "next" + anchor, if ever wanted, belongs in + [JS ranges](/quest/m1/subscribe-ranges/js.md) shared with Rust and lite. +- **No standalone-FETCH fallback.** A relay that refuses the joining FETCH + for a lone subscriber (moqx: "Session has no active subscribe", reported as + [openmoq/moqx#883](https://github.com/openmoq/moqx/issues/883)) has a bug of + its own. A refusal costs only that group's head, as in Rust + ([#6407](https://github.com/moq-dev/moq/pull/6407)). +- **No FETCH_OK deadline**, matching Rust: the joining FETCH lives as long as + the subscription. FILL_TIMEOUT is a separate publisher-side bound. +- **Rust's strictness on duplicates.** A whole group from object 0 that + arrives before the head wins and the FETCH is discarded. One that arrives + after the head is refused. +- **Implementation:** take over [#7089](https://github.com/moq-dev/moq/pull/7089) + (michaelstuurman, maintainer edits allowed). Trim it on its branch: drop + `Track.Join`, `Subscription.join` and the @moq/watch `MEDIA_JOIN` change, + drop the standalone fallback and the 10 s per-request deadline, and add the + draft-22 fill. The author keeps credit, and the review history stays with + the PR. + +Update the notes this makes stale: [IETF fill timeout](/quest/m1/ietf-fill-timeout.md)'s +"JS sends no IETF FETCH", and the interop README's claim that a JS subscriber +requests neither a joining FETCH nor a fill. + +Tests: the unit tests in #7089 minus the knob, plus a draft-22 fill. The +interop late-join lane passes with a JS subscriber against a strict publisher +(a lane that refuses to replay the group). `just ci` runs green, since #7089 +has not yet run CI. + +Public API: none. Wire: the JS subscriber sends Largest Object and a joining +FETCH on draft 18, and a fill on draft 22. + +## Closes + +- [#7088](https://github.com/moq-dev/moq/issues/7088) - a late draft-18 JS subscriber never receives a catalog written once + +## Related + +- [JS publisher answers a late join](/quest/m2/ietf-js-joining-fetch.md) - the publisher half after a quick finish +- [Headless moq-transport groups](/quest/m2/ietf-headless-group.md) - what a subscriber does with a tail whose head never comes diff --git a/quest/m1/push-to-talk/README.md b/quest/m1/push-to-talk/README.md new file mode 100644 index 0000000000..42008789da --- /dev/null +++ b/quest/m1/push-to-talk/README.md @@ -0,0 +1,59 @@ +# Push-to-talk loses no speech + +## Goal + +A listener subscribed while a publisher is muted hears the next utterance from +its first frame. Unmuting costs no catalog round trip, no resubscribe and no +microphone reopen. A muted rendition still costs no media bytes. + +## Plan + +From [#7101](https://github.com/moq-dev/moq/issues/7101), traced 2026-10-28. With +`Audio.Encoder` `enabled: false`, watch deselects the rendition and +unsubscribes. On unmute the listener waits for the catalog flip, then its +SUBSCRIBE reaches the publisher. The demand gate discards everything encoded +before that (`js/publish/src/audio/encoder.ts:686`). At 100 ms RTT that is +three one-way trips plus the probe and a frame (~515 ms to the first fresh +frame). The emptied buffer then refills the 300 ms delay, so the result is +~820-950 ms to audible, with 250-400 ms of speech never encoded. + +Decided 2026-10-28 (maintainer): + +- **Hold, not DTX.** DTX is for silence the app can't predict. Push-to-talk + knows when it is silent, and a held subscription costs no media bytes and no + encoder CPU. DTX is off on main anyway ([#4919](https://github.com/moq-dev/moq/pull/4919)). +- **Consumers hold a disabled rendition**, audio and video, by default, in + every first-party consumer. The hang spec weakens from "must not select" to + "should not select" a disabled rendition. +- **The publisher keeps its probe result across a disable**, and the + element's `muted` and `invisible` leave capture open. +- **Encoding no longer waits for demand** + ([always-on encoding](/quest/m1/eager-encode/README.md)). A listener who + subscribes after the unmute still gets the start within its `maxDelay`. This + replaces a publisher head buffer, which was considered and dropped as the + same idea special-cased. + +The epic's own work: once the children land, measure push-to-talk (22 +presses, 300 ms delay) behind bufferbloat: a rate limit below the media +bitrate, behind a deep but finite queue whose overflow drops packets. That +gates acceptance. A held listener's unmute pays no catalog round trip and no +resubscribe, so unmute → audible is the path's one-way delay plus the 300 ms. +Every frame of the utterance is encoded and sent; any frames dropped come +only from queue overflow. Rerun the reporter's setup (100 ms RTT, 2% random +loss) only as a reproduction of the original numbers, not as acceptance. +Record both in the PR. + +## Required + +- [JS consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-js.md) - watch keeps a disabled audio or video rendition subscribed, and the hang spec says "should not select" +- [Rust consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-rs.md) - moq-cli play, moq-ffi and moq-mux select follow the same rule +- [Re-enable without a probe](/quest/m1/push-to-talk/reenable.md) - the JS audio encoder reuses its probe, so an unmute flips the catalog at once and keeps capture timestamps +- [`muted` and `invisible` keep capture open](/quest/m1/push-to-talk/mute.md) - the element disables only the encoder, so unmute skips getUserMedia and AudioContext setup + +## Closes + +- [#7101](https://github.com/moq-dev/moq/issues/7101) - a disabled audio rendition is deselected, so the start of the next utterance is lost + +## Related + +- [Always-on encoding](/quest/m1/eager-encode/README.md) - encoders run while enabled, so a late subscriber can fetch the start diff --git a/quest/m1/push-to-talk/hold-js.md b/quest/m1/push-to-talk/hold-js.md new file mode 100644 index 0000000000..e4e16123d9 --- /dev/null +++ b/quest/m1/push-to-talk/hold-js.md @@ -0,0 +1,47 @@ +# [M] JS consumers hold a disabled rendition + +## Goal + +`@moq/watch` keeps its selected audio or video rendition, and its +subscription, while the publisher has it `enabled: false` and no enabled +alternative exists. Re-enabling it plays without a resubscribe. The hang spec +says a consumer "should not select" a disabled rendition, where it said "must +not". + +## Plan + +Decided 2026-10-28 (maintainer), default on, audio and video: + +- Initial selection still skips disabled renditions, so a listener who joins + during a pause subscribes on the unmute. +- The selected rendition stays selected while it is disabled, still in the + catalog, and every rendition of its kind is disabled. An enabled + alternative wins, so a simulcast ladder (whose lowest rung never disables) + never holds. A rendition removed from the catalog is dropped. +- Audio: `#runSelected` (`js/watch/src/audio/source.ts:129-133`) and the + decoder's subscription (`audio/decoder.ts:363-396`). Key `#idle` + (`decoder.ts:203-205`) on "held and disabled" as well as a missing config, + so a muted tile still suspends its AudioContext. +- Video: `enabledRenditions` (`js/watch/src/video/source.ts:190-191`) with the + same rule. +- A held subscription gets a clean break from the publisher's discontinuity + marker ([#4880](https://github.com/moq-dev/moq/pull/4880)), or after + [time addressing](/quest/m1/time-addressing/consumers.md) from the gap + between groups, and keeps its + audio graph ([#4881](https://github.com/moq-dev/moq/pull/4881)). + +Spec wording lands with whichever hold quest lands first: the hang draft, +`rs/hang/src/catalog/{audio,video}/mod.rs`, `js/hang/src/catalog/audio.ts` and +`video.ts`, and `doc/concept/hang.md`. +[OBS source renditions](/quest/m2/obs-source-renditions.md) already plans the +same hold. + +Tests: mute, then unmute, keeps one subscription. An enabled alternative takes +over. A rendition removed while held is dropped. A held muted tile suspends +its AudioContext. + +Public API: behavior only. Wire: none. + +## Related + +- [Rust consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-rs.md) - the same rule in Rust diff --git a/quest/m1/push-to-talk/hold-rs.md b/quest/m1/push-to-talk/hold-rs.md new file mode 100644 index 0000000000..c48c8cb650 --- /dev/null +++ b/quest/m1/push-to-talk/hold-rs.md @@ -0,0 +1,35 @@ +# [M] Rust consumers hold a disabled rendition + +## Goal + +Every first-party Rust consumer keeps its selected rendition, and its +subscription, while the publisher has it disabled and no enabled alternative +exists, as [@moq/watch does](/quest/m1/push-to-talk/hold-js.md). + +## Plan + +Decided 2026-10-28 (maintainer): the hold applies in every first-party +consumer, with the same rule as the JS quest. Initial selection skips disabled +renditions, an enabled alternative wins, and a removed rendition is dropped. + +- `rs/moq-mux/src/select.rs:93,121`: the selection filters require + `config.enabled`. +- `rs/moq-cli/src/play/media.rs:192-193,445-452`: `stop_disabled` drops demand + on the playing rendition. +- `rs/moq-ffi/src/audio.rs:613-616`: a disabled rendition returns `NotFound`. + It should keep a held one, and the wrappers' docs follow. + +[moq-audio follows the grant](/quest/m2/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md) +already expects a held listener to keep demand on a single audio rendition +disabled below the codec floor. + +The spec wording lands with whichever hold quest lands first. + +Tests: the JS quest's cases, in moq-mux select and moq-ffi. + +Public API: moq-ffi returns a held disabled rendition where it returned +`NotFound`. Wire: none. + +## Related + +- [JS consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-js.md) - the same rule in JS diff --git a/quest/m1/push-to-talk/mute.md b/quest/m1/push-to-talk/mute.md new file mode 100644 index 0000000000..a033419ad3 --- /dev/null +++ b/quest/m1/push-to-talk/mute.md @@ -0,0 +1,47 @@ +# [S] The publish element's mute and hide keep capture open + +## Goal + +`` and `` stop sending audio or +video without closing the microphone or camera. Unmuting pays no +getUserMedia, no new AudioContext and no worklet load, and unhiding pays no +camera reopen. Removing `source`, or removing the element from the +document, still releases both devices. + +## Plan + +Traced 2026-10-28 from [#7101](https://github.com/moq-dev/moq/issues/7101): +`muted` sets `audioEnabled = enabled && !muted` (`js/publish/src/element.ts:204`), +and that one signal drives the `Source.Microphone`, the Capture and the +encoder (`:430-431`). Turning it off tears down the AudioContext and worklet +(`capture.ts:177`) and stops the track (`microphone.ts:61`). Unmuting then +pays 50-300+ ms on real hardware before any frame exists. `invisible` does +the same to the camera. Toggling `Audio.Encoder.enabled` alone leaves +capture running. + +Decided 2026-10-28 (maintainer): + +- **`muted` keeps the mic open.** It disables only the audio encoder; the + source and Capture follow the element's lifecycle gate (`#enabled`, cleared + by `disconnectedCallback`, `element.ts:356-359`) and `source`, independent + of `muted` and `invisible`. No new attribute: a second mute flag would be + the same control twice. +- **`invisible` does the same for video.** It disables only the video + encoder and leaves the camera open. One rule for both, and removing + `source` is how a page releases the devices. Rejected: keeping `invisible` + releasing the camera as a privacy cue. +- The browser's microphone and camera indicators stay lit while muted or + hidden. The docs say so, and say that releasing one device while keeping + the other takes the library (`Source.Microphone` or `Source.Camera` + `enabled`), since `source` covers both. + +Docs: the attribute table in `doc/lib/js/publish.md:34` ("Disable audio or +video capture") and the element's doc comments. + +Tests: muting keeps the microphone track live and unmuting encodes without a +new getUserMedia. Hiding keeps the camera track live. Removing `source` +stops both tracks, and so does detaching the element; reattaching it +reopens them. + +Public API: behavior change to `muted` and `invisible` (capture stays open). +Wire: none. diff --git a/quest/m1/push-to-talk/reenable.md b/quest/m1/push-to-talk/reenable.md new file mode 100644 index 0000000000..203746400a --- /dev/null +++ b/quest/m1/push-to-talk/reenable.md @@ -0,0 +1,32 @@ +# [S] The JS audio encoder re-enables without a probe + +## Goal + +Re-enabling a disabled `Audio.Encoder` republishes the catalog as enabled and +encodes the first captured frame at once, without building a new probe +encoder. Resumed chunks carry their capture timestamps. + +## Plan + +Traced 2026-10-28 from [#7101](https://github.com/moq-dev/moq/issues/7101): +`enabled = true` builds a new probe `AudioEncoder` (encode and flush, +`js/publish/src/audio/encoder.ts:457-535`), and the catalog stays disabled +until it finishes (`:545-569`). That costs ~10-30 ms. + +Decided 2026-10-28 (maintainer): keep the probe result (the encoder +settings and decoder description) across a disable while the capture format +and settings are unchanged. A changed format or settings probes again, as +today. + +Keep the reset at a timestamp gap (`:693-702`). Chrome stamps encoder output +from its first input plus the samples encoded since, ignoring later jumps. +Without the reset, chunks after a mute would trail capture time by the mute's +length. The reset also drops only chunks from before the gap. Rejected: +rebasing output timestamps by hand to avoid the reset, which adds a second +clock for a cost of under a frame. + +Tests: a disable/enable cycle with unchanged capture flips the catalog in the +same tick, without a new probe. After a multi-second mute, resumed chunk +timestamps match capture time. A changed sample rate probes again. + +Public API: none. Wire: none. diff --git a/quest/m1/time-addressing/consumers.md b/quest/m1/time-addressing/consumers.md index 0a84167946..fa3dcd2d65 100644 --- a/quest/m1/time-addressing/consumers.md +++ b/quest/m1/time-addressing/consumers.md @@ -41,7 +41,9 @@ state again). break. Nested so a later join mechanism that is not a fixed grid can sit beside it. moq-mux declares the interval from its GOP hint, and `@moq/publish` does - too. A broadcast without one is not HLS-renderable. + too. moq-c's `group_duration_us` hint + ([#7068](https://github.com/moq-dev/moq/pull/7068)), which declares the `hls` + duration today, then declares the interval. A broadcast without one is not HLS-renderable. - **Each group belongs to one segment, numbered by grid line**: the first line it contains, so an edge finds segment N from a `Time` start at pts N × interval, the group that resolves to. A group containing no line diff --git a/quest/m1/ts-reliability/README.md b/quest/m1/ts-reliability/README.md index d2a69c1f4b..795e59fa20 100644 --- a/quest/m1/ts-reliability/README.md +++ b/quest/m1/ts-reliability/README.md @@ -88,7 +88,8 @@ owner that retires it. Those quests rank ahead of the remaining rows. Rejected: decode time in every frame container, one shared ingest byte budget, and replacing the demux with GStreamer or another parser (it fixes neither cause). Out of scope for now: decoupling catalog publication from mux-rate -measurement. Re-planned the same day: decode order covers MKV and FLV too and +measurement, except for a supervised restart, which reuses its predecessor's +measurement (decided 2026-10-28). Re-planned the same day: decode order covers MKV and FLV too and absorbs the release audit's export-reorder quest, and the shared parser lands before the open stream-routes and PAT drafts, which rebase onto it. @@ -98,6 +99,9 @@ catalog linger (#5305). ## Required +- [A timebase step restarts the TS import](/quest/m1/ts-reliability/step-restart.md) - an unflagged step or a rewind republishes under a new epoch instead of ending the import and every subscriber's export +- [Export re-acquires a step](/quest/m1/ts-reliability/export-reacquire.md) - `moq export ts` re-acquires an unmarked timeline step instead of exiting on fake drift +- [A restart keeps its catalog](/quest/m1/ts-reliability/restart-catalog.md) - a restarted import publishes its catalog with its first media - [PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md) - a running export follows a moved PMT PID - [TS workflow passes](/quest/m1/ts-reliability/validation-green.md) - the scheduled TS lanes pass on main again, with no lane loosened - [Export SI order](/quest/m1/ts-reliability/export-si-order.md) - a TS export sends SI and media after its PMT and a PCR interval, so a hop keeps first emissions diff --git a/quest/m1/ts-reliability/export-reacquire.md b/quest/m1/ts-reliability/export-reacquire.md new file mode 100644 index 0000000000..256991d7bd --- /dev/null +++ b/quest/m1/ts-reliability/export-reacquire.md @@ -0,0 +1,37 @@ +# [S] The TS export re-acquires an unmarked timeline step + +## Goal + +`moq export ts` keeps running when its source's timestamps step with no +restart or skip marker. It re-acquires the clock as at startup, instead of +exiting with a drift error ("the source's clock runs +172.5 ppm off ours"). +This protects subscribers of TS publishers other than `moq import`. + +## Plan + +Traced 2026-10-28 from [#7102](https://github.com/moq-dev/moq/issues/7102): the ++172.5 ppm is noise, not a measurement of the source. + +- `jumps()` (`rs/moq-mux/src/jitter.rs`) opens a generation only on a marker. + An unmarked step goes to `steer()` on the old clock. +- The 600 s rate window drops every pre-step floor, but `startup` survives, + since only `acquired()` resets it. So `lost` reads as the whole step. +- The bail then fires on the first two-floor rate over 30 ppm (~345 µs of + arrival jitter across one 2 s step). + +Decided 2026-10-28 (maintainer): treat a step outside any marker as a new +acquisition, as `acquired()` does. Keep [#5564](https://github.com/moq-dev/moq/issues/5564)'s +decisions: no minimum span or sample count, and never widen the 30 ppm or +`lost` thresholds. A step is told apart from drift by size: a phase change no +clock rate within the bound could produce in the elapsed time. Check that the +straddling step's slew bound (`MAX_SLEW · elapsed`) cannot swing the drift +across the bound in one measurement. + +Tests: a forward and a backward unmarked step each re-acquire and keep +emitting. The #5564 startup cases still pass unchanged. + +Public API: none. Wire: none. + +## Related + +- [A timebase step restarts the TS import](/quest/m1/ts-reliability/step-restart.md) - the importer-side fix, under which `moq import` never forwards such a step diff --git a/quest/m1/ts-reliability/restart-catalog.md b/quest/m1/ts-reliability/restart-catalog.md new file mode 100644 index 0000000000..114204ffa1 --- /dev/null +++ b/quest/m1/ts-reliability/restart-catalog.md @@ -0,0 +1,28 @@ +# [S] A restarted TS import publishes its catalog at once + +## Goal + +When `moq import ts/srt` republishes after a timebase step, the new epoch's +catalog goes out with its first media. It no longer waits ~2 s for a fresh +mux-rate window, so subscribers lose only the splice, not the catalog hold. + +## Plan + +Measured in [#7102](https://github.com/moq-dev/moq/issues/7102): a supervised +restart cost a subscriber 2.8 s. 1.2 s was the old instance's tail (the +[stitch catch-up](/quest/m1/ts-reliability/stitch-catch-up.md)). 1.6 s was +the restarted import holding its catalog for its first mux-rate window. + +Decided 2026-10-28 (maintainer): scoped to supervised restarts. The new +instance reuses the previous instance's mux-rate measurement, which describes +the same multiplex, and refines it as usual. Catalog publication stays coupled +to the measurement for a first start, so the epic's out-of-scope note stands. + +Tests: replaying a step through the supervisor, the second epoch's catalog +is published with its first group. + +Public API: none. Wire: none. + +## Required + +- [A timebase step restarts the TS import](/quest/m1/ts-reliability/step-restart.md) - the restart this speeds up diff --git a/quest/m1/ts-reliability/step-restart.md b/quest/m1/ts-reliability/step-restart.md new file mode 100644 index 0000000000..57128f7f00 --- /dev/null +++ b/quest/m1/ts-reliability/step-restart.md @@ -0,0 +1,62 @@ +# [M] A timebase step restarts the TS import under a new epoch + +## Goal + +`moq import ts` and `moq import srt` keep a broadcast available across a +source timebase step. A rewind, flagged or not, and an unflagged step in +either direction end the importer's instance. The CLI then republishes in the +same process as a new broadcast under a fresh epoch, instead of exiting. +Subscribers follow the new instance like any republish, and no `moq export ts` +exits on a step it reads as clock drift. + +## Plan + +Root cause of [#7102](https://github.com/moq-dev/moq/issues/7102), established +2026-10-28: the contribution comes from two redundant encoders behind a +changeover switch. Their clocks share a frequency reference (the offset holds +at ~16,844.5 s, a few ppm over days) but start ~4 h 40 m apart, and the switch +sets no `discontinuity_indicator` (ISO/IEC 13818-1 §2.4.3.5, TR 101 290 2.3b). +That is an upstream misconfiguration: redundant encoders are normally +time-locked, or the switch flags the break. MoQ tolerates it anyway, as ffmpeg, +GStreamer and VLC do by inferring a break from the jump. + +Decided 2026-10-28 (maintainer): + +- **Two encoders are two broadcasts.** An unflagged step means a different + source, so it gets a new epoch rather than a splice. This keeps #4543's rule + (group starts never go backwards within an instance) and the m1 decision + that a redundant ingest pair runs on distinct epochs. A flagged forward step + still splices with break markers, as today. +- **Detection:** PCR and every timed PID step together, forward beyond ~10 s + or backward by any amount that is not a 33-bit wrap. These are ffmpeg's + `dts_delta_threshold` bounds, not TR 101 290's 100 ms, which SRT loss can + exceed. `ts/health.rs` already computes the modular step for 2.3b; reuse it. +- **The supervisor lives in the CLI, not moq-mux.** The importer ends its + instance with a typed restart cause. `moq import ts/srt` republishes under a + fresh epoch on the same path, keeping the SRT connection. This is the + supervisor the 2026-10-08 decision allows ("a supervisor that wants to + continue publishes a new broadcast under a new epoch"). An in-process + `import.restart(broadcast)` in moq-mux stays rejected. +- Rejected: inferring the flag forward (a splice across two sources), and + reopening #4543 with a catalog timeline generation (a second epoch under + another name, conflicting with time addressing). + +Tests: a synthetic clip with the whole-multiplex unflagged step both ways (the +reporter offered a public reproducer) restarts under a new epoch. A +subscriber's `moq export ts --stitch` follows it. Lossy PCR gaps under the +bound never restart. + +Docs: `doc/bin/cli.md` ("run it again to publish anew") describes the restart +and names the upstream fix (time-locked encoders, or a flagged switch). + +Public API: the importer's error gains a restart cause. Wire: none. + +## Closes + +- [#7102](https://github.com/moq-dev/moq/issues/7102) - an unflagged timebase step ends `moq import` and `moq export ts` + +## Related + +- [Export re-acquires a step](/quest/m1/ts-reliability/export-reacquire.md) - the export-side defense for non-moq TS sources +- [A restart keeps its catalog](/quest/m1/ts-reliability/restart-catalog.md) - removes the catalog hold a restart pays +- [A stitched switch returns to --delay](/quest/m1/ts-reliability/stitch-catch-up.md) - drops the old instance's tail a restart leaves at each export diff --git a/quest/m2/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md b/quest/m2/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md index 820498bb7b..1bb1d2d1eb 100644 --- a/quest/m2/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md +++ b/quest/m2/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md @@ -44,8 +44,9 @@ grant; `Options::bandwidth` documents that grant below it disables the rendition until bandwidth recovers (decided 2026-10-04 with the enabled flag), through the disable and re-enable rule [simulcast rung disable](/quest/m1/simulcast-rung-disable.md) puts beside - `Control` rather than a second one; the encoder never errors. Once viewers - deselect it, its demand and so its grant go away, so recovery evaluates a + `Control` rather than a second one; the encoder never errors. Viewers with + no enabled alternative hold it ([push-to-talk](/quest/m1/push-to-talk/README.md)), + so its demand can stay while its grant is below the floor; recovery evaluates a hypothetical share against the current estimate, as the ladder controller does. The reservation's ceiling is the configured bitrate; only the policy target moves. diff --git a/quest/m2/obs-source-renditions.md b/quest/m2/obs-source-renditions.md index 016efa201e..016470e480 100644 --- a/quest/m2/obs-source-renditions.md +++ b/quest/m2/obs-source-renditions.md @@ -34,8 +34,12 @@ Decided with the maintainer on 2026-10-22: rendition never wraps), then bitrate. Audio (`hang::catalog::Audio::ranked`) sorts by enabled, then bitrate, then sample rate, then channel count. Skip disabled renditions and take the first one the source can decode. A - section where every rendition is disabled is handled like an absent - section: clear the track, and blank it for video. Don't add a ranking API + section where every rendition is disabled keeps the rendition already + selected, still subscribed, when it remains in the catalog (decided + 2026-10-28, [push-to-talk](/quest/m1/push-to-talk/README.md): consumers + "should not select" a disabled rendition, but hold one). With nothing + selected, it is handled like an absent section: clear the track, and blank + it for video. Don't add a ranking API to moq-ffi until another binding consumer needs one. - Make `epoch` part of rendition identity for both the active and the pending track.