diff --git a/quest/m0/audio-filters.md b/quest/m0/audio-filters.md index 3ce623d..38369eb 100644 --- a/quest/m0/audio-filters.md +++ b/quest/m0/audio-filters.md @@ -15,7 +15,7 @@ Guest, broadcast, media, browser, and replay sound stays unfiltered. Changed 2026-11-02 (maintainer), when the studio moved out of moq: the chain itself, its filters, and the out-of-process plugin host are moq-audio work in [moq-audio filter chain](https://github.com/moq-dev/moq/blob/main/quest/m3/audio-filters.md), -which moves to moq's m3 in [moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421). +in moq's m3. This quest is the studio's side: the panel, the scene-file shape, and wiring each source's chain into the program bus. diff --git a/quest/m0/packaging/macos.md b/quest/m0/packaging/macos.md index 4cb0bbc..4e339b4 100644 --- a/quest/m0/packaging/macos.md +++ b/quest/m0/packaging/macos.md @@ -44,6 +44,13 @@ protected branches can deploy from, for team `D7D5SDDB5Z` and bundle ID The certificate (`6WWG279G9M`, G2) expires 2027-12-11, and the profile with it; see [renewal](/quest/m0/signing-renew.md). -## Waiting +A Mac camera renders only once AVFoundation asks for NV12 instead of `2vuy`, +which [moq-dev/moq#8251](https://github.com/moq-dev/moq/pull/8251) did on moq's +`main` (`2be959ba9`) after the studio's pin; [pin-main](/quest/m0/pin-main.md) +brings it here. Check by hand on a Mac that a camera shows its picture in +program, since the launch check only lists devices and hosted runners have no +camera. -- moq-dev/moq: a Mac camera renders only once AVFoundation asks for NV12 instead of `2vuy`, in review as [moq-dev/moq#8251](https://github.com/moq-dev/moq/pull/8251); done when it merges, `just upgrade-moq` bumps the gitlink here, and a Mac camera shows its picture in program +## Required + +- [The studio is caught up to moq's main](/quest/m0/pin-main.md) - brings moq-dev/moq#8251's NV12 camera fix diff --git a/quest/m0/pin-aac-encode.md b/quest/m0/pin-aac-encode.md index 48e76f0..8afca7b 100644 --- a/quest/m0/pin-aac-encode.md +++ b/quest/m0/pin-aac-encode.md @@ -14,9 +14,7 @@ fixing whatever the new API breaks here. ## Waiting -- moq-dev/moq: both upstream quests above land. They move to moq's m3 in - [moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421), so each link - resolves once that merges. Check: both quests' PRs have +- moq-dev/moq: both upstream quests above land. Check: both quests' PRs have merged on moq's `main`, deleting their files, and moq-audio's AAC encoders are there. A quest file that moved, or went away without its work landing, does not count. diff --git a/quest/m0/pin-audio-filters.md b/quest/m0/pin-audio-filters.md index 0ccffa9..f21a6d9 100644 --- a/quest/m0/pin-audio-filters.md +++ b/quest/m0/pin-audio-filters.md @@ -12,9 +12,7 @@ fixing whatever the new API breaks here. ## Waiting -- moq-dev/moq: the upstream quest above lands. It moves to moq's m3 in - [moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421), so the link - resolves once that merges. Check: that quest's PR has +- moq-dev/moq: the upstream quest above lands. Check: that quest's PR has merged on moq's `main`, deleting its file, and moq-audio's filter chain is there. A quest file that moved, or went away without its work landing, does not count. diff --git a/quest/m0/pin-sync.md b/quest/m0/pin-sync.md index eef7380..a322888 100644 --- a/quest/m0/pin-sync.md +++ b/quest/m0/pin-sync.md @@ -54,29 +54,15 @@ kio's `Instant` loses `+` and `-` with a `Duration`, which `playout.rs` and `source.rs` use throughout) all merged after #8272. [pin-main](/quest/m0/pin-main.md) fixes them. -## Waiting - -- Maintainer: decide what a sound stall does to a broadcast source's picture. - `README.md` says "A sound stall longer than that delay pauses the picture - too and starts both again together", which rested on audio owning the - clock's anchor and on `Consumer::playhead()`. #8272 removed both, after the - 2026-11-05 ruling on - [moq-dev/moq#7941](https://github.com/moq-dev/moq/pull/7941): lip sync in - the steady state, desync under congestion, and one slow track never stalls - or moves another. An agent cannot pick: it changes what a creator sees when - a guest's audio stalls. Options: - 1. Adopt moq's decoupled rule (recommended). The picture keeps playing - through a sound stall, and the sound comes back on the clock: each frame - is written at the bus position `due` gives it, and the late part of a - burst is trimmed, as the positioned writer already does. The studio - plays a broadcast as `moq play` and the browser do, which `playout.rs` - sets out to, and `README.md` drops "Each picture waits for the sound - stamped with it" and the pause sentence. - 2. Keep the studio's rule. The `Speaker` keeps a playhead of its own, - `heard` holds pictures on it, and the studio holds the picture back - itself after a stall, since `sync` no longer lets audio hold video. Sound - and picture never part, at the cost of studio-only coupling over an API - built to prevent it. - - Check: the maintainer answers in `/quest-unblock`, and the choice is - recorded here. +Decided 2026-11-12 (maintainer, /quest-unblock): ✅ adopt moq's decoupled +rule for a sound stall (rejected: keeping the studio's rule, with a `Speaker` +playhead of its own that holds the picture after a stall). The picture keeps +playing through a sound stall, and the sound comes back on the clock: each +frame is written at the bus position `due` gives it, and the late part of a +burst is trimmed, as the positioned writer already does. The studio plays a +broadcast as `moq play` and the browser do, following the 2026-11-05 ruling on +[moq-dev/moq#7941](https://github.com/moq-dev/moq/pull/7941): lip sync in the +steady state, desync under congestion, and one slow track never stalls or +moves another. `README.md` drops "Each picture waits for the sound stamped +with it" and "A sound stall longer than that delay pauses the picture too and +starts both again together", and says the picture plays on through a stall. diff --git a/quest/m2/compose/README.md b/quest/m2/compose/README.md index bcadb24..8642429 100644 --- a/quest/m2/compose/README.md +++ b/quest/m2/compose/README.md @@ -2,13 +2,13 @@ ## Goal -A studio broadcast carries each independently published source as its own -track, plus a layer layout track that says where and when each one shows. -Players composite those sources over the program and mix their audio in, so -a viewer sees the creator's picture while those sources travel, and can be -toggled, as their own tracks. The end goal is moq's own players, `moq play` -and `@moq/watch`, playing a studio broadcast this way from a layer layout -that hang specifies. +The studio publishes a composition: a broadcast whose catalog lists the +program and each independently published source, local or guest, as +broadcasts of their own, and says where and when each one shows. Players +composite those sources over the program and mix their audio in, so a viewer +sees the creator's picture while those sources travel, and can be toggled, as +their own broadcasts. The end goal is moq's own players, `moq play` and +`@moq/watch`, playing a composition that hang specifies. This README's own work is the end-to-end test: a studio broadcast with an independently published camera and microphone plays in `moq play` and @@ -51,12 +51,15 @@ land before replanning: - **Milestone:** ✅ m2 (rejected: m1; staying in m3). The moq quests join moq's m3. -Still settled from 2026-10-26: local sources are tracks in the studio's one -broadcast, and guests stay their own broadcasts. How the layout names a -guest is open in the [layer layout](/quest/m2/compose/layout.md)'s Waiting: -hang's `include` requires an included broadcast to share the listing -catalog's timeline, clock, and join interval, which a guest's does not. +Decided 2026-11-12 (maintainer, /quest-unblock): ✅ a composition, its own +broadcast listing a catalog per source, each on its own timeline with an +offset onto the composition's clock, and every independently published +source, local or guest, its own broadcast (rejected: a layer layout track in +the program's broadcast naming tracks; local sources as tracks of the +program's broadcast, the 2026-10-26 rule this reverses). The studio +prototypes the format first. See [the composition](/quest/m2/compose/layout.md). -moq's side, in its own quest tree (m3): hang specifies a layer layout track, -and `moq play` and `@moq/watch` composite it. +moq's side, in its own quest tree (m3): hang specifies the composition, and +`moq play` and `@moq/watch` composite it. moq's three layers quests were +planned for a layer layout track and need replanning there. diff --git a/quest/m2/compose/hang-layers.md b/quest/m2/compose/hang-layers.md index 1a9e38d..f3b2b43 100644 --- a/quest/m2/compose/hang-layers.md +++ b/quest/m2/compose/hang-layers.md @@ -1,10 +1,9 @@ -# [S] The studio publishes hang's layer layout +# [S] The studio publishes hang's composition ## Goal -The `moq` submodule includes hang's layer layout track and `moq play`'s layer -compositing. The studio publishes hang's format instead of the provisional -`live.kixel.studio.layers`, and the `moq-studio --watch` prototype is gone, with +The `moq` submodule includes hang's composition and `moq play`'s compositing +of it. The studio publishes hang's format instead of its provisional one, and the `moq-studio --watch` prototype is gone, with `README.md` pointing viewers at `moq play` instead. ## Plan @@ -18,21 +17,28 @@ the studio to hang's types, and delete the Decided 2026-11-09 (maintainer, /quest-audit): ✅ the wait names moq's three quests (rejected: "both quests", unnamed, which left out `@moq/watch`). +Decided 2026-11-12 (maintainer, /quest-unblock): hang specifies a +composition, a broadcast listing a catalog per source, rather than a layer +layout track; see [the composition](/quest/m2/compose/layout.md), whose +prototype moq's spec starts from. moq's three quests below were planned for a +layer layout track and are replanned in moq's tree. + ## Waiting -- moq-dev/moq: its three layer quests land, planned in moq's m3 from the - `moq-studio --watch` prototype's format, the README's "Layer layout": +- moq-dev/moq: hang specifies the composition, and `moq play` and + `@moq/watch` composite it, which the [epic](/quest/m2/compose/README.md)'s + test plays in. moq's m3 plans this today as three layer quests, [hang specifies a layer layout track](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/spec.md), [moq play composites layers](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/play.md), and [The watch player composites layers](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/watch.md), - which the [epic](/quest/m2/compose/README.md)'s test plays in. Check: all - three quests' PRs have merged on moq's `main`, deleting their files, hang - exposes a layer layout whose audio layers carry gain, mute, pan, and delay, - and both players apply all four. A quest file that moved, or went away - without its work landing, does not count. + to be replanned there for the composition. Check: moq's `main` has hang's + composition, whose sound entries carry gain, mute, pan, and delay, and both + players apply all four. A quest file that moved or was replanned, without + that work landing, does not count. ## Required +- [The studio publishes a composition](/quest/m2/compose/layout.md) - the prototype hang's spec starts from - [Viewer controls](/quest/m2/compose/controls.md) - toggles and priorities the spec carries - [The studio is caught up to moq's main](/quest/m0/pin-main.md) - moq's breaking changes since moq-dev/moq#8272, so this pin takes only its own diff --git a/quest/m2/compose/layout.md b/quest/m2/compose/layout.md index 9f415c9..830d3bd 100644 --- a/quest/m2/compose/layout.md +++ b/quest/m2/compose/layout.md @@ -1,50 +1,53 @@ -# [M] Guests in the layer layout +# [L] The studio publishes a composition ## Goal -A guest's broadcast source can be published independently, like a local -source: the program leaves it out, and the provisional -`live.kixel.studio.layers` layout track names the guest's broadcast so a player -places it over the program, and mixes its sound, at the right time. +The studio publishes a composition: a broadcast of its own whose catalog +lists other broadcasts, the program and each independently published source, +local or guest, with where and when each shows and how its sound mixes. Each +listed broadcast keeps its own timeline, and the composition carries the +offset that puts it on the composition's media clock. The `moq-studio --watch` +prototype (`rs/moq-studio/src/watch.rs`) plays a composition, guests +included, and it replaces the provisional `live.kixel.studio.layers` track +(`rs/moq-scene/src/layout.rs`, and the README's "Layer layout"). ## Plan -The layout track itself landed for local sources: `rs/moq-scene/src/layout.rs` -and the README's "Layer layout". A guest stays its own broadcast (settled -2026-10-26), so its layer names that broadcast rather than an entry under the -catalog's `live.kixel.studio` `sources`, and carries what a player needs to put -the guest's frames on the program's media clock. Today -`Source::publishes` refuses a broadcast source, since it has no track in the -studio's broadcast. - -Bump the layout's `version` if the change is not additive. - -The `moq-studio --watch` prototype (`rs/moq-studio/src/watch.rs`) plays only -local sources' layers today: its layers play tracks of the program's own -broadcast on the program's clock, so a guest's layer needs the prototype to -follow the named broadcast, on the offset the layout carries. - -## Waiting - -- Maintainer: decide how the layout names a guest's broadcast. The epic - settled on 2026-10-26 that guests stay their own broadcasts, named through - hang's catalog references, then `contributions`, now `include` - ([moq-dev/moq#7943](https://github.com/moq-dev/moq/pull/7943)). The hang - draft at the pin requires an include to share the listing catalog's PTS - timeline, `media.clock`, and `media.join.interval`, and a guest's broadcast - keeps its own, so a player that takes `include` at its word places a - guest's frames at the wrong time. Options: - - The layout names each guest's broadcast itself and carries the offset - from the guest's timeline onto the program's media clock, the one the - studio applies when it plays the guest (recommended): `include` keeps - its meaning, and the timing stays in the track that already says when - each layer shows. - - moq relaxes `include`, or adds a sibling map, for broadcasts on their own - timeline, with the offset in the catalog: a moq quest and a pin, and every - hang consumer learns a second kind of include. - - The studio republishes each guest's tracks in its own broadcast, restamped - onto the program's clock: no spec change, but it reverses the settled - "guests stay their own broadcasts" and relays every guest twice. - - Check: the maintainer answers in /quest-unblock, and the choice is recorded - in this Plan and the epic's. +Decided 2026-11-12 (maintainer, /quest-unblock), settling how the layout +names a guest: + +- **Naming:** ✅ a composition, a catalog of catalogs (rejected: the layer + layout naming each guest's broadcast with an offset; moq relaxing hang's + `include` for broadcasts on their own timeline; republishing guests' + tracks in the studio's broadcast). hang's `include` + ([moq-dev/moq#7943](https://github.com/moq-dev/moq/pull/7943)) requires an + included broadcast to share the listing catalog's PTS timeline, + `media.clock`, and `media.join.interval`, which a guest's does not; a + composition keeps `include`'s meaning and says how each timeline maps. +- **Home:** ✅ its own broadcast (rejected: a section of the program's + catalog, which is `include` with a different clock). The program is one of + its children, and a player that knows no compositions still plays the + program. +- **Local sources:** ✅ every independently published source is its own + broadcast (rejected: layers naming a broadcast and track, with local + sources staying tracks of the program's broadcast at offset zero). This + reverses the 2026-10-26 rule that local sources are tracks in the studio's + one broadcast, so the composition lists only catalogs. +- **Format:** ✅ the studio prototypes it, provisional and versioned, before + moq's hang specifies it (rejected: straight to hang's spec), as the + 2026-11-08 layer layout did. + +Recommended, not decided: the composition's media clock is the program's, so +the program's offset is zero; each child's entry carries what the provisional +layer format carries per layer today (placement, crop, opacity, visibility, +keyframes, toggleability, priority, and gain, mute, pan, and delay for +sound); and a change is a new snapshot group stamped on the composition's +clock, as the layers track is now. + +`Source::publishes` refuses a broadcast source today, since it has no track +in the studio's broadcast; under a composition a guest needs no track of the +studio's. `README.md`'s "Published independently" and "Layer layout" become +the composition's. + +Bump the format's `version`, since naming broadcasts instead of tracks is not +additive. diff --git a/quest/m2/compose/server.md b/quest/m2/compose/server.md index f719930..5dffb04 100644 --- a/quest/m2/compose/server.md +++ b/quest/m2/compose/server.md @@ -3,8 +3,8 @@ ## Goal A headless compositor binary in this repository subscribes to a studio -broadcast's program, its independently published sources, and its layer -layout, and publishes the flattened program, headless (a GPU, or lavapipe +composition's program and its independently published sources, and +publishes the flattened program, headless (a GPU, or lavapipe without one). It can push that program to RTMP and WHIP through the studio's egress library. So destinations that cannot compose, such as RTMP, WHIP, HLS, and legacy players, still see the sources the creator published @@ -26,8 +26,10 @@ moq-cli, but moq-scene moved here with the studio, so the command lives here too and moq never depends on this repository. Settle its name in this quest and document it in `README.md`. -It flattens the layer layout the studio publishes on `live.kixel.studio.layers` -(`rs/moq-scene/src/layout.rs`, and the README's "Layer layout"). +It flattens the composition the studio publishes +([the composition](/quest/m2/compose/layout.md)); until that lands, it +flattens the provisional `live.kixel.studio.layers` track (`rs/moq-scene/src/layout.rs`, +and the README's "Layer layout"). ## Required @@ -35,4 +37,4 @@ It flattens the layer layout the studio publishes on `live.kixel.studio.layers` ## Related -- [Guests in the layer layout](/quest/m2/compose/layout.md) - guests' layers, which it flattens once they land +- [The studio publishes a composition](/quest/m2/compose/layout.md) - the format it flattens