Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quest/m0/audio-filters.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Guest, broadcast, media, browser, and replay sound stays unfiltered.
Changed 2026-11-02 (maintainer), when the studio moved out of moq: the chain
itself, its filters, and the out-of-process plugin host are moq-audio work in
[moq-audio filter chain](https://github.com/moq-dev/moq/blob/main/quest/m3/audio-filters.md),
which moves to moq's m3 in [moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421).
in moq's m3.
This quest is the studio's side: the panel, the scene-file shape, and wiring
each source's chain into the program bus.

Expand Down
11 changes: 9 additions & 2 deletions quest/m0/packaging/macos.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,13 @@ protected branches can deploy from, for team `D7D5SDDB5Z` and bundle ID
The certificate (`6WWG279G9M`, G2) expires 2027-12-11, and the profile with it;
see [renewal](/quest/m0/signing-renew.md).

## Waiting
A Mac camera renders only once AVFoundation asks for NV12 instead of `2vuy`,
which [moq-dev/moq#8251](https://github.com/moq-dev/moq/pull/8251) did on moq's
`main` (`2be959ba9`) after the studio's pin; [pin-main](/quest/m0/pin-main.md)
brings it here. Check by hand on a Mac that a camera shows its picture in
program, since the launch check only lists devices and hosted runners have no
camera.

- moq-dev/moq: a Mac camera renders only once AVFoundation asks for NV12 instead of `2vuy`, in review as [moq-dev/moq#8251](https://github.com/moq-dev/moq/pull/8251); done when it merges, `just upgrade-moq` bumps the gitlink here, and a Mac camera shows its picture in program
## Required

- [The studio is caught up to moq's main](/quest/m0/pin-main.md) - brings moq-dev/moq#8251's NV12 camera fix
4 changes: 1 addition & 3 deletions quest/m0/pin-aac-encode.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,7 @@ fixing whatever the new API breaks here.

## Waiting

- moq-dev/moq: both upstream quests above land. They move to moq's m3 in
[moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421), so each link
resolves once that merges. Check: both quests' PRs have
- moq-dev/moq: both upstream quests above land. Check: both quests' PRs have
merged on moq's `main`, deleting their files, and moq-audio's AAC encoders
are there. A quest file that moved, or went away without its work landing,
does not count.
Expand Down
4 changes: 1 addition & 3 deletions quest/m0/pin-audio-filters.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,7 @@ fixing whatever the new API breaks here.

## Waiting

- moq-dev/moq: the upstream quest above lands. It moves to moq's m3 in
[moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421), so the link
resolves once that merges. Check: that quest's PR has
- moq-dev/moq: the upstream quest above lands. Check: that quest's PR has
merged on moq's `main`, deleting its file, and moq-audio's filter chain is
there. A quest file that moved, or went away without its work landing, does
not count.
Expand Down
38 changes: 12 additions & 26 deletions quest/m0/pin-sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,29 +54,15 @@ kio's `Instant` loses `+` and `-` with a `Duration`, which `playout.rs` and
`source.rs` use throughout) all merged after #8272.
[pin-main](/quest/m0/pin-main.md) fixes them.

## Waiting

- Maintainer: decide what a sound stall does to a broadcast source's picture.
`README.md` says "A sound stall longer than that delay pauses the picture
too and starts both again together", which rested on audio owning the
clock's anchor and on `Consumer::playhead()`. #8272 removed both, after the
2026-11-05 ruling on
[moq-dev/moq#7941](https://github.com/moq-dev/moq/pull/7941): lip sync in
the steady state, desync under congestion, and one slow track never stalls
or moves another. An agent cannot pick: it changes what a creator sees when
a guest's audio stalls. Options:
1. Adopt moq's decoupled rule (recommended). The picture keeps playing
through a sound stall, and the sound comes back on the clock: each frame
is written at the bus position `due` gives it, and the late part of a
burst is trimmed, as the positioned writer already does. The studio
plays a broadcast as `moq play` and the browser do, which `playout.rs`
sets out to, and `README.md` drops "Each picture waits for the sound
stamped with it" and the pause sentence.
2. Keep the studio's rule. The `Speaker` keeps a playhead of its own,
`heard` holds pictures on it, and the studio holds the picture back
itself after a stall, since `sync` no longer lets audio hold video. Sound
and picture never part, at the cost of studio-only coupling over an API
built to prevent it.

Check: the maintainer answers in `/quest-unblock`, and the choice is
recorded here.
Decided 2026-11-12 (maintainer, /quest-unblock): ✅ adopt moq's decoupled
rule for a sound stall (rejected: keeping the studio's rule, with a `Speaker`
playhead of its own that holds the picture after a stall). The picture keeps
playing through a sound stall, and the sound comes back on the clock: each
frame is written at the bus position `due` gives it, and the late part of a
burst is trimmed, as the positioned writer already does. The studio plays a
broadcast as `moq play` and the browser do, following the 2026-11-05 ruling on
[moq-dev/moq#7941](https://github.com/moq-dev/moq/pull/7941): lip sync in the
steady state, desync under congestion, and one slow track never stalls or
moves another. `README.md` drops "Each picture waits for the sound stamped
with it" and "A sound stall longer than that delay pauses the picture too and
starts both again together", and says the picture plays on through a stall.
31 changes: 17 additions & 14 deletions quest/m2/compose/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

## Goal

A studio broadcast carries each independently published source as its own
track, plus a layer layout track that says where and when each one shows.
Players composite those sources over the program and mix their audio in, so
a viewer sees the creator's picture while those sources travel, and can be
toggled, as their own tracks. The end goal is moq's own players, `moq play`
and `@moq/watch`, playing a studio broadcast this way from a layer layout
that hang specifies.
The studio publishes a composition: a broadcast whose catalog lists the
program and each independently published source, local or guest, as
broadcasts of their own, and says where and when each one shows. Players
composite those sources over the program and mix their audio in, so a viewer
sees the creator's picture while those sources travel, and can be toggled, as
their own broadcasts. The end goal is moq's own players, `moq play` and
`@moq/watch`, playing a composition that hang specifies.

This README's own work is the end-to-end test: a studio broadcast with an
independently published camera and microphone plays in `moq play` and
Expand Down Expand Up @@ -51,12 +51,15 @@ land before replanning:
- **Milestone:** ✅ m2 (rejected: m1; staying in m3). The moq quests join
moq's m3.

Still settled from 2026-10-26: local sources are tracks in the studio's one
broadcast, and guests stay their own broadcasts. How the layout names a
guest is open in the [layer layout](/quest/m2/compose/layout.md)'s Waiting:
hang's `include` requires an included broadcast to share the listing
catalog's timeline, clock, and join interval, which a guest's does not.
Decided 2026-11-12 (maintainer, /quest-unblock): ✅ a composition, its own
broadcast listing a catalog per source, each on its own timeline with an
offset onto the composition's clock, and every independently published
source, local or guest, its own broadcast (rejected: a layer layout track in
the program's broadcast naming tracks; local sources as tracks of the
program's broadcast, the 2026-10-26 rule this reverses). The studio
prototypes the format first. See [the composition](/quest/m2/compose/layout.md).

moq's side, in its own quest tree (m3): hang specifies a layer layout track,
and `moq play` and `@moq/watch` composite it.
moq's side, in its own quest tree (m3): hang specifies the composition, and
`moq play` and `@moq/watch` composite it. moq's three layers quests were
planned for a layer layout track and need replanning there.

28 changes: 17 additions & 11 deletions quest/m2/compose/hang-layers.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
# [S] The studio publishes hang's layer layout
# [S] The studio publishes hang's composition

## Goal

The `moq` submodule includes hang's layer layout track and `moq play`'s layer
compositing. The studio publishes hang's format instead of the provisional
`live.kixel.studio.layers`, and the `moq-studio --watch` prototype is gone, with
The `moq` submodule includes hang's composition and `moq play`'s compositing
of it. The studio publishes hang's format instead of its provisional one, and the `moq-studio --watch` prototype is gone, with
`README.md` pointing viewers at `moq play` instead.

## Plan
Expand All @@ -18,21 +17,28 @@ the studio to hang's types, and delete the
Decided 2026-11-09 (maintainer, /quest-audit): ✅ the wait names moq's three
quests (rejected: "both quests", unnamed, which left out `@moq/watch`).

Decided 2026-11-12 (maintainer, /quest-unblock): hang specifies a
composition, a broadcast listing a catalog per source, rather than a layer
layout track; see [the composition](/quest/m2/compose/layout.md), whose
prototype moq's spec starts from. moq's three quests below were planned for a
layer layout track and are replanned in moq's tree.

## Waiting

- moq-dev/moq: its three layer quests land, planned in moq's m3 from the
`moq-studio --watch` prototype's format, the README's "Layer layout":
- moq-dev/moq: hang specifies the composition, and `moq play` and
`@moq/watch` composite it, which the [epic](/quest/m2/compose/README.md)'s
test plays in. moq's m3 plans this today as three layer quests,
[hang specifies a layer layout track](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/spec.md),
[moq play composites layers](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/play.md),
and
[The watch player composites layers](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/watch.md),
which the [epic](/quest/m2/compose/README.md)'s test plays in. Check: all
three quests' PRs have merged on moq's `main`, deleting their files, hang
exposes a layer layout whose audio layers carry gain, mute, pan, and delay,
and both players apply all four. A quest file that moved, or went away
without its work landing, does not count.
to be replanned there for the composition. Check: moq's `main` has hang's
composition, whose sound entries carry gain, mute, pan, and delay, and both
players apply all four. A quest file that moved or was replanned, without
that work landing, does not count.

## Required

- [The studio publishes a composition](/quest/m2/compose/layout.md) - the prototype hang's spec starts from
- [Viewer controls](/quest/m2/compose/controls.md) - toggles and priorities the spec carries
- [The studio is caught up to moq's main](/quest/m0/pin-main.md) - moq's breaking changes since moq-dev/moq#8272, so this pin takes only its own
91 changes: 47 additions & 44 deletions quest/m2/compose/layout.md
Original file line number Diff line number Diff line change
@@ -1,50 +1,53 @@
# [M] Guests in the layer layout
# [L] The studio publishes a composition

## Goal

A guest's broadcast source can be published independently, like a local
source: the program leaves it out, and the provisional
`live.kixel.studio.layers` layout track names the guest's broadcast so a player
places it over the program, and mixes its sound, at the right time.
The studio publishes a composition: a broadcast of its own whose catalog
lists other broadcasts, the program and each independently published source,
local or guest, with where and when each shows and how its sound mixes. Each
listed broadcast keeps its own timeline, and the composition carries the
offset that puts it on the composition's media clock. The `moq-studio --watch`
prototype (`rs/moq-studio/src/watch.rs`) plays a composition, guests
included, and it replaces the provisional `live.kixel.studio.layers` track
(`rs/moq-scene/src/layout.rs`, and the README's "Layer layout").

## Plan

The layout track itself landed for local sources: `rs/moq-scene/src/layout.rs`
and the README's "Layer layout". A guest stays its own broadcast (settled
2026-10-26), so its layer names that broadcast rather than an entry under the
catalog's `live.kixel.studio` `sources`, and carries what a player needs to put
the guest's frames on the program's media clock. Today
`Source::publishes` refuses a broadcast source, since it has no track in the
studio's broadcast.

Bump the layout's `version` if the change is not additive.

The `moq-studio --watch` prototype (`rs/moq-studio/src/watch.rs`) plays only
local sources' layers today: its layers play tracks of the program's own
broadcast on the program's clock, so a guest's layer needs the prototype to
follow the named broadcast, on the offset the layout carries.

## Waiting

- Maintainer: decide how the layout names a guest's broadcast. The epic
settled on 2026-10-26 that guests stay their own broadcasts, named through
hang's catalog references, then `contributions`, now `include`
([moq-dev/moq#7943](https://github.com/moq-dev/moq/pull/7943)). The hang
draft at the pin requires an include to share the listing catalog's PTS
timeline, `media.clock`, and `media.join.interval`, and a guest's broadcast
keeps its own, so a player that takes `include` at its word places a
guest's frames at the wrong time. Options:
- The layout names each guest's broadcast itself and carries the offset
from the guest's timeline onto the program's media clock, the one the
studio applies when it plays the guest (recommended): `include` keeps
its meaning, and the timing stays in the track that already says when
each layer shows.
- moq relaxes `include`, or adds a sibling map, for broadcasts on their own
timeline, with the offset in the catalog: a moq quest and a pin, and every
hang consumer learns a second kind of include.
- The studio republishes each guest's tracks in its own broadcast, restamped
onto the program's clock: no spec change, but it reverses the settled
"guests stay their own broadcasts" and relays every guest twice.

Check: the maintainer answers in /quest-unblock, and the choice is recorded
in this Plan and the epic's.
Decided 2026-11-12 (maintainer, /quest-unblock), settling how the layout
names a guest:

- **Naming:** ✅ a composition, a catalog of catalogs (rejected: the layer
layout naming each guest's broadcast with an offset; moq relaxing hang's
`include` for broadcasts on their own timeline; republishing guests'
tracks in the studio's broadcast). hang's `include`
([moq-dev/moq#7943](https://github.com/moq-dev/moq/pull/7943)) requires an
included broadcast to share the listing catalog's PTS timeline,
`media.clock`, and `media.join.interval`, which a guest's does not; a
composition keeps `include`'s meaning and says how each timeline maps.
- **Home:** ✅ its own broadcast (rejected: a section of the program's
catalog, which is `include` with a different clock). The program is one of
its children, and a player that knows no compositions still plays the
program.
- **Local sources:** ✅ every independently published source is its own
broadcast (rejected: layers naming a broadcast and track, with local
sources staying tracks of the program's broadcast at offset zero). This
reverses the 2026-10-26 rule that local sources are tracks in the studio's
one broadcast, so the composition lists only catalogs.
- **Format:** ✅ the studio prototypes it, provisional and versioned, before
moq's hang specifies it (rejected: straight to hang's spec), as the
2026-11-08 layer layout did.

Recommended, not decided: the composition's media clock is the program's, so
the program's offset is zero; each child's entry carries what the provisional
layer format carries per layer today (placement, crop, opacity, visibility,
keyframes, toggleability, priority, and gain, mute, pan, and delay for
sound); and a change is a new snapshot group stamped on the composition's
clock, as the layers track is now.

`Source::publishes` refuses a broadcast source today, since it has no track
in the studio's broadcast; under a composition a guest needs no track of the
studio's. `README.md`'s "Published independently" and "Layer layout" become
the composition's.

Bump the format's `version`, since naming broadcasts instead of tracks is not
additive.
12 changes: 7 additions & 5 deletions quest/m2/compose/server.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
## Goal

A headless compositor binary in this repository subscribes to a studio
broadcast's program, its independently published sources, and its layer
layout, and publishes the flattened program, headless (a GPU, or lavapipe
composition's program and its independently published sources, and
publishes the flattened program, headless (a GPU, or lavapipe
without one). It can push that program to RTMP and WHIP through the studio's
egress library. So destinations that cannot compose, such as RTMP, WHIP, HLS,
and legacy players, still see the sources the creator published
Expand All @@ -26,13 +26,15 @@ moq-cli, but moq-scene moved here with the studio, so the command lives here
too and moq never depends on this repository. Settle its name in this quest
and document it in `README.md`.

It flattens the layer layout the studio publishes on `live.kixel.studio.layers`
(`rs/moq-scene/src/layout.rs`, and the README's "Layer layout").
It flattens the composition the studio publishes
([the composition](/quest/m2/compose/layout.md)); until that lands, it
flattens the provisional `live.kixel.studio.layers` track (`rs/moq-scene/src/layout.rs`,
and the README's "Layer layout").

## Required

- [Destinations](/quest/m0/destinations.md) - the egress library the server pushes RTMP and WHIP through

## Related

- [Guests in the layer layout](/quest/m2/compose/layout.md) - guests' layers, which it flattens once they land
- [The studio publishes a composition](/quest/m2/compose/layout.md) - the format it flattens