From d5785f730abd03b32e3bb5435b5f5442e4ded540 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 2 Nov 2026 16:49:42 -0800 Subject: [PATCH 1/2] quest: plan HLS listing of catalog contributions Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 1 + quest/m0/hls-contributions.md | 85 +++++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+) create mode 100644 quest/m0/hls-contributions.md diff --git a/quest/m0/README.md b/quest/m0/README.md index 90b85920fba..bd3d9b90d05 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -38,3 +38,4 @@ since cutting segments from first-frame pts waits on nothing in m1. - [More tests under load](/quest/m0/test-flakes-2/README.md) - load-only and queue failures, one quest per flake, fixed at the cause, since each one fails a whole batch - [Backport the dot-segment and auth-redirect refusals](/quest/m0/backport-release-hardening.md) - `release`'s relay refuses dot-segment paths and its auth client refuses redirects - [HLS export on a declared grid](/quest/m0/hls-media-grid.md) - the catalog's `hls` field and record tracks go away, and the export numbers segments on `media.join.interval` from media tracks +- [HLS lists contributions](/quest/m0/hls-contributions.md) - an HLS master lists each catalog contribution's renditions, and a contribution's media is subscribed only while players fetch it diff --git a/quest/m0/hls-contributions.md b/quest/m0/hls-contributions.md new file mode 100644 index 00000000000..b781b38b7ec --- /dev/null +++ b/quest/m0/hls-contributions.md @@ -0,0 +1,85 @@ +# [L] HLS lists a broadcast's contributions without starting their media + +## Goal + +A moq-hls master playlist lists the audio and video renditions of each +contribution in the broadcast's catalog (the root `contributions` map) beside +the broadcast's own. It reads a contribution's catalog to list them, and +subscribes to a contribution rendition's media tracks only once a player +requests that rendition, so an on-demand contribution, such as a transcode +ladder, encodes only the rungs players fetch. Once no player fetches a +contribution rendition, its media subscription ends. + +## Plan + +**Reading the catalog is the listing.** A master playlist variant needs a +`BANDWIDTH` and a media playlist, so moq-hls cannot list a contribution's +renditions without its catalog. The master opens each contribution's catalog +while the broadcast is being served and follows it, but no media track. The +hang draft's Contributions section says a consumer opens a contribution's +catalog only when it selects it; an HLS master selects every contribution it +lists, so add that to the draft, and that a publisher producing media on +demand answers the catalog without starting media. `moq-transcode` already +does: only rung track demand starts an encode. + +**Resolve like a rendition's reference.** Bind each contribution with +`moq_mux::Source`'s binding and pinning (`Source::bind`, as +`rs/moq-hls/src/export/rendition.rs` binds a rendition's `broadcast` +reference), through the server's own origin. A contribution follows the same +rules as a sibling reference today: denied by the origin's scope, it is left +out of the master and its URLs answer `403`; unresolved, it is listed once it +resolves; restarted, its renditions follow the new instance. Do not add a +second resolver. + +**Segments line up with the broadcast's.** Players switch variants by +segment number, so a contribution's renditions share the broadcast's grid: +its catalog must declare the same `media.join.interval` on the same timeline. +A contribution declaring another interval or none is not HLS-renderable with +the broadcast and is left out of the master, as a rendition that cannot be +exported is today. + +**Lazy media.** A contribution rendition holds no media subscription until a +request reaches its URLs. The first one subscribes to that rendition's tracks +only, and its media playlist answers once it lists a segment. The +subscription ends once no request has reached the rendition for longer than a +player reloading at the target duration would allow (RFC 8216 6.3.4); pick the +multiple and say why in the code, and a later request starts it again. +Today's renditions keep their eager subscription: the broadcast's own media is +published anyway, so only contributions change. + +**URLs and names.** A contribution's rendition IDs belong to its own catalog, +so its URLs nest under the contribution ID inside the broadcast's instance +(for example `@{instance}/contribution/{id}/{kind}/{rendition}/media.m3u8`; +the implementation settles the spelling), and audio `NAME`s stay unique within +their group. A path-based gate in front of the server sees the broadcast's +path, so a viewer allowed the broadcast can play its contributions, as with a +sibling reference. Video variants join the broadcast's audio groups. + +**Out of scope, with reasons.** The DASH manifest keeps listing only the +broadcast's own renditions: a `$Time$` timeline needs every listed +representation's media, which defeats the lazy start. The recorder's +`renditions::Consumer` keeps yielding only the broadcast's own renditions: a +contribution is its own broadcast with its own recording. + +**Tests.** A master lists the broadcast's and a contribution's renditions +while the contribution's media tracks have no demand. A media playlist request +starts demand on that rendition's tracks only, idle release ends it, and a +later request starts it again. A contribution that is unannounced at first is +listed once it resolves. A restarted contribution is followed. A denied one is +left out and its URLs answer `403`. A mismatched or missing interval leaves the +contribution out. A contribution rendition sharing a name with one of the +broadcast's own gets distinct URLs and names. Unit tests mock time for the idle +release. + +Update `doc/bin/hls.md` and the hang draft's Contributions section. + +Public API: additive in moq-hls (the new URL routes). Wire: none; the draft +gains the HLS listing rule and the on-demand publisher's catalog answer. + +## Required + +- [HLS export on a declared grid](/quest/m0/hls-media-grid.md) - the export reads media tracks on `media.join.interval`, which both the shared segment numbering and the lazy media subscription build on + +## Related + +- [The HLS server follows each path once](/quest/m2/hls-follow-once.md) - also changes how the server resolves a path From 5fe085a1fd1f4bf2b8ba8ccb781ee5b546ed19ee Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 2 Nov 2026 17:47:45 -0800 Subject: [PATCH 2/2] quest: follow contribution instances and hold demand during requests Records the maintainer's 2026-11-02 decision that the master reads each contribution's catalog. Replaces Source::bind, which never follows a restart, with Source::follow and a per-instance Source. Keeps a rendition's demand while a request is still in progress. Co-Authored-By: Claude Opus 5.5 --- quest/m0/hls-contributions.md | 48 ++++++++++++++++++++++------------- 1 file changed, 31 insertions(+), 17 deletions(-) diff --git a/quest/m0/hls-contributions.md b/quest/m0/hls-contributions.md index b781b38b7ec..5737bb73cb5 100644 --- a/quest/m0/hls-contributions.md +++ b/quest/m0/hls-contributions.md @@ -12,9 +12,12 @@ contribution rendition, its media subscription ends. ## Plan -**Reading the catalog is the listing.** A master playlist variant needs a -`BANDWIDTH` and a media playlist, so moq-hls cannot list a contribution's -renditions without its catalog. The master opens each contribution's catalog +**Reading the catalog is the listing (decided).** A master playlist variant +needs a `BANDWIDTH` and codecs, so moq-hls cannot list a contribution's +renditions without its catalog. The maintainer decided on 2026-11-02 that the +master reads each contribution's catalog when it is served: only the +contribution's media waits for a player's media playlist request, and a +catalog read starts no encoding. The master opens each contribution's catalog while the broadcast is being served and follows it, but no media track. The hang draft's Contributions section says a consumer opens a contribution's catalog only when it selects it; an HLS master selects every contribution it @@ -22,14 +25,21 @@ lists, so add that to the draft, and that a publisher producing media on demand answers the catalog without starting media. `moq-transcode` already does: only rung track demand starts an encode. -**Resolve like a rendition's reference.** Bind each contribution with -`moq_mux::Source`'s binding and pinning (`Source::bind`, as -`rs/moq-hls/src/export/rendition.rs` binds a rendition's `broadcast` -reference), through the server's own origin. A contribution follows the same -rules as a sibling reference today: denied by the origin's scope, it is left -out of the master and its URLs answer `403`; unresolved, it is listed once it -resolves; restarted, its renditions follow the new instance. Do not add a -second resolver. +**Follow each contribution's instances.** A contribution is its own broadcast +with its own catalog, so follow it the way the server follows a broadcast it +serves, not the way a rendition binds a sibling. `Source::bind` pins a path's +first answer and never resolves it again after a replacement +(`rs/moq-mux/src/source.rs`), and a rendition whose bound sibling ends ends +with it (`Media` in `rs/moq-hls/src/export/rendition.rs`). So resolve each +contribution entry with `Source::follow` (`moq_mux::follow::Instances`), +through the server's own origin, and build a `moq_mux::Source` for each +instance it hands out, so the instance's catalog and media reads stay pinned +to that instance. A new instance replaces the contribution's renditions and an +ended one drops them from the master until the next one starts. An entry +naming an `instance` follows only that stream, as the draft's +`instance` rules require. Unresolved, a contribution is listed once it +resolves; denied by the origin's scope, it is left out of the master and its +URLs answer `403`. Do not add a second resolver. **Segments line up with the broadcast's.** Players switch variants by segment number, so a contribution's renditions share the broadcast's grid: @@ -41,9 +51,11 @@ exported is today. **Lazy media.** A contribution rendition holds no media subscription until a request reaches its URLs. The first one subscribes to that rendition's tracks only, and its media playlist answers once it lists a segment. The -subscription ends once no request has reached the rendition for longer than a -player reloading at the target duration would allow (RFC 8216 6.3.4); pick the -multiple and say why in the code, and a later request starts it again. +subscription ends once no request to the rendition is in progress and none has +finished for longer than a player reloading at the target duration would allow +(RFC 8216 6.3.4); a slow response, such as a first media playlist waiting on an +on-demand encode, keeps it. Pick the multiple and say why in the code, and a +later request starts it again. Today's renditions keep their eager subscription: the broadcast's own media is published anyway, so only contributions change. @@ -63,9 +75,11 @@ contribution is its own broadcast with its own recording. **Tests.** A master lists the broadcast's and a contribution's renditions while the contribution's media tracks have no demand. A media playlist request -starts demand on that rendition's tracks only, idle release ends it, and a -later request starts it again. A contribution that is unannounced at first is -listed once it resolves. A restarted contribution is followed. A denied one is +starts demand on that rendition's tracks only, a request still in progress +past the idle window keeps it, idle release ends it, and a later request starts +it again. A contribution that is unannounced at first is +listed once it resolves. A restarted contribution is followed onto its new instance, and one naming an +`instance` is not. A denied one is left out and its URLs answer `403`. A mismatched or missing interval leaves the contribution out. A contribution rendition sharing a name with one of the broadcast's own gets distinct URLs and names. Unit tests mock time for the idle