Skip to content

feat(hang): catalog references may name their broadcast instance - #5699

Merged
kixelated merged 10 commits into
mainfrom
quest/m0/broadcast-epoch/catalog-references
Oct 21, 2026
Merged

kixelated merged 10 commits into
mainfrom
quest/m0/broadcast-epoch/catalog-references

Conversation

@kixelated

@kixelated kixelated commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A catalog reference (broadcast: "./source") names a path, not a publisher instance. A reader that resolves it after the sibling restarted silently reads different content, and nothing in the catalog can say which instance it meant.

Changes

  • Schema (Rust hang, JS @moq/hang): optional epoch beside broadcast on video, audio, text, json and binary entries, and replayEpoch beside the archive's replay. Canonical UUIDv7 text, reusing moq_net::Epoch / Epoch.parse; anything else refuses the catalog. hang::catalog::Reference gains epoch().
  • Shared resolution (moq_mux::Source): bind, resolve, subscribe_track take the reference's epoch. The path's first resolved broadcast stays pinned as before; a binding whose broadcast proves another epoch fails with Error::EpochMismatch instead of falling back, and a later catalog update cannot replace the pin. Over an epochless route it is best effort. Self-references check the catalog broadcast's own epoch.
  • Readers: TS/fMP4/MKV/FLV exporters, data readers, moq play, moqsrc, HLS, and WebRTC egress pass the epoch. moqsrc keys pumps by it, so an epoch-only change replaces the pump. FFI and libmoq pass None for now (see follow-ups).
  • @moq/watch: relativeBroadcast(effect, rel, epoch?) returns nothing while the followed broadcast proves another epoch. A reference without an epoch keeps following restarts (2026-10-14 decision); one with an epoch never follows onto another instance.
  • MSF export warns that it drops an epoch it cannot carry.
  • Docs: Hang draft (field-epoch, replayEpoch, changelog; MoQ Transport with the epoch extension carries an Epoch), doc/concept/hang.md, and MIGRATE.md entries for hang and moq-mux.
  • Quests: deleted catalog-references.md; added m0/broadcast-epoch/reference-announcements.md (in the epic's Required list) and m1/reference-epoch-bindings.md.

Reconciled with main (2026-10-20)

Merged main, including #5925 (origin::Consumer::announced_broadcast). Nothing in this PR duplicated it: Source resolves once per path and never follows announcements, which is the follow-up quest's job. What changed:

  • reference-announcements.md now builds its shared follower on announced_broadcast instead of a hand-rolled loop, following unpinned and checking each instance's epoch as Source::bind does. A pinned announced_broadcast(path, Some(epoch)) waits out an epochless route, which would break the best effort decision 5 keeps.
  • sink-restart-follow.md points at reference-announcements.md, since this PR deletes catalog-references.md; stale restart-followers.md and ietf-epochs.md links are fixed.
  • The upgrade note moved out of doc/setup/upgrade.md (main now keeps only cross-package rollout order there) into rs/moq-mux/MIGRATE.md and rs/hang/MIGRATE.md. The wire change is additive, so it adds no rollout-order entry.

Public API and wire impact

  • Rust hang: new epoch / replay_epoch fields (structs are non_exhaustive). New required Reference::epoch(), which breaks external implementors.
  • Rust moq-mux: breaking signatures for Source::bind(rel, epoch), resolve(rel, epoch), subscribe_track(rel, epoch, name). New Error::EpochMismatch.
  • JS @moq/hang: optional schema fields, plus new exports EpochSchema and ReferenceEpoch. @moq/watch: an optional third argument.
  • Wire: additive Hang catalog fields. No transport framing change. Older readers ignore the fields.

Decisions

Settled by the maintainer (comment):

  1. Scope split
    • ✅ Following referenced announcements is its own m0 quest, and binding exposure is m1.
    • Land following and bindings here.
  2. Reference shape and archive field name
    • ✅ Sibling fields: epoch beside broadcast, and replayEpoch on the archive. Older readers ignore them.
    • A nested "path" | { path, epoch? } reference. Rejected: published readers would refuse the whole catalog.
  3. MSF export
    • ✅ Warn and drop the epoch; the exported reference requests without one.
    • Refuse the rendition.
  4. Malformed epoch
    • ✅ Refuses the whole catalog, like any other malformed field.
    • Drop only the rendition.
  5. Unpinned epoch requests
    • ✅ Ask the origin without an epoch and verify the returned one, so epochless routes stay best effort.
    • Pass the epoch to request_broadcast, refusing epochless routes.

Checks

  • just check passed on head f0b2a0a7da (Rust nextest 3600, JS, Python, Go, Dart). One earlier run hit export::redundant_importers_switch_through_bufferbloat_on_lite_07, the known main flake tracked by redundant-failover-drift; the rerun passed.
  • just test interop --all passed on c011d11e4d. The later merge only brought in main and resolved moq-hls's Media::bind against #6087.
  • just drafts check and quest check passed.

Review

Codex reviewed c011d11e4d with three findings, all declined with replies:

Follow-ups

  • Reference announcements: Rust readers follow referenced announcements on announced_broadcast, and @moq/watch drops epoch-mismatched renditions from out.catalog.
  • Exporters and WebRTC egress drop or repick past a failed referenced rendition, for every failure kind, instead of ending the export or idling the line.
  • Reference epoch bindings: UniFFI and the C ABI expose and honor epoch.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits October 15, 2026 23:55
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add an optional `epoch` beside every rendition and data track `broadcast`
reference, and `replayEpoch` beside the archive's `replay`, in Rust and JS.
moq_mux::Source and @moq/watch fail a reference whose broadcast proves
another epoch rather than reading it, and treat it as best effort over an
epochless route. Split the announcement-following readers and binding
exposure into follow-up quests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Quest outcome: this delivers the catalog-references schema, shared resolution, and docs. The 2026-10-14 announcement-following readers and the binding exposure are split into quest/m0/broadcast-epoch/reference-announcements.md and quest/m1/reference-epoch-bindings.md. The PR stays a draft until a maintainer settles the open decisions listed in the description. Scoped just check and just drafts check pass; just test interop --all was not run.

(Written by Claude Opus 5.5)

…-epoch/catalog-references

# Conflicts:
#	quest/m0/broadcast-epoch/README.md
#	quest/m0/broadcast-epoch/catalog-references.md
…-epoch/catalog-references

# Conflicts:
#	rs/moq-cli/src/play/media.rs
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decisions on the open decisions (relayed from an interactive session, 2026-10-19):

  1. Scope split: accepted.
  2. Keep the sibling fields (epoch beside broadcast, and replayEpoch on the archive). A nested "path" | { path, epoch? } reference was considered and rejected, because published readers would refuse the whole catalog, while a sibling field is ignored by older readers.
  3. MSF export: accepted. MSF can't carry the epoch, so the exported reference requests without one.
  4. A malformed epoch refuses the catalog: accepted.
  5. Unpinned requests ask without the epoch and verify the returned one: accepted.

The PR predates 2026-10-19 on main (distinct importer epochs, #5925's announced_broadcast), so it needs main merged in before it's marked ready.

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 20, 2026 20:07
# Conflicts:
#	doc/setup/upgrade.md
#	drafts/draft-lcurley-moq-hang.md
#	js/hang/src/catalog/root.test.ts
#	js/hang/src/catalog/video.ts
#	quest/m0/broadcast-epoch/catalog-references.md
#	quest/m1/README.md
#	quest/m1/ietf-epochs.md
The reference-announcements quest now builds on moq-net's
announced_broadcast, following unpinned so epochless routes stay best
effort. Move the migration note into the moq-mux and hang MIGRATE.md
guides, and note that MoQ Transport with the epoch extension carries an
Epoch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 21, 2026 03:36
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-21T16:38:21.310168Z 092403b New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 15 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: cb1840a4-66fe-4dfd-881e-cab8e12b6080
📥 Commits

Reviewing files that changed from the base of the PR and between fb28fa5 and 092403b.

📒 Files selected for processing: 47
  • doc/concept/hang.md
  • drafts/draft-lcurley-moq-hang.md
  • js/hang/src/catalog/archive.ts
  • js/hang/src/catalog/audio.ts
  • js/hang/src/catalog/binary.ts
  • js/hang/src/catalog/json.ts
  • js/hang/src/catalog/path.ts
  • js/hang/src/catalog/root.test.ts
  • js/hang/src/catalog/text.ts
  • js/hang/src/catalog/video.ts
  • js/watch/src/audio/config.ts
  • js/watch/src/audio/decoder.ts
  • js/watch/src/broadcast.test.ts
  • js/watch/src/broadcast.ts
  • js/watch/src/text/renderer.ts
  • js/watch/src/video/config.ts
  • js/watch/src/video/decoder.ts
  • quest/m0/broadcast-epoch/README.md
  • quest/m0/broadcast-epoch/catalog-references.md
  • quest/m0/broadcast-epoch/reference-announcements.md
  • quest/m1/README.md
  • quest/m1/announce-takeover.md
  • quest/m1/identity-guide.md
  • quest/m1/reference-epoch-bindings.md
  • quest/m1/sink-restart-follow.md
  • rs/hang/MIGRATE.md
  • rs/hang/src/catalog/archive.rs
  • rs/hang/src/catalog/audio/mod.rs
  • rs/hang/src/catalog/binary.rs
  • rs/hang/src/catalog/json.rs
  • rs/hang/src/catalog/reference.rs
  • rs/hang/src/catalog/root.rs
  • rs/hang/src/catalog/text/mod.rs
  • rs/hang/src/catalog/video/mod.rs
  • rs/moq-c/src/consume.rs
  • rs/moq-cli/src/play/media.rs
  • rs/moq-ffi/src/consumer.rs
  • rs/moq-gst/src/source/imp.rs
  • rs/moq-hls/src/export/rendition.rs
  • rs/moq-mux/MIGRATE.md
  • rs/moq-mux/src/binary.rs
  • rs/moq-mux/src/catalog/producer.rs
  • rs/moq-mux/src/container/source.rs
  • rs/moq-mux/src/error.rs
  • rs/moq-mux/src/json.rs
  • rs/moq-mux/src/source.rs
  • rs/moq-rtc/src/egress.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c011d11e4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/watch/src/broadcast.ts
Comment on lines +429 to +432
const active = this.#relativeActive(effect, rel);
if (!active || epoch === undefined || active.epoch === undefined || active.epoch === epoch) return active;
console.warn("ignoring rendition: its broadcast serves another epoch", active.path, active.epoch, epoch);
return undefined;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Remove epoch-mismatched renditions before selection

When a preferred rendition names an epoch different from the resolved broadcast, this returns undefined, but #runFiltered still retains that rendition because it filters only by path. The video/audio source therefore continues selecting the unusable entry instead of falling back to another valid rendition in the catalog, leaving browser playback blank until the catalog or user selection changes. Include epoch compatibility in the effective-catalog filtering so selection and relativeBroadcast cannot disagree.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining here: this is tracked in reference announcements ("Drop it from the effective catalog too"), which the maintainer split out of this PR (decision 1). #runFiltered only consults announcements and deliberately issues no requests, so filtering by epoch means reading the covering route's epoch from the announcement watch, which is the same machinery that quest builds for following. Until then a stale epoch shows a blank rendition rather than the wrong instance, which is the guarantee this PR adds.

(Written by Claude Opus 5.5)

Comment on lines 164 to 167
let media: HangContainer = config.try_into()?;
let description = config.description.as_ref().filter(|b| !b.is_empty()).cloned();
let Some(request) = source.try_request(config.broadcast.as_ref()) else {
let Some(request) = source.try_request(config) else {
return Ok(None);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Isolate epoch mismatches from the whole container export

When one catalog entry resolves to a broadcast with a different epoch, the binding created here later returns EpochMismatch from ExportSource::poll_event; MKV and FLV propagate that through track.source.poll_read(waiter)?, terminating the entire multiplexed export. Thus one stale reference also removes otherwise valid sibling renditions, rather than failing only the mismatched entry as intended. Resolve/filter this condition per rendition before adding it to the export, or retire only the affected track.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining for this PR. Every failure of a rendition's binding already ends a muxed export this way: an unroutable sibling, a missing track, or a pinned broadcast that ended all propagate through the same poll_read(waiter)? in the TS, fMP4, MKV and FLV exporters. EpochMismatch is one more instance of that policy, not a new class, and failing loudly matches the draft's "fail the rendition" for a container that cannot drop a declared track mid-stream. Isolating per-rendition failures across all of those error kinds is a separate change; I'll suggest it as a follow-up quest.

(Written by Claude Opus 5.5)

Comment thread rs/moq-rtc/src/egress.rs
Comment on lines 281 to 283
let track = source
.subscribe_track(config.broadcast.as_ref(), hang::catalog::Reference::track(config, name))
.subscribe_track(config.broadcast(), config.epoch(), config.track(name))
.await?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Repick after an epoch-mismatched WebRTC subscription

If the initially preferred audio or video rendition resolves with a different epoch, this subscription now fails with EpochMismatch, but the pump only logs the error and leaves repick false. With no existing serving track, it then waits indefinitely for another catalog change and never tries the next compatible rendition, so a catalog containing a stale preferred entry and a valid fallback produces an idle WebRTC media line. Mark the failed rendition unusable and repick immediately.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining for this PR, for the same reason as the container exporters: a subscription that fails for any reason (an unroutable sibling, a missing track) already leaves the line idle until the next catalog change, and EpochMismatch follows that path. Since Source pins a path's first answer, a repick would also need to mark the rendition unusable rather than retry it, which applies equally to the existing errors. I'll suggest repicking past a failed rendition as a follow-up quest covering every failure kind.

(Written by Claude Opus 5.5)

# Conflicts:
#	rs/hang/MIGRATE.md
#	rs/moq-hls/src/export/rendition.rs
#	rs/moq-mux/MIGRATE.md
@kixelated
kixelated added this pull request to the merge queue Oct 21, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 21, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary for 071a17096cdad84fa0fd478379b27a1a9b42af46.

What lands: an optional epoch beside every hang catalog broadcast reference (and replayEpoch on the archive), enforced by moq_mux::Source and @moq/watch: a reference whose broadcast proves another instance fails instead of falling back, and it is best effort over epochless routes. The Hang draft, doc/concept/hang.md, and the hang / moq-mux MIGRATE.md entries cover it.

Decisions: the five open decisions were settled by the maintainer (comment). After #5925, the reference announcements follow-up builds on announced_broadcast, unpinned so epochless routes stay best effort.

Since the last push: a merge of main after the queue removed f0b2a0a7 for a conflict. The only conflict was rs/moq-mux/MIGRATE.md (kept both entries). Every catalog broadcast reference in Rust and JS still carries epoch.

Review: Codex left three findings, declined with replies and left to follow-up quests by the maintainer. The changes since then are main merges with mechanical conflict fixes.

Checks on this head: Rust lint and tests (3605 passed), JS check and tests, Python tests, just drafts check, and just test interop --all all pass. quest check fails only on stream-routes.md, which is broken on main until #6179 lands.

(Written by Claude Opus 5.5)

@kixelated
kixelated enabled auto-merge October 21, 2026 14:39
@kixelated
kixelated added this pull request to the merge queue Oct 21, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 071a17096c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/watch/src/broadcast.ts
Comment on lines +429 to +430
const active = this.#relativeActive(effect, rel);
if (!active || epoch === undefined || active.epoch === undefined || active.epoch === epoch) return active;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Pin the first instance before checking each epoch

When a referenced path initially resolves epoch E1 and later follows a restart to E2, this comparison accepts any rendition whose catalog entry names E2, including one added by a later catalog update. That lets one catalog instance move a previously resolved path onto a replacement publisher, contrary to draft-lcurley-moq-hang.md:687-694; only epochless references may follow restarts. Retain the first resolved instance for epoch-bound references so entries naming E2 continue to fail until a new catalog instance is created.

AGENTS.md reference: AGENTS.md:L67-L68

Useful? React with 👍 / 👎.

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 21, 2026
# Conflicts:
#	rs/moq-mux/src/source.rs
@kixelated
kixelated added this pull request to the merge queue Oct 21, 2026
Merged via the queue into main with commit 8b0d995 Oct 21, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m0/broadcast-epoch/catalog-references branch October 21, 2026 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant