From 739215c88cb855feaddc765ee44524ac16441f27 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 15 Oct 2026 22:58:49 -0700 Subject: [PATCH 1/4] quest: claim catalog-references Co-Authored-By: Claude Opus 5.5 From a543401e3deb62da641caeb2d5a13753bb44e0fc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 15 Oct 2026 23:19:25 -0700 Subject: [PATCH 2/4] feat(hang): catalog references may name their broadcast instance Add an optional `epoch` beside every rendition and data track `broadcast` reference, and `replayEpoch` beside the archive's `replay`, in Rust and JS. moq_mux::Source and @moq/watch fail a reference whose broadcast proves another epoch rather than reading it, and treat it as best effort over an epochless route. Split the announcement-following readers and binding exposure into follow-up quests. Co-Authored-By: Claude Opus 5.5 --- doc/concept/hang.md | 2 +- doc/setup/upgrade.md | 9 + drafts/draft-lcurley-moq-hang.md | 28 +- js/hang/src/catalog/archive.ts | 6 +- js/hang/src/catalog/audio.ts | 7 +- js/hang/src/catalog/binary.ts | 7 +- js/hang/src/catalog/json.ts | 7 +- js/hang/src/catalog/path.ts | 21 +- js/hang/src/catalog/root.test.ts | 43 +++ js/hang/src/catalog/text.ts | 7 +- js/hang/src/catalog/video.ts | 7 +- js/watch/src/audio/config.ts | 2 + js/watch/src/audio/decoder.ts | 2 +- js/watch/src/broadcast.test.ts | 85 ++++++ js/watch/src/broadcast.ts | 17 +- js/watch/src/text/renderer.ts | 2 +- js/watch/src/video/config.ts | 2 + js/watch/src/video/decoder.ts | 2 +- quest/m0/broadcast-epoch/README.md | 9 +- .../m0/broadcast-epoch/catalog-references.md | 91 ------ .../reference-announcements.md | 40 +++ quest/m1/README.md | 1 + quest/m1/announce-takeover.md | 2 +- quest/m1/identity-guide.md | 2 +- quest/m1/ietf-epochs.md | 2 +- quest/m1/reference-epoch-bindings.md | 30 ++ rs/hang/src/catalog/archive.rs | 8 + rs/hang/src/catalog/audio/mod.rs | 11 + rs/hang/src/catalog/binary.rs | 11 + rs/hang/src/catalog/json.rs | 11 + rs/hang/src/catalog/reference.rs | 10 +- rs/hang/src/catalog/root.rs | 45 +++ rs/hang/src/catalog/text/mod.rs | 11 + rs/hang/src/catalog/video/mod.rs | 11 + rs/moq-c/src/consume.rs | 4 +- rs/moq-cli/src/play/media.rs | 4 +- rs/moq-ffi/src/consumer.rs | 4 +- rs/moq-gst/src/source/imp.rs | 4 +- rs/moq-hls/src/export/rendition.rs | 28 +- rs/moq-mux/src/binary.rs | 6 +- rs/moq-mux/src/catalog/producer.rs | 4 + rs/moq-mux/src/container/source.rs | 4 +- rs/moq-mux/src/error.rs | 14 + rs/moq-mux/src/json.rs | 6 +- rs/moq-mux/src/source.rs | 261 ++++++++++++++---- rs/moq-rtc/src/egress.rs | 6 +- 46 files changed, 711 insertions(+), 185 deletions(-) delete mode 100644 quest/m0/broadcast-epoch/catalog-references.md create mode 100644 quest/m0/broadcast-epoch/reference-announcements.md create mode 100644 quest/m1/reference-epoch-bindings.md diff --git a/doc/concept/hang.md b/doc/concept/hang.md index e924764cf93..cd055cd1003 100644 --- a/doc/concept/hang.md +++ b/doc/concept/hang.md @@ -63,7 +63,7 @@ decoder once per track. A few things the catalog can express beyond decoder config: - **Labels.** Any rendition may carry a human-readable `label` for a track picker. The map key stays the rendition's ID, so labels need not be unique and renaming one doesn't rename the track. -- **Renditions in another broadcast.** A rendition may point at a relative broadcast path, so a transcoder can publish a ladder that adds low rungs and references the source's original rendition without re-publishing its bytes. The path resolves against where the consumer found the catalog, so a reference that escapes above the root names nothing and the catalog is rejected. +- **Renditions in another broadcast.** A rendition may point at a relative broadcast path, so a transcoder can publish a ladder that adds low rungs and references the source's original rendition without re-publishing its bytes. The path resolves against where the consumer found the catalog, so a reference that escapes above the root names nothing and the catalog is rejected. A consumer reads each referenced path from the first broadcast it resolves there. An optional `epoch` beside the path names the [publisher instance](/concept/moq-lite#publisher-epochs) it expects: a route proving another instance fails the rendition instead of falling back to it, while a route without an epoch (an older version) cannot prove one, so there it is best effort. - **Track names.** The map key is a rendition's ID and, by default, its track name. A rendition sets `track` only when the two must differ, such as two renditions of same-named tracks in different broadcasts, or a rendition whose media changed and so needs a new name; readers subscribe to `track` when present. An ID can't name both a video and an audio rendition. - **Jitter.** A rendition can say how far its frames fell behind the media clock before the publisher flushed them, in whole milliseconds rounded up. Encoders report the spread of lateness above each rendition's own recent minimum, so a constant encoder delay is not jitter; container imports estimate batch spans without counting ingest delay. It describes the publisher, never the network, only grows over the life of a stream, and a player sizes its buffer to at least this much. A `0` is read as absent. - **Delay.** A rendition can also say how far its frames reach the transport behind the broadcast's earliest rendition, measured the same way from each rendition's minimum lateness, so a video encoder running 200 ms behind audio advertises `delay: 200` on video. It follows the same rules as jitter. A player holds the largest `delay + jitter` among the renditions it subscribes to, and never subtracts one rendition's `delay` from another's. diff --git a/doc/setup/upgrade.md b/doc/setup/upgrade.md index 1afdc054f35..cb91edfe51d 100644 --- a/doc/setup/upgrade.md +++ b/doc/setup/upgrade.md @@ -112,6 +112,15 @@ These land with the next breaking release, not the 2026-09-23 train. `Source` to follow a new instance. moq-hls's `Broadcaster::new` no longer awaits or fails, and a rendition whose sibling broadcast ends now ends its playlist instead of rebinding. +- **Catalog references may name their instance.** Every hang rendition and data + track gains an optional `epoch` beside `broadcast`, and the archive gains + `replayEpoch` beside `replay`. In Rust, moq-mux's `Source::bind(rel)`, + `resolve(rel)`, and `subscribe_track(rel, name)` take the reference's epoch + after `rel`: pass `config.epoch.as_ref()`, or `None` to accept the first + instance resolved as before. A reference whose broadcast proves another epoch + fails with `Error::EpochMismatch`; over a route without an epoch it is best + effort. `@moq/watch`'s `relativeBroadcast(effect, rel, epoch?)` takes the same + optional argument. Omitting the field keeps older catalogs and readers working. - **fMP4 export of Annex-B H.264 and H.265 inits from the catalog.** When the catalog codec string and dimensions are enough, `moq export fmp4` writes an `avc3` or `hev1` init segment before the first keyframe and leaves SPS, PPS, diff --git a/drafts/draft-lcurley-moq-hang.md b/drafts/draft-lcurley-moq-hang.md index cc508b68956..b8d5ee5f8ef 100644 --- a/drafts/draft-lcurley-moq-hang.md +++ b/drafts/draft-lcurley-moq-hang.md @@ -476,7 +476,7 @@ The `deflate` value is the group-scoped DEFLATE of {{compression}}. A `snapshot` group covers a single value (plus any deltas), so its window spans that group alone; a `stream` group's frames compress against the earlier ones in the log. ### broadcast and track {#data-shared} -The `broadcast` and `track` fields carry the same meaning here as they do for a media rendition ({{field-broadcast}}, {{field-track}}). +The `broadcast`, `epoch`, and `track` fields carry the same meaning here as they do for a media rendition ({{field-broadcast}}, {{field-epoch}}, {{field-track}}). ### bitrate, jitter, and delay {#data-estimates} The optional `bitrate` field is the track's maximum bitrate in bits per second. @@ -500,6 +500,7 @@ Audio, video, and text renditions share the following fields, extending the WebC ~~~ type CommonExtensions = { "broadcast": string | undefined, + "epoch": string | undefined, "track": string | undefined, "label": string | undefined, "container": Container, @@ -531,6 +532,27 @@ An absent MSF `namespace` inherits the catalog's broadcast. Since each endpoint sees paths under its own connection root, a `namespace` equal to the catalog broadcast's path, or matching it on whole trailing segments, also names the catalog's broadcast; a gateway MUST NOT emit a different broadcast's `namespace` that matches so. A gateway MUST preserve the namespace and track-name pair or refuse the rendition; it MUST NOT substitute the catalog's broadcast for an unrepresentable reference. +### epoch {#field-epoch} +The optional `epoch` field names the publisher instance of the broadcast a rendition references ({{field-broadcast}}), as the canonical text of its Epoch {{moql}}: a lowercase hyphenated UUIDv7. +Without `broadcast`, it names the instance of the catalog's own broadcast. +A consumer MUST reject a catalog containing any other `epoch` value. + +A consumer reads each path a catalog instance references from the first broadcast it resolves there, so the catalog and its renditions never resolve one path independently across a replacement. +An absent `epoch` accepts that first broadcast, whichever instance it is. +A consumer that follows a referenced broadcast's announcements MAY move a rendition without `epoch` onto a restarted instance. + +When `epoch` is present and the route that resolved the referenced broadcast carries an Epoch, the consumer MUST read the rendition only from that instance. +It MUST fail the rendition rather than fall back to another instance, including one that replaced it before or after resolution, and MUST NOT follow a restart onto another instance for it. +A later catalog update naming another Epoch for an already-resolved path MUST NOT replace the resolved broadcast; the renditions naming the other Epoch fail instead. +A new catalog instance resolves its references afresh. + +A route without an Epoch, such as one over an older moq-lite version or MoQ Transport, cannot prove which instance it serves, so there `epoch` is best effort: the consumer reads the broadcast it resolved. +A consumer MUST NOT treat a catalog's `epoch` as evidence that a subscription may resume across routes. +A consumer that predates the field ignores it and does not gain its guarantee. + +A publisher SHOULD set `epoch` only when it knows the instance it references, and MUST omit it for a broadcast produced on demand, which has no instance until a worker accepts the request. +MSF {{moqmsf}} has no field for an instance, so a gateway converting to MSF drops `epoch` and SHOULD report that it did; its readers accept the first instance they resolve. + ### track {#field-track} A rendition's key in its map is its ID, and by default also the name of its track. The `track` field overrides the name, so that the ID can stay unique when the name cannot: two renditions referencing same-named tracks in different broadcasts ({{field-broadcast}}) need different IDs. @@ -708,6 +730,7 @@ type ArchiveSchema = { "timescale": number | undefined, "durationMax": number | undefined, "replay": string | undefined, + "replayEpoch": string | undefined, "store": string | undefined, "version": number | undefined, } @@ -736,6 +759,8 @@ The `replay` field, if present, is a relative MoQ broadcast path ({{field-broadc Absent, the timelines live on the catalog's own broadcast. A wildcard replay path names no generation. +The `replayEpoch` field, if present, names the publisher instance of `replay`, with the meaning and rules of a rendition's `epoch` ({{field-epoch}}). + The `store` field, if present, is the object-store URL the recording objects ({{recording}}) live under. Authorization for `replay` and `store` is external. @@ -1127,6 +1152,7 @@ This document has no IANA actions. ## moq-hang-04 {:numbered="false"} +- Added an optional `epoch` beside every rendition and data track `broadcast` reference, and `replayEpoch` beside the archive's `replay` ({{field-epoch}}). A consumer reads each referenced path from the first broadcast it resolves for a catalog instance; a present `epoch` fails a rendition proven to resolve another instance, and is best effort over a route without an Epoch. - Derived HLS and DASH exports hand disabled references to enabled renditions at the first record numbered past the advertised ones, keeping the record `sequence` as the HLS media sequence and starting a DASH period. - Fixed a track's identity for its name ({{identity}}): codec, description, container, display aspect, sample rate, and channel count never change, while every other field, including size, bitrate, and framerate, updates in place. A change of identity takes a new track name, which a rendition may name with `track` while keeping its ID, or a new Epoch. diff --git a/js/hang/src/catalog/archive.ts b/js/hang/src/catalog/archive.ts index a99b4ee4f08..df4878c7766 100644 --- a/js/hang/src/catalog/archive.ts +++ b/js/hang/src/catalog/archive.ts @@ -1,6 +1,6 @@ import * as z from "@zod/mini"; import { nonzeroU53Schema, u53, u53Schema } from "./integers"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; /** The moq epoch (2020-01-01T00:00:00Z) in Unix-epoch milliseconds. */ export const MOQ_EPOCH_UNIX_MILLIS = 1_577_836_800_000; @@ -36,6 +36,10 @@ export const ArchiveSchema = z.object({ // generation: compare this path and `store` to tell recordings apart. replay: z.optional(RelativeBroadcastSchema), + // The publisher instance `replay` names, when known. Absent accepts whichever instance + // resolves first, as for a rendition's `epoch`. + replayEpoch: z.optional(EpochSchema), + // The object-store URL the recording objects live under, if the publisher exposes one. store: z.optional(z.url()), diff --git a/js/hang/src/catalog/audio.ts b/js/hang/src/catalog/audio.ts index 0784ff9cef5..9c8427b6ec3 100644 --- a/js/hang/src/catalog/audio.ts +++ b/js/hang/src/catalog/audio.ts @@ -3,7 +3,7 @@ import { compareName, compareNumber } from "./compare"; import { ContainerSchema } from "./container"; import { hexSchema } from "./hex"; import { u53Schema } from "./integers"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; // Backwards compatibility: old track schema const TrackSchema = z.object({ @@ -19,6 +19,11 @@ export const AudioConfigSchema = z.object({ // relative to the broadcast that served this catalog (e.g. "./source"). // If unset, the track lives in the same broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/hang/src/catalog/binary.ts b/js/hang/src/catalog/binary.ts index e49191df24b..be5e1a82049 100644 --- a/js/hang/src/catalog/binary.ts +++ b/js/hang/src/catalog/binary.ts @@ -2,7 +2,7 @@ import * as z from "zod/mini"; import { CompressionSchema } from "./compression"; import { u53Schema } from "./integers"; import { ModeSchema } from "./mode"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; /** * Schema for a single binary track: application data published as opaque payloads. @@ -19,6 +19,11 @@ export const BinaryConfigSchema = z.looseObject({ // relative to the broadcast that served this catalog (e.g. "./source"). // If unset, the track lives in the same broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/hang/src/catalog/json.ts b/js/hang/src/catalog/json.ts index 3e5222980c1..0d5fb05d721 100644 --- a/js/hang/src/catalog/json.ts +++ b/js/hang/src/catalog/json.ts @@ -2,7 +2,7 @@ import * as z from "zod/mini"; import { CompressionSchema } from "./compression"; import { u53Schema } from "./integers"; import { ModeSchema } from "./mode"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; /** * Schema for a single JSON track: application data published as a live JSON document or log. @@ -18,6 +18,11 @@ export const JsonConfigSchema = z.looseObject({ // relative to the broadcast that served this catalog (e.g. "./source"). // If unset, the track lives in the same broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/hang/src/catalog/path.ts b/js/hang/src/catalog/path.ts index 05c6fc7d71e..62eb3b6a385 100644 --- a/js/hang/src/catalog/path.ts +++ b/js/hang/src/catalog/path.ts @@ -1,4 +1,4 @@ -import { Path } from "@moq/net"; +import { Epoch, Path } from "@moq/net"; import * as z from "@zod/mini"; /** @@ -18,3 +18,22 @@ export const RelativeBroadcastSchema: z.ZodMiniType = * out here so the catalog schemas' inferred types stay nameable from this package. */ export type RelativeBroadcast = string & { __brand: "Relative" }; + +function isEpoch(text: string): boolean { + try { + Epoch.parse(text); + return true; + } catch { + return false; + } +} + +/** + * Zod schema for the publisher instance a catalog reference names: an entry's `epoch` beside its + * `broadcast`, or the archive's `replayEpoch`. A lowercase hyphenated UUIDv7, the same text MoQ + * routes carry; anything else refuses the catalog. + */ +export const EpochSchema: z.ZodMiniType = z.pipe( + z.string().check(z.refine(isEpoch, "expected a lowercase hyphenated UUIDv7")), + z.transform((text) => text as Epoch.Valid), +); diff --git a/js/hang/src/catalog/root.test.ts b/js/hang/src/catalog/root.test.ts index f06098efe0e..0cc92855ca0 100644 --- a/js/hang/src/catalog/root.test.ts +++ b/js/hang/src/catalog/root.test.ts @@ -45,6 +45,49 @@ test("rendition broadcast reference is parsed and normalized", () => { expect(parsed.video.renditions.video?.broadcast).toBe("source" as RelativeBroadcast); }); +// Mirrors Rust's `reference_epochs_roundtrip_in_every_section`. +test("every reference-bearing section carries an optional epoch", () => { + const EPOCH = "01890a5d-ac96-774b-bcce-b302099a8057"; + const catalog = (epoch: string) => ({ + video: { + renditions: { + v: { broadcast: "./a", epoch, codec: "avc1.64001f", container: { kind: "legacy" } }, + }, + }, + audio: { + renditions: { + a: { + broadcast: "./a", + epoch, + codec: "opus", + sampleRate: 48000, + numberOfChannels: 2, + container: { kind: "legacy" }, + }, + }, + }, + text: { renditions: { t: { broadcast: "./a", epoch, format: "vtt" } } }, + json: { tracks: { j: { broadcast: "./a", epoch, mode: "snapshot" } } }, + binary: { tracks: { b: { broadcast: "./a", epoch, mode: "snapshot" } } }, + archive: { timelines: {}, replay: "./a", replayEpoch: epoch }, + }); + + const parsed = RootSchema.parse(catalog(EPOCH)); + if (!parsed.video || !("renditions" in parsed.video)) throw new Error("missing video section"); + if (!parsed.audio || !("renditions" in parsed.audio)) throw new Error("missing audio section"); + expect(parsed.video.renditions.v?.epoch).toBe(EPOCH); + expect(parsed.audio.renditions.a?.epoch).toBe(EPOCH); + expect(parsed.text?.renditions.t?.epoch).toBe(EPOCH); + expect(parsed.json?.tracks.j?.epoch).toBe(EPOCH); + expect(parsed.binary?.tracks.b?.epoch).toBe(EPOCH); + expect(parsed.archive?.replayEpoch).toBe(EPOCH); + + // A malformed or noncanonical epoch refuses the catalog, as in Rust. + for (const bad of ["not-a-uuid", EPOCH.toUpperCase(), "01890a5d-ac96-474b-bcce-b302099a8057"]) { + expect(RootSchema.safeParse(catalog(bad)).success).toBe(false); + } +}); + test("rendition parent broadcast reference stays distinct from empty", () => { const catalog = { video: { diff --git a/js/hang/src/catalog/text.ts b/js/hang/src/catalog/text.ts index 2dc6bdf3f6a..77f0401f01f 100644 --- a/js/hang/src/catalog/text.ts +++ b/js/hang/src/catalog/text.ts @@ -1,7 +1,7 @@ import * as z from "zod/mini"; import { ContainerSchema } from "./container"; import { u53Schema } from "./integers"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; /** * The cue serialization format of a text track, mirroring the Rust `TextFormat`. @@ -34,6 +34,11 @@ export const TextConfigSchema = z.object({ // broadcast that served this catalog (e.g. "../source"). If unset, the track lives in the same // broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/hang/src/catalog/video.ts b/js/hang/src/catalog/video.ts index f21150dd351..378b27f2c3b 100644 --- a/js/hang/src/catalog/video.ts +++ b/js/hang/src/catalog/video.ts @@ -3,7 +3,7 @@ import { compareName, compareNumber } from "./compare"; import { ContainerSchema } from "./container"; import { hexSchema } from "./hex"; import { u53Schema } from "./integers"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; // Backwards compatibility: old track schema const TrackSchema = z.object({ @@ -16,6 +16,11 @@ export const VideoConfigSchema = z.object({ // relative to the broadcast that served this catalog (e.g. "./source"). // If unset, the track lives in the same broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/watch/src/audio/config.ts b/js/watch/src/audio/config.ts index abaad956a38..b37cab81748 100644 --- a/js/watch/src/audio/config.ts +++ b/js/watch/src/audio/config.ts @@ -19,6 +19,7 @@ export type DecoderConfig = Pick< /** The routing and decoder state whose changes require a replacement subscription. */ export type PlaybackIdentity = { broadcast: Catalog.AudioConfig["broadcast"]; + epoch: Catalog.AudioConfig["epoch"]; track: Catalog.AudioConfig["track"]; decoder: DecoderConfig; }; @@ -39,6 +40,7 @@ export function decoderConfig(config: Catalog.AudioConfig): DecoderConfig { export function playbackIdentity(config: Catalog.AudioConfig): PlaybackIdentity { return { broadcast: config.broadcast, + epoch: config.epoch, track: config.track, decoder: decoderConfig(config), }; diff --git a/js/watch/src/audio/decoder.ts b/js/watch/src/audio/decoder.ts index e69a2eb3765..fcb1736c3c0 100644 --- a/js/watch/src/audio/decoder.ts +++ b/js/watch/src/audio/decoder.ts @@ -331,7 +331,7 @@ export class Decoder { // tell whether a failure's source is still selected. const active = broadcast && track && identity && (running || this.#failed) - ? broadcast.relativeBroadcast(effect, identity.broadcast) + ? broadcast.relativeBroadcast(effect, identity.broadcast, identity.epoch) : undefined; const key = identity && JSON.stringify(identity); diff --git a/js/watch/src/broadcast.test.ts b/js/watch/src/broadcast.test.ts index 723c1c4790f..ecce914b634 100644 --- a/js/watch/src/broadcast.test.ts +++ b/js/watch/src/broadcast.test.ts @@ -171,6 +171,91 @@ describe("relativeBroadcast", () => { } }); + it("reads a reference only from the epoch it names, best effort without one", async () => { + const owner = new Origin.Producer(); + const [catalogEpoch, siblingEpoch, other] = [Moq.Epoch.mint(), Moq.Epoch.mint(), Moq.Epoch.mint()]; + const catalog = owner.createBroadcast(Path.from("a/b")); + catalog.announce({ epoch: catalogEpoch }); + const sibling = owner.createBroadcast(Path.from("a/source")); + sibling.announce({ epoch: siblingEpoch }); + const plain = publish(owner, Path.from("a/plain")); + const source = new Broadcast({ + origin: owner, + name: Path.from("a/b"), + enabled: true, + announced: false, + catalogFormat: "manual", + }); + const effect = new Effect(); + try { + await settle(); + const rel = Path.normalizeRelative("./source"); + expect(source.relativeBroadcast(effect, rel)?.epoch).toBe(siblingEpoch); + expect(source.relativeBroadcast(effect, rel, siblingEpoch)?.epoch).toBe(siblingEpoch); + const own = source.out.active.peek(); + expect(own?.epoch).toBe(catalogEpoch); + expect(source.relativeBroadcast(effect, undefined, catalogEpoch)).toBe(own); + withoutWarnings(() => { + expect(source.relativeBroadcast(effect, rel, other)).toBeUndefined(); + expect(source.relativeBroadcast(effect, undefined, other)).toBeUndefined(); + expect(source.relativeBroadcast(effect, Path.normalizeRelative("./b"), other)).toBeUndefined(); + }); + // An epochless route cannot prove its instance, so the epoch cannot refuse it. + expect(source.relativeBroadcast(effect, Path.normalizeRelative("./plain"), other)).toBeDefined(); + } finally { + effect.close(); + source.close(); + plain.close(); + sibling.close(); + catalog.close(); + owner.close(); + } + }); + + it("follows a replaced sibling only for a reference that names no epoch", async () => { + const owner = new Origin.Producer(); + const main = publish(owner, Path.from("a/b")); + const [oldEpoch, newEpoch] = [Moq.Epoch.mint(), Moq.Epoch.mint()]; + const old = owner.createBroadcast(Path.from("a/source")); + old.announce({ epoch: oldEpoch }); + const source = new Broadcast({ + origin: owner, + name: Path.from("a/b"), + enabled: true, + catalogFormat: "manual", + }); + const effect = new Effect(); + const rel = Path.normalizeRelative("./source"); + let following: Moq.Broadcast.Consumer | undefined; + let pinned: Moq.Broadcast.Consumer | undefined; + effect.run((nested) => { + following = source.relativeBroadcast(nested, rel); + }); + effect.run((nested) => { + pinned = withoutWarnings(() => source.relativeBroadcast(nested, rel, oldEpoch)); + }); + try { + await settle(); + expect(following?.epoch).toBe(oldEpoch); + expect(pinned?.epoch).toBe(oldEpoch); + + // The announcement is the online signal: the plain reference follows the restart, + // while one naming the old instance never reads the new one. + const replacement = owner.createBroadcast(Path.from("a/source")); + replacement.announce({ epoch: newEpoch }); + await settle(); + expect(following?.epoch).toBe(newEpoch); + expect(pinned).toBeUndefined(); + replacement.close(); + } finally { + effect.close(); + source.close(); + old.close(); + main.close(); + owner.close(); + } + }); + it("uses the catalog's own broadcast when the reference is absent, empty, or self", async () => { const { source, owner } = broadcast("a/b"); const effect = new Effect(); diff --git a/js/watch/src/broadcast.ts b/js/watch/src/broadcast.ts index f3b2deff8b3..5a0f9ab8a26 100644 --- a/js/watch/src/broadcast.ts +++ b/js/watch/src/broadcast.ts @@ -407,8 +407,23 @@ export class Broadcast { * The consumer is scoped to the caller's `effect` (closed on its next run), so a * reference resolves lazily and reacts to `enabled` / connection / announcement * changes exactly like the catalog broadcast. + * + * `epoch` is the rendition's `epoch` field, the publisher instance the reference names. + * Returns `undefined` while the followed broadcast is proven to be another instance, rather + * than reading it. A route without an epoch cannot prove one, so there it is best effort. */ - relativeBroadcast(effect: Effect, rel: Path.Relative | undefined): Moq.Broadcast.Consumer | undefined { + relativeBroadcast( + effect: Effect, + rel: Path.Relative | undefined, + epoch?: Moq.Epoch.Valid, + ): Moq.Broadcast.Consumer | undefined { + const active = this.#relativeActive(effect, rel); + if (!active || epoch === undefined || active.epoch === undefined || active.epoch === epoch) return active; + console.warn("ignoring rendition: its broadcast serves another epoch", active.path, active.epoch, epoch); + return undefined; + } + + #relativeActive(effect: Effect, rel: Path.Relative | undefined): Moq.Broadcast.Consumer | undefined { const target = this.#relativeTarget(effect, rel); if (!target) return undefined; if (target.local) return effect.get(this.out.active); diff --git a/js/watch/src/text/renderer.ts b/js/watch/src/text/renderer.ts index e8c91fde2ab..4a27e4fd291 100644 --- a/js/watch/src/text/renderer.ts +++ b/js/watch/src/text/renderer.ts @@ -208,7 +208,7 @@ export class Renderer { const [_enabled, container, track, config, broadcast] = values; // Honor a per-rendition `broadcast` override: subscribe on the resolved source broadcast. - const active = broadcast.relativeBroadcast(effect, config.broadcast); + const active = broadcast.relativeBroadcast(effect, config.broadcast, config.epoch); if (!active) return; // A new track gets its own chance to complain about its clock. diff --git a/js/watch/src/video/config.ts b/js/watch/src/video/config.ts index a88782c5c83..1dedaa75ab8 100644 --- a/js/watch/src/video/config.ts +++ b/js/watch/src/video/config.ts @@ -14,6 +14,7 @@ export type DecoderConfig = Pick< /** The routing and decoder state whose changes require a new playback pipeline. */ export type PlaybackIdentity = { broadcast: Catalog.VideoConfig["broadcast"]; + epoch: Catalog.VideoConfig["epoch"]; track: Catalog.VideoConfig["track"]; decoder: DecoderConfig; }; @@ -22,6 +23,7 @@ export type PlaybackIdentity = { export function playbackIdentity(config: Catalog.VideoConfig): PlaybackIdentity { return { broadcast: config.broadcast, + epoch: config.epoch, track: config.track, decoder: { codec: config.codec, diff --git a/js/watch/src/video/decoder.ts b/js/watch/src/video/decoder.ts index cc99c163c9f..47247271e1b 100644 --- a/js/watch/src/video/decoder.ts +++ b/js/watch/src/video/decoder.ts @@ -183,7 +183,7 @@ export class Decoder { // tell whether a failure's source is still selected. const active: Moq.Broadcast.Consumer | undefined = broadcast && track && identity && (enabled || this.#failed) - ? broadcast.relativeBroadcast(effect, identity.broadcast) + ? broadcast.relativeBroadcast(effect, identity.broadcast, identity.epoch) : undefined; const key = identity && JSON.stringify(identity); diff --git a/quest/m0/broadcast-epoch/README.md b/quest/m0/broadcast-epoch/README.md index 00de076c73a..51b158d69c6 100644 --- a/quest/m0/broadcast-epoch/README.md +++ b/quest/m0/broadcast-epoch/README.md @@ -69,9 +69,10 @@ Decided: resolved sibling for the catalog instance's lifetime in players and exporters. Optional reference epochs constrain the first resolution on epoch-aware transports; on epochless transports they remain best effort. - This supersedes following the newest sibling by name. The - [catalog references quest](/quest/m0/broadcast-epoch/catalog-references.md) - owns the schemas, readers, and compatibility documentation. + This supersedes following the newest sibling by name. The schemas, shared + resolution, and compatibility documentation landed; readers that follow a + referenced broadcast's announcements (decided 2026-10-14) are owned by + [reference announcements](/quest/m0/broadcast-epoch/reference-announcements.md). - Every first-party publisher that can restart mints its own: the apps, moq-boy, the ingest gateways, moqsink, OBS, and the bindings. Players (`moq play`, `@moq/watch`, demo/web) follow the announce `Restart`. moq-stats mints one per @@ -120,7 +121,7 @@ End+Start. The child quests own their focused regressions. ## Required -- [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - players and exporters pin siblings consistently, with optional reference epochs and documented legacy best effort +- [Reference announcements](/quest/m0/broadcast-epoch/reference-announcements.md) - native readers follow a referenced broadcast's announcements, never past a named epoch ## Related diff --git a/quest/m0/broadcast-epoch/catalog-references.md b/quest/m0/broadcast-epoch/catalog-references.md deleted file mode 100644 index 8c6ed71eec9..00000000000 --- a/quest/m0/broadcast-epoch/catalog-references.md +++ /dev/null @@ -1,91 +0,0 @@ -# [L] Catalog references pin their broadcast instances - -## Goal - -Catalog, media, and data requests use the same pinned broadcast instance -in players and exporters. Each sibling stays on its first resolved -instance for the catalog instance's lifetime. Optional catalog epochs let -a reference name its intended sibling before that first resolution. - -## Plan - -Decided in the 2026-10-10 resume audit: - -- Apply the same pinning policy to players and exporters. This replaces - the earlier rule that a named sibling follows the newest epoch, including - `@moq/watch`'s test that follows a republished rendition independently of - its catalog. A sibling replacement does not silently retarget the old - catalog's reference. A new catalog instance or explicit export stitch - starts fresh pins; same-epoch recovery preserves identity. -- Add optional `epoch` beside every catalog broadcast reference: video, - audio, text, JSON, and binary configs, plus the archive replay reference. - Reuse the canonical UUIDv7 representation and validation from MoQ epochs. - There is no new top-level catalog epoch. Self-references use the already - pinned catalog broadcast; an explicit value must not contradict its - known identity. -- An explicit reference epoch constrains the initial resolution and later - requests wherever the transport can express and verify it. A known - mismatch fails instead of falling back to the current winner. Conflicting - identities for an already-pinned path cannot replace that pin through a - catalog update. -- Lazy work has no anticipated epoch: leave it out of unresolved contribution - references. Once a worker accepts a job it advertises a concrete path with - a unique epoch. A proven Takeover preserves the same pinned instance; a - replacement requires a new catalog instance. An epoch-bound request on an - epoch-capable link requires a matching advertised epoch, not merely an - epoch returned from an epochless claim. -- Omission remains compatible with existing catalogs: pin the first - resolved instance, with no promise about an unresolved sibling's past. -- On epochless transports, explicit catalog epochs are best effort - (maintainer choice). Keep the requested identity in the reference, use - whatever identity the route can verify, and pin the resolved handle. - Unknown identity alone is not a refusal. Document that this cannot prove - the requested epoch, and never promote an unverified catalog value into - transport evidence permitting seamless resume. Older readers can ignore - the additive field and do not gain its guarantee. - -Decided by the maintainer, 2026-10-14 (from #5500's Codex review): readers -follow a referenced broadcast's announcements, since announcement is the -online/offline signal and `@moq/watch` already behaves this way. The "no -retarget" rule above applies only to readers that follow no announcement. - -- `moqsrc`: Start binds the rendition, End idles its pad, and Restart - restarts its pump. -- Exporters: a late first announcement starts the rendition and End ends - it. Restart behavior belongs to - [restart followers](/quest/m1/restart-followers.md). -- Share this in `moq_mux` (a `Source::follow`-style API; the shape is open) - rather than per reader. `Source::bind` binds once today, as - `failed_binding_does_not_retry_a_later_publisher` asserts; that changes. - -Use the shared reference resolution paths rather than adding one rule per -media type. Mirror schemas and behavior in `rs/hang`, `js/hang`, native -readers/exporters and `@moq/watch`; carry the field through catalog producers, -conversions and bindings that expose these configs. A conversion that -cannot preserve the requested identity must report that limitation rather -than silently discard it. Coordinate with catalog rendition IDs, which -changes the neighboring track-name field but is not a prerequisite. - -Test all reference-bearing sections in both languages: omitted/matching/ -mismatched/malformed epochs, same and sibling paths, a replacement before -first resolution and after pinning, late rendition additions, updates that -conflict with a pin, new catalog instances, and an epochless transport with -an explicit reference. Catalog and media must never independently resolve -the same path across a replacement. Include archive replay references and -concurrent old/new consumers. Use existing unit, media and interop CI lanes; -run `just check`, `just test interop --all`, and `just drafts check`. - -Public API: additive optional catalog fields and changed player sibling -following behavior. Wire: additive Hang catalog fields, no transport -framing change. Update the Hang draft, existing catalog/identity docs, and -upgrade notes inline. The [identity guide](/quest/m1/identity-guide.md) -consolidates the behavior separately. Explain both the strong epoch-aware -guarantee and the selected legacy best-effort behavior. - -## Related - -- [Source pin](https://github.com/moq-dev/moq/pull/5252) - exporters share their resolved catalog handle -- [JS consume identity](https://github.com/moq-dev/moq/pull/5254) - JS resolves the actual serving identity -- [Retired requests](https://github.com/moq-dev/moq/pull/5294) - held handles cannot emit unpinned requests after retirement -- [Rust identity enforcement](/quest/m1/catalog-identity-rs.md) - the same for the Rust publishers -- [IETF epochs](/quest/m1/ietf-epochs.md) - verified catalog identity over negotiated IETF sessions diff --git a/quest/m0/broadcast-epoch/reference-announcements.md b/quest/m0/broadcast-epoch/reference-announcements.md new file mode 100644 index 00000000000..dd80b5c2893 --- /dev/null +++ b/quest/m0/broadcast-epoch/reference-announcements.md @@ -0,0 +1,40 @@ +# [L] Rust readers follow referenced broadcasts' announcements + +## Goal + +Native readers follow a catalog-referenced sibling's announcements the way +`@moq/watch` already does, since announcement is the online/offline signal. +A reference that names an `epoch` never follows onto another instance. + +## Plan + +Split from the catalog references work (#5500's Codex review, decided by the +maintainer 2026-10-14). That work landed the optional reference `epoch`, its +shared resolution in `moq_mux::Source` and `@moq/watch`, and the Hang draft +text; this quest owns the following behavior it left out. + +- Share it in `moq_mux` (a `Source::follow`-style API; the shape is open) + rather than per reader. `Source::bind` binds once today, as + `failed_binding_does_not_retry_a_later_publisher` asserts; that changes. + The "no retarget" rule applies only to readers that follow no announcement. +- `moqsrc`: Start binds the rendition, End idles its pad, and Restart + restarts its pump. +- Exporters: a late first announcement starts the rendition and End ends + it. Restart behavior belongs to + [restart followers](/quest/m1/restart-followers.md). +- A reference with an `epoch` follows only its own instance: a Restart onto + another epoch ends that rendition, matching `relativeBroadcast` in + `@moq/watch` (`follows a replaced sibling only for a reference that names + no epoch`). +- `@moq/watch` returns nothing for a rendition whose followed broadcast proves + another epoch, but still lists it in `out.catalog`, so selection can pick a + rendition that never plays. Drop it from the effective catalog too. + +Test late first announcements, End then Start, Restart with and without an +epoch, and concurrent old/new consumers, in the moq-mux unit tests and the +`moqsrc` and exporter suites. + +## Related + +- [Restart followers](/quest/m1/restart-followers.md) - consumers that follow an epochless Restart of their own broadcast +- [Export rendition rebind](/quest/m1/export-rendition-rebind.md) - exporters rebuild a rendition whose reference changes diff --git a/quest/m1/README.md b/quest/m1/README.md index 1d4d1ba60b2..308356ec13a 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -106,6 +106,7 @@ above them. - [Announcement takeover](/quest/m1/announce-takeover.md) - a new independent route explicitly continues instances served through its predecessor, with shared Rust/JS semantics and lite-07 wire - [IETF takeover](/quest/m1/ietf-takeover.md) - carry the shared takeover contract through the negotiated IETF epoch extension - [Identity and recovery guide](/quest/m1/identity-guide.md) - explain pinned catalogs, lazy jobs, takeover, restart and version compatibility +- [Reference epoch bindings](/quest/m1/reference-epoch-bindings.md) - the UniFFI bindings and C ABI expose and honor a catalog reference's optional `epoch` - [Dropped sources](/quest/m1/dropped-sources.md) - track consumers see the producer's real error on every end path, never `Dropped`; a dead route ends Unroutable, and the TS follower lingers on announcements instead of matching errors - [No made-up Dropped](/quest/m1/dropped-internal.md) - internal code names the real cause instead of `Dropped`, which then only means an app dropped a producer unfinished - [Client settings parity](/quest/m1/obs-client-config.md) - moq-ffi offers moq-c's client knobs, and the OBS Advanced settings get back the ones the C++ migration dropped diff --git a/quest/m1/announce-takeover.md b/quest/m1/announce-takeover.md index 490c6e911ac..997fd056441 100644 --- a/quest/m1/announce-takeover.md +++ b/quest/m1/announce-takeover.md @@ -94,6 +94,6 @@ ANNOUNCE_TAKEOVER; published versions retain their framing. ## Related - [IETF takeover](/quest/m1/ietf-takeover.md) - negotiated IETF counterpart -- [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - a proven continuation preserves the same pin +- [Reference announcements](/quest/m0/broadcast-epoch/reference-announcements.md) - a proven continuation keeps a followed reference on its instance - [Retired requests](https://github.com/moq-dev/moq/pull/5294) - replacements never substitute content into held handles - [Identity and recovery guide](/quest/m1/identity-guide.md) - explains takeover alongside restart and legacy behavior diff --git a/quest/m1/identity-guide.md b/quest/m1/identity-guide.md index e0eaa1fcf4d..ad9d4613dc8 100644 --- a/quest/m1/identity-guide.md +++ b/quest/m1/identity-guide.md @@ -39,5 +39,5 @@ Public API and wire: none; this is a user guide for existing feature owners. ## Required -- [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - establishes the shared pinning behavior +- [Reference announcements](/quest/m0/broadcast-epoch/reference-announcements.md) - settles how readers pin and follow catalog references - [Announcement takeover](/quest/m1/announce-takeover.md) - establishes the continuity operation the guide demonstrates diff --git a/quest/m1/ietf-epochs.md b/quest/m1/ietf-epochs.md index a59017af587..497a5fbe836 100644 --- a/quest/m1/ietf-epochs.md +++ b/quest/m1/ietf-epochs.md @@ -80,5 +80,5 @@ extension; no changes to unnegotiated published versions or broadcast paths. - [Announcement takeover](/quest/m1/announce-takeover.md) - shared continuity model and lite-07 operation - [IETF takeover](/quest/m1/ietf-takeover.md) - independently advertised continuation of a claim-served instance - [JS track handover](/quest/m1/js-group-handover.md) - stable JS readers across same-epoch route changes -- [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - optional requested epochs become verifiable on this negotiated transport; epochless fallback remains best effort +- [Hang reference epochs](/drafts/draft-lcurley-moq-hang.md) - a catalog reference's optional `epoch` becomes verifiable on this negotiated transport; epochless routes remain best effort - [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - native wait-budget support is separate from epoch negotiation and adds no age extension diff --git a/quest/m1/reference-epoch-bindings.md b/quest/m1/reference-epoch-bindings.md new file mode 100644 index 00000000000..3e426407772 --- /dev/null +++ b/quest/m1/reference-epoch-bindings.md @@ -0,0 +1,30 @@ +# [M] Bindings carry catalog reference epochs + +## Goal + +The UniFFI bindings and the C ABI expose a catalog reference's optional +`epoch` and honor it when resolving, so a native app gets the same guarantee +as Rust and `@moq/watch`: a broadcast proven to serve another instance fails +the rendition instead of playing it. + +## Plan + +Today `MoqVideo` and `MoqAudio` drop the field, and `MoqBroadcastConsumer::resolve`, +`decode_video`, `decode_audio` and libmoq's rendition resolution pass `None` +to `moq_mux::Source::resolve`, so they read like a reader that predates the +field: the first instance resolved, without its guarantee. + +- Add `epoch` beside `broadcast` on the binding records, and pass it from + `decode_video` and `decode_audio`. +- Decide the `resolve` shape with the maintainer: a second optional argument, + or one reference record carrying `broadcast` and `epoch`. +- Carry it through the hand-written wrappers and docs in the Cross-Package + Sync table (`rs/moq-c`, `py`, `swift`, `kt`, `go/wrapper`, `dart`, `cpp`, + `doc/lib`). + +Test a matching, mismatched and epochless reference through `moq-ffi` and +libmoq, and run `just test interop --all`. + +## Related + +- [Rust readers follow references](/quest/m0/broadcast-epoch/reference-announcements.md) - the following behavior the bindings inherit from `moq_mux` diff --git a/rs/hang/src/catalog/archive.rs b/rs/hang/src/catalog/archive.rs index 9c7199761a5..998d8b6cb9e 100644 --- a/rs/hang/src/catalog/archive.rs +++ b/rs/hang/src/catalog/archive.rs @@ -10,6 +10,7 @@ pub const MOQ_EPOCH_UNIX_MILLIS: u64 = 1_577_836_800_000; /// A live publisher sets the timeline fields (`timelines`, `timescale`, `duration_max`) alone, /// and a recording also names the replay broadcast and object store those spans live under. /// Every advertised span is FETCHable; with a store they are durable. +#[serde_with::serde_as] #[serde_with::skip_serializing_none] #[derive(Serialize, Deserialize, Debug, Clone, PartialEq)] #[serde(rename_all = "camelCase")] @@ -32,6 +33,12 @@ pub struct Archive { #[serde(default)] pub replay: Option, + /// The publisher instance [`replay`](Self::replay) names, when known. Absent accepts the + /// first one a reader resolves; see [`Reference::epoch`](super::Reference::epoch). + #[serde(default)] + #[serde_as(as = "Option")] + pub replay_epoch: Option, + /// The object-store URL the recording objects live under, if exposed by the publisher. #[serde(default)] pub store: Option, @@ -57,6 +64,7 @@ impl Archive { timescale: Self::default_timescale(), duration_max: None, replay: None, + replay_epoch: None, store: None, version: None, } diff --git a/rs/hang/src/catalog/audio/mod.rs b/rs/hang/src/catalog/audio/mod.rs index 01382bb0ea6..d2169e531a4 100644 --- a/rs/hang/src/catalog/audio/mod.rs +++ b/rs/hang/src/catalog/audio/mod.rs @@ -111,6 +111,16 @@ pub struct AudioConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// Human-readable rendition name for track pickers. #[serde(default)] pub label: Option, @@ -195,6 +205,7 @@ impl AudioConfig { Self { broadcast: None, track: None, + epoch: None, label: None, codec: codec.into(), sample_rate, diff --git a/rs/hang/src/catalog/binary.rs b/rs/hang/src/catalog/binary.rs index ebf746fe561..f0b08dab552 100644 --- a/rs/hang/src/catalog/binary.rs +++ b/rs/hang/src/catalog/binary.rs @@ -83,6 +83,16 @@ pub struct BinaryConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// Whether the track is a latest-value blob or an append log. Always stated: see [`Mode`]. #[serde_as(as = "serde_with::DisplayFromStr")] pub mode: Mode, @@ -131,6 +141,7 @@ impl BinaryConfig { Self { broadcast: None, track: None, + epoch: None, mode, compression: None, mime: None, diff --git a/rs/hang/src/catalog/json.rs b/rs/hang/src/catalog/json.rs index 08121614554..10478082256 100644 --- a/rs/hang/src/catalog/json.rs +++ b/rs/hang/src/catalog/json.rs @@ -81,6 +81,16 @@ pub struct JsonConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// Whether the track is a latest-value document or an append log. Always stated: see [`Mode`]. #[serde_as(as = "serde_with::DisplayFromStr")] pub mode: Mode, @@ -129,6 +139,7 @@ impl JsonConfig { Self { broadcast: None, track: None, + epoch: None, mode, compression: None, schema: None, diff --git a/rs/hang/src/catalog/reference.rs b/rs/hang/src/catalog/reference.rs index f134a1addf4..40f982854c6 100644 --- a/rs/hang/src/catalog/reference.rs +++ b/rs/hang/src/catalog/reference.rs @@ -4,11 +4,15 @@ use super::{AudioConfig, BinaryConfig, JsonConfig, TextConfig, VideoConfig}; /// /// An entry's key in the catalog is its ID. The track it names is `track` when set and the ID /// otherwise, in `broadcast` when set and the catalog's own broadcast otherwise. Every reader -/// subscribes through [`track`](Self::track) so the two never diverge. +/// subscribes through [`track`](Self::track) so the two never diverge. [`epoch`](Self::epoch) +/// optionally names the publisher instance the broadcast must be. pub trait Reference { /// The broadcast publishing the track, relative to the catalog's; `None` is the catalog's own. fn broadcast(&self) -> Option<&moq_net::path::RelativeOwned>; + /// The publisher instance the broadcast must be; `None` accepts the first one resolved. + fn epoch(&self) -> Option<&moq_net::Epoch>; + /// The track's name in its broadcast, given this entry's ID. fn track<'a>(&'a self, id: &'a str) -> &'a str; } @@ -20,6 +24,10 @@ macro_rules! reference { self.broadcast.as_ref() } + fn epoch(&self) -> Option<&moq_net::Epoch> { + self.epoch.as_ref() + } + fn track<'a>(&'a self, id: &'a str) -> &'a str { self.track.as_deref().unwrap_or(id) } diff --git a/rs/hang/src/catalog/root.rs b/rs/hang/src/catalog/root.rs index 11144461964..3698856fbb2 100644 --- a/rs/hang/src/catalog/root.rs +++ b/rs/hang/src/catalog/root.rs @@ -295,6 +295,49 @@ mod test { assert_eq!(Catalog::<()>::from_str(&catalog.to_json().unwrap()).unwrap(), catalog); } + /// Every reference-bearing section carries an optional epoch beside its `broadcast`. + #[test] + fn reference_epochs_roundtrip_in_every_section() { + use crate::catalog::Reference; + + const EPOCH: &str = "01890a5d-ac96-774b-bcce-b302099a8057"; + let encoded = format!( + r#"{{ + "video": {{"renditions": {{"v": {{"broadcast": "./a", "epoch": "{EPOCH}", "codec": "avc1.64001f", "container": {{"kind": "legacy"}}}}}}}}, + "audio": {{"renditions": {{"a": {{"broadcast": "./a", "epoch": "{EPOCH}", "codec": "opus", "sampleRate": 48000, "numberOfChannels": 2, "container": {{"kind": "legacy"}}}}}}}}, + "text": {{"renditions": {{"t": {{"broadcast": "./a", "epoch": "{EPOCH}", "format": "vtt"}}}}}}, + "json": {{"tracks": {{"j": {{"broadcast": "./a", "epoch": "{EPOCH}", "mode": "snapshot"}}}}}}, + "binary": {{"tracks": {{"b": {{"broadcast": "./a", "epoch": "{EPOCH}", "mode": "snapshot"}}}}}}, + "archive": {{"timelines": {{}}, "replay": "./a", "replayEpoch": "{EPOCH}"}} + }}"# + ); + let catalog = Catalog::<()>::from_str(&encoded).unwrap(); + let epoch: moq_net::Epoch = EPOCH.parse().unwrap(); + let references: [&dyn Reference; 5] = [ + &catalog.video.renditions["v"], + &catalog.audio.renditions["a"], + &catalog.text.renditions["t"], + &catalog.json.tracks["j"], + &catalog.binary.tracks["b"], + ]; + for reference in references { + assert_eq!(reference.epoch(), Some(&epoch)); + } + assert_eq!(catalog.archive.as_ref().unwrap().replay_epoch, Some(epoch)); + assert_eq!(Catalog::<()>::from_str(&catalog.to_json().unwrap()).unwrap(), catalog); + + // Omitted stays omitted, and a malformed or noncanonical epoch refuses the catalog. + assert!(!Catalog::<()>::default().to_json().unwrap().contains("poch")); + for bad in [ + "not-a-uuid", + "01890A5D-AC96-774B-BCCE-B302099A8057", + "01890a5d-ac96-474b-bcce-b302099a8057", + ] { + let encoded = encoded.replace(EPOCH, bad); + assert!(Catalog::<()>::from_str(&encoded).is_err(), "{bad} must be refused"); + } + } + #[test] fn cross_kind_duplicate_id_is_refused() { let catalog = Catalog::<()>::from_str( @@ -411,6 +454,7 @@ mod test { VideoConfig { broadcast: None, track: None, + epoch: None, label: None, codec: H264 { profile: 0x64, @@ -441,6 +485,7 @@ mod test { AudioConfig { broadcast: None, track: None, + epoch: None, label: None, codec: Opus, sample_rate: 48_000, diff --git a/rs/hang/src/catalog/text/mod.rs b/rs/hang/src/catalog/text/mod.rs index 4976a6fa3b3..fb0de18a2be 100644 --- a/rs/hang/src/catalog/text/mod.rs +++ b/rs/hang/src/catalog/text/mod.rs @@ -89,6 +89,16 @@ pub struct TextConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// The serialization format of each cue payload. #[serde_as(as = "DisplayFromStr")] pub format: TextFormat, @@ -141,6 +151,7 @@ impl TextConfig { Self { broadcast: None, track: None, + epoch: None, format, role: TextRole::default(), lang: None, diff --git a/rs/hang/src/catalog/video/mod.rs b/rs/hang/src/catalog/video/mod.rs index 75a25968476..322a75e03b3 100644 --- a/rs/hang/src/catalog/video/mod.rs +++ b/rs/hang/src/catalog/video/mod.rs @@ -190,6 +190,16 @@ pub struct VideoConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// Human-readable rendition name for track pickers. #[serde(default)] pub label: Option, @@ -303,6 +313,7 @@ impl VideoConfig { Self { broadcast: None, track: None, + epoch: None, label: None, codec: codec.into(), description: None, diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index 48e2c6acbf4..2f7c28c1692 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -87,9 +87,11 @@ async fn resolve( let origin = origin.ok_or_else(|| Error::UnresolvableBroadcast(reference.as_str().to_string()))?; let source = moq_mux::Source::new(origin.inner, broadcast); + // The C ABI does not expose a reference's epoch yet, so it reads like an older reader: the + // first instance resolved, without the epoch's guarantee. origin .driver - .watch(async { Ok(source.resolve(Some(&reference.borrow())).await?) }) + .watch(async { Ok(source.resolve(Some(&reference.borrow()), None).await?) }) .await } diff --git a/rs/moq-cli/src/play/media.rs b/rs/moq-cli/src/play/media.rs index 555eb84165a..fc980a0ad60 100644 --- a/rs/moq-cli/src/play/media.rs +++ b/rs/moq-cli/src/play/media.rs @@ -222,7 +222,7 @@ impl Media { // A rendition pointing at a broadcast we can't reach is that // rendition's problem, not the catalog's: fall through to the // next one like an unsupported codec does. - let rendition = match source.resolve(config.broadcast.as_ref()).await { + let rendition = match source.resolve(config.broadcast.as_ref(), config.epoch.as_ref()).await { Ok(rendition) => rendition, Err(err) => { tracing::warn!(track = name, %err, "cannot resolve video rendition"); @@ -287,7 +287,7 @@ impl Media { if playback.wants(Kind::Audio) { playback.read(Kind::Audio); for (name, config) in snapshot.audio.enabled() { - let rendition = match source.resolve(config.broadcast.as_ref()).await { + let rendition = match source.resolve(config.broadcast.as_ref(), config.epoch.as_ref()).await { Ok(rendition) => rendition, Err(err) => { tracing::warn!(track = name, %err, "cannot resolve audio rendition"); diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index cca2ad2cd15..366e01261e6 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -137,7 +137,9 @@ impl MoqBroadcastConsumer { })?; let source = moq_mux::Source::new(origin, self.inner.clone()); - Ok(source.resolve(Some(&reference)).await?) + // The bindings do not expose a reference's epoch yet, so they read like an older + // reader: the first instance resolved, without the epoch's guarantee. + Ok(source.resolve(Some(&reference), None).await?) }) .await } diff --git a/rs/moq-gst/src/source/imp.rs b/rs/moq-gst/src/source/imp.rs index 231bcd7f2eb..77e7b7f3c49 100644 --- a/rs/moq-gst/src/source/imp.rs +++ b/rs/moq-gst/src/source/imp.rs @@ -389,6 +389,7 @@ struct Shape { container: hang::catalog::Container, track: String, broadcast: Option, + epoch: Option, } /// A pump's progress, shared with its [`ActiveTrack`] so teardown and pad creation can't both @@ -855,6 +856,7 @@ fn reconcile( container: container.clone(), track: reference.track(name).to_string(), broadcast: reference.broadcast().cloned(), + epoch: reference.epoch().cloned(), }; desired.insert(name.clone(), Desired { kind, shape }); } @@ -949,7 +951,7 @@ fn reconcile( // Bind without waiting: an unavailable sibling or track must not park the catalog loop // and hold up every other rendition. The pump resolves both under its cancellation scope. - let binding = match source.bind(d.shape.broadcast.as_ref()) { + let binding = match source.bind(d.shape.broadcast.as_ref(), d.shape.epoch.as_ref()) { Ok(binding) => binding, Err(err) => { gst::warning!(CAT, "ignoring rendition {name}: {err:?}"); diff --git a/rs/moq-hls/src/export/rendition.rs b/rs/moq-hls/src/export/rendition.rs index 9d4c7ca24a8..048b4e8d34e 100644 --- a/rs/moq-hls/src/export/rendition.rs +++ b/rs/moq-hls/src/export/rendition.rs @@ -112,9 +112,9 @@ enum Config { /// never rebound onto whatever serves the path next. struct Media { handle: Mutex, - /// The source and reference a sibling is bound through, to re-issue a bind that failed - /// before it ever resolved. - sibling: Option<(moq_mux::Source, moq_net::path::RelativeOwned)>, + /// The source, reference, and epoch a sibling is bound through, to re-issue a bind that + /// failed before it ever resolved. + sibling: Option<(moq_mux::Source, moq_net::path::RelativeOwned, Option)>, } struct Handle { @@ -124,13 +124,14 @@ struct Handle { } impl Media { - fn bind(source: &moq_mux::Source, rel: Option<&moq_net::path::RelativeOwned>) -> moq_mux::Result { + fn bind(source: &moq_mux::Source, reference: &impl hang::catalog::Reference) -> moq_mux::Result { + let rel = reference.broadcast(); Ok(Self { handle: Mutex::new(Handle { - binding: Arc::new(source.bind(rel)?), + binding: Arc::new(source.bind(rel, reference.epoch())?), ended: false, }), - sibling: sibling(source, rel), + sibling: sibling(source, rel).map(|rel| (source.clone(), rel, reference.epoch().cloned())), }) } @@ -148,7 +149,7 @@ impl Media { /// [`sync`](Self::sync), registering `waiter` to wake when the sibling resolves or ends, so a /// waiter on the window learns of an end that nothing else touches the rendition for. fn poll_sync(&self, waiter: &kio::Waiter, window: &segments::Producer) -> Arc { - let Some((source, rel)) = &self.sibling else { + let Some((source, rel, epoch)) = &self.sibling else { return self.handle.lock().expect("media lock poisoned").binding.clone(); }; let mut handle = self.handle.lock().expect("media lock poisoned"); @@ -170,7 +171,7 @@ impl Media { // re-resolves a sibling. Poll::Ready(Err(_)) if !reissued => { reissued = true; - if let Ok(next) = source.bind(Some(rel)) { + if let Ok(next) = source.bind(Some(rel), epoch.as_ref()) { handle.binding = Arc::new(next); continue; } @@ -186,15 +187,12 @@ impl Media { fn sibling( source: &moq_mux::Source, rel: Option<&moq_net::path::RelativeOwned>, -) -> Option<(moq_mux::Source, moq_net::path::RelativeOwned)> { +) -> Option { let target = source.resolve_reference(rel)?; if source.resolve_reference(None).as_ref() == Some(&target) { return None; } - Some(( - source.clone(), - rel.cloned().unwrap_or_else(moq_net::path::Relative::empty), - )) + Some(rel.cloned().unwrap_or_else(moq_net::path::Relative::empty)) } /// Clear the video fields that don't change how the rendition decodes or is muxed, so a config @@ -361,7 +359,7 @@ impl Rendition { spans: Arc::new(Spans::new()), catalog: source.broadcast(), watcher: Mutex::new(None), - media: Media::bind(source, config.broadcast.as_ref())?, + media: Media::bind(source, config)?, run: Mutex::default(), building: tokio::sync::Mutex::new(()), }) @@ -392,7 +390,7 @@ impl Rendition { spans: Arc::new(Spans::new()), catalog: source.broadcast(), watcher: Mutex::new(None), - media: Media::bind(source, config.broadcast.as_ref())?, + media: Media::bind(source, config)?, run: Mutex::default(), building: tokio::sync::Mutex::new(()), }) diff --git a/rs/moq-mux/src/binary.rs b/rs/moq-mux/src/binary.rs index 33e95470dab..a9940b7a3f8 100644 --- a/rs/moq-mux/src/binary.rs +++ b/rs/moq-mux/src/binary.rs @@ -383,7 +383,11 @@ impl crate::catalog::Entry<'_, BinaryConfig> { /// Errors if the entry declares a mode or compression this build doesn't implement. pub async fn subscribe(&self, source: &crate::Source) -> crate::Result { let track = source - .subscribe_track(self.config().broadcast.as_ref(), self.config().track(self.name())) + .subscribe_track( + self.config().broadcast.as_ref(), + self.config().epoch.as_ref(), + self.config().track(self.name()), + ) .await?; Consumer::from_track(track, self.config()) } diff --git a/rs/moq-mux/src/catalog/producer.rs b/rs/moq-mux/src/catalog/producer.rs index 0b80c54512e..27edfb395c1 100644 --- a/rs/moq-mux/src/catalog/producer.rs +++ b/rs/moq-mux/src/catalog/producer.rs @@ -1136,6 +1136,10 @@ fn to_msf_media(catalog: &hang::Catalog, base: &moq_net::Path) -> let mut names = BTreeSet::new(); let mut reference = |id: &str, config: &dyn Reference| { let name = config.track(id); + if let Some(epoch) = config.epoch() { + // MSF has nowhere to name an instance, so its readers take whichever resolves first. + tracing::warn!(rendition = id, %epoch, "MSF cannot carry a reference epoch; advertising without it"); + } let namespace = match config.broadcast().filter(|rel| !rel.is_empty()) { None => None, Some(rel) => { diff --git a/rs/moq-mux/src/container/source.rs b/rs/moq-mux/src/container/source.rs index dccea4b3bfc..cc476fb8294 100644 --- a/rs/moq-mux/src/container/source.rs +++ b/rs/moq-mux/src/container/source.rs @@ -159,7 +159,7 @@ impl ExportSource { ) -> Result, crate::Error> { let media: HangContainer = config.try_into()?; let description = config.description.as_ref().filter(|b| !b.is_empty()).cloned(); - let Some(request) = source.try_request(config.broadcast.as_ref()) else { + let Some(request) = source.try_request(config) else { return Ok(None); }; @@ -194,7 +194,7 @@ impl ExportSource { ) -> Result, crate::Error> { let media: HangContainer = config.try_into()?; let description = config.description.as_ref().filter(|b| !b.is_empty()).cloned(); - let Some(request) = source.try_request(config.broadcast.as_ref()) else { + let Some(request) = source.try_request(config) else { return Ok(None); }; diff --git a/rs/moq-mux/src/error.rs b/rs/moq-mux/src/error.rs index 7299a9f4b4a..0e8b03275eb 100644 --- a/rs/moq-mux/src/error.rs +++ b/rs/moq-mux/src/error.rs @@ -242,6 +242,20 @@ pub enum Error { #[error("broadcast reference escapes the root: {0}")] EscapingBroadcast(String), + /// A reference's `epoch` names another publisher instance than the one serving its broadcast. + /// + /// The served instance is the one the source resolved first for that path, so a later + /// reference cannot move it; the reference fails instead of reading other content. + #[error("broadcast {path} serves epoch {actual}, not the referenced {expected}")] + EpochMismatch { + /// The broadcast's path, relative to the origin root. + path: String, + /// The epoch the reference names. + expected: moq_net::Epoch, + /// The epoch the resolved broadcast serves. + actual: moq_net::Epoch, + }, + /// A rendition tried to lower jitter already advertised to subscribers. #[error("catalog jitter cannot decrease for a published rendition")] JitterDecreased, diff --git a/rs/moq-mux/src/json.rs b/rs/moq-mux/src/json.rs index 2e65a85154b..f4463bf0c8f 100644 --- a/rs/moq-mux/src/json.rs +++ b/rs/moq-mux/src/json.rs @@ -438,7 +438,11 @@ impl crate::catalog::Entry<'_, JsonConfig> { /// Errors if the entry declares a mode or compression this build doesn't implement. pub async fn subscribe(&self, source: &crate::Source) -> crate::Result> { let track = source - .subscribe_track(self.config().broadcast.as_ref(), self.config().track(self.name())) + .subscribe_track( + self.config().broadcast.as_ref(), + self.config().epoch.as_ref(), + self.config().track(self.name()), + ) .await?; Consumer::from_track(track, self.config()) } diff --git a/rs/moq-mux/src/source.rs b/rs/moq-mux/src/source.rs index 5cb73cb4188..1d95334e739 100644 --- a/rs/moq-mux/src/source.rs +++ b/rs/moq-mux/src/source.rs @@ -29,6 +29,12 @@ type Pins = Arc> /// and an export's track requests). A route replaced after that is never resolved, epoch or /// not, so two publishers never splice into one export; a request on a held broadcast that has /// ended fails instead. Clones share the held broadcasts. +/// +/// A reference may name the instance it expects with an epoch (the catalog's optional `epoch` +/// beside `broadcast`). A held or resolved broadcast serving another epoch fails that reference +/// with [`Error::EpochMismatch`](crate::Error::EpochMismatch) rather than replacing the held +/// one or falling back to it. A broadcast resolved over a route without an epoch cannot prove +/// its instance, so there the epoch is best effort: the reference reads the held broadcast. #[derive(Clone)] pub struct Source { origin: moq_net::origin::Consumer, @@ -104,16 +110,23 @@ impl Source { Binding::new(self.broadcast()) } - /// The broadcast `path` is pinned to, or a lookup that pins its answer. - fn request_path(&self, path: &moq_net::PathOwned) -> Binding { - if let Some(pinned) = self.pins.lock().expect("pins poisoned").get(path) { - return Binding::new(pinned.clone()); + /// The broadcast `path` is pinned to, or a lookup that pins its answer, either checked + /// against `epoch` on every read. + fn request_path(&self, path: &moq_net::PathOwned, epoch: Option<&moq_net::Epoch>) -> Binding { + let bound = match self.pins.lock().expect("pins poisoned").get(path) { + Some(pinned) => Bound::Ready(pinned.clone()), + // Unpinned even with an epoch: an epochless route still answers, best effort, and + // the read refuses a route that proves another instance. + None => Bound::Requested { + request: self.origin.request_broadcast(path, None).into_inner(), + pins: self.pins.clone(), + path: path.clone(), + }, + }; + Binding { + bound, + epoch: epoch.cloned(), } - Binding(Bound::Requested { - request: self.origin.request_broadcast(path, None).into_inner(), - pins: self.pins.clone(), - path: path.clone(), - }) } /// Resolve a rendition's optional broadcast reference to an origin path. @@ -140,14 +153,15 @@ impl Source { }) } - /// The skipping counterpart to [`Self::bind`], returning `None` when `rel` walks above - /// the origin root. + /// The skipping counterpart to [`Self::bind`] for a catalog entry, returning `None` when its + /// broadcast walks above the origin root. /// /// The exporters use it to drop a single bad rendition rather than fail the whole export. /// [`Self::retain_selectable`] already removes those renditions, so this is the second half of /// the same policy rather than a different one. - pub(crate) fn try_request(&self, rel: Option<&moq_net::path::Relative<'_>>) -> Option { - Some(self.request_path(&self.resolve_reference(rel)?)) + pub(crate) fn try_request(&self, reference: &impl hang::catalog::Reference) -> Option { + let path = self.resolve_reference(reference.broadcast())?; + Some(self.request_path(&path, reference.epoch())) } /// Remove the renditions an export must not select: a disabled one (catalog @@ -193,7 +207,8 @@ impl Source { /// /// `rel` is a rendition's catalog `broadcast` field: `None` (or an empty reference) /// resolves the catalog broadcast itself; anything else fetches the referenced - /// broadcast from the origin. Use it when you need the broadcast handle itself + /// broadcast from the origin. `epoch` is the rendition's `epoch` field, the instance the + /// broadcast must be (see [`Source`]). Use it when you need the broadcast handle itself /// (e.g. to FETCH individual groups) rather than a subscription. /// /// A reference that escapes above the origin root is @@ -202,8 +217,9 @@ impl Source { pub async fn resolve( &self, rel: Option<&moq_net::path::Relative<'_>>, + epoch: Option<&moq_net::Epoch>, ) -> crate::Result { - self.bind(rel)?.broadcast().await + self.bind(rel, epoch)?.broadcast().await } /// Start one broadcast request now and retain its result for repeated reads. @@ -217,9 +233,14 @@ impl Source { /// one upstream subscription, and its answer is pinned. Use [`Binding::new`] when the /// intended broadcast is already in hand. /// - /// Rejects an escaping reference exactly as [`resolve`](Self::resolve) does. - pub fn bind(&self, rel: Option<&moq_net::path::Relative<'_>>) -> crate::Result { - Ok(self.request_path(&self.target(rel)?)) + /// Rejects an escaping reference exactly as [`resolve`](Self::resolve) does, and checks + /// `epoch` on every read. + pub fn bind( + &self, + rel: Option<&moq_net::path::Relative<'_>>, + epoch: Option<&moq_net::Epoch>, + ) -> crate::Result { + Ok(self.request_path(&self.target(rel)?, epoch)) } /// Resolve an optional cross-broadcast reference and subscribe to track `name`, @@ -227,7 +248,8 @@ impl Source { /// /// `rel` is a rendition's catalog `broadcast` field: `None` (or an empty reference) /// subscribes on the catalog broadcast; anything else fetches the referenced broadcast - /// from the origin first. A reference that escapes above the origin root is + /// from the origin first, checked against `epoch` as in [`resolve`](Self::resolve). A + /// reference that escapes above the origin root is /// [`Error::EscapingBroadcast`](crate::Error::EscapingBroadcast). /// /// This is the async counterpart to the poll-driven container exporters: consumers @@ -236,9 +258,10 @@ impl Source { pub async fn subscribe_track( &self, rel: Option<&moq_net::path::Relative<'_>>, + epoch: Option<&moq_net::Epoch>, name: &str, ) -> crate::Result { - let broadcast = self.bind(rel)?.broadcast().await?; + let broadcast = self.bind(rel, epoch)?.broadcast().await?; Ok(broadcast.track(name)?.subscribe(None).await?) } } @@ -247,10 +270,15 @@ impl Source { /// /// Reading the binding never looks up its path again, even after a failure or publisher /// replacement. Pending requests follow the origin's routing semantics; issuing a request -/// does not establish an epoch relationship with a separate catalog broadcast. +/// does not establish an epoch relationship with a separate catalog broadcast. A binding made +/// with an epoch refuses a broadcast proven to serve another one. /// /// Build one with [`Source::bind`], or with [`Binding::new`] for a broadcast already in hand. -pub struct Binding(Bound); +pub struct Binding { + bound: Bound, + /// The instance the reference named, if any. + epoch: Option, +} enum Bound { /// A broadcast the caller already holds. @@ -267,22 +295,39 @@ enum Bound { impl Binding { /// Bind to a broadcast already in hand, for a reference that named it. pub fn new(broadcast: moq_net::broadcast::Consumer) -> Self { - Self(Bound::Ready(broadcast)) + Self { + bound: Bound::Ready(broadcast), + epoch: None, + } } /// Poll for the bound broadcast without blocking. /// /// Concurrent polls and cancelled waits retain the same request and result. pub fn poll_broadcast(&self, waiter: &kio::Waiter) -> Poll> { - match &self.0 { - Bound::Ready(broadcast) => Poll::Ready(Ok(broadcast.clone())), + let broadcast = match &self.bound { + Bound::Ready(broadcast) => broadcast.clone(), Bound::Requested { request, pins, path } => { let broadcast = ready!(request.poll_ok(waiter))?; - // A request that raced another for the path takes the one pinned first. + // A request that raced another for the path takes the one pinned first. The path's + // first answer is pinned even when it fails this binding's epoch, so no later + // reference resolves the path independently. let mut pins = pins.lock().expect("pins poisoned"); - Poll::Ready(Ok(pins.entry(path.clone()).or_insert(broadcast).clone())) + pins.entry(path.clone()).or_insert(broadcast).clone() } + }; + // An epochless route cannot prove its instance, so only a known mismatch fails. + let info = broadcast.info(); + if let (Some(expected), Some(actual)) = (&self.epoch, &info.epoch) + && expected != actual + { + return Poll::Ready(Err(crate::Error::EpochMismatch { + path: info.path.to_string(), + expected: expected.clone(), + actual: actual.clone(), + })); } + Poll::Ready(Ok(broadcast)) } /// The bound broadcast, awaiting the origin's answer when it hasn't arrived yet. @@ -358,7 +403,7 @@ mod tests { settle().await; let source = resolved(&origin, "pub").await; let dynamic = origin.dynamic("media", Default::default()).unwrap(); - let binding = source.bind(Some(&Relative::new("media"))).unwrap(); + let binding = source.bind(Some(&Relative::new("media")), None).unwrap(); let request = dynamic.requested_broadcast().await.unwrap(); assert!( @@ -392,11 +437,11 @@ mod tests { settle().await; let source = resolved(&origin, "pub").await; let media = Relative::new("media"); - let binding = source.bind(Some(&media)).unwrap(); + let binding = source.bind(Some(&media), None).unwrap(); assert!(binding.broadcast().await.is_err()); let _publisher = origin.publish("media", Default::default()).unwrap(); settle().await; - assert!(!source.resolve(Some(&media)).await.unwrap().is_closed()); + assert!(!source.resolve(Some(&media), None).await.unwrap().is_closed()); assert!(binding.broadcast().await.is_err()); } @@ -415,9 +460,9 @@ mod tests { settle().await; let source = Source::new(consumer, first.clone()); assert!(source.broadcast().is_clone(&first)); - assert!(source.resolve(None).await.unwrap().is_clone(&first)); + assert!(source.resolve(None, None).await.unwrap().is_clone(&first)); assert!( - source.subscribe_track(None, "late").await.is_err(), + source.subscribe_track(None, None, "late").await.is_err(), "the replacement's track is never read" ); } @@ -431,20 +476,142 @@ mod tests { settle().await; let source = resolved(&origin, "a/pub").await; let sibling = Relative::new("./source"); - let first = source.resolve(Some(&sibling)).await.expect("the sibling resolves"); + let first = source + .resolve(Some(&sibling), None) + .await + .expect("the sibling resolves"); let new = origin.publish("a/source", Default::default()).unwrap(); let _late = new.create_track("late", None).unwrap(); settle().await; - assert!(source.resolve(Some(&sibling)).await.unwrap().is_clone(&first)); + assert!(source.resolve(Some(&sibling), None).await.unwrap().is_clone(&first)); assert!( - source.subscribe_track(Some(&sibling), "late").await.is_err(), + source.subscribe_track(Some(&sibling), None, "late").await.is_err(), "the replacement's track is never read" ); - let bound = source.bind(Some(&sibling)).unwrap().broadcast().await.unwrap(); + let bound = source.bind(Some(&sibling), None).unwrap().broadcast().await.unwrap(); assert!(bound.is_clone(&first), "a binding honors the pin too"); } + fn route(epoch: &moq_net::Epoch) -> moq_net::origin::Route { + moq_net::origin::Route::default().with_epoch(epoch.clone()) + } + + fn is_mismatch(result: crate::Result, expected: &moq_net::Epoch) -> bool { + matches!(result, Err(crate::Error::EpochMismatch { expected: ref named, .. }) if named == expected) + } + + /// An omitted or matching epoch resolves the sibling; another epoch fails instead. + #[tokio::test(start_paused = true)] + async fn a_reference_epoch_must_match_the_resolved_sibling() { + let origin = produce_origin(); + let (catalog, sibling, other) = (moq_net::Epoch::mint(), moq_net::Epoch::mint(), moq_net::Epoch::mint()); + let _catalog = origin.publish("a/pub", route(&catalog)).unwrap(); + let _sibling = origin.publish("a/source", route(&sibling)).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + let rel = Relative::new("./source"); + + let first = source.resolve(Some(&rel), None).await.unwrap(); + assert_eq!(first.info().epoch.as_ref(), Some(&sibling)); + let matching = source.resolve(Some(&rel), Some(&sibling)).await.unwrap(); + assert!(matching.is_clone(&first)); + assert!(is_mismatch(source.resolve(Some(&rel), Some(&other)).await, &other)); + assert!( + source.subscribe_track(Some(&rel), Some(&other), "video").await.is_err(), + "a mismatched reference reads nothing" + ); + } + + /// A replacement before the first resolution fails a reference naming the old instance, + /// rather than falling back to the current winner. + #[tokio::test(start_paused = true)] + async fn a_replaced_sibling_fails_a_reference_to_the_old_epoch() { + let origin = produce_origin(); + let (old_epoch, new_epoch) = (moq_net::Epoch::mint(), moq_net::Epoch::mint()); + let _catalog = origin.publish("a/pub", Default::default()).unwrap(); + let _old = origin.publish("a/source", route(&old_epoch)).unwrap(); + let _new = origin.publish("a/source", route(&new_epoch)).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + let rel = Relative::new("./source"); + + assert!(is_mismatch( + source.resolve(Some(&rel), Some(&old_epoch)).await, + &old_epoch + )); + // The path's first answer is pinned all the same: later references read that instance. + let pinned = source.resolve(Some(&rel), None).await.unwrap(); + assert_eq!(pinned.info().epoch.as_ref(), Some(&new_epoch)); + } + + /// A later reference naming the replacement cannot move a pinned sibling, and a new + /// catalog instance starts fresh pins that resolve it. + #[tokio::test(start_paused = true)] + async fn a_conflicting_epoch_cannot_replace_a_pin() { + let origin = produce_origin(); + let (old_epoch, new_epoch) = (moq_net::Epoch::mint(), moq_net::Epoch::mint()); + let _catalog = origin.publish("a/pub", Default::default()).unwrap(); + let _old = origin.publish("a/source", route(&old_epoch)).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + let rel = Relative::new("./source"); + let first = source.resolve(Some(&rel), Some(&old_epoch)).await.unwrap(); + + let _new = origin.publish("a/source", route(&new_epoch)).unwrap(); + settle().await; + // A rendition added later names the replacement: it fails, and the pin stays. + let binding = source.bind(Some(&rel), Some(&new_epoch)).unwrap(); + assert!(is_mismatch(binding.broadcast().await, &new_epoch)); + assert!(source.resolve(Some(&rel), None).await.unwrap().is_clone(&first)); + assert!( + source + .resolve(Some(&rel), Some(&old_epoch)) + .await + .unwrap() + .is_clone(&first) + ); + + // Concurrent consumers: a new catalog instance resolves the replacement while the old + // source keeps reading its pin. + let fresh = source.switched(&source.broadcast()); + let replaced = fresh.resolve(Some(&rel), Some(&new_epoch)).await.unwrap(); + assert_eq!(replaced.info().epoch.as_ref(), Some(&new_epoch)); + assert!(source.resolve(Some(&rel), None).await.unwrap().is_clone(&first)); + } + + /// A self-reference's epoch must not contradict the catalog broadcast's own. + #[tokio::test(start_paused = true)] + async fn a_self_reference_epoch_checks_the_catalog_instance() { + let origin = produce_origin(); + let (catalog, other) = (moq_net::Epoch::mint(), moq_net::Epoch::mint()); + let _catalog = origin.publish("a/pub", route(&catalog)).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + + for reference in [None, Some("./pub")] { + let rel = reference.map(Relative::new); + let own = source.resolve(rel.as_ref(), Some(&catalog)).await.unwrap(); + assert!(own.is_clone(&source.broadcast()), "{reference:?}"); + assert!(is_mismatch(source.resolve(rel.as_ref(), Some(&other)).await, &other)); + } + } + + /// An epochless route cannot prove its instance, so an explicit epoch is best effort there. + #[tokio::test(start_paused = true)] + async fn an_epochless_route_reads_an_epoch_reference_best_effort() { + let origin = produce_origin(); + let _catalog = origin.publish("a/pub", Default::default()).unwrap(); + let _sibling = origin.publish("a/source", Default::default()).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + let epoch = moq_net::Epoch::mint(); + + let sibling = source.resolve(Some(&Relative::new("./source")), Some(&epoch)).await; + assert!(sibling.unwrap().info().epoch.is_none()); + assert!(source.resolve(None, Some(&epoch)).await.is_ok()); + } + /// Every self-reference serves the broadcast the source was built from, even once it /// ended and a replacement took the path. #[tokio::test(start_paused = true)] @@ -459,7 +626,7 @@ mod tests { for reference in [None, Some(""), Some("live"), Some("./live"), Some("../a/live")] { let rel = reference.map(Relative::new); - let bound = source.bind(rel.as_ref()).unwrap().broadcast().await.unwrap(); + let bound = source.bind(rel.as_ref(), None).unwrap().broadcast().await.unwrap(); assert!( bound.is_closed(), "{reference:?} must keep the original catalog broadcast" @@ -478,14 +645,14 @@ mod tests { // No reference and an empty reference both resolve to the catalog broadcast. source - .bind(None) + .bind(None, None) .expect("no reference is always resolvable") .broadcast() .await .expect("catalog broadcast should resolve"); let empty = Relative::empty(); source - .bind(Some(&empty)) + .bind(Some(&empty), None) .expect("empty reference is always resolvable") .broadcast() .await @@ -503,7 +670,7 @@ mod tests { let source = resolved(&origin, "a/pub").await; source - .subscribe_track(None, "video") + .subscribe_track(None, None, "video") .await .expect("catalog track should resolve"); } @@ -521,7 +688,7 @@ mod tests { // Names the catalog within its own parent. let rel = Relative::new("./pub"); source - .subscribe_track(Some(&rel), "video") + .subscribe_track(Some(&rel), None, "video") .await .expect("self-reference should resolve to the catalog broadcast"); } @@ -551,16 +718,16 @@ mod tests { source.resolve_reference(Some(&rel)).is_none(), "{reference} should escape" ); - assert!(source.bind(Some(&rel)).is_err(), "{reference} should be rejected"); + assert!(source.bind(Some(&rel), None).is_err(), "{reference} should be rejected"); // Neither `elsewhere` nor the catalog broadcast answers it: the rendition names // no broadcast at all. - match source.resolve(Some(&rel)).await { + match source.resolve(Some(&rel), None).await { Err(crate::Error::EscapingBroadcast(_)) => {} Err(err) => panic!("{reference} failed with the wrong error: {err:?}"), Ok(_) => panic!("{reference} should not resolve to any broadcast"), } - match source.subscribe_track(Some(&rel), "video").await { + match source.subscribe_track(Some(&rel), None, "video").await { Err(crate::Error::EscapingBroadcast(_)) => {} Err(err) => panic!("{reference} failed with the wrong error: {err:?}"), Ok(_) => panic!("{reference} should not resolve to any broadcast"), @@ -662,7 +829,7 @@ mod tests { // The reference resolves to `a/source`, whose "video" track answers the subscribe. let rel = Relative::new("./source"); source - .subscribe_track(Some(&rel), "video") + .subscribe_track(Some(&rel), None, "video") .await .expect("referenced track should resolve"); } @@ -682,7 +849,7 @@ mod tests { let source = resolved(&origin, "a/source/transcode").await; let rel = Relative::new("."); source - .subscribe_track(Some(&rel), "video") + .subscribe_track(Some(&rel), None, "video") .await .expect("dot should resolve to the catalog broadcast's parent"); } @@ -702,7 +869,7 @@ mod tests { let source = resolved(&origin, "top").await; let rel = Relative::new("."); source - .subscribe_track(Some(&rel), "video") + .subscribe_track(Some(&rel), None, "video") .await .expect("dot should resolve to the empty root broadcast"); } diff --git a/rs/moq-rtc/src/egress.rs b/rs/moq-rtc/src/egress.rs index 342b46e1ab6..8e865aa2052 100644 --- a/rs/moq-rtc/src/egress.rs +++ b/rs/moq-rtc/src/egress.rs @@ -260,16 +260,18 @@ impl Rendition { /// Subscribe to the rendition, honoring its cross-broadcast reference (the catalog /// `broadcast` field). async fn subscribe(self, source: moq_mux::Source) -> Result { + use hang::catalog::Reference; + match &self { Self::Audio(name, config) => { let track = source - .subscribe_track(config.broadcast.as_ref(), hang::catalog::Reference::track(config, name)) + .subscribe_track(config.broadcast(), config.epoch(), config.track(name)) .await?; codec::Track::opus(track, config) } Self::Video(name, config) => { let track = source - .subscribe_track(config.broadcast.as_ref(), hang::catalog::Reference::track(config, name)) + .subscribe_track(config.broadcast(), config.epoch(), config.track(name)) .await?; codec::Track::video(track, config) } From 10aa69a22ee173679304a66df2a05cbd87022cf1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 15 Oct 2026 23:55:47 -0700 Subject: [PATCH 3/4] fix(hang): name the reference epoch brand inside @moq/hang Co-Authored-By: Claude Opus 5.5 --- js/hang/src/catalog/path.ts | 10 ++++++++-- js/hang/src/catalog/root.test.ts | 4 ++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/js/hang/src/catalog/path.ts b/js/hang/src/catalog/path.ts index 62eb3b6a385..c88194dbe84 100644 --- a/js/hang/src/catalog/path.ts +++ b/js/hang/src/catalog/path.ts @@ -33,7 +33,13 @@ function isEpoch(text: string): boolean { * `broadcast`, or the archive's `replayEpoch`. A lowercase hyphenated UUIDv7, the same text MoQ * routes carry; anything else refuses the catalog. */ -export const EpochSchema: z.ZodMiniType = z.pipe( +export const EpochSchema: z.ZodMiniType = z.pipe( z.string().check(z.refine(isEpoch, "expected a lowercase hyphenated UUIDv7")), - z.transform((text) => text as Epoch.Valid), + z.transform((text) => text as ReferenceEpoch), ); + +/** + * A validated reference epoch: the same brand as `Epoch.Valid` from `@moq/net`, spelled out here + * so the catalog schemas' inferred types stay nameable from this package. + */ +export type ReferenceEpoch = string & { readonly _brand: "epoch" }; diff --git a/js/hang/src/catalog/root.test.ts b/js/hang/src/catalog/root.test.ts index 0cc92855ca0..15e96a3dddc 100644 --- a/js/hang/src/catalog/root.test.ts +++ b/js/hang/src/catalog/root.test.ts @@ -2,7 +2,7 @@ import { expect, test } from "bun:test"; import * as z from "@zod/mini"; import { ARCHIVE_VERSION, ArchiveSchema } from "./archive.ts"; import { u53 } from "./integers.ts"; -import type { RelativeBroadcast } from "./path.ts"; +import type { ReferenceEpoch, RelativeBroadcast } from "./path.ts"; import { checkRenditions, DuplicateRendition, RootSchema } from "./root.ts"; import { track } from "./track.ts"; @@ -47,7 +47,7 @@ test("rendition broadcast reference is parsed and normalized", () => { // Mirrors Rust's `reference_epochs_roundtrip_in_every_section`. test("every reference-bearing section carries an optional epoch", () => { - const EPOCH = "01890a5d-ac96-774b-bcce-b302099a8057"; + const EPOCH = "01890a5d-ac96-774b-bcce-b302099a8057" as ReferenceEpoch; const catalog = (epoch: string) => ({ video: { renditions: { From c011d11e4db3278ec06c5b7c8315c3e917960427 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 20:08:22 -0700 Subject: [PATCH 4/4] Reconcile catalog reference epochs with announced_broadcast The reference-announcements quest now builds on moq-net's announced_broadcast, following unpinned so epochless routes stay best effort. Move the migration note into the moq-mux and hang MIGRATE.md guides, and note that MoQ Transport with the epoch extension carries an Epoch. Co-Authored-By: Claude Opus 5.5 --- drafts/draft-lcurley-moq-hang.md | 10 +++++++++- quest/m0/broadcast-epoch/reference-announcements.md | 11 ++++++++--- rs/hang/MIGRATE.md | 5 +++++ rs/moq-mux/MIGRATE.md | 6 ++++++ 4 files changed, 28 insertions(+), 4 deletions(-) diff --git a/drafts/draft-lcurley-moq-hang.md b/drafts/draft-lcurley-moq-hang.md index c92c2eae8a9..7ec39a1aa6e 100644 --- a/drafts/draft-lcurley-moq-hang.md +++ b/drafts/draft-lcurley-moq-hang.md @@ -35,6 +35,14 @@ normative: date: false informative: + moqepoch: + title: "MoQ Epoch Extension" + target: https://datatracker.ietf.org/doc/draft-lcurley-moq-epoch/ + author: + - + ins: L. Curley + name: Luke Curley + date: false h273: title: "Coding-independent code points for video signal type identification" target: https://www.itu.int/rec/T-REC-H.273 @@ -685,7 +693,7 @@ It MUST fail the rendition rather than fall back to another instance, including A later catalog update naming another Epoch for an already-resolved path MUST NOT replace the resolved broadcast; the renditions naming the other Epoch fail instead. A new catalog instance resolves its references afresh. -A route without an Epoch, such as one over an older moq-lite version or MoQ Transport, cannot prove which instance it serves, so there `epoch` is best effort: the consumer reads the broadcast it resolved. +A route without an Epoch, such as one over an older moq-lite version or MoQ Transport without the epoch extension {{moqepoch}}, cannot prove which instance it serves, so there `epoch` is best effort: the consumer reads the broadcast it resolved. A consumer MUST NOT treat a catalog's `epoch` as evidence that a subscription may resume across routes. A consumer that predates the field ignores it and does not gain its guarantee. diff --git a/quest/m0/broadcast-epoch/reference-announcements.md b/quest/m0/broadcast-epoch/reference-announcements.md index 09dd8acfd7a..cfde1ceeef0 100644 --- a/quest/m0/broadcast-epoch/reference-announcements.md +++ b/quest/m0/broadcast-epoch/reference-announcements.md @@ -14,9 +14,14 @@ shared resolution in `moq_mux::Source` and `@moq/watch`, and the Hang draft text; this quest owns the following behavior it left out. - Share it in `moq_mux` (a `Source::follow`-style API; the shape is open) - rather than per reader. `Source::bind` binds once today, as - `failed_binding_does_not_retry_a_later_publisher` asserts; that changes. - The "no retarget" rule applies only to readers that follow no announcement. + rather than per reader, built on `origin::Consumer::announced_broadcast` + rather than another hand-rolled loop. Follow unpinned and check each + handed-out instance's epoch, as `Source::bind` does: a pinned + `announced_broadcast` waits out an epochless route, which would break the + best effort the reference epoch promises there. `Source::bind` binds once + today, as `failed_binding_does_not_retry_a_later_publisher` asserts; that + changes. The "no retarget" rule applies only to readers that follow no + announcement. - `moqsrc`: Start binds the rendition, End idles its pad, and Restart restarts its pump. - Exporters: a late first announcement starts the rendition and End ends diff --git a/rs/hang/MIGRATE.md b/rs/hang/MIGRATE.md index 6cce9f986bd..0087dd6e439 100644 --- a/rs/hang/MIGRATE.md +++ b/rs/hang/MIGRATE.md @@ -4,6 +4,11 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `catalog::Reference` names its epoch (#5699) + +Every rendition and data track config gains an optional `epoch` beside `broadcast`, and `Archive` gains `replay_epoch` beside `replay`. +`Reference` has a new required `epoch()`: an implementor outside this crate returns the entry's epoch, or `None` when it names none. + ### `M2ts::random_access` covers the whole track (#5967) `random_access` no longer describes the latest two-second window: it is true only while every group the track has published started at a random access point. diff --git a/rs/moq-mux/MIGRATE.md b/rs/moq-mux/MIGRATE.md index 91b85a9b273..8763cb8c6a0 100644 --- a/rs/moq-mux/MIGRATE.md +++ b/rs/moq-mux/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `Source` resolution takes the reference's epoch (#5699) + +`Source::bind(rel)`, `resolve(rel)`, and `subscribe_track(rel, name)` take the catalog entry's optional epoch after `rel`. +Pass `config.epoch.as_ref()` (or `Reference::epoch`), or `None` to accept the first instance resolved as before. +A broadcast proven to serve another epoch fails with the new `Error::EpochMismatch`; over a route without an epoch the check is best effort. + ### JSON and binary consumers return `Timed` (#5677) The `json` and `binary` consumers' `next` / `poll_next` return `moq_net::Timed`; read `.value` and the optional `.at`.