diff --git a/doc/concept/hang.md b/doc/concept/hang.md index 2ff655fb8fb..35a5cab213d 100644 --- a/doc/concept/hang.md +++ b/doc/concept/hang.md @@ -63,7 +63,7 @@ decoder once per track. A few things the catalog can express beyond decoder config: - **Labels.** Any rendition may carry a human-readable `label` for a track picker. The map key stays the rendition's ID, so labels need not be unique and renaming one doesn't rename the track. -- **Renditions in another broadcast.** A rendition may point at a relative broadcast path, so a transcoder can publish a ladder that adds low rungs and references the source's original rendition without re-publishing its bytes. The path resolves against where the consumer found the catalog, so a reference that escapes above the root names nothing and the catalog is rejected. +- **Renditions in another broadcast.** A rendition may point at a relative broadcast path, so a transcoder can publish a ladder that adds low rungs and references the source's original rendition without re-publishing its bytes. The path resolves against where the consumer found the catalog, so a reference that escapes above the root names nothing and the catalog is rejected. A consumer reads each referenced path from the first broadcast it resolves there. An optional `epoch` beside the path names the [publisher instance](/concept/moq-lite#publisher-epochs) it expects: a route proving another instance fails the rendition instead of falling back to it, while a route without an epoch (an older version) cannot prove one, so there it is best effort. - **Track names.** The map key is a rendition's ID and, by default, its track name. A rendition sets `track` only when the two must differ, such as two renditions of same-named tracks in different broadcasts, or a rendition whose media changed and so needs a new name; readers subscribe to `track` when present. An ID can't name both a video and an audio rendition. - **Jitter.** A rendition can say how far its frames fell behind the media clock before the publisher flushed them, in whole milliseconds rounded up. Encoders report the spread of lateness above each rendition's own recent minimum, so a constant encoder delay is not jitter; container imports estimate batch spans without counting ingest delay. It describes the publisher, never the network, only grows over the life of a stream, and a player sizes its buffer to at least this much. A `0` is read as absent. - **Delay.** A rendition can also say how far its frames reach the transport behind the broadcast's earliest rendition, measured the same way from each rendition's minimum lateness, so a video encoder running 200 ms behind audio advertises `delay: 200` on video. It follows the same rules as jitter. A player holds the largest `delay + jitter` among the renditions it subscribes to, and never subtracts one rendition's `delay` from another's. diff --git a/drafts/draft-lcurley-moq-hang.md b/drafts/draft-lcurley-moq-hang.md index 576b331cf17..7ec39a1aa6e 100644 --- a/drafts/draft-lcurley-moq-hang.md +++ b/drafts/draft-lcurley-moq-hang.md @@ -35,6 +35,14 @@ normative: date: false informative: + moqepoch: + title: "MoQ Epoch Extension" + target: https://datatracker.ietf.org/doc/draft-lcurley-moq-epoch/ + author: + - + ins: L. Curley + name: Luke Curley + date: false h273: title: "Coding-independent code points for video signal type identification" target: https://www.itu.int/rec/T-REC-H.273 @@ -615,7 +623,7 @@ The `deflate` value is the group-scoped DEFLATE of {{compression}}. A `snapshot` group covers a single value (plus any deltas), so its window spans that group alone; a `stream` group's frames compress against the earlier ones in the log. ### broadcast and track {#data-shared} -The `broadcast` and `track` fields carry the same meaning here as they do for a media rendition ({{field-broadcast}}, {{field-track}}). +The `broadcast`, `epoch`, and `track` fields carry the same meaning here as they do for a media rendition ({{field-broadcast}}, {{field-epoch}}, {{field-track}}). ### bitrate, jitter, and delay {#data-estimates} The optional `bitrate` field is the track's maximum bitrate in bits per second. @@ -639,6 +647,7 @@ Audio, video, and text renditions share the following fields, extending the WebC ~~~ type CommonExtensions = { "broadcast": string | undefined, + "epoch": string | undefined, "track": string | undefined, "label": string | undefined, "container": Container, @@ -670,6 +679,27 @@ An absent MSF `namespace` inherits the catalog's broadcast. Since each endpoint sees paths under its own connection root, a `namespace` equal to the catalog broadcast's path, or matching it on whole trailing segments, also names the catalog's broadcast; a gateway MUST NOT emit a different broadcast's `namespace` that matches so. A gateway MUST preserve the namespace and track-name pair or refuse the rendition; it MUST NOT substitute the catalog's broadcast for an unrepresentable reference. +### epoch {#field-epoch} +The optional `epoch` field names the publisher instance of the broadcast a rendition references ({{field-broadcast}}), as the canonical text of its Epoch {{moql}}: a lowercase hyphenated UUIDv7. +Without `broadcast`, it names the instance of the catalog's own broadcast. +A consumer MUST reject a catalog containing any other `epoch` value. + +A consumer reads each path a catalog instance references from the first broadcast it resolves there, so the catalog and its renditions never resolve one path independently across a replacement. +An absent `epoch` accepts that first broadcast, whichever instance it is. +A consumer that follows a referenced broadcast's announcements MAY move a rendition without `epoch` onto a restarted instance. + +When `epoch` is present and the route that resolved the referenced broadcast carries an Epoch, the consumer MUST read the rendition only from that instance. +It MUST fail the rendition rather than fall back to another instance, including one that replaced it before or after resolution, and MUST NOT follow a restart onto another instance for it. +A later catalog update naming another Epoch for an already-resolved path MUST NOT replace the resolved broadcast; the renditions naming the other Epoch fail instead. +A new catalog instance resolves its references afresh. + +A route without an Epoch, such as one over an older moq-lite version or MoQ Transport without the epoch extension {{moqepoch}}, cannot prove which instance it serves, so there `epoch` is best effort: the consumer reads the broadcast it resolved. +A consumer MUST NOT treat a catalog's `epoch` as evidence that a subscription may resume across routes. +A consumer that predates the field ignores it and does not gain its guarantee. + +A publisher SHOULD set `epoch` only when it knows the instance it references, and MUST omit it for a broadcast produced on demand, which has no instance until a worker accepts the request. +MSF {{moqmsf}} has no field for an instance, so a gateway converting to MSF drops `epoch` and SHOULD report that it did; its readers accept the first instance they resolve. + ### track {#field-track} A rendition's key in its map is its ID, and by default also the name of its track. The `track` field overrides the name, so that the ID can stay unique when the name cannot: two renditions referencing same-named tracks in different broadcasts ({{field-broadcast}}) need different IDs. @@ -925,6 +955,7 @@ type ArchiveSchema = { }>, "timescale": number | undefined, "replay": string | undefined, + "replayEpoch": string | undefined, "store": string | undefined, "version": number | undefined, } @@ -955,6 +986,8 @@ The `replay` field, if present, is a relative MoQ broadcast path ({{field-broadc Absent, the timelines live on the catalog's own broadcast. A wildcard replay path names no generation. +The `replayEpoch` field, if present, names the publisher instance of `replay`, with the meaning and rules of a rendition's `epoch` ({{field-epoch}}). + The `store` field, if present, is the object-store URL the recording objects ({{recording}}) live under. Authorization for `replay` and `store` is external. @@ -1360,6 +1393,7 @@ This document has no IANA actions. - A data track marks a discontinuity with a group of one object with no bytes, since its empty payload is data. An object with no bytes on an audio or video track, or sharing its group, is malformed. - Each archive timeline declares its own fixed nominal record duration, omitted for an untimed track; removed the broadcast-wide duration bound. - Added the optional video `preference` field: a consumer selects automatically among the highest preference it can decode. +- Added an optional `epoch` beside every rendition and data track `broadcast` reference, and `replayEpoch` beside the archive's `replay` ({{field-epoch}}). A consumer reads each referenced path from the first broadcast it resolves for a catalog instance; a present `epoch` fails a rendition proven to resolve another instance, and is best effort over a route without an Epoch. - Derived HLS and DASH exports hand disabled references to enabled renditions at the first record numbered past the advertised ones, keeping the record `sequence` as the HLS media sequence and starting a DASH period. - Added optional `stats` and `echo` catalog sections, publisher/viewer snapshots keyed by rendition ID, and the `.echo` feedback broadcast convention. diff --git a/js/hang/src/catalog/archive.ts b/js/hang/src/catalog/archive.ts index bfb51d23663..7a57a968d53 100644 --- a/js/hang/src/catalog/archive.ts +++ b/js/hang/src/catalog/archive.ts @@ -1,6 +1,6 @@ import * as z from "@zod/mini"; import { nonzeroU53Schema, u53, u53Schema } from "./integers"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; /** The moq epoch (2020-01-01T00:00:00Z) in Unix-epoch milliseconds. */ export const MOQ_EPOCH_UNIX_MILLIS = 1_577_836_800_000; @@ -40,6 +40,10 @@ export const ArchiveSchema = z.object({ // generation: compare this path and `store` to tell recordings apart. replay: z.optional(RelativeBroadcastSchema), + // The publisher instance `replay` names, when known. Absent accepts whichever instance + // resolves first, as for a rendition's `epoch`. + replayEpoch: z.optional(EpochSchema), + // The object-store URL the recording objects live under, if the publisher exposes one. store: z.optional(z.url()), diff --git a/js/hang/src/catalog/audio.ts b/js/hang/src/catalog/audio.ts index 4c7451700bb..7ac80d5f04b 100644 --- a/js/hang/src/catalog/audio.ts +++ b/js/hang/src/catalog/audio.ts @@ -3,7 +3,7 @@ import { compareName, compareNumber } from "./compare"; import { ContainerSchema } from "./container"; import { hexSchema } from "./hex"; import { u53Schema } from "./integers"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; // Backwards compatibility: old track schema const TrackSchema = z.object({ @@ -15,6 +15,11 @@ const AudioConfigFields = z.object({ // relative to the broadcast that served this catalog (e.g. "./source"). // If unset, the track lives in the same broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/hang/src/catalog/binary.ts b/js/hang/src/catalog/binary.ts index e49191df24b..be5e1a82049 100644 --- a/js/hang/src/catalog/binary.ts +++ b/js/hang/src/catalog/binary.ts @@ -2,7 +2,7 @@ import * as z from "zod/mini"; import { CompressionSchema } from "./compression"; import { u53Schema } from "./integers"; import { ModeSchema } from "./mode"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; /** * Schema for a single binary track: application data published as opaque payloads. @@ -19,6 +19,11 @@ export const BinaryConfigSchema = z.looseObject({ // relative to the broadcast that served this catalog (e.g. "./source"). // If unset, the track lives in the same broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/hang/src/catalog/json.ts b/js/hang/src/catalog/json.ts index 3e5222980c1..0d5fb05d721 100644 --- a/js/hang/src/catalog/json.ts +++ b/js/hang/src/catalog/json.ts @@ -2,7 +2,7 @@ import * as z from "zod/mini"; import { CompressionSchema } from "./compression"; import { u53Schema } from "./integers"; import { ModeSchema } from "./mode"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; /** * Schema for a single JSON track: application data published as a live JSON document or log. @@ -18,6 +18,11 @@ export const JsonConfigSchema = z.looseObject({ // relative to the broadcast that served this catalog (e.g. "./source"). // If unset, the track lives in the same broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/hang/src/catalog/path.ts b/js/hang/src/catalog/path.ts index 05c6fc7d71e..c88194dbe84 100644 --- a/js/hang/src/catalog/path.ts +++ b/js/hang/src/catalog/path.ts @@ -1,4 +1,4 @@ -import { Path } from "@moq/net"; +import { Epoch, Path } from "@moq/net"; import * as z from "@zod/mini"; /** @@ -18,3 +18,28 @@ export const RelativeBroadcastSchema: z.ZodMiniType = * out here so the catalog schemas' inferred types stay nameable from this package. */ export type RelativeBroadcast = string & { __brand: "Relative" }; + +function isEpoch(text: string): boolean { + try { + Epoch.parse(text); + return true; + } catch { + return false; + } +} + +/** + * Zod schema for the publisher instance a catalog reference names: an entry's `epoch` beside its + * `broadcast`, or the archive's `replayEpoch`. A lowercase hyphenated UUIDv7, the same text MoQ + * routes carry; anything else refuses the catalog. + */ +export const EpochSchema: z.ZodMiniType = z.pipe( + z.string().check(z.refine(isEpoch, "expected a lowercase hyphenated UUIDv7")), + z.transform((text) => text as ReferenceEpoch), +); + +/** + * A validated reference epoch: the same brand as `Epoch.Valid` from `@moq/net`, spelled out here + * so the catalog schemas' inferred types stay nameable from this package. + */ +export type ReferenceEpoch = string & { readonly _brand: "epoch" }; diff --git a/js/hang/src/catalog/root.test.ts b/js/hang/src/catalog/root.test.ts index e1fcd4560d0..0d906ab80f0 100644 --- a/js/hang/src/catalog/root.test.ts +++ b/js/hang/src/catalog/root.test.ts @@ -2,7 +2,7 @@ import { expect, test } from "bun:test"; import * as z from "@zod/mini"; import { ARCHIVE_VERSION, ArchiveSchema } from "./archive.ts"; import { u53 } from "./integers.ts"; -import type { RelativeBroadcast } from "./path.ts"; +import type { ReferenceEpoch, RelativeBroadcast } from "./path.ts"; import { checkRenditions, DuplicateRendition, minVersion, RootSchema, UnsupportedVersion, VERSION } from "./root.ts"; import { track } from "./track.ts"; @@ -45,6 +45,49 @@ test("rendition broadcast reference is parsed and normalized", () => { expect(parsed.video.renditions.video?.broadcast).toBe("source" as RelativeBroadcast); }); +// Mirrors Rust's `reference_epochs_roundtrip_in_every_section`. +test("every reference-bearing section carries an optional epoch", () => { + const EPOCH = "01890a5d-ac96-774b-bcce-b302099a8057" as ReferenceEpoch; + const catalog = (epoch: string) => ({ + video: { + renditions: { + v: { broadcast: "./a", epoch, codec: "avc1.64001f", container: { kind: "legacy" } }, + }, + }, + audio: { + renditions: { + a: { + broadcast: "./a", + epoch, + codec: "opus", + sampleRate: 48000, + numberOfChannels: 2, + container: { kind: "legacy" }, + }, + }, + }, + text: { renditions: { t: { broadcast: "./a", epoch, format: "vtt" } } }, + json: { tracks: { j: { broadcast: "./a", epoch, mode: "snapshot" } } }, + binary: { tracks: { b: { broadcast: "./a", epoch, mode: "snapshot" } } }, + archive: { timelines: {}, replay: "./a", replayEpoch: epoch }, + }); + + const parsed = RootSchema.parse(catalog(EPOCH)); + if (!parsed.video || !("renditions" in parsed.video)) throw new Error("missing video section"); + if (!parsed.audio || !("renditions" in parsed.audio)) throw new Error("missing audio section"); + expect(parsed.video.renditions.v?.epoch).toBe(EPOCH); + expect(parsed.audio.renditions.a?.epoch).toBe(EPOCH); + expect(parsed.text?.renditions.t?.epoch).toBe(EPOCH); + expect(parsed.json?.tracks.j?.epoch).toBe(EPOCH); + expect(parsed.binary?.tracks.b?.epoch).toBe(EPOCH); + expect(parsed.archive?.replayEpoch).toBe(EPOCH); + + // A malformed or noncanonical epoch refuses the catalog, as in Rust. + for (const bad of ["not-a-uuid", EPOCH.toUpperCase(), "01890a5d-ac96-474b-bcce-b302099a8057"]) { + expect(RootSchema.safeParse(catalog(bad)).success).toBe(false); + } +}); + test("rendition parent broadcast reference stays distinct from empty", () => { const catalog = { video: { diff --git a/js/hang/src/catalog/text.ts b/js/hang/src/catalog/text.ts index d72a756f902..87667a7f1ce 100644 --- a/js/hang/src/catalog/text.ts +++ b/js/hang/src/catalog/text.ts @@ -1,7 +1,7 @@ import * as z from "zod/mini"; import { ContainerSchema } from "./container"; import { u53Schema } from "./integers"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; /** * The cue serialization format of a text track, mirroring the Rust `TextFormat`. @@ -35,6 +35,11 @@ export const TextConfigSchema = z.object({ // broadcast that served this catalog (e.g. "../source"). If unset, the track lives in the same // broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/hang/src/catalog/video.ts b/js/hang/src/catalog/video.ts index f16f035cc99..1d4ffdd5e86 100644 --- a/js/hang/src/catalog/video.ts +++ b/js/hang/src/catalog/video.ts @@ -3,7 +3,7 @@ import { compareName, compareNumber } from "./compare"; import { ContainerSchema } from "./container"; import { hexSchema } from "./hex"; import { u8Schema, u53Schema } from "./integers"; -import { RelativeBroadcastSchema } from "./path"; +import { EpochSchema, RelativeBroadcastSchema } from "./path"; // Backwards compatibility: old track schema const TrackSchema = z.object({ @@ -89,6 +89,11 @@ export const VideoConfigSchema = z.object({ // relative to the broadcast that served this catalog (e.g. "./source"). // If unset, the track lives in the same broadcast as the catalog. broadcast: z.optional(RelativeBroadcastSchema), + // The publisher instance the reference names, when known. Absent accepts whichever instance + // resolves first. An epoch-aware route proving another instance fails the entry; an epochless + // route cannot prove one, so there it is best effort. Without `broadcast`, it names the + // catalog's own instance. + epoch: z.optional(EpochSchema), // The track's name in its broadcast, when it differs from this entry's ID (its key in the // catalog). Absent means the ID is the track name. Resolve it with `track`. track: z.optional(z.string()), diff --git a/js/watch/src/audio/config.ts b/js/watch/src/audio/config.ts index 27ab8465ef5..cd881a39cb9 100644 --- a/js/watch/src/audio/config.ts +++ b/js/watch/src/audio/config.ts @@ -19,6 +19,7 @@ export type DecoderConfig = Pick< /** The routing and decoder state whose changes require a replacement subscription. */ export type PlaybackIdentity = { broadcast: Catalog.AudioConfig["broadcast"]; + epoch: Catalog.AudioConfig["epoch"]; track: Catalog.AudioConfig["track"]; decoder: DecoderConfig; }; @@ -39,6 +40,7 @@ export function decoderConfig(config: Catalog.AudioConfig): DecoderConfig { export function playbackIdentity(config: Catalog.AudioConfig): PlaybackIdentity { return { broadcast: config.broadcast, + epoch: config.epoch, track: config.track, decoder: decoderConfig(config), }; diff --git a/js/watch/src/audio/decoder.ts b/js/watch/src/audio/decoder.ts index f4bcb37e4f4..80c334add2c 100644 --- a/js/watch/src/audio/decoder.ts +++ b/js/watch/src/audio/decoder.ts @@ -339,7 +339,7 @@ export class Decoder { // tell whether a failure's source is still selected. const active = broadcast && track && identity && (running || this.#failed) - ? broadcast.relativeBroadcast(effect, identity.broadcast) + ? broadcast.relativeBroadcast(effect, identity.broadcast, identity.epoch) : undefined; const key = identity && JSON.stringify(identity); diff --git a/js/watch/src/broadcast.test.ts b/js/watch/src/broadcast.test.ts index 1077c92a6e9..317d0b279dc 100644 --- a/js/watch/src/broadcast.test.ts +++ b/js/watch/src/broadcast.test.ts @@ -270,6 +270,91 @@ describe("relativeBroadcast", () => { } }); + it("reads a reference only from the epoch it names, best effort without one", async () => { + const owner = new Origin.Producer(); + const [catalogEpoch, siblingEpoch, other] = [Moq.Epoch.mint(), Moq.Epoch.mint(), Moq.Epoch.mint()]; + const catalog = owner.createBroadcast(Path.from("a/b")); + catalog.announce({ epoch: catalogEpoch }); + const sibling = owner.createBroadcast(Path.from("a/source")); + sibling.announce({ epoch: siblingEpoch }); + const plain = publish(owner, Path.from("a/plain")); + const source = new Broadcast({ + origin: owner, + name: Path.from("a/b"), + enabled: true, + announced: false, + catalogFormat: "manual", + }); + const effect = new Effect(); + try { + await settle(); + const rel = Path.normalizeRelative("./source"); + expect(source.relativeBroadcast(effect, rel)?.epoch).toBe(siblingEpoch); + expect(source.relativeBroadcast(effect, rel, siblingEpoch)?.epoch).toBe(siblingEpoch); + const own = source.out.active.peek(); + expect(own?.epoch).toBe(catalogEpoch); + expect(source.relativeBroadcast(effect, undefined, catalogEpoch)).toBe(own); + withoutWarnings(() => { + expect(source.relativeBroadcast(effect, rel, other)).toBeUndefined(); + expect(source.relativeBroadcast(effect, undefined, other)).toBeUndefined(); + expect(source.relativeBroadcast(effect, Path.normalizeRelative("./b"), other)).toBeUndefined(); + }); + // An epochless route cannot prove its instance, so the epoch cannot refuse it. + expect(source.relativeBroadcast(effect, Path.normalizeRelative("./plain"), other)).toBeDefined(); + } finally { + effect.close(); + source.close(); + plain.close(); + sibling.close(); + catalog.close(); + owner.close(); + } + }); + + it("follows a replaced sibling only for a reference that names no epoch", async () => { + const owner = new Origin.Producer(); + const main = publish(owner, Path.from("a/b")); + const [oldEpoch, newEpoch] = [Moq.Epoch.mint(), Moq.Epoch.mint()]; + const old = owner.createBroadcast(Path.from("a/source")); + old.announce({ epoch: oldEpoch }); + const source = new Broadcast({ + origin: owner, + name: Path.from("a/b"), + enabled: true, + catalogFormat: "manual", + }); + const effect = new Effect(); + const rel = Path.normalizeRelative("./source"); + let following: Moq.Broadcast.Consumer | undefined; + let pinned: Moq.Broadcast.Consumer | undefined; + effect.run((nested) => { + following = source.relativeBroadcast(nested, rel); + }); + effect.run((nested) => { + pinned = withoutWarnings(() => source.relativeBroadcast(nested, rel, oldEpoch)); + }); + try { + await settle(); + expect(following?.epoch).toBe(oldEpoch); + expect(pinned?.epoch).toBe(oldEpoch); + + // The announcement is the online signal: the plain reference follows the restart, + // while one naming the old instance never reads the new one. + const replacement = owner.createBroadcast(Path.from("a/source")); + replacement.announce({ epoch: newEpoch }); + await settle(); + expect(following?.epoch).toBe(newEpoch); + expect(pinned).toBeUndefined(); + replacement.close(); + } finally { + effect.close(); + source.close(); + old.close(); + main.close(); + owner.close(); + } + }); + it("uses the catalog's own broadcast when the reference is absent, empty, or self", async () => { const { source, owner } = broadcast("a/b"); const effect = new Effect(); diff --git a/js/watch/src/broadcast.ts b/js/watch/src/broadcast.ts index cc22327e3f4..7e76e00741f 100644 --- a/js/watch/src/broadcast.ts +++ b/js/watch/src/broadcast.ts @@ -423,8 +423,23 @@ export class Broadcast { * The consumer is scoped to the caller's `effect` (closed on its next run), so a * reference resolves lazily and reacts to `enabled` / connection / announcement * changes exactly like the catalog broadcast. + * + * `epoch` is the rendition's `epoch` field, the publisher instance the reference names. + * Returns `undefined` while the followed broadcast is proven to be another instance, rather + * than reading it. A route without an epoch cannot prove one, so there it is best effort. */ - relativeBroadcast(effect: Effect, rel: Path.Relative | undefined): Moq.Broadcast.Consumer | undefined { + relativeBroadcast( + effect: Effect, + rel: Path.Relative | undefined, + epoch?: Moq.Epoch.Valid, + ): Moq.Broadcast.Consumer | undefined { + const active = this.#relativeActive(effect, rel); + if (!active || epoch === undefined || active.epoch === undefined || active.epoch === epoch) return active; + console.warn("ignoring rendition: its broadcast serves another epoch", active.path, active.epoch, epoch); + return undefined; + } + + #relativeActive(effect: Effect, rel: Path.Relative | undefined): Moq.Broadcast.Consumer | undefined { const target = this.#relativeTarget(effect, rel); if (!target) return undefined; if (target.local) return effect.get(this.out.active); diff --git a/js/watch/src/text/renderer.ts b/js/watch/src/text/renderer.ts index 78a123f5437..49bcc7b66d2 100644 --- a/js/watch/src/text/renderer.ts +++ b/js/watch/src/text/renderer.ts @@ -213,7 +213,7 @@ export class Renderer { const [_enabled, container, track, config, broadcast] = values; // Honor a per-rendition `broadcast` override: subscribe on the resolved source broadcast. - const active = broadcast.relativeBroadcast(effect, config.broadcast); + const active = broadcast.relativeBroadcast(effect, config.broadcast, config.epoch); if (!active) return; // A new track gets its own chance to complain about its clock. diff --git a/js/watch/src/video/config.ts b/js/watch/src/video/config.ts index a88782c5c83..1dedaa75ab8 100644 --- a/js/watch/src/video/config.ts +++ b/js/watch/src/video/config.ts @@ -14,6 +14,7 @@ export type DecoderConfig = Pick< /** The routing and decoder state whose changes require a new playback pipeline. */ export type PlaybackIdentity = { broadcast: Catalog.VideoConfig["broadcast"]; + epoch: Catalog.VideoConfig["epoch"]; track: Catalog.VideoConfig["track"]; decoder: DecoderConfig; }; @@ -22,6 +23,7 @@ export type PlaybackIdentity = { export function playbackIdentity(config: Catalog.VideoConfig): PlaybackIdentity { return { broadcast: config.broadcast, + epoch: config.epoch, track: config.track, decoder: { codec: config.codec, diff --git a/js/watch/src/video/decoder.ts b/js/watch/src/video/decoder.ts index 5ab876c31a4..96909001632 100644 --- a/js/watch/src/video/decoder.ts +++ b/js/watch/src/video/decoder.ts @@ -188,7 +188,7 @@ export class Decoder { // tell whether a failure's source is still selected. const active: Moq.Broadcast.Consumer | undefined = broadcast && track && identity && (enabled || this.#failed) - ? broadcast.relativeBroadcast(effect, identity.broadcast) + ? broadcast.relativeBroadcast(effect, identity.broadcast, identity.epoch) : undefined; const key = identity && JSON.stringify(identity); diff --git a/quest/m0/broadcast-epoch/README.md b/quest/m0/broadcast-epoch/README.md index 8e7a412e0f5..396e06f4d22 100644 --- a/quest/m0/broadcast-epoch/README.md +++ b/quest/m0/broadcast-epoch/README.md @@ -72,8 +72,9 @@ Decided: for the catalog instance's lifetime. Optional reference epochs constrain resolution on epoch-aware transports and remain best effort on epochless transports. Following never turns a replacement into seamless resume. The - [catalog references quest](/quest/m0/broadcast-epoch/catalog-references.md) - owns the schemas, readers, and compatibility documentation. + schemas, shared resolution, and compatibility documentation landed; + [reference announcements](/quest/m0/broadcast-epoch/reference-announcements.md) + owns the native readers that follow announcements. - Every first-party publisher that can restart mints its own: the apps, moq-boy, the ingest gateways, moqsink, OBS, and the bindings. Players (`moq play`, `@moq/watch`, demo/web) follow the announce `Restart`. moq-stats mints one per @@ -125,7 +126,7 @@ End+Start. The child quests own their focused regressions. ## Required -- [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - announcement followers restart replaced siblings; other readers retain pins, with optional epochs and legacy best effort +- [Reference announcements](/quest/m0/broadcast-epoch/reference-announcements.md) - native readers restart a replaced referenced sibling on its announcements, never past a named epoch ## Related diff --git a/quest/m0/broadcast-epoch/catalog-references.md b/quest/m0/broadcast-epoch/catalog-references.md deleted file mode 100644 index a62a6081e51..00000000000 --- a/quest/m0/broadcast-epoch/catalog-references.md +++ /dev/null @@ -1,94 +0,0 @@ -# [L] Catalog references pin their broadcast instances - -## Goal - -Catalog, media, and data requests use the same broadcast identity in players -and exporters. Announcement-following readers follow a referenced sibling's -Start, End, and Restart; readers without announcement following pin their -first resolution for the catalog's lifetime. Optional reference epochs -constrain the intended identity in either case. - -## Plan - -Decided in the 2026-10-10 resume audit: - -- Apply the same identity policy to players and exporters. Per the - 2026-10-14 decision below, announcement-following readers handle a sibling - replacement as Restart, never as seamless same-epoch resume. Readers that - follow no announcements keep their first pin until a new catalog instance - or explicit export stitch. Same-epoch recovery preserves identity. -- Add optional `epoch` beside every catalog broadcast reference: video, - audio, text, JSON, and binary configs, plus the archive replay reference. - Reuse the canonical UUIDv7 representation and validation from MoQ epochs. - There is no new top-level catalog epoch. Self-references use the already - pinned catalog broadcast; an explicit value must not contradict its - known identity. -- An explicit reference epoch constrains the initial resolution and later - requests wherever the transport can express and verify it. A known - mismatch fails instead of falling back to the current winner. Conflicting - identities for an already-pinned path cannot replace that pin through a - catalog update. -- Lazy work has no anticipated epoch: leave it out of unresolved contribution - references. Once a worker accepts a job it advertises a concrete path with - a unique epoch. A proven Takeover preserves the same pinned instance; a - replacement follows Restart for announcement-following readers and requires - a new catalog instance for other readers. An epoch-bound request on an - epoch-capable link requires a matching advertised epoch, not merely an - epoch returned from an epochless claim. -- Omission remains compatible with existing catalogs: bind the first - resolved instance, then follow announcements only where the reader opts - into that lifecycle. Make no promise about an unresolved sibling's past. -- On epochless transports, explicit catalog epochs are best effort - (maintainer choice). Keep the requested identity in the reference, use - whatever identity the route can verify, and pin the resolved handle. - Unknown identity alone is not a refusal. Document that this cannot prove - the requested epoch, and never promote an unverified catalog value into - transport evidence permitting seamless resume. Older readers can ignore - the additive field and do not gain its guarantee. - -Decided by the maintainer, 2026-10-14 (from #5500's Codex review): readers -follow a referenced broadcast's announcements, since announcement is the -online/offline signal. The "no retarget" rule applies only to readers that -follow no announcement. An explicit epoch still constrains every resolution; -following does not authorize substituting a different requested epoch. - -- `moqsrc`: Start binds the rendition, End idles its pad, and Restart - restarts its pump. -- Exporters: a late first announcement starts the rendition and End ends - it. Restart behavior belongs to - [sink restart follow](/quest/m1/sink-restart-follow.md). -- Share this in `moq_mux` (a `Source::follow`-style API; the shape is open) - rather than per reader. `Source::bind` binds once today, as - `failed_binding_does_not_retry_a_later_publisher` asserts; that changes. - -Use the shared reference resolution paths rather than adding one rule per -media type. Mirror schemas and behavior in `rs/hang`, `js/hang`, native -readers/exporters and `@moq/watch`; carry the field through catalog producers, -conversions and bindings that expose these configs. A conversion that -cannot preserve the requested identity must report that limitation rather -than silently discard it. Coordinate with catalog rendition IDs, which -changes the neighboring track-name field but is not a prerequisite. - -Test all reference-bearing sections in both languages: omitted/matching/ -mismatched/malformed epochs, same and sibling paths, a replacement before -first resolution and after pinning, late rendition additions, updates that -conflict with a pin, new catalog instances, and an epochless transport with -an explicit reference. Catalog and media must never independently resolve -the same path across a replacement. Include archive replay references and -concurrent old/new consumers. Use existing unit, media and interop CI lanes; -run `just check`, `just test interop --all`, and `just drafts check`. - -Public API: additive optional catalog fields and changed player sibling -following behavior. Wire: additive Hang catalog fields, no transport -framing change. Update the Hang draft, existing catalog/identity docs, and -upgrade notes inline. The [identity guide](/quest/m1/identity-guide.md) -consolidates the behavior separately. Explain both the strong epoch-aware -guarantee and the selected legacy best-effort behavior. - -## Related - -- [Source pin](https://github.com/moq-dev/moq/pull/5252) - exporters share their resolved catalog handle -- [JS consume identity](https://github.com/moq-dev/moq/pull/5254) - JS resolves the actual serving identity -- [Retired requests](https://github.com/moq-dev/moq/pull/5294) - held handles cannot emit unpinned requests after retirement -- [Rust identity enforcement](/quest/m1/catalog-identity-rs.md) - the same for the Rust publishers -- [MoQ Epoch](/drafts/draft-lcurley-moq-epoch.md) - verified catalog identity over negotiated IETF sessions diff --git a/quest/m0/broadcast-epoch/reference-announcements.md b/quest/m0/broadcast-epoch/reference-announcements.md new file mode 100644 index 00000000000..cfde1ceeef0 --- /dev/null +++ b/quest/m0/broadcast-epoch/reference-announcements.md @@ -0,0 +1,48 @@ +# [L] Rust readers follow referenced broadcasts' announcements + +## Goal + +Native readers follow a catalog-referenced sibling's announcements the way +`@moq/watch` already does, since announcement is the online/offline signal. +A reference that names an `epoch` never follows onto another instance. + +## Plan + +Split from the catalog references work (#5500's Codex review, decided by the +maintainer 2026-10-14). That work landed the optional reference `epoch`, its +shared resolution in `moq_mux::Source` and `@moq/watch`, and the Hang draft +text; this quest owns the following behavior it left out. + +- Share it in `moq_mux` (a `Source::follow`-style API; the shape is open) + rather than per reader, built on `origin::Consumer::announced_broadcast` + rather than another hand-rolled loop. Follow unpinned and check each + handed-out instance's epoch, as `Source::bind` does: a pinned + `announced_broadcast` waits out an epochless route, which would break the + best effort the reference epoch promises there. `Source::bind` binds once + today, as `failed_binding_does_not_retry_a_later_publisher` asserts; that + changes. The "no retarget" rule applies only to readers that follow no + announcement. +- `moqsrc`: Start binds the rendition, End idles its pad, and Restart + restarts its pump. +- Exporters: a late first announcement starts the rendition and End ends + it. Restart behavior belongs to + [sink restart follow](/quest/m1/sink-restart-follow.md). +- A replacement is a Restart for an announcement follower, never a seamless + same-epoch resume. A proven Takeover preserves the same instance. +- A reference with an `epoch` follows only its own instance: following never + substitutes a different epoch, so a Restart onto another one ends that + rendition, matching `relativeBroadcast` in + `@moq/watch` (`follows a replaced sibling only for a reference that names + no epoch`). +- `@moq/watch` returns nothing for a rendition whose followed broadcast proves + another epoch, but still lists it in `out.catalog`, so selection can pick a + rendition that never plays. Drop it from the effective catalog too. + +Test late first announcements, End then Start, Restart with and without an +epoch, and concurrent old/new consumers, in the moq-mux unit tests and the +`moqsrc` and exporter suites. + +## Related + +- [Sink restart follow](/quest/m1/sink-restart-follow.md) - exporters and sinks that follow an epochless Restart of their own broadcast +- [Export rendition rebind](/quest/m1/export-rendition-rebind.md) - exporters rebuild a rendition whose reference changes diff --git a/quest/m1/README.md b/quest/m1/README.md index 0609e7c04ca..20052e29c1a 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -79,6 +79,7 @@ lands before the cut. - [Announcement takeover](/quest/m1/announce-takeover.md) - a new independent route explicitly continues instances served through its predecessor, with shared Rust/JS semantics and lite-07 wire - [IETF takeover](/quest/m1/ietf-takeover.md) - carry the shared takeover contract through the negotiated IETF epoch extension - [Identity and recovery guide](/quest/m1/identity-guide.md) - explain pinned catalogs, lazy jobs, takeover, restart and version compatibility +- [Reference epoch bindings](/quest/m1/reference-epoch-bindings.md) - the UniFFI bindings and C ABI expose and honor a catalog reference's optional `epoch` - [Generated C bindings](/quest/m1/c/README.md) - C generated from moq-ffi ships as `moq-c` 0.8.0 and replaces the hand-written libmoq - [Scope track priority](/quest/m1/track-priority-scope.md) - priority orders one owner's streams, and a shared cluster session is fair across tenants - [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the playout target is default in both languages; a manual browser proof and a native trace replay remain diff --git a/quest/m1/announce-takeover.md b/quest/m1/announce-takeover.md index df845df6c34..593b11e5366 100644 --- a/quest/m1/announce-takeover.md +++ b/quest/m1/announce-takeover.md @@ -94,6 +94,6 @@ ANNOUNCE_TAKEOVER; published versions retain their framing. - [Claim answer epochs](https://github.com/moq-dev/moq/pull/5678) - `Request::prepare` then `publish` answers a claim at the exact path under a fresh epoch, the operation TAKEOVER continues - [IETF takeover](/quest/m1/ietf-takeover.md) - negotiated IETF counterpart -- [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - a proven continuation preserves the same pin +- [Reference announcements](/quest/m0/broadcast-epoch/reference-announcements.md) - a proven continuation keeps a followed reference on its instance - [Retired requests](https://github.com/moq-dev/moq/pull/5294) - replacements never substitute content into held handles - [Identity and recovery guide](/quest/m1/identity-guide.md) - explains takeover alongside restart and legacy behavior diff --git a/quest/m1/identity-guide.md b/quest/m1/identity-guide.md index 03c83d358e9..27d5b876af0 100644 --- a/quest/m1/identity-guide.md +++ b/quest/m1/identity-guide.md @@ -40,5 +40,5 @@ Public API and wire: none; this is a user guide for existing feature owners. ## Required -- [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - establishes the shared pinning behavior +- [Reference announcements](/quest/m0/broadcast-epoch/reference-announcements.md) - settles how readers pin and follow catalog references - [Announcement takeover](/quest/m1/announce-takeover.md) - establishes the continuity operation the guide demonstrates diff --git a/quest/m1/reference-epoch-bindings.md b/quest/m1/reference-epoch-bindings.md new file mode 100644 index 00000000000..3e426407772 --- /dev/null +++ b/quest/m1/reference-epoch-bindings.md @@ -0,0 +1,30 @@ +# [M] Bindings carry catalog reference epochs + +## Goal + +The UniFFI bindings and the C ABI expose a catalog reference's optional +`epoch` and honor it when resolving, so a native app gets the same guarantee +as Rust and `@moq/watch`: a broadcast proven to serve another instance fails +the rendition instead of playing it. + +## Plan + +Today `MoqVideo` and `MoqAudio` drop the field, and `MoqBroadcastConsumer::resolve`, +`decode_video`, `decode_audio` and libmoq's rendition resolution pass `None` +to `moq_mux::Source::resolve`, so they read like a reader that predates the +field: the first instance resolved, without its guarantee. + +- Add `epoch` beside `broadcast` on the binding records, and pass it from + `decode_video` and `decode_audio`. +- Decide the `resolve` shape with the maintainer: a second optional argument, + or one reference record carrying `broadcast` and `epoch`. +- Carry it through the hand-written wrappers and docs in the Cross-Package + Sync table (`rs/moq-c`, `py`, `swift`, `kt`, `go/wrapper`, `dart`, `cpp`, + `doc/lib`). + +Test a matching, mismatched and epochless reference through `moq-ffi` and +libmoq, and run `just test interop --all`. + +## Related + +- [Rust readers follow references](/quest/m0/broadcast-epoch/reference-announcements.md) - the following behavior the bindings inherit from `moq_mux` diff --git a/quest/m1/sink-restart-follow.md b/quest/m1/sink-restart-follow.md index c3910d78ceb..cc29cd39c3c 100644 --- a/quest/m1/sink-restart-follow.md +++ b/quest/m1/sink-restart-follow.md @@ -29,7 +29,7 @@ and fix the rest the same way: on the new instance. Share one helper (a `moq_mux` `Source::follow`-style API per -[catalog references](/quest/m0/broadcast-epoch/catalog-references.md)) +[reference announcements](/quest/m0/broadcast-epoch/reference-announcements.md)) rather than one rule per sink, built on `origin::Consumer::announced_broadcast` rather than another hand-rolled loop. Each fix lands with a test that replaces the publisher while the sink runs. diff --git a/rs/hang/MIGRATE.md b/rs/hang/MIGRATE.md index ea49dc3e358..dde7dbb8fe0 100644 --- a/rs/hang/MIGRATE.md +++ b/rs/hang/MIGRATE.md @@ -4,6 +4,11 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `catalog::Reference` names its epoch (#5699) + +Every rendition and data track config gains an optional `epoch` beside `broadcast`, and `Archive` gains `replay_epoch` beside `replay`. +`Reference` has a new required `epoch()`: an implementor outside this crate returns the entry's epoch, or `None` when it names none. + ### A lost catalog track is `Error::Moq` (#6114) `catalog::Consumer::next` and `poll_next` return `Error::Moq` when the catalog track fails, instead of flattening it into `Error::Json`. diff --git a/rs/hang/src/catalog/archive.rs b/rs/hang/src/catalog/archive.rs index 0d83ea0e1e1..ffd463b5075 100644 --- a/rs/hang/src/catalog/archive.rs +++ b/rs/hang/src/catalog/archive.rs @@ -34,6 +34,7 @@ impl Timeline { /// A live publisher sets the timeline fields (`timelines`, `timescale`) alone, /// and a recording also names the replay broadcast and object store those spans live under. /// Every advertised span is FETCHable; with a store they are durable. +#[serde_with::serde_as] #[serde_with::skip_serializing_none] #[derive(Serialize, Deserialize, Debug, Clone, PartialEq)] #[serde(rename_all = "camelCase")] @@ -53,6 +54,12 @@ pub struct Archive { #[serde(default)] pub replay: Option, + /// The publisher instance [`replay`](Self::replay) names, when known. Absent accepts the + /// first one a reader resolves; see [`Reference::epoch`](super::Reference::epoch). + #[serde(default)] + #[serde_as(as = "Option")] + pub replay_epoch: Option, + /// The object-store URL the recording objects live under, if exposed by the publisher. #[serde(default)] pub store: Option, @@ -77,6 +84,7 @@ impl Archive { timelines: BTreeMap::new(), timescale: Self::default_timescale(), replay: None, + replay_epoch: None, store: None, version: None, } diff --git a/rs/hang/src/catalog/audio/mod.rs b/rs/hang/src/catalog/audio/mod.rs index d36c5491c91..96173a796c4 100644 --- a/rs/hang/src/catalog/audio/mod.rs +++ b/rs/hang/src/catalog/audio/mod.rs @@ -113,6 +113,16 @@ pub struct AudioConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// Human-readable rendition name for track pickers. #[serde(default)] pub label: Option, @@ -204,6 +214,7 @@ impl AudioConfig { Self { broadcast: None, track: None, + epoch: None, label: None, codec: codec.into(), sample_rate, diff --git a/rs/hang/src/catalog/binary.rs b/rs/hang/src/catalog/binary.rs index ebf746fe561..f0b08dab552 100644 --- a/rs/hang/src/catalog/binary.rs +++ b/rs/hang/src/catalog/binary.rs @@ -83,6 +83,16 @@ pub struct BinaryConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// Whether the track is a latest-value blob or an append log. Always stated: see [`Mode`]. #[serde_as(as = "serde_with::DisplayFromStr")] pub mode: Mode, @@ -131,6 +141,7 @@ impl BinaryConfig { Self { broadcast: None, track: None, + epoch: None, mode, compression: None, mime: None, diff --git a/rs/hang/src/catalog/json.rs b/rs/hang/src/catalog/json.rs index 08121614554..10478082256 100644 --- a/rs/hang/src/catalog/json.rs +++ b/rs/hang/src/catalog/json.rs @@ -81,6 +81,16 @@ pub struct JsonConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// Whether the track is a latest-value document or an append log. Always stated: see [`Mode`]. #[serde_as(as = "serde_with::DisplayFromStr")] pub mode: Mode, @@ -129,6 +139,7 @@ impl JsonConfig { Self { broadcast: None, track: None, + epoch: None, mode, compression: None, schema: None, diff --git a/rs/hang/src/catalog/reference.rs b/rs/hang/src/catalog/reference.rs index f134a1addf4..40f982854c6 100644 --- a/rs/hang/src/catalog/reference.rs +++ b/rs/hang/src/catalog/reference.rs @@ -4,11 +4,15 @@ use super::{AudioConfig, BinaryConfig, JsonConfig, TextConfig, VideoConfig}; /// /// An entry's key in the catalog is its ID. The track it names is `track` when set and the ID /// otherwise, in `broadcast` when set and the catalog's own broadcast otherwise. Every reader -/// subscribes through [`track`](Self::track) so the two never diverge. +/// subscribes through [`track`](Self::track) so the two never diverge. [`epoch`](Self::epoch) +/// optionally names the publisher instance the broadcast must be. pub trait Reference { /// The broadcast publishing the track, relative to the catalog's; `None` is the catalog's own. fn broadcast(&self) -> Option<&moq_net::path::RelativeOwned>; + /// The publisher instance the broadcast must be; `None` accepts the first one resolved. + fn epoch(&self) -> Option<&moq_net::Epoch>; + /// The track's name in its broadcast, given this entry's ID. fn track<'a>(&'a self, id: &'a str) -> &'a str; } @@ -20,6 +24,10 @@ macro_rules! reference { self.broadcast.as_ref() } + fn epoch(&self) -> Option<&moq_net::Epoch> { + self.epoch.as_ref() + } + fn track<'a>(&'a self, id: &'a str) -> &'a str { self.track.as_deref().unwrap_or(id) } diff --git a/rs/hang/src/catalog/root.rs b/rs/hang/src/catalog/root.rs index 4fc866506aa..8a7a266a904 100644 --- a/rs/hang/src/catalog/root.rs +++ b/rs/hang/src/catalog/root.rs @@ -490,6 +490,49 @@ mod test { assert_eq!(Catalog::<()>::from_str(&catalog.to_json().unwrap()).unwrap(), catalog); } + /// Every reference-bearing section carries an optional epoch beside its `broadcast`. + #[test] + fn reference_epochs_roundtrip_in_every_section() { + use crate::catalog::Reference; + + const EPOCH: &str = "01890a5d-ac96-774b-bcce-b302099a8057"; + let encoded = format!( + r#"{{ + "video": {{"renditions": {{"v": {{"broadcast": "./a", "epoch": "{EPOCH}", "codec": "avc1.64001f", "container": {{"kind": "legacy"}}}}}}}}, + "audio": {{"renditions": {{"a": {{"broadcast": "./a", "epoch": "{EPOCH}", "codec": "opus", "sampleRate": 48000, "numberOfChannels": 2, "container": {{"kind": "legacy"}}}}}}}}, + "text": {{"renditions": {{"t": {{"broadcast": "./a", "epoch": "{EPOCH}", "format": "vtt"}}}}}}, + "json": {{"tracks": {{"j": {{"broadcast": "./a", "epoch": "{EPOCH}", "mode": "snapshot"}}}}}}, + "binary": {{"tracks": {{"b": {{"broadcast": "./a", "epoch": "{EPOCH}", "mode": "snapshot"}}}}}}, + "archive": {{"timelines": {{}}, "replay": "./a", "replayEpoch": "{EPOCH}"}} + }}"# + ); + let catalog = Catalog::<()>::from_str(&encoded).unwrap(); + let epoch: moq_net::Epoch = EPOCH.parse().unwrap(); + let references: [&dyn Reference; 5] = [ + &catalog.video.renditions["v"], + &catalog.audio.renditions["a"], + &catalog.text.renditions["t"], + &catalog.json.tracks["j"], + &catalog.binary.tracks["b"], + ]; + for reference in references { + assert_eq!(reference.epoch(), Some(&epoch)); + } + assert_eq!(catalog.archive.as_ref().unwrap().replay_epoch, Some(epoch)); + assert_eq!(Catalog::<()>::from_str(&catalog.to_json().unwrap()).unwrap(), catalog); + + // Omitted stays omitted, and a malformed or noncanonical epoch refuses the catalog. + assert!(!Catalog::<()>::default().to_json().unwrap().contains("poch")); + for bad in [ + "not-a-uuid", + "01890A5D-AC96-774B-BCCE-B302099A8057", + "01890a5d-ac96-474b-bcce-b302099a8057", + ] { + let encoded = encoded.replace(EPOCH, bad); + assert!(Catalog::<()>::from_str(&encoded).is_err(), "{bad} must be refused"); + } + } + #[test] fn cross_kind_duplicate_id_is_refused() { let catalog = Catalog::<()>::from_str( @@ -632,6 +675,7 @@ mod test { VideoConfig { broadcast: None, track: None, + epoch: None, label: None, codec: H264 { profile: 0x64, @@ -666,6 +710,7 @@ mod test { AudioConfig { broadcast: None, track: None, + epoch: None, label: None, codec: Opus, sample_rate: 48_000, diff --git a/rs/hang/src/catalog/text/mod.rs b/rs/hang/src/catalog/text/mod.rs index 4976a6fa3b3..fb0de18a2be 100644 --- a/rs/hang/src/catalog/text/mod.rs +++ b/rs/hang/src/catalog/text/mod.rs @@ -89,6 +89,16 @@ pub struct TextConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// The serialization format of each cue payload. #[serde_as(as = "DisplayFromStr")] pub format: TextFormat, @@ -141,6 +151,7 @@ impl TextConfig { Self { broadcast: None, track: None, + epoch: None, format, role: TextRole::default(), lang: None, diff --git a/rs/hang/src/catalog/video/mod.rs b/rs/hang/src/catalog/video/mod.rs index 1a4feba1fe9..2d005d4af3e 100644 --- a/rs/hang/src/catalog/video/mod.rs +++ b/rs/hang/src/catalog/video/mod.rs @@ -200,6 +200,16 @@ pub struct VideoConfig { #[serde(default)] pub track: Option, + /// The publisher instance the reference names, when the catalog knows it. Absent names + /// whichever instance the reader resolves first, as in every catalog before this field. + /// + /// A reader on an epoch-aware route refuses another instance rather than fall back to it; + /// on an epochless route it is best effort, since the route cannot prove the instance. + /// Without [`broadcast`](Self::broadcast), it names the catalog's own instance. + #[serde(default)] + #[serde_as(as = "Option")] + pub epoch: Option, + /// Human-readable rendition name for track pickers. #[serde(default)] pub label: Option, @@ -339,6 +349,7 @@ impl VideoConfig { Self { broadcast: None, track: None, + epoch: None, label: None, codec: codec.into(), description: None, diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index 0e18ac0bcc3..8e46e7f44c5 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -124,9 +124,11 @@ async fn resolve( let origin = origin.ok_or_else(|| Error::UnresolvableBroadcast(reference.as_str().to_string()))?; let source = moq_mux::Source::new(origin.inner, broadcast); + // The C ABI does not expose a reference's epoch yet, so it reads like an older reader: the + // first instance resolved, without the epoch's guarantee. origin .driver - .watch(async { Ok(source.resolve(Some(&reference.borrow())).await?) }) + .watch(async { Ok(source.resolve(Some(&reference.borrow()), None).await?) }) .await } diff --git a/rs/moq-cli/src/play/media.rs b/rs/moq-cli/src/play/media.rs index f83c1e195ea..3c8bb8fae94 100644 --- a/rs/moq-cli/src/play/media.rs +++ b/rs/moq-cli/src/play/media.rs @@ -237,7 +237,7 @@ impl Media { // A rendition pointing at a broadcast we can't reach is that // rendition's problem, not the catalog's: fall through to the // next one like an unsupported codec does. - let rendition = match source.resolve(config.broadcast.as_ref()).await { + let rendition = match source.resolve(config.broadcast.as_ref(), config.epoch.as_ref()).await { Ok(rendition) => rendition, Err(err) => { tracing::warn!(track = name, %err, "cannot resolve video rendition"); @@ -304,7 +304,7 @@ impl Media { if playback.finished(&reference) { continue; } - let rendition = match source.resolve(config.broadcast.as_ref()).await { + let rendition = match source.resolve(config.broadcast.as_ref(), config.epoch.as_ref()).await { Ok(rendition) => rendition, Err(err) => { tracing::warn!(track = name, %err, "cannot resolve audio rendition"); diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index 40be2cadba4..1265eb12ab4 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -225,7 +225,9 @@ impl MoqBroadcastConsumer { })?; let source = moq_mux::Source::new(origin, self.inner.clone()); - Ok(source.resolve(Some(&reference)).await?) + // The bindings do not expose a reference's epoch yet, so they read like an older + // reader: the first instance resolved, without the epoch's guarantee. + Ok(source.resolve(Some(&reference), None).await?) }) .await } diff --git a/rs/moq-gst/src/source/imp.rs b/rs/moq-gst/src/source/imp.rs index 122f7e051fd..03a86098da3 100644 --- a/rs/moq-gst/src/source/imp.rs +++ b/rs/moq-gst/src/source/imp.rs @@ -404,6 +404,7 @@ struct Shape { container: hang::catalog::Container, track: String, broadcast: Option, + epoch: Option, } /// A pump's progress, shared with its [`ActiveTrack`] so teardown and pad creation can't both @@ -880,6 +881,7 @@ fn reconcile( container: container.clone(), track: reference.track(name).to_string(), broadcast: reference.broadcast().cloned(), + epoch: reference.epoch().cloned(), }; desired.insert(name.clone(), Desired { kind, shape }); } @@ -982,7 +984,7 @@ fn reconcile( // Bind without waiting: an unavailable sibling or track must not park the catalog loop // and hold up every other rendition. The pump resolves both under its cancellation scope. - let binding = match source.bind(d.shape.broadcast.as_ref()) { + let binding = match source.bind(d.shape.broadcast.as_ref(), d.shape.epoch.as_ref()) { Ok(binding) => binding, Err(err) => { gst::warning!(CAT, "ignoring rendition {name}: {err:?}"); diff --git a/rs/moq-hls/src/export/rendition.rs b/rs/moq-hls/src/export/rendition.rs index 5ae3016b491..71a84778d31 100644 --- a/rs/moq-hls/src/export/rendition.rs +++ b/rs/moq-hls/src/export/rendition.rs @@ -113,9 +113,9 @@ enum Config { struct Media { handle: Mutex, track: String, - /// The source and reference a sibling is bound through, to re-issue a bind that failed - /// before it ever resolved. - sibling: Option<(moq_mux::Source, moq_net::path::RelativeOwned)>, + /// The source, reference, and epoch a sibling is bound through, to re-issue a bind that + /// failed before it ever resolved. + sibling: Option<(moq_mux::Source, moq_net::path::RelativeOwned, Option)>, } struct Handle { @@ -139,13 +139,10 @@ impl Handle { } impl Media { - fn bind( - source: &moq_mux::Source, - rel: Option<&moq_net::path::RelativeOwned>, - track: &str, - ) -> moq_mux::Result { + fn bind(source: &moq_mux::Source, reference: &impl hang::catalog::Reference, track: &str) -> moq_mux::Result { + let rel = reference.broadcast(); let mut handle = Handle { - binding: Arc::new(source.bind(rel)?), + binding: Arc::new(source.bind(rel, reference.epoch())?), track: None, ended: false, }; @@ -155,7 +152,7 @@ impl Media { Ok(Self { handle: Mutex::new(handle), track: track.to_string(), - sibling: sibling(source, rel), + sibling: sibling(source, rel).map(|rel| (source.clone(), rel, reference.epoch().cloned())), }) } @@ -189,8 +186,8 @@ impl Media { // successful result, so a replacement can never answer these old rows. Poll::Ready(Err(_)) if !reissued => { reissued = true; - if let Some((source, rel)) = &self.sibling - && let Ok(next) = source.bind(Some(rel)) + if let Some((source, rel, epoch)) = &self.sibling + && let Ok(next) = source.bind(Some(rel), epoch.as_ref()) { handle.binding = Arc::new(next); continue; @@ -220,15 +217,12 @@ impl Media { fn sibling( source: &moq_mux::Source, rel: Option<&moq_net::path::RelativeOwned>, -) -> Option<(moq_mux::Source, moq_net::path::RelativeOwned)> { +) -> Option { let target = source.resolve_reference(rel)?; if source.resolve_reference(None).as_ref() == Some(&target) { return None; } - Some(( - source.clone(), - rel.cloned().unwrap_or_else(moq_net::path::Relative::empty), - )) + Some(rel.cloned().unwrap_or_else(moq_net::path::Relative::empty)) } /// Clear the video fields that don't change how the rendition decodes or is muxed, so a config @@ -382,7 +376,7 @@ impl Rendition { section: Archive, clock: Option, ) -> moq_mux::Result { - let media = Media::bind(source, config.broadcast.as_ref(), config.track(&name))?; + let media = Media::bind(source, config, config.track(&name))?; Ok(Self { name, kind: Kind::Video, @@ -415,7 +409,7 @@ impl Rendition { section: Archive, clock: Option, ) -> moq_mux::Result { - let media = Media::bind(source, config.broadcast.as_ref(), config.track(&name))?; + let media = Media::bind(source, config, config.track(&name))?; Ok(Self { name, kind: Kind::Audio, diff --git a/rs/moq-mux/MIGRATE.md b/rs/moq-mux/MIGRATE.md index 4f91eaa5d1a..0dcc889ce36 100644 --- a/rs/moq-mux/MIGRATE.md +++ b/rs/moq-mux/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `Source` resolution takes the reference's epoch (#5699) + +`Source::bind(rel)`, `resolve(rel)`, and `subscribe_track(rel, name)` take the catalog entry's optional epoch after `rel`. +Pass `config.epoch.as_ref()` (or `Reference::epoch`), or `None` to accept the first instance resolved as before. +A broadcast proven to serve another epoch fails with the new `Error::EpochMismatch`; over a route without an epoch the check is best effort. + ### `ts::stats::Export`'s release counters move into `release` (#5849) `dropped`, `trimmed`, `gaps`, `drift`, and `out_of_tolerance` are now fields of `Export::release`, a `ts::stats::Release`, which `ts::Release::stats` also returns on its own. diff --git a/rs/moq-mux/src/binary.rs b/rs/moq-mux/src/binary.rs index 236802643aa..62be7161fa9 100644 --- a/rs/moq-mux/src/binary.rs +++ b/rs/moq-mux/src/binary.rs @@ -396,7 +396,11 @@ impl crate::catalog::Entry<'_, BinaryConfig> { /// Errors if the entry declares a mode or compression this build doesn't implement. pub async fn subscribe(&self, source: &crate::Source) -> crate::Result { let track = source - .subscribe_track(self.config().broadcast.as_ref(), self.config().track(self.name())) + .subscribe_track( + self.config().broadcast.as_ref(), + self.config().epoch.as_ref(), + self.config().track(self.name()), + ) .await?; Consumer::from_track(track, self.config()) } diff --git a/rs/moq-mux/src/catalog/producer.rs b/rs/moq-mux/src/catalog/producer.rs index 65108d16292..01c286a7638 100644 --- a/rs/moq-mux/src/catalog/producer.rs +++ b/rs/moq-mux/src/catalog/producer.rs @@ -1306,6 +1306,10 @@ fn to_msf_media(catalog: &hang::Catalog, base: &moq_net::Path) -> let mut names = BTreeSet::new(); let mut reference = |id: &str, config: &dyn Reference| { let name = config.track(id); + if let Some(epoch) = config.epoch() { + // MSF has nowhere to name an instance, so its readers take whichever resolves first. + tracing::warn!(rendition = id, %epoch, "MSF cannot carry a reference epoch; advertising without it"); + } let namespace = match config.broadcast().filter(|rel| !rel.is_empty()) { None => None, Some(rel) => { diff --git a/rs/moq-mux/src/container/source.rs b/rs/moq-mux/src/container/source.rs index 7a77072a90b..0cf76f5315c 100644 --- a/rs/moq-mux/src/container/source.rs +++ b/rs/moq-mux/src/container/source.rs @@ -163,7 +163,7 @@ impl ExportSource { ) -> Result, crate::Error> { let media: HangContainer = config.try_into()?; let description = config.description.as_ref().filter(|b| !b.is_empty()).cloned(); - let Some(request) = source.try_request(config.broadcast.as_ref()) else { + let Some(request) = source.try_request(config) else { return Ok(None); }; @@ -201,7 +201,7 @@ impl ExportSource { ) -> Result, crate::Error> { let media: HangContainer = config.try_into()?; let description = config.description.as_ref().filter(|b| !b.is_empty()).cloned(); - let Some(request) = source.try_request(config.broadcast.as_ref()) else { + let Some(request) = source.try_request(config) else { return Ok(None); }; diff --git a/rs/moq-mux/src/error.rs b/rs/moq-mux/src/error.rs index 407bef3bcc2..3358abdf0a7 100644 --- a/rs/moq-mux/src/error.rs +++ b/rs/moq-mux/src/error.rs @@ -251,6 +251,20 @@ pub enum Error { #[error("broadcast reference escapes the root: {0}")] EscapingBroadcast(String), + /// A reference's `epoch` names another publisher instance than the one serving its broadcast. + /// + /// The served instance is the one the source resolved first for that path, so a later + /// reference cannot move it; the reference fails instead of reading other content. + #[error("broadcast {path} serves epoch {actual}, not the referenced {expected}")] + EpochMismatch { + /// The broadcast's path, relative to the origin root. + path: String, + /// The epoch the reference names. + expected: moq_net::Epoch, + /// The epoch the resolved broadcast serves. + actual: moq_net::Epoch, + }, + /// A rendition tried to lower jitter already advertised to subscribers. #[error("catalog jitter cannot decrease for a published rendition")] JitterDecreased, diff --git a/rs/moq-mux/src/json.rs b/rs/moq-mux/src/json.rs index 85b6381e9ab..be61879db91 100644 --- a/rs/moq-mux/src/json.rs +++ b/rs/moq-mux/src/json.rs @@ -451,7 +451,11 @@ impl crate::catalog::Entry<'_, JsonConfig> { /// Errors if the entry declares a mode or compression this build doesn't implement. pub async fn subscribe(&self, source: &crate::Source) -> crate::Result> { let track = source - .subscribe_track(self.config().broadcast.as_ref(), self.config().track(self.name())) + .subscribe_track( + self.config().broadcast.as_ref(), + self.config().epoch.as_ref(), + self.config().track(self.name()), + ) .await?; Consumer::from_track(track, self.config()) } diff --git a/rs/moq-mux/src/source.rs b/rs/moq-mux/src/source.rs index 5325f93b6e2..565911b29bf 100644 --- a/rs/moq-mux/src/source.rs +++ b/rs/moq-mux/src/source.rs @@ -29,6 +29,12 @@ type Pins = Arc> /// and an export's track requests). A route replaced after that is never resolved, epoch or /// not, so two publishers never splice into one export; a request on a held broadcast that has /// ended fails instead. Clones share the held broadcasts. +/// +/// A reference may name the instance it expects with an epoch (the catalog's optional `epoch` +/// beside `broadcast`). A held or resolved broadcast serving another epoch fails that reference +/// with [`Error::EpochMismatch`](crate::Error::EpochMismatch) rather than replacing the held +/// one or falling back to it. A broadcast resolved over a route without an epoch cannot prove +/// its instance, so there the epoch is best effort: the reference reads the held broadcast. #[derive(Clone)] pub struct Source { origin: moq_net::origin::Consumer, @@ -127,16 +133,23 @@ impl Source { Binding::new(self.broadcast()) } - /// The broadcast `path` is pinned to, or a lookup that pins its answer. - fn request_path(&self, path: &moq_net::PathOwned) -> Binding { - if let Some(pinned) = self.pins.lock().expect("pins poisoned").get(path) { - return Binding::new(pinned.clone()); + /// The broadcast `path` is pinned to, or a lookup that pins its answer, either checked + /// against `epoch` on every read. + fn request_path(&self, path: &moq_net::PathOwned, epoch: Option<&moq_net::Epoch>) -> Binding { + let bound = match self.pins.lock().expect("pins poisoned").get(path) { + Some(pinned) => Bound::Ready(pinned.clone()), + // Unpinned even with an epoch: an epochless route still answers, best effort, and + // the read refuses a route that proves another instance. + None => Bound::Requested { + request: self.origin.request_broadcast(path, None).into_inner(), + pins: self.pins.clone(), + path: path.clone(), + }, + }; + Binding { + bound, + epoch: epoch.cloned(), } - Binding(Bound::Requested { - request: self.origin.request_broadcast(path, None).into_inner(), - pins: self.pins.clone(), - path: path.clone(), - }) } /// Resolve a rendition's optional broadcast reference to an origin path. @@ -163,14 +176,15 @@ impl Source { }) } - /// The skipping counterpart to [`Self::bind`], returning `None` when `rel` walks above - /// the origin root. + /// The skipping counterpart to [`Self::bind`] for a catalog entry, returning `None` when its + /// broadcast walks above the origin root. /// /// The exporters use it to drop a single bad rendition rather than fail the whole export. /// [`Self::retain_selectable`] already removes those renditions, so this is the second half of /// the same policy rather than a different one. - pub(crate) fn try_request(&self, rel: Option<&moq_net::path::Relative<'_>>) -> Option { - Some(self.request_path(&self.resolve_reference(rel)?)) + pub(crate) fn try_request(&self, reference: &impl hang::catalog::Reference) -> Option { + let path = self.resolve_reference(reference.broadcast())?; + Some(self.request_path(&path, reference.epoch())) } /// Remove the renditions an export must not select: a disabled one (catalog @@ -216,7 +230,8 @@ impl Source { /// /// `rel` is a rendition's catalog `broadcast` field: `None` (or an empty reference) /// resolves the catalog broadcast itself; anything else fetches the referenced - /// broadcast from the origin. Use it when you need the broadcast handle itself + /// broadcast from the origin. `epoch` is the rendition's `epoch` field, the instance the + /// broadcast must be (see [`Source`]). Use it when you need the broadcast handle itself /// (e.g. to FETCH individual groups) rather than a subscription. /// /// A reference that escapes above the origin root is @@ -225,8 +240,9 @@ impl Source { pub async fn resolve( &self, rel: Option<&moq_net::path::Relative<'_>>, + epoch: Option<&moq_net::Epoch>, ) -> crate::Result { - self.bind(rel)?.broadcast().await + self.bind(rel, epoch)?.broadcast().await } /// Start one broadcast request now and retain its result for repeated reads. @@ -240,9 +256,14 @@ impl Source { /// one upstream subscription, and its answer is pinned. Use [`Binding::new`] when the /// intended broadcast is already in hand. /// - /// Rejects an escaping reference exactly as [`resolve`](Self::resolve) does. - pub fn bind(&self, rel: Option<&moq_net::path::Relative<'_>>) -> crate::Result { - Ok(self.request_path(&self.target(rel)?)) + /// Rejects an escaping reference exactly as [`resolve`](Self::resolve) does, and checks + /// `epoch` on every read. + pub fn bind( + &self, + rel: Option<&moq_net::path::Relative<'_>>, + epoch: Option<&moq_net::Epoch>, + ) -> crate::Result { + Ok(self.request_path(&self.target(rel)?, epoch)) } /// Resolve an optional cross-broadcast reference and subscribe to track `name`, @@ -250,7 +271,8 @@ impl Source { /// /// `rel` is a rendition's catalog `broadcast` field: `None` (or an empty reference) /// subscribes on the catalog broadcast; anything else fetches the referenced broadcast - /// from the origin first. A reference that escapes above the origin root is + /// from the origin first, checked against `epoch` as in [`resolve`](Self::resolve). A + /// reference that escapes above the origin root is /// [`Error::EscapingBroadcast`](crate::Error::EscapingBroadcast). /// /// This is the async counterpart to the poll-driven container exporters: consumers @@ -259,9 +281,10 @@ impl Source { pub async fn subscribe_track( &self, rel: Option<&moq_net::path::Relative<'_>>, + epoch: Option<&moq_net::Epoch>, name: &str, ) -> crate::Result { - let broadcast = self.bind(rel)?.broadcast().await?; + let broadcast = self.bind(rel, epoch)?.broadcast().await?; Ok(broadcast.track(name)?.subscribe(None).await?) } } @@ -270,10 +293,15 @@ impl Source { /// /// Reading the binding never looks up its path again, even after a failure or publisher /// replacement. Pending requests follow the origin's routing semantics; issuing a request -/// does not establish an epoch relationship with a separate catalog broadcast. +/// does not establish an epoch relationship with a separate catalog broadcast. A binding made +/// with an epoch refuses a broadcast proven to serve another one. /// /// Build one with [`Source::bind`], or with [`Binding::new`] for a broadcast already in hand. -pub struct Binding(Bound); +pub struct Binding { + bound: Bound, + /// The instance the reference named, if any. + epoch: Option, +} enum Bound { /// A broadcast the caller already holds. @@ -290,22 +318,39 @@ enum Bound { impl Binding { /// Bind to a broadcast already in hand, for a reference that named it. pub fn new(broadcast: moq_net::broadcast::Consumer) -> Self { - Self(Bound::Ready(broadcast)) + Self { + bound: Bound::Ready(broadcast), + epoch: None, + } } /// Poll for the bound broadcast without blocking. /// /// Concurrent polls and cancelled waits retain the same request and result. pub fn poll_broadcast(&self, waiter: &kio::Waiter) -> Poll> { - match &self.0 { - Bound::Ready(broadcast) => Poll::Ready(Ok(broadcast.clone())), + let broadcast = match &self.bound { + Bound::Ready(broadcast) => broadcast.clone(), Bound::Requested { request, pins, path } => { let broadcast = ready!(request.poll_ok(waiter))?; - // A request that raced another for the path takes the one pinned first. + // A request that raced another for the path takes the one pinned first. The path's + // first answer is pinned even when it fails this binding's epoch, so no later + // reference resolves the path independently. let mut pins = pins.lock().expect("pins poisoned"); - Poll::Ready(Ok(pins.entry(path.clone()).or_insert(broadcast).clone())) + pins.entry(path.clone()).or_insert(broadcast).clone() } + }; + // An epochless route cannot prove its instance, so only a known mismatch fails. + let info = broadcast.info(); + if let (Some(expected), Some(actual)) = (&self.epoch, &info.epoch) + && expected != actual + { + return Poll::Ready(Err(crate::Error::EpochMismatch { + path: info.path.to_string(), + expected: expected.clone(), + actual: actual.clone(), + })); } + Poll::Ready(Ok(broadcast)) } /// The bound broadcast, awaiting the origin's answer when it hasn't arrived yet. @@ -381,7 +426,7 @@ mod tests { settle().await; let source = resolved(&origin, "pub").await; let dynamic = origin.dynamic_route("media", Default::default()).unwrap(); - let binding = source.bind(Some(&Relative::new("media"))).unwrap(); + let binding = source.bind(Some(&Relative::new("media")), None).unwrap(); let request = dynamic.requested_broadcast().await.unwrap(); assert!( @@ -415,11 +460,11 @@ mod tests { settle().await; let source = resolved(&origin, "pub").await; let media = Relative::new("media"); - let binding = source.bind(Some(&media)).unwrap(); + let binding = source.bind(Some(&media), None).unwrap(); assert!(binding.broadcast().await.is_err()); let _publisher = origin.publish("media", Default::default()).unwrap(); settle().await; - assert!(!source.resolve(Some(&media)).await.unwrap().is_closed()); + assert!(!source.resolve(Some(&media), None).await.unwrap().is_closed()); assert!(binding.broadcast().await.is_err()); } @@ -438,9 +483,9 @@ mod tests { settle().await; let source = Source::new(consumer, first.clone()); assert!(source.broadcast().is_clone(&first)); - assert!(source.resolve(None).await.unwrap().is_clone(&first)); + assert!(source.resolve(None, None).await.unwrap().is_clone(&first)); assert!( - source.subscribe_track(None, "late").await.is_err(), + source.subscribe_track(None, None, "late").await.is_err(), "the replacement's track is never read" ); } @@ -454,20 +499,142 @@ mod tests { settle().await; let source = resolved(&origin, "a/pub").await; let sibling = Relative::new("./source"); - let first = source.resolve(Some(&sibling)).await.expect("the sibling resolves"); + let first = source + .resolve(Some(&sibling), None) + .await + .expect("the sibling resolves"); let new = origin.publish("a/source", Default::default()).unwrap(); let _late = new.create_track("late", None).unwrap(); settle().await; - assert!(source.resolve(Some(&sibling)).await.unwrap().is_clone(&first)); + assert!(source.resolve(Some(&sibling), None).await.unwrap().is_clone(&first)); assert!( - source.subscribe_track(Some(&sibling), "late").await.is_err(), + source.subscribe_track(Some(&sibling), None, "late").await.is_err(), "the replacement's track is never read" ); - let bound = source.bind(Some(&sibling)).unwrap().broadcast().await.unwrap(); + let bound = source.bind(Some(&sibling), None).unwrap().broadcast().await.unwrap(); assert!(bound.is_clone(&first), "a binding honors the pin too"); } + fn route(epoch: &moq_net::Epoch) -> moq_net::origin::Route { + moq_net::origin::Route::default().with_epoch(epoch.clone()) + } + + fn is_mismatch(result: crate::Result, expected: &moq_net::Epoch) -> bool { + matches!(result, Err(crate::Error::EpochMismatch { expected: ref named, .. }) if named == expected) + } + + /// An omitted or matching epoch resolves the sibling; another epoch fails instead. + #[tokio::test(start_paused = true)] + async fn a_reference_epoch_must_match_the_resolved_sibling() { + let origin = produce_origin(); + let (catalog, sibling, other) = (moq_net::Epoch::mint(), moq_net::Epoch::mint(), moq_net::Epoch::mint()); + let _catalog = origin.publish("a/pub", route(&catalog)).unwrap(); + let _sibling = origin.publish("a/source", route(&sibling)).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + let rel = Relative::new("./source"); + + let first = source.resolve(Some(&rel), None).await.unwrap(); + assert_eq!(first.info().epoch.as_ref(), Some(&sibling)); + let matching = source.resolve(Some(&rel), Some(&sibling)).await.unwrap(); + assert!(matching.is_clone(&first)); + assert!(is_mismatch(source.resolve(Some(&rel), Some(&other)).await, &other)); + assert!( + source.subscribe_track(Some(&rel), Some(&other), "video").await.is_err(), + "a mismatched reference reads nothing" + ); + } + + /// A replacement before the first resolution fails a reference naming the old instance, + /// rather than falling back to the current winner. + #[tokio::test(start_paused = true)] + async fn a_replaced_sibling_fails_a_reference_to_the_old_epoch() { + let origin = produce_origin(); + let (old_epoch, new_epoch) = (moq_net::Epoch::mint(), moq_net::Epoch::mint()); + let _catalog = origin.publish("a/pub", Default::default()).unwrap(); + let _old = origin.publish("a/source", route(&old_epoch)).unwrap(); + let _new = origin.publish("a/source", route(&new_epoch)).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + let rel = Relative::new("./source"); + + assert!(is_mismatch( + source.resolve(Some(&rel), Some(&old_epoch)).await, + &old_epoch + )); + // The path's first answer is pinned all the same: later references read that instance. + let pinned = source.resolve(Some(&rel), None).await.unwrap(); + assert_eq!(pinned.info().epoch.as_ref(), Some(&new_epoch)); + } + + /// A later reference naming the replacement cannot move a pinned sibling, and a new + /// catalog instance starts fresh pins that resolve it. + #[tokio::test(start_paused = true)] + async fn a_conflicting_epoch_cannot_replace_a_pin() { + let origin = produce_origin(); + let (old_epoch, new_epoch) = (moq_net::Epoch::mint(), moq_net::Epoch::mint()); + let _catalog = origin.publish("a/pub", Default::default()).unwrap(); + let _old = origin.publish("a/source", route(&old_epoch)).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + let rel = Relative::new("./source"); + let first = source.resolve(Some(&rel), Some(&old_epoch)).await.unwrap(); + + let _new = origin.publish("a/source", route(&new_epoch)).unwrap(); + settle().await; + // A rendition added later names the replacement: it fails, and the pin stays. + let binding = source.bind(Some(&rel), Some(&new_epoch)).unwrap(); + assert!(is_mismatch(binding.broadcast().await, &new_epoch)); + assert!(source.resolve(Some(&rel), None).await.unwrap().is_clone(&first)); + assert!( + source + .resolve(Some(&rel), Some(&old_epoch)) + .await + .unwrap() + .is_clone(&first) + ); + + // Concurrent consumers: a new catalog instance resolves the replacement while the old + // source keeps reading its pin. + let fresh = source.switched(&source.broadcast()); + let replaced = fresh.resolve(Some(&rel), Some(&new_epoch)).await.unwrap(); + assert_eq!(replaced.info().epoch.as_ref(), Some(&new_epoch)); + assert!(source.resolve(Some(&rel), None).await.unwrap().is_clone(&first)); + } + + /// A self-reference's epoch must not contradict the catalog broadcast's own. + #[tokio::test(start_paused = true)] + async fn a_self_reference_epoch_checks_the_catalog_instance() { + let origin = produce_origin(); + let (catalog, other) = (moq_net::Epoch::mint(), moq_net::Epoch::mint()); + let _catalog = origin.publish("a/pub", route(&catalog)).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + + for reference in [None, Some("./pub")] { + let rel = reference.map(Relative::new); + let own = source.resolve(rel.as_ref(), Some(&catalog)).await.unwrap(); + assert!(own.is_clone(&source.broadcast()), "{reference:?}"); + assert!(is_mismatch(source.resolve(rel.as_ref(), Some(&other)).await, &other)); + } + } + + /// An epochless route cannot prove its instance, so an explicit epoch is best effort there. + #[tokio::test(start_paused = true)] + async fn an_epochless_route_reads_an_epoch_reference_best_effort() { + let origin = produce_origin(); + let _catalog = origin.publish("a/pub", Default::default()).unwrap(); + let _sibling = origin.publish("a/source", Default::default()).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + let epoch = moq_net::Epoch::mint(); + + let sibling = source.resolve(Some(&Relative::new("./source")), Some(&epoch)).await; + assert!(sibling.unwrap().info().epoch.is_none()); + assert!(source.resolve(None, Some(&epoch)).await.is_ok()); + } + /// Every self-reference serves the broadcast the source was built from, even once it /// ended and a replacement took the path. #[tokio::test(start_paused = true)] @@ -482,7 +649,7 @@ mod tests { for reference in [None, Some(""), Some("live"), Some("./live"), Some("../a/live")] { let rel = reference.map(Relative::new); - let bound = source.bind(rel.as_ref()).unwrap().broadcast().await.unwrap(); + let bound = source.bind(rel.as_ref(), None).unwrap().broadcast().await.unwrap(); assert!( bound.is_closed(), "{reference:?} must keep the original catalog broadcast" @@ -501,14 +668,14 @@ mod tests { // No reference and an empty reference both resolve to the catalog broadcast. source - .bind(None) + .bind(None, None) .expect("no reference is always resolvable") .broadcast() .await .expect("catalog broadcast should resolve"); let empty = Relative::empty(); source - .bind(Some(&empty)) + .bind(Some(&empty), None) .expect("empty reference is always resolvable") .broadcast() .await @@ -526,7 +693,7 @@ mod tests { let source = resolved(&origin, "a/pub").await; source - .subscribe_track(None, "video") + .subscribe_track(None, None, "video") .await .expect("catalog track should resolve"); } @@ -544,7 +711,7 @@ mod tests { // Names the catalog within its own parent. let rel = Relative::new("./pub"); source - .subscribe_track(Some(&rel), "video") + .subscribe_track(Some(&rel), None, "video") .await .expect("self-reference should resolve to the catalog broadcast"); } @@ -574,16 +741,16 @@ mod tests { source.resolve_reference(Some(&rel)).is_none(), "{reference} should escape" ); - assert!(source.bind(Some(&rel)).is_err(), "{reference} should be rejected"); + assert!(source.bind(Some(&rel), None).is_err(), "{reference} should be rejected"); // Neither `elsewhere` nor the catalog broadcast answers it: the rendition names // no broadcast at all. - match source.resolve(Some(&rel)).await { + match source.resolve(Some(&rel), None).await { Err(crate::Error::EscapingBroadcast(_)) => {} Err(err) => panic!("{reference} failed with the wrong error: {err:?}"), Ok(_) => panic!("{reference} should not resolve to any broadcast"), } - match source.subscribe_track(Some(&rel), "video").await { + match source.subscribe_track(Some(&rel), None, "video").await { Err(crate::Error::EscapingBroadcast(_)) => {} Err(err) => panic!("{reference} failed with the wrong error: {err:?}"), Ok(_) => panic!("{reference} should not resolve to any broadcast"), @@ -685,7 +852,7 @@ mod tests { // The reference resolves to `a/source`, whose "video" track answers the subscribe. let rel = Relative::new("./source"); source - .subscribe_track(Some(&rel), "video") + .subscribe_track(Some(&rel), None, "video") .await .expect("referenced track should resolve"); } @@ -705,7 +872,7 @@ mod tests { let source = resolved(&origin, "a/source/transcode").await; let rel = Relative::new("."); source - .subscribe_track(Some(&rel), "video") + .subscribe_track(Some(&rel), None, "video") .await .expect("dot should resolve to the catalog broadcast's parent"); } @@ -725,7 +892,7 @@ mod tests { let source = resolved(&origin, "top").await; let rel = Relative::new("."); source - .subscribe_track(Some(&rel), "video") + .subscribe_track(Some(&rel), None, "video") .await .expect("dot should resolve to the empty root broadcast"); } diff --git a/rs/moq-rtc/src/egress.rs b/rs/moq-rtc/src/egress.rs index 80ab1130953..4440828d417 100644 --- a/rs/moq-rtc/src/egress.rs +++ b/rs/moq-rtc/src/egress.rs @@ -274,16 +274,18 @@ impl Rendition { /// Subscribe to the rendition, honoring its cross-broadcast reference (the catalog /// `broadcast` field). async fn subscribe(self, source: moq_mux::Source) -> Result { + use hang::catalog::Reference; + match &self { Self::Audio(name, config) => { let track = source - .subscribe_track(config.broadcast.as_ref(), hang::catalog::Reference::track(config, name)) + .subscribe_track(config.broadcast(), config.epoch(), config.track(name)) .await?; codec::Track::opus(track, config) } Self::Video(name, config) => { let track = source - .subscribe_track(config.broadcast.as_ref(), hang::catalog::Reference::track(config, name)) + .subscribe_track(config.broadcast(), config.epoch(), config.track(name)) .await?; codec::Track::video(track, config) }