Skip to content

feat(moq-net)!: address groups by their start - #8638

Merged
moq-bot[bot] merged 102 commits into
mainfrom
quest/m2/time-addressing/model
Nov 15, 2026
Merged

moq-bot[bot] merged 102 commits into
mainfrom
quest/m2/time-addressing/model

Conversation

@kixelated

@kixelated kixelated commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Claims quest/m2/time-addressing/model.md.

Rust groups are addressed by their start. A group's key is a u64 in its track's timescale (a bare number on a scale-less track), and moq-net's API and model carry no group sequence numbers. Legacy wires (lite-01 to 06, moq-transport) carry the key verbatim as the group ID, so group 6 leaves as group 6. Every Rust crate, moq-ffi, moq-c and the interop clients move with it. JS keeps numbering groups until its own quest.

A scale-less track's frames now carry a Timestamp too: Timestamp { value, scale: Option<Timescale> }, where Timestamp::unscaled(key) is the group key (a datagram's sequence). Every write takes a required Timestamp, in Rust, JS, moq-ffi, moq-c and every binding.

Decisions

Following the Plan's maintainer decisions (2026-10-25, 2026-10-26, 2026-11-03, 2026-11-07, 2026-11-13); these are the calls I made inside them.

Maintainer decisions (2026-11-13, /quest-complete)

  • A scale-less key is a Timestamp whose scale is None. ✅ Timestamp { value: u64, scale: Option<Timescale> }, Timestamp::unscaled(key), and Timescale stays NonZero. Every write takes a required Timestamp: group and track write_frame (the track one now works on a scale-less track, at the key), Frame::new, datagram append, moq-json's Timed::at, and the JS, moq-ffi, moq-c and binding counterparts. A mismatched scale, or an unscaled key naming another group, stays TimestampMismatch. Rejected: a bare u64 key with an always-scaled Timestamp, and Option<Timestamp>.
  • moq-json Timed::at: ✅ required, with the unscaled key on a scale-less track. Rejected: Option<Timestamp>, and refusing scale-less JSON tracks.
  • lite-05/06 egress: ✅ as built. TRACK_INFO declares milliseconds on a scale-less track, because those published drafts treat Timescale = 0 as invalid. The group ID is the key, and frames carry the group key. lite::Version::wire_timescale and the draft's lite-07 changelog say this is forced by the published wire. Rejected: main's send-time stamps (feat(lite): lite-07 carries an untimed track untimed #5461), and refusing scale-less tracks on lite-05/06.
  • lite-01..04 lag by the hole budget: ✅ accepted. Rejected: extending the legacy-ingest rule so keys below the newest arrival count as never arriving.
  • Released @moq/hang players reading new Legacy audio over lite-05/06: ✅ accept the gaps. The packages release independently, and old players report gaps until they upgrade (see Impact). Rejected: shipping @moq/hang before or with the Rust publishers.
  • A resume after a pause over wires without declared skips: ✅ accepted. Rejected: a heuristic for older wires.

Taken by the agent under /quest-afk, for the maintainer's review (2026-11-13)

Decided 2026-11-14 (maintainer, /quest-complete): ✅ accepted

  • Own-end reach ships before the wire carries the end. An origin whose producer calls finish(end) ages a paused track's last group out after its span. A relay receives that group with a bare FIN (rs/moq-net/src/recv.rs closes it), so it still bounds the group by its successor, as main does on every hop. Relays serve nothing main wouldn't, and the gap closes when lite-07's GROUP_END carries the end (lite07.md). Rejected: carrying the end on the wire here, which is lite07's wire, draft and JS change; and holding own-end reach back until then, which reverts the 2026-10-26 decision.
  • A transcode fill locates a gap through one source read. On a miss, the fill reads the source once, in key order, for the groups around it, and drops the span between them in one call. A source that holds no group at all can't place that read until it holds one. A live rung then waits for the source's next group, as any time-anchored reader of the source would, and holds no encoder slot while it waits. A retired rung's drain drops the miss and refuses the rest of the fill instead, since nothing says the source lost it. Rejected: a fetch per microsecond, which costs millions of requests per GOP gap; and dropping the rest when the read can't answer at once, which loses groups a remote or handler-served source has. The long-term answer is moq-net serving a fill from another track's range, or an at-or-after fetch, as a relay serves one upstream. That is new API in every language, so it is listed in Follow-ups.

Pullfrog's review (my calls, 2026-11-14)

  • HLS reanchors with fmp4::Import::rejoin(): ✅ new. A reanchor seeked to the packed HLS counter, 2^48 + n after a discontinuity, so every later keyframe sat below it and one group took the rest of the epoch. rejoin() closes every track's group without an end and holds video until a sync sample, as seek does, and keys the next groups at their own times. Rejected: seeking to the segment's start, which HLS can't know before reading the segment; and dropping the seek, which ends the group before a skip whole at the next keyframe, hiding the lost segments, and lets video continue on delta frames.
  • moq-ffi and moq-c refuse a scaled time anchor: ✅ TimestampMismatch. They read an anchor's value in track units whatever its scale said, so 1 s at a millisecond scale on a 90 kHz track named tick 1000. A time anchor is now an unscaled key in the track's timescale. Rejected: converting it now, which needs the track's timescale, unknown when the bindings parse a subscription (the bindings quest's follow-up); and keeping the silent reading. Accepting a scaled anchor later breaks nothing.
  • moq-archive resolves a time by group key: ✅ as the live cache does, rather than by each group's first frame, which differs for a seeked group. The index drops its first-frame time set, since object keys already sort by start.
  • The warmup video lane joins with a time start: ✅ Anchor::Time(edge - 1), the group containing the key just below the live edge the min-delay lane found, in moq-video and moq-cli. Its Relative(1) start failed every read over lite-01..06, whose copies refuse a relative start past the edge. lite-07 resolves the time start to the group before the edge, and older wires to the edge alone. moq-cli waits for the edge only for a warmup lane, as main waited for every lane. Rejected: having lite-01..06 copies count a relative start back through what they hold, a moq-net semantic change their doc refuses as a guess; and moving the CLI's video opening off its playback loop, which restructures playback this PR doesn't touch.
  • @moq/net's Time.keyOf is public: ✅ @moq/json opened a scale-less group through Timestamp.value, which throws on a key past 2^53. keyOf reads the exact bigint key. Rejected: a second accessor on Timestamp beside the internal one.
  • A scale-less track ages keys by their distance over the whole key range: ✅ Freshness converted keys to milliseconds, which fails past 2^61 and left those groups to the wall clock. Keys stay unscaled, and their distance counts as milliseconds.
  • Layered runs follow-up: moved to m3: breaks nothing, so it shouldn't gate the cut (rejected: m2). It changes no API or wire, so under the questline's rule it belongs in m3, not in m2, where it would gate the 0.22.0 cut. Taken under the coordinator's brief while the maintainer is away, for review.
  • Time anchors against publishers that still number groups densely: ✅ stated in Impact. JS and previous-release Rust publishers number groups densely. A time anchor goes upstream as group t, so against them it names the wrong group until they key by start. Rejected: keeping main's widen-to-group-0 on wires without time anchors, which reads every group back to 0 to find one. Today's time-anchored readers are moq-transcode, which reads Rust publishers, and the bindings.

Timestamp shape (my calls inside the 2026-11-13 rule)

  • Conversions that need a scale are fallible rather than confined to a second type: convert, as_scale, as_secs, as_millis, as_micros and as_nanos return Result<_, TimeOverflow>, erring when unscaled, and Duration is TryFrom<Timestamp>. A second, always-scaled type would add a type to every binding for what one Option already says.
  • No Ord. PartialOrd is temporal across scales, compares keys when both are unscaled, and returns None between scaled and unscaled. Debug prints an unscaled timestamp as #<key>.
  • unscaled is infallible over the whole u64, since a legacy wide key passes 2^61. Scaled values stay bounded at 2^61 - 1.
  • track::Info::rescale(ts) checks the scale. The new public rescale_in(ts, key) also checks the key, so a signer or encrypter refuses a frame before it spends anything.
  • append_group(start) and group::Info { start } stay: a multi-frame group opens before its first frame is written.
  • On a scale-less datagram track, append(unscaled(seq)) writes at seq, and push/insert refuse a timestamp that names another sequence. moq-crypto's e2ee datagrams follow suit.
  • moq-json on a scale-less track: a snapshot or header write opens its group at at.value(), which must pass every group held. A delta, op, or stream frame carries the open group's key.
  • moq-crypto signs a scale-less frame as (0, 0) for its timescale and value, as before. The key is already signed as the group, so existing signatures still verify.
  • Media consumers (moq-mux and the crates downstream of it) refuse an unscaled frame with TimestampMismatch and never panic. hang's catalog Clock refuses an unscaled wall clock.
  • JS keys are bigint, so an unscaled JS key past 2^53 throws only when value is read, never on receipt.
  • moq-c has no Option, so moq_timestamp gains has_scale, following its has_* fields.

Model

  • The key is a u64 in the track timescale, not a Timestamp, as JS's bigint starts are: group::Info { start }, append_group(start), Position::group (now #[non_exhaustive]).
  • Starts strictly increase. A group whose first frame ties or rewinds past a neighbour's is refused with TimestampMismatch, so two groups never share a start.
  • Frame timestamps are a required Timestamp: a time on a timed track, and Timestamp::unscaled(group key) on a scale-less one.
  • Anchor is {Oldest, Relative(n), Time(t)} with inclusive ends. A time start is the containing group: the newest key at or before t. Time(0) is always group 0. Relative(n) counts back through the groups held, however sparse their keys.
  • Skips are declared, never inferred. append_group skips the starts between it and the previous append, skip_groups skips a run, and lite-07 forwards both as SUBSCRIBE_SKIP. A reader learns them through two additions:
    • Subscriber::next_in_range now passes over declared skips.
    • Subscriber::is_skipped(range) is new.
      Without them, an in-order reader can't tell a skipped span from lost groups. moq-mux's consumer reported spurious gaps until it used them.
  • A group ends whole with group::Producer::finish(end), or without a known end with close() (the old finish()). A whole group ages from its own end, so a finished or paused track's last group expires without a successor, but not before that end.
  • A scale-less track's max age counts bare key units, as milliseconds.
  • Timestamp::now() is gone. Producers stamp from their own clock.
  • Timed moves from moq-net to moq-json, which owns it as @moq/json does. It becomes a struct with a required at, built with Timed::new(value, at).

Wires

  • Legacy ingest keys a group by its wire ID. Legacy egress writes the key back.
  • A legacy inclusive start S resolves to the first group keyed at or above S, never to an earlier group covering it (sparse 4 and 7 with a start of 5 select 7). An exclusive end E maps to Time(E - 1), and E = 0 drops the range rather than saturating to group 0.
  • lite-07-wip decodes a wire absolute start as Time(S), the containing group, and encodes Time(t) as absolute t. Its encoder no longer emits wire Time.
  • A lite-07 copy places a time start only once it knows where it falls, in this order:
    1. The key t is held.
    2. Upstream's SUBSCRIBE_START or STARTED answer for it was recorded.
    3. The newest key held at or before t is held, and holes or a later group prove nothing lies between them.
    4. Holes cover everything up to t.
      Until then the span reads, and asks fills for, the groups from t on, and Subscriber::poll_start waits.
      Legacy copies resolve on their keys as their upstream does.
  • A recovery subscription records the publisher's SUBSCRIBE_SKIP instead of failing it as a protocol violation.
  • IETF relative filters count groups through the keys held. Priority wraps wide keys to their low 48 bits.
  • lite-05/06 egress on a scale-less track keeps TRACK_INFO at milliseconds, forced because those published drafts treat Timescale = 0 as invalid. Its frames carry the group key, and its datagrams their sequence. A lite-02 reader of a scale-less track sees unscaled(key).
  • Old lite versions: every version carries a start group, so a time start goes out as its key. lite-01 to 05 have no oldest start, and an Oldest start asks with none, which they read as the latest group. That is unchanged from main.

Producers and readers

  • Producers key each group at its first frame's time in the track timescale. Clock-stamped writes (catalog, JSON, table revisions) that land in the same unit as the last are stamped one unit later instead of refused.
  • moq-mux's container producer skips every start below a fresh track's first group, saying nothing came before it. moq-audio's decoder reads is_skipped(0..group) in place of group == 0 to tell a track's first packet, which needs no join warmup.
  • A seek names the next group's start. A discontinuity after a seek stamps its marker at the live edge, or one unit past the newest group when the edge is unknown.
  • moq-audio ends each packet group whole at its last packet's end, so consecutive groups read as contiguous.
  • moq-audio never stamps a packet below where the packets before it end: a reanchor that would rewind (a capture block clamped to now, AAC resuming inside its priming window) is raised there, and AAC priming packets clamped to 0 share one group.
  • A discontinuity marker is a group of its own, so it sits one unit past the newest group's start and never below the last frame. Go's TestDynamicTrackRequestCanPublishAudio now reads the encoder's end at 20001us, not 20000us.
  • Legacy audio states its packet durations. moq-mux adds Container::LegacyAudio, built from an AudioConfig whose codec states them (Opus TOC, AAC-LC's 1024 samples, PCM length), and @moq/hang's legacy format mirrors it. A reader then tells a group that ends where the next one starts from lost audio, which sparse starts no longer say.
  • moq-mux's consumer:
    • A run the publisher declared skipped holds no group: it steps over the run with no gap.
    • Any other run between groups is waited for as a missing sequence was, then lost, unless the next group's first frame continues the timeline within 1 ms.
    • A run dropped over a wire that can't say skipped (lite-05/06) still waits for that first frame before judging.
    • The old marker shortcut is gone.
  • A latest video join counts its lookback back in groups with Anchor::Relative(lookback) (moq-video, moq-cli), since largest - 1 no longer names the previous group.

Bindings (stand-in until the bindings quest)

  • moq-ffi and moq-c keep their exported names and shapes. A sequence argument is the group's start in the track timescale.
  • An FFI or C append_group() without a start takes the next key past the largest.
  • A MoqAnchor time value is taken in track units. An absolute end E maps to Time(E - 1), and E = 0 drops the range.
  • Their fill handlers name groups, so a time lookup (Fill::time) is dropped unanswered. The reader then falls back to the groups held, as a handler that can't resolve times does.

Tests restated

  • Tests that counted sequences name groups by start: transcode, Python, moq-mux's release and recovery republishers (which now forward the import's skips as a lite-07 relay would), passthrough markers, and main's new held re-offer, paused group, and ingress tests.
  • The Kotlin smoke test fetches the group at 5,000,000us, its first packet's time.
  • moq-c's raw_track_relative_update_counts_from_the_track_edge is removed. Under containing semantics an open range far ahead holds the newest group, so it delivered with or without the update.
  • moq-hls's a_cursor_records_a_publisher_without_fetch gives lite-01 to 04 a 45 s budget, since those wires lag by the hole budget (accepted 2026-11-13).

Bugs this found

  • lite-05/06 SUBSCRIBE_FINISH before SUBSCRIBE_START. An end declared while the start was unresolved went out first. Those drafts make SUBSCRIBE_START the first message, and @moq/net reads a FINISH without one as "nothing served", settling the tail at once. The end now waits for the start, or goes out alone once none can arrive.
  • A widened finished track ended without its group. A resumed reader's copy that ran out under a narrower cap or ranges stayed marked done, so an update widening it ended the read without the group it now admits.
    • The publisher then dropped that group as not found, and a relay applied the drop to every reader of its copy.
    • In interop, a Go history read [0, start) widened by a later reader made one JS reader in three read nothing in the finish case.
    • Fixed in resume.rs: a copy reads again whenever the reader's terms or bounds move.
  • A frame that lands mid-poll was discarded as lost. Measuring the current group's reach could buffer a frame that landed after its read came back empty. With codec durations that frame reaches the next group's start, so the coverage skip threw it away and reported a gap. That failed about half the interop runs of the JS publisher's audio into Rust, Go, C++ and Python readers.
    • Fixed in moq-mux: the frame is delivered before any skip is judged.
    • @moq/hang runs on one thread, so no frame lands mid-poll there.

Impact

Breaking for every Rust user of moq-net, moq-mux, moq-json, moq-crypto, moq-relay and moq-cli. Wire formats are unchanged except that lite-07-wip stops sending wire Time anchors. Migration:

  • group::Info { sequence } → group::Info { start }, and .sequence → .start, in the track's timescale.
  • append_group() → append_group(start): the first frame's time, pts.convert(timescale)?.value().
  • The old group.finish() → group.close(). finish(end) now ends a group whole at an exclusive end.
  • Anchor::Absolute(g) start → Anchor::Time(g). Exclusive Absolute(E) end → Time(E - 1), and drop the range when E is 0.
  • Range::groups(..) and Subscription::with_range(..) → Subscription::with_ranges([Range { start, end }]).
  • Anchor::Time takes a u64 in the track timescale, not a Timestamp. Fill::time() returns Option<u64>.
  • Position { .. } literals → Position::new(group, frame) or Position::group(g).
  • Timestamp::now() → stamp from your own clock.
  • moq_net::Timed → moq_json::Timed, a struct: Timed::from(v).at(t) → Timed::new(v, t), and at is a required Timestamp.
  • Timestamp::scale() returns Option<Timescale>. convert, as_scale, as_secs, as_millis, as_micros and as_nanos return Result<_, TimeOverflow>. Duration::from(ts) → Duration::try_from(ts)?. Timestamp has no Ord: replace .max(), .min() and .sort() with partial_cmp or sort_by.
  • Every untimed write takes the key: write_frame(None, p) → write_frame(Timestamp::unscaled(group.start), p), Frame::new(None, ..) → Frame::new(Timestamp::unscaled(key), ..), and datagram append(None, p) → append(Timestamp::unscaled(sequence), p). frame.timestamp, datagram.timestamp and Timed::at are no longer Option: on a scale-less track they're unscaled(key), and scale() is None.
  • moq-crypto: write_frame, datagram append and insert take a required Timestamp, and read frames and datagrams carry one.
  • moq-ffi and the bindings: MoqTimestamp.scale is optional (null means unscaled, with value the key). MoqFrame.timestamp, MoqDatagram.timestamp, MoqJsonValue.timestamp, MoqJsonWindowTimed.timestamp and datagram append(timestamp, ..) are required. Untimed writes pass unscaled(key). In Go, TimestampToDuration returns (time.Duration, error). In Dart, Timestamp is an extension type, so annotate reads as MoqTimestamp.
  • moq-c: moq_timestamp gains has_scale at its end, which changes the ABI of every record holding one, so recompile. A zeroed moq_timestamp is now the unscaled key 0: set {value, scale, true}, because an old positional {value, scale} silently becomes unscaled. moq_publish_track_frame, moq_publish_group_frame, moq_publish_datagrams_append and moq_publish_datagrams_insert refuse a NULL timestamp with MOQ_ERROR_INVALID_POINTER. On a scale-less track, pass the unscaled key: the new moq_publish_group_sequence reads a group's. A read's timestamp is never NULL. moq_audio_encoder_receive and moq_audio_encoder_packet fill a new moq_audio_packet instead of a moq_frame.
  • The generated C and C++ bindings (c/moq, cpp/moq) follow moq-ffi. The C moq_timestamp.scale is a moq_optional_u64, and the C++ moq::Timestamp::scale is a std::optional<uint64_t>. Frames carry their timestamp by value, and moq_frame_default() is gone.
  • @moq/net: Timestamp.scale may be undefined. Timestamp.unscaled(key) is new, and as, asMillis and asMicros throw when unscaled. Frame.timestamp, datagram timestamps and Timed.at are required.
  • @moq/hang and the Rust publishers release independently: until a @moq/hang player upgrades, it reports gaps between the sparse Legacy audio groups that lite-05/06 relays forward as drops.
  • moq-mux: GroupConsumer::sequence() → start(). Producer::seek takes a start, and reads' group fields carry starts.
  • moq-mux: catalog::hang::Container gains a LegacyAudio variant, which TryFrom<&AudioConfig> builds for Legacy audio whose codec states its packet durations. A match on the enum needs an arm for it.
  • moq-relay: /fetch?group=N names the group starting at N. moq-cli: fetch --group takes a start.
  • @moq/net: Time.keyOf(timestamp) is public, the exact bigint key of an unscaled timestamp.
  • moq-mux: fmp4::Import::rejoin() is new. moq-hls keys each group after a join, a skip or a discontinuity at its keyframe, and its SequenceOverflow reads "too large" rather than "too large to encode".
  • moq-ffi, moq-c and the bindings: a time anchor takes an unscaled timestamp in the track's timescale, and a scaled one fails with a TimestampMismatch protocol error. Anchor.TIME(Timestamp(v, scale)) → Anchor.TIME(Timestamp.unscaled(v_in_track_units)), and in C set has_scale to false.
  • A time anchor against a publisher that still numbers groups densely (JS, previous-release Rust) names group t, so it misresolves until that publisher keys by start.
  • A JS reader's relative range against a Rust publisher's sparse keys resolves by subtraction (latest key minus n), so it can skip the groups it means, until js.md counts groups as Rust does.
  • An HLS edge that loses a group inside a multi-group segment serves that segment without it, where main's dense ranges failed the fetch. The edge can't tell a lost group from none until lite-07's GROUP carries the previous group's start (lite07.md). Single-group segments, the usual video case, are unaffected.

Validation

At a2942bacc, which merges main at 98ed0f827 (#8739's send-stream hook, #8777's egress billing, #8728), just check passes with 12,902 Rust tests, and just test interop --all passes (211, 0 failed). Under the full check's load the Python socket test test_abort_and_endpoint_close_are_errors failed once (Unroutable where Closed was expected). That is main's endpoint-ending race (socket-endpoint-end.md); the py lane passed on rerun, and its file passed 20 of 20 runs alone. Codex round 8 accepted the JS refutation and holds the HLS one open.

At e0761bbf5, which merges main at d1c025add (with #8772's TCP-less sockets, #8771's p2p removal, #8712's resume tests, #8725's transcode session budget, #8778's HLS cut tags and #8764's feature review), and fixes Codex's segment-membership bound:

  • just check passes, with 12,887 Rust tests and the JS, Python, CARLA, bindings and API-report checks. Main's new code and tests moved to start keys: socket messages carry their group's key, a cut HLS segment is named by its last group, and a copy's resume point now bounds what it judges (see the merge commit).
  • just test interop --all: 211 passed, 0 failed (at 7b89888fd; the commit after it touches only moq-hls).
  • One full just check run failed test_cancelled_dial_releases_the_original_subscription (NotFound where Cancelled was expected, a race under load between the cancelled dial and the accept). The socket model is main's apart from the start anchor, and the test passed in 8 reruns of its file and in the py lane's rerun.
  • The local Codex review (round 7) confirmed the bound fixed. It holds two findings open, both refuted in Impact: an HLS edge losing a group inside a multi-group segment, and JS relative ranges against sparse keys.

At 82ca7af3b, which merges main at d4b46eec2 (adding #8741's chunk-streaming socket, #8745's key checks, #8749 and #8768's quest moves):

  • just check passes, with 14,161 Rust tests and the JS, Python, CARLA and API-report checks. The merge moved main's socket (Rust, tokio and its tests), the watch include tests and the CARLA sim's session replies to start keys. Each CARLA reply is a group keyed one past the last.
  • just test interop --all: 214 passed, 1 failed. The failure was CLI FETCH rust -> rust, which reported "group 1 of tail not found" over lite-06 right after the announce. The publisher writes every group before it connects, the relay log shows nothing past the cancel, and the case passed in three --tail reruns and in every earlier full run on this branch. I'm treating it as a flake, not a regression.
  • The local Codex review (round 5) found nothing on this head.

At bcd17e487, which merges main at 6f8b12729 (adding #8698's JS head fills, #8715's declared-clock ties, #8720's picture arrivals and #8636's VA-API reorder), just check passes, with 13,167 Rust tests, and just test interop --all passes. Main's new tests and code moved to start keys and fallible time conversions in the merges.

At 142e7faba, which merges main at 99d789eef (with #8699's lite-07 FETCH Start Byte), fixes Pullfrog's review, and fixes the local Codex review's findings:

  • just check passes, including 13,156 Rust tests and the Go, Kotlin, Dart, Python, C, C++, OBS, JS and API-report checks.
  • just test interop --all: all checks passed. One earlier full run failed tail rust -> rust with Error: old. The track is timed and none of this round's changes touch timed freshness. The lane passed in the run before it, in three --tail reruns, and in the full rerun. It reads as a wall-clock staleness verdict under the full run's load (1 MiB groups, a 5 s budget).
  • Each fix's test fails without it:
    • a_media_sequence_jump_keys_each_later_group_at_its_keyframe (moq-hls, new). It and the restated discontinuity tests now read three groups through real fMP4, and check that each group starts at its first frame. With the old seek, the groups after the reanchor started at the packed counter.
    • a_time_resolves_by_key_not_first_frame (moq-archive, new) resolved 2.2 s to the group at 1 s, not the seeked one keyed at 2 s.
    • time_anchors_start_at_the_containing_group (moq-ffi) and raw_track_update_refuses_a_scaled_time_anchor (moq-c) cover the scaled-anchor refusal. The Python local test now passes an unscaled anchor.
    • a key past the safe number range opens its group (@moq/json, new) threw a RangeError with the old helper.
    • a_scale_less_track_ages_wide_keys_by_their_distance (moq-net, new) delivered a stale group from keys at 2^61.
  • The warmup lane's time start has no lite-01..06 test of its own. Range::is_wire (rs/moq-net/src/model/subscription.rs:248) admits a time start on those copies, where it refused Relative(1).

At 4bbf407db, which merges main at 93cd63eb9 (with #8694, the interop refused session fix):

  • cargo nextest run --workspace --all-features: 14,143 passed. cargo clippy --workspace --all-features --tests --benches --examples -D warnings is clean.
  • just check passes, including the Go, Kotlin, Dart, Python, C, C++, OBS, JS and API-report checks. Swift compiles on Linux, but its tests need a Mac.
  • just test interop --all: 215 passed, 0 failed.

At 0bf2e87cb, after the local Codex review rounds (moq-archive, moq-json and moq-transcode only, so no wire, moq-ffi or binding change and no interop rerun):

  • just check passes, including 13,065 Rust tests across the changed crates and their dependents, clippy, fmt, docs and the quest check.
  • Each review fix has a test that fails without it:
    • a_re_added_track_holding_the_last_stored_group_stores_it_once (moq-archive) stored group 1000 twice.
    • a_fill_drops_a_gap_between_source_groups_at_once (moq-transcode) times out at nextest's 60 s with the old one-group drop. It now finishes in ~8 ms over a one-hour gap of microsecond keys.
    • Both moq-json a_refresh_after_older_*_waits_for_a_later_one tests fail with the write-wake removed.
  • The other moq-transcode tests cover the remaining paths: the descending, retired and empty-source gap fills (a_*_fill_drops_a_gap_*, a_fill_over_an_empty_source_drops_the_gap_once_the_source_has_a_group), and a_missing_group_fetched_alone_is_answered_at_once.

From the redesign itself:

  • test_unscaled_has_no_time (moq-net) covers the unscaled Timestamp: fallible conversions, comparison with scaled and unscaled timestamps, and the full key range.
  • scale_less_datagrams_write_at_their_unscaled_sequence (moq-crypto) covers the sequence-naming rule, and the unchanged signing vectors cover the (0, 0) signature.
  • raw_untimed_frames_roundtrip (moq-ffi) covers a track write at the key and a group write refused at another group's key.
  • moq-json's tests key their scale-less writes explicitly, and assert each group starts at the key of the write that opened it.

Before the merge with main at f952d10d7, the earlier fixes were validated as follows:

  • The finish case with every subscriber passed 4 of 4 runs. Before the resume fix, about 1 run in 3 failed.
  • The JS publisher into Rust, Go, C++, C and Python passed 4 of 4 runs. Before the moq-mux fix, about 1 run in 2 failed.

Each earlier fix's test fails without it:

  • a_pre07_end_declared_before_the_first_group_follows_the_start
  • a_widened_reader_reads_a_finished_tracks_group_it_ran_out_before
  • late_tail_reader::an_empty_history_read_leaves_the_group_to_readers_joining_meanwhile

The mid-poll race needs a frame to land between two reads in one poll, which a single-threaded test can't stage, so only the interop runs cover it. The verify-resume TLA+ model wasn't rerun: its scope leaves out range updates and local ranges, which is all the resume fix touches.

Follow-ups

  • Collapse a window's layered runs (m3): time ends settle at once on keys, so the moving layer is dead weight.
  • moq-c reads hand out a non-null const moq_timestamp*. Returning the timestamp by value, as moq-ffi now does, would save an allocation per read.
  • The released smoke clients build against the published bindings, so they move to unscaled with the next release.
  • Bindings quest:
    • Key FFI and C append_group at the first frame.
    • Accept scaled anchor times, converting them with the track timescale. They're refused until then.
    • Rename sequence parameters.
    • Expose Fill::time lookups.
    • Expose is_skipped.
  • JS: mirror is_skipped and the skip-aware next_in_range when @moq/net moves to start keys. @moq/hang's container/consumer.ts then tells a Rust track's first group by its declared skips, not sequence === 0n, which a Rust track keyed at its first frame never passes. Until then, a JS viewer joining a track younger than its reach waits out the elapsed fallback.
  • moq-net: serve a fill from another track's range, or fetch the group at or after a key, with definitive answers. A transcode rung could then forward a fill's gaps to its source as a relay does, rather than locating them with a time-anchored read that waits on a source holding no group.
  • lite-07 GROUP_END (lite07.md) carries a whole group's end, so relays age a paused track's last group as its origin does.
  • doc/concept/identity.md still describes group sequences. It's shared with JS, which still numbers groups, so it moves with the JS quest.

Models

Size: XL

Claude Opus 5.5 (xhigh) wrote moq-net, the bindings, the interop fixes, and the merge with main, and delegated the downstream crates to parallel Claude Opus 5.5 agents.

Implementer: Claude Code, claude-opus-5-5, high effort

That iteration ported the unscaled Timestamp through moq-net, moq-json, moq-crypto, moq-archive, moq-ffi and the leaf crates, and delegated moq-mux, moq-video, moq-audio, JS, moq-c with C/C++, and the uniffi wrappers to parallel Claude Opus 5.5 (high) agents.

Claude Code, claude-opus-5-5, iterated on 2026-11-14: merged main, fixed Pullfrog's review and the local Codex review's findings, and recorded the maintainer's acceptance.

Implementer: Claude Code, claude-opus-5-5, unknown effort (iterate, XL)

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Head: a2942ba, named in full once no reviewer reports a finding

  • Base: origin/main at 98ed0f8

  • Risk: high, Public group-addressing and range semantics change for publishers and cross-language subscribers, creating interoperability risks across deployed versions that a single-component revert may not resolve.

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort

  • Session: 01a1de07-7fc5-7fa2-a224-3ac33ef3e5d8, round 3

  • Score: 6/10, The recorded plan supports the staged JavaScript migration, but the HLS completeness regression remains unresolved.

  • Raised the risk to high: Public group-addressing and range semantics change for publishers and cross-language subscribers, creating interoperability risks across deployed versions that a single-component revert may not resolve.

  • Rounds: 8; findings 7 (high 2, medium 5): fixed 4, refuted 2, open 1

  • Findings, open until the next round settles them:

    • High, rs/moq-hls/src/export/rendition.rs:1729: Preserve dropped-group accounting in segment membership
  • The last round's findings, at e0761bb:

    • Open: high, rs/moq-hls/src/export/rendition.rs:1729: Preserve dropped-group accounting in segment membership; rendition.rs:2154 still records only delivered groups and :1729 fetches only that membership, while quest/m2/time-addressing/lite07.md:14 describes a future completeness mechanism rather than one implemented at this head.
    • Refuted: medium, rs/moq-net/src/model/track.rs:1053: Update JavaScript relative anchors to count sparse groups; The base already contains the maintainer-planned dependency chain in quest/m2/time-addressing/js.md:27 and :36 through lite07.md:93 to the Rust model, establishing that the JavaScript migration was intentionally sequenced separately.

kixelated and others added 30 commits November 13, 2026 03:19
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…op Timestamp::now

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Time starts wait for a group on a hop that resolves its own groups, and a
key prefix that is all gone resolves before it. The front adopts copy-resolved
time starts exactly, and Subscription::end skips empty ranges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oups

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Time anchors are group IDs on every wire, so the tests of resolving a time
end against the cache, refusing an unknown one, and refusing a time anchor
on a wire without timestamps go. A time start past an open newest group
names that group, so a narrowing there still asks from it. One-frame groups
start at their frame. Send-time stamping on lite-05/06 is gone, so its
tests go, and the relayed timestamps are read on groups keyed by them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A copy received over a wire without ranges (lite-03..06, moq-transport)
reads a time start as the first key at or after it, as its publisher does,
and a lite publisher starts its cursor at that floor. A time start before
every group held misses from the time, not the oldest group held. A multi-group
history read on moq-transport subscribes rather than fetching one group per
key. A received copy whose clean end sits at its newest group stops waiting on
the keys below it. Fuzz and bench groups start in key order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recovering stale or evicted history over sparse starts spans the keys
between groups, which the publisher names with SUBSCRIBE_SKIP. The
recovery refused that as a protocol violation and tore the session down;
it now records the skip on the copy and answers a fill owing those keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A copy holds only part of the track, so the newest group it holds before
a time need not contain it. It now starts there only once the keys
between are skipped or a later group arrived, starts at a group starting
at the time, or takes where its publisher reported starting the range it
asked from that time (SUBSCRIBE_START and SUBSCRIBE_STARTED). Until then
the reader reads the groups from the time, which are in range whatever
contains it, and a publisher serving it waits to report its start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A snapshot, window, or stream group starts at its first frame's timestamp
in the track's timescale. A scale-less write starts one past the newest
start the track holds, standing in until every write carries a timestamp.
A refresh whose newest timestamp is still the open group's own start waits
for a later write, since two groups never share a start. A new group is
appended before the previous one closes, so a refused start changes
nothing. Groups end with close(), their end being unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sign and e2ee track producers take the group's start: append_group(start)
and create_group(start) mirror moq-net, and both still refuse a start at or
below one already signed or allocated. Group handles expose start() in place
of sequence(), and close() in place of finish(), mirroring moq-net's group.
The e2ee nonce and the signed coordinates bind the start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stats frames are stamped from the ambient clock's origin (wall time since
the MoQ epoch), not moq-net's model clock. Each snapshot group starts at
its drain's timestamp in the track's milliseconds and is appended, so the
span since the last group is skipped. A track still opens at most one group
per interval, and a drain whose millisecond ties the newest group waits for
the next. Bucket tracks no longer count groups from 0: a run's first group
is where its counters start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The recorder subscribes from Anchor::Time at the next group it wants and
reads group.start; the replayer creates each stored group at its key and
closes it rather than finishing it, since a stored group says nothing
about where the next starts. A look-up by time answers a scale-less track
by key, now that moq-net hands those a bare start, instead of leaving it
unanswered. The storage format still calls a group's key its sequence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Producers key each group by its first frame's timestamp in the track's
timescale. A keyframe tying the open group continues it; with none open it
is refused. Clock-stamped writes (catalog, SI, `timestamp(None)`) step one
unit past the newest start. A discontinuity marker sits one unit below the
live edge, so media resuming at the edge still follows it.

A group cut at a known bound (the next keyframe, an explicit end, a media
track's estimated end, or the marker after it) now ends whole with
`finish(end)`. The container consumer counts that end as presented and
steps past a key gap at once when the next group's first frame or marker
bound continues it. Otherwise it waits the hole budget as it did for a
missing sequence. A resumed source carries its reach into the new
consumer.

`GroupConsumer::sequence()` is now `start()`, and every `seek(start)`
takes a group start that has to pass the starts the track holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The producer stamps frames from this process's monotonic clock in
milliseconds and starts each group at its keyframe, a millisecond past the
last start when two land in one millisecond.

Group starts are times, so the drain counts holes by the group number the
keyframe header already carries rather than by the start. A group that
fails before its header names it is counted once the frontier passes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pullfrog[bot]
pullfrog Bot previously approved these changes Nov 14, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The new commits are two merges from main, resolved against start keys, plus a quest file move and a test format fix.

Reviewed changes

I reviewed the commits between 142e7fa and bcd17e4.

  • Merged main code now uses start keys: the new sim integration tests (quic_sim.rs, sim_client.rs) and the moq-video decoder tests now use group::Info { start } and read group.start. The JS head-fill tests now pass the required Timestamp.unscaled(key) to writeFrame.
  • Fallible timestamp conversions in the merged clocks: moq-mux's declared clock and the FLV and MKV exports now use as_millis()? and as_nanos()?. An unscaled frame now returns an error instead of reaching an infallible conversion.
  • Arrival matching in the moq-video decoder: receive() matches an arrival only when the frame has microseconds. A frame with no scale reports no arrival, and a refused access unit pops the arrival it queued.
  • Quest moved: layered-runs.md moved from m2/time-addressing to m3.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 14, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 14, 2026
@moq-bot

moq-bot Bot commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

quest land watch: the merge queue removed this pull request at bcd17e4 because it conflicts with main (merge_conflict).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

(written by quest land watch)

Carries the start-keyed model into main's chunk-streaming socket (#8741), the
watch include tests, and the CARLA sim's session replies, which now key each
reply group one past the last.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@moq-bot

moq-bot Bot commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

quest land watch: Another session holds this pull request, so the watcher stops acting on it and leaves it to that session: the hold ends the pick, and a maintainer enqueues it again once that session is done.

(written by quest land watch)

pullfrog[bot]
pullfrog Bot previously approved these changes Nov 15, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The new commit merges main and moves the incoming socket byte-stream code onto start keys.

Reviewed changes

I reviewed the merge between bcd17e4 and 82ca7af.

  • Socket pump on keys: a tcp half is one scale-less group at key 0. The pump copies each peer frame's Timestamp::unscaled(0) into the local key-0 group, and ends the half with close(), since the new finish takes an end key. A peer frame stamped with anything else fails create_frame with TimestampMismatch, which resets only that session.
  • moq-tokio TcpStream: AsyncWrite stamps each write with Timestamp::unscaled(writer.start), and poll_shutdown ends the half with close(). The tests now build groups with group::Info { start: 0 }.
  • Merged tests and demos: the socket integration tests, the @moq/watch include tests and the CARLA demo's Python replies now pass the required timestamp. The CARLA replies key each group one past largest().
  • Docs: the moq-net page picks up dial_tcp(server, priority) and accept(priority) from main.

I ran cargo test -p moq-tokio --lib socket (49 passed) and cargo test -p moq-net --features async --test integration socket (31 passed).

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

kixelated and others added 4 commits November 14, 2026 16:25
Main's #8712 resume tests move to start keys. Two of them caught the start-keyed model
diverging from main's:

- A serving copy judged whether a bounded read ran out from where its time start
  resolved, which can lie under its resume point, so a gap the reader gave up there kept
  the read open. The copy now judges from its resume point, as the logical reader judges
  what lies under it.
- A front's reader re-anchored a relative start that reached back to the oldest group
  as a time start naming group 0, so it reported a floor a plain reader did not. It
  re-anchors as the oldest start now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main dropped TCP sockets (#8772) and peer-to-peer (#8771). The track socket's
messages, main's new tests, and the interop socket clients now key each message's
group, since a socket track is scale-less. A lite-07 test helper stamps its frame
with its group's key, as the first frame's absolute delta must.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A segment lists its groups by start, so groups whose pts stalls in one segment
grew its list without bound. Past one more than any request fetches, it only
counts them: such a segment is listed as a gap or refused anyway.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pullfrog[bot]
pullfrog Bot previously approved these changes Nov 15, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The new commits are three merges from main, resolved against start keys, plus a fix that bounds how many group starts a moq-hls segment keeps in memory.

Reviewed changes

I reviewed the commits between 82ca7af and e0761bb, including the hand-resolved hunks of each merge (git show --cc).

  • Bounded segment group lists in moq-hls: grid::Groups now lists at most MAX_SEGMENT_GROUPS + 1 starts and only counts the rest. A publisher whose pts stalls inside one segment can no longer grow that segment's list without bound. Such a segment still reports more groups than max_groups() allows, so row() lists it as a gap and Fetching::Start refuses it, as before.
  • Resume reconciled with the merged mutation tests (#8712): in resume.rs resolve, a time or relative start the copy floored stays Anchor::Time(start). One that reached back to the oldest group becomes Anchor::Oldest. A copy whose failover cap empties its range resolves nothing. is_exhausted and exhausted_for skip groups below a copy's resume point.
  • Sockets without the tcp protocol (#8772): only TrackStream remains. It subscribes from Anchor::Time(0), which covers the whole session track under start keys. The FFI, Python, JS and interop socket tests now write only timestamped frames at key 0.
  • Join backlog (#8761) and group header cap (#8774): moq-mux and moq-audio compare group.start against the joined largest key. The lite Group size-cap test now uses the key field.
  • HLS cut tags (#8778) and transcode session budget (#8725): a cut segment's tag names its last group start (Option<u64>) rather than main's exclusive group end. reaches_live matches Anchor::Time(last) >= edge, and model ends are only Relative or Time, so the check covers every end a fill run can carry.

I ran cargo test -p moq-net --lib --features async (3453 passed) and cargo test -p moq-hls --lib export:: (170 passed).

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The one new commit merges main, and its hand-resolved hunks use start keys and fallible timestamp conversions correctly.

Reviewed changes

I reviewed merge a2942ba against e0761bb, including the hand-resolved hunks (git show --cc).

  • Gateway egress billing (#8777): ExportSource keeps this branch's reached resume point next to the new bill. The FLV export pushes (arrival, record, bill) with the branch's fallible as_millis()?. The TS export uses nanos(decode) and records each frame's charge. The moq-webrtc codec reads as_micros() through moq_mux::Error. The merged origin billing test opens its group with append_group(0) and ends it with close().
  • Path router interest (#8728) and the SendStream ack hook (#8739): both merged without conflicts. Neither reads group keys or timestamps.

I ran cargo check -p moq-net -p moq-mux -p moq-webrtc --all-targets (clean), cargo test -p moq-net --lib --features async billed (3 passed) and cargo test -p moq-webrtc --lib egress (30 passed).

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@moq-bot
moq-bot Bot added this pull request to the merge queue Nov 15, 2026
Merged via the queue into main with commit 7d1bee7 Nov 15, 2026
8 checks passed
@moq-bot
moq-bot Bot deleted the quest/m2/time-addressing/model branch November 15, 2026 08:40
kixelated added a commit that referenced this pull request Nov 15, 2026
- quest/m2/cluster-routes: main declined the fresh publisher's exemption
  (#8735) and deleted its quest, so its link to the switch goes with it;
  item 28 keeps main's text with the switch linked to its PR.
- `waits_out_the_doubt` writes its groups as #8638's start-addressed API
  does (`append_next`, `at_start`, `close`), as the other route-change
  tests now do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant