Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions drafts/draft-lcurley-moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -2096,8 +2096,9 @@ A Group Stream also wins over a stale drop of its sequence that arrives after it
Either way the Group Stream accounts for the sequence, and the next SUBSCRIBE_UPDATE does not reopen it: the publisher, having served it, neither serves it again nor names it in another drop.

For every sequence in the subscription's range, the publisher MUST send a Group Stream or name it in SUBSCRIBE_SKIP or SUBSCRIBE_DROP.
This includes streams reset before their headers were delivered.
A reliably delivered reset retaining the Group Stream header accounts for that group instead, unless it is YIELDED.
This includes streams reset before their headers were delivered, and streams the subscriber stopped (STOP_SENDING), which it may have discarded unread; such a drop carries the STOP_SENDING's Error Code.
Comment thread
kixelated marked this conversation as resolved.
Resolved
A drop naming a stopped duplicate of a Group that another Group Stream already completed changes nothing for that Group: its readers keep it whole.
A reliably delivered reset retaining the Group Stream header accounts for that group instead, unless it is YIELDED or answers a STOP_SENDING.
A [paused](#yield) Group is accounted for only once a continuation completes it, a reset of one other than YIELDED ends it, or a SUBSCRIBE_DROP names it.
The publisher SHOULD name a sequence as soon as it gives it up, without waiting for SUBSCRIBE_FINISH.

Expand Down Expand Up @@ -2488,6 +2489,7 @@ An application-specific payload.
- Added `Error Code` to SUBSCRIBE_FINISH: a track can end with an error while keeping the groups below its end.
- Restored SUBSCRIBE_DROP (type 0x2), replacing Stream Count on SUBSCRIBE_FINISH. Every sequence is accounted for by a Group Stream or a drop; a publisher SHOULD send a drop as soon as it gives a sequence up.
- SUBSCRIBE_DROP's Error Code is a stream error code, which ends the dropped groups' readers, including one still arriving.
- A Group Stream the subscriber stopped (STOP_SENDING) is named in SUBSCRIBE_DROP with the STOP_SENDING's Error Code, even after a reliable reset.
- Refuse a SUBSCRIBE_DROP whose Group Start is greater than Group End plus one on every earlier version that carries it.
- SUBSCRIBE_DROP and SUBSCRIBE_SKIP carry `Group Start` and `Group Count` in place of an inclusive `Group End`. A range past 2^64-1 is a PROTOCOL_VIOLATION.
- Added SUBSCRIBE_SKIP (type 0x3) for sequences the publisher declared it will never produce, which ends no reader and is final: a Group is never created at a skipped sequence. A skip overrides an earlier drop of the same sequences, whatever its Error Code, without re-delivering to readers the drop already ended, and never overrides a Group that arrived. A gap in the sequence numbers is not a skip, since Groups may be created out of order; a sequence never had and never declared skipped is named in SUBSCRIBE_DROP. SUBSCRIBE_DROP no longer names declared skips.
Expand Down
31 changes: 31 additions & 0 deletions js/net/src/lite/publisher.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -894,6 +894,13 @@ async function servedSubscription(
}
return { sequence: header.sequence, frameStart: header.frameStart, payloads };
},
// Stop the next group stream the publisher opened with `code`, before reading any of it,
// as a subscriber that discards a stream unread does.
async stopServed(code: number): Promise<void> {
const next = await opened.read();
if (next.done) throw new Error("the publisher opened no group stream");
await next.value.cancel({ source: "stream", streamErrorCode: code });
},
// The sequence alone, for tests that only care which groups reached the wire.
async servedSequence(): Promise<bigint | undefined> {
return (await this.servedGroup())?.sequence;
Expand Down Expand Up @@ -1249,6 +1256,30 @@ for (const version of [Version.DRAFT_06, Version.DRAFT_07]) {
}
});

// A subscriber may stop a group stream before reading which group it carried, so the drop
// is its only account of it. Mirrors the Rust `a_stopped_group_stream_is_dropped_with_its_stop_code`.
test(`lite ${version}: a stopped group stream is dropped with ${version === Version.DRAFT_07 ? "its stop code" : "no code"}`, async () => {
const sub = await servedSubscription({ version, startGroup: 0n });
try {
// Left open, so only the stop ends its stream.
const group = sub.track.createGroup(0n);
group.writeFrame(textFrame("hello", Timestamp.fromMillis(0)));
await sub.stopServed(StreamCode.Evicted);
for (;;) {
const resp = await decodeSubscribeResponse(sub.client.reader, version);
if (!("drop" in resp)) continue;
expect([resp.drop.start, resp.drop.end, resp.drop.error]).toEqual([
0n,
1n,
version === Version.DRAFT_07 ? StreamCode.Evicted : 0,
]);
break;
}
} finally {
await sub.close();
}
});

test(`lite ${version}: a routed track forwards its serving route's drop`, async () => {
const sub = await servedSubscription({ version, startGroup: 0n, routed: true });
try {
Expand Down
5 changes: 3 additions & 2 deletions quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,8 +247,9 @@ what would need a breaking release had it landed right after the cut:
tier and partition tests run before it (maintainer; rejected: moving it to
m3 and fixing late routing flaws in lite-08).
- ✅ [Group drops](/quest/m2/group-drops.md) split: its lite-07 wire question
is [its own quest](/quest/m2/group-drops-wire.md), which starts now and
gates the freeze; the public `Ordered` API waits for the model.
was its own quest, settled without a wire-format change in
[moq-dev/moq#8774](https://github.com/moq-dev/moq/pull/8774); the public
`Ordered` API waits for the model.
[Catalog at timestamp](/quest/m2/catalog-at-timestamp.md) no longer waits
for `@moq/net` by start, and [the bindings](/quest/m2/time-addressing/bindings.md)
shrink to a rename, since the model already fills each field with the start.
64 changes: 0 additions & 64 deletions quest/m2/group-drops-wire.md

This file was deleted.

16 changes: 11 additions & 5 deletions quest/m2/group-drops.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,16 @@ Planned 2026-11-04 with the maintainer, from
reports a group the subscriber dropped without `recv_group` returning it, so a
reader that paces on delivered-or-evicted groups stalls. Split 2026-11-14 (the
m2 triage before the cut, under /quest-afk, for the maintainer's review):
[the wire half](/quest/m2/group-drops-wire.md) makes every abandoned group
known and decides lite-07's part; this publishes it. It stays in m2 because
the `Ordered` change below is breaking.
the wire half made every abandoned group known and decided lite-07's part;
this publishes it. It stays in m2 because the `Ordered` change below is
breaking.

The wire half ([moq-dev/moq#8774](https://github.com/moq-dev/moq/pull/8774))
found every group the overcommitted cells lose already in the drop set: a
scratch reader polling `poll_drop` beside `recv_group` finished every cell of
the receive-windows benchmark with delivered, evicted-after-header and dropped
groups summing to those sent, on lite-06 and lite-07. So surfacing
`poll_drop` alone unskips those cells.

- ✅ **Milestone** (maintainer, 2026-11-04): it changes public API, so it
lands before the cut.
Expand Down Expand Up @@ -56,12 +63,11 @@ Callers match the new item, and a caller that ignores drops takes the group
arm; the PR's MIGRATE notes say so. `recv_group` and the unordered cursor are
unchanged, so a reader that needs drop accounting reads through `Ordered`.
`@moq/net` and the bindings mirror it only with a named consumer.
Wire: none; [the wire half](/quest/m2/group-drops-wire.md) owns it.
Wire: none.

## Required

- [Address groups by timestamp](/quest/m2/time-addressing/model.md) - the drop range is keyed by start
- [A moq-lite subscriber always learns which group it abandoned](/quest/m2/group-drops-wire.md) - the benchmark's cells finish only once no abandoned group is lost

## Related

Expand Down
11 changes: 6 additions & 5 deletions quest/m2/lite07-finalize.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,10 +147,12 @@ would need a breaking release had it landed right after the cut:
the signed transcript. Rejected: keeping the pin in m2, where the freeze
waits on nixpkgs as well as the cut; dropping the check, which would find a
transcript flaw only after the freeze.
- ✅ The wire half of group drops split out as
[its own quest](/quest/m2/group-drops-wire.md), which this waits on in
place of [the public drop signal](/quest/m2/group-drops.md), an API change
that doesn't touch the wire.
- ✅ The wire half of group drops split out as its own quest, which this
waited on in place of [the public drop signal](/quest/m2/group-drops.md),
an API change that doesn't touch the wire. It settled without a
wire-format change ([moq-dev/moq#8774](https://github.com/moq-dev/moq/pull/8774)):
both publishers already name a Group Stream the subscriber stopped in
SUBSCRIBE_DROP, which the draft now states, even after a reliable reset.

## Required

Expand All @@ -163,7 +165,6 @@ would need a breaking release had it landed right after the cut:
- [Yield on relay links](/quest/m2/yield-relay-links.md) - narrows YIELDED from `ietf` sessions to tracks carrying moq-transport records
- [Subscriber Min Age](/quest/m2/subscriber-min-age.md) - removes SUBSCRIBE's minimum age, which nothing reads
- [AUTH_ERROR retries](/quest/m2/auth-retry-codes.md) - writes down which refusal codes a presenter retries
- [A moq-lite subscriber always learns which group it abandoned](/quest/m2/group-drops-wire.md) - may add SUBSCRIBE_DROP for a stopped group; leaves this list if it takes the no-wire path

## Related

Expand Down
4 changes: 4 additions & 0 deletions rs/moq-net/src/coding/varint.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ pub struct BoundsExceeded;
/// The largest value the QUIC form can carry: `2^62 - 1`.
pub(crate) const MAX_QUIC: u64 = (1 << 62) - 1;

/// The longest varint either form decodes, minimal or not: the leading-ones form's 9 bytes,
/// past QUIC's 8.
pub(crate) const MAX_LEN: usize = 9;

/// How a protocol version lays out a varint on the wire.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum Form {
Expand Down
64 changes: 64 additions & 0 deletions rs/moq-net/src/lite/group.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,17 @@ pub struct Group {
pub frame_start: u64,
}

impl Group {
/// The varints the newest layout carries: Subscribe ID, Group Sequence and Frame Start.
const FIELDS: usize = 3;
}

impl<'a> Message<'a> for Group {
/// Every field at its longest varint, so a header still arriving holds few bytes at its
/// [unidentified](crate::priority::RECV_UNIDENTIFIED) receive rank. Versions before
/// lite-07 carry no more fields, in a shorter varint form.
const MAX_SIZE: usize = Self::FIELDS * crate::coding::varint::MAX_LEN;

fn decode_msg(r: &mut Decoder<'a>, version: Version) -> Result<Self, DecodeError> {
let subscribe = r.varint()?;
let sequence = r.varint()?;
Expand Down Expand Up @@ -91,4 +101,58 @@ mod test {
crate::coding::decode_buf(&mut buf.as_slice(), Version::Lite05, |r, v| Group::decode_msg(r, v)).unwrap();
assert_eq!(got.frame_start, 0);
}

/// The size cap admits the longest header every version can carry: each field at its
/// form's largest value, whose minimal encoding is already that form's longest. Lite-07
/// fills the cap exactly, and one byte more is refused at the length prefix.
#[test]
fn the_longest_header_reads_back_under_the_cap() {
let versions = [
Version::Lite01,
Version::Lite02,
Version::Lite03,
Version::Lite04,
Version::Lite05,
Version::Lite06,
Version::Lite07,
];
for version in versions {
let (max, longest) = match Form::from(version) {
Form::Quic => (crate::coding::varint::MAX_QUIC, 8),
Form::LeadingOnes => (u64::MAX, crate::coding::varint::MAX_LEN),
};
let fields = if version.has_frame_bounds() { 3 } else { 2 };
let msg = Group {
subscribe: max,
sequence: max,
frame_start: if version.has_frame_bounds() { max } else { 0 },
};

let mut body = Vec::new();
msg.encode_msg(&mut Encoder::new(&mut body, version.into()), version)
.unwrap();
assert_eq!(body.len(), fields * longest, "{version:?}");
assert!(body.len() <= <Group as Message>::MAX_SIZE, "{version:?}");

let mut wire = Vec::new();
msg.encode(&mut Encoder::new(&mut wire, version.into()), version)
.unwrap();
let (got, used) = Group::decode_slice(&wire, version).unwrap();
assert_eq!(used, wire.len(), "{version:?}");
assert_eq!(
(got.subscribe, got.sequence, got.frame_start),
(msg.subscribe, msg.sequence, msg.frame_start),
"{version:?}"
);
}

assert_eq!(<Group as Message>::MAX_SIZE, 27);
let over = <Group as Message>::MAX_SIZE + 1;
let mut wire = vec![over as u8];
wire.resize(1 + over, 0);
assert!(matches!(
Group::decode_slice(&wire, Version::Lite07),
Err(DecodeError::MessageTooLarge { size: 28, max: 27 })
));
}
}
38 changes: 38 additions & 0 deletions rs/moq-net/src/lite/publisher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9344,6 +9344,44 @@ mod serve_group_test {
assert_eq!(drops(&mut writer, &log, version).await, [(1, 1)]);
}

/// A group stream the subscriber stopped is named in SUBSCRIBE_DROP, with the stop's code
/// where the wire carries one: a stream stopped before its header was read leaves the
/// subscriber no other way to learn which group it carried.
#[kio::test]
async fn a_stopped_group_stream_is_dropped_with_its_stop_code() {
for version in [Version::Lite05, Version::Lite06, Version::Lite07] {
let mut track = track::Producer::new(
Arc::new(broadcast::Info::default()),
"test",
crate::track::Info::timed(),
);
let subscriber = track.subscribe(None);
let evicted = version.encode_stream_code(&crate::StreamError::Evicted);
let session = SinkSession::new(Log::default()).with_stopped_unis(evicted);
let (mut run, mut writer, log) = lite_run(version, session, subscriber);
write_group(&mut track, 0, 0);
while run.poll_step(&mut writer, &kio::Waiter::noop()).is_ready() {}

kio::wait(|waiter| writer.poll_flush(waiter)).await.unwrap();
let writes = log.writes.lock().unwrap().clone();
let mut buf = writes.as_slice();
let mut dropped = Vec::new();
while !buf.is_empty() {
let (msg, size) = lite::SubscribeResponse::decode_slice(buf, version).unwrap();
buf = &buf[size..];
if let lite::SubscribeResponse::Drop(drop) = msg {
dropped.push((drop.groups.clone(), drop.error(version)));
}
}
let error = match version.has_drop_code() {
true => crate::StreamError::Evicted,
// An application code, which this side reads as stale.
false => crate::StreamError::Old,
};
assert_eq!(dropped, [(0..1, error)], "{version:?}");
}
}

/// A wide upstream DROP is held and forwarded as a range, never one entry per sequence,
/// and a wider one later forwards only what is new.
#[kio::test]
Expand Down
23 changes: 15 additions & 8 deletions rs/moq-net/src/lite/subscriber.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1675,11 +1675,10 @@ impl Drop for SetupWait {

#[kio::machine(self.state: GroupRecvState)]
impl GroupRecv {
fn new(subscriber: Subscriber, mut reader: Reader<crate::transport::boxed::RecvStream, Version>) -> Self {
// Until its group opens, the stream may wait unread on the peer's SETUP or its
// subscription's TRACK_INFO, so a full window stops it before the stream either
// arrives on.
reader.set_priority(crate::priority::RECV_PARKED);
/// The stream keeps its [unidentified](crate::priority::RECV_UNIDENTIFIED) rank until its
/// header is read, and ranks [parked](crate::priority::RECV_PARKED) only while it waits
/// unread, so a full window stops what holds the most first.
fn new(subscriber: Subscriber, reader: Reader<crate::transport::boxed::RecvStream, Version>) -> Self {
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
Self {
subscriber,
reader,
Expand All @@ -1693,12 +1692,16 @@ impl GroupRecv {
let header: (bool, lite::Group) = wait!(Header => {
// Before the header, which a wait after it would lose.
let Poll::Ready(ietf) = self.subscriber.peer_setup.poll_ietf(waiter) else {
let waits = &self.subscriber.setup_waits;
self.setup_wait.get_or_insert_with(|| SetupWait::new(waits));
if self.setup_wait.is_none() {
self.setup_wait = Some(SetupWait::new(&self.subscriber.setup_waits));
self.reader.set_priority(crate::priority::RECV_PARKED);
}
// Unread meanwhile, so a reset or an eviction would otherwise go unseen.
return self.reader.poll_reset(waiter).map(Err);
};
self.setup_wait = None;
if self.setup_wait.take().is_some() {
self.reader.set_priority(crate::priority::RECV_UNIDENTIFIED);
}
self.reader.poll_decode::<lite::Group>(waiter).map_ok(|hdr| (ietf, hdr))
})?;

Expand All @@ -1709,6 +1712,7 @@ impl GroupRecv {
match subs.get_mut(&header.1.subscribe) {
Some(entry) if entry.copy.is_none() => {
self.park_wait = Some(ParkWait::new(&self.subscriber.park_waits));
self.reader.set_priority(crate::priority::RECV_PARKED);
Some(entry.park(&header.1))
}
_ => None,
Expand Down Expand Up @@ -1748,6 +1752,9 @@ impl GroupRecv {
}
};
let mut open: Option<Open> = Some(open?);
if matches!(open, Some(Open::Duplicate(_))) {
self.reader.set_priority(crate::priority::RECV_PARKED);
}
// A duplicate of a copy still arriving waits, unread, for that copy to settle.
let opened: Opened = wait!(Hold => loop {
let mut duplicate = match open.take() {
Expand Down
Loading