diff --git a/quest/m1/cut-audit/README.md b/quest/m1/cut-audit/README.md index a84b26c68c0..cd6d5c031ff 100644 --- a/quest/m1/cut-audit/README.md +++ b/quest/m1/cut-audit/README.md @@ -54,7 +54,7 @@ leaving JS-only redesigns of `js/net` to [rs2ts](/quest/m2/rs2ts/README.md). ## Required -- [One fMP4 sample resolver](/quest/m1/cut-audit/fmp4-sample-resolver.md) - zero-size samples cannot cost gigabytes, and `base_data_offset` input imports or is refused +- [Counted field-less `trun` entries](/quest/m1/cut-audit/fmp4-sample-resolver.md) - a moof of field-less runs cannot materialize millions of sample entries - [A compressed announce path never outgrows its literal form](/quest/m1/cut-audit/announce-path-bytes.md) - lite-07 announce compression cannot build multi-MiB paths - [A route too large to re-encode is skipped](/quest/m1/cut-audit/announce-reencode.md) - one oversized route never resets a downstream announce stream - [A reserved group survives a route returning](/quest/m1/cut-audit/reserved-route-return.md) - a route flip A to B to A no longer panics a peeked reader diff --git a/quest/m1/cut-audit/fmp4-sample-resolver.md b/quest/m1/cut-audit/fmp4-sample-resolver.md index 444e1852fbe..2d871671ee3 100644 --- a/quest/m1/cut-audit/fmp4-sample-resolver.md +++ b/quest/m1/cut-audit/fmp4-sample-resolver.md @@ -1,45 +1,43 @@ -# [M] One fMP4 sample resolver for import and decode +# [M] Counted field-less trun entries ## Goal -The fMP4 importer and the CMAF subscriber decode resolve samples through -one resolver that enforces non-empty, in-bounds, disjoint, and correctly -based samples. A moof of field-less truns can no longer cost gigabytes and -hundreds of thousands of groups, and ffmpeg output that uses -`base_data_offset` (the common `frag_keyframe+empty_moov` pipe recipe) -imports or is refused with a typed error instead of failing on a bogus -range. +Parsing a moof allocates in proportion to its bytes: a `trun` whose samples +inherit every field from `tfhd`/`trex` decodes as a count, not one +materialized entry per sample. Land that counted representation in +kixelated/mp4-atom, release it, and adopt it in moq-mux so field-less runs +cannot amplify before the sample resolver sees the mdat. ## Plan -Found by the 2026-10-24 pre-cut audit at 151a529670. +[moq-dev/moq#6864](https://github.com/moq-dev/moq/pull/6864) put fMP4 +import, subtitle import, and CMAF decode on one lazy `sample::Resolver`. It +refuses empty samples, out-of-bounds or overlapping ranges, and absolute +`base_data_offset`, so output is bounded by the mdat. The parser is not: +mp4-atom 0.16.2 pushes one `TrunEntry` per sample even when the run carries +no per-sample field, capped only at 4096 per run. A 16 KB moof of 1000 such +runs materializes 4.1 million entries before the resolver refuses anything. -- **Zero-size samples** (reproduced; per-sample fan-out new on main from - [moq-dev/moq#6250](https://github.com/moq-dev/moq/pull/6250)): a moof of - 1000 field-less truns of 4096 entries with tfhd default size 0 is 4.1M - samples in 16 KB, since mp4-atom 0.16.2 materializes 4096 entries for a - trun without per-entry fields. Through import it took 6.3 GB RSS, 66-81 s - of CPU, and about 720k groups. Paths: - `rs/moq-mux/src/container/fmp4/import.rs:1052` (`with_capacity(total_samples)`), - `:1095-1128` (a 0-byte sample passes), `fmp4/sample.rs:87` - (`Claimed::take` returns early on empty), `import.rs:1380` (each sync - sample opens a group), and the subscriber-side `Wire::decode` - (`fmp4/mod.rs:533`), reached by any Rust subscriber of a cmaf track. -- **`base_data_offset`** (reproduced with ffmpeg 9.0.2, inherited): - `import.rs:1049-1069` and `fmp4/text.rs:72` treat the absolute file offset - as an mdat-relative one, failing the first fragment with - `SampleRangeOutOfBounds`. Resolve it against the moof's absolute position - with a per-source byte counter, or refuse it with a typed error as - `decode()` does at `mod.rs:523`. +Settled (2026-10-26, maintainer): a counted representation, not a refusal. +[kixelated/mp4-atom#248](https://github.com/kixelated/mp4-atom/pull/248) +refuses nonempty field-less runs and breaks the AV1, BBB, and ESDS fixtures, +which are valid files; close or rework it rather than weaken those tests. Do +not add another sample-count cap or duplicate the atom parser in moq-mux. -Fix: refuse zero-size media samples in the shared resolver, so each sample -costs at least one input byte, and make field-less truns bounded by design -in mp4-atom (refuse, or store a count) instead of its 4096 constant. The two -sample loops have drifted through repeated fixes -([moq-dev/moq#5078](https://github.com/moq-dev/moq/pull/5078), -[moq-dev/moq#6379](https://github.com/moq-dev/moq/pull/6379), -[moq-dev/moq#6686](https://github.com/moq-dev/moq/pull/6686)); one resolver -is a net deletion. If the mp4-atom change needs a release, land the -resolver here and track the bump separately. +- mp4-atom: replace `Trun.entries: Vec` with a shape that stores a + field-less run (including a first-sample-flags-only run) as a count and + yields entries through an iterator, without allocating from + `sample_count`. Encode must round-trip both forms. The API shape is the + maintainer's call; propose it in the mp4-atom PR. This is breaking, so + aim for the 0.17.0 release that + [VR fields](/quest/m2/mux-vr-fields.md) is waiting on. +- moq-mux re-exports `mp4_atom`, so the bump is a breaking change to its + public API. Migrate `sample::Resolver::resolve`, the segment-start scan, + and `kept`/`fragment` in `import.rs`, plus the tests that build `Trun` + literals. +- Verify with the fMP4 tests and the interleaved container benchmark, + including its 1-byte samples, and a test that a moof of many field-less + runs decodes without a per-sample allocation. -Public API: possibly a new `fmp4::Error` variant. Wire: none. +Public API: breaking `mp4_atom::Trun` through moq-mux's re-export. MoQ wire: +none. diff --git a/quest/m2/media-fuzz.md b/quest/m2/media-fuzz.md index 0a40f634900..aa8ea8fbcc9 100644 --- a/quest/m2/media-fuzz.md +++ b/quest/m2/media-fuzz.md @@ -13,7 +13,7 @@ From the 2026-10-24 pre-cut audit at 151a529670. `rs/moq-mux` and `rs/hang` have no fuzz targets; only `rs/moq-net/fuzz` and `rs/moq-quic/fuzz` exist, and [wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) puts media importers out of scope. The audit's -[zero-size sample](/quest/m1/cut-audit/fmp4-sample-resolver.md) and +[zero-size sample](https://github.com/moq-dev/moq/pull/6864) and malformed avc1 codec-string findings are the kind `cargo fuzz` finds in minutes; they are reachable through RTMP, SRT, HLS, and fMP4 ingest and through catalog subscription. Include RTMP's AMF0 diff --git a/rs/moq-mux/benches/containers.rs b/rs/moq-mux/benches/containers.rs index b2d13b25d39..33e240bf4ab 100644 --- a/rs/moq-mux/benches/containers.rs +++ b/rs/moq-mux/benches/containers.rs @@ -259,27 +259,30 @@ fn interleaved_runs(c: &mut Criterion) { )); let init = runtime.block_on(export("cmaf", source, stream, true)); drop(fixture); - for runs in [1, 16] { - let data = interleaved(&init, tracks, runs, frames, &payload); - for passthrough in [false, true] { - let (catalog, import) = import_cmaf(cmaf_import(passthrough), &data); - assert_eq!(catalog.snapshot().video.renditions.len(), tracks); - drop(import); - } - for (direction, passthrough) in [("import", true), ("unwrap", false)] { - let mut group = c.benchmark_group(format!("cmaf-interleaved-{direction}")); - group.throughput(Throughput::Bytes(data.len() as u64)); - group.bench_function( - BenchmarkId::new(format!("{tracks}tracks"), format!("{runs}runs")), - |b| { - b.iter_batched( - || cmaf_import(passthrough), - |import| import_cmaf(import, std::hint::black_box(&data)), - BatchSize::PerIteration, - ); - }, - ); - group.finish(); + for size in [1, payload.len()] { + let sample = payload.slice(..size); + for runs in [1, 16] { + let data = interleaved(&init, tracks, runs, frames, &sample); + for passthrough in [false, true] { + let (catalog, import) = import_cmaf(cmaf_import(passthrough), &data); + assert_eq!(catalog.snapshot().video.renditions.len(), tracks); + drop(import); + } + for (direction, passthrough) in [("import", true), ("unwrap", false)] { + let mut group = c.benchmark_group(format!("cmaf-interleaved-{direction}")); + group.throughput(Throughput::Bytes(data.len() as u64)); + group.bench_function( + BenchmarkId::new(format!("{tracks}tracks"), format!("{runs}runs-{size}bytes")), + |b| { + b.iter_batched( + || cmaf_import(passthrough), + |import| import_cmaf(import, std::hint::black_box(&data)), + BatchSize::PerIteration, + ); + }, + ); + group.finish(); + } } } } diff --git a/rs/moq-mux/src/container/fmp4/import.rs b/rs/moq-mux/src/container/fmp4/import.rs index 9e58d677b7c..534c1e23f6b 100644 --- a/rs/moq-mux/src/container/fmp4/import.rs +++ b/rs/moq-mux/src/container/fmp4/import.rs @@ -69,7 +69,8 @@ pub struct Import { // The latest moof header moof: Option, - moof_size: usize, + // Bytes consumed since that moof began, including intervening boxes. + moof_span: usize, // Bytes carried across calls: a partial atom at the tail of one `decode` waits // here for the rest to arrive on the next call. @@ -249,7 +250,7 @@ impl Import { skipped: HashSet::default(), moov: None, moof: None, - moof_size: 0, + moof_span: 0, broadcast, buffer: BytesMut::new(), pending_cut: false, @@ -313,6 +314,9 @@ impl Import { } /// Decode a buffer of bytes. + /// + /// Samples must be nonempty and use moof-relative offsets; absolute `base_data_offset` + /// addressing is unsupported. With ffmpeg, use `default_base_moof`. pub fn decode(&mut self, data: &[u8]) -> Result<()> { self.buffer.extend_from_slice(data); self.drain() @@ -345,6 +349,9 @@ impl Import { let consumed = self.buffer.split_to(position).freeze(); for (atom, start, size) in parsed { + if self.moof.is_some() { + self.moof_span = self.moof_span.checked_add(size).ok_or(Error::InvalidDataOffset)?; + } match atom { Any::Ftyp(_) => {} Any::Emsg(_) => { @@ -364,7 +371,7 @@ impl Import { return Err(Error::DuplicateMoof.into()); } self.moof.replace(moof); - self.moof_size = size; + self.moof_span = size; } Any::Mdat(mdat) => { let raw = consumed.slice(start..start + size); @@ -922,7 +929,6 @@ impl Import { fn extract(&mut self, mdat: Mdat, raw: Bytes) -> Result<()> { let has_video = self.tracks.values().any(|t| t.kind == Kind::Video); let moof = self.moof.take().ok_or(Error::NoMoof)?; - let moof_size = self.moof_size; let header_size = raw.len() - mdat.data.len(); // The mdat payload as a view of the input, so each unwrapped sample is sliced, not copied. let payload = raw.slice(header_size..); @@ -942,7 +948,12 @@ impl Import { return Err(Error::NoMoov.into()); } - let mut claimed = sample::Claimed::default(); + // Offsets include any boxes between moof and mdat, across decode calls too. + let data_start = self + .moof_span + .checked_sub(payload.len()) + .ok_or(Error::InvalidDataOffset)?; + let mut resolver = sample::Resolver::new(data_start, payload.len()); // Loop over all of the traf boxes in the moof. for traf in &moof.traf { @@ -952,9 +963,7 @@ impl Import { super::text::Fragment { traf, data: &payload, - moof_size, - header_size, - claimed: &mut claimed, + resolver: &mut resolver, }, &mut self.timebase, )?; @@ -971,12 +980,9 @@ impl Import { // A fragment wholly before the edit or the first sync sample, such as priming samples or // a mid-GOP join. Skipped before the segment is counted as opened, or the next keyframe // would roll a second segment. - let Some(run) = track.run(traf, moof_size, header_size, payload.len())? else { + let Some(run) = track.run(traf, &mut resolver)? else { continue; }; - for sample in &run.samples { - claimed.take(sample.range.clone())?; - } // Shift onto the catalog clock. Anchor before releasing the reservation, so the first // snapshot carries the final clock; the moov declared every track, so any track's @@ -1018,9 +1024,8 @@ struct Placement { } impl Fmp4Track { - /// Resolve the samples `traf` places in an mdat whose payload is `len` bytes, or `None` when - /// the edit list drops every one. - fn run(&mut self, traf: &mp4_atom::Traf, moof_size: usize, header_size: usize, len: usize) -> Result> { + /// Apply this track's edit list to the resolved samples, or return `None` if none remain. + fn run(&mut self, traf: &mp4_atom::Traf, resolver: &mut sample::Resolver) -> Result> { let tfdt = traf.tfdt.as_ref().ok_or(Error::MissingTfdt)?; let timescale = self.timescale; let mut dts = tfdt.base_media_decode_time; @@ -1040,100 +1045,51 @@ impl Fmp4Track { self.last_decode_time = Some(dts); - if traf.trun.is_empty() { - return Err(Error::MissingTrun.into()); - } - - let tfhd = &traf.tfhd; - let mut offset = tfhd.base_data_offset.unwrap_or_default() as usize; let mut earliest = None; let mut end = None; - let total_samples: usize = traf.trun.iter().map(|t| t.entries.len()).sum(); - let mut samples = Vec::with_capacity(total_samples); - let mut index = 0; + let mut samples = Vec::new(); let mut dropped = false; - - for trun in &traf.trun { - if let Some(data_offset) = trun.data_offset { - let base_offset = tfhd.base_data_offset.unwrap_or_default() as usize; - let data_offset: usize = data_offset.try_into().map_err(|_| Error::InvalidDataOffset)?; - - let relative_offset = data_offset - .checked_sub(moof_size) - .and_then(|v| v.checked_sub(header_size)) - .ok_or(Error::InvalidDataOffset)?; - - offset = base_offset - .checked_add(relative_offset) - .ok_or(Error::InvalidDataOffset)?; + for resolved in resolver.resolve(traf, self.defaults)? { + let sample::Resolved { sample, cts, range } = resolved?; + let duration = sample.duration; + + // Checked: a negative composition offset must not wrap into a huge u64 PTS. + let pts = dts.checked_add_signed(cts as i64).ok_or(Error::PtsOverflow)?; + let duration_ts = duration + .map(|duration| Timestamp::new(duration as u64, timescale)) + .transpose()?; + if let Some(duration) = duration_ts { + self.sample_duration = Some(self.sample_duration.map_or(duration, |min| min.min(duration))); } - for entry in &trun.entries { - let sample = self.defaults.resolve(tfhd, entry); - let duration = sample.duration; - - // A non-final sample with no resolvable duration leaves the rest of the - // fragment's DTS ambiguous, so reject it rather than collapse timestamps. - if duration.is_none() && index + 1 != total_samples { - return Err(Error::MissingSampleDuration.into()); - } - - // Checked: a negative composition offset must not wrap into a huge u64 PTS. - let pts = dts - .checked_add_signed(entry.cts.unwrap_or_default() as i64) - .ok_or(Error::PtsOverflow)?; - let duration_ts = duration - .map(|duration| Timestamp::new(duration as u64, timescale)) - .transpose()?; - if let Some(duration) = duration_ts { - self.sample_duration = Some(self.sample_duration.map_or(duration, |min| min.min(duration))); - } - - // Each sample must lie inside the mdat, or the sizes and offsets disagree with the - // data and it cannot be addressed safely. - let next = offset - .checked_add(sample.size as usize) - .ok_or(Error::InvalidDataOffset)?; - if next > len { - return Err(Error::SampleRangeOutOfBounds { - start: offset, - end: next, - len, - } - .into()); - } - - let presented = self.edit.present(pts)?; - if sample.is_sync() { - self.orphaned = presented.is_none(); - } - if let Some(presented) = presented.filter(|_| !self.orphaned) { - let timestamp = Timestamp::new(presented, timescale)?; - let sample_end = match duration_ts.or(self.sample_duration) { - Some(duration) => timestamp.checked_add(duration)?, - None => timestamp, - }; - end = Some(end.map_or(sample_end, |max: Timestamp| max.max(sample_end))); - earliest = Some(earliest.map_or(timestamp, |min: Timestamp| min.min(timestamp))); - - samples.push(Sample { - dts, - pts, - timestamp, - duration, - flags: sample.flags, - sync: sample.is_sync(), - range: offset..next, - }); - } else { - dropped = true; - } + let presented = self.edit.present(pts)?; + if sample.is_sync() { + self.orphaned = presented.is_none(); + } + if let Some(presented) = presented.filter(|_| !self.orphaned) { + let timestamp = Timestamp::new(presented, timescale)?; + let sample_end = match duration_ts.or(self.sample_duration) { + Some(duration) => timestamp.checked_add(duration)?, + None => timestamp, + }; + end = Some(end.map_or(sample_end, |max: Timestamp| max.max(sample_end))); + earliest = Some(earliest.map_or(timestamp, |min: Timestamp| min.min(timestamp))); + + samples.push(Sample { + dts, + pts, + timestamp, + duration, + flags: sample.flags, + sync: sample.is_sync(), + range, + }); + } else { + dropped = true; + } - if let Some(duration) = duration { - dts = dts.checked_add(duration as u64).ok_or(Error::PtsOverflow)?; - } - offset = next; - index += 1; + if let Some(duration) = duration { + dts = dts.checked_add(duration as u64).ok_or(Error::PtsOverflow)?; } } @@ -1176,10 +1132,8 @@ fn fragment( // Apply structural (flag-presence) changes BEFORE measuring the encoded // size, otherwise the rewritten data_offset values are computed from a // stale size and the resulting fragment misaddresses sample data. - // In particular: clearing tfhd.base_data_offset removes 8 bytes per traf, - // and ensuring trun.data_offset is Some(...) reserves 4 bytes per trun. + // Ensuring trun.data_offset is Some(...) reserves 4 bytes per trun. for traf_mut in &mut adjusted_moof.traf { - traf_mut.tfhd.base_data_offset = None; // A zero default/sample duration is "unknown", not "instantaneous": drop it so // the re-emitted fragment carries no bogus zero that a decoder would honor. if traf_mut.tfhd.default_sample_duration == Some(0) { diff --git a/rs/moq-mux/src/container/fmp4/import_test.rs b/rs/moq-mux/src/container/fmp4/import_test.rs index b3904269fe0..84dbc6970dc 100644 --- a/rs/moq-mux/src/container/fmp4/import_test.rs +++ b/rs/moq-mux/src/container/fmp4/import_test.rs @@ -630,12 +630,16 @@ async fn separated_runs_keep_their_samples() { // Offsets count from the moof's first byte, so measure it with every offset present. let mut buf = Vec::new(); moof.encode(&mut buf).unwrap(); - let data = buf.len() as i32 + 8; + let padding = mp4_atom::Any::Unknown(mp4_atom::FourCC::new(b"free"), vec![0; 16]); + let mut padding_bytes = Vec::new(); + padding.encode(&mut padding_bytes).unwrap(); + let data = (buf.len() + padding_bytes.len() + 8) as i32; moof.traf[0].trun[0].data_offset = Some(data + at(0) as i32); moof.traf[0].trun[1].data_offset = Some(data + at(5) as i32); moof.traf[1].trun[0].data_offset = Some(data + at(3) as i32); let mut fragment = Vec::new(); moof.encode(&mut fragment).unwrap(); + fragment.extend_from_slice(&padding_bytes); mp4_atom::Mdat { data: layout.iter().flat_map(|(_, bytes)| bytes.iter().copied()).collect(), } @@ -665,7 +669,9 @@ async fn separated_runs_keep_their_samples() { let catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); let mut fmp4 = crate::container::fmp4::Import::new(broadcast, catalog.reserve()).with_passthrough(passthrough); fmp4.decode(&init).unwrap(); - fmp4.decode(&fragment).unwrap(); + for chunk in fragment.chunks(7) { + fmp4.decode(chunk).unwrap(); + } fmp4.finish().unwrap(); let [got_video, got_audio] = published(&consumer, &catalog).await; assert_eq!(got_video, video, "video, passthrough {passthrough}"); @@ -673,6 +679,103 @@ async fn separated_runs_keep_their_samples() { } } +/// A sample must own input bytes even when its size comes from a default. +#[test] +fn empty_samples_are_refused() { + let (init, (video_id, timescale), _) = bbb_init(); + for explicit in [false, true] { + let moof = mp4_atom::Moof { + mfhd: mp4_atom::Mfhd { sequence_number: 1 }, + traf: vec![mp4_atom::Traf { + tfhd: mp4_atom::Tfhd { + track_id: video_id, + default_sample_size: Some(0), + default_sample_duration: Some(1), + ..Default::default() + }, + tfdt: Some(mp4_atom::Tfdt { + base_media_decode_time: 0, + }), + trun: vec![mp4_atom::Trun { + data_offset: None, + entries: vec![ + mp4_atom::TrunEntry { + size: explicit.then_some(0), + ..Default::default() + }; + 4 + ], + }], + ..Default::default() + }], + }; + let mut fragment = Vec::new(); + moof.encode(&mut fragment).unwrap(); + mp4_atom::Mdat { data: vec![] }.encode(&mut fragment).unwrap(); + for passthrough in [false, true] { + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let mut import = super::Import::new(broadcast, catalog.reserve()).with_passthrough(passthrough); + import.decode(&init).unwrap(); + assert!( + matches!( + import.decode(&fragment), + Err(crate::Error::Cmaf(super::Error::EmptySample)) + ), + "empty import: explicit={explicit}, passthrough={passthrough}" + ); + } + assert!(matches!( + super::decode(fragment.into(), None, super::Track::new(timescale, super::Kind::Video)), + Err(super::Error::EmptySample) + )); + } +} + +/// A file-relative base is explicitly unsupported on both input paths. +#[test] +fn absolute_sample_bases_are_refused() { + let (init, (video_id, timescale), _) = bbb_init(); + for base in [0, 1024, u64::MAX] { + let moof = mp4_atom::Moof { + mfhd: mp4_atom::Mfhd { sequence_number: 1 }, + traf: vec![mp4_atom::Traf { + tfhd: mp4_atom::Tfhd { + track_id: video_id, + base_data_offset: Some(base), + default_sample_size: Some(1), + ..Default::default() + }, + tfdt: Some(mp4_atom::Tfdt { + base_media_decode_time: 0, + }), + trun: vec![mp4_atom::Trun { + entries: vec![Default::default()], + ..Default::default() + }], + ..Default::default() + }], + }; + let mut fragment = Vec::new(); + moof.encode(&mut fragment).unwrap(); + mp4_atom::Mdat { data: vec![1] }.encode(&mut fragment).unwrap(); + for passthrough in [false, true] { + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let mut import = super::Import::new(broadcast, catalog.reserve()).with_passthrough(passthrough); + import.decode(&init).unwrap(); + assert!(matches!( + import.decode(&fragment), + Err(crate::Error::Cmaf(super::Error::UnsupportedBaseDataOffset)) + )); + } + assert!(matches!( + super::decode(fragment.into(), None, super::Track::new(timescale, super::Kind::Video)), + Err(super::Error::UnsupportedBaseDataOffset) + )); + } +} + /// A run pointing at bytes an earlier sample took is refused, passthrough or unwrapped, whether /// that sample is in the same track or another, rather than copying or publishing those bytes once /// per run: a moof of many such runs would multiply its mdat. diff --git a/rs/moq-mux/src/container/fmp4/mod.rs b/rs/moq-mux/src/container/fmp4/mod.rs index b38d42b8baa..da8926f3043 100644 --- a/rs/moq-mux/src/container/fmp4/mod.rs +++ b/rs/moq-mux/src/container/fmp4/mod.rs @@ -187,6 +187,14 @@ pub enum Error { #[error("invalid data offset")] InvalidDataOffset, + /// Absolute file offsets cannot be resolved in a standalone fragment. + #[error("absolute base_data_offset is unsupported; use moof-relative offsets")] + UnsupportedBaseDataOffset, + + /// Every media sample must contain at least one byte. + #[error("empty media sample")] + EmptySample, + /// The frame timestamp carries a fragment's earliest presentation time exactly only when the /// track counts in the same ticks as the media. #[error("track timescale {track:?} doesn't match the CMAF mdhd timescale {mdhd:?}")] @@ -518,80 +526,35 @@ pub(crate) fn decode(data: Bytes, timestamp: Option, track: Track) -> let (mdat_start, mdat_data) = mdat.ok_or(Error::NoMdat)?; let traf = moof.traf.first().ok_or(Error::NoTraf)?; - // Samples are read from the mdat front to back, so each run must start where the previous - // one ended. A run's data_offset counts from the moof's first byte (CMAF's default-base-is-moof). - if traf.tfhd.base_data_offset.is_some() { - return Err(Error::InvalidDataOffset); - } let data_start = mdat_start.checked_sub(moof_start).ok_or(Error::InvalidDataOffset)?; let tfdt = traf.tfdt.as_ref().ok_or(Error::NoTfdt)?; let base_dts = tfdt.base_media_decode_time; - - // DTS is reconstructed by accumulating each sample's duration. A non-final sample - // with no resolvable duration would leave every following sample stuck at the same - // DTS, silently collapsing their timestamps, so reject that fragment instead. - let total_samples: usize = traf.trun.iter().map(|t| t.entries.len()).sum(); + let mut resolver = sample::Resolver::new(data_start, mdat_data.len()); // Rounded the way `encode` stamps it, so a timestamp the track carried at another scale // lands back on the tick it was written at. let anchor = timestamp.map(|t| timestamp_ticks(t, timescale)).transpose()?; let mut frames = Vec::new(); - let mut ptss = Vec::with_capacity(total_samples); - let mut offset = 0usize; + let mut ptss = Vec::new(); let mut dts = i128::from(base_dts); - let mut sample_index = 0usize; - - for trun in &traf.trun { - if let Some(data_offset) = trun.data_offset - && usize::try_from(data_offset).ok() != Some(data_start + offset) - { - return Err(Error::InvalidDataOffset); - } - - for entry in &trun.entries { - let sample = defaults.resolve(&traf.tfhd, entry); - let end = offset + sample.size as usize; - - if end > mdat_data.len() { - return Err(Error::SampleRangeOutOfBounds { - start: offset, - end, - len: mdat_data.len(), - }); - } - - ptss.push(dts + i128::from(entry.cts.unwrap_or_default())); - let payload = Bytes::copy_from_slice(&mdat_data[offset..end]); - let keyframe = kind == Kind::Video && sample.is_sync(); - - // Carry the sample-duration through at the track's scale when present, so - // the jitter buffer can use it and an exporter can write it back. - let sample_duration = sample.duration; - - // The last sample needs no duration (nothing follows it to time), but any - // earlier sample without one makes the rest of the fragment's DTS ambiguous. - let is_last = sample_index + 1 == total_samples; - if sample_duration.is_none() && !is_last { - return Err(Error::MissingSampleDuration); - } - - let duration = sample_duration - .map(|d| Timestamp::new(d as u64, timescale)) - .transpose()?; - - frames.push(Frame { - // Placeholder until the earliest PTS is known. - timestamp: Timestamp::ZERO, - payload, - keyframe, - duration, - }); - - offset = end; - dts += i128::from(sample_duration.unwrap_or(0)); - sample_index += 1; - } + for resolved in resolver.resolve(traf, defaults)? { + let sample::Resolved { sample, cts, range } = resolved?; + ptss.push(dts + i128::from(cts)); + let payload = Bytes::copy_from_slice(&mdat_data[range]); + let keyframe = kind == Kind::Video && sample.is_sync(); + let duration = sample + .duration + .map(|d| Timestamp::new(d as u64, timescale)) + .transpose()?; + frames.push(Frame { + // Placeholder until the earliest PTS is known. + timestamp: Timestamp::ZERO, + payload, + keyframe, + duration, + }); + dts += i128::from(sample.duration.unwrap_or(0)); } let (anchor, earliest) = match anchor { @@ -1950,10 +1913,9 @@ mod tests { assert!(matches!(err, Error::TimescaleMismatch { .. }), "got {err:?}"); } - /// Samples are read front to back, so a run whose data_offset points anywhere but the next - /// sample is refused rather than sliced from the wrong bytes. + /// A run whose samples extend beyond the payload is refused. #[test] - fn decode_refuses_a_run_that_skips_bytes() { + fn decode_refuses_a_run_past_the_payload() { use mp4_atom::{DecodeMaybe, Encode}; let timescale = moq_net::Timescale::MICRO; @@ -1972,7 +1934,7 @@ mod tests { skewed.extend_from_slice(mdat); let err = decode(Bytes::from(skewed), None, Track::new(timescale, Kind::Video)).unwrap_err(); - assert!(matches!(err, Error::InvalidDataOffset), "got {err:?}"); + assert!(matches!(err, Error::SampleRangeOutOfBounds { .. }), "got {err:?}"); } /// A frame timestamp the track carries at another scale anchors to the nearest tick, the way diff --git a/rs/moq-mux/src/container/fmp4/sample.rs b/rs/moq-mux/src/container/fmp4/sample.rs index a7582926ad7..261114c4090 100644 --- a/rs/moq-mux/src/container/fmp4/sample.rs +++ b/rs/moq-mux/src/container/fmp4/sample.rs @@ -74,18 +74,110 @@ impl Sample { } } +/// Resolve one fragment's samples against its mdat, sharing byte ownership across tracks. +pub(crate) struct Resolver { + data_start: usize, + len: usize, + claimed: Claimed, +} + +/// A sample whose byte range and duration have been validated. +pub(crate) struct Resolved { + pub sample: Sample, + pub cts: i32, + pub range: std::ops::Range, +} + +impl Resolver { + pub fn new(data_start: usize, len: usize) -> Self { + Self { + data_start, + len, + claimed: Claimed::default(), + } + } + + /// Walk samples without allocating from the untrusted sample count. + pub fn resolve<'a>( + &'a mut self, + traf: &'a mp4_atom::Traf, + defaults: Defaults, + ) -> Result> + 'a, super::Error> { + if traf.tfhd.base_data_offset.is_some() { + return Err(super::Error::UnsupportedBaseDataOffset); + } + let mut remaining: usize = traf.trun.iter().map(|run| run.entries.len()).sum(); + if remaining == 0 { + return Err(super::Error::MissingTrun); + } + let mut runs = traf.trun.iter(); + let mut entries = [].iter(); + let mut offset = 0usize; + Ok(std::iter::from_fn(move || { + if remaining == 0 { + return None; + } + let result = (|| { + let entry = loop { + if let Some(entry) = entries.next() { + break entry; + } + let run = runs.next().ok_or(super::Error::MissingTrun)?; + if let Some(start) = run.data_offset { + offset = usize::try_from(start) + .ok() + .and_then(|start| start.checked_sub(self.data_start)) + .ok_or(super::Error::InvalidDataOffset)?; + } + entries = run.entries.iter(); + }; + let sample = defaults.resolve(&traf.tfhd, entry); + if sample.size == 0 { + return Err(super::Error::EmptySample); + } + let end = offset + .checked_add(sample.size as usize) + .ok_or(super::Error::InvalidDataOffset)?; + if end > self.len { + return Err(super::Error::SampleRangeOutOfBounds { + start: offset, + end, + len: self.len, + }); + } + let range = offset..end; + self.claimed.take(range.clone())?; + if sample.duration.is_none() && remaining > 1 { + return Err(super::Error::MissingSampleDuration); + } + offset = end; + remaining -= 1; + Ok(Resolved { + sample, + cts: entry.cts.unwrap_or_default(), + range, + }) + })(); + if result.is_err() { + remaining = 0; + } + Some(result) + })) + } +} + /// The `mdat` bytes a fragment's samples have taken, so no two samples share any. /// /// Each sample's bytes are copied or published once per sample, so runs that all pointed at one /// range would turn a small fragment into output as large as its run count times its `mdat`. #[derive(Default)] -pub(crate) struct Claimed(std::collections::BTreeMap); +struct Claimed(std::collections::BTreeMap); impl Claimed { /// Take `range` of the `mdat` payload, or fail if an earlier sample holds any of it. pub fn take(&mut self, range: std::ops::Range) -> Result<(), super::Error> { if range.is_empty() { - return Ok(()); + return Err(super::Error::EmptySample); } // The taken ranges are disjoint, so the last one starting before `range` ends reaches furthest. if let Some((_, &end)) = self.0.range(..range.end).next_back() @@ -165,20 +257,107 @@ mod tests { assert_eq!(sample.duration, None); } - /// Adjacent and empty ranges are taken; any shared byte is refused, wherever it falls. + /// Adjacent ranges are taken; empty ranges and any shared byte are refused. #[test] fn claimed_ranges_are_disjoint() { let mut claimed = Claimed::default(); claimed.take(10..20).unwrap(); claimed.take(20..30).unwrap(); claimed.take(0..10).unwrap(); - claimed.take(15..15).unwrap(); + assert!(claimed.take(15..15).is_err()); for overlap in [10..20, 5..11, 29..40, 12..13, 0..40] { assert!(claimed.take(overlap.clone()).is_err(), "{overlap:?}"); } claimed.take(30..31).unwrap(); } + fn traf(offsets: &[Option]) -> mp4_atom::Traf { + mp4_atom::Traf { + tfhd: mp4_atom::Tfhd { + default_sample_size: Some(2), + default_sample_duration: Some(1), + ..Default::default() + }, + trun: offsets + .iter() + .map(|&data_offset| mp4_atom::Trun { + data_offset, + entries: vec![Default::default()], + }) + .collect(), + ..Default::default() + } + } + + #[test] + fn resolves_disjoint_runs_in_decode_order() { + let traf = traf(&[Some(104), Some(100), None]); + let mut resolver = Resolver::new(100, 6); + let ranges: Vec<_> = resolver + .resolve(&traf, Defaults::default()) + .unwrap() + .map(|sample| sample.unwrap().range) + .collect(); + assert_eq!(ranges, [4..6, 0..2, 2..4]); + // Ownership includes other tracks of the same fragment. + let other = traf.clone(); + assert!(matches!( + resolver.resolve(&other, Defaults::default()).unwrap().next().unwrap(), + Err(super::super::Error::InvalidDataOffset) + )); + } + + #[test] + fn refuses_incorrect_sample_ranges() { + use super::super::Error; + for offset in [i32::MIN, -1, 99, 105, i32::MAX] { + let traf = traf(&[Some(offset)]); + let mut resolver = Resolver::new(100, 6); + let mut samples = resolver.resolve(&traf, Defaults::default()).unwrap(); + assert!(samples.next().unwrap().is_err(), "offset {offset}"); + assert!(samples.next().is_none(), "errors end iteration"); + } + let mut traf = traf(&[None]); + for size in [None, Some(0)] { + traf.tfhd.default_sample_size = size; + let mut resolver = Resolver::new(100, 6); + assert!(matches!( + resolver.resolve(&traf, Defaults::default()).unwrap().next().unwrap(), + Err(Error::EmptySample) + )); + } + for base in [0, 100, u64::MAX] { + traf.tfhd.base_data_offset = Some(base); + let mut resolver = Resolver::new(100, 6); + assert!(matches!( + resolver.resolve(&traf, Defaults::default()), + Err(Error::UnsupportedBaseDataOffset) + )); + } + } + + #[test] + fn only_the_final_sample_may_lack_a_duration() { + use super::super::Error; + let mut traf = traf(&[None, None]); + traf.trun[1].entries[0].duration = Some(0); + // Empty trailing runs don't make the last sample non-final. + traf.trun.push(Default::default()); + let mut resolver = Resolver::new(100, 4); + let durations: Vec<_> = resolver + .resolve(&traf, Defaults::default()) + .unwrap() + .map(|sample| sample.unwrap().sample.duration) + .collect(); + assert_eq!(durations, [Some(1), None]); + traf.trun[0].entries[0].duration = Some(0); + let mut resolver = Resolver::new(100, 4); + assert!(matches!( + resolver.resolve(&traf, Defaults::default()).unwrap().next().unwrap(), + Err(Error::MissingSampleDuration) + )); + } + #[test] fn sync_rule() { let sync = |flags| { diff --git a/rs/moq-mux/src/container/fmp4/text.rs b/rs/moq-mux/src/container/fmp4/text.rs index ae0df610809..26f76c4bfdc 100644 --- a/rs/moq-mux/src/container/fmp4/text.rs +++ b/rs/moq-mux/src/container/fmp4/text.rs @@ -47,10 +47,8 @@ pub(super) struct Track { pub(super) struct Fragment<'a> { pub traf: &'a mp4_atom::Traf, pub data: &'a [u8], - pub moof_size: usize, - pub header_size: usize, - /// The `mdat` bytes the moof's earlier samples took. - pub claimed: &'a mut sample::Claimed, + /// Resolve sample ranges shared with the moof's other tracks. + pub resolver: &'a mut sample::Resolver, } impl Track { @@ -71,64 +69,40 @@ impl Track { self.producer.discontinuity()?; self.discontinuity = false; } - let base: usize = traf - .tfhd - .base_data_offset - .unwrap_or_default() - .try_into() - .map_err(|_| Error::InvalidDataOffset)?; - let mut offset = base; - for trun in &traf.trun { - if let Some(value) = trun.data_offset { - let value: usize = value.try_into().map_err(|_| Error::InvalidDataOffset)?; - offset = value - .checked_sub(input.moof_size) - .and_then(|value| value.checked_sub(input.header_size)) - .and_then(|value| value.checked_add(base)) - .ok_or(Error::InvalidDataOffset)?; - } - for entry in &trun.entries { - let sample = self.defaults.resolve(&traf.tfhd, entry); - let duration = sample.duration.ok_or(Error::MissingSampleDuration)?; - let pts = dts - .checked_add_signed(entry.cts.unwrap_or_default() as i64) - .ok_or(Error::PtsOverflow)?; - let duration = Timestamp::new(duration as u64, self.timescale)?; - let next = offset - .checked_add(sample.size as usize) - .ok_or(Error::InvalidDataOffset)?; - let data = input.data.get(offset..next).ok_or(Error::InvalidDataOffset)?; - input.claimed.take(offset..next)?; - offset = next; - dts = dts.checked_add(duration.value()).ok_or(Error::PtsOverflow)?; + for resolved in input.resolver.resolve(traf, self.defaults)? { + let sample::Resolved { sample, cts, range } = resolved?; + let duration = sample.duration.ok_or(Error::MissingSampleDuration)?; + let pts = dts.checked_add_signed(cts as i64).ok_or(Error::PtsOverflow)?; + let duration = Timestamp::new(duration as u64, self.timescale)?; + let data = &input.data[range]; + dts = dts.checked_add(duration.value()).ok_or(Error::PtsOverflow)?; - // A cue starting before the edit is dropped, like a media sample. - let Some(presented) = self.edit.present(pts)? else { - continue; - }; - // TTML maps the document's media times from `source`, so they move with the edit too. - let source = Timestamp::new(pts, self.timescale)?; - let timestamp = timebase.shift(Timestamp::new(presented, self.timescale)?)?; - let end = timestamp.checked_add(duration)?; - let payload = match &self.format { - Format::Vtt { header } => vtt(header, data, timestamp, end)?, - // The frame timestamp is serialized in hang's scale, which the body's begin must match. - Format::Ttml => ttml::rewrite( - data, - source, - timestamp.convert(hang::container::TIMESCALE)?..end.convert(hang::container::TIMESCALE)?, - ) - .map_err(Error::InvalidSubtitle)? - .into(), - }; - self.producer.write(crate::container::Frame { - timestamp, - duration: Some(duration), - keyframe: true, - payload, - })?; - self.producer.cut(None)?; - } + // A cue starting before the edit is dropped, like a media sample. + let Some(presented) = self.edit.present(pts)? else { + continue; + }; + // TTML maps the document's media times from `source`, so they move with the edit too. + let source = Timestamp::new(pts, self.timescale)?; + let timestamp = timebase.shift(Timestamp::new(presented, self.timescale)?)?; + let end = timestamp.checked_add(duration)?; + let payload = match &self.format { + Format::Vtt { header } => vtt(header, data, timestamp, end)?, + // The frame timestamp is serialized in hang's scale, which the body's begin must match. + Format::Ttml => ttml::rewrite( + data, + source, + timestamp.convert(hang::container::TIMESCALE)?..end.convert(hang::container::TIMESCALE)?, + ) + .map_err(Error::InvalidSubtitle)? + .into(), + }; + self.producer.write(crate::container::Frame { + timestamp, + duration: Some(duration), + keyframe: true, + payload, + })?; + self.producer.cut(None)?; } self.end_decode_time = Some(dts); Ok(())