Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion quest/m1/cut-audit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ leaving JS-only redesigns of `js/net` to [rs2ts](/quest/m2/rs2ts/README.md).

## Required

- [Counted field-less `trun` entries](/quest/m1/cut-audit/fmp4-sample-resolver.md) - a moof of field-less runs cannot materialize millions of sample entries
- [A reserved group survives a route returning](/quest/m1/cut-audit/reserved-route-return.md) - a route flip A to B to A no longer panics a peeked reader
- [A description-less AAC rendition means one thing](/quest/m1/cut-audit/aac-description-less.md) - every player and exporter agrees on the frame format
- [The export clock keeps only the rules admission still needs](/quest/m1/cut-audit/jitter-admission.md) - the clock rules the time and byte admission bound made redundant are measured and removed
Expand Down
43 changes: 0 additions & 43 deletions quest/m1/cut-audit/fmp4-sample-resolver.md

This file was deleted.

1 change: 0 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,6 @@ lite-07; bitrate enforcement is not part of this release.
- [Shared fullscreen, formatters, and stats cards in @moq/ui](/quest/m2/ui-shared-controls.md) - watch and publish stop keeping copies of their shared controls
- [moqsrc A/V sync](/quest/m2/moqsrc-av-sync.md) - a nightly test measures A/V skew from `moqsrc` through clock-synced sinks
- [MP4 export](/quest/m2/mp4-export.md) - `moq export mp4 --output` records crash-safe fragments, then finishes a regular MP4 with moov at the end
- [Importers fill the VR fields](/quest/m2/mux-vr-fields.md) - MP4, Matroska, and Opus imports set the catalog's projection, stereo, and ambisonic layout
- [fMP4 ambisonic layout](/quest/m2/mux-vr-ambisonic-fmp4.md) - fMP4 Opus import fills `ambisonic` from the `dOps` mapping table
- [Linux decoded frames](/quest/m2/obs-decode-linux.md) - present supported native decoded surfaces with visible CPU fallback
- [Windows decoded frames](/quest/m2/obs-decode-windows.md) - present decoded D3D11 surfaces in OBS without CPU readback
Expand Down
13 changes: 8 additions & 5 deletions quest/m2/cmaf-opus-surround.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,14 @@ family 0), the shape drafted in kixelated/mp4-atom#246. That is a breaking
Decided 2026-10-17: the `mp4-atom` change is not worth landing on its own.
Alone it only swaps `mp4-atom`'s generic decode error for
`UnsupportedMappingFamily`; the payoff is surround Opus. So the former
`mp4-atom-dops-mapping` quest (draft moq-dev/moq#5819) folded in here, and
the breaking `mp4-atom` release lands only when this quest does.

Order: land kixelated/mp4-atom#246, release `mp4-atom` 0.17, then one moq PR
that bumps `rs/moq-mux` to `mp4-atom = "0.17"` and does the import and export.
`mp4-atom-dops-mapping` quest (draft moq-dev/moq#5819) folded in here.

Decided 2026-10-28 (maintainer): kixelated/mp4-atom#246 ships in 0.17.0
beside kixelated/mp4-atom#254, so downstream takes one breaking bump.
0.17.0 is released (2026-10-30), and moq-mux takes it for the counted
trun (moq-dev/moq#7043, carried by moq-dev/moq#7308), still refusing a
nonzero family explicitly, since `mp4-atom` no longer does.
This quest then only does the import and export.
A git patch to the branch cannot stand in for the release, since `cargo deny`
refuses unknown git sources.

Expand Down
11 changes: 6 additions & 5 deletions quest/m2/mux-vr-ambisonic-fmp4.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,21 @@

An fMP4 Opus track with channel mapping family 2 imports with the catalog's
`ambisonic` field, under the same RFC 8486 rules the Matroska and raw Opus
importers apply in [VR fields](/quest/m2/mux-vr-fields.md).
importers apply (`opus::Mapping::new` refuses the family 2 channel counts the
catalog cannot carry, and the OpusHead's `AudioConfig` sets the layout).

## Plan

Split from VR fields on 2026-10-22 (maintainer): only this part needs the
`dOps` channel mapping table, so the rest of VR fields does not wait on it.
Split from the VR fields quest on 2026-10-22 (maintainer), which landed the
Matroska and raw Opus side: only this part needs the `dOps` channel mapping
table.
Read the mapping from the `dOps` that
[CMAF surround Opus](/quest/m2/cmaf-opus-surround.md) carries through the
OpusHead, and reuse VR fields' family 2 validation. Test with an fMP4
OpusHead, and reuse that family 2 validation. Test with an fMP4
first-order ambisonic fixture and a refused non-diegetic stereo one.

Public API: none. Wire: none.

## Required

- [CMAF surround Opus](/quest/m2/cmaf-opus-surround.md) - the `dOps` mapping table an ambisonic fMP4 track needs
- [VR fields](/quest/m2/mux-vr-fields.md) - the ambisonic validation and catalog plumbing this reuses
58 changes: 0 additions & 58 deletions quest/m2/mux-vr-fields.md

This file was deleted.

4 changes: 0 additions & 4 deletions quest/m3/watch-webxr.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,3 @@ into CI; a manual run on a headset browser proves playback.

Public API: possibly a `<moq-watch>` attribute to enter or disable immersive
playback. Wire: none.

## Related

- [moq-mux importers fill the VR fields](/quest/m2/mux-vr-fields.md) - native sources that carry these fields
2 changes: 1 addition & 1 deletion rs/moq-hls/src/export/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -597,7 +597,7 @@ mod tests {
while let Some(atom) = mp4_atom::Any::decode_maybe(&mut cursor).unwrap() {
if let mp4_atom::Any::Moof(moof) = atom {
return moof.traf[0].trun[0]
.entries
.samples
.iter()
.map(|entry| entry.duration)
.collect();
Expand Down
5 changes: 5 additions & 0 deletions rs/moq-mux/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- MP4 and Matroska import fill a video rendition's `projection` and `stereo` from Spherical Video V2's `sv3d`/`st3d` boxes and Matroska's `Projection`/`StereoMode`, and Opus channel mapping family 2 (raw Opus, Matroska, and other OpusHead sources) imports with `layout: "ambisonic"`. What the catalog cannot describe is refused with `fmp4::Error::UnsupportedSpatial`, `mkv::Error::UnsupportedSpatial`, or `opus::Error::UnsupportedChannelCount`: a cubemap or mesh projection, a nonzero pose, uncentred or vertically cropped equirectangular bounds, a stereo packing other than mono, left-right, or top-bottom with the left eye first, and family 2 at zeroth order or with RFC 8486's non-diegetic stereo pair.

### Changed

- [**breaking**] Declare per-track archive durations from GOP hints or the first completed record.
- [**breaking**] Update the re-exported `mp4_atom` to 0.17, whose `Trun.entries` is now `Trun.samples`: a run without per-sample fields decodes as a count, so a moof of field-less runs no longer materializes an entry per sample before import refuses it. Read entries with `trun.samples.iter()` and build them with `samples: entries.into()`. `TrunEntry::cts` widens to `i64` (a version 0 offset is unsigned), `EsDescriptor` gains its optional fields, and `Dref.urls` becomes typed `entries`. `Dops` gains `channel_mapping: Option<OpusChannelMapping>` (`None` for family 0); the fMP4 importer still refuses a nonzero family.

### Fixed

Expand Down
2 changes: 1 addition & 1 deletion rs/moq-mux/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ moq-json = { workspace = true }
moq-loc = { workspace = true }
moq-msf = { workspace = true }
moq-net = { workspace = true }
mp4-atom = { version = "0.16.2", features = ["tokio", "bytes", "serde"] }
mp4-atom = { version = "0.17.0", features = ["tokio", "bytes", "serde"] }
mpeg2ts = "0.6.0"
num_enum = { workspace = true }
roxmltree = "0.21.1"
Expand Down
5 changes: 3 additions & 2 deletions rs/moq-mux/benches/containers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -207,12 +207,13 @@ fn interleaved(init: &[u8], tracks: usize, runs: usize, fragments: usize, payloa
trun: (0..runs)
.map(|run| mp4_atom::Trun {
data_offset: Some(0),
entries: vec![mp4_atom::TrunEntry {
samples: vec![mp4_atom::TrunEntry {
duration: Some(trak.mdia.mdhd.timescale / 50),
size: Some(payload.len() as u32),
flags: Some(if run == 0 { 0 } else { 0x0001_0000 }),
cts: None,
}],
}]
.into(),
})
.collect(),
..Default::default()
Expand Down
1 change: 1 addition & 0 deletions rs/moq-mux/src/codec/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ pub(crate) mod legacy;
pub(crate) mod mp2;
pub mod mp3;
pub mod opus;
pub(crate) mod spatial;
pub(crate) mod video;
pub mod vp8;
pub mod vp9;
29 changes: 29 additions & 0 deletions rs/moq-mux/src/codec/opus/import.rs
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,11 @@ impl From<Config> for hang::catalog::AudioConfig {
fn from(config: Config) -> Self {
let mut audio = hang::catalog::AudioConfig::new(hang::catalog::AudioCodec::Opus, 48_000, config.channel_count);
audio.warmup = Some(super::WARMUP);
// Family 2 is ambisonics in AmbiX order (RFC 8486 §3.1); its mapping refuses the channel
// counts the catalog's layout cannot carry.
if config.mapping.is_some_and(|mapping| mapping.family() == 2) {
audio.layout = Some(hang::catalog::AudioLayout::Ambisonic);
}
audio.description = config.encode().ok();
audio.container = hang::catalog::Container::Legacy;
audio
Expand All @@ -143,6 +148,30 @@ mod tests {
assert_eq!(config.sample_rate, 48_000);
}

#[test]
fn family_two_is_ambisonic() {
// First order ambisonics, one stream per channel, in ACN order.
let mapping = crate::codec::opus::Mapping::new(crate::codec::opus::mapping::Config {
family: 2,
streams: 4,
coupled: 0,
table: &[0, 1, 2, 3],
})
.unwrap();
let mut head = super::Config::new(48_000, 4);
head.mapping = Some(mapping);
let config = super::config(&head.encode().unwrap()).unwrap();
assert_eq!(config.layout, Some(hang::catalog::AudioLayout::Ambisonic));
assert_eq!(config.channel_count, 4);

// Family 1 surround and family 0 stereo stay plain channels.
let mut head = super::Config::new(48_000, 6);
head.mapping = Some(crate::codec::opus::Mapping::vorbis(6).unwrap());
assert_eq!(super::config(&head.encode().unwrap()).unwrap().layout, None);
let head = super::Config::new(48_000, 2).encode().unwrap();
assert_eq!(super::config(&head).unwrap().layout, None);
}

#[tokio::test(start_paused = true)]
async fn a_loc_reservation_reaches_the_wire_and_the_catalog() {
let mut broadcast = moq_net::broadcast::Info::new().produce();
Expand Down
21 changes: 20 additions & 1 deletion rs/moq-mux/src/codec/opus/mapping.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,9 @@ impl Mapping {

/// Check `config` into a mapping with one output channel per table entry.
///
/// Refuses family 0, a channel count the family does not allow, and a table
/// Refuses family 0, a channel count the family does not allow (family 2
/// allows only the 4, 9, 16, ... channels of first order ambisonics and up,
/// without RFC 8486's non-diegetic stereo pair), and a table
/// that names no streams, more coupled streams than streams, or a channel
/// past the decoded ones.
pub fn new(config: Config<'_>) -> Result<Self> {
Expand All @@ -77,6 +79,13 @@ impl Mapping {
if table.is_empty() || table.len() > max_channels {
return Err(Error::UnsupportedChannelCount(table.len() as u32));
}
// RFC 8486 §3.1 gives family 2 (1 + n)² ambisonic channels for n in 0..=14, plus 0 or 2
// non-diegetic stereo ones. The catalog's ambisonic layout carries neither zeroth order nor
// the stereo pair, so only first order and up without the pair is accepted.
let order = (table.len() as u32).isqrt();
if family == 2 && (order < 2 || order * order != table.len() as u32) {
return Err(Error::UnsupportedChannelCount(table.len() as u32));
}

let decoded = streams as u32 + coupled as u32;
if streams == 0 || coupled > streams || decoded > 255 {
Expand Down Expand Up @@ -169,6 +178,16 @@ mod tests {
assert!(matches!(new(0, 1, 1, &[0, 1]), Err(Error::UnsupportedMappingFamily(0))));
assert!(matches!(new(1, 1, 0, &[]), Err(Error::UnsupportedChannelCount(0))));
assert!(matches!(new(1, 9, 0, &[0; 9]), Err(Error::UnsupportedChannelCount(9))));
assert_eq!(new(2, 4, 0, &[0, 1, 2, 3]).unwrap().family(), 2);
assert_eq!(new(2, 9, 0, &[0, 1, 2, 3, 4, 5, 6, 7, 8]).unwrap().family(), 2);
// Zeroth order, first order with RFC 8486's non-diegetic stereo pair, and counts it forbids.
for channels in [1u8, 3, 6, 11, 2, 5, 8, 10] {
let table: Vec<u8> = (0..channels).collect();
assert!(
matches!(new(2, channels, 0, &table), Err(Error::UnsupportedChannelCount(c)) if c == channels as u32),
"{channels} channels"
);
}
for (streams, coupled, table) in [(0, 0, &[0][..]), (1, 2, &[0]), (1, 1, &[2]), (200, 100, &[0])] {
assert!(
matches!(new(1, streams, coupled, table), Err(Error::InvalidMappingTable)),
Expand Down
5 changes: 3 additions & 2 deletions rs/moq-mux/src/codec/opus/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,9 @@ pub enum Error {
#[error("unsupported OpusHead version {0}")]
UnsupportedVersion(u8),

/// The channel count is zero, or more than the channel mapping family allows:
/// family 0 covers mono/stereo and family 1 up to eight channels.
/// The channel count is zero, or one the channel mapping family does not allow:
/// family 0 covers mono/stereo, family 1 up to eight channels, and family 2 the
/// square counts of first order ambisonics and up.
#[error("channel mapping family does not allow {0} channels")]
UnsupportedChannelCount(u32),

Expand Down
Loading
Loading