[pkajaba]

The software-host parity work from #702 (the "parity: subtitles, PiP, scrub thumbnails" thread). The software host (the Dolby Vision FEL route and video the hardware decoder cannot play) has no AVPlayer, so several things the AVPlayer path gives for free were missing. Nine pieces, on one branch. It also includes the changes from #856 by @PieterVO (review items 1 to 4 and the fixes from his Apple TV run): commits 6e88cb8, a1e7fb2, bf48b1c, 1d1b2f5, 6bd9b89, 1912cd3 and 527e2c1, plus his merges of main (51b4137, 1f287d6, 4628c0f, 6ddadc1). They are merged in, not rebased, so his authorship is kept.

What's in it

• Now-Playing card and remote commands, opt-in.

  • ownsSoftwareNowPlaying, off by default and independent of ownsVideoNowPlayingSession, so the AVPlayer paths are unchanged.
  • With it on, the engine publishes title, artwork, duration, elapsed and rate to MPNowPlayingInfoCenter.default() and registers play, pause, toggle, skip (remoteSkipInterval, default 10 s) and change-position on MPRemoteCommandCenter.shared(). It republishes on seek, pause, speed change, stall and end.
  • softwareNowPlayingActive tells a host when to add its own next and previous.
  • The centers are process-wide, so at stop the engine removes only its own targets, restores each command's isEnabled unless another owner changed it, and clears the info only while the center still holds what it last published.

• System caption style on the software host. Subtitles composited into software-path PiP frames use the user's Settings > Accessibility > Subtitles & Captioning choices (font, size, text colour and opacity, background and window colour and opacity, edge style), and re-render when the setting changes. A field counts as chosen only where it differs from the system default. SubtitleCaptionStyle.current() and observeChanges(_:) expose the same style to a host that draws its own overlay. Bitmap subtitles are untouched.

• Scrub thumbnails on a software session that plays a local file. scrubThumbnail and scrubPreviewFrame answered nil there (supportsCacheBackedStills was false), so a FEL title had no scrub stills unless the host built its own FrameExtractor. The engine now decodes them from the file (the base layer for a dual-layer title), through the same calls the AVPlayer path answers, with scrubPreviewFrame honouring isCancelled and the 750 ms / 900 packet bound. A Dolby Vision Profile 5 / 10.0 file without the DolbyVisionStillReshaping trait keeps supportsCacheBackedStills false (review item 4). Remote software sessions already served stills from the packet cache.

• Atmos on the software host. The host decoded every track to PCM, so a JOC stream played as its 5.1 core and TrueHD Atmos or DTS:X as its 7.1 bed. With LoadOptions.objectAudioRendering set (the engine's existing option for object-audio delivery), E-AC-3 JOC is handed to the audio renderer compressed, after the system's Atmos decoder is registered once per process, and TrueHD Atmos and DTS:X are rendered into the chosen bed and encoded as APAC. .off (the default), live sessions and every other codec keep the PCM decode. The automatic audio pick prefers a TrueHD Atmos or DTS:X track on the software host too. Two fixes came with it:

  • the APAC format description handed to the renderer had no cookie and no packet size, so the renderer failed every packet and played silence; it now carries both
  • the E-AC-3 look at the stream is made only inside the demuxer's stream guard, so a stop that closes the demuxer cannot free the codec parameters mid-read

  From the feat(software-host): parity with the AVPlayer path: Now-Playing and remote commands, caption style, local scrub stills, Atmos, audio route and interruption handling #808 review (items 1 to 4) and the Apple TV run in fix(software-host): #808 review follow-ups (stacked on #808) #856:

  • DTS:X is rendered only into 7.1.4 or 9.1.6 (main's one-to-one rule); a TrueHD or DTS track kept on PCM says why in the log; a TrueHD track whose profile the probe missed is read from its own major sync, as on the loopback route (fix(audio): read TrueHD Atmos from the major sync when the probe never reached the track (7.40.1) #838)
  • a packet read before a seek can no longer anchor the APAC bridge at the old position: the producer drops it under the same lock as the seek's flush (review item 3)
  • the clock's "audio is not recovering" fallback waits for the APAC route's first buffer instead of anchoring every TrueHD load on video after 50 packets
  • on tvOS the Atmos output activator runs beside the software host while its renderer gets Dolby Atmos (TrueHD Atmos as APAC, compressed E-AC-3 JOC; not DTS:X), follows pause (riding through a short pause for the 60 s grace from HDMI Atmos badge drops on pause/resume — follow-up to #834 #848), setRate(0) and interruption resumes, and detaches with the session
  • audioDelivery reports .bridged (APAC) and .streamCopy (compressed JOC) on the software host, activeAudioDecoder names the route ("TRUEHD Atmos → APAC 7.1.4"), and background audio counts both as audio

• Pause when the output route is removed. On iOS and tvOS the renderer paths (software host and audio host) now pause when the active output goes away (headphones unplugged, a wireless output lost). The decision is platform-neutral and unit-tested; it fires only while playing or seeking. It pauses for headphones, Bluetooth (A2DP, LE, HFP), AirPlay and USB outputs only. HDMI is left out, so an ARC / eARC receiver switching on or off on an Apple TV keeps playing (review item 2). The AVPlayer path already does this itself and is untouched.

• Follow a change of the output channel count. When the route changes (say a 7.1 receiver to 2.0 speakers), the engine re-evaluates the preferred output channel count against the source and the new maximum, and writes it only when it differs, on the shared audio-session queue, so a repeated notification ends at the first no-op.

• play() during an audio-session interruption. On the renderer paths a play press during an interruption is held instead of starting a stalled clock; the session is reactivated and the held play runs when the interruption ends. A second press while one is held reactivates and plays at once, and a pause, a stop or a reload cancels the held play. Without a renderer path nothing is held.

• FEL yields to an AirPlay video receiver, opt-in. On iOS, an AirPlay output holds the audio route and the system reports every AirPlay output as the same port type, so the engine cannot tell an Apple TV from a HomePod. The host says it with setAirPlayReceiverShowsVideo(_:) (default false, so nothing changes until a host calls it): with it on and a wireless AirPlay route active, a Profile 7 title keeps the native route (8.1 where the display takes Dolby Vision; the enhancement layer is lost), because a video receiver takes a stream, not a composed picture. An audio-only output such as a HomePod keeps FEL. The decision is the new DualLayerRouteDecision.Reason.airPlayVideoReceiver, logged with the rest of the route line, and is read once per load (a URL source decides again on reloadAtCurrentPosition(); a custom source keeps its route until the next load()).

  • setScreenMirrored(_:) (iOS) lets a host say the screen is mirrored, so airPlayPictureStaysLocal reads false while it is on: the picture goes to the receiver too.

• Software play waits for the display mode switch (tvOS). The software host now goes through the same display-criteria gate the AVPlayer path uses before it starts the clock, so a rate-only or HDR mode switch finishes before the first frame instead of after it. The gate does nothing when the display criteria did not change or the load is a secondary output (nobody writes criteria for it), and it has no effect off tvOS.

• docs/api.md, docs/architecture.md, docs/formats.md, docs/dolby-vision-fel.md and the README row describe the contracts. CHANGELOG entries are under Unreleased.

How it was tested

The branch is merged with the current origin/main (20a2272, including #848, #854 and #855), and with the commits from #856 listed above.

Tested on Mac

On the merged tip (7e8b09e):

• The suites this branch touches pass (131 tests in 15 suites): Now-Playing, remote commands, the AirPlay route decision, the display-mode play gate, caption style, scrub stills, the Atmos routes, the audio delivery reporting, route removal, channel count, interruption, the APAC format description, the activator (including the pause grace) and the panel HDR proof from fix(display): an accepted HDR master proves the panel for every engine in the process #855. With the TrueHDObjectAudio, DTSXObjectAudio, DualLayerDolbyVision and DolbyVisionStillReshaping traits on, the same suites plus the spatial bridge (including the stale packet) pass (156 tests in 17 suites).
• Generic tvOS and iOS xcodebuild build-for-testing pass. The tvOS build is the only compile check of the activator's software-host branch.
• The full suite ran before the last merge of main, on c90d499: swift test 4950 Swift Testing tests in 685 suites and 800 XCTest cases (783 + 17 in the SMB target, 1 skipped), all passing; the strict-pool run (LIBDISPATCH_COOPERATIVE_POOL_STRICT=1) passed except the known wall-clock test VideoPrerollTests "holds the clock past the plain timeout", which passes alone. It was not re-run after merging main 20a2272, which changed the panel HDR proof in the display criteria, the demuxer's PGS probe parking and a preroll test (9 files). Only the panel HDR proof suite of those was run on the merged tip (above); main's own tests for the PGS probe parking and the audio probe come with the merge and were not re-run here.
• New suites: SoftwareNowPlayingTests, SubtitleCaptionStyleTests, SoftwareLocalScrubStillTests, SoftwareAtmosRouteTests, SoftwareAPACFormatDescriptionTests, RouteRemovalPauseTests, ChannelCountChangeTests, InterruptionPlayGuardTests and AirPlayParityTests (the host-signal and mirror truth tables, the setter defaults, the software play-gate grace, and with the trait on the video-receiver route decision, including that an audio-only output keeps FEL), plus additions to FrameDecodeContextClampTests and ItemSwapStillTests.
• New in this round: RouteRemovalPauseTests.hdmiDoesNotPause and portTypesMatchTheSession, SoftwareAtmosRouteTests.dtsxNeedsAOneToOneBed and dolbyAtmosRoutes, SoftwareLocalScrubStillTests.profile5FollowsTheStillVerdict, SpatialAudioBridgeTests.stalePacketDoesNotAnchorThePostSeekRun, AudioDeliveryClassificationTests.atmosRoutesReportTheirDelivery, the activator's transport truth table in Issue834AtmosActivatorRouteTests, and the renderer path's pause grace in Issue848AtmosActivatorPauseGraceTests.
• Atmos APAC timing was measured on macOS: the audio renderer plays the APAC output with no decode errors, and with the packets stamped at the PTS minus the 2048 priming frames the APAC lines up with the PCM decode at 0.00 ms (unshifted it is +42.69 ms). The compressed E-AC-3 JOC route has the unit tests above but no playback measurement.
• Load flakes seen on a busy machine (SoftwarePacketMemoryFIFOTests, SoftwarePacketCacheStorageTests, the refusal tests, DoviFelFuzzTests, VideoPrerollTests, and once the wall-clock Issue848AtmosActivatorPauseGraceTests "a pause keeps the stream playing until the grace runs out") are wall-clock or free-disk based and pass when re-run alone.

Tested on Apple TV (by @PieterVO, #856)

Apple TV 4K (1st generation), tvOS 26.6, HDMI to a Dolby Vision TV through an Atmos receiver: a Profile 7 FEL title with TrueHD Atmos takes the dual-layer route with TrueHD rendered to APAC 7.1.4; the TV shows Dolby Vision and the receiver Atmos; over three sessions 0 lost frames, 2 to 39 display-layer drops per session (mostly at start), no rebuffers, the clock armed from audio. A Profile 7 MEL title on the loopback route: the activator is active on HDMI.

Still needs Apple TV

This stays a draft until these have run on a device:

• Atmos on tvOS: compressed E-AC-3 JOC on the software host, DTS:X as APAC, and the APAC priming offset (TrueHD Atmos as APAC is done, above).
• The receiver's Atmos indicator during the tvOS display-mode switch wait: the activator starts while the load is still .loading. On the software path the state may read .paused before play when the load reaches readiness during the wait (the same .paused as the play-gate item below), so the activator can play, hold through the 60 s pause grace, and play again. Check that the receiver flips to Atmos once and stays there.
• Now-Playing card and remote commands: the card contents, play, pause, skip and change-position from the remote, and that stop leaves another owner's targets alone.
• System caption style in software-path PiP frames, and a change of the setting while playing.
• Scrub stills on a local software session: the 750 ms / 900 packet bound on device storage.
• Route removal (AOS5): unplug or lose the active output during playback on the software host and the audio host; playback pauses and does not resume by itself. Headphones or Bluetooth removal pauses; an HDMI receiver toggle does not.
• Channel-count change (AOS6): play a 7.1 title to a 7.1 receiver, then switch to 2.0 speakers; the preferred channel count follows, with a downmix and not silence, and any extra gap is short. One write per real change, none on repeats.
• Interruption (Siri, an alarm): press play while interrupted; the session reactivates, the clock runs and playback starts when the interruption ends. A pause press cancels it.
• AirPlay video receiver (iOS): with setAirPlayReceiverShowsVideo(true) and an Apple TV picked as the AirPlay output, a Profile 7 title loads on the native route (the log line names airPlayVideoReceiver); with a HomePod as the output the same title keeps FEL. The setter is inert until a host calls it.
• Software play gate (tvOS): on a real rate-only switch and on an HDR switch, the first frame waits for the mode switch and the picture does not flash. The state shown while it waits (.paused) is a caveat to look at.

Follow-up (not in this PR)

Open items from the review, to be taken separately:

• the channel preference against the 12 or 16 channels fed to the renderer
• a fallback when an Atmos route stays silent
• LiveTelemetry.objectAudio is nil on the software host
• the interruption hold when a begin never gets its end
• one sentence in docs/api.md on host Now-Playing targets beside softwareNowPlayingActive
• local scrub stills: no keyframe fallback at the 750 ms bound, and network mounts (volumeIsLocalKey)
• caption text does not wrap at large sizes

When spatial rendering is unavailable, the software host falls back to the PCM decode, so asking for Atmos never costs a session its audio. audioDelivery and activeAudioDecoder now say which route a session took (.bridged, .streamCopy, .decoded), but the reason for a fallback is still only a log line. It happens when:

• objectAudioRendering is .off, or the session is live
• the E-AC-3 track is not confirmed JOC, or is not 6 channels at 48 kHz in 1536-sample packets, or the dec3 box cannot be built from the first packets, or the passthrough producer cannot be created
• for TrueHD Atmos and DTS:X: the track declares a rate other than 48 kHz (spatialRenderingLayout returns nil), the OS is older than 26 or has no APAC encoder, the build lacks the TrueHDObjectAudio or DTSXObjectAudio trait, a DTS-HD MA track is not DTS:X, or the object decoder or the APAC bridge fails to build

In those cases JOC plays as its 5.1 core and TrueHD Atmos or DTS:X as its 7.1 bed, as before this PR. The follow-up is to publish why a session fell back so a host can show it, and to decide whether any of these cases should do something better than PCM.

[superuser404notfound]

This landed on main together with #856: 1ed65a90 merges this branch at 82805511 (including the AirPlay video-receiver gate), 96381521 merges PieterVO's follow-ups on top. Thank you, Pavel, for this big parity push; the stop-path care made it easy to land. Closing the draft since every commit is now in main. The device checks from the original list are still welcome as issues or comments here.

[superuser404notfound]

The four commits pushed after the merge are handled in d2cb2b98 on main:

• c90d499e (renderer-path pause grace test) is cherry-picked as is.
• a168689a and 5d75e124 are folded into one commit under your name: the softwareAtmosOutputPlaying truth table, your first-buffer comment (the 2048 priming frames plus the 2048-frame chunk, which matches the APAC encoder notes better than the wording main had), and the changelog entries for the channel-count follow and the interruption play hold. The blank line in the activator's doc comment was already gone on main.
• 3cb9dee1 is left out on purpose: main now depends on LibTrueHD 3.1.0 for the object renderer, so that Package.resolved pin is real.

Thanks again, Pavel.