Skip to content

[VCWG-G144/G133] bitstream conformance for bitstreams contains SWICTH frames - #5433

Draft
jkeui wants to merge 12 commits into
AOMediaCodec:mainfrom
jkeui:VCWG-G144
Draft

jkeui wants to merge 12 commits into
AOMediaCodec:mainfrom
jkeui:VCWG-G144

Conversation

@jkeui

@jkeui jkeui commented Sep 23, 2026 •

Copy link
Copy Markdown

VCWG-133 and 144 (resolve #5418)
1) add requirement checks to encoder and decoder
1-1) refMVbank=0
1-2) allow_intrabc=0, allow_bawp=0 if refMVbank=0 after a switch frame
1-3) oplf=0 RefineMV=0
2) add a CLI option(--sframe-conformance-option) to apply one of the requirement(constraint) to the encoder
--sframe-conformance-option=0 : (default) --enable-refmvbank=0 + BAWP is disabled after a switch frame
--sframe-conformance-option=1 : intraBC is disabled after a switch frame if --enable-refmvbank=1 + BAWP is disalbed after a switch frame
--sframe-conformance-option=2 : --enable-opfl-refine=0 --enable-refinemv=0

Also check git@github.com:mabenjak/avm.git(Martin Benjak)'s switch_DVMR_experiments branch (commits 47a8c2c, 31b14ea)

This branch also includes three other encoder items which can be disabled.

  1. 2d85428 - Removing the encoder-side limitation for restricted switch frames
  2. 4140cf6 - sframe-replace-kf : changes the coding structure when switch frames are used
  3. 97146c6 - sframe-refresh-all : the switch frame refreshes all RPL slots or just one

@jkeui
jkeui force-pushed the VCWG-G144 branch 3 times, most recently from 7ea9dd5 to 535412d Compare September 24, 2026 02:42
@jkeui

jkeui commented Sep 24, 2026 •

Copy link
Copy Markdown
Author
  1. Note that if a non-restricted switch frame is used, reference frames are all refreshed regardless of sframe-refresh-all
  2. Also note sframe_mode is set as 0 for multi-layer tests since "non-restricted switch frame + sframe-refresh-all=0" may bring inevitable decoder crash. With 1, "non-restricted switch frame + sframe-refresh-all=0" is prevented.
  3. These are all encoder settings.

@peterderivaz peterderivaz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added checks assumes a bitstream only contains a single layer with a switch.

There are currently no restrictions in the specification which force this.

In particular, this code would not match the specification if:

  1. Using multiple extended layers (perhaps the new risk variables should be added to av2_store_xlayer_context and av2_restore_xlayer_context?)
  2. Using multiple embedded or temporal layers (perhaps the new risk variables should be stored per embedded and temporal layer?)

@jkeui

jkeui commented Oct 2, 2026

Copy link
Copy Markdown
Author

The added checks assumes a bitstream only contains a single layer with a switch.

There are currently no restrictions in the specification which force this.

In particular, this code would not match the specification if:

  1. Using multiple extended layers (perhaps the new risk variables should be added to av2_store_xlayer_context and av2_restore_xlayer_context?)
  2. Using multiple embedded or temporal layers (perhaps the new risk variables should be stored per embedded and temporal layer?)

I have updated the code.
I also added a few more things.

  1. TLayerPresenceMap[][][]
  2. The scope of the restriction for OLK at the decoder : it ends at the regular TU after the OLK TU.

@urvangjoshi
urvangjoshi marked this pull request as draft October 13, 2026 18:24
mabenjak and others added 12 commits October 18, 2026 18:34
The function prune_ref_by_selective_ref_frame() returned 1 for all blocks that reference restricted references. This caused WARPMV to be the only prediction mode to be considered for restricted references by the encoder, severely hindering the compression performance of restricted switch frames.

Removed this limitation and set display_order_hint to REF_RESTRICTED_DOH to ensure that all now accessible prediction modes are correctly encoded. Added is_restricted checks to av2_get_refresh_frame_flags() because the display_order_hint of restricted references is always >= the current display_order_hint.
The encoder replaces key frames with switch frames in a repeating pattern, if --sframe-replace-kf is set to a non-zero value. The pattern follows this rule:

--sframe-replace-kf =0: disabled

--sframe-replace-kf =1: K-S-K-S…,

--sframe-replace-kf =2: K-S-S-K-S-S…

--sframe-replace-kf =3: …
1) add requirement checks to encoder and decoder
1-1) refMVbank=0
1-2) allow_intrabc=0, allow_bawp=0 if refMVbank=0 after a switch frame
1-3) oplf=0 RefineMV=0
2) add a CLI option(--sframe-conformance-option) to apply one of the requirement(constraint) to the encoder
--sframe-conformance-option=0 : (default) --enable-refmvbank=0 + BAWP is disabled after a switch frame
--sframe-conformance-option=1 : intraBC is disabled after a switch frame if --enable-refmvbank=1 + BAWP is disalbed after a switch frame
--sframe-conformance-option=2 : --enable-opfl-refine=0 --enable-refinemv=0

Also check git@github.com:mabenjak/avm.git(Martin Benjak)'s
switch_DVMR_experiments branch (commits 47a8c2c, 31b14ea)
When a switch frame is not restricted and its reference frames are not
fully refreshed, a decoder crash/mismatch is expected: only a
restricted switch frame's reference marking (is_restricted,
sample-values-only) makes partial reference refresh well-defined.
So a switch frame must either be restricted, or fully refresh its
reference frames. This test exercises restricted switch frames, so
set sframe_mode = 0 explicitly instead of relying on the default
(sframe_mode == 1), which produced a non-restricted switch frame with
default partial refresh and triggered exactly the undefined case above.
Save and restore the switch_risk_window in av2_store_xlayer_context() and
av2_restore_xlayer_context()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Conformance requirement for switch frames

4 participants