Skip to content

Reset the DC prediction at restart markers when optimizing Huffman tables - #42

Open
wleeaf wants to merge 1 commit into
vstroebel:mainfrom
wleeaf:optimized-huffman-restart-dc
Open

wleeaf wants to merge 1 commit into
vstroebel:mainfrom
wleeaf:optimized-huffman-restart-dc

Conversation

@wleeaf

@wleeaf wleeaf commented Oct 26, 2026

Copy link
Copy Markdown

With both set_optimized_huffman_tables(true) and set_restart_interval(n), the encoder can write a JPEG that doesn't decode.

optimize_huffman_table counts the DC differences of each component's blocks as if the prediction ran through the whole scan. When the blocks are written, though, each restart marker resets the prediction to 0, so the first DC difference after a marker is the DC value itself. That difference often falls in a category the optimized table has no code for. get_for_value then returns a zero-length code: release builds write nothing for the symbol, and debug builds hit the debug_assert! in huffman.rs.

For example, I encoded a 17x1 image with RGB, quality 95, F_1_1, a restart interval of 1 and optimized tables:

  • libjpeg-turbo's djpeg and jpeginfo -c report "Corrupt JPEG data: 7 extraneous bytes before marker 0xd1".
  • The decoded pixels are 15 dB PSNR from the source; the same image without the restart interval is 49 dB.

I tried every combination of Luma and RGB; 17x1, 300x3, 37x29 and 64x64; restart intervals 0, 1, 2, 3 and 7; optimized tables on and off; progressive on and off; and F_1_1 and F_2_2.

  • Before: all 70 corrupt or degraded files had optimized tables together with a restart interval.
  • After: none are flagged by jpeginfo -c, and every image decodes within 1 dB of the same image encoded without a restart interval.

The DC statistics now reset the prediction at the same blocks where encode_image_sequential and the DC scan of encode_image_progressive write a restart marker. With optimized tables, every scan is non-interleaved, so that is every restart_interval blocks of a component. AC statistics don't depend on the prediction and are unchanged.

Tests: test_restart_interval_optimized (RGB, interval 32) and test_restart_interval_optimized_progressive (grayscale, interval 1). Both fail on main at the debug_assert! in huffman.rs.

cargo test passes with the default features, --features simd, --features use_wide and --no-default-features --tests. cargo fmt -- --check is clean. The clippy warnings are the ones already on main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant