Repository navigation
Encode and decode depth maps in one pass over the data - #722
Conversation
encode_depth_map indexed the map element by element and wrote each value with a bounds-checked append_big_endian; decode_depth_map read each value with a bounds-checked read, pushed it into a temporary vector and then copied that into the map. For a row-major map (the default layout) both now make one pass between the map's storage and the byte buffer, after the existing size checks. Other layouts keep element indexing. The bytes and the API are unchanged. On a Raspberry Pi CM5, a 1280x720 map encodes in 1.0 ms instead of 3.0 ms and decodes in 0.77 ms instead of 4.3 ms at -O2 (0.45-0.65 ms and 0.32-0.40 ms at -O3, where the loops vectorize). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| if (m.layout() == xt::layout_type::row_major) { | ||
| const uint16_t* in = m.data(); | ||
| for (size_t i = 0; i < height * width; ++i) { | ||
| write_big_endian_u16(out + 2 * i, in[i]); |
There was a problem hiding this comment.
can you look into xtensor's documentation, i am pretty sure they have iterator/range helpers to make it so you never have to do the type of manual indexing that is happening here, same with the read loop below
There was a problem hiding this comment.
Done in d652835: both directions now use linear_cbegin()/linear_cend() and linear_begin()/linear_end(), with no indexing.
depth_map is xt::xarray<uint16_t>, whose layout is fixed row-major, so I dropped the other-layout fallback and added a static_assert that depth_map::static_layout is row-major.
Raspberry Pi CM5, 1280×720, median per call:
| main | this PR | |
|---|---|---|
encode, -O2 |
3.0–3.3 ms | 0.97 ms |
decode, -O2 |
3.1–4.2 ms | 0.95 ms |
encode, -O3 |
2.8–3.1 ms | 0.53–0.65 ms |
decode, -O3 |
2.8–2.9 ms | 0.30–0.32 ms |
| return value; | ||
| } | ||
|
|
||
| // Writes a depth value in big-endian order. Unlike append_big_endian it does no bounds check, |
There was a problem hiding this comment.
like other parts of this file we should be using boost.endian rather than doing these conversions manually, which may result in us removing this helper entirely. in particular it seems like a poor choice to implement the read/write ops on a raw pointer which is assumed to be a buffer which is always two bytes. consider doing this with the boost endian conversion helpers or big_uint16_t
There was a problem hiding this comment.
Done in d652835, and the hand-written helpers are gone.
- Encode constructs each value as a
boost::endian::big_uint16_buf_tdirectly in the payload withstd::uninitialized_copy, so the buffer objects are actually created there rather than aliased. - Decode copies the payload into the map's storage, then converts each value with
boost::endian::big_to_native_inplace. That's the same memcpy-then-convertread_big_endianalready does for the header. - A
static_assertchecks thatbig_uint16_buf_tis two unaligned bytes.
Decode is now two passes, which costs about 0.18 ms at -O2 against the previous commit (1280×720 on a CM5) and nothing at -O3. Reading through a const big_uint16_buf_t* overlay would keep it to one pass, but those objects don't exist in the input buffer, so I went with the well-defined copy.
Per review: encode_depth_map copies the map's row-major storage (linear_cbegin/linear_cend) straight into the payload with std::uninitialized_copy, constructing each value as a boost::endian::big_uint16_buf_t. decode_depth_map copies the payload into the map's storage and converts each value with big_to_native_inplace, the same memcpy-then-convert that read_big_endian does for the header. The hand-written byte helpers and the non-row-major fallback are gone; a depth_map (xt::xarray) is always row-major, which a static_assert now checks, along with big_uint16_buf_t being two unaligned bytes. The bytes and the API are unchanged. On a Raspberry Pi CM5 at -O2, a 1280x720 map encodes in 0.97 ms (3.0-3.3 ms on main) and decodes in 0.95 ms (3.1-4.2 ms on main); at -O3, 0.53-0.65 ms and 0.30-0.32 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lia (lia-viam)
left a comment
There was a problem hiding this comment.
minor comment but lgtm
| // row-major storage, then convert each value from big-endian. | ||
| xt::xarray<uint16_t> m = xt::xarray<uint16_t>::from_shape({height, width}); | ||
| std::copy(depth_values.begin(), depth_values.end(), m.begin()); | ||
| std::memcpy(m.data(), data.data() + offset, m.size() * sizeof(uint16_t)); |
There was a problem hiding this comment.
minor but let's prefer to use std::copy here rather than a manual memcpy. for contiguous storage pretty much any standard library implementation will do this as a memmove (see eg https://github.com/google/libcxx/blob/master/include/algorithm#L1517) and we get better bounds/type safety
There was a problem hiding this comment.
Done in 4f8d4e1: std::copy(data.begin() + offset, data.end(), reinterpret_cast<unsigned char*>(m.data())). The source range is now the vector's own [offset, end) rather than a computed byte count. test_camera passes.
Timed on a CM5 (median of 200 runs each, two runs), it matches the memcpy version:
| std::copy | memcpy | |
|---|---|---|
| 1280x720 | 1.06 / 1.07 ms | 0.96 / 1.07 ms |
| 848x480 | 0.39 / 0.43 ms | 0.39 / 0.43 ms |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ethan (stuqdog)
left a comment
There was a problem hiding this comment.
looks reasonable to me, CI compilation issues aside
clang-tidy's bugprone-narrowing-conversions rejects adding a size_t to a vector iterator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Ethan (@stuqdog) thanks! The CI failure was from my Checked locally in the CI image ( The workflow runs for cd3c6ca are waiting for a maintainer to approve them. |
Summary
Camera::encode_depth_mapandCamera::decode_depth_mapprocessed one value at a time.m(i, j)and wrote it with a bounds-checkedappend_big_endian.read_big_endian, pushed it into a temporarystd::vector, then copied that into the map.Now, after the existing header and size checks:
linear_cbegin()/linear_cend()) straight into the payload withstd::uninitialized_copy, constructing each value as aboost::endian::big_uint16_buf_t.boost::endian::big_to_native_inplace.depth_map(xt::xarray<uint16_t>) is always row-major, the encoding's order. Astatic_assertchecks that, and another checks thatbig_uint16_buf_tis two unaligned bytes.The encoded bytes and the API are unchanged.
Why
Modules encode a depth frame on every
GetImagescall that returns depth, and readers decode one per frame. We're reading depth more often, and on a Raspberry Pi CM5 this was several ms per frame.Measurements
Raspberry Pi Compute Module 5, one core, three runs. Built with Debian bookworm's g++ 12, the image CI uses. Every version produces identical bytes and maps. Median ms per call:
-O2-O2-O3-O3Tests
test_depth_map_encoding_byteschecks the exact encoded bytes for a 2×3 map: theDEPTHMAPmagic, big-endian width and height, then big-endian values including0x00FF,0xFF00and0xFFFF.test_depth_map_round_trip_camera_sizeround-trips an 848×480 map whose values cover the whole 16-bit range.test_camerapasses 12/12 in thecpp-sdk-system-debian:bookwormimage (arm64), built with-Wall -Werror.camera.cpp. Formatted with the image's clang-format.🤖 Generated with Claude Code