Skip to content

[Perf][C++] Avoid memcpy of the RLE pattern in read_length_and_bitwidth #1076

Open

Description

@ColinLeeo

Problem

Int64RleDecoder::read_length_and_bitwidth / Int32RleDecoder::read_length_and_bitwidth
copy the whole encoded pattern into a temporary buffer (tmp_buf_) on every
pattern, then wrap byte_cache_ over that copy. Profiling a wide-table read
showed this memcpy at ~7.5% of the query.

Proposed changes

When the caller's ByteStream is wrapped and has at least length_ bytes left,
wrap byte_cache_ directly over the caller's buffer and advance its read
position; fall back to the copy for paged or short streams. The caller's stream
outlives the pattern decode, so no lifetime issue.

Implemented in #1072.

Related: #1072

Activity

  1. added this to the 2.5.1 milestone on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions