Skip to content

Add Reader::NewReader(initial_pos, length_hint), and let GcsReader bound the request to the hinted range. - #45

Merged
copybara-service[bot] merged 1 commit into
masterfrom
test_1009577927
Nov 17, 2026
Merged

copybara-service[bot] merged 1 commit into
masterfrom
test_1009577927

Conversation

@copybara-service

Copy link
Copy Markdown

Add Reader::NewReader(initial_pos, length_hint), and let GcsReader bound the request to the hinted range.

Random single-record reads of ArrayRecord files on GCS create a child reader
with NewReader(offset) and read one chunk. GcsReader sent an open-ended
request (Range: bytes=offset-), and the request was cancelled when the reader
was destroyed after the chunk was read. The caller knows how much it will read,
but NewReader(Position) cannot express it.

NewReader(initial_pos, length_hint) is like NewReader(initial_pos), but
hints that about length_hint bytes will be read from initial_pos. Reading
beyond the hinted range is supported. By default the hint is ignored, by the
new virtual NewReaderWithLengthHintImpl() calling NewReaderImpl().

GcsReader overrides it: if the hinted range ends before the object (or the
range given to the constructor) does, the request is ReadRange(pos, pos + length_hint), so it completes normally and the connection is reused. The
request is sent when the reader is created, as for NewReader(initial_pos), so
request failures and metadata are reported the same way. A read which needs
more than what is left of the range continues with an open-ended request from
the current position. A hint reaching the end of the object sends the same
request as NewReader(initial_pos); in particular a read of the whole object
from offset 0 still sends no Range header and keeps checksum validation.

All other requests are unchanged.

ArrayRecord passes the chunk length in a follow-up change.

@google-cla

google-cla Bot commented Oct 30, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@copybara-service
copybara-service Bot force-pushed the test_1009577927 branch 2 times, most recently from 1f7f712 to 5104549 Compare October 30, 2026 23:04
@copybara-service
copybara-service Bot force-pushed the test_1009577927 branch 2 times, most recently from 1c40a3a to d5b0459 Compare November 17, 2026 16:00
…bound the request to the hinted range.

Random single-record reads of ArrayRecord files on GCS create a child reader
with `NewReader(offset)` and read one chunk. `GcsReader` sent an open-ended
request (`Range: bytes=offset-`), and the request was cancelled when the reader
was destroyed after the chunk was read. The caller knows how much it will read,
but `NewReader(Position)` cannot express it.

`NewReader(initial_pos, length_hint)` is like `NewReader(initial_pos)`, but
hints that about `length_hint` bytes will be read from `initial_pos`. Reading
beyond the hinted range is supported. By default the hint is ignored, by the
new virtual `NewReaderWithLengthHintImpl()` calling `NewReaderImpl()`.

`GcsReader` overrides it: if the hinted range ends before the object (or the
range given to the constructor) does, the request is `ReadRange(pos, pos +
length_hint)`, so it completes normally and the connection is reused. The
request is sent when the reader is created, as for `NewReader(initial_pos)`, so
request failures and metadata are reported the same way. A read which needs
more than what is left of the range continues with an open-ended request from
the current position. A hint reaching the end of the object sends the same
request as `NewReader(initial_pos)`; in particular a read of the whole object
from offset 0 still sends no `Range` header and keeps checksum validation.

All other requests are unchanged.

ArrayRecord passes the chunk length in a follow-up change.

PiperOrigin-RevId: 1023749871
@copybara-service
copybara-service Bot merged commit 64d553d into master Nov 17, 2026
4 checks passed
@copybara-service
copybara-service Bot deleted the test_1009577927 branch November 17, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants