Repository navigation
Enforce daily telemetry event storage limits - #326
Open
gmemuriuki wants to merge 3 commits into
Open
gmemuriuki wants to merge 3 commits into
gmemuriuki wants to merge 3 commits into
Conversation
gmemuriuki
added this pull request to stack #331
October 8, 2026 18:34
gmemuriuki
force-pushed
the
telemetry/event-storage-limits
branch
2 times, most recently
from
October 14, 2026 07:51
f5031c8 to
037e0e4
Compare
nikomatsakis
force-pushed
the
telemetry/event-storage-limits
branch
from
October 16, 2026 09:02
037e0e4 to
e584e4a
Compare
Introduce a typed LowVolumeRow boundary so aggregate metric rows cannot enter the append-only event log. EventBatch validates one authoritative UTC day and serializes the complete ordered batch before opening a file. Append batches through LockedStorage using the canonical daily path. Existing partial final lines are closed before new rows are written, owner-only permissions are restored where supported, and failures identify the preparation, inspection, repair, or write stage. Share event and metric filename rules with daily-file discovery and document the tail-repair and storage-allowance behavior in the telemetry contract and module guide.
Add the low-volume event admission policy for the shared 8 MiB daily ceiling. Measure event and metric files under the telemetry lock, reserve the largest version 1 storage-limit marker, and make whole-batch append, marker, or suppression decisions with saturating arithmetic. Bound final-line inspection and treat complete unterminated markers as authoritative so a failed append cannot reopen a closed day. Track the stopped UTC day in private state while keeping inspection and marker preparation failures scoped to one attempt. Document the wire limit, state lifecycle, and event-before-snapshot ordering. Add boundary coverage for exact capacity, partial-tail repair, marker recovery, and failure precedence.
Keep filename-to-kind pairing in DailyFileKind and expose one tolerant canonical-file scan for archive inspection. The high-water lookup remains strict because overlooking a later day could reopen closed storage.
gmemuriuki
force-pushed
the
telemetry/event-storage-limits
branch
from
October 20, 2026 13:26
e73d304 to
8d31feb
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds bounded low-volume event storage on top of the private storage foundation.
Complete relationship batches are serialized before the event file is opened and appended with one batch write. If an earlier failed append left an unterminated line, the writer closes that line before appending new rows so the new batch cannot be swallowed by malformed data.
The daily size policy accounts for both event and metric files, reserves space for the largest marker the running build can emit, and writes at most one
storage_limitmarker before stopping event appends for that day. Aggregate recording remains independent of a stopped event day.Tests cover exact-fit and one-byte-over boundaries, tail repair, whole-batch rejection, marker recovery from the file, stopped-day rollover, permission handling, and inspection or marker-preparation failures.
Disclosure questions
AI disclosure.
Questions for reviewers.
Could reviewers check the distinction between a capacity-closed day and an attempt-scoped suppression? Only a real capacity decision persists a stopped day; inspection and marker-preparation failures leave it open.