Skip to content

assurance: probe binaries, the trend tier and per-commit measure history #81

Open

Description

@debuggingfuture

Measures that need a process-wide allocator, a fresh process for RSS or wall-clock timing run as tools/probe binaries. A scheduled workflow runs every tier on the default branch and attaches each run report under refs/notes/measures (assurance.measure.history).

Ledger entries: audit-open-bounded-heap (retained heap across 1k and 100k entries) and audit-query-digest-keyed (unkeyed dictionary matches), each with its clause, and the trend entries.

Activity

  1. debuggingfuture commented on Oct 16, 2026

    @debuggingfuture
    MemberAuthor

    #563 settles the note credential and remote attestation (assurance.measure.note-credential, assurance.measure.remote-attest) and pins assurance.measure.run-report: measure-report reads FlareDispatch execution ids as run ids. The nightly contextful-measures run already attaches reports under refs/notes/measures. The remaining step is fractalboxdev/flare-dispatch#199, which builds the note line with contextful-ci measure-report so nightly notes carry trend annotations; this issue closes once it deploys.

  2. debuggingfuture commented on Oct 18, 2026

    @debuggingfuture
    MemberAuthor

    The 2026-10-18 nightly run is the first to post a flare-dispatch/contextful-measures check run. It was admitted at 02:17 UTC and is still queued an hour later: every sandbox slot is held, with no run queued ahead of it, and its admission deadline is 08:17 UTC. The newest refs/notes/measures note is still the 2026-10-15 run.

    That pattern fits the missing notes: a nightly run that never gets a slot writes no note. Several check executions on merged pull requests are still in_progress hours after dispatch, which points to stalled executions holding the pool rather than to a failing measures step.

    Next: read this run's terminal state. If it times out in admission, the fix belongs on the dispatcher side, either by releasing slots held by stalled executions or by reserving capacity for scheduled runs, rather than in this repository.

  3. debuggingfuture commented on Oct 18, 2026

    @debuggingfuture
    MemberAuthor

    At 03:10 UTC on 2026-10-18 the pool is busy with gate checks, not with measures. check:* children dispatched between 02:30 and 03:00 UTC are running for pull requests that have already merged (#608, #612, #613, #614, #618), next to children still in_progress since 2026-10-16 and 2026-10-17 (#605, #607). Merged pull requests keep consuming sandbox capacity, and the scheduled measures run waits behind them for its whole admission window.

    Two dispatcher-side fixes would address this: cancel a pull request's gate executions when it merges or closes, and give scheduled runs admission ahead of pull-request checks.

  4. debuggingfuture commented on Oct 18, 2026

    @debuggingfuture
    MemberAuthor

    Tonight's run did get a sandbox slot. It then ended at 03:59 UTC with Failing step: measure — StepFailed: ExecFailed: exec failed (exit -1): Connection closed: this Durable Object instance is no longer active. The container was recycled mid-measure, and the measure step ran with retries: 0, so no note was written. This is the failure that removes notes. The queueing described above delays a run but does not by itself drop a note.

    The fix is fractalboxdev/flare-dispatch#204, merged as 759aaa5a. The measure step now rebuilds a recycled checkout inside its retry and retries ExecFailed and CheckoutFailed twice, and the report steps read the workspace it measured. This issue closes once a nightly note with annotations lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions