Skip to content

Track skipped final-result checkpoint acknowledgment and late worker admission #787

Open

Description

@zhongkechen

Summary

Track the potential loss of an oversized execution result when its final checkpoint is skipped, together with the related late user-worker admission boundary. This is the follow-up requested in the #767 review.

This report records static code-path analysis, not a runtime-confirmed reproduction. The reviewer explicitly noted that the trace has not been run. The concern applies to current main 1064d0ec32e5b1650b4fa08e65ca960361c7cd1b, independently of the fallback-root telemetry change.

Potential failure path

  1. The root handler outcome is selected before result preparation.
  2. A continuation can subsequently fail and complete invocation control before or during preparation.
  3. CheckpointManager.checkpointBatch can return without sending an update when that control is already exceptional.
  4. The batcher can nevertheless complete the update's observation normally.
  5. For an output beyond the inline result limit, handleLargePayload treats that completion as persistence and returns an empty inline result.

If the described interleaving is reachable, the response could report SUCCEEDED with an empty inline result while its required EXECUTION/SUCCEED checkpoint was never stored. Closing new continuation registration before serialization narrows admission but does not itself prove delivery of that mandatory checkpoint.

Related admission boundary

Review r4262386190 identifies an already-admitted continuation that passes a closing check, waits for earlier work or dispatch, and then starts a fresh user worker after root selection. Registration admission and later worker admission need to be distinguished; a registration gate alone is not a guarantee that all accepted work has stopped.

Follow-up requirements

  • A required checkpoint that was not sent must not be acknowledged as persisted. An empty inline result used for durable result offloading must correspond to confirmed checkpoint completion.
  • Validate the suspected result-preparation and worker-admission interleavings described in the linked reviews, including caller outcome, persisted result and replay behavior. No dynamic reproduction is claimed in this issue.
  • Define success/control arbitration and admission guarantees explicitly before adding a stronger global outcome or cleanup policy. Preserve original failures and the behavior of completed replayed work.

Earlier discussions: r4233196696, r4233420625, and r4262385920. This is a tracking report; no automated implementation is requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions