fix: apply invocation deadlines to SDK waits and cleanup

[zhongkechen]

Issue Link, if available

Partial fix for #726. Stacked on #731 (feature/lmi-invocation-scope); keep this PR in Draft. Latest upstream main (67bad28cd9ef1b0808bed193f8a15be75ce457ed) and #731 (d9b251c5003c03e997df6d07dd05116441509a88) are included.

Description

An invocation deadline or interrupted cleanup must not leave the runtime caller in an unbounded SDK wait, return PENDING while cooperative root cleanup is still running, or turn an interruption into a durable business failure. This change adds deadline-aware admission and cleanup to #731's invocation-owned task tracking while preserving current main's worker-owned Start, handler, drain, result preparation, and End ordering.

The runtime caller observes the handler outcome and separately waits for actual task exit. Graceful cleanup gives admitted work an opportunity to settle before requesting cooperative cancellation, closes ordinary admission atomically with the final empty task snapshot, and uses the remaining budget for checkpoint and batcher shutdown. A handler-drain interruption continues best-effort cleanup so an admitted late step can still checkpoint its result for replay; a caller interruption or deadline cancels invocation-owned work. Both paths attempt remaining cleanup and restore the interrupt flag.

Deadline checks cover user callbacks, checkpoint dispatch, metadata pagination, and response-critical large-result checkpoints. An accepted terminal execution result remains authoritative at a pagination cutoff without starting additional metadata requests. Unrelated service failures retain their classification, and an available JVM fatal keeps its original identity rather than being replaced by a timeout or an acknowledged-result shortcut.

If an uncooperative root exits after the caller has timed out, the original worker still runs its matching End hook once to restore Start-installed scopes. That late path cannot admit new durable work or prepare another result. Existing durable identities, stored success/failure replay, shared-executor ownership, token-revocation registration semantics, and fatal-error propagation are retained.

Internal limits are 500 ms execution headroom, 100 ms response headroom, up to 5 seconds graceful user-work drain, up to 1 second cancellation drain, and a 30-second local/null-context cleanup cap. A real Lambda deadline constrains these further; root result preparation and End use the remaining overall budget after the handler body exits.

This is a partial lifecycle fix, not an unconditional bounded-return or worker-reclamation guarantee. Arbitrary application I/O, synchronous initial-state reads, custom clients, and plugin hooks cannot be forcibly terminated. An executor that runs the root inline (including a saturated CallerRunsPolicy) can block the runtime caller before it reaches the timeout-aware wait. Shared fixed-pool starvation remains separate work in #735; cloud LMI acceptance is still pending. The configuration guide documents these limits and the need for finite downstream I/O timeouts.

Testing

Final head 242b88fcda0f1bdf92bb515809b95acdf2393c57 additionally includes the reviewed #728 harness-only follow-up and an observer-only test stabilization. Production source at 947daa1bc16cff037bbdd29ef6e38535f8d0131d includes the atomic task-publication follow-up from #731:

• Java 17 full mvn -o -B clean verify: all 10 reactor modules passed; 3,022 tests passed, 34 conditional/cloud tests skipped. This includes 553 integration tests and 810 OTel tests.
• Focused lifecycle/admission/replay controls cover physical cancellation before completion observers, interrupted cleanup with cooperative exit, persisted late-step success and replay, fatal precedence, terminal acknowledgment plus pagination cutoff, eventual End scope restoration on worker reuse, and atomic final admission closure. New race regressions were checked against the pre-fix behavior.
• Both opt-in LMI lifecycle controls passed: root-finally/PENDING ordering and deadline cancellation with bounded SDK cleanup.
• Java 25 focused lifecycle/admission/replay controls and Spotless: 40 passed before the atomic-publication follow-up; that follow-up separately passed all 11 Java 25 task-lifecycle controls in feat: add invocation-scoped task lifecycle foundation #731.
• All 179 Python LMI harness tests passed on the final merged head; the Java production tree is unchanged from the full reactor validation above.
• Observer-only integration/OTel fixture updates recognize the caller's timeout-aware get wait and renamed drain helper. The final admission gate retains its observed BLOCKED state instead of racing a second snapshot; all admission/exit assertions remain. All 25 Java 25 cleanup/task-lifecycle controls and Spotless passed at c2e6c5ac; the final merge changes only Python harness files and their documentation.

Cloud LMI execution was not run; #728 supplies the validation workflow. No PR merge or release is included.

Checklist

• Preserved replay operation identity and existing public durable APIs.
• Never shuts down the shared user executor.
• Keeps the PR in Draft state for stacked review.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.