Skip to content

fix: preserve OTel failure and replay semantics - #769

Open
zhongkechen wants to merge 96 commits into
mainfrom
test/otel-conformance-21-24
Open

zhongkechen wants to merge 96 commits into
mainfrom
test/otel-conformance-21-24

Conversation

@zhongkechen

@zhongkechen zhongkechen commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Description

Failed durable operations now produce ERROR spans even when the service supplies no error details. FAILED, TIMED_OUT and STOPPED determine status independently of exception diagnostics; statusless virtual child failures also remain ERROR. Only real exception details produce an exception event. A handler can catch the operation failure and return successfully without losing the failed operation's telemetry or inventing an exception.

Execution-view failures with a known, nonzero backend end place the public exception event at that end, using it as the start only when no start was observed. Standard exception attributes and provider event limits apply. Invocation-view and unknown-end native recording remain unchanged; starts are not clamped across clock skew. Existing user-context, sampling, continuation-parent and input-update delivery fixes remain included.

Cached failed waitForCallback contexts whose child details have been omitted now throw the existing ChildContextFailedException with the actual failed context and stored error, instead of IllegalStateException("Unknown waitForCallback status"). Specific callback, timeout and submitter exceptions are retained when their details are available. Contradictory fully present state still fails internally. Completed bodies are not repeated, and missing child operations or failure categories are not fabricated.

The local testing runner now matches the service's child-retention behavior, corresponding to aws/aws-durable-execution-sdk-js#1002:

  • Invocation input and GetDurableExecutionState omit a descendant when any ancestor CONTEXT is SUCCEEDED or FAILED without ReplayChildren=true. The walk includes running intermediate ancestors and is independent of store order; a retaining inner context cannot override a pruning outer context.
  • Only the closing SUCCEED/FAIL update determines ReplayChildren; START ignores the option, and an absent closing option becomes false.
  • Checkpoint response operations, UpdatedOperationIds, raw storage and public result/history inspection retain their existing contents. The local state API returns a single page; this change does not add pagination.

Public-handler tests exercise first execution, suspension and replay, including small completed contexts, legitimately retained large child results, nested retention, failure containers and skipped completed side effects. Map name-validation tests distinguish retained child metadata from omitted metadata: the SDK still rejects changed names when the old names are visible, but cannot reconstruct omitted names from an aggregate result. Operation names must remain deterministic in either case.

Conformance and validation

Cases 21–26 remain paired with shared #131 catalog 05960beb6c81d7b24b53114f2581b3b6f9fef6d4. Case 25 catches a failed callback without details, crosses two replay barriers and returns successfully while retaining ERROR with no invented exception on the failed leaf. Case 26 uses a real root callback followed by replay barriers. These handlers use public APIs, not synthetic completion notifications.

Current head c853af6eaf360907a00a72df31171a1b409dc033 incorporates main ea4b68fb4237ca6a43bb2c722455332e9489086e. This latest base update changes only the reusable AI-review workflow reference to 8a06a1e5c82083a9224af9d489ddd030be44d978 (#797). The workflow YAML parses and matches the upstream file; events, permissions and inputs are unchanged. Every other tracked file, including SDK/runtime code, tests, dependencies, conformance fixtures and release metadata, is identical to the previously validated PR head.

Prior Java 17 validation on the unchanged SDK and test source built all nine reactor modules: 3,186 tests passed, 31 conditional/cloud tests skipped, and zero failures/errors. All 49 ExecutionManagerTest cases, full Spotless checks and the 16/16 installed core/plugin API compatibility cases passed. New-head CI, including the supported Java-version matrix and deployed conformance, is tracked separately. The testing-runner pruning behavior, conformance handlers/catalog reference and OTel implementation are unchanged by this base integration.

The earlier Java head aee9443b passed all 112 cloud checks against the same 05960beb catalog in run 38637008099. That is historical evidence, not this head's CI result. New-head CI is tracked separately. Shared #131's newer chained-invoke Parent assertions are not qualified by the older catalog or these local tests.

Boundaries

This runner parity change does not fix the independent virtual FLAT duplicate-export issue in aws/aws-durable-execution-sdk-js#997, including the case where a map is still STARTED. No first-completion-losing heuristic or durable exporter acknowledgement is added. Shared-plugin factory/lifetime changes remain in #782. Visible-provider flush waiting remains bounded to five seconds; opaque providers are not privately unwrapped, and arbitrary blocking processor code is not promised to be cancellable. No package, tag or release is published.

@zhongkechen
zhongkechen marked this pull request as ready for review October 3, 2026 00:03
@zhongkechen
zhongkechen requested a review from a team October 3, 2026 00:03
@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 00:03 — with GitHub Actions Active
@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 00:03 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 01:55 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 02:05 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 02:18 — with GitHub Actions Active
Comment thread sdk/src/main/java/software/amazon/lambda/durable/execution/DurableExecutor.java Outdated
Comment thread sdk/src/main/java/software/amazon/lambda/durable/execution/DurableExecutor.java Outdated
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 03:08 — with GitHub Actions Active
Comment thread sdk/src/main/java/software/amazon/lambda/durable/execution/DurableExecutor.java Outdated
@zhongkechen zhongkechen changed the title test: cover OTel replay and user callback context in Java fix(otel): preserve operation telemetry across replay Oct 14, 2026
@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 14, 2026 19:54 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 14, 2026 21:02 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 14, 2026 21:21 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen zhongkechen changed the title fix(otel): preserve operation telemetry across replay fix: preserve OTel failure and replay semantics Oct 15, 2026
@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 15, 2026 00:46 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 16, 2026 19:54 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 17, 2026 01:52 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown
Contributor

Codex AI review

No actionable findings in the PR changes. Residual test risk: tests and cloud conformance were not executed during this read-only review.

Reviewed commit c853af6eaf360907a00a72df31171a1b409dc033. Workflow run

This branch was successfully deployed

1 active deployment
ai-pr-review-runtime — c853af6e Deployed Oct 17, 2026 by zhongkechen via ai-pr-review / Codex review / Generate Codex review #2416
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants