Emit vault get-secrets capability execution event for CRE UI by wentzeld · Pull Request #24039 · smartcontractkit/chainlink

wentzeld · 2026-10-21T15:33:00Z

Wrap the Vault get-secrets host entry points so each vault.secrets.get batch call emits a
workflows.v2 CapabilityExecutionStarted/Finished pair, matching the events the engine
already emits for other capability calls (see callCapability). The event uses the real
workflow execution ID and finishes errored on failure. No new protos or consumer changes.

Previously Vault getSecrets bypassed the generic capability-executor emit path (the SDK
routes secrets to the host-side secretsFetcher, which calls the vault capability
directly), so Vault reads never appeared in the CRE UI. This change emits the same event
type/fields as every other capability call, so it shows up as a first-class execution step
with no consumer changes.

Covers both host entry points:

ExecutionHelper.GetSecrets — normal workflows.
executionHelperWithRawSecrets.GetRawSecretsResponse — confidential (TEE) workflows.
(GetRawSecrets delegates to GetRawSecretsResponse, so it emits once.)

Both delegate to a shared emitVaultGetSecrets helper.

Design

One Started/Finished pair per batch call (each host entry point is the batch boundary).
capability_id = vault@1.0.0, method = vault.secrets.get, step_ref = CallbackId.
workflow_execution_id = the real engine execution ID; labels from base.eventLabels().
Non-nil error → Finished emitted with errored and the error string.
No secret values are included in the event (only id/method/step/status/error).
design doc

Test plan

go test ./core/services/workflows/v2/ -run 'TestExecutionHelper_GetSecrets|TestExecutionHelper_GetRawSecretsResponse' -v
- GetSecrets success + errored-on-failure (normal path).
- GetRawSecretsResponse success + errored-on-failure (confidential/TEE path).
Assert capability_id = vault@1.0.0, method = vault.secrets.get, real execution ID,
step_ref = CallbackId, and status/error on both paths.

Requires

None.

Supports

None.

Deployment Validation Steps

Low blast radius: additive observability only, no change to request/response behavior,
no proto or consumer changes.

Deploy to staging.
Run a normal workflow that calls runtime.GetSecrets against a populated secret.
Run a confidential workflow that fetches secrets, and confirm the CRE UI execution view
shows a vault@1.0.0 / vault.secrets.get step for it too.
Confirm the step shows status = succeeded, tied to the correct execution ID.
Force a failure (e.g. missing secret / vault error) on each path and confirm the step
shows status = errored with the error surfaced.
Confirm no secret values appear in the emitted event and no duplicate events are produced.

github-actions · 2026-10-21T15:33:22Z

👋 wentzeld, thanks for creating this pull request!

To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team.

Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks!

github-actions · 2026-10-21T15:34:16Z

✅ No conflicts with other open PRs targeting develop

github-actions · 2026-10-21T15:34:24Z

I see you updated files related to core. Please run make gocs in the root directory to add a changeset as well as in the text include at least one of the following tags:

#added For any new functionality added.
#breaking_change For any functionality that requires manual action for the node to boot.
#bugfix For bug fixes.
#changed For any change to the existing functionality.
#db_update For any feature that introduces updates to database schema.
#deprecation_notice For any upcoming deprecation functionality.
#internal For changesets that need to be excluded from the final changelog.
#nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
#removed For any functionality/config that is removed.
#updated For any functionality that is updated.
#wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

trunk-io · 2026-10-21T15:50:06Z

_{View Full Report ↗︎ ⋅ Docs}

wentzeld · 2026-10-21T17:07:57Z

Re this Eth Smoke test and Intgration test failure

Neither failure is PR code. Both trace to a single root cause.
ETH Smoke Tests is just the aggregator. Its "Check all E2E test results" step reported CRE_REGRESSION_RESULT: failure — it fails whenever the regression shard fails.
The real failure is Test_CRE_V2_EVM_WriteReport_Invalid_Gas_Regression, and it failed in setup-go, not in the test.

From the job log:
##[error]Process completed with exit code 1.
##[end-action id=setup-go.__run_3;outcome=failure;conclusion=failure;duration_ms=552840]
The cause is transient Go module-proxy errors while downloading dependencies:

chainlink-ccip/chains/solana/gobindings@v0.0.0-...: read "https://proxy.golang.org/...zip": stream error: stream ID 4049; INTERNAL_ERROR; received from peer

apache/arrow-go/v18@v18.6.0: ... stream error: stream ID 4821; INTERNAL_ERROR; received from peer
google/pprof@...: ... stream error: stream ID 4269; INTERNAL_ERROR; received from peer
The test never executed — the downstream steps confirm it:

##[error]Failed to read input file: open /tmp/junit-report-regression.xml: no such file or directory
[J Total: 0 Pass: 0 Fail: 0 Quarantined: 0 Pass Ratio: N/A

Total: 0 = no tests ran; the JUnit enhancer then failed on the missing report, which is what flipped the job red. Those deps (Solana CCIP gobindings, arrow, pprof) are unrelated to a workflow-engine event change, and nothing in the diff touches them.

Conclusion: transient proxy.golang.org flakiness (I hit the same class of stream error ... received from peer when fetching logs). No code change needed

bolekk

LGTM overall :)

bolekk · 2026-10-21T19:51:58Z

+// call so it surfaces in the CRE UI like any other capability call (see
+// callCapability). One pair is emitted per batch call; a failed call finishes as
+// errored with the returned error.
+func emitVaultGetSecrets[T any](


nit: should probably be called "runWithGetSecretsEvents" or something along those lines. It doesn't just emit, it actually runs the provided function.

Making this change

bolekk · 2026-10-21T19:54:58Z

+	run func() (T, error),
+) (T, error) {
+	stepRef := strconv.FormatInt(int64(callbackID), 10)
+	_ = events.EmitCapabilityStartedEvent(


Any risks during rollout? What happens if a subset of nodes are upgraded and we fail to aggregate those events? Will the user see some broken state or nothing at all?

If there isn't quorum, then they just won't see this particular message. I'll confirm

bolekk · 2026-10-21T19:57:54Z

+
+	v, err := run()
+
+	status := store.StatusCompleted


Flagged by Claude: Missing or unauthorized secrets were already reported inside the SecretResponse entries with a nil err. Before the PR nobody reported a status for them. The PR now reports succeeded for those calls, which is a new inaccuracy, not a changed behavior.

Probably not a big deal but potentially confusing to users.

@bolekk this one has two options for fixing, and I'm leaning toward the cheaper option.
Option 1: Add partial success to Capability proto. I don't want to do this now as it is a much bigger piece of work across repos as we don't have partial success on capability calls.
The one I'm implementing: Option 2: Show workflow status in the UI as completed with errors. The capability will show success with an error message saying 2 of 5 fetches failed, while the workflow execution status will be completed with errors (if it succeeded) or failed (if it failed)

Wrap ExecutionHelper.GetSecrets so each vault.secrets.get batch call emits a workflows.v2 CapabilityExecutionStarted/Finished pair, matching the events the engine already emits for other capability calls. The event uses the real workflow execution ID and finishes errored on failure. No new protos or consumer changes.

Emit the capability execution Started/Finished pair from executionHelperWithRawSecrets.GetRawSecretsResponse as well as ExecutionHelper.GetSecrets, via a shared emitVaultGetSecrets helper, so confidential (TEE) workflows surface in the CRE UI like normal ones. GetRawSecrets delegates to GetRawSecretsResponse, so it emits once.

These tests can't call t.Parallel(): they assert on events emitted via beholder.GetEmitter(), the process-global client, so they must swap the global client with beholdertest.NewObserver, which is not parallel-safe. Matches the existing convention (e.g. engine_drop_paths_test.go).

Classify each get-secrets batch by per-secret outcome: all succeed -> completed, some fail -> completed with an error summary (so the CRE UI derives completed_with_errors), all fail -> errored. The call still returns its responses; only the emitted event status/error changes.

cl-sonarqube-production · 2026-10-23T23:44:50Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

wentzeld requested a review from a team as a code owner October 21, 2026 15:33

product-security-plaid-production Bot requested a review from DylanTinianov October 21, 2026 15:49

russell-stern previously approved these changes Oct 21, 2026

View reviewed changes

bolekk reviewed Oct 21, 2026

View reviewed changes

wentzeld dismissed russell-stern’s stale review via 632a92e October 23, 2026 19:59

wentzeld added 5 commits October 23, 2026 16:23

Emit vault get-secrets events via injected EventEmitter

f392a90

wentzeld force-pushed the add-getSecrets-event branch from 632a92e to f392a90 Compare October 23, 2026 23:26

Use tagged switch in vaultSecretBatchOutcome (staticcheck QF1002)

47deb2e

Conversation

wentzeld commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Design

Test plan

Requires

Supports

Deployment Validation Steps

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

trunk-io Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wentzeld commented Oct 21, 2026

Uh oh!

bolekk left a comment

Choose a reason for hiding this comment

Uh oh!

bolekk Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

wentzeld Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

bolekk Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

wentzeld Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

bolekk Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

wentzeld Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

cl-sonarqube-production Bot commented Oct 23, 2026

Quality Gate passed

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wentzeld commented Oct 21, 2026 •

edited

Loading

trunk-io Bot commented Oct 21, 2026 •

edited

Loading