[akcll]

What

This PR works in conjunction with this PR in the CRE Storage Service,
https://github.com/smartcontractkit/cre-storage-service/pull/526

It allows for LocalCRE to be extended with a DockerCompose include, plus DNS Aliases, such that Central Services can define their own Docker Compose and provide extension to Local CRE.

Why

LocalCRE needs the ability to test with Centralized Services, for more sophisticated Integration Testing both locally and in CI.
See this doc by @elatoskinas : https://docs.google.com/document/d/1fhzq-pVEJXNLepO_QVfrnJgwIcBZoN0MrFfmIk0Mg4A/edit?tab=t.0#heading=h.4e7ng0tekbkc

How

Prior version of this PR used a brand new Topology, but after some feedback from @Tofel and @elatoskinas, centralized services are now an opt-in that works with any existing topology. Adding an [extensions] section to the TOML config (configs/extensions/extensions.toml as an overlay in CTF_CONFIGS) points the workflow DON nodes at a real storage service instead of the file fetcher stub. The nodes address the services by DNS aliases (cre-workflow-source, cre-artifact-api, cre-artifact-store).

When extensions are configured, env start stands the services up on ctf after the DONs, passing them the registry chain, CapabilitiesRegistry address and DON family from the running environment, and env stop removes them. Each service answers to the aliases the DONs rely on, so services can be composed for testing. The first of these is the Storage Service, which has a new Docker Compose to support Local CRE here.

I used --with-billing as a prior art example to drive these changes.

This also changes the vault feature to set only the WorkflowRegistry fields it owns, rather than replacing the whole section, so that settings like WorkflowStorage and AdditionalSources survive.

Reproduction Steps

I've run these steps myself to confirm the latest versions of both PRs hang together correctly. These steps may pose challenge depending on what environment you are trying to run them (like, I don't know if this is how you'd launch the services in CI)

Assumes chainlink and cre-storage-service are checked out side by side. If not, set storage_compose in configs/extensions/extensions.toml to your cre-storage-service/local-stack/compose.yaml.

1. Unit test

cd chainlink/core/scripts/cre/environment
go test ./environment/ -run TestExtensionsNodeConfig -count=1

2. Build the storage images

The images aren't published, so build them locally.

cd cre-storage-service
GITHUB_TOKEN=$(gh auth token) docker compose --env-file local-stack/standalone.env \
  -f local-stack/compose.yaml -f local-stack/compose.build.yaml build

3. Start Local CRE with extensions

cd chainlink/core/scripts/cre/environment
CTF_CONFIGS=configs/workflow-gateway-don.toml,configs/extensions/extensions.toml go run . env start

The cre-storage-<id>-* containers come up healthy after the DONs, started through the CTF cre_storage_service component (dockercompose is pinned to chainlink-testing-framework#2869's head until it's released). env start logs the host URLs of the roles, which are on free ports. You don't need -e, a separate docker compose up, or a node restart.

4. Check the wiring

# Workflow node config generated in Go
docker logs workflow-node1 2>&1 | grep -m1 '# Input Configuration' \
  | grep -o 'cre-artifact-api:50052\|cre-artifact-store:4566\|cre-workflow-source:50052' | sort -u

# Stack inputs taken from the running environment
docker inspect $(docker ps -q --filter name=storage-node-service) --format '{{range .Config.Env}}{{println .}}{{end}}' \
  | grep 'TENANT_CR_RPC_URL\|DON_FAMILY_WORKFLOW'

# Nodes polling the storage service
docker logs workflow-node1 2>&1 | grep 'Loaded workflows from GRPC source' | tail -1

Expected:

• the three role addresses;
• TENANT_CR_RPC_URL=http://anvil-1337:8545 and {"1":{"test-don-family":100}};
• a recent "Loaded workflows from GRPC source" line for test-don-family.

5. Compile a workflow

go run . env workflow compile -w ./examples/workflows/cron/main.go -n extensioncron03

Note the path it prints ("Workflow saved to …/extensioncron03.br.b64").

6. Build deploy-cli for Linux

deploy-cli currently builds only against the local registryclient, so this uses a workspace file.

cat > $TMPDIR/deploycli.work <<EOF
go $(go env GOVERSION | sed 's/^go//')

use (
      $PWD/../../../../../cre-storage-service/tools/deploy-cli
      $PWD/../../../../../cre-storage-service/registryclient
)
EOF
(cd ../../../../../cre-storage-service/tools/deploy-cli && \
  GOWORK=$TMPDIR/deploycli.work GOOS=linux CGO_ENABLED=0 go build -o $TMPDIR/deploy-cli-linux .)

7. Deploy through cre-deploy-api

Artifact URLs use cre-artifact-store, which only resolves on ctf, so the deploy runs in a container on that network.

WF=$(dirname <path printed in step 5>)
CFG=$PWD/examples/workflows/cron
deploy() {
  docker run --rm --network ctf \
    -v $TMPDIR/deploy-cli-linux:/deploy-cli:ro -v $WF:/wf:ro -v $CFG:/cfg:ro \
    busybox:uclibc /deploy-cli --addr cre-deploy-api:50051 --tls=false \
    --key-id cre-connect --secret local-cre-connect-hmac-key --org-id org_local \
    --don-family test-don-family "$@"
}
SINCE=$(date -u +%Y-%m-%dT%H:%M:%SZ)
deploy --workflow-name extensioncron03 --binary-path /wf/extensioncron03.br.b64 --config-path /cfg/config.json

Expected: Status: WORKFLOW_STATUS_ACTIVE and URLs on cre-artifact-store:4566. Keep the workflow ID as ID=<workflow id>.

8. Confirm the nodes run it

Wait about 45s.

for c in workflow-node0 workflow-node1 workflow-node2 workflow-node3; do
  echo "== $c"
  docker logs --since $SINCE $c 2>&1 \
    | grep -E 'handled event \(WorkflowActivated\)|Workflow execution finished successfully' \
    | grep -o '"ts":"[^"]*"\|"msg":"[^"]*"' | paste -sd' ' - | head -2
done
docker logs --since $SINCE $(docker ps -q --filter name=storage-node-service) 2>&1 | grep -o 'method":"[^"]*' | sort | uniq -c

Expected:

• every node logs handled event (WorkflowActivated) and then Workflow execution finished successfully;
• node-service logs ListWorkflowMetadata and DownloadArtifact calls.

9. Lifecycle (optional)

Space the steps at least 30s apart; nodes poll every ~12s.

deploy --pause $ID
deploy --activate $ID
deploy --delete $ID
docker logs --since $SINCE workflow-node0 2>&1 | grep -o 'handled event ([A-Za-z]*)' | uniq

10. Teardown

go run . env stop
docker ps -a --format '{{.Names}}' | grep cre-storage   # expect nothing

Tool Use Notice

All of this code was generated, but ultimately reviewed and and revised by me. Selection of specific values (like port number) were done based on prior art, but I don't have strong feelings about any of them if there is reason to change them.

[github-actions]

✅ No conflicts with other open PRs targeting develop

[github-actions]

I see you updated files related to core. Please run make gocs in the root directory to add a changeset as well as in the text include at least one of the following tags:

• #added For any new functionality added.
• #breaking_change For any functionality that requires manual action for the node to boot.
• #bugfix For bug fixes.
• #changed For any change to the existing functionality.
• #db_update For any feature that introduces updates to database schema.
• #deprecation_notice For any upcoming deprecation functionality.
• #internal For changesets that need to be excluded from the final changelog.
• #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
• #removed For any functionality/config that is removed.
• #updated For any functionality that is updated.
• #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

[trunk-io]

   

Failed Test	Failure Summary	Logs
Test_CCIP_TokenTransfer_EVM2Sui_PoolReleaseOrMintTransmitterOwned_Rejected	The log is truncated before the failure details, so the underlying cause of the CCIP EVM-to-Sui token transfer test failure cannot be determined.	Logs ↗︎

View Full Report ↗︎ ⋅ Docs

[cl-sonarqube-production]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube