Repository navigation
beholder: inject meter in workflow monitoring, gateway, heartbeat (Phase 5) - #23973
Conversation
|
✅ No conflicts with other open PRs targeting |
|
I see you updated files related to
|
00d477b to
4612cc8
Compare
4612cc8 to
62b7b70
Compare
62b7b70 to
9f8fdf8
Compare
acffd94 to
c147c06
Compare
PR Triage · Risk 🟡 Medium · Complexity 🔴 HighSuggests 1 approval, including one from an owner. Suggested reviewers: @bolekk, @MStreet3.
Start Review Here
VisualizationsScore mapflowchart LR
R["Risk: Medium"]
rs["1 signal → floor Medium"]
s0["hotspot core/services/workflows/v2/config.go"] --> rs
rs --> R
seq["sequential changes (2 in 7d)"] -->|"lifts Low → Medium"| R
C["Complexity: High"]
m0["logic added 80 ≥ 45"] --> C
RT["Suggests 1 approval · one from an owner"]
R --> RT
classDef low fill:#dafbe1,stroke:#1a7f37,color:#1f2328;
classDef medium fill:#fff8c5,stroke:#9a6700,color:#1f2328;
classDef high fill:#ffebe9,stroke:#cf222e,color:#1f2328;
class R medium;
class C high;
Change shapepie title Change shape
"Tests" : 960
"Source" : 209
— distributed across 12 files Files: 12 changed
Why these reviewers
Others who know this code: agparadiso 11%, jmank88 8%, mchain0 8%. Ranked by commits to the changed files. Commits in the 360 days before the head commit count, newer ones more. Requested everyone with at least 20% of the score, up to 4. Added 1 reviewer below the share bar to reach the approval floor. Excluded the PR author, @kalverra. Considered only current organization members. Metrics and policy
Risk signals: pr-triage v0.0.0-20261023193103-09473e7e1439 · policy |
c147c06 to
41ec02e
Compare
6c6fdad to
d39df3e
Compare
ff13d18 to
c8047de
Compare
c8047de to
8b4e5dc
Compare
8b4e5dc to
6cac96d
Compare
…ase 5) - monitoring: InitMonitoringResourcesWithMeter(meter) holds the moved body; InitMonitoringResources() wraps it on the global meter (cre.go caller unchanged) - engine: EngineConfig.Meter, nil defaults to beholder.GetMeter() at construction in newBaseEngine (not resolved in Validate) - tests: one newTestMeter helper per test package replaces setupTestMeter/setupInitFailureTestMeter global client swaps; all listed tests now t.Parallel() with private meters, assertions tightened to exact counts - gateway metrics_test: NewGatewayMetrics(meter) directly - heartbeat: WithTracer option (resolves tracer override TODO), dead WithMapLabels statement dropped; test asserts emitted BaseMessages via beholdertest.NewRecorder instead of swapping the global client After this, no test in core/ calls beholder.SetClient.
6cac96d to
689af79
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Four workflow metric registrations still bypass the injected meter, sending measurements to the global provider instead.
1 open issue
What changed in this PR
This PR extends explicit telemetry injection to workflow monitoring and heartbeat services, reducing reliance on process-global Beholder state and allowing isolated tests to run in parallel.
Changes:
- Adds an engine meter option and a meter-aware monitoring constructor while retaining the existing default entry point.
- Adds heartbeat tracer injection and replaces stdout-based assertions with an injected message recorder.
- Uses private meters in workflow and gateway tests, enabling parallel execution and exact counter assertions.
| File | Description |
|---|---|
| core/services/workflows/v2/init_failures_test.go | Uses private meters for parallel initialization-failure tests. |
| core/services/workflows/v2/engine_test.go | Injects meters and asserts exact subscription counts. |
| core/services/workflows/v2/engine_org_id_missing_test.go | Introduces a private-meter helper and parallelizes counter tests. |
| core/services/workflows/v2/engine_limit_metrics_test.go | Isolates limit-degradation metrics in parallel tests. |
| core/services/workflows/v2/engine_init_failure_metrics_test.go | Replaces global swaps with injected meters and exact assertions. |
| core/services/workflows/v2/config.go | Adds the optional engine meter dependency. |
| core/services/workflows/v2/base_engine.go | Resolves the configured meter at engine construction. |
| core/services/workflows/monitoring/monitoring.go | Adds meter-aware instrument registration. |
| core/services/gateway/monitoring/metrics_test.go | Supplies a private meter directly to gateway metrics. |
| core/services/chainlink/heartbeat.go | Adds tracer injection and removes a discarded labeler call. |
| core/services/chainlink/heartbeat_test.go | Replaces global writer-client assertions with a message recorder. |
Review effort: Balanced
|





Intent
Layer 4 of the beholder emitter-injection stack (DX-5066): engine-metrics tests swapped the process-global beholder client to install a manual meter and the heartbeat test asserted on a writer client's stdout, serializing them and racing parallel engine tests; this PR injects meters, tracers, and emitters explicitly so no core test calls
beholder.SetClient.Notes
EngineConfig.Meteris deliberately not defaulted inValidate—newBaseEnginefalls back tobeholder.GetMeter()at construction, preserving today's read timing.InitMonitoringResources()remains forcre.go; it now delegates toInitMonitoringResourcesWithMeter(beholder.GetMeter())with identical timing.TODO allow override of tracer:WithTracerdefaults tobeholder.GetTracer()at construction, and the dead discardedcme.WithMapLabels(labels)call is removed.Changes
Where the engine's meter comes from:
InitMonitoringResourcesWithMeter(meter)One
newTestMeterhelper replaces both global-swap helperssetupTestMeter/setupInitFailureTestMeterbecome a private-meter helper per test package (example); assertions tighten from "tolerate foreign increments" to exact counts (TestEngine_Init).Heartbeat asserts on a recorder, not stdout
TestHeartbeat_MeterEventsinjectsWithEmitter(NewLabelerWithEmitter(rec))and asserts on decoded BaseMessages; thebyteCollector/writer client and the restore-to-noop bug are deleted.Gateway metrics test builds its meter directly
NewGatewayMetrics(meterProvider.Meter("gateway-test"))with no global swap (metrics_test.go).Requires
#23972 (rest of the DX-5066 beholder stack)
Supports
Follow-up DX-5066 parallel-test layers (02–07)
Deployment Validation Steps
Run a node locally with telemetry enabled and confirm engine metrics (
platform_engine_*) and heartbeat messages still reach the collector; no metric names or labels changed.Stack created with GitHub Stacks CLI • Give Feedback 💬