Skip to content

Recover tracing after span plugin failures - #130

Draft
Stephen Belanger (Qard) wants to merge 3 commits into
mainfrom
t3code/investigate-sdk-471
Draft

Stephen Belanger (Qard) wants to merge 3 commits into
mainfrom
t3code/investigate-sdk-471

Conversation

@Qard

Copy link
Copy Markdown
Collaborator

Summary

  • Raise the span plugin call timeout from 50 ms to 1 second for larger traces.
  • Pause delivery for only the affected source/session/route after a plugin failure, retaining journaled events and the exact failure offset. Retry when that same plugin file changes; leave other sessions and routes active.
  • Show the affected session, span, journal backlog, recovery state, and full local error in doctor. Emit a payload-free failure marker on the affected span ID while paused, then replace it with the normally transformed span and clear the marker error after replay succeeds.
  • Keep paused journals and session actors alive until recovery.

Validation

  • BT_DAEMON_CONFIG=/tmp/sdk-471-missing-settings.json cargo test --manifest-path bt-daemon/Cargo.toml --locked --all-features --quiet
  • cargo clippy --manifest-path bt-daemon/Cargo.toml --locked --all-targets --all-features -- -D warnings
  • cargo fmt --manifest-path bt-daemon/Cargo.toml --check and git diff --check
  • Live Braintrust check with a locally built bt using this daemon: fetched a failed span in project test with bt view span, edited only its failing plugin, then fetched the same span and full trace with bt view span/bt view trace. The span ID stayed the same, its normal transformed name and input appeared, and its error became null without another hook event.

SDK-471

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant