Repository navigation
feat: add run telemetry probes to diagnostics - #205
Merged
Merged
Conversation
Adds packages/tools/src/telemetry with probes that translate polled host and process data into the existing Diagnostics sink: run context, host resources (load, CPU split, vm_stat memory pressure), Node runtime (memory, event loop lag), lifecycle driven process snapshots and a platform collector bridge. Probes are started through one composition point and cost nothing when diagnostics are disabled.
waitForBoot now returns the last stage and elapsed time that simctl bootstatus reported. Rather than adding diagnostics calls to the runner, instrumented() accepts optional resultAttrs/errorAttrs derivers; the simctl module is wrapped with ones that attach the stage to the platform.ios.simctl.waitForBoot span, including when the wait times out or is aborted (the case that matters most). Platform runners can expose an optional createTelemetryCollector(); the simulator runner implements it to count guest launchd services at platform.ready and dispose through the (instrumented) simctl module.
diagnostics can now be a boolean or an object with enabled, probes (host, node, snapshots, platform) and sampleIntervalMs. isDiagnosticsEnabled keeps its semantics (true, an object that does not opt out, or RN_HARNESS_DIAGNOSTICS), and resolveDiagnosticsOptions fills in defaults.
The session now starts telemetry through one composition point (startSessionTelemetry) and only emits lifecycle phases (session.created, metro.ready, platform.ready, app.ready, test-file.start/end, dispose); probes translate them into diagnostics entries. End-of-run collection is awaited, with a bound, before the platform is torn down. Entries are appended to .harness/diagnostics/<sessionId>.jsonl as they arrive (batched, flushed on exit and on SIGINT/SIGTERM) so a cancelled CI job keeps its data. Gauges are written to the Chrome trace as counter events so Perfetto graphs them. summarizeRunKpis reduces a run to boot wait, time to ready, host pressure peaks and the result; it is printed, written to <runId>.summary.json and appended to GITHUB_STEP_SUMMARY when set.
TelemetryCollector.collect now receives an abort signal that fires when the probe stops or the context aborts, and the iOS guest process listing passes it to its spawn, so teardown never waits on a measurement nobody will read. run.dispose() no longer waits for work whose context was already aborted, and bounds the wait otherwise. Queued probe jobs are skipped once a probe stops or aborts, so no collect() runs after collector.dispose() and no AbortError snapshot marks are written. The process snapshot is now taken at platform.ready, the first app.ready and dispose only, instead of after every app relaunch. Samplers pause on the new run.end phase and resume on run.start or dispose, so a watch-mode session no longer accumulates idle samples while a single run still samples from session start to teardown. Diagnostics.subscribe gains a replay option for sinks attached after entries were recorded. guest_process_count now counts only launchd rows with a PID (running processes), not loaded jobs.
The appender now writes asynchronously, one chunk at a time, so recording an entry never blocks on disk; only the exit handler writes synchronously (and also writes chunks still pending). SIGINT/SIGTERM handlers are removed: the session owns those signals and leaves through process.exit, and a re-raise that depends on listener order can fight other handlers. The appender replays entries recorded before it attached, so session.config.load is in the log. Entries recorded after a run's report was written (the dispose-phase process snapshot and guest process count, dispose.total) are appended to that run's trace when the session is disposed. executeRun emits run.start and run.end so samplers pause between runs in watch mode. Telemetry is also stopped when setup fails before the main try block.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The runtime package typechecks the tools sources with the DOM lib, where setInterval and setTimeout return numbers, so calling unref() on the result and annotating it as NodeJS.Timeout failed CI. Timers now go through a guarded unrefTimer helper and ReturnType-based annotations that hold under both libs.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this?
Harness diagnostics now record resource telemetry alongside the existing timing spans. When diagnostics are enabled, a run samples host load, CPU and memory pressure, the Harness process's memory and event-loop lag, takes process listings at a few run milestones on macOS, and reports the iOS simulator's boot stage and guest process count. Entries are written incrementally, so a cancelled or timed-out CI job keeps its data, and each run ends with a short key-number summary.
This closes the gap that made iOS E2E failures on small CI runners impossible to diagnose: the simulator boot wait there takes minutes instead of seconds, the box runs out of memory and into the compressor, and the only trace was written at the end of the run, which never came when the job hit its timeout.
How does it work?
The existing
Diagnosticsobject stays the single sink and gains agaugeentry kind. Probes live in a separatetelemetrymodule in@react-native-harness/tools, one per file: run context, host resources (vm_staton macOS,/proc/meminfoon Linux, load and CPU deltas everywhere), Node runtime, process snapshot, and a platform collector. They are started from one composition point in the Jest package and react to lifecycle phases the session emits (session.created,metro.ready,platform.ready,app.ready,run.start,test-file.start,test-file.end,run.end,dispose). Business code only emits those phases; it makes no diagnostics calls. Platforms can expose an optionalcreateTelemetryCollector(); the iOS simulator runner uses it for the guest process count, and the boot stage reported bysimctl bootstatuslands as attributes on the existing boot span through the instrumented proxy.Timer-driven sampling only uses cheap sources and skips overlapping samples. Heavy collection (
top,launchctl listinside the simulator) runs only atplatform.ready, the firstapp.ready, anddispose, and is bounded on teardown. When diagnostics are off, no probe, timer, listener, or process is started. Samplers pause between runs in watch mode.The
diagnosticsconfig option accepts an object (enabled,probes: { host, node, snapshots, platform },sampleIntervalMs) in addition totrue, andRN_HARNESS_DIAGNOSTICSkeeps working. Entries stream to.harness/diagnostics/<sessionId>.jsonl; the Chrome trace still goes to<runId>.json, with gauges as counter tracks that Perfetto graphs; KPIs (boot wait, time to ready, peak load, peak compressed memory, minimum free memory, swapouts, event-loop lag) are printed, saved to<runId>.summary.json, and appended to the GitHub Actions job summary when available.Why is this useful?
It turns "the simulator did not start" into numbers: how long the boot waited and at which stage, what the load average and free memory were at the time, whether the Harness process itself was starved, and which processes held the memory. Those numbers survive a cancelled job and can be compared across runs, so decisions about runner size, simulator slimming, or startup ordering can be made from evidence instead of guesses. The data has zero cost for users who leave diagnostics off.