Repository navigation
trunk-merge/pr-1265/6837f838-0dbd-4dd0-bc2b-335ca189367e - #1271
Closed
trunk-io[bot] wants to merge 20 commits into
Closed
trunk-io[bot] wants to merge 20 commits into
trunk-io[bot] wants to merge 20 commits into
Conversation
Adds tests that compare the plain-text summary line by line for upload, validate and ErrorReport, so the summary-as-data refactors (TRUNK-22401, TRUNK-22402) can show they change nothing. Each type goes through one helper (upload_summary, validate_summary, error_summary), which is the only thing those refactors should need to change. Refs TRUNK-22399 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The lock-down tests move from src/ into one integration test crate, cli/tests/summary, with a file each for upload, validate and ErrorReport. The existing ErrorReport output test moves with them and now checks every line. report_limiting becomes public so the tests can set --validation-report. The validate tests now run run_validate on fixture files instead of building ValidateRunResult, whose fields are private. Refs TRUNK-22399 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
UploadReport, in the new report module, works out everything the upload summary says once: counts, failure groups, links, validation results, the closing message (Outcome) and the exit code. The terminal output is now a text renderer over that report plus TextOptions (--validation-report and --show-failure-messages), and main exits with report.exit_code. The report derives Serialize, for the JSON summary to come. ErrorReport now classifies its error once as an ErrorKind; ErrorSummary holds what gets printed. run_upload collects deprecation warnings and BEP parse errors as notices, still only logged. The lock-down tests pass with only their helpers changed. `test` keeps exiting with the test run's exit code when the upload fails, so UploadReport::new takes the command. Refs TRUNK-22401 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
"report" already means the RSpec test report and JUnit reports in this repo. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re text in the upload summary - GroupKey sorts files, then suites, then the rest, so one ordered map replaces the three accumulators. - The summary records which command produced it. - A failure's message attribute and text are kept apart. bundle::Test still uploads the text-else-message as failure_message; the split fields are serde(skip) so the bundle payload doesn't change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ering
ValidateReport, in the report module, works out everything the validate
summary says once: the matched files, one FileValidation per file, the
CODEOWNERS check, the counts and the exit code. main exits with
report.exit_code, and the terminal output, including the EndOutput impl
for ValidateRunResult, moves into the text renderer. The report derives
Serialize, for the JSON output to come.
validate and upload's validation section now read the same
FileValidation. It gains test suite and test case counts, which are null
for a file that wasn't read as exactly one report. Its parse error is now
an error plus its causes, like ErrorSummary, because validate prints the
error the way anyhow's {:?} does. validate collects its --hide-banner
deprecation and BEP parse errors as notices, like upload.
The lock-down tests pass unchanged.
Refs TRUNK-22402
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--summary-output-file <path> (or TRUNK_SUMMARY_OUTPUT_FILE) writes the upload report as JSON to a file. The terminal output doesn't change. The file carries a schemaVersion and a type: "upload" with the same report the text is rendered from, or "error" with the ErrorSummary when the run fails outright. The flag is in UploadArgs, so `test` gets it too. The JSON goes to a temporary file next to the target, which is then renamed into place. Missing parent directories are created. On success, "Summary written to <path>" follows the summary on stderr. If the write fails, a warning goes there instead and the exit code doesn't change. Refs TRUNK-22403 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`trunk-analytics-cli summary-output-schema` prints the JSON Schema for what --summary-output-file writes. schemars generates it from the report types, so it follows the serde attributes (camelCase, the `type` tag) and takes its descriptions from their doc comments. It is generated for serialization, so the fields that are always written, even as null, are required. Refs TRUNK-22403 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When TRUNK_SUMMARY_OUTPUT_FILE is set, the RSpec gem writes the same JSON summary as `upload --summary-output-file` after uploading, or the error object when the upload fails outright. "Summary written to <path>" goes to stderr, as with the CLI, because the gem's own logs only reach the console at error level. The gem's summary has no exitCode, because RSpec decides the exit code. UploadCommand::RSpec leaves the report's exit_code unset, and the field is skipped when unset. `upload` and `test` always set it. The upload trusts the gem's quarantine results without looking them up again, so quarantineLookupFailed comes from the gem's own lookup. ErrorReport::unlogged builds the error object without logging it, since the gem already logs that error to Sentry. Refs TRUNK-22403 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The summary gets dryRun, from --dry-run (TRUNK_DRY_RUN in the RSpec gem). A dry run still checks quarantining but uploads nothing, so its summary has no upload ID or upload link and no error. dryRun says why. Refs TRUNK-22403 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`quarantining` replaces `quarantineLookupFailed` in the summary. Its
value is one of:
- CHECKED
- NOT_NEEDED
- DISABLED_VIA_CLI (--disable-quarantining)
- DISABLED_VIA_API (quarantining is turned off for the repo in Trunk)
- LOOKUP_FAILED
So a reader can tell why no failure was quarantined. Telemetry's
quarantine query result now comes from the same status. The RSpec gem
takes the status from its own lookup, because it quarantines during
the run even when TRUNK_DISABLE_QUARANTINING is set.
Every enum value in the summary is now SCREAMING_SNAKE_CASE: `type`,
`outcome`, the error and notice `kind`, and `quarantining`. Keys stay
camelCase, as in the protobuf JSON mapping. A group's key becomes
{"type": "FILE", "name": ...} or {"type": "OTHER"}, so its kind is a
value rather than a key.
Refs TRUNK-22403
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion Add `ExitCode` (cli/src/exit_code.rs) with a variant per code and a doc comment on each saying when it's used, plus `PassedThrough(code)` for the test command's code and --test-process-exit-code. Two exit codes are equal when the process exits with the same number. Every exit code decision and every field that holds one now uses it: quarantining, ErrorKind and ErrorReport, UploadReport (including cli_exit_code()), ValidateReport, ErrorSummary, the test command, main.rs and shutdown.rs. It serializes as a plain number, so the JSON doesn't change. The doc comments generate the user-facing docs: - an "Exit codes" section in --help (via clap's after_long_help), on the top-level command and each subcommand; - the ExitCode description in summary-output-schema. Remove EXIT_SUCCESS and EXIT_FAILURE from constants and drop the exitcode dependency. No behavior change: the tests only change how they build their inputs and name their expected codes; the expected numbers and text are the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
strum's EnumMessage derive exposes each variant's doc comment through get_documentation(), and EnumIter lists the variants, so ExitCode no longer needs macro_rules! to keep its doc comments as strings. The numbers move into a plain match in code(). --help and summary-output-schema print exactly the same text as before. strum 0.27.2 was already in Cargo.lock through other dependencies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the test reports have no tests, the summary only said "No tests were found in the provided test results", even when the run exits with the test step's non-zero code. A test step that failed before running any tests, such as a Playwright global setup that timed out, looked like an empty report rather than a failure. The summary now names the exit code and points at the step's own output:⚠️ No tests were found in the provided test results, but non zero exit code provided: 1 The test step appears to have failed without running tests -- see above for the original failure. With exit code 0, the single line is unchanged. The code is the one the CLI exits with, which is the test run's code wherever the outcome is printed: an error with its own exit code stops the summary before the outcome. The JSON summary already has `outcome: NO_TESTS_FOUND` and `exitCode`, so it doesn't change. Refs TRUNK-19074 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`trunk-analytics-cli github-annotations --summary-file <path>` reads the summary that --summary-output-file wrote and prints GitHub Actions workflow commands saying why the run failed. The analytics-uploader action calls it when the CLI exits non-zero, so the CLI words the failure and the action never reads the summary's format. - Exits 0 only when it printed an annotation. A summary with nothing to say, or one it can't read, exits 1 so the action falls back to its own message. - Runs before Sentry, logging and the renderer start, so stdout carries only the workflow commands. - Follows the text summary: an error that blocks quarantining is the only annotation; any other upload error is a warning; then the outcome. Each failure that wasn't quarantined gets its own annotation with its file, up to 9, since GitHub shows 10 errors per step. - The wording is shared with the text summary through constants in report/text.rs, including the "failed without running tests" line. To read the file back, the report types derive Deserialize, ExitCode reads a number back as its named code (or PassedThrough), and json::from_str rejects a schemaVersion newer than it knows. Refs TRUNK-19074 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r schemas that still parse
- Report::{schema, to_json, write_file}, and Summary::read_file plus FromStr, replace json::* free functions.
- A newer schemaVersion only fails when the summary doesn't parse; one that does is read with a warning.
- An unknown error kind from a newer CLI reads as OTHER instead of failing the whole summary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1271 +/- ##
==========================================
+ Coverage 84.43% 85.61% +1.18%
==========================================
Files 78 86 +8
Lines 18633 19235 +602
==========================================
+ Hits 15732 16468 +736
+ Misses 2901 2767 -134 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
|
trunk-io
Bot
deleted the
trunk-merge/pr-1265/6837f838-0dbd-4dd0-bc2b-335ca189367e
branch
October 29, 2026 17:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request was created and is being managed by Trunk Merge.
This pull request is based on the main branch at SHA 2da0f157db87cbd95bf9901b6dc11ed9437eb2c8.
See more details here.
When CI completes, this pull request will be closed automatically.
Pull Requests Being Tested
This pull request is testing the changes from pull request 1265, stacked on pull requests 1235, 1236, 1254, 1246, 1255, and 1253.