Skip to content

fix(cli)!: exit 2 for invalid flag and argument values [RED-1277] [ship] - #1584

Merged
sorccu merged 1 commit into
mainfrom
simo/red-1277-usage-error-exit-code
Oct 26, 2026
Merged

sorccu merged 1 commit into
mainfrom
simo/red-1277-usage-error-exit-code

Conversation

@sorccu

@sorccu sorccu commented Oct 26, 2026

Copy link
Copy Markdown
Member

Linear: RED-1277

The exit-code contract says 2 is a usage error and 1 is a failure. Many checks that reject input which is invalid on its own (a malformed or out-of-range value, or flags that cannot be combined) threw a plain Error or called exit(1), so they exited 1 like a failure. A script or agent could not tell "fix your arguments" from "something failed". These checks now exit 2.

Exit code 1 still covers well-formed input that names something remote that does not exist or does not fit the remote data:

  • an unknown --location, --private-location or --plan-id;
  • a member email that matches several members;
  • assets download needing --type for a result with several assets;
  • unknown incident service IDs.

This matches the treatment of account plan <unknown key> in RED-1274.

How it works

  • InvalidInputError (helpers/invalid-input-error.ts) carries oclif = { exit: 2 }. oclif's Command.catch and its error handler read the exit code from that field. It prints the same way as before (Error: <message> on stderr). oclif's custom flag parse path rethrows the same object, so deploy --schedule-on-deploy-threshold abc exits 2 as well.
  • Validation helpers throw InvalidInputError: number.ts, usage.ts, result-assets.ts (the source flags), api-fields.ts, flags.ts, and the --detach reporter check in test-helper.ts. So do the assets download --force --skip-existing conflict and the import plan resource specifier.
  • Catch blocks that print their own error document set process.exitCode = exitCodeFor(err), which returns 2 for invalid input and 1 otherwise. That covers alert-channels get/logs/list and account usage terms/summary/series/term/projections. Under --output json these still print the JSON error document, now with exit 2.
  • deploy's two flag-conflict checks and import's --all + --plan-id conflict call exit(2).
  • docs/output-contract.md and AGENTS.md now say which mechanism to use where.

Breaking changes (for the v10 release notes)

Invalid input now exits 2 instead of 1:

  • checkly account usage terms/summary/series/term/projections: invalid dates, --from after --to, --cursor, --limit, --pricing-rule/--group-by/--include values, an invalid term ID or --account-id, --as-of, and the removed --check-type.
  • checkly alert-channels get/logs/list: a non-numeric ID, --page, --limit, --from, or --to.
  • checkly assets list/download: both or neither of --check-id/--test-session-id. assets download also exits 2 for --force with --skip-existing.
  • checkly api -F: a malformed field or invalid JSON.
  • checkly import plan <resource>: an invalid resource specifier.
  • checkly import cancel: --all with --plan-id.
  • checkly deploy:
    • --no-plan combined with a plan-only flag;
    • a scheduling-refinement flag without --schedule-on-deploy;
    • a non-numeric --schedule-on-deploy-threshold or --schedule-on-deploy-min-frequency.
  • checkly test/pw-test/trigger --detach with a json or github reporter. This also exits 2 when that reporter comes from cli.reporters in the config, because --detach itself is on the command line.

Not covered

An invalid regex in test --grep or in a file pattern still throws a raw SyntaxError, which exits 1. It is compiled for each check during bundling, after the API calls. Fixing it needs the patterns compiled once up front.

Testing

  • Unit tests:
    • Each helper throws InvalidInputError.
    • Each command path ends with exit 2: the propagating path, the JSON-document path, the deploy and import exits, and an API failure that still exits 1.
    • 31 of these assertions fail without the fix.
  • e2e (cli-parse-error.spec.ts): api -F broken, deploy --schedule-on-deploy-threshold abc and import plan bogus:1 each exit 2.
  • The full unit suite and the full e2e suite pass locally against production.

🤖 Generated with Claude Code

Input that is invalid on its own (a malformed or out-of-range value, or flags
that cannot be combined) was often rejected with a plain Error or exit(1), so
it exited 1 like a failure. It now exits 2, the usage-error code.

Validation helpers throw InvalidInputError, which carries oclif's exit 2.
Catch blocks that print their own (JSON) error document take the code from
exitCodeFor(err). deploy's flag conflicts and import's --all/--plan-id
conflict exit 2. Well-formed input that names something remote that does not
exist still exits 1.

BREAKING CHANGE: invalid flag and argument values exit 2 instead of 1 in the
commands listed in the PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sorccu sorccu changed the title fix(cli)!: exit 2 for invalid flag and argument values [RED-1277] fix(cli)!: exit 2 for invalid flag and argument values [RED-1277] [ship] Oct 26, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved: ship/show PR from a same-repo branch.

@sorccu
sorccu merged commit 4f88d60 into main Oct 26, 2026
16 checks passed
@sorccu
sorccu deleted the simo/red-1277-usage-error-exit-code branch October 26, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant