Skip to content

Typecheck Checkly CLI snippets against checkly@10 - #737

Open
stefanjudis wants to merge 25 commits into
mainfrom
snippet-typecheck
Open

stefanjudis wants to merge 25 commits into
mainfrom
snippet-typecheck

Conversation

@stefanjudis

@stefanjudis stefanjudis commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Typechecks the docs' Checkly CLI code snippets against checkly@10.0.0 on every PR, so examples can't drift from the constructs. It also fixes the snippets that didn't typecheck.

Affected Components

  • Content & Marketing
  • Pricing
  • Test
  • Docs
  • Learn
  • Other (CI)

What changed

Checker (.github/snippet-typecheck/)

  • Extracts TS/JS code blocks from all .mdx files (.mintignore dirs, samples/ and node_modules excluded) and typechecks them in memory with the TypeScript compiler API against the latest CLI. Runs in about a second.
  • The folder is its own npm package with checkly, @checkly/playwright-helpers, @playwright/test and @types/node on latest and no lockfile (.npmrc: package-lock=false). Every run checks against the newest CLI release, independently of the docs project's own checkly version. TypeScript is pinned to ^6: 7 (the Go-native compiler) has no JavaScript compiler API.
  • Errors point at the MDX file and line, as inline PR annotations.
  • npm run check:snippets runs the check, npm run test:snippets runs its 29 node:test tests. Both install the checker's dependencies first, so they use the latest CLI.

What gets checked (579 snippets)

  • Complete snippets, meaning ones that import or require from checkly, checkly/constructs or checkly/util.
  • Fragments that use a checkly export (new BrowserCheck(, Frequency.EVERY_5M, UrlAssertionBuilder.…), with no marker needed. Exports that are also globals, like Request, don't count.
  • Fragments marked with an MDX comment on the line before the fence:
    • {/* typecheck-wrap: ApiCheck */}: property fragments like request: {...} are checked inside new ApiCheck('snippet', {...}). Fragments that already call the constructor only get imports added.
    • {/* typecheck-wrap */}: statements that don't use a construct directly, like export default defineConfig(...). These only get imports added.
    • Fragments get every checkly/constructs and checkly/util export, defineConfig and path auto-imported. Readers don't see any of it.
  • {/* typecheck-skip */} opts a block out. It's only used on the 6 AgenticCheck blocks, which document CLI v9 until the December 1, 2026 shutdown.

Rules that keep it from failing on intentional shortcuts

  • /* More options... */: the docs already use this 84 times to mark shortened snippets. In those, missing required props are allowed; wrong or unknown props still fail.

  • Relative imports in multi-file examples (./group.check) resolve to any.

  • Undeclared lowercase names in fragments (statusPage, group) count as page context. Unknown PascalCase names (a removed or misspelled construct) still fail.

  • // Equivalent to: in assertion examples: the comment and the raw object after it are dropped, and the builder call above it is checked.

Left unchecked on purpose: nested option fragments (sslConfig: {...}, reminders: {...}), Playwright configs, and non-Checkly code.

CI and dependencies

  • New blocking step in static-docs-checks.yml. It installs the checker's dependencies fresh and runs on docs changes, and also when anything in .github/snippet-typecheck/ changes. It doesn't need the root npm ci.
  • Trade-off of latest: a CLI release that changes the types fails the snippet step on every open docs PR until the snippets are fixed on main.
  • The root checkly (used for this repo's own monitoring) goes from 8.2.0 to 10.0.0. Apart from that, the root package.json only gains the two snippet scripts. v10 requires Node ≥ 22.13, so deploy-checks, preview-checks and static-docs-checks move from Node 20 to 22.
  • .playwright-cli/ and .playwright-mcp/ are now gitignored.

Doc fixes the check found

  • Missing construct imports: browser-check, check-group(-v2), multistep-check, playwright-check, tcp-monitor.
  • Missing required projectName/logicalId in shortened checkly.config.ts examples: Playwright Check Suite pages, including-checks, checkly-runtimes. Highlight ranges are shifted to match.
  • Unclosed ApiCheck block, which also lacked a request: cli/checkly-runtimes.mdx.
  • ApiCheck inline setup example with no request method: constructs/api-check.mdx.
  • Invalid bare ... placeholders: cli/importing.mdx.
  • page.getByLabel('email', { name: /email/i }): getByLabel has no name option, so it was ignored. Now getByLabel(/email/i) (and password) in constructs/browser-check.mdx.
  • Trailing commas after three assertion examples in api-check.mdx and tcp-monitor.mdx.
  • /* More options... */ added to usage examples that omit required props: url, dns, icmp, tcp, heartbeat, multistep, overview.

Notes for the Reviewer

  • How I verified: the snippet check reports 0 errors. Snippet tests, API-nav tests, frontmatter, redirect destinations and broken links all pass locally. I rendered all 48 changed pages, plus one page that includes a changed snippet, with mint dev: no marker text is visible, and every CodeGroup has the same tab count as its source.
  • One highlight change isn't caused by inserted lines: in custom-dependencies.mdx the range was already off by two. It now points at the include line.

New Dependency Submission

  • Root: checkly 8.2.0 → 10.0.0 (devDependency), for this repo's own monitoring.
  • .github/snippet-typecheck/ (separate package, not installed with the root project): checkly, @checkly/playwright-helpers (seven pages import it), @playwright/test, @types/node on latest, and typescript ^6.

Screenshots

No visual changes. The markers are MDX comments and don't render.

checkly@10 requires Node >=22.13.
Extracts code blocks that import from checkly, typechecks them with the
TypeScript compiler API against the installed CLI, and reports errors at
the MDX line. Opt out per block with {/* typecheck-skip */}. Runs
report-only in static docs checks for now.
Add required projectName/logicalId to abbreviated config examples, close
the unterminated ApiCheck example and give it a request, and shift
highlight ranges accordingly.
Snippets marked with /* More options... */ may omit required props; wrong
or unknown props still fail the typecheck.
AgenticCheck was removed in checkly@10; these pages document v9 until
the December 1, 2026 shutdown.
{/* typecheck-wrap: ApiCheck */} before a fence auto-imports the checkly
exports and wraps property fragments in the named construct.
Mark fragments with typecheck-wrap, auto-import path for entrypoints, and
add the missing request method to the inline setup script example.
Mark fragments with typecheck-wrap so they're checked against checkly@10.
Add /* More options... */ to usage examples that omit required props,
and projectName/logicalId to abbreviated checkly.config.ts examples.
@mintlify

mintlify Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
checkly-422f444a 🟢 Ready View Preview Oct 28, 2026, 3:18 PM

# Conflicts:
#	package-lock.json
#	package.json

This branch was successfully deployed

1 active deployment
staging — 0f45a6fe Deployed Oct 28, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant