Skip to content

test(expo): add a verify skill that drives the expo-native fixture - #10053

Draft
mikepitre wants to merge 6 commits into
mike/expo-verify-hostfrom
mike/expo-verify-skill
Draft

mikepitre wants to merge 6 commits into
mike/expo-verify-hostfrom
mike/expo-verify-skill

Conversation

@mikepitre

@mikepitre mikepitre commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds verify-clerk-expo, a skill and CLI that an agent uses to prove a @clerk/expo change on an iOS simulator or Android emulator, with video, screenshots, and app state as evidence. It drives the expo-native fixture through the launch inputs from #10052, against a Clerk application that it creates for the worktree and deletes afterwards. The skill is at .claude/skills/verify-clerk-expo/, the CLI is bin/control-clerk-expo inside it, and .cursor/skills/verify-clerk-expo is a symlink so Cursor reads the same files. packages/expo/AGENTS.md has the short version for agents, and the root AGENTS.md points to it.

About 2,700 of the 16,500 added lines are written for this repo. The commits are in reading order:

  1. The skill's package.json, tsconfig.json, the .prettierignore entries, the Cursor symlink, and the changeset.
  2. package-lock.json alone (2,528 lines).
  3. The copies, 61 files and 11,194 lines: src/core/, src/platform/, specs/fixtures.ts, testing/, and 21 test files. They are byte copies of the verification skills in clerk/clerk-ios (feat(verify): add the verify-clerk-ios verification skill clerk-ios#624) and clerk/clerk-android (feat(verify): add the clerk-android verify skill clerk-android#1042), where they are reviewed. src/core/MANIFEST lists the hashes of the core files, and a unit test and control-clerk-expo doctor fail when one drifts.
  4. This repo's own code: src/host.ts, src/fixture.ts, and src/freshness.ts (the fixture build, Metro and the watch build, and the check that the app loads current JS), their tests, specs/native.ts, and specs/golden/ (ten spec files for six features).
  5. SKILL.md, features/, references/, and the pointers for agents.
  6. A CI job.

What the CLI does:

  • up --platform ios|android builds the fixture as a Debug dev client, creates the worktree's Clerk application, takes a device the skill owns (a cloned simulator or a read-only emulator), installs the app, and starts tsdown --watch in packages/expo and expo start on a port fixed per device.
  • run <feature> runs that feature's specs and writes video.mp4, screenshots, and every verify.state the app reported under .claude/skills/verify-clerk-expo/.verify/runs/<run-id>/. The features are native-auth-view, user-button-and-profile, custom-flow-sign-in, custom-flow-sign-up, token-cache-persistence, and native-js-sync.
  • down releases the devices, stops Metro and the watch build, and deletes the application with every user in it. It keeps the evidence.
  • doctor only reads. It creates no file, device, or application.
  • A change under packages/expo/src reaches the app on the next run with no native build. Before the specs start, run waits until the watch build has caught up and Metro serves a bundle built from the current dist, and fails with NOT_READY if it never does. A change to a native input (the list is nativeInputs in src/fixture.ts) rebuilds the dev client.
  • @clerk/expo, @clerk/expo-biometrics, and @clerk/expo-google-signin are built into the dev client. For @clerk/expo's other workspace dependencies, such as @clerk/clerk-js and @clerk/shared, up and run refuse to launch when a dist was built from different source, and print the pnpm turbo build command to run.

Things a reviewer may ask:

  • The skill uses no shared test instance. Each worktree gets one application in a workspace that holds nothing else (org_3KHungJxbvIscuSvy8oos5MHAli), configured from src/core/instances/base.json. The credential is a Platform API key from the environment or from a 1Password reference kept on the machine. A machine with none fails doctor with one fix line.
  • CI does not run these specs. The tests in integration/tests/expo-native (ci(e2e): Replace maestro with e2e in the expo native integration tests #10032) stay the regression gate, on a Release build. This skill is the development loop: a Debug dev client, workspace source, and evidence to attach to a PR. Both use the same fixture and the same e2e runner.
  • The new Verify Skill Tests job in ci.yml runs the skill's unit tests and typecheck on a Linux runner, with no device and no secret, when a PR changes the skill or a path its tests read.
  • The skill installs e2e 0.15.2 and @e2e-dev/mobile 0.9.0 with npm ci from its own lockfile. It is outside the pnpm workspace, and the copied files are tested against those versions.
  • The skill runs on a Mac. On any other machine doctor, up, and run fail with UNSUPPORTED. The PR above this one adds a device on a CI runner for those machines.
  • The auth specs use +clerk_test users. Four specs type a password or the test code 424242 and are tagged form-entry: custom-flow-sign-in/complete, custom-flow-sign-up/request-code, custom-flow-sign-up/complete, and native-auth-view/complete. None has a passing run yet. custom-flow-sign-up/request-code ran once and failed on the password field, which iOS covered with its strong-password sheet. test(expo): drive the expo-native fixture from verify launch inputs #10052 now keeps that sheet away, and a run that only focuses the field shows the keyboard, but no run has typed a password since. Each sign-in flow was run up to its code screen with --skip form-entry. Sign-up has no spec that runs without typing.
  • One spec is tagged known-bug and skipped unless --include known-bug is passed: the close button of an inline AuthView does not fire onDismiss.
  • Android specs find native views by text, because the pinned clerk-android release has no test tags. iOS specs use the clerk.* accessibility identifiers.
  • A worktree can hold an iOS and an Android device at once, and they share one application and one watch build. down --platform <p> releases that platform's device and stops its Metro, and keeps what the other platform still uses. Their runs should not overlap, for two reasons. A run that finds an edited sibling package stops every Metro of the worktree while it rebuilds. And with both platforms running specs on the shared application at once, a ticket sign-in failed with resource_not_found in two of four tries, once on each platform. That did not happen in any run with one platform at a time, and the cause is not known. SKILL.md says to run them one after the other.
  • .prettierignore skips the copied .ts files so the pre-commit hook does not reformat them.

Checklist

  • pnpm test runs as expected.
  • pnpm build runs as expected.
  • (If applicable) JSDoc comments have been added or updated for any package exports
  • (If applicable) Documentation has been updated

Type of change

  • 🐛 Bug fix
  • 🌟 New feature
  • 🔨 Breaking change
  • 📖 Refactoring / dependency upgrade / documentation
  • other: test tooling

🤖 Generated with Claude Code

@changeset-bot

changeset-bot Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 45a50d0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
clerk-js-sandbox Ready Ready Preview Oct 6, 2026 6:43am UTC
swingset Ready Ready Preview Oct 6, 2026 6:43am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the expo label Oct 3, 2026
@mikepitre
mikepitre force-pushed the mike/expo-verify-skill branch from 65e54c1 to fe47a59 Compare October 3, 2026 08:27
@mikepitre
mikepitre force-pushed the mike/expo-verify-host branch from 4901448 to 2add795 Compare October 5, 2026 23:26
@mikepitre
mikepitre force-pushed the mike/expo-verify-skill branch from fbef48a to 1d0dda9 Compare October 6, 2026 00:36
@mikepitre
mikepitre force-pushed the mike/expo-verify-skill branch from 2328a39 to 482deff Compare October 6, 2026 03:04
@mikepitre
mikepitre force-pushed the mike/expo-verify-skill branch from 482deff to 435e872 Compare October 6, 2026 06:03
mikepitre and others added 6 commits October 6, 2026 02:41
The skill is outside the pnpm workspace and installs its pinned e2e with npm from its own lockfile.
.cursor/skills/verify-clerk-expo is a symlink to the skill, and .prettierignore keeps the pre-commit hook
off the files that are copied from other repositories.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
src/core, src/platform/ios, specs/fixtures.ts, testing, e2e.config.ts, and these tests are byte copies of
clerk/clerk-ios. src/platform/android and test/android.test.ts are byte copies of clerk/clerk-android.
src/core/MANIFEST pins the core.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… check, and the golden specs

src/host.ts builds the expo-native fixture as a Debug dev client, starts the watch build and one Metro per
lane, and waits until Metro serves current JS before specs start. specs/golden holds ten spec files for
six features.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…them

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

2 active deployments
Preview – swingset — 45a50d01 Deployed Oct 6, 2026 by vercel[bot]
Preview – clerk-js-sandbox — 45a50d01 Deployed Oct 6, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant