Repository navigation
feat(platform-ios): ship prebuilt XCTest agent artifacts and sign them locally on devices - #209
Merged
Merged
Conversation
…seam Artifact preparation was inlined in the agent controller: `prepare()` knew only how to run `xcodebuild build-for-testing`, and the launch input it produced (project plus scheme, versus a supplied `.xctestrun`) leaked into `ensureStarted()`. Publishing prebuilt artifacts needs a path that runs *ahead* of the build, so split the concerns first: - `xctest-agent-artifacts.ts` defines the contract. A provider either returns a launch input, declines so the next one is tried, or throws to fail the run. `resolveXCTestAgentArtifacts` walks providers in order. - `xctest-agent-build.ts` holds the build logic, moved verbatim, plus the fallback provider that wraps it. - `xctest-agent.ts` keeps the controller and owns process launch and readiness only. It now takes a provider list, defaulting to external-artifact then build, which reproduces the previous precedence. The build-inputs hash moved with the code that computes it and is exported, so tests read the real fingerprint instead of reimplementing it. Behaviour is unchanged.
…kage Building the agent with `xcodebuild build-for-testing` costs 10-22 s and needs a working Xcode on every machine that runs Harness. The built agent carries no device, bundle-identity, or checkout path, so one unsigned build can serve every consumer on a matching toolchain. `scripts/pack-xctest-agent-artifacts.mjs` runs as `prepack`, which npm and pnpm trigger during `nx release publish`, so artifacts reach the tarball without touching the release commands. It copies the `build-for-testing` product tree minus dSYMs, Swift modules, and symbol maps (2.0 MB instead of 3.9 MB) into `xctest-agent-prebuilds/`, alongside an index recording the agent sources hash, package version, Xcode, and SDK it was built with. Packing is idempotent and degrades to a warning when Xcode is unavailable, so a release from a machine that cannot build for iOS still publishes; `HARNESS_REQUIRE_XCTEST_PREBUILDS=1` makes such failures fatal, which the iOS CI jobs set so the shipped path is exercised rather than only the fallback. `createPrebuiltXCTestAgentProvider` is now consulted before the build provider. It matches the index against the local toolchain and copies a matching artifact into `.harness/cache` — the copy is what `test-without-building` writes into, keeping the installed package pristine. It declines, leaving the build path to run, when Xcode, SDK, agent sources, or package version differ, when the artifact is incomplete, or when the index is unreadable. Two hashes are needed because the old one mixed the agent sources with the build module's own source: that digest differs between `src` in this repository and `dist` in an installed package, so a packed artifact could never be recognised by a consumer. `getAgentProjectSourcesHash` covers only files shipped verbatim; local build reuse keeps the stricter hash. Verified against a real Xcode 27.0 build: packed, resolved from a separate project root, and launched to a healthy agent on an iOS 26.4 simulator, which also confirms the artifact is not tied to the iOS runtime version.
…evices A device build cannot ship ready to run: a runnable bundle carries the consumer's certificate and a profile listing their device. Building one therefore stayed the only device path, costing an Xcode build per machine and, under automatic signing, a reachable Developer Portal. `createResignedDeviceXCTestAgentProvider` takes the unsigned device artifact shipped with the package, copies it into `.harness/cache`, and signs that copy: the nested XCTest bundle with no entitlements, then the runner and the app under test with entitlements matching what `xcodebuild` emits for the same team and bundle id, confirmed by diffing against a real signed build. The order matters, because the XCTest bundle lives inside the runner. Signing takes about a second and needs no network. Selecting what to sign with is most of the work. `codesign --sign "Apple Development"` fails as ambiguous once a keychain holds two development certificates, which is routine for a personal plus a work account, so the identity is resolved to a SHA-1 by intersecting the certificates embedded in a profile with the usable keychain identities — precise, and unable to name another account's certificate. Profiles are decoded key by key because `plutil -convert json` refuses their binary certificate blobs, and one is used only when it covers the agent's bundle ids and the target device, has not expired, and matches an installed certificate. Otherwise the provider declines and the build path runs, since that path can reach the portal. Bundle ids are rewritten when `device.codeSign.bundleIdPrefix` is set, for accounts that cannot mint a profile covering `com.callstackincubator.*`. The three Info.plists alone are not enough: a `.xctestrun` names the runner bundle id per target and lists the ids whose crash reports belong to the agent, and a stale runner id makes the session look for an app that was never installed. Those keys sit under `TestConfigurations`/`TestTargets`, not at the top level, so they are resolved per target. An explicit prefix decides the ids even when a profile would also cover the shipped ones, so bundle identity does not shift with whichever profiles happen to be installed. Both destinations now ship (1.0 MB to 1.5 MB compressed), which keeps the device path reachable in practice; consumers who never touch hardware pay one larger download and nothing at runtime, because the provider declines before doing any work. Re-signing records a marker only once verification passes and removes a partially signed copy on failure, so a cache is never reused half-done, and the packaged artifact is never signed.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
… work across Xcode versions Pack into a fresh derived data folder with coverage, indexing, previews and debug info disabled, and refuse to pack artifacts that carry a path from the packing machine, an embedded profile, or a non-Apple signature. Use a shipped artifact from Xcode 26 up to the major version that packed it, and build locally on any other Xcode. Re-sign the XCTest frameworks Xcode 26 embeds in the runner, unpack and re-sign atomically, and read the toolchain from Xcode's own files instead of invoking xcodebuild.
V3RON
marked this pull request as ready for review
October 18, 2026 15:14
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this?
iOS runs no longer compile the permission helper from scratch. Harness now uses the XCTest agent that ships prebuilt with the Apple platform package, so a run reaches a ready agent in seconds instead of waiting on an Xcode build, and physical devices sign that shipped artifact locally rather than building one per machine. Building from source remains the fallback and runs whenever the shipped artifact cannot be used.
Building the agent with
xcodebuild build-for-testingcost 10-22 s on every machine and, on a device, required a working signing setup with a reachable Developer Portal. Neither cost was necessary: an unsigned build carries no device, account, or checkout path, so one build can serve every consumer. This is the distribution model Appium already ships for WebDriverAgent.How does it work?
Harness obtains agent artifacts from the first provider that applies: an
.xctestrunyou point it at, the artifact shipped with the package, or a localbuild-for-testing. Each provider returns artifacts, declines so the next one runs, or throws to fail the run.Packing runs as
prepack, which npm and pnpm trigger duringnx release publish, so artifacts reach the tarball without touching the release commands. It builds one simulator and one device artifact with the selected Xcode into a fresh folder, never reusing a local build that may carry the packer's signature. Builds disable code coverage, indexing, previews, the debug dylib, and debug info, which cuts a clean build by about a third and keeps the products free of the build machine's paths. Packing then refuses any artifact that still embeds a path from the machine, a provisioning profile, or a signature from a team other than Apple's, so a release packed on a maintainer's machine cannot publish their identity. The tarball grows by about 1 MB.A shipped artifact is used from Xcode 26 up to the major Xcode version that packed it: artifacts packed with Xcode 26.4.1 and 27.0 were each verified to run under the other on simulators and on a physical device. Any other Xcode, older or newer, builds locally, so a new Xcode costs one local build until a release ships artifacts packed with it. The toolchain is read from Xcode's own
version.plistandSDKSettings.jsoninstead of twoxcodebuildinvocations, saving about a second per run.Device artifacts ship unsigned, and Harness signs a private copy in
.harness/cacheoffline, inside out: the XCTest frameworks Xcode 26 embeds in the runner, the test bundle, the runner, then the app under test. The copy is staged, signed, and only then renamed into place, so an interrupted run or two parallel runners never leave a half-signed copy. The certificate is chosen by intersecting the certificates embedded in your profile with usable keychain identities, and a profile is chosen only when it covers the agent's bundle ids and your device and has not expired.device.codeSign.bundleIdPrefixrewrites the agent's bundle ids for accounts that cannot mint a profile coveringcom.callstackincubator.*. SetHARNESS_IOS_XCTEST_PREBUILT_ROOTto point at artifacts packed elsewhere, orHARNESS_SKIP_XCTEST_PREBUILDS=1to always build locally.Why is this useful?
Every iOS run on a supported Xcode starts roughly ten to twenty seconds sooner and no longer needs to compile the permission helper. CI can drop the cached agent build, and the iOS jobs pack artifacts with failures made fatal so the shipped path is exercised rather than only the fallback. On physical devices the signing step stops requiring Developer Portal access at run time, which makes device runs usable in restricted and offline environments. When anything about the Xcode, the profile, or the certificate is unsuitable, Harness says so and builds the way it always has, so the fast path can only save time, never change results.