Skip to content

perf(ts): check std once per toolchain as a Nix sysroot - #237

Merged
kixelated merged 6 commits into
mainfrom
quest/m2/ts-std-build
Oct 30, 2026
Merged

kixelated merged 6 commits into
mainfrom
quest/m2/ts-std-build

Conversation

@kixelated

@kixelated kixelated commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Completes and deletes quest/m2/ts-std-build.md, rewritten to the plan decided here.

Problem

rust2-ts checks std from source with -Zbuild-std in every cold target directory, because translation needs std's un-inlined MIR. The prebuilt std doesn't work: its optimized MIR changes snapshots and the census, and it refuses LocalKey's methods (measured in this PR's first attempt). On every cold run, CI pays for a std check in each of the fixture suite's 3 target directories.

Approach

  • flake.nix adds rust2-ts-std, a derivation that checks std once per toolchain. It runs -Zbuild-std offline from the crates rust-src vendors, with the same profile translation uses, and keeps the lib*.rmeta files as a sysroot (72 MB). Its 23 rmeta files are byte-identical to the ones -Zbuild-std writes into a target directory, and the -C metadata hashes are the same.
  • The dev shell and the ts package set RUST2_TS_SYSROOT to it. The CLI resolves that path to an absolute one and passes it on. The wrapper gives it to rustc as --sysroot, only for crates built for wasm32; build scripts and proc macros keep the host std.
  • Outside Nix it falls back to -Zbuild-std, rather than refusing, so the translator keeps working without the flake. A RUST2_TS_SYSROOT that holds no std for wasm32 is refused, naming the path.
  • Both modes read the profile from ts/src/profile.toml, one --config per line. The CLI embeds it and the derivation passes it to cargo, so the two can't drift. The crate list core,alloc,std,panic_abort appears in both cli.rs and flake.nix, with a comment at each.
  • A new nightly step runs the fixtures with RUST2_TS_SYSROOT= (empty), so the fallback stays held to the same snapshots.

Proof

  • Fixtures: every generated file and census in the suite's scratch directories is byte-identical to main's binary. The only differences are absolute checkout paths in test harness files (tsconfig.json, caller.ts). All snapshot and differential tests pass in both modes.
  • moq: just ts census (moq-net, model, kio) is byte-identical to main. just ts moq gives identical .ts files; the source maps differ only in source paths, because my moq checkout was a symlink.

Measurements

CPU-s is user plus sys time. These were measured on a 32-core host with a load average around 30 from other jobs, so absolute numbers are inflated, but each pair below ran back to back.

-Zbuild-std sysroot
fixture (codec), cold, median of 3 74 CPU-s, 37 s wall 16 CPU-s, 7 s wall
fixture (codec), warm 1.1 to 1.7 CPU-s 0.7 to 0.9 CPU-s
fixture suite, cold (2 runs) 799 / 782 CPU-s 540 / 447 CPU-s
fixture suite, warm (2 runs) 399 / 374 CPU-s 412 / 403 CPU-s
derivation, once per toolchain 58 CPU-s, 46 s wall

CI starts every cargo target directory cold, so a Check run saves the cold difference, about 250 to 350 CPU-s on this host. The derivation is rebuilt only when the nightly or profile.toml changes, on runners that keep /nix/store. The nightly fallback step adds one fixture-suite run per night.

Impact

  • New environment variable RUST2_TS_SYSROOT, which the ts package and the dev shell set. Unset, or set to empty, the translator checks std with -Zbuild-std as before. No other API change, so no MIGRATE.md entry.
  • Every dev shell now depends on rust2-ts-std. The first nix develop after a nightly bump builds it, about 1 CPU-minute.

Alternatives

  • Translate against the prebuilt std: rejected, because its optimized MIR breaks lang and changes snapshots and the census.
  • Pass the sysroot through target.<triple>.rustflags: that would put it in cargo's fingerprint, but a RUSTFLAGS environment variable silently overrides it. The wrapper flag can't be overridden that way.
  • Refuse outside Nix: simpler, but the translator would no longer work without the flake.

Follow-ups

  • In one target directory, changing only the core/alloc/std lines of profile.toml, with the same nightly, leaves cached dependencies built against the old std. rustc then fails loudly with E0460 until the directory is cleaned. Any toolchain bump or change to the shared flags rebuilds everything already.

Review

Codex gpt-6.1-sol, medium effort, quest review round 2 on head 4232079: no findings.

  • Round 1 (head a1ac238): a relative RUST2_TS_SYSROOT was resolved from the package directory cargo runs the wrapper in. Fixed: the CLI now resolves it to an absolute path and passes that to cargo.

Checks: just check and just test passed on a1ac238. just ts check and just ts test passed on 4232079, which changes only ts/src/cli.rs.

Merge of main at d77d3835c517f93e11bc4888db60b2b8cf4abb86: the only conflict was this PR deleting its finished quest file while #249 updated that file's links; resolved by deleting it. Nothing else changed since the reviewed head 42320797, so this merge is a recorded skip, too trivial to review.

(written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 5 commits October 29, 2026 14:32
Quest-Claim: 87e6d34c667dfd46
The flake checks std for wasm32 with -Zbuild-std once per toolchain, with the
profile translation reads its MIR at, and the dev shell and ts package pass it
as RUST2_TS_SYSROOT, which the wrapper gives the target's crates as --sysroot.
Without it, each target directory checks std with -Zbuild-std as before, from
the same profile, now in ts/src/profile.toml. A nightly step runs the fixtures
that way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…apper elsewhere

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Translate against the prebuilt std perf(ts): check std once per toolchain as a Nix sysroot Oct 30, 2026
@kixelated
kixelated marked this pull request as ready for review October 30, 2026 02:40
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-30T02:42:47.162788Z 4232079 Draft marked ready
🔒 Security Review 🔄 Running since 2026-10-30T02:41:07.155833Z 4232079 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 4232079

No actionable new bugs found in the reviewed paths. A checked-std sysroot preserves the required MIR profile rather than relying on optimized prebuilt std; sharing profile.toml and retaining the source-build fallback are proportionate choices. Resolving the sysroot before Cargo changes directories fixes the relative-path issue, and target-only injection keeps host build scripts and proc macros on their normal sysroot.

The acknowledged warm-cache limitation remains: changing only std-specific profile settings does not invalidate dependent crates through Cargo's fingerprint, so a clean target directory can still be required after E0460. Coverage for relative paths, invalid sysroots and switching modes in a warm target directory would help protect these boundaries.

Verification: independent static inspection of the Nix derivation, CLI/wrapper, shared profile and fallback CI step. No Nix builds, fixture tests, census comparisons or performance measurements run; the reported equivalence and timings were not independently reproduced.

(Written by OpenAI)

# Conflicts:
#	quest/m2/ts-std-build.md
@coderabbitai

coderabbitai Bot commented Oct 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0b37662f-289e-47d5-9868-aa146a908b7a
Commits

Reviewing files that changed from the base of the PR and between e51ecad and d77d383.

Files selected for processing: 9
  • .github/workflows/nightly.yml
  • flake.nix
  • quest/m2/README.md
  • quest/m2/ts-std-build.md
  • ts/README.md
  • ts/src/cli.rs
  • ts/src/config.rs
  • ts/src/driver.rs
  • ts/src/profile.toml
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d77d3835c5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ts/src/driver.rs
if args.windows(2).any(|w| w[0] == "--target" && w[1] == cli::TARGET)
&& let Some(sysroot) = cli::sysroot()?
{
args.extend(["--sysroot".into(), sysroot]);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include the selected sysroot in Cargo's cache identity

When the sysroot changes without a toolchain or shared-rustflags change, Cargo can reuse dependencies checked against the previous std because this flag is added inside the wrapper, after Cargo computes its fingerprints. For example, changing only profile.dev.package.core.rustflags in profile.toml rebuilds the Nix sysroot but leaves cached target dependencies intact. Check::run cleans only the configured package, so its subsequent compilation fails with E0460 against those dependencies until the target directory is manually cleaned. Include the resolved sysroot in Cargo-visible fingerprint inputs or isolate target artifacts by sysroot identity so updating the dev shell does not break an existing translation directory.

Useful? React with 👍 / 👎.

@kixelated
kixelated merged commit 59c5e06 into main Oct 30, 2026
5 checks passed
@kixelated
kixelated deleted the quest/m2/ts-std-build branch October 30, 2026 05:28
kixelated added a commit that referenced this pull request Oct 30, 2026
* quest: plan follow-ups of #237 and #241

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* quest: fold #247's follow-ups into ts-dyn and ts-drop-order

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Luke Curley <432854+kixelated@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant