Skip to content

ci: serve the Nix cache from nix.moq.dev instead of Cachix - #5693

Merged
kixelated merged 7 commits into
mainfrom
quest/m0/ci/nix-cache
Oct 20, 2026
Merged

kixelated merged 7 commits into
mainfrom
quest/m0/ci/nix-cache

Conversation

@kixelated

@kixelated kixelated commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Replaces kixelated.cachix.org with nix.moq.dev, an R2-backed binary cache behind a Worker. Completes quest/m0/ci/nix-cache.md.

Changes

  • infra/nix: a Worker for nix.moq.dev that serves two R2 buckets as one cache.
    • nix-moq-dev holds released packages and never expires.
    • nix-ci-moq-dev holds CI dev shells. lifecycle.json deletes its objects after 30 days.
    • Releases are checked first. The Worker serves nix-cache-info itself, with Priority: 41, so Nix asks here only after cache.nixos.org.
    • A .narinfo gets a 5-minute Cache-Control and nar/ gets immutable.
    • Recipes: just infra nix create | lifecycle | deploy | keygen | push. keygen stores the signing key in cache-write without writing it to disk, then prints the public key.
  • .github/actions/nix: a single composite action replaces every direct nix-installer-action call, including the Nightly jobs added on main since. It sets accept-flake-config = true, so flake.nix is the only file that names the cache and its key. The CI checkout is already trusted code, so this adds no trust. Nightly's tooling test now requires every dev-shell job to use it.
  • flake.nix: nixConfig now names https://nix.moq.dev, trusts nix.moq.dev-1:BZ0DtCypDFv1ESX3hxuB42yc0vitSA1YHojSmVwB5bk=, and sets fallback = true. Expiry can delete a dependency before the path that needs it, and fallback builds the missing piece instead of failing.
  • cache.yml: after ci: cache Rust builds in R2, written from the merge queue #5685 it warms only the dev shell, and that push now goes to nix-ci-moq-dev from the cache-write Environment. Each warm uploads only missing paths, which also restores anything that expired.
  • cachix.yml -> release-nix.yml ("Release Nix"): pushes the tagged package to nix-moq-dev from the cache-write Environment, which also admits the five release tag patterns. cachix pin is no longer needed because that bucket never expires. alert.yml follows the rename.
  • infra/README.md: adds the bootstrap beside the mbx one. The doc, overlay, and scrub comments now point at the new cache.
  • Quest: deletes nix-cache.md and adds nix-cache-verify.md, a condition quest that waits on this merge, then verifies the cache and retires Cachix. Links from audit.md, mbx-r2-verify.md and the epic README now point at it.

Maintainer actions

Done 2026-10-19 and 2026-10-20 (README "Bootstrapping the Nix cache"):

  • Tag ruleset 25452857 limits moq-{relay,cli,boy,c,gst}-v* to moq-bot and RepositoryRole 2.
  • Buckets nix-moq-dev and nix-ci-moq-dev with the CI lifecycle rule; the Worker serves nix.moq.dev.
  • NIX_SIGNING_KEY in cache-write; its public half is in flake.nix.
  • The five tag patterns are in cache-write's deployment policy, and the unused nix-release Environment is gone.
  • One R2 token, Object Read & Write on both buckets, as NIX_R2_ACCESS_KEY_ID / NIX_R2_SECRET_ACCESS_KEY in cache-write.
  • The Worker is redeployed (just infra nix deploy) without the stray nix_moq_dev binding that wrangler r2 bucket create had added to wrangler.jsonc. create now passes --update-config=false.

Remaining after merging: delete CACHIX_AUTH_TOKEN (tracked in the verify quest).

cache-write also admits the merge queue since #5685, so merge-queue jobs can reach the signing key too. The maintainer accepted that: enqueuing a PR is treated as landing it.

Public API and wire impact

No API or wire changes; this touches CI and infra only. Downstream users who run --accept-flake-config on a new tag will now trust nix.moq.dev instead of Cachix.

Checks

Decisions (confirmed by the maintainer 2026-10-19)

  1. Two buckets instead of one. One bucket with age-based expiry would also expire released packages, which cachix pin used to protect. Nix's S3 store can't write under a prefix, so a lifecycle rule can't tell the two apart.
  2. Stop pushing to Cachix now. Old tags keep their Cachix pins. A downstream flake that hard-codes Cachix will build new releases from source until it adds nix.moq.dev.
  3. accept-flake-config instead of naming the cache in each workflow. One source of truth in flake.nix.
  4. Push the full closure. A Nix binary cache refuses a path whose references are missing, so paths that cache.nixos.org already serves can't be filtered out. Storage and the first upload are larger as a result; revisit with niks3 or attic only if size matters.
  5. One signing key, one Environment. Anything reaching the key can sign any path, so a separate release Environment only guarded the bucket token; nix-release folded into cache-write. Two buckets stay so CI paths expire and released ones don't.
  6. 30-day CI expiry, tuned in the verify quest.

Suggested follow-ups

  • nix-cache-verify (added here).
  • Dependabot scans only /, so composite-action pins under .github/actions/* drift: setup-packaging was already on an older nix-installer-action. Add directories.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 2 commits October 15, 2026 22:58
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
R2 behind a Worker replaces kixelated.cachix.org: a release bucket that
never expires and a CI bucket that a lifecycle rule expires after 30 days.
cache.yml pushes the dev shell from main and release-nix.yml (formerly
cachix.yml) pushes tagged packages, each through a GitHub Environment that
holds the signing key. Workflows install Nix through one local action that
accepts the flake's nixConfig, so flake.nix is the only place naming the
cache. The public key is a placeholder until the maintainer generates it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Quest outcome: implementation done, left as a draft. It is blocked on the maintainer bootstrap in infra/README.md and on the real public key replacing the placeholder in flake.nix. Six open decisions are listed in the description, each with the option I went with.

(Written by Claude Opus 5.5)

kixelated and others added 4 commits October 16, 2026 13:02
# Conflicts:
#	.github/workflows/check.yml
#	.github/workflows/ts.yml
#	.github/workflows/uring.yml
flake.nix now trusts the generated nix.moq.dev-1 key instead of the
all-zero placeholder.

With one signing key, a second Environment for releases only guarded the
bucket token: anything reaching the key can sign any path. So release-nix
deploys to cache-write, which also admits the release tags, and one R2
token covers both buckets. keygen writes only there, and bucket creation
no longer offers to add bindings to wrangler.jsonc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep main's queue-only Check and Test, its alert list without Cache and
Check, and its removal of the drill-sensitivity job. Route the two new
Nightly jobs through the nix composite, and point audit.md and the epic
README at nix-cache-verify.md, carrying main's Nightly baseline into it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Take #5685's dev-shell-only cache.yml and push it to nix.moq.dev from
cache-write. List the mbx and nix infra side by side, and point the
Nightly tooling test and the remaining quest links at the nix composite
and nix-cache-verify.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 20, 2026 15:53
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 19 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ca55ffab-03e3-470e-aacd-89003f5879ab
📥 Commits

Reviewing files that changed from the base of the PR and between 34a01e7 and 5c2096b.

⛔ Files ignored due to path filters: 1
  • infra/nix/bun.lock is excluded by !**/*.lock
📒 Files selected for processing: 41
  • .github/actions/nix/action.yml
  • .github/actions/setup-packaging/action.yml
  • .github/workflows/alert.yml
  • .github/workflows/audio-quality.yml
  • .github/workflows/cache.yml
  • .github/workflows/check.yml
  • .github/workflows/interop.yml
  • .github/workflows/moq-c.yml
  • .github/workflows/moq-gst.yml
  • .github/workflows/moq-sh.yml
  • .github/workflows/nightly.yml
  • .github/workflows/obs.yml
  • .github/workflows/quest.yml
  • .github/workflows/release-binary.yml
  • .github/workflows/release-js.yml
  • .github/workflows/release-nix.yml
  • .github/workflows/release-py.yml
  • .github/workflows/release-rs.yml
  • .github/workflows/ts.yml
  • .github/workflows/uring.yml
  • .github/workflows/video.yml
  • .github/workflows/wasm.yml
  • .github/workflows/x11.yml
  • doc/setup/install.md
  • flake.nix
  • infra/README.md
  • infra/justfile
  • infra/nix/justfile
  • infra/nix/lifecycle.json
  • infra/nix/package.json
  • infra/nix/src/worker.ts
  • infra/nix/wrangler.jsonc
  • nix/overlay.nix
  • quest/m0/audit-perf.md
  • quest/m0/ci/README.md
  • quest/m0/ci/audit.md
  • quest/m0/ci/mbx-r2-verify.md
  • quest/m0/ci/nix-cache-verify.md
  • quest/m0/ci/nix-cache.md
  • rs/moq-gst/scrub.sh
  • sh/gh/nightly.test.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T15:57:10.808016Z 805776c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 805776c92e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread infra/nix/lifecycle.json Resolved
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit 69cbcc8 Oct 20, 2026
4 checks passed
@kixelated
kixelated deleted the quest/m0/ci/nix-cache branch October 20, 2026 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant