Skip to content

Bring runtime up to the production hosted runtime source (fdb123ea) - #1075

Merged
callumalpass merged 1 commit into
mainfrom
converge/runtime-fdb123ea
Oct 30, 2026
Merged

callumalpass merged 1 commit into
mainfrom
converge/runtime-fdb123ea

Conversation

@callumalpass

@callumalpass callumalpass commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Brings runtime/ up to the mdbase runtime source that production hosted collections run on (fdb123ea), so the next Worker and SDK releases can be built from Connect without losing production fixes. Migration-only code is left out on purpose: the production migration Worker is being retired, and Connect main's existing migration code will be deleted with it.

Do not merge until the beta.148 release Worker build has run (main is frozen for it). A second PR stacked on this one brings the remaining serving fixes (fdb123ea..f9da6acd).

Merge base

origin/main at 9d8037993 (#1078 on top of #1079). Source range: mdbase runtime 004c642e..fdb123ea, the commits since the last runtime update in Connect.

What this ports

Not ported: migration-only code

Retirement of the production migration Worker was chosen (plan reviewed 2026-10-30), so migration-only changes are not ported. That covers crates/hosted-worker/src/migration*, crates/migrate, crates/migrate-portable, deploy/hosted-worker/src/migration-*, src/legacy-source, the migration tests, fixtures and SQL, and the migration paths inside the shared files (worker.ts, engine.ts, control.ts, log.ts, abi.rs, lib.rs, replica/gen0.rs). These paths stay exactly as on Connect main. The serving changes in those shared files were replayed onto Connect main's versions, commit by commit.

Kept from Connect

Not ported: release tooling

mdbase-next release tooling (workflows, Worker publication, SDK packaging handoff scripts, release docs) is not ported; Connect has its own release path.

SDK size ceiling

The root sdk bundle measures 197.1 KiB gzip here, against the 197 KiB ceiling. The growth is the ported Viewer grants, explicit-choice hold UI, bounded cold first sync and content-free opening progress, and nothing in it can be trimmed without dropping a ported fix, so the ceiling goes to 199 KiB. The stacked PR takes it back down to 176 KiB: #975 there loads the UI session runtime on activation, and the bundle drops to 173.4 KiB.

Measurement scripts

packages/sdk/scripts/unindexed-client-read-browser.mjs and unindexed-client-read-heap.mjs stay manual. They measure memory for one canonical 8 MiB Blob read against a production runtime and its signed manifest that the caller builds and supplies, in a Playwright-driven Chromium, and write a report. They aren't pass/fail regression tests, and runtime CI has neither the browser nor a production runtime build to give them. The bounds they measured are recorded in docs/contracts/unindexed-blob-read-custody.md and oversized-markdown-bootstrap-memory.md.

CI for the new store-schema suites

#1078's native test shards already run every runtime crate's tests, including platform-native sql_schema:: (versioned SQLite schema and reindex on open) and store-pg schema_version:: (the downgrade guard), with a Postgres 17 service and MDBN_TEST_PG_URL set. This PR adds MDBN_TEST_PG_REQUIRED to that step, so the store-pg schema tests fail instead of skipping if the URL goes missing.

Changes since the 893a8409 review

  • Local build: restored Connect's build-local-application.mjs (selector and output both runtime-local.wasm), and added a positive test in local-application-build.test.ts that builds a local application from a temporary stack and runs its packaging step. It fails on 893a8409 and passes now.
  • Contracts: completed the 330183cc port. FileView key 10 and file-content-profile are in wire.cddl alongside Connect's API 1.3 additions, with the matching contract docs. check-wire-cddl.sh matches client/file-view-* fixtures, and the full CDDL check passes with all five file-view fixtures validating. The other runtime docs and scripts the first coverage pass had misfiled as release tooling are ported too (contract docs, app-wasm-smoke.mjs, the release feature denylist).
  • CI: the Postgres-required guard on Make native runtime tests block merge qualification #1078's native test step (above), instead of a separate job.
  • SDK size: ceiling to 199 KiB with the reason (above).
  • Cargo.lock: restored to main; the earlier head carried a stale mdbn-migrate → mdbn-hosted-worker edge that broke --locked builds.
  • deploy/hosted-worker/test/log.test.mjs: back to main plus Portal: one searchable collection consent picker #904's reads assertions (the Release collection directory handles after runtime shutdown #908 tests there were migration-only). crates/hosted-worker/tests/log_cache_originals.rs gains the PRAGMA user_version mock the schema floor needs.
  • Migration code and test state from the earlier head are gone.

Known flaky tests (also on main)

  • hosted-worker test/legacy-source-bridge-workerd.test.mjs: composition tests fail under parallel load. Main 53260ccde fails 8 to 10 of them per run with four suites in parallel; they pass run alone.
  • SDK test/app-http-log-network.test.ts, "actual loopback HTTP object transfer…": intermittent; failed in 2 of 3 runs on main.

Follow-up

Checks

  • cargo check --locked --workspace --tests: clean. crates/hosted-worker wasm builds with --locked.
  • cargo test --workspace --no-fail-fast (excluding the unchanged daemon): passes (the two platform-native Bases fixtures that failed on main are fixed by Make native runtime tests block merge qualification #1078, which this is based on).
  • sql_schema:: 7 passed, 1 ignored. schema_version:: compiles and runs, but skips locally (no Postgres on the remote builder); CI runs it against Postgres.
  • scripts/check-wire-cddl.sh: passes.
  • SDK: typecheck clean; build OK; public API matches public-api.txt (the only change is the collection_not_yet_migrated outcome); size 197.1 / 199 KiB. Tests pass except wasm-runtime.test, which needs cargo xtask wasm (CI runs it), and the known loopback flake.
  • hosted-worker tests with a freshly built hosted.wasm: 788 passed, 18 skipped (real Postgres; CI runs them); the only failures are the known legacy-source bridge flake.

Per-commit coverage (004c642e..fdb123ea)

"Serving parts ported; migration parts left out" means every line from the commit that survives in fdb123ea is in this tree, except lines in migration-only code.

Commit Subject Status
ea4637a43 Upstream runtime changes made directly in Connect's vendored runtime/ (#887) Connect copy kept: this change originated in Connect; the remaining differences are migration scoped-recovery tooling
1b86dc73e hosted-worker: migration verification status, digest sample, staging legacy admission (#88… ported; not ported: migration parts and release tooling or docs
5f49ea6a1 hosted-worker: bundled, digest-pinned, cumulative KMS collection allowlist; 409 for verdic… ported; not ported: release tooling or docs
b0e569bc1 ci: run the hosted Worker suite and native workerd qualification on PRs (#890) not ported: release tooling or docs
10bbcb5a3 hosted-worker: back off migration waits from 1 s and retry inside the wake (#891) ported; not ported: migration parts and release tooling or docs
afc7b3665 hosted-worker: retry a hold check once when the kept connection died idle (#893) not ported: migration-only
a64643087 Enable release-owned production SDK packaging and typed collection upgrades (#894) ported; Connect already has a superset of the SDK error codes
17f4ab641 Require bundled KMS allowlists for staging and production (#892) ported; not ported: release tooling or docs
c22cb642d Share authenticated release handoff intake across build tools (#896) ported; not ported: release tooling or docs
194ac11bd deps: update rustls for RUSTSEC advisories (#898) ported
96641f920 Embed managed environment trust in staged and production daemon artifacts (#897) ported; not ported: release tooling or docs; Connect’s pins or daemon kept
b2d162fe5 Enable trust-bound staging escrow service-device generation (#895) ported; the import line keeps Connect’s localBuild export
22704e9ab Add protected source-qualified staging Worker publication (#899) not ported: release tooling (the attest-log-worker.mjs exports only serve mdbase-next’s Worker publisher)
135c08130 Fix legacy hold transport retries without replaying migration custody (#901) not ported: migration-only
d0f84194e Reduce migration scan round trips and transient prepare waits (#900) not ported: migration-only
32a35bb7e Preserve migration snapshot liveness and diagnose source failures (#902) not ported: migration-only
89cb6ba08 Batch bounded metadata-only import preflight (#903) not ported: migration-only
a5757dd3f fix(hosted): honor idle backoff after cold opens (#904) ported; the cold-idle expectation adds Connect’s collection-mode-device key
e5d2b87b1 Fix migration claim rate-limit retries and prepare diagnostics (#905) serving parts ported; migration parts left out
0413be783 Recover pre-base generation-zero UNKNOWNs with audited cancellation (#906) serving parts ported; migration parts left out
b29f1a6ce Finish native Base installation within bounded migration pumps (#908) serving parts ported; migration parts left out
66b249353 Reuse hold frontends within migration wakes (#907) not ported: migration-only
74f32cf0c Recover detached migration Drivers on fresh owned claims (#911) serving parts ported; migration parts left out
e9aad3e61 Preserve feedback-activated migration claim pacing on main (#913) serving parts ported; migration parts left out
9cc94eb6d Fence UNKNOWN AppendBase publication before receipted rollback (#912) serving parts ported; migration parts left out
fe446894f Ignore unmaintained zbus-stack advisories reached via notify-rust (#916) Connect’s deny.toml advisory policy kept
ab9c7d4d8 Focus the original application approval window on repeat gestures (#915) ported
0e44e1e10 Fix paced native read admission and initial busy mint retries (#914) ported
192809d0e perf: overlap sealed migration-window uploads with cap2 and full drain (#917) not ported: migration-only
6120bcdc9 fix: bound ReadNew and resource inventory SQL replies (#918) ported; not ported: migration parts and release tooling or docs
ddff6d815 SDK: make cold first sync progress-bounded (#919) ported
85de89626 Bound shared snapshot/backfill and retained-state SQL replies by encoded bytes (#920) ported; not ported: migration parts and release tooling or docs
f705122dc Refuse unsupported Gen0 records before original action custody (#921) serving parts ported; migration parts left out
a350db1ca Stream ReadNew records through canonical bounded byte groups (#923) ported; not ported: migration parts and release tooling or docs; Connect’s pins or daemon kept
52912e66c test: cover member attachment reads after bounded snapshot scans (#924) ported
0fabaf205 sdk: refresh bounded application runtime from main (#925) ported; not ported: release tooling or docs; Connect’s pins or daemon kept
bd131b16c Report content-free collection opening progress (#926) ported; not ported: release tooling or docs
f5ede226d Version durable SQL schemas and rebuild derived metadata on open ported; not ported: release tooling or docs
d54e4a861 Qualify upgrade open cost and align host/snapshot fixtures ported
f6c91b724 Map the store schema floor onto supported DO SQLite storage ported; not ported: release tooling or docs
3b849b15e Merge pull request #928 merge commit
cc492f3f3 fix(sdk): make Private Add a null-hint unified chooser alias (#929) ported; not ported: release tooling or docs
f9b90e9ce Ask the rollout check before an explicit takeover stops the old connector (#910) not ported: Connect’s daemon or SDK pins kept
18f32bd5a Import legacy hosted mirror edits through the canonical daemon (#931) not ported: Connect’s daemon or SDK pins kept
37fa32177 Support canonical oversized Markdown in hosted migration (#930) serving parts ported; migration parts left out
3e3177908 test: cover member removal key and live-delivery fences (#932) ported
6ec021f60 Fix whole-source object requests during unindexed cold open (#933) ported
4cc036746 Test canonical unindexed snapshot cold open through real LS and daemon (#934) not ported: Connect’s daemon or SDK pins kept
cae8e5eb7 fix: complete bounded Direct object reads in hosted bootstrap (#935) ported without the migration nativePending argument
b1710dcf9 docs: record measured oversized Markdown bootstrap memory exception (#936) ported
2cf0a9207 fix(sdk): remove every stored Private grant on sign-out, and report failure (#937) ported
17033e7ca chore(sdk): pin accepted app runtime with oversized-note fix (#939) not ported: Connect’s daemon or SDK pins kept
d64ca522a chore: drop private provenance wording from fixture and contract docs (#938) ported; Connect’s pins or daemon kept
ff190f963 chore(sdk): follow the a573 runtime pin in the method manifest and asset test (#940) ported; Connect’s pins or daemon kept
330183cc4 feat: add lease-bound full-source reads for canonical unindexed Blob Markdown ported; the wire-CDDL fixture matcher takes file-view-* as a glob
e488e0874 fix: keep AttachmentV1 callback retirement unchanged for Blob staging ported
725d72b2e refactor: box metadata pins without changing leased attachment storage ported
05c987406 fix: preserve unknown profile metadata and qualify approved read memory bound ported
f7a100b8b fix: refuse Hosted unindexed Blob reads after native preflight ported; not ported: migration parts
4000dfb92 feat(sdk): support staging application packaging (#941) ported
bc21d76b2 Merge pull request #942 ported
a3023aee3 chore(sdk): adopt accepted application Blob-read runtime (#943) ported; Connect’s pins or daemon kept
1eccc064a feat(worker): share legacy source admission and provider unwrap across tiers serving parts ported; legacy-source admission and its wrangler.jsonc comments are migration-only
70f09d9e1 test(worker): consume real provider wrapping vectors across tiers ported; not ported: migration parts and release tooling or docs
eaa4e697f fix(sdk): retain file stream cancellation across open (#944) ported
991043dbb fix(worker): validate stored provider key references before decrypt not ported: migration-only
db21c1573 fix(hosted): support keyed blob-part migration addresses not ported: migration-only
2923b3519 Merge pull request #945 merge commit
fadddd5ee Merge pull request #946 merge commit
67e2afc1b fix(hosted): allow absent dedicated provider KMS custody only in LAB not ported: migration-only
ae3e73883 Merge pull request #947 merge commit
9598d138d Persist managed Gen0/AppendBase cleanup obligations for negative disposition (#948) serving parts ported; migration parts left out
82647c493 Accept exact-main CI dispatches for staging Worker publication (#949) not ported: release tooling or docs
6f8e1824c Admit fixed production tier in integrated Worker publisher (#950) not ported: release tooling or docs
377790c59 Fix integrated Worker setup and remove private provenance citations (#951) ported; not ported: migration parts and release tooling or docs
0356a99c1 Bump rustls-webpki to 0.103.16 (RUSTSEC-2026-0382) (#952) ported
bfdc26e38 Fix release-workers Postgres health-cmd quoting (#953) not ported: release tooling or docs
2b88ffeb6 Make release-workers hosted qualification pass on Node 24 with zero skips (#954) not ported: migration-only
483db20da Accept Node 24 spec-reporter summary in release-workers skip gate (#955) not ported: release tooling or docs
a79868b82 Update release-workers workflow pin for the dual-reporter skip gate (#956) not ported: release tooling or docs
e76d7e66e Do not publish Wrangler source maps in managed Worker bundles (#957) not ported: release tooling or docs
3c3a50a81 Allow-all KMS module for CP hosted/escrow; enable production escrow generation (#958) ported; not ported: release tooling or docs
fdb123eac log: allow browser reads from any origin (#959) ported; not ported: release tooling or docs; Connect’s pins or daemon kept

@callumalpass
callumalpass force-pushed the converge/runtime-fdb123ea branch from 893a840 to 9f1ca55 Compare October 30, 2026 05:27
@callumalpass callumalpass changed the title Bring runtime up to the production hosted runtime source Bring runtime up to the production hosted runtime source (fdb123ea) Oct 30, 2026
Ports the serving changes from mdbase runtime 004c642e..fdb123ea that
production hosted collections and the current app SDK run but Connect's
runtime lacked: byte-bounded SQL replies, full-source unindexed Blob
reads, idle backoff after cold opens, versioned store schemas, bundled
KMS allowlists, escrow service-device generation, SDK cold-sync and
opening-progress fixes, and their contract docs.

Migration-only changes are not ported; the production migration Worker
is being retired. Connect's daemon, log Worker, SDK runtime pins and
local build script are kept. The native test shards now require
Postgres for the store-pg schema tests instead of skipping them.
@callumalpass
callumalpass force-pushed the converge/runtime-fdb123ea branch from 9f1ca55 to 6c396b7 Compare October 30, 2026 05:59
@callumalpass
callumalpass added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit a841032 Oct 30, 2026
53 checks passed
@callumalpass
callumalpass deleted the converge/runtime-fdb123ea branch October 30, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant