Repository navigation
feat(bb.js): generated bb.js-api package with an in-process wasm transport - #25433
Open
charlielye wants to merge 2 commits into
Open
charlielye wants to merge 2 commits into
charlielye wants to merge 2 commits into
Conversation
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
from
September 14, 2026 16:03
df6b806 to
f8f51f3
Compare
charlielye
force-pushed
the
cl/bbjs-api-codegen
branch
from
September 14, 2026 16:11
8f33256 to
a2d11b4
Compare
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
from
September 14, 2026 16:45
f8f51f3 to
1c5d177
Compare
charlielye
force-pushed
the
cl/bbjs-api-codegen
branch
from
September 14, 2026 16:46
e7367ed to
a6a19a5
Compare
ludamad
approved these changes
Sep 14, 2026
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
from
September 25, 2026 12:56
1c5d177 to
3cbcfe2
Compare
charlielye
force-pushed
the
cl/bbjs-api-codegen
branch
from
September 25, 2026 12:56
5e58051 to
d5be2aa
Compare
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
from
September 29, 2026 17:43
3cbcfe2 to
c79d6db
Compare
charlielye
force-pushed
the
cl/bbjs-api-codegen
branch
from
September 29, 2026 17:43
d5be2aa to
c9da307
Compare
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
from
September 30, 2026 10:04
c79d6db to
9ff4d14
Compare
charlielye
force-pushed
the
cl/bbjs-api-codegen
branch
3 times, most recently
from
September 30, 2026 16:44
4313c16 to
322205a
Compare
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
from
October 1, 2026 18:06
b4c4cc3 to
53105d0
Compare
charlielye
force-pushed
the
cl/bbjs-api-codegen
branch
3 times, most recently
from
October 8, 2026 13:15
49f4fab to
81a7021
Compare
Collaborator
Flakey Tests🤖 says: This CI run detected 2 tests that failed, but were tolerated due to a .test_patterns.yml entry. |
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
from
October 13, 2026 20:02
9f50140 to
b8a89ca
Compare
…sport bb.js reached bb's wasm through roughly 1,400 lines of hand-written plumbing (comlink workers, WASI stubs, a heap allocator, pako inflation, base64-embedded modules) sitting next to a generated client. This moves "how a client reaches a service module" into ipc-codegen/ipc-runtime, generates the bb API as its own package, and leaves bb.js a facade over it. ipc-runtime gains a generic wasm FFI backend for any WASI reactor implementing the contract (<svc>_ipc_ffi_entry plus its allocator pair): an async backend that runs the instance in a worker with wasi threads on further workers, a synchronous one that runs on the calling thread, node and browser entries chosen by export condition, streaming compilation with gzip support, and a WASI shim. Workers are spawned through factories holding the literal new URL(..., import.meta.url) that bundlers must see. ipc-codegen emits the FFI entry itself for C++ and Rust, with symbols carrying the service name so several services can link into one binary, and its TS --package gains a wasm transport. The generated package owns backend selection: the spawned process when the binary resolves, else the wasm module, with an explicit choice forcing one and no silent downgrade. A failed command raises a generated error class rather than an injected factory. bb's wasm is now a plain WASI reactor. It imported a logger, an abort hook and a thread count because barretenberg/wasi stubbed out the WASI calls the old host did not implement; ipc-runtime implements them, so the stubs go and the three functions are defined against stderr, the environment and proc_exit instead. bb.js keeps its public API. Its own uds/shm/wasm backend files, the base64 wasm embedding and the bundled bb binary are gone; bb resolves through bb.js-api's per-platform packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Prk3bcqkopGENJUuUpT52q
charlielye
force-pushed
the
cl/bbjs-api-codegen
branch
from
October 15, 2026 17:47
81a7021 to
a1a0938
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bb.js reached bb's wasm through ~1,400 lines of hand-written plumbing (comlink workers, WASI stubs, a heap allocator, pako inflation, base64-embedded modules) next to a generated client. This PR moves "how a client reaches a service module" into ipc-codegen/ipc-runtime, generates the bb API as its own package, and makes bb.js a facade over it.
ipc-runtime:
@aztec-foundation/ipc-runtime/wasmA generic backend for any wasi reactor implementing the FFI contract (
<svc>_ipc_ffi_entry+<svc>_ipc_ffi_alloc/_free, found by suffix when not named):WasmFfiBackend(async; main instance in a worker, wasi threads on further workers) andWasmFfiBackendSync, implementing the runtime'sIpcClientAsync/IpcClientSyncso the generatedAsyncApi/SyncApisit on them unchanged.WebAssembly.compileStreamingloading with gzip support (browsers that cache compiled wasm start warm on a repeat visit); memory created to the module's declared shape (a threads build gets a shared memory even for one thread; maximum probed down, iOS-sized in browsers); a small WASI shim (clock, random, stdout/stderr to the logger, environ).hostImportsfor modules whose platform layer imports functions beyond WASI (bb:logstr,throw_or_abort_impl,env_hardware_concurrency).new Worker(new URL(..., import.meta.url), { type: 'module' }), the expression bundlers need to see to bundle a worker's dependency graph.ipc-codegen
--server --ffinow emits the FFI entry itself: C++<svc>_ffi.{hpp,cpp}(the service defines one hook,ipc_ffi_dispatcher()), Rust<svc>_ffi.rswith anexport_<svc>_ffi!(path, Handler)macro over aHandler + Defaulttype. The exported symbols carry the service name (bb_ipc_ffi_entry) so libraries of several services can be statically linked into one binary; the Rust/Zig clientffi_backendfiles are generated to match instead of copied templates. The echo example exercises all three languages: a C++ffi_testbinary, a Rust in-process round trip (FfiBackend→ own entry), and the Zigffi_check.--packagegains thewasmtransport (--package-transports uds,shm,wasm,--package-wasm-module,--package-wasm-threads-module,--package-wasm-host-imports): the package shipswasm/, abrowserentry, its own worker scripts,createWasmBackend(Sync)and<Svc>Service.wasm().main/typesfields added for CommonJS consumers' resolution.<Svc>Service.create(options)(and<Svc>ServiceSync.create) picks the spawned process when the binary resolves, else the wasm module;backend: 'process' | 'wasm' | <object>forces one with no fallback (asking for more wasm threads than the host can give is an error, not a downgrade);createBackend/createBackendSyncexpose the same policy to facades. A spawned process receivesthreadsasHARDWARE_CONCURRENCYandRAYON_NUM_THREADS. One entry per host with its own types: node (everything),browser(wasm only),react-native(no built-in backend: a native package callsregisterBackend, or the app passes one).--outand--clientare implied by--package(<package>/src/generated; the package is the client package); the bootstraps drop both lines.SCHEMA_SPEC.md("FFI entry"); flags in the README.bb
Warmupcommand: runs the prover's hot loops once (a 2^14 Pippenger MSM over2^i·Gpoints, batch inversion, Poseidon2) so a tiering wasm engine has optimized them before real work. Exposed in bb.js asBarretenberg.new({ warmup: true }).bb_ffi.cppprovidesbb_ipc_ffi_entry/_alloc/_free;c_bind.cppshrinks to the dispatcher hook. barretenberg-rs regenerates against the new name (its FFI tests pass againstlibbb-external.a). The generated source is listed explicitly in CMake because the module's configure-time glob cannot see a build-time output on a clean tree.bb.js
barretenberg/ts/bb.js-api(@aztec-foundation/bb.js-api), generated bybarretenberg/ts/bootstrap.shlike bb-avm-sim/cdb (build_bb_js_apistages the two wasm builds and the native bb). bb.js depends on it for types,AsyncApi/SyncApi, curve constants and the wasm backends.BackendTypeoptions onto the package'screateBackend/createBackendSync(bb's thread defaults, shm ring sizes and process lifetime settings live in that mapping); its own uds/shm/wasm backend files are gone. bb resolves through bb.js-api's per-platform packages (findBbBinaryis re-exported from there); bb.js no longer bundlesbuild/<platform>/bb. The facades are unchanged.src/barretenberg_wasm/**,bb_backends/{wasm,node/native_socket,node/native_shm,node/native_shm_async}.ts,scripts/generate.sh,scripts/copy_wasm.sh, the comlink dependency and the base64 wasm embedding in the browser postprocess.Behaviour to note
new URL(..., import.meta.url)(streaming compile + browser code cache). This needs the same bundler support the worker scripts already required (Vite:optimizeDeps.excludefor bb.js and bb.js-api).wasmPath/BB_WASM_PATHstill override the module.@aztec-foundation/bb(bb-cli) is under separate review.Testing
yarn test(23 tests, incl. new module-source tests).ffi_test, Rustcargo test --features ffi, Zigffi_check, ts_package over uds and shm (incl. thecreate()default-policy check);schema_visitortests. bb-avm-sim (process-only) and wsdb regenerate and build.bbapi_tests --gtest_filter='CBind.*'(incl. newWarmupSucceeds); both wasm builds export the three prefixed FFI symbols; barretenberg-rscargo test --release --features ffi(72 tests) againstlibbb-external.a.dest: existing suites against native bb (uds + shm via the NAPI addon) and with the binary hidden (wasm fallback); newbackends/wasm.test.tscoveringWasmandWasmWorkerat 4 threads,warmup: true, the sync backend, and error propagation.backends/node/native_shm.test.ts(from refactor(bbapi): migrate bb + bb.js to ipc-codegen/ipc-runtime; delete legacy bb::ipc #25362, moved ontocreateSyncBackend) still sees a killed bb promptly.yarn formattingclean.🤖 Generated with Claude Code
https://claude.ai/code/session_0136VgtizjT6krfyRB6TeqKG