Skip to content

feat(bb.js): generated bb.js-api package with an in-process wasm transport - #25433

Open
charlielye wants to merge 2 commits into
nextfrom
cl/bbjs-api-codegen
Open

charlielye wants to merge 2 commits into
nextfrom
cl/bbjs-api-codegen

Conversation

@charlielye

@charlielye charlielye commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

bb.js reached bb's wasm through ~1,400 lines of hand-written plumbing (comlink workers, WASI stubs, a heap allocator, pako inflation, base64-embedded modules) next to a generated client. This PR moves "how a client reaches a service module" into ipc-codegen/ipc-runtime, generates the bb API as its own package, and makes bb.js a facade over it.

ipc-runtime: @aztec-foundation/ipc-runtime/wasm

A generic backend for any wasi reactor implementing the FFI contract (<svc>_ipc_ffi_entry + <svc>_ipc_ffi_alloc/_free, found by suffix when not named):

  • WasmFfiBackend (async; main instance in a worker, wasi threads on further workers) and WasmFfiBackendSync, implementing the runtime's IpcClientAsync/IpcClientSync so the generated AsyncApi/SyncApi sit on them unchanged.
  • Node and browser entries selected by export condition; WebAssembly.compileStreaming loading with gzip support (browsers that cache compiled wasm start warm on a repeat visit); memory created to the module's declared shape (a threads build gets a shared memory even for one thread; maximum probed down, iOS-sized in browsers); a small WASI shim (clock, random, stdout/stderr to the logger, environ).
  • hostImports for modules whose platform layer imports functions beyond WASI (bb: logstr, throw_or_abort_impl, env_hardware_concurrency).
  • Workers are spawned through factories holding the literal new Worker(new URL(..., import.meta.url), { type: 'module' }), the expression bundlers need to see to bundle a worker's dependency graph.

ipc-codegen

  • --server --ffi now emits the FFI entry itself: C++ <svc>_ffi.{hpp,cpp} (the service defines one hook, ipc_ffi_dispatcher()), Rust <svc>_ffi.rs with an export_<svc>_ffi!(path, Handler) macro over a Handler + Default type. The exported symbols carry the service name (bb_ipc_ffi_entry) so libraries of several services can be statically linked into one binary; the Rust/Zig client ffi_backend files are generated to match instead of copied templates. The echo example exercises all three languages: a C++ ffi_test binary, a Rust in-process round trip (FfiBackend → own entry), and the Zig ffi_check.
  • TS --package gains the wasm transport (--package-transports uds,shm,wasm, --package-wasm-module, --package-wasm-threads-module, --package-wasm-host-imports): the package ships wasm/, a browser entry, its own worker scripts, createWasmBackend(Sync) and <Svc>Service.wasm(). main/types fields added for CommonJS consumers' resolution.
  • The client package owns backend selection: <Svc>Service.create(options) (and <Svc>ServiceSync.create) picks the spawned process when the binary resolves, else the wasm module; backend: 'process' | 'wasm' | <object> forces one with no fallback (asking for more wasm threads than the host can give is an error, not a downgrade); createBackend/createBackendSync expose the same policy to facades. A spawned process receives threads as HARDWARE_CONCURRENCY and RAYON_NUM_THREADS. One entry per host with its own types: node (everything), browser (wasm only), react-native (no built-in backend: a native package calls registerBackend, or the app passes one).
  • --out and --client are implied by --package (<package>/src/generated; the package is the client package); the bootstraps drop both lines.
  • Contract documented in SCHEMA_SPEC.md ("FFI entry"); flags in the README.

bb

  • Warmup command: runs the prover's hot loops once (a 2^14 Pippenger MSM over 2^i·G points, batch inversion, Poseidon2) so a tiering wasm engine has optimized them before real work. Exposed in bb.js as Barretenberg.new({ warmup: true }).
  • The generated bb_ffi.cpp provides bb_ipc_ffi_entry/_alloc/_free; c_bind.cpp shrinks to the dispatcher hook. barretenberg-rs regenerates against the new name (its FFI tests pass against libbb-external.a). The generated source is listed explicitly in CMake because the module's configure-time glob cannot see a build-time output on a clean tree.

bb.js

  • New gitignored workspace barretenberg/ts/bb.js-api (@aztec-foundation/bb.js-api), generated by barretenberg/ts/bootstrap.sh like bb-avm-sim/cdb (build_bb_js_api stages the two wasm builds and the native bb). bb.js depends on it for types, AsyncApi/SyncApi, curve constants and the wasm backends.
  • bb.js maps its BackendType options onto the package's createBackend/createBackendSync (bb's thread defaults, shm ring sizes and process lifetime settings live in that mapping); its own uds/shm/wasm backend files are gone. bb resolves through bb.js-api's per-platform packages (findBbBinary is re-exported from there); bb.js no longer bundles build/<platform>/bb. The facades are unchanged.
  • Deleted: src/barretenberg_wasm/**, bb_backends/{wasm,node/native_socket,node/native_shm,node/native_shm_async}.ts, scripts/generate.sh, scripts/copy_wasm.sh, the comlink dependency and the base64 wasm embedding in the browser postprocess.

Behaviour to note

  • Browser bundles no longer inline the wasm as base64: the modules are assets referenced with new URL(..., import.meta.url) (streaming compile + browser code cache). This needs the same bundler support the worker scripts already required (Vite: optimizeDeps.exclude for bb.js and bb.js-api). wasmPath/BB_WASM_PATH still override the module.
  • The bb binary's published home is now bb.js-api's per-platform packages (bb.js used to bundle its own copy). @aztec-foundation/bb (bb-cli) is under separate review.

Testing

  • ipc-runtime: yarn test (23 tests, incl. new module-source tests).
  • ipc-codegen echo example: C++ ffi_test, Rust cargo test --features ffi, Zig ffi_check, ts_package over uds and shm (incl. the create() default-policy check); schema_visitor tests. bb-avm-sim (process-only) and wsdb regenerate and build.
  • bb: bbapi_tests --gtest_filter='CBind.*' (incl. new WarmupSucceeds); both wasm builds export the three prefixed FFI symbols; barretenberg-rs cargo test --release --features ffi (72 tests) against libbb-external.a.
  • bb.js jest from dest: existing suites against native bb (uds + shm via the NAPI addon) and with the binary hidden (wasm fallback); new backends/wasm.test.ts covering Wasm and WasmWorker at 4 threads, warmup: true, the sync backend, and error propagation. backends/node/native_shm.test.ts (from refactor(bbapi): migrate bb + bb.js to ipc-codegen/ipc-runtime; delete legacy bb::ipc #25362, moved onto createSyncBackend) still sees a killed bb promptly. yarn formatting clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_0136VgtizjT6krfyRB6TeqKG

@AztecBot

AztecBot commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Flakey Tests

🤖 says: This CI run detected 2 tests that failed, but were tolerated due to a .test_patterns.yml entry.

\033FLAKED\033 (8;;http://ci.aztec-labs.com/24a4f0ea633f0d82�24a4f0ea633f0d828;;�): cd labs && export root= ci3= &&  yarn-project/end-to-end/scripts/run_test.sh simple src/single-node/cross-chain/inbox_bot.test.ts (287s) (code: 0) group:e2e-p2p-epoch-flakes
\033FLAKED\033 (8;;http://ci.aztec-labs.com/a428cb4d42a20e56�a428cb4d42a20e568;;�): cd labs && export root= ci3= &&  yarn-project/end-to-end/scripts/run_test.sh simple src/multi-node/slashing/inactivity_slash.test.ts (252s) (code: 0) group:e2e-p2p-epoch-flakes

@charlielye
charlielye force-pushed the cl/ipc-bb-bbjs-migrate branch from 9f50140 to b8a89ca Compare October 13, 2026 20:02
Base automatically changed from cl/ipc-bb-bbjs-migrate to next October 15, 2026 17:32
charlielye and others added 2 commits October 15, 2026 17:34
…sport

bb.js reached bb's wasm through roughly 1,400 lines of hand-written plumbing
(comlink workers, WASI stubs, a heap allocator, pako inflation, base64-embedded
modules) sitting next to a generated client. This moves "how a client reaches a
service module" into ipc-codegen/ipc-runtime, generates the bb API as its own
package, and leaves bb.js a facade over it.

ipc-runtime gains a generic wasm FFI backend for any WASI reactor implementing
the contract (<svc>_ipc_ffi_entry plus its allocator pair): an async backend that
runs the instance in a worker with wasi threads on further workers, a synchronous
one that runs on the calling thread, node and browser entries chosen by export
condition, streaming compilation with gzip support, and a WASI shim. Workers are
spawned through factories holding the literal new URL(..., import.meta.url) that
bundlers must see.

ipc-codegen emits the FFI entry itself for C++ and Rust, with symbols carrying the
service name so several services can link into one binary, and its TS --package
gains a wasm transport. The generated package owns backend selection: the spawned
process when the binary resolves, else the wasm module, with an explicit choice
forcing one and no silent downgrade. A failed command raises a generated error
class rather than an injected factory.

bb's wasm is now a plain WASI reactor. It imported a logger, an abort hook and a
thread count because barretenberg/wasi stubbed out the WASI calls the old host did
not implement; ipc-runtime implements them, so the stubs go and the three
functions are defined against stderr, the environment and proc_exit instead.

bb.js keeps its public API. Its own uds/shm/wasm backend files, the base64 wasm
embedding and the bundled bb binary are gone; bb resolves through bb.js-api's
per-platform packages.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants