Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
86bc621
feat(ipc-codegen): napi transport over the generated FFI entry
charlielye Sep 30, 2026
2c959a5
feat(bb): wsdb-ref, the reference world state over a wsdb base contract
charlielye Sep 30, 2026
166046a
feat(bb.js): generated @aztec-foundation/wsdb-ref package
charlielye Sep 30, 2026
0ea04f6
feat(bb): wsdb-ref serves the full wsdb schema, request for request l…
charlielye Oct 27, 2026
e6eff34
fix(wsdb): atomic indexed inserts, a filled Commit status, empty slot…
charlielye Oct 28, 2026
db47469
feat(bb): wsdb-ref owns the wsdb schema and ships a server binary
charlielye Oct 28, 2026
8e0974e
feat(bb): InitGenesis carries the tree heights and initial sizes
charlielye Oct 28, 2026
a3e6426
chore(bb.js): install wsdb-ref.wasm through the package codegen
charlielye Nov 12, 2026
cb62937
fix(wsdb): word InitGenesis and its aztec-wsdb error without assuming…
charlielye Nov 12, 2026
0110275
chore(bb): wsdb-ref's genesis seeds and roots, regenerated for next's…
charlielye Nov 12, 2026
a5c16e9
chore(bb): wsdb-ref's generated code lives in the build tree
charlielye Nov 12, 2026
3f7ce32
feat(wsdb): Init, the first command, replaces InitGenesis; wsdb-ref h…
charlielye Nov 12, 2026
8064133
docs(bb): an interactive explainer of the reference world state's per…
charlielye Nov 13, 2026
0741494
docs(bb): link the treap explainer from a wsdb_ref README, not the ro…
charlielye Nov 13, 2026
b1eccac
feat(wsdb): in-process differential test over both FFI entries; libws…
charlielye Nov 13, 2026
b2ac210
chore(bb): clang-format poseidon2_capi.cpp
charlielye Nov 13, 2026
bfc2542
fix(wsdb): stage the arch packages through the generated package's pr…
charlielye Nov 13, 2026
fdda20b
fix(kvdb): cross-build the addon so the release publishes all four pl…
charlielye Nov 13, 2026
06cdc23
refactor(bb): use upstream msgpack-c 7.0.0 instead of the fork; drop …
charlielye Nov 14, 2026
c01acdd
refactor(wsdb): build wsdb-ref from source instead of linking libwsdb…
charlielye Nov 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 18 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ bench-labs: labs-bench
bench: bench-foundation bench-labs

# Release. Everything plus copy bb cross compiles to ts projects.
release-foundation: fast-foundation bb-cpp-release-dir bb-ts-cross-copy bb-avm-sim-cross-copy bb-bin-cross-copy ipc-runtime-cross
release-foundation: fast-foundation bb-cpp-release-dir bb-ts-cross-copy bb-avm-sim-cross-copy bb-wsdb-ref-cross-copy bb-bin-cross-copy ipc-runtime-cross

release-labs: fast-labs

Expand Down Expand Up @@ -143,7 +143,7 @@ avm-transpiler-cross: avm-transpiler-cross-amd64-macos avm-transpiler-cross-arm6
#==============================================================================

# Barretenberg - Aggregate target for all barretenberg sub-projects.
barretenberg: bb-cpp bb-ts bb-avm-sim bb-cdb bb-bin bb-rs bb-acir bb-docs bb-sol bb-bbup bb-crs
barretenberg: bb-cpp bb-ts bb-avm-sim bb-cdb bb-wsdb-ref bb-bin bb-rs bb-acir bb-docs bb-sol bb-bbup bb-crs

# BB C++ - Main aggregate target.
bb-cpp: bb-cpp-native bb-cpp-wasm bb-cpp-wasm-threads
Expand Down Expand Up @@ -290,6 +290,15 @@ bb-avm-sim-cross-copy: bb-avm-sim bb-cdb bb-ts-cross-copy bb-cpp-cross
bb-cdb: ipc-codegen ipc-runtime bb-avm-sim
$(call build,$@,barretenberg/ts,build_cdb)

# @aztec-foundation/wsdb-ref: the in-memory reference world state, run in-process through its
# wasm module or its Node-API addon. Ordered after bb-cdb because every barretenberg/ts target
# regenerates the same workspaces and installs into the same node_modules.
bb-wsdb-ref: ipc-codegen ipc-runtime bb-cpp-native bb-cpp-wasm bb-cdb
$(call build,$@,barretenberg/ts,build_wsdb_ref)

bb-wsdb-ref-cross-copy: bb-wsdb-ref bb-avm-sim-cross-copy bb-cpp-cross
$(call build,$@,barretenberg/ts,cross_copy_wsdb_ref)

# bb and bb-avm as npm packages (meta + one package per platform).
bb-bin: bb-cpp-native
$(call build,$@,barretenberg/ts,build_bb_bin)
Expand Down Expand Up @@ -347,6 +356,9 @@ bb-ts-tests: bb-ts
bb-avm-sim-tests: bb-avm-sim bb-cdb
$(call test,$@,barretenberg/ts,bb_avm_sim)

bb-wsdb-ref-tests: bb-wsdb-ref
$(call test,$@,barretenberg/ts,wsdb_ref)

bb-sol-tests: bb-sol
$(call test,$@,barretenberg/sol)

Expand All @@ -359,7 +371,7 @@ bb-bbup-tests: bb-bbup
bb-rs-tests: bb-rs
$(call test,$@,barretenberg/rust)

bb-tests: bb-cpp-native-tests bb-cpp-wasm-tests bb-acir-tests bb-ts-tests bb-avm-sim-tests bb-sol-tests bb-bbup-tests bb-docs-tests bb-rs-tests
bb-tests: bb-cpp-native-tests bb-cpp-wasm-tests bb-acir-tests bb-ts-tests bb-avm-sim-tests bb-wsdb-ref-tests bb-sol-tests bb-bbup-tests bb-docs-tests bb-rs-tests

bb-full-tests: bb-cpp-wasm-threads-tests bb-cpp-asan-tests bb-cpp-smt-tests

Expand Down Expand Up @@ -415,10 +427,11 @@ ipc-runtime-cross: ipc-runtime ipc-runtime-cross-arm64-linux ipc-runtime-cross-a
lmdblib:
$(call build,$@,native-packages/lmdblib)

kvdb: lmdblib
kvdb:
$(call build,$@,native-packages/kvdb)

wsdb: lmdblib
# wsdb builds barretenberg's wsdb-ref sources itself (see native-packages/wsdb/README.md), so it needs no bb build.
wsdb:
$(call build,$@,native-packages/wsdb)

# Native-package tests. lmdblib and wsdb are self-contained gtest binaries. kvdb has no
Expand Down
11 changes: 6 additions & 5 deletions barretenberg/cpp/CMakePresets.json
Original file line number Diff line number Diff line change
Expand Up @@ -770,7 +770,8 @@
"barretenberg.wasm.gz",
"barretenberg-debug.wasm",
"bb-avm-sim.wasm",
"avm_ffi_tests"
"avm_ffi_tests",
"wsdb-ref.wasm"
]
},
{
Expand Down Expand Up @@ -803,25 +804,25 @@
"name": "amd64-linux",
"configurePreset": "amd64-linux",
"inheritConfigureEnvironment": true,
"targets": ["bb", "bb-external", "bb-avm-sim"]
"targets": ["bb", "bb-external", "bb-avm-sim", "wsdb-ref", "wsdb_ref_napi"]
},
{
"name": "arm64-linux",
"configurePreset": "arm64-linux",
"inheritConfigureEnvironment": true,
"targets": ["bb", "bb-avm", "bb-external", "bb-avm-sim"]
"targets": ["bb", "bb-avm", "bb-external", "bb-avm-sim", "wsdb-ref", "wsdb_ref_napi"]
},
{
"name": "amd64-macos",
"configurePreset": "amd64-macos",
"inheritConfigureEnvironment": true,
"targets": ["bb", "bb-external", "bb-avm-sim"]
"targets": ["bb", "bb-external", "bb-avm-sim", "wsdb-ref", "wsdb_ref_napi"]
},
{
"name": "arm64-macos",
"configurePreset": "arm64-macos",
"inheritConfigureEnvironment": true,
"targets": ["bb", "bb-external", "bb-avm-sim"]
"targets": ["bb", "bb-external", "bb-avm-sim", "wsdb-ref", "wsdb_ref_napi"]
},
{
"name": "amd64-windows",
Expand Down
12 changes: 0 additions & 12 deletions barretenberg/cpp/cmake/module.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,6 @@ function(barretenberg_module_with_sources MODULE_NAME)
endif()
endif()

# enable msgpack downloading via dependency (solves race condition)
add_dependencies(${MODULE_NAME} msgpack-c)
add_dependencies(${MODULE_NAME}_objects msgpack-c)

# enable lmdb downloading via dependency (solves race condition)
if(NOT CMAKE_SYSTEM_PROCESSOR MATCHES "wasm32" AND NOT BB_LITE)
add_dependencies(${MODULE_NAME} lmdb_repo)
Expand Down Expand Up @@ -193,10 +189,6 @@ function(barretenberg_module_with_sources MODULE_NAME)
${TBB_IMPORTED_TARGETS}
)

# enable msgpack downloading via dependency (solves race condition)
add_dependencies(${MODULE_NAME}_test_objects msgpack-c)
add_dependencies(${MODULE_NAME}_tests msgpack-c)

# enable lmdb downloading via dependency (solves race condition)
if(NOT CMAKE_SYSTEM_PROCESSOR MATCHES "wasm32" AND NOT BB_LITE)
add_dependencies(${MODULE_NAME}_test_objects lmdb_repo)
Expand Down Expand Up @@ -307,10 +299,6 @@ function(barretenberg_module_with_sources MODULE_NAME)
)
endif()

# enable msgpack downloading via dependency (solves race condition)
add_dependencies(${BENCHMARK_NAME}_bench_objects msgpack-c)
add_dependencies(${BENCHMARK_NAME}_bench msgpack-c)

# enable lmdb downloading via dependency (solves race condition)
if(NOT CMAKE_SYSTEM_PROCESSOR MATCHES "wasm32" AND NOT BB_LITE)
add_dependencies(${BENCHMARK_NAME}_bench_objects lmdb_repo)
Expand Down
27 changes: 11 additions & 16 deletions barretenberg/cpp/cmake/msgpack.cmake
Original file line number Diff line number Diff line change
@@ -1,18 +1,13 @@
include(ExternalProject)
include(FetchContent)

# External project: Download msgpack-c from GitHub
set(MSGPACK_PREFIX "${CMAKE_BINARY_DIR}/_deps/msgpack-c")
set(MSGPACK_INCLUDE "${MSGPACK_PREFIX}/src/msgpack-c/include")

ExternalProject_Add(
msgpack-c
PREFIX ${MSGPACK_PREFIX}
# We need to go through some hoops to do a shallow clone of a fixed commit (as opposed to a tag).
DOWNLOAD_COMMAND
sh -c "mkdir -p ${MSGPACK_PREFIX}/src/msgpack-c && cd ${MSGPACK_PREFIX}/src/msgpack-c && git init --quiet && (git remote add origin https://github.com/AztecProtocol/msgpack-c.git 2>/dev/null || true) && git fetch --depth 1 origin --quiet c0334576ed657fb3b3c49e8e61402989fb84146d && git reset --quiet --hard FETCH_HEAD"
SOURCE_DIR ${MSGPACK_PREFIX}/src/msgpack-c
CONFIGURE_COMMAND "" # No configure step
BUILD_COMMAND "" # No build step
INSTALL_COMMAND "" # No install step
UPDATE_COMMAND "" # No update step
# msgpack-c is used header-only. SOURCE_SUBDIR names a directory without a CMakeLists.txt, so
# FetchContent_MakeAvailable only fetches it and never configures msgpack-c's own targets.
FetchContent_Declare(
msgpack
GIT_REPOSITORY https://github.com/msgpack/msgpack-c.git
GIT_TAG cpp-7.0.0
GIT_SHALLOW TRUE
SOURCE_SUBDIR include
)
FetchContent_MakeAvailable(msgpack)
set(MSGPACK_INCLUDE "${msgpack_SOURCE_DIR}/include")
14 changes: 10 additions & 4 deletions barretenberg/cpp/src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ endif()

# The build tree's src/ holds generated code (e.g. barretenberg/bbapi/generated/), included by the same
# paths as sources.
include_directories(${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${MSGPACK_INCLUDE} ${TRACY_INCLUDE} ${LMDB_INCLUDE} ${LIBDEFLATE_INCLUDE} ${HTTPLIB_INCLUDE} ${BACKWARD_INCLUDE} ${NLOHMANN_JSON_INCLUDE} ${CMAKE_CURRENT_SOURCE_DIR}/../../../ipc-runtime/cpp)
include_directories(SYSTEM ${MSGPACK_INCLUDE})
include_directories(${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${TRACY_INCLUDE} ${LMDB_INCLUDE} ${LIBDEFLATE_INCLUDE} ${HTTPLIB_INCLUDE} ${BACKWARD_INCLUDE} ${NLOHMANN_JSON_INCLUDE} ${CMAKE_CURRENT_SOURCE_DIR}/../../../ipc-runtime/cpp)

# Add avm-transpiler include path when library is provided
if(AVM_TRANSPILER_LIB)
Expand Down Expand Up @@ -121,25 +122,29 @@ add_subdirectory(barretenberg/vm2_stub)
# AVM simulator (aztec, world_state_reference, yes yes - FUZZING_AVM
# vm2; vm2 defines vm2_sim whatever AVM says)
# bb-avm-sim (avm) yes yes - -
# wsdb-ref (wsdb_ref) yes yes - -
# wsdb and cdb IPC clients (vm2_wsdb, cdb) yes - - -
# ipc-runtime yes yes - -
# world_state_reference and api in the archives yes - - -
#
# The AVM fuzzer (FUZZING_AVM) builds on top of either a fuzzing or a native build.
set(BB_WITH_AVM_SIM OFF)
set(BB_WITH_AVM_SERVICE OFF)
set(BB_WITH_WSDB_REF OFF)
set(BB_WITH_IPC_CLIENTS OFF)
set(BB_WITH_IPC_RUNTIME OFF)
set(BB_ARCHIVE_WORLD_STATE_REFERENCE OFF)
if(WASM)
set(BB_WITH_AVM_SIM ON)
set(BB_WITH_AVM_SERVICE ON)
set(BB_WITH_WSDB_REF ON)
set(BB_WITH_IPC_RUNTIME ON)
elseif(FUZZING)
set(BB_WITH_AVM_SIM ${FUZZING_AVM})
elseif(NOT BB_LITE)
set(BB_WITH_AVM_SIM ON)
set(BB_WITH_AVM_SERVICE ON)
set(BB_WITH_WSDB_REF ON)
set(BB_WITH_IPC_CLIENTS ON)
set(BB_WITH_IPC_RUNTIME ON)
set(BB_ARCHIVE_WORLD_STATE_REFERENCE ON)
Expand All @@ -151,6 +156,9 @@ if(BB_WITH_AVM_SIM)
# NOTE: Do not conditionally base this on the AVM flag as it defines a necessary vm2_sim library.
add_subdirectory(barretenberg/vm2)
endif()
if(BB_WITH_WSDB_REF)
add_subdirectory(barretenberg/wsdb_ref)
endif()
if(BB_WITH_IPC_CLIENTS)
add_subdirectory(barretenberg/vm2_wsdb)
add_subdirectory(barretenberg/cdb)
Expand Down Expand Up @@ -224,14 +232,12 @@ set(BARRETENBERG_TARGET_OBJECTS
# The core object list: what FFI consumers of bb-external need (bb_ipc_ffi_entry()).
set(BB_EXTERNAL_TARGET_OBJECTS ${BARRETENBERG_TARGET_OBJECTS})

# world_state_reference is the in-memory reference merkle DB and its bb_wsref_* C ABI, which external
# world-state implementations link for their conformance tests. It is vm2-free; vm2_stub satisfies its
# world_state_reference is the in-memory reference world state. It is vm2-free; vm2_stub satisfies its
# Stats references.
if(BB_ARCHIVE_WORLD_STATE_REFERENCE)
list(APPEND BARRETENBERG_TARGET_OBJECTS
$<TARGET_OBJECTS:world_state_reference_objects>
$<TARGET_OBJECTS:api_objects>)
list(APPEND BB_EXTERNAL_TARGET_OBJECTS $<TARGET_OBJECTS:world_state_reference_objects>)
endif()

add_library(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ target_link_libraries(
vm2_stub
libdeflate::libdeflate_static
)
add_dependencies(acir_components_check_lib msgpack-c)
if(NOT CMAKE_SYSTEM_PROCESSOR MATCHES "wasm32" AND NOT BB_LITE)
add_dependencies(acir_components_check_lib lmdb_repo)
endif()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
#include "barretenberg/crypto/merkle_tree/types.hpp"
#include "barretenberg/ecc/curves/bn254/fr.hpp"
#include "barretenberg/serialize/msgpack.hpp"
#include "barretenberg/stdlib/primitives/field/field.hpp"

namespace bb::crypto::merkle_tree {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
*
* Attack vector: An attacker crafts raw ACIR bytecode (bypassing the Noir compiler)
* containing a BlackBoxFuncCall opcode where a fixed-size array field is encoded as
* msgpack NIL (0xc0). Without the fix, the AztecProtocol/msgpack-c fork silently
* msgpack NIL (0xc0). Without the fix, msgpack-c's shared_ptr adaptor silently
* converts NIL to a null shared_ptr, which is then dereferenced unconditionally.
* With the fix, deserialization rejects NIL for required fields and throws.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,4 @@ file(GLOB_RECURSE SOURCE_FILES *.cpp)
list(FILTER SOURCE_FILES EXCLUDE REGEX ".*\\.(fuzzer|test|bench)\\.cpp$")

add_library(vm2_stub OBJECT ${SOURCE_FILES})
add_dependencies(vm2_stub msgpack-c)
target_link_libraries(vm2_stub PUBLIC barretenberg_headers)
11 changes: 4 additions & 7 deletions barretenberg/cpp/src/barretenberg/vm2_wsdb/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
# VM2 binding to the wsdb service: the generated IPC client plus the AVM adapter
# (WsdbIpcMerkleDB) that drives native AVM simulation against aztec-wsdb.
#
# The wsdb wire contract (wsdb_schema.jsonc) and the server live in the
# top-level native-packages/wsdb package, which owns the service. bb only
# needs the generated CLIENT, so it codegens from that external schema path.
# This is a deliberate (and currently the only) build-order inversion:
# bb reads native-packages/wsdb/wsdb_schema.jsonc at configure/build time.
# The wsdb wire contract (wsdb_schema.jsonc) lives with wsdb-ref, its reference implementation, in
# ../wsdb_ref. The AVM only needs the generated client.
if(TARGET vm2_sim)
set(WSDB_SCHEMA ${CMAKE_SOURCE_DIR}/../../native-packages/wsdb/wsdb_schema.jsonc)
set(WSDB_SCHEMA ${CMAKE_CURRENT_SOURCE_DIR}/../wsdb_ref/wsdb_schema.jsonc)
# Generated code is per build tree (each preset generates its own, so presets building at once never
# share a file). Remove the copy older checkouts generated into the source tree: it would shadow this one.
file(REMOVE_RECURSE ${CMAKE_CURRENT_SOURCE_DIR}/generated)
Expand Down Expand Up @@ -37,7 +34,7 @@ if(TARGET vm2_sim)
--cpp-namespace bb::wsdb
--cpp-include-dir barretenberg/vm2_wsdb/generated
DEPENDS ${WSDB_SCHEMA} ${IPC_CODEGEN_SRC}
COMMENT "Generating WSDB IPC client from native-packages/wsdb/wsdb_schema.jsonc"
COMMENT "Generating WSDB IPC client from wsdb_ref/wsdb_schema.jsonc"
VERBATIM
)
add_custom_target(wsdb_ipc_generated DEPENDS ${WSDB_GEN_OUTPUTS})
Expand Down
2 changes: 1 addition & 1 deletion barretenberg/cpp/src/barretenberg/vm2_wsdb/wsdb_wire.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
* @file wsdb_wire.hpp
* @brief The boundary between bb's merkle-tree domain types and the generated wsdb wire types.
*
* The generated `wire::` records (native-packages/wsdb/wsdb_schema.jsonc) are the serialisation
* The generated `wire::` records (wsdb_ref/wsdb_schema.jsonc) are the serialisation
* types; bb's own merkle types are what the AVM computes with. `Wire<T>` names a domain type's
* wire counterpart and converts both ways:
*
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// GENERATED FILE - DO NOT EDIT.
// Regenerate with noir-projects/fnd/scripts/regenerate_genesis_constants.sh.
#pragma once

#include "barretenberg/ecc/curves/bn254/fr.hpp"

#include <vector>

namespace bb::world_state {

/**
* @brief The protocol contracts' registration nullifiers, seeded into the nullifier tree of a production genesis.
*
* Derived from the protocol contract artifacts: per protocol contract, the class registration nullifier siloed by
* ContractClassRegistry and the instance publication nullifier siloed by ContractInstanceRegistry. Sorted ascending
* because the indexed nullifier tree requires unique, strictly increasing prefilled leaves. These determine the
* three roots below, and GENESIS_NULLIFIER_TREE_ROOT, GENESIS_BLOCK_HEADER_HASH and GENESIS_ARCHIVE_ROOT in
* constants.nr, so all of them are rewritten together and never on their own.
*/
inline std::vector<bb::fr> genesis_protocol_nullifiers()
{
return {
bb::fr("0x04287df03953b0d68b0e83e08a89067afc602626defd6002277cd6df34696818"),
bb::fr("0x0d99507b7ecac720c73bf197a0e7366a5ed80c1c1b0afe8ff8c6ecc7b5a7aefe"),
bb::fr("0x0eb50b367fb754d3a7d1238bfc105cc9b391a02e187be69038876ae9a502e877"),
bb::fr("0x16f917edd8e333013e5bb0633f7f484725bd19403854d6112992dccef4961e32"),
bb::fr("0x1cea539e01abaa5db980e7ff52ef0d2a7772310306ac625783ae435756ee326d"),
bb::fr("0x268b77cfa59397698ab19812d17573156d9a964fae96ab2d80929882f4a93837"),
};
}

/**
* @brief The genesis roots the seed vector above produces.
*
* Measured from the protocol contract artifacts together with the seeds, so that this package can check its tree
* implementation against them without reaching for the protocol constants: the GENESIS_* macros in
* common/aztec_constants.hpp come from the release named in foundation.pin, which lags this tree whenever the genesis
* moves. constants.nr remains the protocol's source of truth, and is held to this same measurement by the script that
* writes this file.
*/
inline bb::fr genesis_nullifier_tree_root()
{
return bb::fr("0x238a0a0523fa750b77e243f8d6f485f03e90f1319fe76f5876f4af3d88b3fa89");
}

inline bb::fr genesis_block_header_hash()
{
return bb::fr("0x0a72ed6f85f785d34e0a09a57c971b1623bb579531f3b85325e1d60f8209b3b8");
}

inline bb::fr genesis_archive_root()
{
return bb::fr("0x1fdfe7b76afd4dc9b9b6bd09f5fd6fb354f22cf7055ef1f11490b55a4e197291");
}

} // namespace bb::world_state
Loading
Loading