Repository navigation
feat: embed ethrex as the beacon node's execution client (include-ethrex) - #730
Draft
MegaRedHand wants to merge 5 commits into
Draft
MegaRedHand wants to merge 5 commits into
MegaRedHand wants to merge 5 commits into
Conversation
The embedded execution client (next commits) needs ethrex v29.0.1, the release the servers run. Moving discovery to the same tag keeps a single copy of ethrex in the graph, and putting the pin in the workspace manifest makes the shared version explicit. v25 was only the latest release when discovery landed. Against v29 the discovery server takes two new arguments: a shared local identity, which we hand over and never read back, and an execution fork id channel, which a consensus record has no use for, so it starts at None with its sender dropped. `DiscoveryConfig` gains `nat_extip_set`, tied to `--discovery.advertise-ip` so IP voting cannot override an address the operator chose.
Behind a new `include-ethrex` feature, `EngineClient::in_process` drives an ethrex running in this process through its own Engine API handlers instead of JSON-RPC over HTTP. `EngineClient` keeps one public API over a private `Transport` enum, so the chain actor and the Beacon API handlers that hold a client do not change. Requests are typed: each method builds the ethrex handler struct it names and calls `RpcHandler::handle`. ethrex keeps most of its `ExecutionPayload` fields crate-private, so a payload goes through `from_block` with the consensus layer's claimed block hash put back, which keeps ethrex checking that claim rather than a recomputation. A payload no ethrex block can carry (an undecodable transaction, a base fee past u64) is malformed by definition, and is handed to the handler's own JSON `parse` so ethrex gives the verdict it gives over the wire. Answers are the JSON values ethrex would send, decoded by the types the HTTP transport already uses; errors keep their wire codes. There is no retry ladder, since every in-process error is an answer, but the per-method timeouts stay: the chain actor awaits `newPayload` inline.
…rex) With the `include-ethrex` feature, `ethlambda beacon` without `--execution-endpoint` starts ethrex in-process and validates payloads through the in-process Engine API transport. An explicit endpoint still selects an external client over HTTP, so one image serves both setups. The new `ethlambda-ethrex` crate owns the embedded node: - `start` is ethrex's `init_l1` without its RPC servers. `init_l1` builds its RPC context inside `bind_api` and never returns it, so the steps are repeated and the context is assembled from the same parts. The crate docs say to re-diff them whenever the ethrex pin moves. - `--ethrex.*` flags are a subset of ethrex's own. Everything else keeps the standalone binary's defaults and `ETHREX_*` overrides. The data directory and the network are derived from the beacon node's. - An opt-in execution JSON-RPC server (`--ethrex.http-port`) answers `eth_*`/`net_*`/`web3_*` over the shared context, since ethrex's own server would build a second one with its own sync manager. - Shutdown runs after the chain actor joins, so no Engine API call is in flight while ethrex flushes its store. An ethrex subsystem that cancels the node takes the beacon node down too, with an error. ethrex's ports are checked against ours before anything binds. Its tests run in their own make target and CI job (`test-ethrex`), because ethrex's RPC and node crates would push the node half of the suite past the runner's disk ceiling. An extra clippy pass covers the binary's feature-gated code. The lockfile is held to the 14-day publish-age cooldown for the crates the new dependencies pulled in.
Seen on the first live run: ethrex's metrics server repeats every lean_* series and --metrics-port carries a few ethrex_* ones, since both gather the process-wide default registry. The prefixes keep them from colliding, but the page claimed the two were kept apart.
The job was written before #713 locked every workflow's actions to exact versions, so it still named `checkout@v6` and `rust-cache@v2`, neither of which `actions.lock` lists. It now uses the pinned versions and the free-disk path the other jobs use.
MegaRedHand
force-pushed
the
feat/include-ethrex
branch
from
October 29, 2026 20:49
4424b51 to
1f1b715
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🗒️ Description / Motivation
With the
include-ethrexfeature,ethlambda beaconwithout--execution-endpointstarts ethrex in-process and validates payloads through an in-process Engine API transport: everynewPayload,forkchoiceUpdatedandgetPayloadis a direct call into ethrex's own handler, with no socket, no JWT and no JSON request body. An explicit endpoint still selects an external client over HTTP, so one image serves both setups.Ported from
nightlytomain. The branch was first built onnightly, and that build (4424b51) is what ran on mainnet (see Tests). Moving it ontomainchanged two things beyond the commits themselves:Cargo.lock: every versionmainlocks is kept, and the 367 crates the ethrex dependencies add are at the versions thenightlybuild ran, exceptclipboard-win(Windows-only), moved from 5.4.1 (since yanked; 5.4.2 and 5.4.3 are younger than the cooldown) to 5.1.0 somake cooldown-checkpasses.ethrexCI job pins its actions like the rest ofci.yml(ci: pin GitHub action versions #713), sogh actions-lock --verifypasses.What Changed
Cargo.toml,crates/net/p2pDiscoveryServer(a shared local identity, an execution fork id channel started atNone);DiscoveryConfiggainsnat_extip_set, tied to--discovery.advertise-ipso IP voting cannot override an address the operator chosecrates/net/engineEngineClientkeeps one public API over a privateTransportenum (Http/InProcess), so the chain actor and Beacon API handlers do not change.in_process.rsbuilds each ethrex handler struct and callsRpcHandler::handlecrates/execution/ethrex(newethlambda-ethrex)startis ethrex'sinit_l1without its RPC servers;--ethrex.*flag subset withETHREX_*env for the rest; opt-in execution JSON-RPC server over the shared context; shutdown after the chain actor joinsbin/ethlambda/src/{cli,main}.rsMakefile,.github/workflows/ci.ymlmake test-ethrex+ CI jobethrex(kept out oftest-nodefor the runner's disk ceiling); extra clippy pass over the binary's feature-gated codedocs/ethrex.md,CLAUDE.mdCorrectness / Behavior Guarantees
--execution-endpointgiveninclude-ethrexinclude-ethrexethlambda node(lean), but the dependency does.ExecutionPayloadfields crate-private, so a payload goes throughfrom_blockwith the consensus layer's claimed block hash put back: ethrex checks that claim rather than a recomputation. A payload no ethrex block can carry (undecodable transaction, base fee pastu64) goes to the handler's own JSONparse, so ethrex gives the verdict it gives over the wire.-38005,-38006, ...). No retry ladder in-process (every error is an answer), but the per-method timeouts stay, since the chain actor awaitsnewPayloadinline.startrepeatsinit_l1's steps, so when the ethrex pin moves,init_l1,init_rpc_apiand the binary'sserver_shutdownmust be re-diffed againstcrates/execution/ethrex/src/lib.rs(also in the crate docs).Tests Added / Run
crates/execution/ethrex/tests/engine_round_trip.rs: Engine API round trips against an embedded ethrex, on Osaka and Amsterdam genesis fixtures. Run withmake test-ethrex.nightlybuild): mainnet follower on eth-5, binary built from 4424b51, one process for beacon + ethrex, from 2026-10-24 17:20Z to 2026-10-26 21:14Z. A fresh checkpoint sync on 2026-10-26 (anchor 15516640, ethrex DB kept, ~5500 blocks behind) went optimistic until ethrex caught up, thensync_distance 0andis_optimistic: false~14 min after the restart, head matching publicnode.Related Issues / PRs
feat/ethrex-discovery-contact-table, also changescrates/net/p2p/src/discovery/mod.rs; will need the v29DiscoveryServersignature)✅ Verification Checklist
make fmt— cleanmake lint(clippy with-D warnings) — cleanmake test(test-consensusplustest-node, atrelease-fast) — all passing