Repository navigation
feat(rpc): serve and publish builder blocks (fulu) - #704
MegaRedHand wants to merge 23 commits into
Conversation
🤖 Kimi Code ReviewI'll review this PR which implements the Fulu builder flow (MEV-Boost integration) for ethlambda. Let me analyze the key components systematically. Overall AssessmentThis is a substantial, well-structured PR adding builder/MEV-Boost support. The code is generally high quality with good test coverage, but I found several correctness and safety issues that need attention. Critical Issues1. Integer Overflow in
|
| Severity | Count | Items |
|---|---|---|
| Critical | 1 | Item 1: Integer overflow/precision loss in boost calculation |
| High | 1 | Item 7: Division by zero-like behavior in choice (same root cause) |
| Medium | 3 | Items 12, 13, 14 |
| Low/Style | 5 | Items 15, 16, 17, 18, 10 |
Required Changes
- Fix
choice.rsline 58 — usesaturating_mul(bid).saturating_div(100)instead ofsaturating_mul(bid / 100) - Add error logging in
post_blinded_blockforadvance_to_slotfailure - Fix typo "gloas" → "gas" in bid.rs comment
- Verify/document
submit_blinded_blocktimeout behavior - Add cross-reference in
rpc.mdtospec_deviations.mdfor fee recipient
Good Practices Observed
- Comprehensive test coverage with property-based style tests
- Clear separation of concerns (bid validation, breaker, choice, cache)
- Proper use of
tokio::spawnfor builder submission to prevent client hangup cancellation PublishedProposalsprevents equivocation across full and blinded block paths- Metrics for
Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt
🤖 Codex Code ReviewOverall this looks solid. The new builder flow is thoughtfully guarded: bid validation is fairly strict, the anti-equivocation slot claim is shared across both publish paths, and the circuit-breaker logic is explicit and tested.
If you want, I can turn these into ready-to-post GitHub review comments. Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Claude Code ReviewReview of PR 704: builder flow for fulu block productionThis covers the builder bid fetch, bid validation, the circuit breaker, the local-vs-builder choice, What looks good
Minor points
Tests Automated review by Claude (Anthropic) · sonnet · custom prompt |
69df203 to
dbf6fbb
Compare
The Engine API's fulu answer carries the execution client's wish to build locally whatever a builder pays; the type dropped it, so the builder flow could not honour it.
Block production asks the builder only for proposers that registered through this node, and needs their registered gas limit to check a bid. A newer registration replaces an older one, and one not resent for two epochs is gone, since validator clients resend every epoch.
builder-specs requires proposers to stop using builders when the chain looks unhealthy and leaves the conditions to the client. These follow Lighthouse's defaults and read only the proposal state, where an empty slot shows as a block root repeated from the slot before.
A bid must pay something, sit on the proposal's parent with its prev_randao, timestamp and withdrawals, carry blob gas for exactly its blobs, step its gas limit toward the proposer's registered target, and be signed by its builder. Each refusal has its own reason so the metric can tell a misbehaving relay from a misconfigured one. The gas rule is gloas's gossip predicate, pinned to Lighthouse's vectors.
beacon-APIs' builder_boost_factor rule taken literally, with the execution client's shouldOverrideBuilder above it and a tie going to the local payload, which never depends on a relay to reveal it.
The builder client, the registrations it was sent, the circuit breaker's thresholds and the sync status travel together, layered only when --builder is set, so every handler and test router without one keeps the local path untouched. register_validator now records what it forwards, which block production reads next. The breaker's thresholds are flags, defaulting to Lighthouse's.
With a builder configured, the proposer registered and the circuit breaker quiet, the bid is fetched alongside the local payload. A bid that validates is built into a blinded block, the state root computed by assemble_blinded_block, and wins if it pays more after the caller's builder_boost_factor; the execution client's shouldOverrideBuilder keeps the local payload whatever the bid. Either side failing leaves the other, and with no builder in play nothing changes.
publishBlindedBlockV2 checks a blinded block the way publishBlock checks a full one, the signature through a root-based check the two now share, then submits it on its own task so a validator client hanging up cannot cancel the reveal. Both endpoints claim the slot first: a second, different block for a slot is refused, since this node handing out two would get its validator slashed.
produceBlockV3's builder path, the blinded_blocks endpoint and the slot guard both publish endpoints share, the new metrics, and the one deviation from builder-specs: the bid's fee recipient is not checked, since relays pay the proposer with a transaction instead.
A claim below the finalized slot was inserted and then pruned, so the same slot could be claimed again with a different root; it is now refused. The blinded_blocks route gets the publish body limit, and the builder's answer to a signed block is logged from the submission task, which survives a validator client hanging up. A bid's block number is checked against the parent's, which is always known. produceBlockV3 now checks the local payload and the bid without a state transition, picks one, and builds only that block, falling back to the other only if the first fails to build. Tests now assert the blinded header, the value and the number of getHeader calls, and cover the cross-endpoint guard, a failed submit, an unreachable relay and both sides failing, plus the circuit breaker's edges, bid signature variants and the fallback flags.
The 413 test passed with the body limit removed, since its body was over axum's default too; a 3 MiB sibling now fails without the layer. Also covers a winning bid whose block and the local block both fail to build, tightens the status assertions of the no-block and unreachable-builder cases, and skips the packing, whose sync aggregate costs a BLS verification, when there is nothing to build a block from. A refused second block now logs the root that holds the slot.
The README's beacon flag table, rewritten on main by #716, predates `--builder`, and CLAUDE.md described the builder containers and the blinded state root but not the node's flow that uses them. The retry arm's comment also differed from main's: main says the sync aggregate "was verified above", which stops holding here once `pack` does that check, and this PR kept the older wording. It now names `pack`, so the diff against main reads as the move it is.
27cdf9b to
3ade3a4
Compare
|
Security read of this PR, with the bid validation and the blinded-block path reviewed separately. No critical or high issues: the bid checks match Lighthouse's 1. Bound the slot before
2. The registration cache trusts unsigned registrations, before the relay answers
3. Any gossip block can be replayed through A blinded block's signing root equals the full block's, so a local caller can blind any recent network block and it passes parent, proposer and signature. The node claims the slot and spawns an uncancellable Smaller
Tests I'd add: blinded-then-full on the same slot (only full-then-blinded is covered); a claim on the gloas Checked and clean: parent hash, prev_randao, timestamp and withdrawals root from the advanced state; block number with |
- Refuse a block slot that has not started yet (gossip's `is_future_slot`: more than MAXIMUM_GOSSIP_CLOCK_DISPARITY, 500 ms, ahead of this node's clock) before any state is advanced to it, on produceBlockV3, produceBlockV4, both publishBlockV2 paths and blinded_blocks. Without it a caller naming a far-future slot had the node run every epoch transition up to it. produceBlockV4 is outside this PR's diff but had the same unbounded advance. A validator client asks once its slot has started, and a block further ahead would be ignored on gossip anyway. - Accept a bid that pays nothing: it no longer fails validation, and `choose` never lets it beat a local payload whatever the boost factor, so it is served only when the local payload or its block fails. The `zero_value` rejection label is gone. - Tell a builder's 2xx other than 202 from a refusal: a new `BuilderError::UnexpectedSuccess` (`unexpected_success` outcome label), its own error log and 500 message saying the builder may still publish the block. Document that only 202 is success and that `broadcast_validation` is not read on blinded_blocks. - Write out line breaks in a builder's error text as `\n` and `\r` before the 1 KiB cap, so it cannot forge a log line (`truncate_error_text` becomes `error_text`). This and the 202 change touch builder_client, which comes from the base PRs. - Log why advancing a state failed at all five advance sites instead of dropping the error. - Tests asked for in review: blinded-then-full on one slot, a claim on the gloas publish path, a refused blinded block leaving the slot unclaimed, two different blocks posted at once, a slot that has not started on every endpoint, and a builder answering 200. Tests that produced a block on a clock still in the slot before now pin the clock at the proposal slot. Claude-Session: https://claude.ai/code/session_011vghNneoZ1Ta9W6JNqzjxm
|
@pablodeymo on point 3 (a gossip block replayed through
ethlambda already checks more here than any of them: the proposer and its signature are verified against the parent's state before the slot is claimed or anything is sent. I've left the bound as a follow-up after merge, in the PR description: either a semaphore on in-flight submissions, or forwarding only headers this node served. |
…ock, and document the unchecked builder payment - The consecutive-skips check walked back through `block_roots` one slot at a time until it found a block, with no cap. The run is now the distance from the head block, which the advanced proposal state keeps as its latest block header: `slot - 1 - latest_block_header.slot`, Lighthouse's `head_skips`. The per-epoch count still reads `block_roots`, over a fixed 32 slots. A new test advances a real state through slot processing to pin that the header stays at the head block. - `bid::validate` now says, where the check would go, why a bid's fee recipient is not compared with the registered one (builders name themselves and pay the proposer with the payload's last transaction), that the payment cannot be checked there either (the header carries no transactions, and the builder publishes the block after signing), and that the relay checks it when it validates a builder's submission. spec_deviations.md says the same at length; rpc.md and the debug log's comment point at it. Claude-Session: https://claude.ai/code/session_011vghNneoZ1Ta9W6JNqzjxm
…rations only once the builder accepts them - `choose` compared the local payload with `factor * (bid / 100)`, the beacon-APIs text's order, which rounds the bid down to a multiple of 100 wei before weighing it: at the default factor a bid up to 99 wei better than the local payload lost. It now uses `factor * bid / 100`, rounded down once at the end and saturating at `u128::MAX` (`choice::boosted`, split so no type wider than `u128` is needed). This matches Teku, Lodestar, Nimbus and our own gloas `choose_payload`; spec_deviations.md records the departure from the spec's order. - `register_validator` recorded the kept registrations for block production before forwarding them, whatever the builder answered, and this node does not verify their signatures. A registration the builder refused (a forged `gas_limit = 1` under a later timestamp, say) replaced the accepted one and made every real bid fail the gas-limit check. They are now recorded only once the builder accepts them, inside the forwarding task, as Prysm, Teku and Lodestar do. Fee recipients are still set either way. Claude-Session: https://claude.ai/code/session_011vghNneoZ1Ta9W6JNqzjxm
…deBuilder The override always puts the local block first, so the builder's block is reached under it only once the local one failed to build. That fallback stays, since a block beats a missed slot, but it now logs a warning saying the execution client's override was set aside. `local_block` also logs why assembly failed, which it dropped before, so the cause shows next to it. Claude-Session: https://claude.ai/code/session_011vghNneoZ1Ta9W6JNqzjxm
|
Thanks for the reviews. The changes are in 074b459, 086f50c, d0b6752 and dc7e26f. @pablodeymo's security review
Smaller points:
Tests: all six you listed are added:
Automated reviews
|
Brings in the register_validator review (6fb5e57). Conflicts resolved: - registrations.rs: the base's flow (fee recipients recorded before the builder is looked for, one registration per validator, an empty list answered early) with this branch's recording for block production only once the builder accepts them. The epoch stays computed beside the head read, since the record needs it. The base's new tests build an `Arc<Builder>` through `builder_for`, and `unreachable_builder` wraps the base's unreachable URL. - builder_client/error.rs: both new variants, `UnexpectedSuccess` and `Redirected`, with their labels. The redirect target goes through `error_text`, which escapes line breaks, `truncate_error_text`'s new name. - main.rs, docs/cli.md, docs/metrics.md, docs/rpc.md: both sides kept. Claude-Session: https://claude.ai/code/session_011vghNneoZ1Ta9W6JNqzjxm
|
/review |
🤖 Codex Code Review
Tests could not run because rustup attempted to write to a read-only directory. Automated review by OpenAI Codex · gpt-6.1-sol · custom prompt |
🤖 Gemini Code ReviewThis PR is well-designed and thorough. The Fulu builder flow (
No issues found — looks good to merge. Automated review by Gemini · gemini-4-argon · custom prompt |
🤖 Claude Code ReviewReview: fulu builder flow (MEV-Boost), PR 704Overall this is well structured. The bid checks, breaker, choice and publish guard are separate, testable units with good coverage. The decisions that diverge from the spec (fee recipient, boost-factor rounding) are documented. Comments below are in priority order. Correctness / security
Robustness
Minor / nits
Automated review by Claude · claude-opus-5-5 · custom prompt |
🤖 Grok Code ReviewI'll inspect the builder-flow control paths and a few helpers the diff depends on before calling out concrete issues.The equivocation claim and the slot-advance guard look like the highest-risk spots. I'll confirm how those helpers behave before writing the review. Automated review by Grok (xAI) · grok-4.7 · custom prompt |
🤖 Kimi Code ReviewReviewNo blocking correctness or security issues found. The bid validation, proposer-signature refactor, payload-selection fallback ordering, circuit-breaker arithmetic, and 202-only submission handling look sound. The same-slot claim is atomic, and the concurrent-publication and failed-submission tests cover the important races and equivocation cases well. One non-blocking caveat:
Otherwise, the PR is unusually well documented and tested, including malformed bids, zero-value bids, Automated review by Kimi (Moonshot AI) · kimi-k3.1-pro · custom prompt |
🤖 DeepSeek Code ReviewAutomated review by DeepSeek |
Claude-Session: https://claude.ai/code/session_011vghNneoZ1Ta9W6JNqzjxm # Conflicts: # crates/net/rpc/src/beacon/builder_client/error.rs # crates/net/rpc/src/beacon/registrations.rs # docs/cli.md
… behind finality, escape builder error text
Bot review follow-ups on the fulu builder flow:
- produceBlockV3: once a builder bid has validated, the local payload
has LOCAL_BUILD_DEADLINE (2 s, Lodestar's race cutoff and Lighthouse's
getPayload timeout) before it is given up on with a warning and the
bid is served through the existing fallback. Without a valid bid the
local build is still waited for, since giving up then only loses the
block.
- engine: getPayloadV5/V6 are one attempt at ENGINE_GET_PAYLOAD_TIMEOUT
(1 s) instead of the 3 x 8 s ladder. execution-apis #958 dropped the
method's timeout ("MUST respond immediately"); 1 s is the value it
replaced and what Nimbus and Prysm (pre-gloas) use.
- publishBlockV2 (fulu and gloas) and publishBlindedBlockV2 refuse,
before advancing the parent's state, a block at or below the finalized
slot and a parent at or below it that is not the finalized block
itself (gossip's IGNORE and REJECT rules), so an unauthenticated
request cannot make the node advance an old held state for epochs.
- builder error text escapes every control character and U+2028/U+2029,
not just CR/LF, and the 1 KiB cap never cuts an escape in two.
- docs: the bid signature is checked against the key the bid names,
which guards integrity, not the builder's identity; say so, with the
other clients' behaviour (all verifying clients do the same).
Claude-Session: https://claude.ai/code/session_011vghNneoZ1Ta9W6JNqzjxm
|
Thanks for the second round. The changes are in 60c697a, with #696's latest merged in at 0d4dadf. I checked each point against Lighthouse, Prysm, Teku, Lodestar and Nimbus. Changed
Checked, not changed
|
Fourth step of MEV-Boost builder support. With this PR, a fulu proposer registered through
register_validatorcan receive a builder's blinded block fromproduceBlockV3and publish it back throughblinded_blocks. Stacked on #696.Scope: the beacon node only. The validator client does not yet register, ask for blinded blocks or publish them; that is the next PR. Gloas (the builder API alongside gossip bids) comes after.
What it adds
shouldOverrideBuilderengine_getPayloadV5, carried on the built payload--builder-fallback-skips(3) consecutive empty slots (counted from the head block, as Lighthouse does), on more than--builder-fallback-skips-per-epoch(8) empty slots in the last 32, or on more than--builder-fallback-epochs-since-finalization(3) epochs since the state's finalized checkpointDOMAIN_APPLICATION_BUILDER). Every refusal is counted by reasonbuilder_boost_factor. The local payload wins ifflocal >= factor * bid / 100, multiplied first and rounded down once, so the default of 100 compares exactly (the spec's text rounds the bid first; seedocs/spec_deviations.md). A tie goes local;shouldOverrideBuilderbeats everything;u64::MAXmeans builder; the default factor is 100. The winner is chosen before assembly, so only one state transition runs. If the chosen block fails to assemble, the other one is used. This includes a local choice falling back to the bid, which Lighthouse would fail instead, following beacon-APIs' "prefer local unless it is not viable". UndershouldOverrideBuilderthat fallback logs a warning, along with why the local block failed. A bid that pays nothing is accepted but never beats a local payload, whatever the factor: it is served only when the local payload or its block failsproduceBlockV3Eth-Execution-Payload-Blinded: trueand the bid's value. A local win answers as before. Once a valid bid is in hand, the local build gets 2 s from the request (Lodestar's race cutoff; Lighthouse'sgetPayloadtimeout) before the bid is served instead; with no valid bid the local build is awaited as beforeengine_getPayloadPOST /eth/v2/beacon/blinded_blocksunexpected_success), since that builder may still publish the block. There is no local fallback, since that would be a second block.broadcast_validationis not readpost_block, the gloaspost_blockandblinded_blocksall claim the slot after the signature check and before anything leaves the node. A different root for a claimed slot gets a 400. Before any state is advanced, a block not after the finalized slot, or whose parent cannot descend from the finalized checkpoint, gets a 400 too: gossip's own rules, which bound how far an unauthenticated publish can make the node advanceproduceBlockV3,produceBlockV4, bothpublishBlockV2paths andblinded_blocksrefuse a slot that has not started (more thanMAXIMUM_GOSSIP_CLOCK_DISPARITY, 500 ms, ahead of the node's clock) before advancing any state to it, as Nimbus does. A block further ahead would be ignored on gossip anywaylean_beacon_builder_bid_rejections_total{reason},lean_beacon_builder_payload_source_total{source},lean_beacon_builder_fallback_total{reason}Known limitations
--builder-header-timeout(1 s), so it adds at most the difference.docs/spec_deviations.md);Follow-ups (to review after merge)
blinded_blocksforwards. It sends on any blinded block whose proposer signature verifies, so a local caller can blind a recent gossip block, have it pass, and make this node submit it to the relay. Nothing bounds the submissions in flight. This is not an equivocation path, but it is relay spam from this node's address. No other client limits it either. Lighthouse, Prysm, Teku, Nimbus and Lodestar all forward blinded blocks whose headers they did not serve, and none caps concurrent submissions. Two options: a semaphore on in-flight submissions, or forwarding only headers this node served.Tests
block_rootswrap;register_validatorrecording only what the builder accepts: a forged registration under a later timestamp leaves the accepted one in place.produceBlockV3against the mock relay:blinded_blocks:make lintpasses, as do the rpc lib and integration tests and the binary's tests.