Repository navigation
feat(beacon): fast confirmation rule, its event and the engine safe hash - #749
Draft
MegaRedHand wants to merge 2 commits into
Draft
MegaRedHand wants to merge 2 commits into
MegaRedHand wants to merge 2 commits into
Conversation
MegaRedHand
added a commit
that referenced
this pull request
Nov 2, 2026
…ash (#749) contributoor subscribes to the Beacon API `fast_confirmation` event, which this node accepted but never emitted for lack of a Fast Confirmation Rule. `forkchoiceUpdated` had the same gap: the spec's `get_safe_execution_block_hash` is built on the rule, so the safe block was a stand-in (the justified checkpoint's payload). Implement the rule from consensus-specs `master` at f993cf2 (not the pinned beta.2, which ships no FCR fixtures and lacks the gloas empty-slot discount, precise committee weight, the restart-resilient root and the `support_discount <= maximum_support` assertion), one Rust function per spec function in `beacon::fast_confirmation`. The beacon actor runs it once per slot at the tick, after the head recompute and before `forkchoiceUpdated`, and skips it (resetting its variables) while the head is more than 32 slots behind the clock. Each run publishes a `fast_confirmation` event and its confirmed block becomes the engine's safe hash, which removes the documented spec deviation. The spec's helpers walk the registry or the committees per block, which is tens of passes per slot at mainnet size, so a run flattens each balance source once, folds every block's attestation score from the latest messages in one pass and sums cached per-slot committee weights; only epoch-crossing and empty-slot ranges deduplicate validators. Tests check those sums against the spec's set-based definitions and `get_attestation_score`. Cherry-picked from feat/beacon-fast-confirmation at 5247749. Adapted onto #748: its gloas `ChainEvent` variants sit beside `BeaconFastConfirmation`, and `docs/rpc.md` lists both PRs' events, with `fast_confirmation` dropped from the never-emitted list.
MegaRedHand
added a commit
that referenced
this pull request
Nov 2, 2026
feat(rpc): emit single_attestation for gossip-accepted subnet attestations ethPandaOps' contributoor subscribes to single_attestation and never to attestation, and ethlambda accepted that topic without ever sending it, so the lab showed no attestation data for our nodes. Subnet attestations are validated in ethlambda-p2p and never reach the chain actor, so p2p reports an accepted one through the GossipEventSink that #748 added, and the event bus maps it to a ChainEvent on Topic::SingleAttestation. It is the highest-rate event the node has (every attestation on its backbone subnets), so the attestation is cloned only when a client is subscribed, and the payload is boxed to keep the broadcast ring sized for the smaller variants. The payload is the existing electra SingleAttestation container, so its JSON form is the specification's. Second commit: feat(rpc): emit single_attestation for API-submitted attestations This is the specification's "(from P2P or API)" half of single_attestation: POST /eth/v2/beacon/pool/attestations now reports each attestation that passed validation and was handed to the network actor, the way the other API handlers report what a validator client or builder submits. It is a separate commit because emitting our own validators' attestations, with near-zero propagation time, can link validator indices to this node, which is why contributoor makes the topic opt-in. Dropping this commit leaves the gossip-accepted events alone; docs/spec_deviations.md would then need a line saying API submissions are not reported. Cherry-picked from feat/beacon-single-attestation-event at 6a62106. Adapted onto #749: events.rs keeps both the BeaconFastConfirmation and SingleAttestation variants and topic arms.
MegaRedHand
force-pushed
the
feat/beacon-fast-confirmation
branch
from
November 5, 2026 19:47
e9e9c45 to
b448415
Compare
contributoor subscribes to the Beacon API `fast_confirmation` event, which this node accepted but never emitted for lack of a Fast Confirmation Rule. `forkchoiceUpdated` had the same gap: the spec's `get_safe_execution_block_hash` is built on the rule, so the safe block was a stand-in (the justified checkpoint's payload). Implement the rule from consensus-specs `master` at f993cf2 (not the pinned beta.2, which ships no FCR fixtures and lacks the gloas empty-slot discount, precise committee weight, the restart-resilient root and the `support_discount <= maximum_support` assertion), one Rust function per spec function in `beacon::fast_confirmation`. The beacon actor runs it once per slot at the tick, after the head recompute and before `forkchoiceUpdated`, and skips it (resetting its variables) while the head is more than 32 slots behind the clock. Each run publishes a `fast_confirmation` event and its confirmed block becomes the engine's safe hash, which removes the documented spec deviation. The spec's helpers walk the registry or the committees per block, which is tens of passes per slot at mainnet size, so a run flattens each balance source once, folds every block's attestation score from the latest messages in one pass and sums cached per-slot committee weights; only epoch-crossing and empty-slot ranges deduplicate validators. Tests check those sums against the spec's set-based definitions and `get_attestation_score`.
…ches On mainnet a block (c2302e2b, slot 15587649) took 826 ms to import, 766 ms of it queued in the chain actor's mailbox behind the slot tick, whose FCR run took 1-2 s. The rule runs on the actor, so it has to cost about what the head computation does. It now reads what fork choice already keeps: - get_current_target_score skips latest messages from other epochs, which can never vote for the current target. Resolving their checkpoints walked roots below the live chain index, decoding a block per parent hop; that walk was the >1 s tail. - Balance sources come from the store's per-checkpoint JustifiedBalances, which now also carry the active-but-slashed validators and the total active balance, instead of a registry pass per source per run. The pulled-up head reads the current target's entry unless the head state shows a slashing processed in the current epoch, and then builds from the pulled-up state as before: activity and effective balances are fixed within an epoch, and slashings[epoch % EPOCHS_PER_SLASHINGS_VECTOR] starts every epoch at zero. - Attestation scores are the proto-array's PENDING subtree totals (#775) when the source is the justified checkpoint's, which it is on every run but an epoch's first; the remaining fold hashes roots with foldhash. - Committee weights are memoized per (source, slot), and a range across an epoch boundary reuses the later part's members, shared by every range ending at the same slot; equivocators are a bitmap. At 1M validators (the ignored fast_confirmation_timing bench), per run: mid-epoch ~27 ms -> ~0.5 ms (get_head: ~80 us); an epoch's first slot 113 ms -> 14.7 ms (get_head: 12.9 ms); its second and third ~40 ms -> 2.3-2.8 ms. New tests compare the cached scores with the fold, pre-gloas and under gloas; the filtered target score with the unfiltered sum; and the reused pulled-up balances with the pulled-up state's, with and without a slashing in the epoch. Claude-Session: https://claude.ai/code/session_011vghNmndKPkZinHRkVTwVB
MegaRedHand
force-pushed
the
feat/beacon-fast-confirmation
branch
from
November 5, 2026 19:50
b448415 to
c2fd35f
Compare
MegaRedHand
added a commit
that referenced
this pull request
Nov 5, 2026
…ash (#749): run off the fork choice's caches On mainnet a block (c2302e2b, slot 15587649) took 826 ms to import, 766 ms of it queued in the chain actor's mailbox behind the slot tick, whose FCR run took 1-2 s. The rule runs on the actor, so it has to cost about what the head computation does. It now reads what fork choice already keeps: - get_current_target_score skips latest messages from other epochs, which can never vote for the current target. Resolving their checkpoints walked roots below the live chain index, decoding a block per parent hop; that walk was the >1 s tail. - Balance sources come from the store's per-checkpoint JustifiedBalances, which now also carry the active-but-slashed validators and the total active balance, instead of a registry pass per source per run. The pulled-up head reads the current target's entry unless the head state shows a slashing processed in the current epoch, and then builds from the pulled-up state as before: activity and effective balances are fixed within an epoch, and slashings[epoch % EPOCHS_PER_SLASHINGS_VECTOR] starts every epoch at zero. - Attestation scores are the proto-array's PENDING subtree totals (#775) when the source is the justified checkpoint's, which it is on every run but an epoch's first; the remaining fold hashes roots with foldhash. - Committee weights are memoized per (source, slot), and a range across an epoch boundary reuses the later part's members, shared by every range ending at the same slot; equivocators are a bitmap. At 1M validators (the ignored fast_confirmation_timing bench), per run: mid-epoch ~27 ms -> ~0.5 ms (get_head: ~80 us); an epoch's first slot 113 ms -> 14.7 ms (get_head: 12.9 ms); its second and third ~40 ms -> 2.3-2.8 ms. New tests compare the cached scores with the fold, pre-gloas and under gloas; the filtered target score with the unfiltered sum; and the reused pulled-up balances with the pulled-up state's, with and without a slashing in the epoch. Claude-Session: https://claude.ai/code/session_011vghNmndKPkZinHRkVTwVB
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #775. The rule reads that PR's node array and per-checkpoint balances, so this PR's base is
perf/beacon-proto-array; retarget it tomainonce #775 lands.Summary
beacon::fast_confirmationinethlambda-state-transition, one Rust function per spec function, named as the spec names it.fast_confirmationevent ({"block", "slot", "current_slot"}, integers quoted) after every run, changed or not.forkchoiceUpdated'ssafehash is now the spec'sget_safe_execution_block_hash(the confirmed block's payload hash; gloas: its bid'sparent_block_hash), replacing the justified-checkpoint stand-in. The matching deviation indocs/spec_deviations.mdis gone.safehash the EL receives now comes from this rule, and no consensus-specs fixtures exist for it, only the hand-ported scenarios listed below. Its cost is benchmarked at 1M validators (see "Cost per run"), but not yet measured on a live mainnet follower; the pulled-up-head balance reuse described there is the one piece of reasoning that is not spec-literal.Design notes
Spec commit. consensus-specs
masteratf993cf2, recorded in the module doc, not the pinnedv1.7.0-beta.2(no FCR fixtures exist there, so nothing ties us to it). Since beta.2 master fixed the gloas empty-slot support discount (#5672), made committee weight precise (#5670), added the restart-resilient confirmed root (#5673) and assertssupport_discount <= maximum_support(#5788). A later bump should diffphase0,bellatrixandgloasfast-confirmation.mdagainst that commit.Where state lives. The six variables of
FastConfirmationStore(not a copy of the fork-choice store) areFastConfirmationVariablesinethlambda-types, held in theStore'sBeaconScratchnext to the other in-memory fork-choice extras (Store::fast_confirmation).FastConfirmationStorein the new module is the spec's store: a borrowedStoreplus those variables, plus a per-run memo. Initialization is lazy and equalsget_fast_confirmation_store: the first run of a process, or the first after a reset, starts every variable at the finalized checkpoint. Onlyconfirmed_rootis persisted (Metadata["beacon_fast_confirmed_root"], written when it changes; noDB_VERSIONbump, an older directory just has no row). The first synced run of a process offers that row toget_restart_resilient_confirmed_root(the spec's implementation-definedget_root_confirmed_before_restart), once.When it runs. In the beacon actor's per-slot tick (
recompute_beacon_head_with(true)): after the store clock advance, the deferred aggregates (so attestations from past slots are applied) and the head recompute, and beforeforkchoiceUpdated, so that call carries this slot's safe block. Not on block imports, which recompute the head many times a slot: the spec requiresupdate_fast_confirmation_variablesexactly once per slot, beforeget_attestation_due_ms.Sync gating. Same staleness gate as the
headevent: the run is skipped while the head is more thanHEAD_EVENT_RECENCY_SLOTS(32) behind the wall clock, and the variables are reset to the finalized checkpoint, since the rule assumes synchrony from the start of the previous slot and the stored heads would be stale and eventually pruned. It is also skipped when the head recompute failed. A failing run logs a warning, publishes nothing and leaves the variables as they were.Cost per run. The rule runs on the chain actor, so it has to cost about what
get_headdoes: every block queued behind the tick waits for it. On mainnet an earlier version of this branch made a block (c2302e2b, slot 15587649) wait 766 ms in the mailbox behind a 1-2 s run. The spec's helpers each walk the registry or the committees, once per block of the confirmed chain (up to ~64 blocks), which at ~1M validators would be tens of passes per slot. The rule instead reads what fork choice already keeps:JustifiedBalances(the snapshotget_headreads its justified balances from, perf(beacon): cache fork-choice vote weights in a node array #775), which now also carry the active-but-slashed validators and the total active balance. A registry pass happens once per checkpoint, usually when its boundary block is imported, not once per source per run. The pulled-up head reads the current target's entry unless the head state shows a slashing processed in the current epoch, in which case it builds from the pulled-up state as the spec does. That is exact because activity and effective balances are fixed within an epoch andslashings[epoch % EPOCHS_PER_SLASHINGS_VECTOR]starts every epoch at zero.PENDINGsubtree totals when the source is the justified checkpoint's balances, which it is on every run but an epoch's first. That run folds the latest messages bottom-up once (roots hashed with foldhash).get_current_target_scoreresolves only current-epoch votes. An older vote can never vote for the current target, and resolving its checkpoint walked roots below the live chain index, decoding a block per parent hop: that walk caused the >1 s runs.(source, slot); a range across an epoch boundary subtracts the members its later part already counted, shared by every range ending at the same slot. Equivocators are a bitmap, collected once per slot. Committees come from the store's shared committee cache.Measured with the ignored
fast_confirmation_timingbench (1M validators, three epochs, the rule timed separately fromget_head), mean per run:get_headSlot 0 is bounded by the previous epoch's balance source, which no fork-choice cache holds. Not covered by the bench (it has no empty slots):
get_node_support_between_slotsover an empty-slot range still reads each participant's latest message under the store lock. Alean_beacon_fast_confirmation_time_secondshistogram times each run.Safe-hash change.
forkchoice_plancallsget_safe_execution_block_hash(store): the confirmed root'scheckpoint_hash(bellatrix+: its payload hash; gloas: its bid'sparent_block_hash; a block with no known payload, such as pre-bellatrix: zero, exactly as the finalized hash already behaves). Before the rule's first run, or while it is reset, the confirmed root is the finalized block, as the spec's initialization sets it.Departures from the Python (all noted in the module doc): the live chain index with a block-table fallback for
store.blocks; a missingunrealized_justifications[root]falls back to the store's justified checkpoint (asget_voting_sourcedoes) instead of raising;is_one_confirmedis false for an optimistic (NOT_VALIDATED) block, as the spec's prose requires;get_slot_committeereturns aVec(one slot's committees are disjoint); a validator index past a balance source's registry counts as inactive there;CONFIRMATION_BYZANTINE_THRESHOLDis a constant25(the key is not aConfigfield and stays in the ignored-keys warning).Ported tests
No FCR fixtures exist, so the pyspec scenarios are Rust unit tests in the module, on a fork-choice store built directly (fabricated blocks and post-states; votes cast into the latest messages one slot late as
on_attestationapplies them). The harness emulates the FFG justification a given participation produces (honest-chain rule), since it does not run the state transition. The tests are preset-agnostic and pass on both presets.Ported (pyspec name, or the scenario):
test_basic:test_fast_confirm_an_epoch,..._with_low_participation,..._with_asynchrony_at_epoch_boundary.test_is_one_confirmed:..._passes_with_full_participation(including the inequality terms),..._fails_with_low_participation,..._support_accumulates_over_slots,..._slashing_non_supporters_helps,..._slashing_supporters_does_not_hurt,..._empty_slot_discount,..._epoch_crossing_block,..._fails_with_competing_branch,test_is_confirmed_chain_safe_passes_full_chain; plus an optimistic-block case from the spec's prose.test_empty_slots:test_fcr_handles_single_empty_slot.test_revert_finality:..._reverts_to_finalized_when_confirmed_not_canonical_mid_epoch.test_variables: slot heads updated every slot, observed-justified rotation at the epoch start, initial values.get_restart_resilient_confirmed_root,block_should_be_finalized, and the persisted-root hand-off (once per process).test_is_one_confirmed_delayed_with_mixed_payload_votes_in_empty_slot(only FULL votes count as parent support in the empty slot, against every vote pre-gloas) and thePENDINGnode.fork_choice::get_attestation_score; ancestor roots;is_full_validator_set_covered.JustifiedBalances::active_balancefor an active slashed validator.events.rs: event shape against the beacon-APIs example, and one event per run.beacon_payloads.rs:safefollows the confirmed root (pre-gloas head, gloas head, pre-gloas confirmed block under a gloas head).Not ported, and why:
test_fcr_current_epoch_*andtest_fcr_previous_epoch_*(130 generated scenarios): they drive real attestations through the state transition so FFG, unrealized justification and late inclusion fall out of them; this harness emulates the honest rule only, so porting them would test the emulation.test_ffg(will_*_be_justifiedat exactly one and two thirds, new validator activated in the head state),test_restart_gu, and thetest_revert_finalityscenarios built on late equivocations or a non-descendant unrealized justification: they need exact integer participation fractions and states with deposits and activations.test_reconfirmationand the rest oftest_variablesbeyond what is listed.test_is_one_confirmedscenarios with large, recently activated or inactive validators (custom registries),..._attester_in_two_consecutive_slots_*,..._precise_committee_weight, and thecompute_committee_weight_between_slots_matches_fork_choice_weight_*pairs (the differential tests above cover the sums).Test plan
cargo fmt --all -- --checkcargo clippy --locked --workspace --all-targets -- -D warningscargo test --locked -p ethlambda-types -p ethlambda-storage -p ethlambda-state-transition -p ethlambda-blockchain --profile release-fast --lib(mainnet preset)cargo test -p ethlambda-state-transition --profile release-fast --lib --features beacon-spec-tests,preset-minimal -- fast_confirmation(asmake test-beacon-minimalselects it)cargo test -p ethlambda-state-transition --profile release-fast --lib -- --ignored fast_confirmation_timing --nocapture(1M validators by default,FCR_BENCH_VALIDATORSto change)fast_confirmationevent on a live follower (curl -N '.../eth/v1/events?topics=fast_confirmation') and the run's time inlean_beacon_fast_confirmation_time_secondson mainnet