Skip to content

feat(beacon): fast confirmation rule, its event and the engine safe hash - #749

Draft
MegaRedHand wants to merge 2 commits into
perf/beacon-proto-arrayfrom
feat/beacon-fast-confirmation
Draft

MegaRedHand wants to merge 2 commits into
perf/beacon-proto-arrayfrom
feat/beacon-fast-confirmation

Conversation

@MegaRedHand

@MegaRedHand MegaRedHand commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #775. The rule reads that PR's node array and per-checkpoint balances, so this PR's base is perf/beacon-proto-array; retarget it to main once #775 lands.

Summary

  • Implements the Fast Confirmation Rule (FCR) for the beacon follower: beacon::fast_confirmation in ethlambda-state-transition, one Rust function per spec function, named as the spec names it.
  • The actor runs it once per slot and publishes the Beacon API fast_confirmation event ({"block", "slot", "current_slot"}, integers quoted) after every run, changed or not.
  • forkchoiceUpdated's safe hash is now the spec's get_safe_execution_block_hash (the confirmed block's payload hash; gloas: its bid's parent_block_hash), replacing the justified-checkpoint stand-in. The matching deviation in docs/spec_deviations.md is gone.
  • contributoor (ethPandaOps) subscribes to this event; before this the node accepted the topic and never sent it.
  • Review focus: the safe hash the EL receives now comes from this rule, and no consensus-specs fixtures exist for it, only the hand-ported scenarios listed below. Its cost is benchmarked at 1M validators (see "Cost per run"), but not yet measured on a live mainnet follower; the pulled-up-head balance reuse described there is the one piece of reasoning that is not spec-literal.

Design notes

Spec commit. consensus-specs master at f993cf2, recorded in the module doc, not the pinned v1.7.0-beta.2 (no FCR fixtures exist there, so nothing ties us to it). Since beta.2 master fixed the gloas empty-slot support discount (#5672), made committee weight precise (#5670), added the restart-resilient confirmed root (#5673) and asserts support_discount <= maximum_support (#5788). A later bump should diff phase0, bellatrix and gloas fast-confirmation.md against that commit.

Where state lives. The six variables of FastConfirmationStore (not a copy of the fork-choice store) are FastConfirmationVariables in ethlambda-types, held in the Store's BeaconScratch next to the other in-memory fork-choice extras (Store::fast_confirmation). FastConfirmationStore in the new module is the spec's store: a borrowed Store plus those variables, plus a per-run memo. Initialization is lazy and equals get_fast_confirmation_store: the first run of a process, or the first after a reset, starts every variable at the finalized checkpoint. Only confirmed_root is persisted (Metadata["beacon_fast_confirmed_root"], written when it changes; no DB_VERSION bump, an older directory just has no row). The first synced run of a process offers that row to get_restart_resilient_confirmed_root (the spec's implementation-defined get_root_confirmed_before_restart), once.

When it runs. In the beacon actor's per-slot tick (recompute_beacon_head_with(true)): after the store clock advance, the deferred aggregates (so attestations from past slots are applied) and the head recompute, and before forkchoiceUpdated, so that call carries this slot's safe block. Not on block imports, which recompute the head many times a slot: the spec requires update_fast_confirmation_variables exactly once per slot, before get_attestation_due_ms.

Sync gating. Same staleness gate as the head event: the run is skipped while the head is more than HEAD_EVENT_RECENCY_SLOTS (32) behind the wall clock, and the variables are reset to the finalized checkpoint, since the rule assumes synchrony from the start of the previous slot and the stored heads would be stale and eventually pruned. It is also skipped when the head recompute failed. A failing run logs a warning, publishes nothing and leaves the variables as they were.

Cost per run. The rule runs on the chain actor, so it has to cost about what get_head does: every block queued behind the tick waits for it. On mainnet an earlier version of this branch made a block (c2302e2b, slot 15587649) wait 766 ms in the mailbox behind a 1-2 s run. The spec's helpers each walk the registry or the committees, once per block of the confirmed chain (up to ~64 blocks), which at ~1M validators would be tens of passes per slot. The rule instead reads what fork choice already keeps:

  • Balance sources are the store's per-checkpoint JustifiedBalances (the snapshot get_head reads its justified balances from, perf(beacon): cache fork-choice vote weights in a node array #775), which now also carry the active-but-slashed validators and the total active balance. A registry pass happens once per checkpoint, usually when its boundary block is imported, not once per source per run. The pulled-up head reads the current target's entry unless the head state shows a slashing processed in the current epoch, in which case it builds from the pulled-up state as the spec does. That is exact because activity and effective balances are fixed within an epoch and slashings[epoch % EPOCHS_PER_SLASHINGS_VECTOR] starts every epoch at zero.
  • Attestation scores are the proto-array's PENDING subtree totals when the source is the justified checkpoint's balances, which it is on every run but an epoch's first. That run folds the latest messages bottom-up once (roots hashed with foldhash).
  • get_current_target_score resolves only current-epoch votes. An older vote can never vote for the current target, and resolving its checkpoint walked roots below the live chain index, decoding a block per parent hop: that walk caused the >1 s runs.
  • Committee weights are memoized per (source, slot); a range across an epoch boundary subtracts the members its later part already counted, shared by every range ending at the same slot. Equivocators are a bitmap, collected once per slot. Committees come from the store's shared committee cache.

Measured with the ignored fast_confirmation_timing bench (1M validators, three epochs, the rule timed separately from get_head), mean per run:

Slot of epoch get_head before after
3-31 ~80 µs ~27 ms ~0.5 ms
2 ~70 µs ~39 ms 2.8 ms
1 ~80 µs ~41 ms 2.3 ms
0 12.9 ms 113 ms 14.7 ms

Slot 0 is bounded by the previous epoch's balance source, which no fork-choice cache holds. Not covered by the bench (it has no empty slots): get_node_support_between_slots over an empty-slot range still reads each participant's latest message under the store lock. A lean_beacon_fast_confirmation_time_seconds histogram times each run.

Safe-hash change. forkchoice_plan calls get_safe_execution_block_hash(store): the confirmed root's checkpoint_hash (bellatrix+: its payload hash; gloas: its bid's parent_block_hash; a block with no known payload, such as pre-bellatrix: zero, exactly as the finalized hash already behaves). Before the rule's first run, or while it is reset, the confirmed root is the finalized block, as the spec's initialization sets it.

Departures from the Python (all noted in the module doc): the live chain index with a block-table fallback for store.blocks; a missing unrealized_justifications[root] falls back to the store's justified checkpoint (as get_voting_source does) instead of raising; is_one_confirmed is false for an optimistic (NOT_VALIDATED) block, as the spec's prose requires; get_slot_committee returns a Vec (one slot's committees are disjoint); a validator index past a balance source's registry counts as inactive there; CONFIRMATION_BYZANTINE_THRESHOLD is a constant 25 (the key is not a Config field and stays in the ignored-keys warning).

Ported tests

No FCR fixtures exist, so the pyspec scenarios are Rust unit tests in the module, on a fork-choice store built directly (fabricated blocks and post-states; votes cast into the latest messages one slot late as on_attestation applies them). The harness emulates the FFG justification a given participation produces (honest-chain rule), since it does not run the state transition. The tests are preset-agnostic and pass on both presets.

Ported (pyspec name, or the scenario):

  • test_basic: test_fast_confirm_an_epoch, ..._with_low_participation, ..._with_asynchrony_at_epoch_boundary.
  • test_is_one_confirmed: ..._passes_with_full_participation (including the inequality terms), ..._fails_with_low_participation, ..._support_accumulates_over_slots, ..._slashing_non_supporters_helps, ..._slashing_supporters_does_not_hurt, ..._empty_slot_discount, ..._epoch_crossing_block, ..._fails_with_competing_branch, test_is_confirmed_chain_safe_passes_full_chain; plus an optimistic-block case from the spec's prose.
  • test_empty_slots: test_fcr_handles_single_empty_slot.
  • test_revert_finality: ..._reverts_to_finalized_when_confirmed_not_canonical_mid_epoch.
  • test_variables: slot heads updated every slot, observed-justified rotation at the epoch start, initial values.
  • Restart: the four cases of get_restart_resilient_confirmed_root, block_should_be_finalized, and the persisted-root hand-off (once per process).
  • Gloas: the first step of test_is_one_confirmed_delayed_with_mixed_payload_votes_in_empty_slot (only FULL votes count as parent support in the empty slot, against every vote pre-gloas) and the PENDING node.
  • Beyond pyspec: the committee-range weight and equivocation score against the spec's set-based definitions for every range over three epochs; every block's score against fork_choice::get_attestation_score; ancestor roots; is_full_validator_set_covered.
  • Cache reuse: the proto-array scores against the fold, pre-gloas and under gloas (votes split FULL/EMPTY on a parent); the epoch-filtered target score against the unfiltered sum; the reused pulled-up balances against the pulled-up state's, with and without a slashing in the epoch; JustifiedBalances::active_balance for an active slashed validator.
  • events.rs: event shape against the beacon-APIs example, and one event per run.
  • beacon_payloads.rs: safe follows the confirmed root (pre-gloas head, gloas head, pre-gloas confirmed block under a gloas head).

Not ported, and why:

  • test_fcr_current_epoch_* and test_fcr_previous_epoch_* (130 generated scenarios): they drive real attestations through the state transition so FFG, unrealized justification and late inclusion fall out of them; this harness emulates the honest rule only, so porting them would test the emulation.
  • test_ffg (will_*_be_justified at exactly one and two thirds, new validator activated in the head state), test_restart_gu, and the test_revert_finality scenarios built on late equivocations or a non-descendant unrealized justification: they need exact integer participation fractions and states with deposits and activations.
  • test_reconfirmation and the rest of test_variables beyond what is listed.
  • The test_is_one_confirmed scenarios with large, recently activated or inactive validators (custom registries), ..._attester_in_two_consecutive_slots_*, ..._precise_committee_weight, and the compute_committee_weight_between_slots_matches_fork_choice_weight_* pairs (the differential tests above cover the sums).
  • The gloas scenario's accumulation steps (no gloas chain builder).

Test plan

  • cargo fmt --all -- --check
  • cargo clippy --locked --workspace --all-targets -- -D warnings
  • cargo test --locked -p ethlambda-types -p ethlambda-storage -p ethlambda-state-transition -p ethlambda-blockchain --profile release-fast --lib (mainnet preset)
  • cargo test -p ethlambda-state-transition --profile release-fast --lib --features beacon-spec-tests,preset-minimal -- fast_confirmation (as make test-beacon-minimal selects it)
  • cargo test -p ethlambda-state-transition --profile release-fast --lib -- --ignored fast_confirmation_timing --nocapture (1M validators by default, FCR_BENCH_VALIDATORS to change)
  • Observe a fast_confirmation event on a live follower (curl -N '.../eth/v1/events?topics=fast_confirmation') and the run's time in lean_beacon_fast_confirmation_time_seconds on mainnet

MegaRedHand added a commit that referenced this pull request Nov 2, 2026
…ash (#749)

contributoor subscribes to the Beacon API `fast_confirmation` event, which
this node accepted but never emitted for lack of a Fast Confirmation Rule.
`forkchoiceUpdated` had the same gap: the spec's `get_safe_execution_block_hash`
is built on the rule, so the safe block was a stand-in (the justified
checkpoint's payload).

Implement the rule from consensus-specs `master` at f993cf2 (not the pinned
beta.2, which ships no FCR fixtures and lacks the gloas empty-slot discount,
precise committee weight, the restart-resilient root and the
`support_discount <= maximum_support` assertion), one Rust function per spec
function in `beacon::fast_confirmation`. The beacon actor runs it once per
slot at the tick, after the head recompute and before `forkchoiceUpdated`, and
skips it (resetting its variables) while the head is more than 32 slots behind
the clock. Each run publishes a `fast_confirmation` event and its confirmed
block becomes the engine's safe hash, which removes the documented
spec deviation.

The spec's helpers walk the registry or the committees per block, which is
tens of passes per slot at mainnet size, so a run flattens each balance source
once, folds every block's attestation score from the latest messages in one
pass and sums cached per-slot committee weights; only epoch-crossing and
empty-slot ranges deduplicate validators. Tests check those sums against the
spec's set-based definitions and `get_attestation_score`.

Cherry-picked from feat/beacon-fast-confirmation at 5247749.

Adapted onto #748: its gloas `ChainEvent` variants sit beside `BeaconFastConfirmation`, and `docs/rpc.md` lists both PRs' events, with `fast_confirmation` dropped from the never-emitted list.
MegaRedHand added a commit that referenced this pull request Nov 2, 2026
feat(rpc): emit single_attestation for gossip-accepted subnet attestations

ethPandaOps' contributoor subscribes to single_attestation and never to
attestation, and ethlambda accepted that topic without ever sending it, so
the lab showed no attestation data for our nodes.

Subnet attestations are validated in ethlambda-p2p and never reach the chain
actor, so p2p reports an accepted one through the GossipEventSink that #748
added, and the event bus maps it to a ChainEvent on Topic::SingleAttestation.
It is the highest-rate event the node has (every attestation on its backbone
subnets), so the attestation is cloned only when a client is subscribed, and
the payload is boxed to keep the broadcast ring sized for the smaller
variants. The payload is the existing electra SingleAttestation container, so
its JSON form is the specification's.

Second commit: feat(rpc): emit single_attestation for API-submitted attestations

This is the specification's "(from P2P or API)" half of single_attestation:
POST /eth/v2/beacon/pool/attestations now reports each attestation that
passed validation and was handed to the network actor, the way the other
API handlers report what a validator client or builder submits.

It is a separate commit because emitting our own validators' attestations,
with near-zero propagation time, can link validator indices to this node,
which is why contributoor makes the topic opt-in. Dropping this commit leaves
the gossip-accepted events alone; docs/spec_deviations.md would then need a
line saying API submissions are not reported.

Cherry-picked from feat/beacon-single-attestation-event at 6a62106.
Adapted onto #749: events.rs keeps both the BeaconFastConfirmation and SingleAttestation variants and topic arms.
@MegaRedHand
MegaRedHand force-pushed the feat/beacon-fast-confirmation branch from e9e9c45 to b448415 Compare November 5, 2026 19:47
@MegaRedHand
MegaRedHand changed the base branch from main to perf/beacon-proto-array November 5, 2026 19:47
contributoor subscribes to the Beacon API `fast_confirmation` event, which
this node accepted but never emitted for lack of a Fast Confirmation Rule.
`forkchoiceUpdated` had the same gap: the spec's `get_safe_execution_block_hash`
is built on the rule, so the safe block was a stand-in (the justified
checkpoint's payload).

Implement the rule from consensus-specs `master` at f993cf2 (not the pinned
beta.2, which ships no FCR fixtures and lacks the gloas empty-slot discount,
precise committee weight, the restart-resilient root and the
`support_discount <= maximum_support` assertion), one Rust function per spec
function in `beacon::fast_confirmation`. The beacon actor runs it once per
slot at the tick, after the head recompute and before `forkchoiceUpdated`, and
skips it (resetting its variables) while the head is more than 32 slots behind
the clock. Each run publishes a `fast_confirmation` event and its confirmed
block becomes the engine's safe hash, which removes the documented
spec deviation.

The spec's helpers walk the registry or the committees per block, which is
tens of passes per slot at mainnet size, so a run flattens each balance source
once, folds every block's attestation score from the latest messages in one
pass and sums cached per-slot committee weights; only epoch-crossing and
empty-slot ranges deduplicate validators. Tests check those sums against the
spec's set-based definitions and `get_attestation_score`.
…ches

On mainnet a block (c2302e2b, slot 15587649) took 826 ms to import, 766 ms
of it queued in the chain actor's mailbox behind the slot tick, whose FCR
run took 1-2 s. The rule runs on the actor, so it has to cost about what
the head computation does. It now reads what fork choice already keeps:

- get_current_target_score skips latest messages from other epochs, which
  can never vote for the current target. Resolving their checkpoints
  walked roots below the live chain index, decoding a block per parent
  hop; that walk was the >1 s tail.
- Balance sources come from the store's per-checkpoint JustifiedBalances,
  which now also carry the active-but-slashed validators and the total
  active balance, instead of a registry pass per source per run. The
  pulled-up head reads the current target's entry unless the head state
  shows a slashing processed in the current epoch, and then builds from
  the pulled-up state as before: activity and effective balances are fixed
  within an epoch, and slashings[epoch % EPOCHS_PER_SLASHINGS_VECTOR]
  starts every epoch at zero.
- Attestation scores are the proto-array's PENDING subtree totals (#775)
  when the source is the justified checkpoint's, which it is on every run
  but an epoch's first; the remaining fold hashes roots with foldhash.
- Committee weights are memoized per (source, slot), and a range across
  an epoch boundary reuses the later part's members, shared by every range
  ending at the same slot; equivocators are a bitmap.

At 1M validators (the ignored fast_confirmation_timing bench), per run:
mid-epoch ~27 ms -> ~0.5 ms (get_head: ~80 us); an epoch's first slot
113 ms -> 14.7 ms (get_head: 12.9 ms); its second and third ~40 ms ->
2.3-2.8 ms.

New tests compare the cached scores with the fold, pre-gloas and under
gloas; the filtered target score with the unfiltered sum; and the reused
pulled-up balances with the pulled-up state's, with and without a slashing
in the epoch.

Claude-Session: https://claude.ai/code/session_011vghNmndKPkZinHRkVTwVB
@MegaRedHand
MegaRedHand force-pushed the feat/beacon-fast-confirmation branch from b448415 to c2fd35f Compare November 5, 2026 19:50
MegaRedHand added a commit that referenced this pull request Nov 5, 2026
…ash (#749): run off the fork choice's caches

On mainnet a block (c2302e2b, slot 15587649) took 826 ms to import, 766 ms
of it queued in the chain actor's mailbox behind the slot tick, whose FCR
run took 1-2 s. The rule runs on the actor, so it has to cost about what
the head computation does. It now reads what fork choice already keeps:

- get_current_target_score skips latest messages from other epochs, which
  can never vote for the current target. Resolving their checkpoints
  walked roots below the live chain index, decoding a block per parent
  hop; that walk was the >1 s tail.
- Balance sources come from the store's per-checkpoint JustifiedBalances,
  which now also carry the active-but-slashed validators and the total
  active balance, instead of a registry pass per source per run. The
  pulled-up head reads the current target's entry unless the head state
  shows a slashing processed in the current epoch, and then builds from
  the pulled-up state as before: activity and effective balances are fixed
  within an epoch, and slashings[epoch % EPOCHS_PER_SLASHINGS_VECTOR]
  starts every epoch at zero.
- Attestation scores are the proto-array's PENDING subtree totals (#775)
  when the source is the justified checkpoint's, which it is on every run
  but an epoch's first; the remaining fold hashes roots with foldhash.
- Committee weights are memoized per (source, slot), and a range across
  an epoch boundary reuses the later part's members, shared by every range
  ending at the same slot; equivocators are a bitmap.

At 1M validators (the ignored fast_confirmation_timing bench), per run:
mid-epoch ~27 ms -> ~0.5 ms (get_head: ~80 us); an epoch's first slot
113 ms -> 14.7 ms (get_head: 12.9 ms); its second and third ~40 ms ->
2.3-2.8 ms.

New tests compare the cached scores with the fold, pre-gloas and under
gloas; the filtered target score with the unfiltered sum; and the reused
pulled-up balances with the pulled-up state's, with and without a slashing
in the epoch.

Claude-Session: https://claude.ai/code/session_011vghNmndKPkZinHRkVTwVB

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant