[MegaRedHand]

🗒️ Description / Motivation

At block import, data_availability_for read every custody column sidecar back from DataColumns (decoding each), and fork choice's is_data_available_columns then re-ran the structural check and the KZG cell batch on each of them. Every column in DataColumns was already fully verified in p2p before the chain actor stored it, so this repeated work on the import thread: ~8 ms of CPU per block in verify_cell_kzg_proof_batch under on_block on the mainnet follower, plus the reads. Lighthouse verifies a column once and carries that in its type (KzgVerifiedCustodyDataColumn -> AvailableBlock).

What Changed

• fork_choice.rs: new DataAvailability::ColumnsVerified, documented as a promise its constructor makes (every sampled sidecar passed verify_data_column_sidecar, the inclusion proof and the KZG batch, and is stored under the block's root). is_data_available_columns accepts it without running anything; Columns(sidecars) still runs the full spec check, so the fork_choice fixtures keep exercising it.
• crates/blockchain/src/lib.rs: data_availability_for returns ColumnsVerified once custody_columns_present holds, with no sidecar reads or decodes.
• Gloas envelopes: check_execution_payload_envelope_columns_verified skips only the availability step; beacon_envelope.rs no longer reads the sampled columns back.
• docs/data_storage.md note.

Correctness / Behavior Guarantees

Audit before the change:

• The only writer of DataColumns is keep_data_column, reached only through on_checked_data_columns (NewDataColumnSidecars).
• Its senders: gossip Accept (cheap + stateful checks: signature, proposer, inclusion proof, KZG); gossip Queue/Overloaded, fetched columns and replayed parked columns through column::chain_checks_for.
• One exception: this node's own proposal's custody columns (publish_beacon_block) are built locally and handed over unchecked; debug builds still re-check every write (on_checked_data_columns runs chain_checks_for under debug_assertions).
• Unverified columns live in PendingDataColumns, which the availability gate never reads.

Benchmarks

The import replay cannot measure this: BlockChainServer::for_replay has an empty custody set, so neither path reads or verifies a column (the leg came out -2.1%, consistent across rounds, but its code does not run there). The expected saving is the ~8 ms of import-thread CPU per block seen in the live perf profile, plus the sidecar decodes; it needs a live measurement.

Tests Added / Run

• Updated/added in lib.rs: complete custody columns yield ColumnsVerified; undecodable stored rows still yield ColumnsVerified (proves nothing is read); a missing column still yields None.
• cargo test --profile release-fast -p ethlambda-state-transition --lib (630), -p ethlambda-blockchain --lib (293), and with --features beacon-spec-tests (336, gloas envelope fixtures)
• Beacon spec fixtures: fork_choice/ 206, networking/ 34

Related Issues / PRs

• Part of a set of Lighthouse ports from the same profile, each its own PR.

✅ Verification Checklist

• Ran make fmt — clean
• Ran make lint (clippy with -D warnings) — clean
• Ran make test — not run; the crate lib tests and spec filters above were run instead

[MegaRedHand]

Combined result for the four Lighthouse ports (#759 pubkey index, #760 column KZG, #761 progressive target balances, #762 tree-backed participation), squashed onto nightly b1fc143 as 080bd40..1d06bf9.

Offline import replay on ethlambda-5, same three mainnet corpora, both legs built at the commit and run with --warm-committee-sums, 2 rounds, steady state n=164, paired per block:

	nightly b1fc143	+ all four (1d06bf9)	change	95% CI
STF per block	140.7 ms	20.7 ms	-120.0 ms (-85.3%)	[-123.5, -117.3]
wall (import + fork choice)	162.5 ms	43.8 ms	-118.7 ms (-73.0%)	[-122.1, -116.0]

Per STF step, ordinary blocks (n=162): sync_aggregate 71.4 -> 0.9 ms (#759), execution_requests 5.3 -> 0.0 ms (#759), state_root 24.8 -> 4.0 ms (#762), unattributed 28.0 -> 1.3 ms (#761's removed state clone and scan, #762's cheaper clone). Regressions being fixed: attestations +2.9 ms, process_epoch +9.2 ms at epoch boundaries, get_head +1.0 ms; a per-item attribution run is in progress.

Live on the ethlambda-2 mainnet follower (nightly 1d06bf9): steady stf ~30-40 ms per gossip block (nightly before the ports: ~175 ms).

[MegaRedHand]

Per-PR attribution on nightly, where the four ports are stacked in the order #759, #760 + #761, #762. Offline import replay on ethlambda-5, three mainnet corpora, --warm-committee-sums, 2 rounds, steady state n=164, paired per block:

nightly at	STF per block	change vs before the ports	95% CI
b1fc143 (before the ports)	140.1 ms
+ #759 (080bd40)	65.1 ms	-75.0 ms (-53.5%)	[-78.5, -72.3]
+ #760, #761 (33551eb)	39.9 ms	-100.2 ms (-71.5%)	[-103.7, -97.5]
+ #762 (1d06bf9)	20.8 ms	-119.4 ms (-85.2%)	[-122.9, -116.6]
+ follow-ups in #761 and #762 (2b51a33)	19.1 ms

#760 does not run in the replay (no custody columns), so the second step is #761's.

Regressions found by the per-step split, and what the follow-up commits (now pushed to #761 and #762) did about them:

step	from	added	after the follow-ups
attestations, ordinary blocks	#761: second validator read per attester	+1.4 ms	5.8 -> 4.3 ms together (before the ports: 2.9)
attestations, ordinary blocks	#762: per-attester tree read and write	+1.5 ms
process_epoch, epoch boundaries	#762	+9.1 ms	106.2 -> 104.8 ms (before the ports: 97.2)
get_head	the #760 + #761 step	+1.3 ms	unchanged

get_head's code is untouched by either PR; a perf profile puts the extra time in walk_head's own code, so it looks like codegen rather than work. The same profile shows std SipHash in fork choice's vote tally as most of get_head's ~17 ms per block; that is a separate change, not part of these PRs.

ethlambda-2 (mainnet follower) runs nightly 2b51a33 since 2026-11-03 14:42 UTC.