lighthouse/consensus/types/src
Jimmy Chen 46be05f728 Cache target attester balances for unrealized FFG progression calculation (#4362)
## Issue Addressed

#4118 

## Proposed Changes

This PR introduces a "progressive balances" cache on the `BeaconState`, which keeps track of the accumulated target attestation balance for the current & previous epochs. The cached values are utilised by fork choice to calculate unrealized justification and finalization (instead of converting epoch participation arrays to balances for each block we receive).

This optimization will be rolled out gradually to allow for more testing. A new `--progressive-balances disabled|checked|strict|fast` flag is introduced to support this:
- `checked`: enabled with checks against participation cache, and falls back to the existing epoch processing calculation if there is a total target attester balance mismatch. There is no performance gain from this as the participation cache still needs to be computed. **This is the default mode for now.**
- `strict`: enabled with checks against participation cache, returns error if there is a mismatch. **Used for testing only**.
- `fast`: enabled with no comparative checks and without computing the participation cache. This mode gives us the performance gains from the optimization. This is still experimental and not currently recommended for production usage, but will become the default mode in a future release.
- `disabled`: disable the usage of progressive cache, and use the existing method for FFG progression calculation. This mode may be useful if we find a bug and want to stop the frequent error logs.

### Tasks

- [x] Initial cache implementation in `BeaconState`
- [x] Perform checks in fork choice to compare the progressive balances cache against results from `ParticipationCache`
- [x] Add CLI flag, and disable the optimization by default
- [x] Testing on Goerli & Benchmarking
- [x]  Move caching logic from state processing to the `ProgressiveBalancesCache` (see [this comment](https://github.com/sigp/lighthouse/pull/4362#discussion_r1230877001))
- [x] Add attesting balance metrics



Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
2023-06-30 01:13:06 +00:00
..
beacon_state Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
test_utils Realized unrealized experimentation (#3322) 2022-07-25 23:53:26 +00:00
aggregate_and_proof.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
application_domain.rs Register validator api (#3194) 2022-06-30 00:49:21 +00:00
attestation_data.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
attestation_duty.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
attestation.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
attester_slashing.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
beacon_block_body.rs Add broadcast validation routes to Beacon Node HTTP API (#4316) 2023-06-29 12:02:38 +00:00
beacon_block_header.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
beacon_block.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
beacon_committee.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
beacon_state.rs Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
bls_to_execution_change.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
builder_bid.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
chain_spec.rs Schedule Capella for Gnosis chain (#4433) 2023-06-27 01:06:51 +00:00
checkpoint.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
config_and_preset.rs Shift subnet backbone structure (attnets revamp) (#4304) 2023-05-30 06:15:56 +00:00
consts.rs Clean capella (#4019) 2023-03-01 03:19:02 +00:00
contribution_and_proof.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
deposit_data.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
deposit_message.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
deposit_tree_snapshot.rs Use JSON by default for Deposit Snapshot Sync (#4397) 2023-06-15 08:42:20 +00:00
deposit.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
enr_fork_id.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
eth1_data.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
eth_spec.rs Update gnosis capella preset (#4302) 2023-05-19 10:07:12 +00:00
execution_block_hash.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
execution_block_header.rs Verify blockHash with withdrawals 2023-01-13 12:46:54 +11:00
execution_payload_header.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
execution_payload.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
fork_context.rs Clean capella (#4019) 2023-03-01 03:19:02 +00:00
fork_data.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
fork_name.rs Implement el_offline and use it in the VC (#4295) 2023-05-17 05:51:56 +00:00
fork_versioned_response.rs Remove the unused ExecutionOptimisticForkVersionedResponse type (#4160) 2023-04-12 01:48:21 +00:00
fork.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
graffiti.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
historical_batch.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
historical_summary.rs Merge remote-tracking branch 'origin/unstable' into capella 2023-01-12 16:22:00 +11:00
indexed_attestation.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
lib.rs Remove the unused ExecutionOptimisticForkVersionedResponse type (#4160) 2023-04-12 01:48:21 +00:00
light_client_bootstrap.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
light_client_finality_update.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
light_client_optimistic_update.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
light_client_update.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
participation_flags.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
participation_list.rs Rust 1.54.0 lints (#2483) 2021-07-30 01:11:47 +00:00
payload.rs Reduce false positive logging for late builder blocks (#4073) 2023-03-17 00:44:03 +00:00
pending_attestation.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
preset.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
proposer_preparation_data.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
proposer_slashing.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
relative_epoch.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
selection_proof.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
shuffling_id.rs Optimize validator duties (#2243) 2021-03-17 05:09:57 +00:00
signed_aggregate_and_proof.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
signed_beacon_block_header.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
signed_beacon_block.rs Clean capella (#4019) 2023-03-01 03:19:02 +00:00
signed_bls_to_execution_change.rs Broadcast address changes at Capella (#3919) 2023-02-07 17:13:49 +11:00
signed_contribution_and_proof.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
signed_voluntary_exit.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
signing_data.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
slot_data.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
slot_epoch_macros.rs Use SmallVec for TreeHash packed encoding (#3581) 2022-09-16 08:54:06 +00:00
slot_epoch.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
sqlite.rs Directory Restructure (#1163) 2020-05-18 21:24:23 +10:00
subnet_id.rs Shift subnet backbone structure (attnets revamp) (#4304) 2023-05-30 06:15:56 +00:00
sync_aggregate.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
sync_aggregator_selection_data.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
sync_committee_contribution.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
sync_committee_message.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
sync_committee_subscription.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
sync_committee.rs Fix Arbitrary implementations (#3867) 2023-01-12 15:17:03 +11:00
sync_duty.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
sync_selection_proof.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
sync_subnet_id.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
tree_hash_impls.rs Fixes to make EF Capella tests pass (#3719) 2022-11-14 13:14:31 -06:00
validator_registration_data.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
validator_subscription.rs Implement standard eth2.0 API (#1569) 2020-10-01 11:12:36 +10:00
validator.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
voluntary_exit.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
withdrawal.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00