lighthouse/beacon_node/beacon_chain/src
Jimmy Chen 46be05f728 Cache target attester balances for unrealized FFG progression calculation (#4362)
## Issue Addressed

#4118 

## Proposed Changes

This PR introduces a "progressive balances" cache on the `BeaconState`, which keeps track of the accumulated target attestation balance for the current & previous epochs. The cached values are utilised by fork choice to calculate unrealized justification and finalization (instead of converting epoch participation arrays to balances for each block we receive).

This optimization will be rolled out gradually to allow for more testing. A new `--progressive-balances disabled|checked|strict|fast` flag is introduced to support this:
- `checked`: enabled with checks against participation cache, and falls back to the existing epoch processing calculation if there is a total target attester balance mismatch. There is no performance gain from this as the participation cache still needs to be computed. **This is the default mode for now.**
- `strict`: enabled with checks against participation cache, returns error if there is a mismatch. **Used for testing only**.
- `fast`: enabled with no comparative checks and without computing the participation cache. This mode gives us the performance gains from the optimization. This is still experimental and not currently recommended for production usage, but will become the default mode in a future release.
- `disabled`: disable the usage of progressive cache, and use the existing method for FFG progression calculation. This mode may be useful if we find a bug and want to stop the frequent error logs.

### Tasks

- [x] Initial cache implementation in `BeaconState`
- [x] Perform checks in fork choice to compare the progressive balances cache against results from `ParticipationCache`
- [x] Add CLI flag, and disable the optimization by default
- [x] Testing on Goerli & Benchmarking
- [x]  Move caching logic from state processing to the `ProgressiveBalancesCache` (see [this comment](https://github.com/sigp/lighthouse/pull/4362#discussion_r1230877001))
- [x] Add attesting balance metrics



Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
2023-06-30 01:13:06 +00:00
..
attestation_verification Attestation verification uses head state fork (#4263) 2023-05-15 02:10:41 +00:00
schema_change DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
attestation_rewards.rs Tweaks to reward APIs (#3957) 2023-02-10 06:19:42 +00:00
attestation_verification.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
attester_cache.rs Add early attester cache (#2872) 2022-01-11 01:35:55 +00:00
beacon_block_reward.rs Update block rewards API for Capella 2023-02-14 12:09:40 +11:00
beacon_block_streamer.rs Reconstruct Payloads using Payload Bodies Methods (#4028) 2023-03-19 23:15:59 +00:00
beacon_chain.rs Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
beacon_fork_choice_store.rs DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
beacon_proposer_cache.rs Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
beacon_snapshot.rs Capella eip 4844 cleanup (#3652) 2022-10-26 15:15:26 -04:00
block_reward.rs Capella eip 4844 cleanup (#3652) 2022-10-26 15:15:26 -04:00
block_times_cache.rs Add BlockTimesCache to allow additional block delay metrics (#2546) 2021-09-30 04:31:41 +00:00
block_verification.rs Add broadcast validation routes to Beacon Node HTTP API (#4316) 2023-06-29 12:02:38 +00:00
builder.rs Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
canonical_head.rs Enshrine head state shuffling in the shuffling_cache (#4296) 2023-05-19 05:13:05 +00:00
capella_readiness.rs Suggestions for Capella beacon_chain (#3999) 2023-02-21 11:05:36 +11:00
chain_config.rs Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
early_attester_cache.rs Fix some typos (#3376) 2022-07-27 00:51:06 +00:00
errors.rs Add broadcast validation routes to Beacon Node HTTP API (#4316) 2023-06-29 12:02:38 +00:00
eth1_chain.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
eth1_finalization_cache.rs Deposit Cache Finalization & Fast WS Sync (#2915) 2022-10-30 04:04:24 +00:00
events.rs Fix order of arguments to log_count (#4060) 2023-03-13 01:40:01 +00:00
execution_payload.rs Make more noise when the EL is broken (#3986) 2023-03-17 00:44:02 +00:00
fork_choice_signal.rs Clippy 1.67 (#3916) 2023-01-27 09:48:42 +00:00
fork_revert.rs Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
head_tracker.rs Fix rust 1.65 lints (#3682) 2022-11-04 07:43:43 +00:00
historical_blocks.rs Backfill blocks only to the WSP by default (#4082) 2023-05-05 03:49:23 +00:00
lib.rs Add broadcast validation routes to Beacon Node HTTP API (#4316) 2023-06-29 12:02:38 +00:00
light_client_finality_update_verification.rs Adding light_client gossip topics (#3693) 2022-12-13 06:24:51 +00:00
light_client_optimistic_update_verification.rs light client optimistic update reprocessing (#3799) 2023-01-24 22:17:50 +00:00
merge_readiness.rs exchangeCapabilities & Capella Readiness Logging (#3918) 2023-01-31 18:26:23 +01:00
metrics.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
migrate.rs Fix rust 1.65 lints (#3682) 2022-11-04 07:43:43 +00:00
naive_aggregation_pool.rs Clippy lints for rust 1.66 (#3810) 2022-12-16 04:04:00 +00:00
observed_aggregates.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
observed_attesters.rs Add more metrics for tracking sync messages (#4308) 2023-05-19 05:13:07 +00:00
observed_block_producers.rs Add broadcast validation routes to Beacon Node HTTP API (#4316) 2023-06-29 12:02:38 +00:00
observed_operations.rs Use head state for exit verification (#4183) 2023-04-14 01:11:46 +00:00
otb_verification_service.rs Initial Commit of Retrospective OTB Verification (#3372) 2022-07-30 00:22:38 +00:00
persisted_beacon_chain.rs Fix head tracker concurrency bugs (#1771) 2020-10-19 05:58:39 +00:00
persisted_fork_choice.rs DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
pre_finalization_cache.rs Separate execution payloads in the DB (#3157) 2022-05-12 00:42:17 +00:00
proposer_prep_service.rs Enable proposer boost re-orging (#2860) 2022-12-13 09:57:26 +00:00
schema_change.rs DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
shuffling_cache.rs Enshrine head state shuffling in the shuffling_cache (#4296) 2023-05-19 05:13:05 +00:00
snapshot_cache.rs Enable proposer boost re-orging (#2860) 2022-12-13 09:57:26 +00:00
state_advance_timer.rs Enable proposer boost re-orging (#2860) 2022-12-13 09:57:26 +00:00
sync_committee_rewards.rs Update sync rewards API for abstract exec payload 2023-01-25 15:46:47 +11:00
sync_committee_verification.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
test_utils.rs Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
timeout_rw_lock.rs Add flag to disable lock timeouts (#2714) 2021-10-19 00:30:40 +00:00
validator_monitor.rs add attestation inclusion distance in http api (#4148) 2023-04-26 01:12:35 +00:00
validator_pubkey_cache.rs Fix regression in DB write atomicity (#3931) 2023-02-13 03:32:01 +00:00