lighthouse/beacon_node/beacon_chain/src
Michael Sproul 20067b9465 Remove checkpoint alignment requirements and enable historic state pruning (#4610)
## Issue Addressed

Closes #3210
Closes #3211

## Proposed Changes

- Checkpoint sync from the latest finalized state regardless of its alignment.
- Add the `block_root` to the database's split point. This is _only_ added to the in-memory split in order to avoid a schema migration. See `load_split`.
- Add a new method to the DB called `get_advanced_state`, which looks up a state _by block root_, with a `state_root` as fallback. Using this method prevents accidental accesses of the split's unadvanced state, which does not exist in the hot DB and is not guaranteed to exist in the freezer DB at all. Previously Lighthouse would look up this state _from the freezer DB_, even if it was required for block/attestation processing, which was suboptimal.
- Replace several state look-ups in block and attestation processing with `get_advanced_state` so that they can't hit the split block's unadvanced state.
- Do not store any states in the freezer database by default. All states will be deleted upon being evicted from the hot database unless `--reconstruct-historic-states` is set. The anchor info which was previously used for checkpoint sync is used to implement this, including when syncing from genesis.

## Additional Info

Needs further testing. I want to stress-test the pruned database under Hydra.

The `get_advanced_state` method is intended to become more relevant over time: `tree-states` includes an identically named method that returns advanced states from its in-memory cache.

Co-authored-by: realbigsean <seananderson33@gmail.com>
2023-08-21 05:02:32 +00:00
..
attestation_verification Attestation verification uses head state fork (#4263) 2023-05-15 02:10:41 +00:00
schema_change DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
attestation_rewards.rs Fix incorrect ideal rewards calculation (#4520) 2023-07-31 01:53:06 +00:00
attestation_verification.rs Shift networking configuration (#4426) 2023-08-03 01:51:47 +00:00
attester_cache.rs Add early attester cache (#2872) 2022-01-11 01:35:55 +00:00
beacon_block_reward.rs Update block rewards API for Capella 2023-02-14 12:09:40 +11:00
beacon_block_streamer.rs Reconstruct Payloads using Payload Bodies Methods (#4028) 2023-03-19 23:15:59 +00:00
beacon_chain.rs Remove checkpoint alignment requirements and enable historic state pruning (#4610) 2023-08-21 05:02:32 +00:00
beacon_fork_choice_store.rs Remove checkpoint alignment requirements and enable historic state pruning (#4610) 2023-08-21 05:02:32 +00:00
beacon_proposer_cache.rs Fix some typos (#4565) 2023-08-14 00:29:43 +00:00
beacon_snapshot.rs Capella eip 4844 cleanup (#3652) 2022-10-26 15:15:26 -04:00
block_reward.rs Capella eip 4844 cleanup (#3652) 2022-10-26 15:15:26 -04:00
block_times_cache.rs Add BlockTimesCache to allow additional block delay metrics (#2546) 2021-09-30 04:31:41 +00:00
block_verification.rs Remove checkpoint alignment requirements and enable historic state pruning (#4610) 2023-08-21 05:02:32 +00:00
builder.rs Remove checkpoint alignment requirements and enable historic state pruning (#4610) 2023-08-21 05:02:32 +00:00
canonical_head.rs Remove checkpoint alignment requirements and enable historic state pruning (#4610) 2023-08-21 05:02:32 +00:00
capella_readiness.rs Suggestions for Capella beacon_chain (#3999) 2023-02-21 11:05:36 +11:00
chain_config.rs Use BeaconProcessor for API requests (#4462) 2023-08-08 23:30:15 +00:00
early_attester_cache.rs Fix some typos (#3376) 2022-07-27 00:51:06 +00:00
errors.rs Phase 0 attestation rewards via Beacon API (#4474) 2023-07-18 01:48:40 +00:00
eth1_chain.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
eth1_finalization_cache.rs Deposit Cache Finalization & Fast WS Sync (#2915) 2022-10-30 04:04:24 +00:00
events.rs Don't kill SSE stream if channel fills up (#4500) 2023-08-17 02:37:29 +00:00
execution_payload.rs Make more noise when the EL is broken (#3986) 2023-03-17 00:44:02 +00:00
fork_choice_signal.rs Clippy 1.67 (#3916) 2023-01-27 09:48:42 +00:00
fork_revert.rs Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
head_tracker.rs Fix rust 1.65 lints (#3682) 2022-11-04 07:43:43 +00:00
historical_blocks.rs Backfill blocks only to the WSP by default (#4082) 2023-05-05 03:49:23 +00:00
lib.rs Shift networking configuration (#4426) 2023-08-03 01:51:47 +00:00
light_client_finality_update_verification.rs Shift networking configuration (#4426) 2023-08-03 01:51:47 +00:00
light_client_optimistic_update_verification.rs Shift networking configuration (#4426) 2023-08-03 01:51:47 +00:00
merge_readiness.rs Deprecate exchangeTransitionConfiguration functionality (#4517) 2023-07-31 23:51:39 +00:00
metrics.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
migrate.rs Remove checkpoint alignment requirements and enable historic state pruning (#4610) 2023-08-21 05:02:32 +00:00
naive_aggregation_pool.rs Clippy lints for rust 1.66 (#3810) 2022-12-16 04:04:00 +00:00
observed_aggregates.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
observed_attesters.rs Add more metrics for tracking sync messages (#4308) 2023-05-19 05:13:07 +00:00
observed_block_producers.rs Add broadcast validation routes to Beacon Node HTTP API (#4316) 2023-06-29 12:02:38 +00:00
observed_operations.rs Use head state for exit verification (#4183) 2023-04-14 01:11:46 +00:00
otb_verification_service.rs Initial Commit of Retrospective OTB Verification (#3372) 2022-07-30 00:22:38 +00:00
persisted_beacon_chain.rs Fix head tracker concurrency bugs (#1771) 2020-10-19 05:58:39 +00:00
persisted_fork_choice.rs DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
pre_finalization_cache.rs Separate execution payloads in the DB (#3157) 2022-05-12 00:42:17 +00:00
proposer_prep_service.rs Enable proposer boost re-orging (#2860) 2022-12-13 09:57:26 +00:00
schema_change.rs DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
shuffling_cache.rs Enshrine head state shuffling in the shuffling_cache (#4296) 2023-05-19 05:13:05 +00:00
snapshot_cache.rs Enable proposer boost re-orging (#2860) 2022-12-13 09:57:26 +00:00
state_advance_timer.rs Enable proposer boost re-orging (#2860) 2022-12-13 09:57:26 +00:00
sync_committee_rewards.rs Update sync rewards API for abstract exec payload 2023-01-25 15:46:47 +11:00
sync_committee_verification.rs Shift networking configuration (#4426) 2023-08-03 01:51:47 +00:00
test_utils.rs Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
timeout_rw_lock.rs fix typo (#4555) 2023-08-02 23:50:41 +00:00
validator_monitor.rs add attestation inclusion distance in http api (#4148) 2023-04-26 01:12:35 +00:00
validator_pubkey_cache.rs Fix regression in DB write atomicity (#3931) 2023-02-13 03:32:01 +00:00