lighthouse/consensus/types/src
realbigsean b22ac95d7f v1.1.6 Fork Choice changes (#2822)
## Issue Addressed

Resolves: https://github.com/sigp/lighthouse/issues/2741
Includes: https://github.com/sigp/lighthouse/pull/2853 so that we can get ssz static tests passing here on v1.1.6. If we want to merge that first, we can make this diff slightly smaller

## Proposed Changes

- Changes the `justified_epoch` and `finalized_epoch` in the `ProtoArrayNode` each to an `Option<Checkpoint>`. The `Option` is necessary only for the migration, so not ideal. But does allow us to add a default logic to `None` on these fields during the database migration.
- Adds a database migration from a legacy fork choice struct to the new one, search for all necessary block roots in fork choice by iterating through blocks in the db.
- updates related to https://github.com/ethereum/consensus-specs/pull/2727
  -  We will have to update the persisted forkchoice to make sure the justified checkpoint stored is correct according to the updated fork choice logic. This boils down to setting the forkchoice store's justified checkpoint to the justified checkpoint of the block that advanced the finalized checkpoint to the current one. 
  - AFAICT there's no migration steps necessary for the update to allow applying attestations from prior blocks, but would appreciate confirmation on that
- I updated the consensus spec tests to v1.1.6 here, but they will fail until we also implement the proposer score boost updates. I confirmed that the previously failing scenario `new_finalized_slot_is_justified_checkpoint_ancestor` will now pass after the boost updates, but haven't confirmed _all_ tests will pass because I just quickly stubbed out the proposer boost test scenario formatting.
- This PR now also includes proposer boosting https://github.com/ethereum/consensus-specs/pull/2730

## Additional Info
I realized checking justified and finalized roots in fork choice makes it more likely that we trigger this bug: https://github.com/ethereum/consensus-specs/pull/2727

It's possible the combination of justified checkpoint and finalized checkpoint in the forkchoice store is different from in any block in fork choice. So when trying to startup our store's justified checkpoint seems invalid to the rest of fork choice (but it should be valid). When this happens we get an `InvalidBestNode` error and fail to start up. So I'm including that bugfix in this branch.

Todo:

- [x] Fix fork choice tests
- [x] Self review
- [x] Add fix for https://github.com/ethereum/consensus-specs/pull/2727
- [x] Rebase onto Kintusgi 
- [x] Fix `num_active_validators` calculation as @michaelsproul pointed out
- [x] Clean up db migrations

Co-authored-by: realbigsean <seananderson33@gmail.com>
2021-12-13 20:43:22 +00:00
..
beacon_state v1.1.1 spec updates (#2684) 2021-12-02 14:26:54 +11:00
test_utils Kintsugi review comments (#2831) 2021-12-02 14:29:59 +11:00
aggregate_and_proof.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
attestation_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
attestation_duty.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
attestation.rs Web3Signer support for VC (#2522) 2021-09-16 03:26:33 +00:00
attester_slashing.rs Update for clippy 1.50 (#2193) 2021-02-15 00:09:12 +00:00
beacon_block_body.rs Initial merge changes 2021-12-02 14:26:50 +11:00
beacon_block_header.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
beacon_block.rs Kintsugi on_merge_block tests (#2811) 2021-12-02 14:29:20 +11:00
beacon_committee.rs Directory Restructure (#1163) 2020-05-18 21:24:23 +10:00
beacon_state.rs Fix consensus, SSZ, tree hash & run merge EF tests (#2622) 2021-12-02 14:26:50 +11:00
chain_spec.rs v1.1.6 Fork Choice changes (#2822) 2021-12-13 20:43:22 +00:00
checkpoint.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
config_and_preset.rs Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
consts.rs v1.1.6 Fork Choice changes (#2822) 2021-12-13 20:43:22 +00:00
contribution_and_proof.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
deposit_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
deposit_message.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
deposit.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
enr_fork_id.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
eth1_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
eth_spec.rs Increase network limits (#2796) 2021-12-02 14:29:20 +11:00
execution_payload_header.rs Merge devnet 3 (#2859) 2021-12-12 09:04:21 +00:00
execution_payload.rs Merge devnet 3 (#2859) 2021-12-12 09:04:21 +00:00
fork_context.rs Initial merge changes 2021-12-02 14:26:50 +11:00
fork_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
fork_name.rs Kintsugi rebase patches (#2769) 2021-12-02 14:26:54 +11:00
fork.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
free_attestation.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
graffiti.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
historical_batch.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
indexed_attestation.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
lib.rs Removed PowBlock struct that never got used (#2813) 2021-12-02 14:29:20 +11:00
participation_flags.rs Quoted altair fields (2.0) (#2712) 2021-10-14 02:58:12 +00:00
participation_list.rs Rust 1.54.0 lints (#2483) 2021-07-30 01:11:47 +00:00
pending_attestation.rs Get arbitrary check passing (2.0) (#2710) 2021-10-14 02:58:11 +00:00
preset.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
proposer_slashing.rs Implement slasher (#1567) 2020-11-23 03:43:22 +00:00
relative_epoch.rs Remove saturating arith from state_processing (#1644) 2020-09-25 05:18:21 +00:00
selection_proof.rs Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
shuffling_id.rs Optimize validator duties (#2243) 2021-03-17 05:09:57 +00:00
signed_aggregate_and_proof.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
signed_beacon_block_header.rs Update for clippy 1.50 (#2193) 2021-02-15 00:09:12 +00:00
signed_beacon_block.rs Initial merge changes 2021-12-02 14:26:50 +11:00
signed_contribution_and_proof.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
signed_voluntary_exit.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
signing_data.rs Consensus updates for v0.12 (#1228) 2020-06-03 14:56:54 +10:00
slot_data.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
slot_epoch_macros.rs Updated to comply with new clippy formatting rules (#2336) 2021-05-10 00:53:09 +00:00
slot_epoch.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sqlite.rs Directory Restructure (#1163) 2020-05-18 21:24:23 +10:00
subnet_id.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_aggregate.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
sync_aggregator_selection_data.rs Web3Signer support for VC (#2522) 2021-09-16 03:26:33 +00:00
sync_committee_contribution.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_committee_message.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_committee_subscription.rs Add quoted u64/u64_vec to SyncCommitteeSubscription (#2589) 2021-09-11 23:56:16 +00:00
sync_committee.rs Validator monitor support for sync committees (#2476) 2021-08-31 23:31:36 +00:00
sync_duty.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_selection_proof.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
sync_subnet_id.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
tree_hash_impls.rs Rust 1.54.0 lints (#2483) 2021-07-30 01:11:47 +00:00
validator_subscription.rs Implement standard eth2.0 API (#1569) 2020-10-01 11:12:36 +10:00
validator.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
voluntary_exit.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00