lighthouse/consensus
ethDreamer e8604757a2 Deposit Cache Finalization & Fast WS Sync (#2915)
## Summary

The deposit cache now has the ability to finalize deposits. This will cause it to drop unneeded deposit logs and hashes in the deposit Merkle tree that are no longer required to construct deposit proofs. The cache is finalized whenever the latest finalized checkpoint has a new `Eth1Data` with all deposits imported.

This has three benefits:

1. Improves the speed of constructing Merkle proofs for deposits as we can just replay deposits since the last finalized checkpoint instead of all historical deposits when re-constructing the Merkle tree.
2. Significantly faster weak subjectivity sync as the deposit cache can be transferred to the newly syncing node in compressed form. The Merkle tree that stores `N` finalized deposits requires a maximum of `log2(N)` hashes. The newly syncing node then only needs to download deposits since the last finalized checkpoint to have a full tree.
3. Future proofing in preparation for [EIP-4444](https://eips.ethereum.org/EIPS/eip-4444) as execution nodes will no longer be required to store logs permanently so we won't always have all historical logs available to us.

## More Details

Image to illustrate how the deposit contract merkle tree evolves and finalizes along with the resulting `DepositTreeSnapshot`
![image](https://user-images.githubusercontent.com/37123614/151465302-5fc56284-8a69-4998-b20e-45db3934ac70.png)

## Other Considerations

I've changed the structure of the `SszDepositCache` so once you load & save your database from this version of lighthouse, you will no longer be able to load it from older versions.

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>
2022-10-30 04:04:24 +00:00
..
cached_tree_hash Update and consolidate dependencies (#3136) 2022-04-04 00:26:16 +00:00
fork_choice Strict count unrealized (#3522) 2022-09-05 04:50:47 +00:00
int_to_bytes Update to Rust 1.59 and 2021 edition (#3038) 2022-02-25 00:10:17 +00:00
merkle_proof Deposit Cache Finalization & Fast WS Sync (#2915) 2022-10-30 04:04:24 +00:00
proto_array Strict count unrealized (#3522) 2022-09-05 04:50:47 +00:00
safe_arith Inline safe_arith methods (#3229) 2022-05-31 06:09:12 +00:00
serde_utils Add a u256_hex_be module to encode/decode U256 types (#3321) 2022-07-15 07:31:21 +00:00
ssz Deposit Cache Finalization & Fast WS Sync (#2915) 2022-10-30 04:04:24 +00:00
ssz_derive Update to Rust 1.59 and 2021 edition (#3038) 2022-02-25 00:10:17 +00:00
ssz_types New rust lints for rustc 1.64.0 (#3602) 2022-09-23 03:52:46 +00:00
state_processing Deposit Cache Finalization & Fast WS Sync (#2915) 2022-10-30 04:04:24 +00:00
swap_or_not_shuffle Update and consolidate dependencies (#3136) 2022-04-04 00:26:16 +00:00
tree_hash Use SmallVec for TreeHash packed encoding (#3581) 2022-09-16 08:54:06 +00:00
tree_hash_derive Use SmallVec for TreeHash packed encoding (#3581) 2022-09-16 08:54:06 +00:00
types Deposit Cache Finalization & Fast WS Sync (#2915) 2022-10-30 04:04:24 +00:00