be4e261e74
## Overview
This rather extensive PR achieves two primary goals:
1. Uses the finalized/justified checkpoints of fork choice (FC), rather than that of the head state.
2. Refactors fork choice, block production and block processing to `async` functions.
Additionally, it achieves:
- Concurrent forkchoice updates to the EL and cache pruning after a new head is selected.
- Concurrent "block packing" (attestations, etc) and execution payload retrieval during block production.
- Concurrent per-block-processing and execution payload verification during block processing.
- The `Arc`-ification of `SignedBeaconBlock` during block processing (it's never mutated, so why not?):
- I had to do this to deal with sending blocks into spawned tasks.
- Previously we were cloning the beacon block at least 2 times during each block processing, these clones are either removed or turned into cheaper `Arc` clones.
- We were also `Box`-ing and un-`Box`-ing beacon blocks as they moved throughout the networking crate. This is not a big deal, but it's nice to avoid shifting things between the stack and heap.
- Avoids cloning *all the blocks* in *every chain segment* during sync.
- It also has the potential to clean up our code where we need to pass an *owned* block around so we can send it back in the case of an error (I didn't do much of this, my PR is already big enough 😅)
- The `BeaconChain::HeadSafetyStatus` struct was removed. It was an old relic from prior merge specs.
For motivation for this change, see https://github.com/sigp/lighthouse/pull/3244#issuecomment-1160963273
## Changes to `canonical_head` and `fork_choice`
Previously, the `BeaconChain` had two separate fields:
```
canonical_head: RwLock<Snapshot>,
fork_choice: RwLock<BeaconForkChoice>
```
Now, we have grouped these values under a single struct:
```
canonical_head: CanonicalHead {
cached_head: RwLock<Arc<Snapshot>>,
fork_choice: RwLock<BeaconForkChoice>
}
```
Apart from ergonomics, the only *actual* change here is wrapping the canonical head snapshot in an `Arc`. This means that we no longer need to hold the `cached_head` (`canonical_head`, in old terms) lock when we want to pull some values from it. This was done to avoid deadlock risks by preventing functions from acquiring (and holding) the `cached_head` and `fork_choice` locks simultaneously.
## Breaking Changes
### The `state` (root) field in the `finalized_checkpoint` SSE event
Consider the scenario where epoch `n` is just finalized, but `start_slot(n)` is skipped. There are two state roots we might in the `finalized_checkpoint` SSE event:
1. The state root of the finalized block, which is `get_block(finalized_checkpoint.root).state_root`.
4. The state root at slot of `start_slot(n)`, which would be the state from (1), but "skipped forward" through any skip slots.
Previously, Lighthouse would choose (2). However, we can see that when [Teku generates that event](
|
||
|---|---|---|
| .. | ||
| src | ||
| .gitignore | ||
| Cargo.toml | ||
| Makefile | ||
| README.md | ||
state_transition_vectors
This crate contains test vectors for Lighthouse state transition functions.
This crate serves two purposes:
- Outputting the test vectors to disk via
make. - Running the vectors against our code via
make test.
Outputting vectors to disk
Whilst we don't actually need to write the vectors to disk to test them, we provide this functionality so we can generate corpra for the fuzzer and also so they can be of use to other clients.
To create the files in ./vectors (directory relative to this crate), run:
make
This will produce a directory structure that looks roughly like this:
vectors
└── exit
├── invalid_bad_signature
│ ├── block.ssz
│ ├── error.txt
│ └── pre.ssz
├── invalid_duplicate
│ ├── block.ssz
│ ├── error.txt
│ └── pre.ssz
├── invalid_exit_already_initiated
│ ├── block.ssz
│ ├── error.txt
│ └── pre.ssz
├── invalid_future_exit_epoch
│ ├── block.ssz
│ ├── error.txt
│ └── pre.ssz
├── invalid_not_active_after_exit_epoch
│ ├── block.ssz
│ ├── error.txt
│ └── pre.ssz
├── invalid_not_active_before_activation_epoch
│ ├── block.ssz
│ ├── error.txt
│ └── pre.ssz
├── invalid_too_young_by_a_lot
│ ├── block.ssz
│ ├── error.txt
│ └── pre.ssz
├── invalid_too_young_by_one_epoch
│ ├── block.ssz
│ ├── error.txt
│ └── pre.ssz
├── invalid_validator_unknown
│ ├── block.ssz
│ ├── error.txt
│ └── pre.ssz
├── valid_genesis_epoch
│ ├── block.ssz
│ ├── post.ssz
│ └── pre.ssz
└── valid_previous_epoch
├── block.ssz
├── post.ssz
└── pre.ssz