lighthouse/beacon_node
Michael Sproul c574f8136e Fix block backfill with genesis skip slots (#4820)
## Issue Addressed

Closes #4817.

## Proposed Changes

- Fill in the linear block roots array between 0 and the slot of the first block (e.g. slots 0 and 1 on Holesky).
- Backport the `--freezer`, `--skip` and `--limit` options for `lighthouse db inspect` from tree-states. This allows us to easily view the database corruption of 4817 using `lighthouse db inspect --network holesky --freezer --column bbr --output values --limit 2`.
- Backport the `iter_column_from` change and `MemoryStore` overhaul from tree-states. These are required to enable `lighthouse db inspect`.
- Rework `freezer_upper_limit` to allow state lookups for slots below the `state_lower_limit`. Currently state lookups will fail until state reconstruction completes entirely.

There is a new regression test for the main bug, but no test for the `freezer_upper_limit` fix because we don't currently support running state reconstruction partially (see #3026). This will be fixed once we merge `tree-states`! In lieu of an automated test, I've tested manually on a Holesky node while it was reconstructing.

## Additional Info

Users who backfilled Holesky to slot 0 (e.g. using `--reconstruct-historic-states`) need to either:

- Re-sync from genesis.
- Re-sync using checkpoint sync and the changes from this PR.

Due to the recency of the Holesky genesis, writing a custom pass to fix up broken databases (which would require its own thorough testing) was deemed unnecessary. This is the primary reason for this PR being marked `backwards-incompat`.

This will create few conflicts with Deneb, which I've already resolved on `tree-states-deneb` and will be happy to backport to Deneb once this PR is merged to unstable.
2023-10-27 05:08:49 +00:00
..
beacon_chain Fix block backfill with genesis skip slots (#4820) 2023-10-27 05:08:49 +00:00
beacon_processor Merge unstable 20230925 into deneb-free-blobs. 2023-09-26 10:32:18 +10:00
builder_client Merge unstable 20230925 into deneb-free-blobs. 2023-09-26 10:32:18 +10:00
client fix typos (#4838) 2023-10-19 22:05:15 +00:00
eth1 fix typos (#4838) 2023-10-19 22:05:15 +00:00
execution_layer Upgrade to v1.4.0-beta.3 (#4862) 2023-10-21 13:49:27 +00:00
genesis Move dependencies to workspace (#4650) 2023-09-22 04:30:56 +00:00
http_api Reduce calls to network channel (#4863) 2023-10-20 19:42:47 +00:00
http_metrics Move dependencies to workspace (#4650) 2023-09-22 04:30:56 +00:00
lighthouse_network fix typos (#4838) 2023-10-19 22:05:15 +00:00
network deneb related logging improvements (#4859) 2023-10-25 15:30:17 +00:00
operation_pool Enable BLS portable feature on all CI tests (#4868) 2023-10-20 07:30:27 +00:00
src Upgrade to v1.4.0-beta.3 (#4862) 2023-10-21 13:49:27 +00:00
store Fix block backfill with genesis skip slots (#4820) 2023-10-27 05:08:49 +00:00
tests Appease Clippy 1.68 and refactor http_api (#4068) 2023-03-13 01:40:03 +00:00
timer Move dependencies to workspace (#4650) 2023-09-22 04:30:56 +00:00
Cargo.toml Merge unstable 20230925 into deneb-free-blobs. 2023-09-26 10:32:18 +10:00