lighthouse/beacon_node
Michael Sproul 6c375205fb Fix HTTP state API bug and add --epochs-per-migration (#4236)
## Issue Addressed

Fix an issue observed by `@zlan` on Discord where Lighthouse would sometimes return this error when looking up states via the API:

> {"code":500,"message":"UNHANDLED_ERROR: ForkChoiceError(MissingProtoArrayBlock(0xc9cf1495421b6ef3215d82253b388d77321176a1dcef0db0e71a0cd0ffc8cdb7))","stacktraces":[]}

## Proposed Changes

The error stems from a faulty assumption in the HTTP API logic: that any state in the hot database must have its block in fork choice. This isn't true because the state's hot database may update much less frequently than the fork choice store, e.g. if reconstructing states (where freezer migration pauses), or if the freezer migration runs slowly. There could also be a race between loading the hot state and checking fork choice, e.g. even if the finalization migration of DB+fork choice were atomic, the update could happen between the 1st and 2nd calls.

To address this I've changed the HTTP API logic to use the finalized block's execution status as a fallback where it is safe to do so. In the case where a block is non-canonical and prior to finalization (permanently orphaned) we default `execution_optimistic` to `true`.

## Additional Info

I've also added a new CLI flag to reduce the frequency of the finalization migration as this is useful for several purposes:

- Spacing out database writes (less frequent, larger batches)
- Keeping a limited chain history with high availability, e.g. the last month in the hot database.

This new flag made it _substantially_ easier to test this change. It was extracted from `tree-states` (where it's called `--db-migration-period`), which is why this PR also carries the `tree-states` label.
2023-07-17 00:14:12 +00:00
..
beacon_chain Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
beacon_processor Move the BeaconProcessor into a new crate (#4435) 2023-07-10 07:45:54 +00:00
builder_client Tidy formatting of Reqwest errors (#4336) 2023-06-27 01:06:50 +00:00
client Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
eth1 Replace ganache-cli with anvil (#3555) 2023-05-15 07:22:02 +00:00
execution_layer Remove hidden re-exports to appease Rust 1.73 (#4495) 2023-07-12 07:06:00 +00:00
genesis Replace ganache-cli with anvil (#3555) 2023-05-15 07:22:02 +00:00
http_api Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
http_metrics Appease clippy in Rust 1.70 (#4365) 2023-06-02 03:17:40 +00:00
lighthouse_network Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
network Move the BeaconProcessor into a new crate (#4435) 2023-07-10 07:45:54 +00:00
operation_pool Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
src Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
store Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
tests Appease Clippy 1.68 and refactor http_api (#4068) 2023-03-13 01:40:03 +00:00
timer Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
Cargo.toml Move the BeaconProcessor into a new crate (#4435) 2023-07-10 07:45:54 +00:00