lighthouse/beacon_node/beacon_chain/src
Michael Sproul 6c375205fb Fix HTTP state API bug and add --epochs-per-migration (#4236)
## Issue Addressed

Fix an issue observed by `@zlan` on Discord where Lighthouse would sometimes return this error when looking up states via the API:

> {"code":500,"message":"UNHANDLED_ERROR: ForkChoiceError(MissingProtoArrayBlock(0xc9cf1495421b6ef3215d82253b388d77321176a1dcef0db0e71a0cd0ffc8cdb7))","stacktraces":[]}

## Proposed Changes

The error stems from a faulty assumption in the HTTP API logic: that any state in the hot database must have its block in fork choice. This isn't true because the state's hot database may update much less frequently than the fork choice store, e.g. if reconstructing states (where freezer migration pauses), or if the freezer migration runs slowly. There could also be a race between loading the hot state and checking fork choice, e.g. even if the finalization migration of DB+fork choice were atomic, the update could happen between the 1st and 2nd calls.

To address this I've changed the HTTP API logic to use the finalized block's execution status as a fallback where it is safe to do so. In the case where a block is non-canonical and prior to finalization (permanently orphaned) we default `execution_optimistic` to `true`.

## Additional Info

I've also added a new CLI flag to reduce the frequency of the finalization migration as this is useful for several purposes:

- Spacing out database writes (less frequent, larger batches)
- Keeping a limited chain history with high availability, e.g. the last month in the hot database.

This new flag made it _substantially_ easier to test this change. It was extracted from `tree-states` (where it's called `--db-migration-period`), which is why this PR also carries the `tree-states` label.
2023-07-17 00:14:12 +00:00
..
attestation_verification Attestation verification uses head state fork (#4263) 2023-05-15 02:10:41 +00:00
schema_change DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
attestation_rewards.rs Tweaks to reward APIs (#3957) 2023-02-10 06:19:42 +00:00
attestation_verification.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
attester_cache.rs Add early attester cache (#2872) 2022-01-11 01:35:55 +00:00
beacon_block_reward.rs Update block rewards API for Capella 2023-02-14 12:09:40 +11:00
beacon_block_streamer.rs Reconstruct Payloads using Payload Bodies Methods (#4028) 2023-03-19 23:15:59 +00:00
beacon_chain.rs Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
beacon_fork_choice_store.rs DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
beacon_proposer_cache.rs Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
beacon_snapshot.rs Capella eip 4844 cleanup (#3652) 2022-10-26 15:15:26 -04:00
block_reward.rs Capella eip 4844 cleanup (#3652) 2022-10-26 15:15:26 -04:00
block_times_cache.rs Add BlockTimesCache to allow additional block delay metrics (#2546) 2021-09-30 04:31:41 +00:00
block_verification.rs Add broadcast validation routes to Beacon Node HTTP API (#4316) 2023-06-29 12:02:38 +00:00
builder.rs Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
canonical_head.rs Enshrine head state shuffling in the shuffling_cache (#4296) 2023-05-19 05:13:05 +00:00
capella_readiness.rs Suggestions for Capella beacon_chain (#3999) 2023-02-21 11:05:36 +11:00
chain_config.rs Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
early_attester_cache.rs Fix some typos (#3376) 2022-07-27 00:51:06 +00:00
errors.rs Add broadcast validation routes to Beacon Node HTTP API (#4316) 2023-06-29 12:02:38 +00:00
eth1_chain.rs Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
eth1_finalization_cache.rs Deposit Cache Finalization & Fast WS Sync (#2915) 2022-10-30 04:04:24 +00:00
events.rs Fix order of arguments to log_count (#4060) 2023-03-13 01:40:01 +00:00
execution_payload.rs Make more noise when the EL is broken (#3986) 2023-03-17 00:44:02 +00:00
fork_choice_signal.rs Clippy 1.67 (#3916) 2023-01-27 09:48:42 +00:00
fork_revert.rs Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
head_tracker.rs Fix rust 1.65 lints (#3682) 2022-11-04 07:43:43 +00:00
historical_blocks.rs Backfill blocks only to the WSP by default (#4082) 2023-05-05 03:49:23 +00:00
lib.rs Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
light_client_finality_update_verification.rs Adding light_client gossip topics (#3693) 2022-12-13 06:24:51 +00:00
light_client_optimistic_update_verification.rs light client optimistic update reprocessing (#3799) 2023-01-24 22:17:50 +00:00
merge_readiness.rs exchangeCapabilities & Capella Readiness Logging (#3918) 2023-01-31 18:26:23 +01:00
metrics.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
migrate.rs Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
naive_aggregation_pool.rs Clippy lints for rust 1.66 (#3810) 2022-12-16 04:04:00 +00:00
observed_aggregates.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
observed_attesters.rs Add more metrics for tracking sync messages (#4308) 2023-05-19 05:13:07 +00:00
observed_block_producers.rs Add broadcast validation routes to Beacon Node HTTP API (#4316) 2023-06-29 12:02:38 +00:00
observed_operations.rs Use head state for exit verification (#4183) 2023-04-14 01:11:46 +00:00
otb_verification_service.rs Initial Commit of Retrospective OTB Verification (#3372) 2022-07-30 00:22:38 +00:00
persisted_beacon_chain.rs Fix head tracker concurrency bugs (#1771) 2020-10-19 05:58:39 +00:00
persisted_fork_choice.rs DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
pre_finalization_cache.rs Separate execution payloads in the DB (#3157) 2022-05-12 00:42:17 +00:00
proposer_prep_service.rs Enable proposer boost re-orging (#2860) 2022-12-13 09:57:26 +00:00
schema_change.rs DB migration for fork choice cleanup (#4265) 2023-05-15 02:10:42 +00:00
shuffling_cache.rs Enshrine head state shuffling in the shuffling_cache (#4296) 2023-05-19 05:13:05 +00:00
snapshot_cache.rs Enable proposer boost re-orging (#2860) 2022-12-13 09:57:26 +00:00
state_advance_timer.rs Enable proposer boost re-orging (#2860) 2022-12-13 09:57:26 +00:00
sync_committee_rewards.rs Update sync rewards API for abstract exec payload 2023-01-25 15:46:47 +11:00
sync_committee_verification.rs Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
test_utils.rs Fix HTTP state API bug and add --epochs-per-migration (#4236) 2023-07-17 00:14:12 +00:00
timeout_rw_lock.rs Add flag to disable lock timeouts (#2714) 2021-10-19 00:30:40 +00:00
validator_monitor.rs add attestation inclusion distance in http api (#4148) 2023-04-26 01:12:35 +00:00
validator_pubkey_cache.rs Fix regression in DB write atomicity (#3931) 2023-02-13 03:32:01 +00:00