lighthouse/beacon_node
Michael Sproul 54cf94ea59 Fix per-slot timer in presence of clock changes (#3243)
## Issue Addressed

Fixes a timing issue that results in spurious fork choice notifier failures:

```
WARN Error signalling fork choice waiter     slot: 3962270, error: ForkChoiceSignalOutOfOrder { current: Slot(3962271), latest: Slot(3962270) }, service: beacon
```

There’s a fork choice run that is scheduled to run at the start of every slot by the `timer`, which creates a 12s interval timer when the beacon node starts up. The problem is that if there’s a bit of clock drift that gets corrected via NTP (or a leap second for that matter) then these 12s intervals will cease to line up with the start of the slot. This then creates the mismatch in slot number that we see above.

Lighthouse also runs fork choice 500ms before the slot begins, and these runs are what is conflicting with the start-of-slot runs. This means that the warning in current versions of Lighthouse is mostly cosmetic because fork choice is up to date with all but the most recent 500ms of attestations (which usually isn’t many).

## Proposed Changes

Fix the per-slot timer so that it continually re-calculates the duration to the start of the next slot and waits for that.

A side-effect of this change is that we may skip slots if the per-slot task takes >12s to run, but I think this is an unlikely scenario and an acceptable compromise.
2022-06-06 23:52:32 +00:00
..
beacon_chain Avoid parallel fork choice runs during sync (#3217) 2022-05-25 03:27:30 +00:00
client Fix per-slot timer in presence of clock changes (#3243) 2022-06-06 23:52:32 +00:00
eth1 Improve eth1 block cache sync (for Ropsten) (#3234) 2022-06-03 06:05:03 +00:00
execution_layer Emit log when fee recipient values are inconsistent (#3202) 2022-06-03 03:22:54 +00:00
genesis Update and consolidate dependencies (#3136) 2022-04-04 00:26:16 +00:00
http_api Use genesis slot for node/syncing (#3226) 2022-05-31 06:09:11 +00:00
http_metrics Support IPv6 in BN and VC HTTP APIs (#3104) 2022-03-24 00:04:49 +00:00
lighthouse_network Fix: PeerManager doesn't remove "outbound only" peers which should be pruned (#3236) 2022-06-06 05:51:10 +00:00
network reduce reprocess queue/channel sizes (#3239) 2022-06-06 23:52:31 +00:00
operation_pool Remove DB migrations for legacy database schemas (#3181) 2022-05-17 04:54:39 +00:00
src Improve eth1 block cache sync (for Ropsten) (#3234) 2022-06-03 06:05:03 +00:00
store Separate execution payloads in the DB (#3157) 2022-05-12 00:42:17 +00:00
tests Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
timer Fix per-slot timer in presence of clock changes (#3243) 2022-06-06 23:52:32 +00:00
Cargo.toml v2.3.0 (#3222) 2022-05-30 01:35:10 +00:00