a764c3b247
## Issue Addressed
NA
## Problem this PR addresses
There's an issue where Lighthouse is banning a lot of peers due to the following sequence of events:
1. Gossip block 0xabc arrives ~200ms early
- It is propagated across the network, with respect to [`MAXIMUM_GOSSIP_CLOCK_DISPARITY`](https://github.com/ethereum/eth2.0-specs/blob/v1.0.0/specs/phase0/p2p-interface.md#why-is-there-maximum_gossip_clock_disparity-when-validating-slot-ranges-of-messages-in-gossip-subnets).
- However, it is not imported to our database since the block is early.
2. Attestations for 0xabc arrive, but the block was not imported.
- The peer that sent the attestation is down-voted.
- Each unknown-block attestation causes a score loss of 1, the peer is banned at -100.
- When the peer is on an attestation subnet there can be hundreds of attestations, so the peer is banned quickly (before the missed block can be obtained via rpc).
## Potential solutions
I can think of three solutions to this:
1. Wait for attestation-queuing (#635) to arrive and solve this.
- Easy
- Not immediate fix.
- Whilst this would work, I don't think it's a perfect solution for this particular issue, rather (3) is better.
1. Allow importing blocks with a tolerance of `MAXIMUM_GOSSIP_CLOCK_DISPARITY`.
- Easy
- ~~I have implemented this, for now.~~
1. If a block is verified for gossip propagation (i.e., signature verified) and it's within `MAXIMUM_GOSSIP_CLOCK_DISPARITY`, then queue it to be processed at the start of the appropriate slot.
- More difficult
- Feels like the best solution, I will try to implement this.
**This PR takes approach (3).**
## Changes included
- Implement the `block_delay_queue`, based upon a [`DelayQueue`](https://docs.rs/tokio-util/0.6.3/tokio_util/time/delay_queue/struct.DelayQueue.html) which can store blocks until it's time to import them.
- Add a new `DelayedImportBlock` variant to the `beacon_processor::WorkEvent` enum to handle this new event.
- In the `BeaconProcessor`, refactor a `tokio::select!` to a struct with an explicit `Stream` implementation. I experienced some issues with `tokio::select!` in the block delay queue and I also found it hard to debug. I think this explicit implementation is nicer and functionally equivalent (apart from the fact that `tokio::select!` randomly chooses futures to poll, whereas now we're deterministic).
- Add a testing framework to the `beacon_processor` module that tests this new block delay logic. I also tested a handful of other operations in the beacon processor (attns, slashings, exits) since it was super easy to copy-pasta the code from the `http_api` tester.
- To implement these tests I added the concept of an optional `work_journal_tx` to the `BeaconProcessor` which will spit out a log of events. I used this in the tests to ensure that things were happening as I expect.
- The tests are a little racey, but it's hard to avoid that when testing timing-based code. If we see CI failures I can revise. I haven't observed *any* failures due to races on my machine or on CI yet.
- To assist with testing I allowed for directly setting the time on the `ManualSlotClock`.
- I gave the `beacon_processor::Worker` a `Toolbox` for two reasons; (a) it avoids changing tons of function sigs when you want to pass a new object to the worker and (b) it seemed cute.
52 lines
1.5 KiB
TOML
52 lines
1.5 KiB
TOML
[package]
|
|
name = "network"
|
|
version = "0.2.0"
|
|
authors = ["Sigma Prime <contact@sigmaprime.io>"]
|
|
edition = "2018"
|
|
|
|
[dev-dependencies]
|
|
sloggers = "1.0.1"
|
|
genesis = { path = "../genesis" }
|
|
lazy_static = "1.4.0"
|
|
matches = "0.1.8"
|
|
tempfile = "3.1.0"
|
|
exit-future = "0.2.0"
|
|
slog-term = "2.6.0"
|
|
slog-async = "2.5.0"
|
|
logging = { path = "../../common/logging" }
|
|
environment = { path = "../../lighthouse/environment" }
|
|
discv5 = { version = "0.1.0-beta.3" }
|
|
|
|
[dependencies]
|
|
beacon_chain = { path = "../beacon_chain" }
|
|
store = { path = "../store" }
|
|
eth2_libp2p = { path = "../eth2_libp2p" }
|
|
hashset_delay = { path = "../../common/hashset_delay" }
|
|
types = { path = "../../consensus/types" }
|
|
state_processing = { path = "../../consensus/state_processing" }
|
|
slot_clock = { path = "../../common/slot_clock" }
|
|
slog = { version = "2.5.2", features = ["max_level_trace"] }
|
|
hex = "0.4.2"
|
|
eth2_ssz = "0.1.2"
|
|
eth2_ssz_types = { path = "../../consensus/ssz_types" }
|
|
tree_hash = "0.1.1"
|
|
futures = "0.3.7"
|
|
error-chain = "0.12.4"
|
|
tokio = { version = "1.1.0", features = ["full"] }
|
|
tokio-stream = "0.1.3"
|
|
parking_lot = "0.11.0"
|
|
smallvec = "1.6.1"
|
|
rand = "0.7.3"
|
|
fnv = "1.0.7"
|
|
rlp = "0.5.0"
|
|
lazy_static = "1.4.0"
|
|
lighthouse_metrics = { path = "../../common/lighthouse_metrics" }
|
|
task_executor = { path = "../../common/task_executor" }
|
|
igd = "0.11.1"
|
|
itertools = "0.9.0"
|
|
num_cpus = "1.13.0"
|
|
lru_cache = { path = "../../common/lru_cache" }
|
|
if-addrs = "0.6.4"
|
|
strum = { version = "0.20"}
|
|
tokio-util = { version = "0.6.3", features = ["time"] }
|