Initial work towards v0.2.0 (#924)

* Remove ping protocol * Initial renaming of network services * Correct rebasing relative to latest master * Start updating types * Adds HashMapDelay struct to utils * Initial network restructure * Network restructure. Adds new types for v0.2.0 * Removes build artefacts * Shift validation to beacon chain * Temporarily remove gossip validation This is to be updated to match current optimisation efforts. * Adds AggregateAndProof * Begin rebuilding pubsub encoding/decoding * Signature hacking * Shift gossipsup decoding into eth2_libp2p * Existing EF tests passing with fake_crypto * Shifts block encoding/decoding into RPC * Delete outdated API spec * All release tests passing bar genesis state parsing * Update and test YamlConfig * Update to spec v0.10 compatible BLS * Updates to BLS EF tests * Add EF test for AggregateVerify And delete unused hash2curve tests for uncompressed points * Update EF tests to v0.10.1 * Use optional block root correctly in block proc * Use genesis fork in deposit domain. All tests pass * Fast aggregate verify test * Update REST API docs * Fix unused import * Bump spec tags to v0.10.1 * Add `seconds_per_eth1_block` to chainspec * Update to timestamp based eth1 voting scheme * Return None from `get_votes_to_consider` if block cache is empty * Handle overflows in `is_candidate_block` * Revert to failing tests * Fix eth1 data sets test * Choose default vote according to spec * Fix collect_valid_votes tests * Fix `get_votes_to_consider` to choose all eligible blocks * Uncomment winning_vote tests * Add comments; remove unused code * Reduce seconds_per_eth1_block for simulation * Addressed review comments * Add test for default vote case * Fix logs * Remove unused functions * Meter default eth1 votes * Fix comments * Progress on attestation service * Address review comments; remove unused dependency * Initial work on removing libp2p lock * Add LRU caches to store (rollup) * Update attestation validation for DB changes (WIP) * Initial version of should_forward_block * Scaffold * Progress on attestation validation Also, consolidate prod+testing slot clocks so that they share much of the same implementation and can both handle sub-slot time changes. * Removes lock from libp2p service * Completed network lock removal * Finish(?) attestation processing * Correct network termination future * Add slot check to block check * Correct fmt issues * Remove Drop implementation for network service * Add first attempt at attestation proc. re-write * Add version 2 of attestation processing * Minor fixes * Add validator pubkey cache * Make get_indexed_attestation take a committee * Link signature processing into new attn verification * First working version * Ensure pubkey cache is updated * Add more metrics, slight optimizations * Clone committee cache during attestation processing * Update shuffling cache during block processing * Remove old commented-out code * Fix shuffling cache insert bug * Used indexed attestation in fork choice * Restructure attn processing, add metrics * Add more detailed metrics * Tidy, fix failing tests * Fix failing tests, tidy * Address reviewers suggestions * Disable/delete two outdated tests * Modification of validator for subscriptions * Add slot signing to validator client * Further progress on validation subscription * Adds necessary validator subscription functionality * Add new Pubkeys struct to signature_sets * Refactor with functional approach * Update beacon chain * Clean up validator <-> beacon node http types * Add aggregator status to ValidatorDuty * Impl Clone for manual slot clock * Fix minor errors * Further progress validator client subscription * Initial subscription and aggregation handling * Remove decompressed member from pubkey bytes * Progress to modifying val client for attestation aggregation * First draft of validator client upgrade for aggregate attestations * Add hashmap for indices lookup * Add state cache, remove store cache * Only build the head committee cache * Removes lock on a network channel * Partially implement beacon node subscription http api * Correct compilation issues * Change `get_attesting_indices` to use Vec * Fix failing test * Partial implementation of timer * Adds timer, removes exit_future, http api to op pool * Partial multiple aggregate attestation handling * Permits bulk messages accross gossipsub network channel * Correct compile issues * Improve gosispsub messaging and correct rest api helpers * Added global gossipsub subscriptions * Update validator subscriptions data structs * Tidy * Re-structure validator subscriptions * Initial handling of subscriptions * Re-structure network service * Add pubkey cache persistence file * Add more comments * Integrate persistence file into builder * Add pubkey cache tests * Add HashSetDelay and introduce into attestation service * Handles validator subscriptions * Add data_dir to beacon chain builder * Remove Option in pubkey cache persistence file * Ensure consistency between datadir/data_dir * Fix failing network test * Peer subnet discovery gets queued for future subscriptions * Reorganise attestation service functions * Initial wiring of attestation service * First draft of attestation service timing logic * Correct minor typos * Tidy * Fix todos * Improve tests * Add PeerInfo to connected peers mapping * Fix compile error * Fix compile error from merge * Split up block processing metrics * Tidy * Refactor get_pubkey_from_state * Remove commented-out code * Rename state_cache -> checkpoint_cache * Rename Checkpoint -> Snapshot * Tidy, add comments * Tidy up find_head function * Change some checkpoint -> snapshot * Add tests * Expose max_len * Remove dead code * Tidy * Fix bug * Add sync-speed metric * Add first attempt at VerifiableBlock * Start integrating into beacon chain * Integrate VerifiableBlock * Rename VerifableBlock -> PartialBlockVerification * Add start of typed methods * Add progress * Add further progress * Rename structs * Add full block verification to block_processing.rs * Further beacon chain integration * Update checks for gossip * Add todo * Start adding segement verification * Add passing chain segement test * Initial integration with batch sync * Minor changes * Tidy, add more error checking * Start adding chain_segment tests * Finish invalid signature tests * Include single and gossip verified blocks in tests * Add gossip verification tests * Start adding docs * Finish adding comments to block_processing.rs * Rename block_processing.rs -> block_verification * Start removing old block processing code * Fixes beacon_chain compilation * Fix project-wide compile errors * Remove old code * Correct code to pass all tests * Fix bug with beacon proposer index * Fix shim for BlockProcessingError * Only process one epoch at a time * Fix loop in chain segment processing * Correct tests from master merge * Add caching for state.eth1_data_votes * Add BeaconChain::validator_pubkey * Revert "Add caching for state.eth1_data_votes" This reverts commit cd73dcd6434fb8d8e6bf30c5356355598ea7b78e. Co-authored-by: Grant Wuerker <gwuerker@gmail.com> Co-authored-by: Michael Sproul <michael@sigmaprime.io> Co-authored-by: Michael Sproul <micsproul@gmail.com> Co-authored-by: pawan <pawandhananjay@gmail.com> Co-authored-by: Paul Hauner <paul@paulhauner.com>
2020-03-17 17:24:44 +11:00
parent c198bddf9e
commit 95c8e476bc
161 changed files with 9771 additions and 5266 deletions
@@ -35,7 +35,7 @@

 use super::network_context::SyncNetworkContext;
 use super::range_sync::{Batch, BatchProcessResult, RangeSync};
-use crate::message_processor::PeerSyncInfo;
+use crate::router::processor::PeerSyncInfo;
 use crate::service::NetworkMessage;
 use beacon_chain::{BeaconChain, BeaconChainTypes, BlockProcessingOutcome};
 use eth2_libp2p::rpc::methods::*;
@@ -153,7 +153,7 @@ pub struct SyncManager<T: BeaconChainTypes> {
    input_channel: mpsc::UnboundedReceiver<SyncMessage<T::EthSpec>>,

    /// A network context to contact the network service.
-    network: SyncNetworkContext,
+    network: SyncNetworkContext<T::EthSpec>,

    /// The object handling long-range batch load-balanced syncing.
    range_sync: RangeSync<T>,
@@ -180,7 +180,7 @@ pub struct SyncManager<T: BeaconChainTypes> {
 pub fn spawn<T: BeaconChainTypes>(
    executor: &tokio::runtime::TaskExecutor,
    beacon_chain: Weak<BeaconChain<T>>,
-    network_send: mpsc::UnboundedSender<NetworkMessage>,
+    network_send: mpsc::UnboundedSender<NetworkMessage<T::EthSpec>>,
    log: slog::Logger,
 ) -> (
    mpsc::UnboundedSender<SyncMessage<T::EthSpec>>,
@@ -391,7 +391,7 @@ impl<T: BeaconChainTypes> SyncManager<T> {

        // we have the correct block, try and process it
        if let Some(chain) = self.chain.upgrade() {
-            match chain.process_block(block.clone()) {
+            match BlockProcessingOutcome::shim(chain.process_block(block.clone())) {
                Ok(outcome) => {
                    match outcome {
                        BlockProcessingOutcome::Processed { block_root } => {
@@ -597,7 +597,7 @@ impl<T: BeaconChainTypes> SyncManager<T> {
                    .downloaded_blocks
                    .pop()
                    .expect("There is always at least one block in the queue");
-                match chain.process_block(newest_block.clone()) {
+                match BlockProcessingOutcome::shim(chain.process_block(newest_block.clone())) {
                    Ok(BlockProcessingOutcome::ParentUnknown { .. }) => {
                        // need to keep looking for parents
                        // add the block back to the queue and continue the search
@@ -642,7 +642,7 @@ impl<T: BeaconChainTypes> SyncManager<T> {
            while let Some(block) = parent_request.downloaded_blocks.pop() {
                // check if the chain exists
                if let Some(chain) = self.chain.upgrade() {
-                    match chain.process_block(block) {
+                    match BlockProcessingOutcome::shim(chain.process_block(block)) {
                        Ok(BlockProcessingOutcome::Processed { .. })
                        | Ok(BlockProcessingOutcome::BlockIsAlreadyKnown { .. }) => {} // continue to the next block

@@ -5,9 +5,4 @@ pub mod manager;
 mod network_context;
 mod range_sync;

-/// Currently implemented sync methods.
-pub enum SyncMethod {
-    SimpleSync,
-}
-
 pub use manager::SyncMessage;
@@ -1,7 +1,7 @@
 //! Provides network functionality for the Syncing thread. This fundamentally wraps a network
 //! channel and stores a global RPC ID to perform requests.

-use crate::message_processor::status_message;
+use crate::router::processor::status_message;
 use crate::service::NetworkMessage;
 use beacon_chain::{BeaconChain, BeaconChainTypes};
 use eth2_libp2p::rpc::methods::*;
@@ -10,20 +10,21 @@ use eth2_libp2p::PeerId;
 use slog::{debug, trace, warn};
 use std::sync::Weak;
 use tokio::sync::mpsc;
+use types::EthSpec;

 /// Wraps a Network channel to employ various RPC related network functionality for the Sync manager. This includes management of a global RPC request Id.

-pub struct SyncNetworkContext {
+pub struct SyncNetworkContext<T: EthSpec> {
    /// The network channel to relay messages to the Network service.
-    network_send: mpsc::UnboundedSender<NetworkMessage>,
+    network_send: mpsc::UnboundedSender<NetworkMessage<T>>,

    request_id: RequestId,
    /// Logger for the `SyncNetworkContext`.
    log: slog::Logger,
 }

-impl SyncNetworkContext {
-    pub fn new(network_send: mpsc::UnboundedSender<NetworkMessage>, log: slog::Logger) -> Self {
+impl<T: EthSpec> SyncNetworkContext<T> {
+    pub fn new(network_send: mpsc::UnboundedSender<NetworkMessage<T>>, log: slog::Logger) -> Self {
        Self {
            network_send,
            request_id: 0,
@@ -31,9 +32,9 @@ impl SyncNetworkContext {
        }
    }

-    pub fn status_peer<T: BeaconChainTypes>(
+    pub fn status_peer<U: BeaconChainTypes>(
        &mut self,
-        chain: Weak<BeaconChain<T>>,
+        chain: Weak<BeaconChain<U>>,
        peer_id: PeerId,
    ) {
        if let Some(chain) = chain.upgrade() {
@@ -117,7 +118,7 @@ impl SyncNetworkContext {
    pub fn send_rpc_request(
        &mut self,
        peer_id: PeerId,
-        rpc_request: RPCRequest,
+        rpc_request: RPCRequest<T>,
    ) -> Result<RequestId, &'static str> {
        let request_id = self.request_id;
        self.request_id += 1;
@@ -125,7 +126,11 @@ impl SyncNetworkContext {
        Ok(request_id)
    }

-    fn send_rpc_event(&mut self, peer_id: PeerId, rpc_event: RPCEvent) -> Result<(), &'static str> {
+    fn send_rpc_event(
+        &mut self,
+        peer_id: PeerId,
+        rpc_event: RPCEvent<T>,
+    ) -> Result<(), &'static str> {
        self.network_send
            .try_send(NetworkMessage::RPC(peer_id, rpc_event))
            .map_err(|_| {
@@ -1,7 +1,7 @@
 use super::batch::Batch;
-use crate::message_processor::FUTURE_SLOT_TOLERANCE;
+use crate::router::processor::FUTURE_SLOT_TOLERANCE;
 use crate::sync::manager::SyncMessage;
-use beacon_chain::{BeaconChain, BeaconChainTypes, BlockProcessingOutcome};
+use beacon_chain::{BeaconChain, BeaconChainTypes, BlockError};
 use slog::{debug, error, trace, warn};
 use std::sync::{Arc, Weak};
 use tokio::sync::mpsc;
@@ -54,116 +54,78 @@ fn process_batch<T: BeaconChainTypes>(
    batch: &Batch<T::EthSpec>,
    log: &slog::Logger,
 ) -> Result<(), String> {
-    let mut successful_block_import = false;
-    for block in &batch.downloaded_blocks {
-        if let Some(chain) = chain.upgrade() {
-            let processing_result = chain.process_block(block.clone());
-
-            if let Ok(outcome) = processing_result {
-                match outcome {
-                    BlockProcessingOutcome::Processed { block_root } => {
-                        // The block was valid and we processed it successfully.
-                        trace!(
-                            log, "Imported block from network";
-                            "slot" => block.slot(),
-                            "block_root" => format!("{}", block_root),
-                        );
-                        successful_block_import = true;
-                    }
-                    BlockProcessingOutcome::ParentUnknown { parent, .. } => {
-                        // blocks should be sequential and all parents should exist
-                        warn!(
-                            log, "Parent block is unknown";
-                            "parent_root" => format!("{}", parent),
-                            "baby_block_slot" => block.slot(),
-                        );
-                        if successful_block_import {
-                            run_fork_choice(chain, log);
-                        }
-                        return Err(format!(
-                            "Block at slot {} has an unknown parent.",
-                            block.slot()
-                        ));
-                    }
-                    BlockProcessingOutcome::BlockIsAlreadyKnown => {
-                        // this block is already known to us, move to the next
-                        debug!(
-                            log, "Imported a block that is already known";
-                            "block_slot" => block.slot(),
-                        );
-                    }
-                    BlockProcessingOutcome::FutureSlot {
-                        present_slot,
-                        block_slot,
-                    } => {
-                        if present_slot + FUTURE_SLOT_TOLERANCE >= block_slot {
-                            // The block is too far in the future, drop it.
-                            warn!(
-                                log, "Block is ahead of our slot clock";
-                                "msg" => "block for future slot rejected, check your time",
-                                "present_slot" => present_slot,
-                                "block_slot" => block_slot,
-                                "FUTURE_SLOT_TOLERANCE" => FUTURE_SLOT_TOLERANCE,
-                            );
-                            if successful_block_import {
-                                run_fork_choice(chain, log);
-                            }
-                            return Err(format!(
-                                "Block at slot {} is too far in the future",
-                                block.slot()
-                            ));
-                        } else {
-                            // The block is in the future, but not too far.
-                            debug!(
-                                log, "Block is slightly ahead of our slot clock, ignoring.";
-                                "present_slot" => present_slot,
-                                "block_slot" => block_slot,
-                                "FUTURE_SLOT_TOLERANCE" => FUTURE_SLOT_TOLERANCE,
-                            );
-                        }
-                    }
-                    BlockProcessingOutcome::WouldRevertFinalizedSlot { .. } => {
-                        debug!(
-                            log, "Finalized or earlier block processed";
-                            "outcome" => format!("{:?}", outcome),
-                        );
-                        // block reached our finalized slot or was earlier, move to the next block
-                    }
-                    BlockProcessingOutcome::GenesisBlock => {
-                        debug!(
-                            log, "Genesis block was processed";
-                            "outcome" => format!("{:?}", outcome),
-                        );
-                    }
-                    _ => {
-                        warn!(
-                            log, "Invalid block received";
-                            "msg" => "peer sent invalid block",
-                            "outcome" => format!("{:?}", outcome),
-                        );
-                        if successful_block_import {
-                            run_fork_choice(chain, log);
-                        }
-                        return Err(format!("Invalid block at slot {}", block.slot()));
-                    }
+    if let Some(chain) = chain.upgrade() {
+        match chain.process_chain_segment(batch.downloaded_blocks.clone()) {
+            Ok(roots) => {
+                trace!(
+                    log, "Imported blocks from network";
+                    "count" => roots.len(),
+                );
+            }
+            Err(BlockError::ParentUnknown(parent)) => {
+                // blocks should be sequential and all parents should exist
+                warn!(
+                    log, "Parent block is unknown";
+                    "parent_root" => format!("{}", parent),
+                );
+            }
+            Err(BlockError::BlockIsAlreadyKnown) => {
+                // this block is already known to us, move to the next
+                debug!(
+                    log, "Imported a block that is already known";
+                );
+            }
+            Err(BlockError::FutureSlot {
+                present_slot,
+                block_slot,
+            }) => {
+                if present_slot + FUTURE_SLOT_TOLERANCE >= block_slot {
+                    // The block is too far in the future, drop it.
+                    warn!(
+                        log, "Block is ahead of our slot clock";
+                        "msg" => "block for future slot rejected, check your time",
+                        "present_slot" => present_slot,
+                        "block_slot" => block_slot,
+                        "FUTURE_SLOT_TOLERANCE" => FUTURE_SLOT_TOLERANCE,
+                    );
+                } else {
+                    // The block is in the future, but not too far.
+                    debug!(
+                        log, "Block is slightly ahead of our slot clock, ignoring.";
+                        "present_slot" => present_slot,
+                        "block_slot" => block_slot,
+                        "FUTURE_SLOT_TOLERANCE" => FUTURE_SLOT_TOLERANCE,
+                    );
                }
-            } else {
+            }
+            Err(BlockError::WouldRevertFinalizedSlot { .. }) => {
+                debug!(
+                    log, "Finalized or earlier block processed";
+                );
+                // block reached our finalized slot or was earlier, move to the next block
+            }
+            Err(BlockError::GenesisBlock) => {
+                debug!(
+                    log, "Genesis block was processed";
+                );
+            }
+            Err(BlockError::BeaconChainError(e)) => {
                warn!(
                    log, "BlockProcessingFailure";
                    "msg" => "unexpected condition in processing block.",
-                    "outcome" => format!("{:?}", processing_result)
+                    "outcome" => format!("{:?}", e)
+                );
+            }
+            other => {
+                warn!(
+                    log, "Invalid block received";
+                    "msg" => "peer sent invalid block",
+                    "outcome" => format!("{:?}", other),
                );
-                if successful_block_import {
-                    run_fork_choice(chain, log);
-                }
-                return Err(format!(
-                    "Unexpected block processing error: {:?}",
-                    processing_result
-                ));
            }
-        } else {
-            return Ok(()); // terminate early due to dropped beacon chain
        }
+    } else {
+        return Ok(()); // terminate early due to dropped beacon chain
    }

    // Batch completed successfully, run fork choice.
@@ -17,7 +17,7 @@ use types::{Hash256, SignedBeaconBlock, Slot};
 /// downvote peers with poor bandwidth. This can be set arbitrarily high, in which case the
 /// responder will fill the response up to the max request size, assuming they have the bandwidth
 /// to do so.
-pub const BLOCKS_PER_BATCH: u64 = 50;
+pub const BLOCKS_PER_BATCH: u64 = 64;

 /// The number of times to retry a batch before the chain is considered failed and removed.
 const MAX_BATCH_RETRIES: u8 = 5;
@@ -141,7 +141,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    /// batch.
    pub fn on_block_response(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        request_id: RequestId,
        beacon_block: &Option<SignedBeaconBlock<T::EthSpec>>,
    ) -> Option<()> {
@@ -161,7 +161,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    /// failed indicating that further batches are required.
    fn handle_completed_batch(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        batch: Batch<T::EthSpec>,
    ) {
        // An entire batch of blocks has been received. This functions checks to see if it can be processed,
@@ -255,7 +255,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    /// of the batch processor.
    pub fn on_batch_process_result(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        processing_id: u64,
        batch: &mut Option<Batch<T::EthSpec>>,
        result: &BatchProcessResult,
@@ -385,7 +385,11 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    // TODO: Batches could have been partially downloaded due to RPC size-limit restrictions. We
    // need to add logic for partial batch downloads. Potentially, if another peer returns the same
    // batch, we try a partial download.
-    fn handle_invalid_batch(&mut self, network: &mut SyncNetworkContext, batch: Batch<T::EthSpec>) {
+    fn handle_invalid_batch(
+        &mut self,
+        network: &mut SyncNetworkContext<T::EthSpec>,
+        batch: Batch<T::EthSpec>,
+    ) {
        // The current batch could not be processed, indicating either the current or previous
        // batches are invalid

@@ -415,7 +419,11 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    ///
    /// If the re-downloaded batch is different to the original and can be processed, the original
    /// peer will be downvoted.
-    fn reprocess_batch(&mut self, network: &mut SyncNetworkContext, mut batch: Batch<T::EthSpec>) {
+    fn reprocess_batch(
+        &mut self,
+        network: &mut SyncNetworkContext<T::EthSpec>,
+        mut batch: Batch<T::EthSpec>,
+    ) {
        // marks the batch as attempting to be reprocessed by hashing the downloaded blocks
        batch.original_hash = Some(batch.hash());

@@ -455,7 +463,11 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    /// This chain has been requested to start syncing.
    ///
    /// This could be new chain, or an old chain that is being resumed.
-    pub fn start_syncing(&mut self, network: &mut SyncNetworkContext, local_finalized_slot: Slot) {
+    pub fn start_syncing(
+        &mut self,
+        network: &mut SyncNetworkContext<T::EthSpec>,
+        local_finalized_slot: Slot,
+    ) {
        // A local finalized slot is provided as other chains may have made
        // progress whilst this chain was Stopped or paused. If so, update the `processed_batch_id` to
        // accommodate potentially downloaded batches from other chains. Also prune any old batches
@@ -490,7 +502,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    /// Add a peer to the chain.
    ///
    /// If the chain is active, this starts requesting batches from this peer.
-    pub fn add_peer(&mut self, network: &mut SyncNetworkContext, peer_id: PeerId) {
+    pub fn add_peer(&mut self, network: &mut SyncNetworkContext<T::EthSpec>, peer_id: PeerId) {
        self.peer_pool.insert(peer_id.clone());
        // do not request blocks if the chain is not syncing
        if let ChainSyncingState::Stopped = self.state {
@@ -503,7 +515,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    }

    /// Sends a STATUS message to all peers in the peer pool.
-    pub fn status_peers(&self, network: &mut SyncNetworkContext) {
+    pub fn status_peers(&self, network: &mut SyncNetworkContext<T::EthSpec>) {
        for peer_id in self.peer_pool.iter() {
            network.status_peer(self.chain.clone(), peer_id.clone());
        }
@@ -517,7 +529,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    /// this chain.
    pub fn inject_error(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        peer_id: &PeerId,
        request_id: RequestId,
    ) -> Option<ProcessingResult> {
@@ -541,7 +553,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    /// `MAX_BATCH_RETRIES`.
    pub fn failed_batch(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        mut batch: Batch<T::EthSpec>,
    ) -> ProcessingResult {
        batch.retries += 1;
@@ -575,7 +587,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {

    /// Attempts to request the next required batches from the peer pool if the chain is syncing. It will exhaust the peer
    /// pool and left over batches until the batch buffer is reached or all peers are exhausted.
-    fn request_batches(&mut self, network: &mut SyncNetworkContext) {
+    fn request_batches(&mut self, network: &mut SyncNetworkContext<T::EthSpec>) {
        if let ChainSyncingState::Syncing = self.state {
            while self.send_range_request(network) {}
        }
@@ -583,7 +595,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {

    /// Requests the next required batch from a peer. Returns true, if there was a peer available
    /// to send a request and there are batches to request, false otherwise.
-    fn send_range_request(&mut self, network: &mut SyncNetworkContext) -> bool {
+    fn send_range_request(&mut self, network: &mut SyncNetworkContext<T::EthSpec>) -> bool {
        // find the next pending batch and request it from the peer
        if let Some(peer_id) = self.get_next_peer() {
            if let Some(batch) = self.get_next_batch(peer_id) {
@@ -669,7 +681,11 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
    }

    /// Requests the provided batch from the provided peer.
-    fn send_batch(&mut self, network: &mut SyncNetworkContext, batch: Batch<T::EthSpec>) {
+    fn send_batch(
+        &mut self,
+        network: &mut SyncNetworkContext<T::EthSpec>,
+        batch: Batch<T::EthSpec>,
+    ) {
        let request = batch.to_blocks_by_range_request();
        if let Ok(request_id) = network.blocks_by_range_request(batch.current_peer.clone(), request)
        {
@@ -4,7 +4,7 @@
 //! with this struct to to simplify the logic of the other layers of sync.

 use super::chain::{ChainSyncingState, SyncingChain};
-use crate::message_processor::PeerSyncInfo;
+use crate::router::processor::PeerSyncInfo;
 use crate::sync::manager::SyncMessage;
 use crate::sync::network_context::SyncNetworkContext;
 use beacon_chain::{BeaconChain, BeaconChainTypes};
@@ -103,7 +103,11 @@ impl<T: BeaconChainTypes> ChainCollection<T> {
    ///
    /// This removes any out-dated chains, swaps to any higher priority finalized chains and
    /// updates the state of the collection.
-    pub fn update_finalized(&mut self, network: &mut SyncNetworkContext, log: &slog::Logger) {
+    pub fn update_finalized(
+        &mut self,
+        network: &mut SyncNetworkContext<T::EthSpec>,
+        log: &slog::Logger,
+    ) {
        let local_slot = match self.beacon_chain.upgrade() {
            Some(chain) => {
                let local = match PeerSyncInfo::from_chain(&chain) {
@@ -197,7 +201,7 @@ impl<T: BeaconChainTypes> ChainCollection<T> {
    #[allow(clippy::too_many_arguments)]
    pub fn new_head_chain(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        remote_finalized_slot: Slot,
        target_head: Hash256,
        target_slot: Slot,
@@ -277,7 +281,11 @@ impl<T: BeaconChainTypes> ChainCollection<T> {
    ///
    /// This removes chains with no peers, or chains whose start block slot is less than our current
    /// finalized block slot.
-    pub fn purge_outdated_chains(&mut self, network: &mut SyncNetworkContext, log: &slog::Logger) {
+    pub fn purge_outdated_chains(
+        &mut self,
+        network: &mut SyncNetworkContext<T::EthSpec>,
+        log: &slog::Logger,
+    ) {
        // Remove any chains that have no peers
        self.finalized_chains
            .retain(|chain| !chain.peer_pool.is_empty());
@@ -349,7 +357,7 @@ impl<T: BeaconChainTypes> ChainCollection<T> {
    /// This will re-status the chains peers on removal. The index must exist.
    pub fn remove_chain(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        index: usize,
        log: &slog::Logger,
    ) {
@@ -42,7 +42,7 @@
 use super::chain::ProcessingResult;
 use super::chain_collection::{ChainCollection, SyncState};
 use super::{Batch, BatchProcessResult};
-use crate::message_processor::PeerSyncInfo;
+use crate::router::processor::PeerSyncInfo;
 use crate::sync::manager::SyncMessage;
 use crate::sync::network_context::SyncNetworkContext;
 use beacon_chain::{BeaconChain, BeaconChainTypes};
@@ -108,7 +108,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {
    /// prioritised by peer-pool size.
    pub fn add_peer(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        peer_id: PeerId,
        remote: PeerSyncInfo,
    ) {
@@ -228,7 +228,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {
    /// This request could complete a chain or simply add to its progress.
    pub fn blocks_by_range_response(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        peer_id: PeerId,
        request_id: RequestId,
        beacon_block: Option<SignedBeaconBlock<T::EthSpec>>,
@@ -255,7 +255,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {

    pub fn handle_block_process_result(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        processing_id: u64,
        batch: Batch<T::EthSpec>,
        result: BatchProcessResult,
@@ -326,7 +326,11 @@ impl<T: BeaconChainTypes> RangeSync<T> {

    /// A peer has disconnected. This removes the peer from any ongoing chains and mappings. A
    /// disconnected peer could remove a chain
-    pub fn peer_disconnect(&mut self, network: &mut SyncNetworkContext, peer_id: &PeerId) {
+    pub fn peer_disconnect(
+        &mut self,
+        network: &mut SyncNetworkContext<T::EthSpec>,
+        peer_id: &PeerId,
+    ) {
        // if the peer is in the awaiting head mapping, remove it
        self.awaiting_head_peers.remove(&peer_id);

@@ -340,7 +344,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {
    /// When a peer gets removed, both the head and finalized chains need to be searched to check which pool the peer is in. The chain may also have a batch or batches awaiting
    /// for this peer. If so we mark the batch as failed. The batch may then hit it's maximum
    /// retries. In this case, we need to remove the chain and re-status all the peers.
-    fn remove_peer(&mut self, network: &mut SyncNetworkContext, peer_id: &PeerId) {
+    fn remove_peer(&mut self, network: &mut SyncNetworkContext<T::EthSpec>, peer_id: &PeerId) {
        if let Some((index, ProcessingResult::RemoveChain)) =
            self.chains.head_finalized_request(|chain| {
                if chain.peer_pool.remove(peer_id) {
@@ -370,7 +374,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {
    /// been too many failed attempts for the batch, remove the chain.
    pub fn inject_error(
        &mut self,
-        network: &mut SyncNetworkContext,
+        network: &mut SyncNetworkContext<T::EthSpec>,
        peer_id: PeerId,
        request_id: RequestId,
    ) {