Initial work towards v0.2.0 (#924)
* Remove ping protocol * Initial renaming of network services * Correct rebasing relative to latest master * Start updating types * Adds HashMapDelay struct to utils * Initial network restructure * Network restructure. Adds new types for v0.2.0 * Removes build artefacts * Shift validation to beacon chain * Temporarily remove gossip validation This is to be updated to match current optimisation efforts. * Adds AggregateAndProof * Begin rebuilding pubsub encoding/decoding * Signature hacking * Shift gossipsup decoding into eth2_libp2p * Existing EF tests passing with fake_crypto * Shifts block encoding/decoding into RPC * Delete outdated API spec * All release tests passing bar genesis state parsing * Update and test YamlConfig * Update to spec v0.10 compatible BLS * Updates to BLS EF tests * Add EF test for AggregateVerify And delete unused hash2curve tests for uncompressed points * Update EF tests to v0.10.1 * Use optional block root correctly in block proc * Use genesis fork in deposit domain. All tests pass * Fast aggregate verify test * Update REST API docs * Fix unused import * Bump spec tags to v0.10.1 * Add `seconds_per_eth1_block` to chainspec * Update to timestamp based eth1 voting scheme * Return None from `get_votes_to_consider` if block cache is empty * Handle overflows in `is_candidate_block` * Revert to failing tests * Fix eth1 data sets test * Choose default vote according to spec * Fix collect_valid_votes tests * Fix `get_votes_to_consider` to choose all eligible blocks * Uncomment winning_vote tests * Add comments; remove unused code * Reduce seconds_per_eth1_block for simulation * Addressed review comments * Add test for default vote case * Fix logs * Remove unused functions * Meter default eth1 votes * Fix comments * Progress on attestation service * Address review comments; remove unused dependency * Initial work on removing libp2p lock * Add LRU caches to store (rollup) * Update attestation validation for DB changes (WIP) * Initial version of should_forward_block * Scaffold * Progress on attestation validation Also, consolidate prod+testing slot clocks so that they share much of the same implementation and can both handle sub-slot time changes. * Removes lock from libp2p service * Completed network lock removal * Finish(?) attestation processing * Correct network termination future * Add slot check to block check * Correct fmt issues * Remove Drop implementation for network service * Add first attempt at attestation proc. re-write * Add version 2 of attestation processing * Minor fixes * Add validator pubkey cache * Make get_indexed_attestation take a committee * Link signature processing into new attn verification * First working version * Ensure pubkey cache is updated * Add more metrics, slight optimizations * Clone committee cache during attestation processing * Update shuffling cache during block processing * Remove old commented-out code * Fix shuffling cache insert bug * Used indexed attestation in fork choice * Restructure attn processing, add metrics * Add more detailed metrics * Tidy, fix failing tests * Fix failing tests, tidy * Address reviewers suggestions * Disable/delete two outdated tests * Modification of validator for subscriptions * Add slot signing to validator client * Further progress on validation subscription * Adds necessary validator subscription functionality * Add new Pubkeys struct to signature_sets * Refactor with functional approach * Update beacon chain * Clean up validator <-> beacon node http types * Add aggregator status to ValidatorDuty * Impl Clone for manual slot clock * Fix minor errors * Further progress validator client subscription * Initial subscription and aggregation handling * Remove decompressed member from pubkey bytes * Progress to modifying val client for attestation aggregation * First draft of validator client upgrade for aggregate attestations * Add hashmap for indices lookup * Add state cache, remove store cache * Only build the head committee cache * Removes lock on a network channel * Partially implement beacon node subscription http api * Correct compilation issues * Change `get_attesting_indices` to use Vec * Fix failing test * Partial implementation of timer * Adds timer, removes exit_future, http api to op pool * Partial multiple aggregate attestation handling * Permits bulk messages accross gossipsub network channel * Correct compile issues * Improve gosispsub messaging and correct rest api helpers * Added global gossipsub subscriptions * Update validator subscriptions data structs * Tidy * Re-structure validator subscriptions * Initial handling of subscriptions * Re-structure network service * Add pubkey cache persistence file * Add more comments * Integrate persistence file into builder * Add pubkey cache tests * Add HashSetDelay and introduce into attestation service * Handles validator subscriptions * Add data_dir to beacon chain builder * Remove Option in pubkey cache persistence file * Ensure consistency between datadir/data_dir * Fix failing network test * Peer subnet discovery gets queued for future subscriptions * Reorganise attestation service functions * Initial wiring of attestation service * First draft of attestation service timing logic * Correct minor typos * Tidy * Fix todos * Improve tests * Add PeerInfo to connected peers mapping * Fix compile error * Fix compile error from merge * Split up block processing metrics * Tidy * Refactor get_pubkey_from_state * Remove commented-out code * Rename state_cache -> checkpoint_cache * Rename Checkpoint -> Snapshot * Tidy, add comments * Tidy up find_head function * Change some checkpoint -> snapshot * Add tests * Expose max_len * Remove dead code * Tidy * Fix bug * Add sync-speed metric * Add first attempt at VerifiableBlock * Start integrating into beacon chain * Integrate VerifiableBlock * Rename VerifableBlock -> PartialBlockVerification * Add start of typed methods * Add progress * Add further progress * Rename structs * Add full block verification to block_processing.rs * Further beacon chain integration * Update checks for gossip * Add todo * Start adding segement verification * Add passing chain segement test * Initial integration with batch sync * Minor changes * Tidy, add more error checking * Start adding chain_segment tests * Finish invalid signature tests * Include single and gossip verified blocks in tests * Add gossip verification tests * Start adding docs * Finish adding comments to block_processing.rs * Rename block_processing.rs -> block_verification * Start removing old block processing code * Fixes beacon_chain compilation * Fix project-wide compile errors * Remove old code * Correct code to pass all tests * Fix bug with beacon proposer index * Fix shim for BlockProcessingError * Only process one epoch at a time * Fix loop in chain segment processing * Correct tests from master merge * Add caching for state.eth1_data_votes * Add BeaconChain::validator_pubkey * Revert "Add caching for state.eth1_data_votes" This reverts commit cd73dcd6434fb8d8e6bf30c5356355598ea7b78e. Co-authored-by: Grant Wuerker <gwuerker@gmail.com> Co-authored-by: Michael Sproul <michael@sigmaprime.io> Co-authored-by: Michael Sproul <micsproul@gmail.com> Co-authored-by: pawan <pawandhananjay@gmail.com> Co-authored-by: Paul Hauner <paul@paulhauner.com>
This commit is contained in:
Age Manning
GitHub
parent
c198bddf9e
commit
95c8e476bc
5179
Cargo.lock
generated
5179
Cargo.lock
generated
File diff suppressed because it is too large
Load Diff
17
Cargo.toml
17
Cargo.toml
@ -13,12 +13,14 @@ members = [
|
||||
"eth2/utils/eth2_testnet_config",
|
||||
"eth2/utils/logging",
|
||||
"eth2/utils/eth2_hashing",
|
||||
"eth2/utils/hashmap_delay",
|
||||
"eth2/utils/lighthouse_metrics",
|
||||
"eth2/utils/lighthouse_bootstrap",
|
||||
"eth2/utils/merkle_proof",
|
||||
"eth2/utils/int_to_bytes",
|
||||
"eth2/utils/serde_hex",
|
||||
"eth2/utils/slot_clock",
|
||||
"eth2/utils/rest_types",
|
||||
"eth2/utils/ssz",
|
||||
"eth2/utils/ssz_derive",
|
||||
"eth2/utils/ssz_types",
|
||||
@ -28,14 +30,15 @@ members = [
|
||||
"eth2/utils/tree_hash_derive",
|
||||
"eth2/utils/test_random_derive",
|
||||
"beacon_node",
|
||||
"beacon_node/store",
|
||||
"beacon_node/client",
|
||||
"beacon_node/rest_api",
|
||||
"beacon_node/network",
|
||||
"beacon_node/eth2-libp2p",
|
||||
"beacon_node/version",
|
||||
"beacon_node/eth1",
|
||||
"beacon_node/beacon_chain",
|
||||
"beacon_node/client",
|
||||
"beacon_node/eth1",
|
||||
"beacon_node/eth2-libp2p",
|
||||
"beacon_node/network",
|
||||
"beacon_node/rest_api",
|
||||
"beacon_node/store",
|
||||
"beacon_node/timer",
|
||||
"beacon_node/version",
|
||||
"beacon_node/websocket_server",
|
||||
"tests/simulator",
|
||||
"tests/ef_tests",
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "beacon_node"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Paul Hauner <paul@paulhauner.com>", "Age Manning <Age@AgeManning.com"]
|
||||
edition = "2018"
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "beacon_chain"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Paul Hauner <paul@paulhauner.com>", "Age Manning <Age@AgeManning.com>"]
|
||||
edition = "2018"
|
||||
|
||||
@ -33,10 +33,10 @@ eth2_ssz_derive = "0.1.0"
|
||||
state_processing = { path = "../../eth2/state_processing" }
|
||||
tree_hash = "0.1.0"
|
||||
types = { path = "../../eth2/types" }
|
||||
tokio = "0.1.22"
|
||||
eth1 = { path = "../eth1" }
|
||||
websocket_server = { path = "../websocket_server" }
|
||||
futures = "0.1.25"
|
||||
exit-future = "0.1.3"
|
||||
genesis = { path = "../genesis" }
|
||||
integer-sqrt = "0.1"
|
||||
rand = "0.7.2"
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -5,14 +5,14 @@ use types::{BeaconState, EthSpec, Hash256, SignedBeaconBlock};
|
||||
/// Represents some block and its associated state. Generally, this will be used for tracking the
|
||||
/// head, justified head and finalized head.
|
||||
#[derive(Clone, Serialize, PartialEq, Debug, Encode, Decode)]
|
||||
pub struct CheckPoint<E: EthSpec> {
|
||||
pub struct BeaconSnapshot<E: EthSpec> {
|
||||
pub beacon_block: SignedBeaconBlock<E>,
|
||||
pub beacon_block_root: Hash256,
|
||||
pub beacon_state: BeaconState<E>,
|
||||
pub beacon_state_root: Hash256,
|
||||
}
|
||||
|
||||
impl<E: EthSpec> CheckPoint<E> {
|
||||
impl<E: EthSpec> BeaconSnapshot<E> {
|
||||
/// Create a new checkpoint.
|
||||
pub fn new(
|
||||
beacon_block: SignedBeaconBlock<E>,
|
||||
801
beacon_node/beacon_chain/src/block_verification.rs
Normal file
801
beacon_node/beacon_chain/src/block_verification.rs
Normal file
@ -0,0 +1,801 @@
|
||||
//! Provides `SignedBeaconBlock` verification logic.
|
||||
//!
|
||||
//! Specifically, it provides the following:
|
||||
//!
|
||||
//! - Verification for gossip blocks (i.e., should we gossip some block from the network).
|
||||
//! - Verification for normal blocks (e.g., some block received on the RPC during a parent lookup).
|
||||
//! - Verification for chain segments (e.g., some chain of blocks received on the RPC during a
|
||||
//! sync).
|
||||
//!
|
||||
//! The primary source of complexity here is that we wish to avoid doing duplicate work as a block
|
||||
//! moves through the verification process. For example, if some block is verified for gossip, we
|
||||
//! do not wish to re-verify the block proposal signature or re-hash the block. Or, if we've
|
||||
//! verified the signatures of a block during a chain segment import, we do not wish to verify each
|
||||
//! signature individually again.
|
||||
//!
|
||||
//! The incremental processing steps (e.g., signatures verified but not the state transition) is
|
||||
//! represented as a sequence of wrapper-types around the block. There is a linear progression of
|
||||
//! types, starting at a `SignedBeaconBlock` and finishing with a `Fully VerifiedBlock` (see
|
||||
//! diagram below).
|
||||
//!
|
||||
//! ```ignore
|
||||
//! START
|
||||
//! |
|
||||
//! ▼
|
||||
//! SignedBeaconBlock
|
||||
//! |---------------
|
||||
//! | |
|
||||
//! | ▼
|
||||
//! | GossipVerifiedBlock
|
||||
//! | |
|
||||
//! |---------------
|
||||
//! |
|
||||
//! ▼
|
||||
//! SignatureVerifiedBlock
|
||||
//! |
|
||||
//! ▼
|
||||
//! FullyVerifiedBlock
|
||||
//! |
|
||||
//! ▼
|
||||
//! END
|
||||
//!
|
||||
//! ```
|
||||
use crate::validator_pubkey_cache::ValidatorPubkeyCache;
|
||||
use crate::{
|
||||
beacon_chain::{BLOCK_PROCESSING_CACHE_LOCK_TIMEOUT, VALIDATOR_PUBKEY_CACHE_LOCK_TIMEOUT},
|
||||
metrics, BeaconChain, BeaconChainError, BeaconChainTypes, BeaconSnapshot,
|
||||
};
|
||||
use parking_lot::RwLockReadGuard;
|
||||
use state_processing::{
|
||||
block_signature_verifier::{
|
||||
BlockSignatureVerifier, Error as BlockSignatureVerifierError, G1Point,
|
||||
},
|
||||
per_block_processing, per_slot_processing, BlockProcessingError, BlockSignatureStrategy,
|
||||
SlotProcessingError,
|
||||
};
|
||||
use std::borrow::Cow;
|
||||
use store::{Error as DBError, StateBatch};
|
||||
use types::{
|
||||
BeaconBlock, BeaconState, BeaconStateError, ChainSpec, CloneConfig, EthSpec, Hash256,
|
||||
RelativeEpoch, SignedBeaconBlock, Slot,
|
||||
};
|
||||
|
||||
mod block_processing_outcome;
|
||||
|
||||
pub use block_processing_outcome::BlockProcessingOutcome;
|
||||
|
||||
/// Maximum block slot number. Block with slots bigger than this constant will NOT be processed.
|
||||
const MAXIMUM_BLOCK_SLOT_NUMBER: u64 = 4_294_967_296; // 2^32
|
||||
|
||||
/// Returned when a block was not verified. A block is not verified for two reasons:
|
||||
///
|
||||
/// - The block is malformed/invalid (indicated by all results other than `BeaconChainError`.
|
||||
/// - We encountered an error whilst trying to verify the block (a `BeaconChainError`).
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum BlockError {
|
||||
/// The parent block was unknown.
|
||||
ParentUnknown(Hash256),
|
||||
/// The block slot is greater than the present slot.
|
||||
FutureSlot {
|
||||
present_slot: Slot,
|
||||
block_slot: Slot,
|
||||
},
|
||||
/// The block state_root does not match the generated state.
|
||||
StateRootMismatch { block: Hash256, local: Hash256 },
|
||||
/// The block was a genesis block, these blocks cannot be re-imported.
|
||||
GenesisBlock,
|
||||
/// The slot is finalized, no need to import.
|
||||
WouldRevertFinalizedSlot {
|
||||
block_slot: Slot,
|
||||
finalized_slot: Slot,
|
||||
},
|
||||
/// Block is already known, no need to re-import.
|
||||
BlockIsAlreadyKnown,
|
||||
/// The block slot exceeds the MAXIMUM_BLOCK_SLOT_NUMBER.
|
||||
BlockSlotLimitReached,
|
||||
/// The proposal signature in invalid.
|
||||
ProposalSignatureInvalid,
|
||||
/// A signature in the block is invalid (exactly which is unknown).
|
||||
InvalidSignature,
|
||||
/// The provided block is from an earlier slot than its parent.
|
||||
BlockIsNotLaterThanParent { block_slot: Slot, state_slot: Slot },
|
||||
/// At least one block in the chain segement did not have it's parent root set to the root of
|
||||
/// the prior block.
|
||||
NonLinearParentRoots,
|
||||
/// The slots of the blocks in the chain segment were not strictly increasing. I.e., a child
|
||||
/// had lower slot than a parent.
|
||||
NonLinearSlots,
|
||||
/// The block failed the specification's `per_block_processing` function, it is invalid.
|
||||
PerBlockProcessingError(BlockProcessingError),
|
||||
/// There was an error whilst processing the block. It is not necessarily invalid.
|
||||
BeaconChainError(BeaconChainError),
|
||||
}
|
||||
|
||||
impl From<BlockSignatureVerifierError> for BlockError {
|
||||
fn from(e: BlockSignatureVerifierError) -> Self {
|
||||
BlockError::BeaconChainError(BeaconChainError::BlockSignatureVerifierError(e))
|
||||
}
|
||||
}
|
||||
|
||||
impl From<BeaconChainError> for BlockError {
|
||||
fn from(e: BeaconChainError) -> Self {
|
||||
BlockError::BeaconChainError(e)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<BeaconStateError> for BlockError {
|
||||
fn from(e: BeaconStateError) -> Self {
|
||||
BlockError::BeaconChainError(BeaconChainError::BeaconStateError(e))
|
||||
}
|
||||
}
|
||||
|
||||
impl From<SlotProcessingError> for BlockError {
|
||||
fn from(e: SlotProcessingError) -> Self {
|
||||
BlockError::BeaconChainError(BeaconChainError::SlotProcessingError(e))
|
||||
}
|
||||
}
|
||||
|
||||
impl From<DBError> for BlockError {
|
||||
fn from(e: DBError) -> Self {
|
||||
BlockError::BeaconChainError(BeaconChainError::DBError(e))
|
||||
}
|
||||
}
|
||||
|
||||
/// Verify all signatures (except deposit signatures) on all blocks in the `chain_segment`. If all
|
||||
/// signatures are valid, the `chain_segment` is mapped to a `Vec<SignatureVerifiedBlock>` that can
|
||||
/// later be transformed into a `FullyVerifiedBlock` without re-checking the signatures. If any
|
||||
/// signature in the block is invalid, an `Err` is returned (it is not possible to known _which_
|
||||
/// signature was invalid).
|
||||
///
|
||||
/// ## Errors
|
||||
///
|
||||
/// The given `chain_segement` must span no more than two epochs, otherwise an error will be
|
||||
/// returned.
|
||||
pub fn signature_verify_chain_segment<T: BeaconChainTypes>(
|
||||
chain_segment: Vec<(Hash256, SignedBeaconBlock<T::EthSpec>)>,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<Vec<SignatureVerifiedBlock<T>>, BlockError> {
|
||||
let (mut parent, slot) = if let Some(block) = chain_segment.first().map(|(_, block)| block) {
|
||||
let parent = load_parent(&block.message, chain)?;
|
||||
(parent, block.slot())
|
||||
} else {
|
||||
return Ok(vec![]);
|
||||
};
|
||||
|
||||
let highest_slot = chain_segment
|
||||
.last()
|
||||
.map(|(_, block)| block.slot())
|
||||
.unwrap_or_else(|| slot);
|
||||
|
||||
let state = cheap_state_advance_to_obtain_committees(
|
||||
&mut parent.beacon_state,
|
||||
highest_slot,
|
||||
&chain.spec,
|
||||
)?;
|
||||
|
||||
let pubkey_cache = get_validator_pubkey_cache(chain)?;
|
||||
let mut signature_verifier = get_signature_verifier(&state, &pubkey_cache, &chain.spec);
|
||||
|
||||
for (block_root, block) in &chain_segment {
|
||||
signature_verifier.include_all_signatures(block, Some(*block_root))?;
|
||||
}
|
||||
|
||||
if signature_verifier.verify().is_err() {
|
||||
return Err(BlockError::InvalidSignature);
|
||||
}
|
||||
|
||||
drop(pubkey_cache);
|
||||
|
||||
let mut signature_verified_blocks = chain_segment
|
||||
.into_iter()
|
||||
.map(|(block_root, block)| SignatureVerifiedBlock {
|
||||
block,
|
||||
block_root,
|
||||
parent: None,
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
if let Some(signature_verified_block) = signature_verified_blocks.first_mut() {
|
||||
signature_verified_block.parent = Some(parent);
|
||||
}
|
||||
|
||||
Ok(signature_verified_blocks)
|
||||
}
|
||||
|
||||
/// A wrapper around a `SignedBeaconBlock` that indicates it has been approved for re-gossiping on
|
||||
/// the p2p network.
|
||||
pub struct GossipVerifiedBlock<T: BeaconChainTypes> {
|
||||
block: SignedBeaconBlock<T::EthSpec>,
|
||||
block_root: Hash256,
|
||||
parent: BeaconSnapshot<T::EthSpec>,
|
||||
}
|
||||
|
||||
/// A wrapper around a `SignedBeaconBlock` that indicates that all signatures (except the deposit
|
||||
/// signatures) have been verified.
|
||||
pub struct SignatureVerifiedBlock<T: BeaconChainTypes> {
|
||||
block: SignedBeaconBlock<T::EthSpec>,
|
||||
block_root: Hash256,
|
||||
parent: Option<BeaconSnapshot<T::EthSpec>>,
|
||||
}
|
||||
|
||||
/// A wrapper around a `SignedBeaconBlock` that indicates that this block is fully verified and
|
||||
/// ready to import into the `BeaconChain`. The validation includes:
|
||||
///
|
||||
/// - Parent is known
|
||||
/// - Signatures
|
||||
/// - State root check
|
||||
/// - Per block processing
|
||||
///
|
||||
/// Note: a `FullyVerifiedBlock` is not _forever_ valid to be imported, it may later become invalid
|
||||
/// due to finality or some other event. A `FullyVerifiedBlock` should be imported into the
|
||||
/// `BeaconChain` immediately after it is instantiated.
|
||||
pub struct FullyVerifiedBlock<T: BeaconChainTypes> {
|
||||
pub block: SignedBeaconBlock<T::EthSpec>,
|
||||
pub block_root: Hash256,
|
||||
pub state: BeaconState<T::EthSpec>,
|
||||
pub parent_block: SignedBeaconBlock<T::EthSpec>,
|
||||
pub intermediate_states: StateBatch<T::EthSpec>,
|
||||
}
|
||||
|
||||
/// Implemented on types that can be converted into a `FullyVerifiedBlock`.
|
||||
///
|
||||
/// Used to allow functions to accept blocks at various stages of verification.
|
||||
pub trait IntoFullyVerifiedBlock<T: BeaconChainTypes> {
|
||||
fn into_fully_verified_block(
|
||||
self,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<FullyVerifiedBlock<T>, BlockError>;
|
||||
|
||||
fn block(&self) -> &SignedBeaconBlock<T::EthSpec>;
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> GossipVerifiedBlock<T> {
|
||||
/// Instantiates `Self`, a wrapper that indicates the given `block` is safe to be re-gossiped
|
||||
/// on the p2p network.
|
||||
///
|
||||
/// Returns an error if the block is invalid, or if the block was unable to be verified.
|
||||
pub fn new(
|
||||
block: SignedBeaconBlock<T::EthSpec>,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<Self, BlockError> {
|
||||
// Do not gossip or process blocks from future slots.
|
||||
//
|
||||
// TODO: adjust this to allow for clock disparity tolerance.
|
||||
let present_slot = chain.slot()?;
|
||||
if block.slot() > present_slot {
|
||||
return Err(BlockError::FutureSlot {
|
||||
present_slot,
|
||||
block_slot: block.slot(),
|
||||
});
|
||||
}
|
||||
|
||||
// Do not gossip a block from a finalized slot.
|
||||
//
|
||||
// TODO: adjust this to allow for clock disparity tolerance.
|
||||
check_block_against_finalized_slot(&block.message, chain)?;
|
||||
|
||||
// TODO: add check for the `(block.proposer_index, block.slot)` tuple once we have v0.11.0
|
||||
|
||||
let mut parent = load_parent(&block.message, chain)?;
|
||||
let block_root = get_block_root(&block);
|
||||
|
||||
let state = cheap_state_advance_to_obtain_committees(
|
||||
&mut parent.beacon_state,
|
||||
block.slot(),
|
||||
&chain.spec,
|
||||
)?;
|
||||
|
||||
let pubkey_cache = get_validator_pubkey_cache(chain)?;
|
||||
|
||||
let mut signature_verifier = get_signature_verifier(&state, &pubkey_cache, &chain.spec);
|
||||
|
||||
signature_verifier.include_block_proposal(&block, Some(block_root))?;
|
||||
|
||||
if signature_verifier.verify().is_ok() {
|
||||
Ok(Self {
|
||||
block,
|
||||
block_root,
|
||||
parent,
|
||||
})
|
||||
} else {
|
||||
Err(BlockError::ProposalSignatureInvalid)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> IntoFullyVerifiedBlock<T> for GossipVerifiedBlock<T> {
|
||||
/// Completes verification of the wrapped `block`.
|
||||
fn into_fully_verified_block(
|
||||
self,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<FullyVerifiedBlock<T>, BlockError> {
|
||||
let fully_verified = SignatureVerifiedBlock::from_gossip_verified_block(self, chain)?;
|
||||
fully_verified.into_fully_verified_block(chain)
|
||||
}
|
||||
|
||||
fn block(&self) -> &SignedBeaconBlock<T::EthSpec> {
|
||||
&self.block
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> SignatureVerifiedBlock<T> {
|
||||
/// Instantiates `Self`, a wrapper that indicates that all signatures (except the deposit
|
||||
/// signatures) are valid (i.e., signed by the correct public keys).
|
||||
///
|
||||
/// Returns an error if the block is invalid, or if the block was unable to be verified.
|
||||
pub fn new(
|
||||
block: SignedBeaconBlock<T::EthSpec>,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<Self, BlockError> {
|
||||
let mut parent = load_parent(&block.message, chain)?;
|
||||
let block_root = get_block_root(&block);
|
||||
|
||||
let state = cheap_state_advance_to_obtain_committees(
|
||||
&mut parent.beacon_state,
|
||||
block.slot(),
|
||||
&chain.spec,
|
||||
)?;
|
||||
|
||||
let pubkey_cache = get_validator_pubkey_cache(chain)?;
|
||||
|
||||
let mut signature_verifier = get_signature_verifier(&state, &pubkey_cache, &chain.spec);
|
||||
|
||||
signature_verifier.include_all_signatures(&block, Some(block_root))?;
|
||||
|
||||
if signature_verifier.verify().is_ok() {
|
||||
Ok(Self {
|
||||
block,
|
||||
block_root,
|
||||
parent: Some(parent),
|
||||
})
|
||||
} else {
|
||||
Err(BlockError::InvalidSignature)
|
||||
}
|
||||
}
|
||||
|
||||
/// Finishes signature verification on the provided `GossipVerifedBlock`. Does not re-verify
|
||||
/// the proposer signature.
|
||||
pub fn from_gossip_verified_block(
|
||||
from: GossipVerifiedBlock<T>,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<Self, BlockError> {
|
||||
let mut parent = from.parent;
|
||||
let block = from.block;
|
||||
|
||||
let state = cheap_state_advance_to_obtain_committees(
|
||||
&mut parent.beacon_state,
|
||||
block.slot(),
|
||||
&chain.spec,
|
||||
)?;
|
||||
|
||||
let pubkey_cache = get_validator_pubkey_cache(chain)?;
|
||||
|
||||
let mut signature_verifier = get_signature_verifier(&state, &pubkey_cache, &chain.spec);
|
||||
|
||||
signature_verifier.include_all_signatures_except_proposal(&block)?;
|
||||
|
||||
if signature_verifier.verify().is_ok() {
|
||||
Ok(Self {
|
||||
block,
|
||||
block_root: from.block_root,
|
||||
parent: Some(parent),
|
||||
})
|
||||
} else {
|
||||
Err(BlockError::InvalidSignature)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> IntoFullyVerifiedBlock<T> for SignatureVerifiedBlock<T> {
|
||||
/// Completes verification of the wrapped `block`.
|
||||
fn into_fully_verified_block(
|
||||
self,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<FullyVerifiedBlock<T>, BlockError> {
|
||||
let block = self.block;
|
||||
let parent = self
|
||||
.parent
|
||||
.map(Result::Ok)
|
||||
.unwrap_or_else(|| load_parent(&block.message, chain))?;
|
||||
|
||||
FullyVerifiedBlock::from_signature_verified_components(
|
||||
block,
|
||||
self.block_root,
|
||||
parent,
|
||||
chain,
|
||||
)
|
||||
}
|
||||
|
||||
fn block(&self) -> &SignedBeaconBlock<T::EthSpec> {
|
||||
&self.block
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> IntoFullyVerifiedBlock<T> for SignedBeaconBlock<T::EthSpec> {
|
||||
/// Verifies the `SignedBeaconBlock` by first transforming it into a `SignatureVerifiedBlock`
|
||||
/// and then using that implementation of `IntoFullyVerifiedBlock` to complete verification.
|
||||
fn into_fully_verified_block(
|
||||
self,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<FullyVerifiedBlock<T>, BlockError> {
|
||||
SignatureVerifiedBlock::new(self, chain)?.into_fully_verified_block(chain)
|
||||
}
|
||||
|
||||
fn block(&self) -> &SignedBeaconBlock<T::EthSpec> {
|
||||
&self
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> FullyVerifiedBlock<T> {
|
||||
/// Instantiates `Self`, a wrapper that indicates that the given `block` is fully valid. See
|
||||
/// the struct-level documentation for more information.
|
||||
///
|
||||
/// Note: this function does not verify block signatures, it assumes they are valid. Signature
|
||||
/// verification must be done upstream (e.g., via a `SignatureVerifiedBlock`
|
||||
///
|
||||
/// Returns an error if the block is invalid, or if the block was unable to be verified.
|
||||
pub fn from_signature_verified_components(
|
||||
block: SignedBeaconBlock<T::EthSpec>,
|
||||
block_root: Hash256,
|
||||
parent: BeaconSnapshot<T::EthSpec>,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<Self, BlockError> {
|
||||
// Reject any block if its parent is not known to fork choice.
|
||||
//
|
||||
// A block that is not in fork choice is either:
|
||||
//
|
||||
// - Not yet imported: we should reject this block because we should only import a child
|
||||
// after its parent has been fully imported.
|
||||
// - Pre-finalized: if the parent block is _prior_ to finalization, we should ignore it
|
||||
// because it will revert finalization. Note that the finalized block is stored in fork
|
||||
// choice, so we will not reject any child of the finalized block (this is relevant during
|
||||
// genesis).
|
||||
if !chain.fork_choice.contains_block(&block.parent_root()) {
|
||||
return Err(BlockError::ParentUnknown(block.parent_root()));
|
||||
}
|
||||
|
||||
/*
|
||||
* Perform cursory checks to see if the block is even worth processing.
|
||||
*/
|
||||
|
||||
check_block_relevancy(&block, Some(block_root), chain)?;
|
||||
|
||||
/*
|
||||
* Advance the given `parent.beacon_state` to the slot of the given `block`.
|
||||
*/
|
||||
|
||||
let catchup_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_CATCHUP_STATE);
|
||||
|
||||
// Keep a batch of any states that were "skipped" (block-less) in between the parent state
|
||||
// slot and the block slot. These will be stored in the database.
|
||||
let mut intermediate_states = StateBatch::new();
|
||||
|
||||
// The block must have a higher slot than its parent.
|
||||
if block.slot() <= parent.beacon_state.slot {
|
||||
return Err(BlockError::BlockIsNotLaterThanParent {
|
||||
block_slot: block.slot(),
|
||||
state_slot: parent.beacon_state.slot,
|
||||
});
|
||||
}
|
||||
|
||||
// Transition the parent state to the block slot.
|
||||
let mut state = parent.beacon_state;
|
||||
let distance = block.slot().as_u64().saturating_sub(state.slot.as_u64());
|
||||
for i in 0..distance {
|
||||
let state_root = if i == 0 {
|
||||
parent.beacon_block.state_root()
|
||||
} else {
|
||||
// This is a new state we've reached, so stage it for storage in the DB.
|
||||
// Computing the state root here is time-equivalent to computing it during slot
|
||||
// processing, but we get early access to it.
|
||||
let state_root = state.update_tree_hash_cache()?;
|
||||
intermediate_states.add_state(state_root, &state)?;
|
||||
state_root
|
||||
};
|
||||
|
||||
per_slot_processing(&mut state, Some(state_root), &chain.spec)?;
|
||||
}
|
||||
|
||||
metrics::stop_timer(catchup_timer);
|
||||
|
||||
/*
|
||||
* Build the committee caches on the state.
|
||||
*/
|
||||
|
||||
let committee_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_COMMITTEE);
|
||||
|
||||
state.build_committee_cache(RelativeEpoch::Previous, &chain.spec)?;
|
||||
state.build_committee_cache(RelativeEpoch::Current, &chain.spec)?;
|
||||
|
||||
metrics::stop_timer(committee_timer);
|
||||
|
||||
/*
|
||||
* Perform `per_block_processing` on the block and state, returning early if the block is
|
||||
* invalid.
|
||||
*/
|
||||
|
||||
let core_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_CORE);
|
||||
|
||||
if let Err(err) = per_block_processing(
|
||||
&mut state,
|
||||
&block,
|
||||
Some(block_root),
|
||||
// Signatures were verified earlier in this function.
|
||||
BlockSignatureStrategy::NoVerification,
|
||||
&chain.spec,
|
||||
) {
|
||||
match err {
|
||||
// Capture `BeaconStateError` so that we can easily distinguish between a block
|
||||
// that's invalid and one that caused an internal error.
|
||||
BlockProcessingError::BeaconStateError(e) => return Err(e.into()),
|
||||
other => return Err(BlockError::PerBlockProcessingError(other)),
|
||||
}
|
||||
};
|
||||
|
||||
metrics::stop_timer(core_timer);
|
||||
|
||||
/*
|
||||
* Calculate the state root of the newly modified state
|
||||
*/
|
||||
|
||||
let state_root_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_STATE_ROOT);
|
||||
|
||||
let state_root = state.update_tree_hash_cache()?;
|
||||
|
||||
metrics::stop_timer(state_root_timer);
|
||||
|
||||
/*
|
||||
* Check to ensure the state root on the block matches the one we have calculated.
|
||||
*/
|
||||
|
||||
if block.state_root() != state_root {
|
||||
return Err(BlockError::StateRootMismatch {
|
||||
block: block.state_root(),
|
||||
local: state_root,
|
||||
});
|
||||
}
|
||||
|
||||
Ok(Self {
|
||||
block,
|
||||
block_root,
|
||||
state,
|
||||
parent_block: parent.beacon_block,
|
||||
intermediate_states,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns `Ok(())` if the block is later than the finalized slot on `chain`.
|
||||
///
|
||||
/// Returns an error if the block is earlier or equal to the finalized slot, or there was an error
|
||||
/// verifying that condition.
|
||||
fn check_block_against_finalized_slot<T: BeaconChainTypes>(
|
||||
block: &BeaconBlock<T::EthSpec>,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<(), BlockError> {
|
||||
let finalized_slot = chain
|
||||
.head_info()?
|
||||
.finalized_checkpoint
|
||||
.epoch
|
||||
.start_slot(T::EthSpec::slots_per_epoch());
|
||||
|
||||
if block.slot <= finalized_slot {
|
||||
Err(BlockError::WouldRevertFinalizedSlot {
|
||||
block_slot: block.slot,
|
||||
finalized_slot,
|
||||
})
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Performs simple, cheap checks to ensure that the block is relevant to imported.
|
||||
///
|
||||
/// `Ok(block_root)` is returned if the block passes these checks and should progress with
|
||||
/// verification (viz., it is relevant).
|
||||
///
|
||||
/// Returns an error if the block fails one of these checks (viz., is not relevant) or an error is
|
||||
/// experienced whilst attempting to verify.
|
||||
pub fn check_block_relevancy<T: BeaconChainTypes>(
|
||||
signed_block: &SignedBeaconBlock<T::EthSpec>,
|
||||
block_root: Option<Hash256>,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<Hash256, BlockError> {
|
||||
let block = &signed_block.message;
|
||||
|
||||
// Do not process blocks from the future.
|
||||
if block.slot > chain.slot()? {
|
||||
return Err(BlockError::FutureSlot {
|
||||
present_slot: chain.slot()?,
|
||||
block_slot: block.slot,
|
||||
});
|
||||
}
|
||||
|
||||
// Do not re-process the genesis block.
|
||||
if block.slot == 0 {
|
||||
return Err(BlockError::GenesisBlock);
|
||||
}
|
||||
|
||||
// This is an artificial (non-spec) restriction that provides some protection from overflow
|
||||
// abuses.
|
||||
if block.slot >= MAXIMUM_BLOCK_SLOT_NUMBER {
|
||||
return Err(BlockError::BlockSlotLimitReached);
|
||||
}
|
||||
|
||||
// Do not process a block from a finalized slot.
|
||||
check_block_against_finalized_slot(block, chain)?;
|
||||
|
||||
let block_root = block_root.unwrap_or_else(|| get_block_root(&signed_block));
|
||||
|
||||
// Check if the block is already known. We know it is post-finalization, so it is
|
||||
// sufficient to check the fork choice.
|
||||
if chain.fork_choice.contains_block(&block_root) {
|
||||
return Err(BlockError::BlockIsAlreadyKnown);
|
||||
}
|
||||
|
||||
Ok(block_root)
|
||||
}
|
||||
|
||||
/// Returns the canonical root of the given `block`.
|
||||
///
|
||||
/// Use this function to ensure that we report the block hashing time Prometheus metric.
|
||||
pub fn get_block_root<E: EthSpec>(block: &SignedBeaconBlock<E>) -> Hash256 {
|
||||
let block_root_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_BLOCK_ROOT);
|
||||
|
||||
let block_root = block.canonical_root();
|
||||
|
||||
metrics::stop_timer(block_root_timer);
|
||||
|
||||
block_root
|
||||
}
|
||||
|
||||
/// Load the parent snapshot (block and state) of the given `block`.
|
||||
///
|
||||
/// Returns `Err(BlockError::ParentUnknown)` if the parent is not found, or if an error occurs
|
||||
/// whilst attempting the operation.
|
||||
fn load_parent<T: BeaconChainTypes>(
|
||||
block: &BeaconBlock<T::EthSpec>,
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<BeaconSnapshot<T::EthSpec>, BlockError> {
|
||||
let db_read_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_DB_READ);
|
||||
|
||||
// Reject any block if its parent is not known to fork choice.
|
||||
//
|
||||
// A block that is not in fork choice is either:
|
||||
//
|
||||
// - Not yet imported: we should reject this block because we should only import a child
|
||||
// after its parent has been fully imported.
|
||||
// - Pre-finalized: if the parent block is _prior_ to finalization, we should ignore it
|
||||
// because it will revert finalization. Note that the finalized block is stored in fork
|
||||
// choice, so we will not reject any child of the finalized block (this is relevant during
|
||||
// genesis).
|
||||
if !chain.fork_choice.contains_block(&block.parent_root) {
|
||||
return Err(BlockError::ParentUnknown(block.parent_root));
|
||||
}
|
||||
|
||||
// Load the parent block and state from disk, returning early if it's not available.
|
||||
let result = chain
|
||||
.snapshot_cache
|
||||
.try_write_for(BLOCK_PROCESSING_CACHE_LOCK_TIMEOUT)
|
||||
.and_then(|mut snapshot_cache| snapshot_cache.try_remove(block.parent_root))
|
||||
.map(|snapshot| Ok(Some(snapshot)))
|
||||
.unwrap_or_else(|| {
|
||||
// Load the blocks parent block from the database, returning invalid if that block is not
|
||||
// found.
|
||||
//
|
||||
// We don't return a DBInconsistent error here since it's possible for a block to
|
||||
// exist in fork choice but not in the database yet. In such a case we simply
|
||||
// indicate that we don't yet know the parent.
|
||||
let parent_block = if let Some(block) = chain.get_block(&block.parent_root)? {
|
||||
block
|
||||
} else {
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
// Load the parent blocks state from the database, returning an error if it is not found.
|
||||
// It is an error because if we know the parent block we should also know the parent state.
|
||||
let parent_state_root = parent_block.state_root();
|
||||
let parent_state = chain
|
||||
.get_state(&parent_state_root, Some(parent_block.slot()))?
|
||||
.ok_or_else(|| {
|
||||
BeaconChainError::DBInconsistent(format!(
|
||||
"Missing state {:?}",
|
||||
parent_state_root
|
||||
))
|
||||
})?;
|
||||
|
||||
Ok(Some(BeaconSnapshot {
|
||||
beacon_block: parent_block,
|
||||
beacon_block_root: block.parent_root,
|
||||
beacon_state: parent_state,
|
||||
beacon_state_root: parent_state_root,
|
||||
}))
|
||||
})
|
||||
.map_err(BlockError::BeaconChainError)?
|
||||
.ok_or_else(|| BlockError::ParentUnknown(block.parent_root));
|
||||
|
||||
metrics::stop_timer(db_read_timer);
|
||||
|
||||
result
|
||||
}
|
||||
|
||||
/// Performs a cheap (time-efficient) state advancement so the committees for `slot` can be
|
||||
/// obtained from `state`.
|
||||
///
|
||||
/// The state advancement is "cheap" since it does not generate state roots. As a result, the
|
||||
/// returned state might be holistically invalid but the committees will be correct (since they do
|
||||
/// not rely upon state roots).
|
||||
///
|
||||
/// If the given `state` can already serve the `slot`, the committees will be built on the `state`
|
||||
/// and `Cow::Borrowed(state)` will be returned. Otherwise, the state will be cloned, cheaply
|
||||
/// advanced and then returned as a `Cow::Owned`. The end result is that the given `state` is never
|
||||
/// mutated to be invalid (in fact, it is never changed beyond a simple committee cache build).
|
||||
fn cheap_state_advance_to_obtain_committees<'a, E: EthSpec>(
|
||||
state: &'a mut BeaconState<E>,
|
||||
block_slot: Slot,
|
||||
spec: &ChainSpec,
|
||||
) -> Result<Cow<'a, BeaconState<E>>, BlockError> {
|
||||
let block_epoch = block_slot.epoch(E::slots_per_epoch());
|
||||
|
||||
if state.current_epoch() == block_epoch {
|
||||
state.build_committee_cache(RelativeEpoch::Current, spec)?;
|
||||
|
||||
Ok(Cow::Borrowed(state))
|
||||
} else if state.slot > block_slot {
|
||||
Err(BlockError::BlockIsNotLaterThanParent {
|
||||
block_slot,
|
||||
state_slot: state.slot,
|
||||
})
|
||||
} else {
|
||||
let mut state = state.clone_with(CloneConfig::committee_caches_only());
|
||||
|
||||
while state.current_epoch() < block_epoch {
|
||||
// Don't calculate state roots since they aren't required for calculating
|
||||
// shuffling (achieved by providing Hash256::zero()).
|
||||
per_slot_processing(&mut state, Some(Hash256::zero()), spec).map_err(|e| {
|
||||
BlockError::BeaconChainError(BeaconChainError::SlotProcessingError(e))
|
||||
})?;
|
||||
}
|
||||
|
||||
state.build_committee_cache(RelativeEpoch::Current, spec)?;
|
||||
|
||||
Ok(Cow::Owned(state))
|
||||
}
|
||||
}
|
||||
|
||||
/// Obtains a read-locked `ValidatorPubkeyCache` from the `chain`.
|
||||
fn get_validator_pubkey_cache<T: BeaconChainTypes>(
|
||||
chain: &BeaconChain<T>,
|
||||
) -> Result<RwLockReadGuard<ValidatorPubkeyCache>, BlockError> {
|
||||
chain
|
||||
.validator_pubkey_cache
|
||||
.try_read_for(VALIDATOR_PUBKEY_CACHE_LOCK_TIMEOUT)
|
||||
.ok_or_else(|| BeaconChainError::ValidatorPubkeyCacheLockTimeout)
|
||||
.map_err(BlockError::BeaconChainError)
|
||||
}
|
||||
|
||||
/// Produces an _empty_ `BlockSignatureVerifier`.
|
||||
///
|
||||
/// The signature verifier is empty because it does not yet have any of this block's signatures
|
||||
/// added to it. Use `Self::apply_to_signature_verifier` to apply the signatures.
|
||||
fn get_signature_verifier<'a, E: EthSpec>(
|
||||
state: &'a BeaconState<E>,
|
||||
validator_pubkey_cache: &'a ValidatorPubkeyCache,
|
||||
spec: &'a ChainSpec,
|
||||
) -> BlockSignatureVerifier<'a, E, impl Fn(usize) -> Option<Cow<'a, G1Point>> + Clone> {
|
||||
BlockSignatureVerifier::new(
|
||||
state,
|
||||
move |validator_index| {
|
||||
// Disallow access to any validator pubkeys that are not in the current beacon
|
||||
// state.
|
||||
if validator_index < state.validators.len() {
|
||||
validator_pubkey_cache
|
||||
.get(validator_index)
|
||||
.map(|pk| Cow::Borrowed(pk.as_point()))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
},
|
||||
spec,
|
||||
)
|
||||
}
|
||||
@ -0,0 +1,105 @@
|
||||
use crate::{BeaconChainError, BlockError};
|
||||
use state_processing::BlockProcessingError;
|
||||
use types::{Hash256, Slot};
|
||||
|
||||
/// This is a legacy object that is being kept around to reduce merge conflicts.
|
||||
///
|
||||
/// As soon as this is merged into master, it should be removed as soon as possible.
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum BlockProcessingOutcome {
|
||||
/// Block was valid and imported into the block graph.
|
||||
Processed {
|
||||
block_root: Hash256,
|
||||
},
|
||||
InvalidSignature,
|
||||
/// The proposal signature in invalid.
|
||||
ProposalSignatureInvalid,
|
||||
/// The parent block was unknown.
|
||||
ParentUnknown(Hash256),
|
||||
/// The block slot is greater than the present slot.
|
||||
FutureSlot {
|
||||
present_slot: Slot,
|
||||
block_slot: Slot,
|
||||
},
|
||||
/// The block state_root does not match the generated state.
|
||||
StateRootMismatch {
|
||||
block: Hash256,
|
||||
local: Hash256,
|
||||
},
|
||||
/// The block was a genesis block, these blocks cannot be re-imported.
|
||||
GenesisBlock,
|
||||
/// The slot is finalized, no need to import.
|
||||
WouldRevertFinalizedSlot {
|
||||
block_slot: Slot,
|
||||
finalized_slot: Slot,
|
||||
},
|
||||
/// Block is already known, no need to re-import.
|
||||
BlockIsAlreadyKnown,
|
||||
/// The block slot exceeds the MAXIMUM_BLOCK_SLOT_NUMBER.
|
||||
BlockSlotLimitReached,
|
||||
/// The provided block is from an earlier slot than its parent.
|
||||
BlockIsNotLaterThanParent {
|
||||
block_slot: Slot,
|
||||
state_slot: Slot,
|
||||
},
|
||||
/// At least one block in the chain segement did not have it's parent root set to the root of
|
||||
/// the prior block.
|
||||
NonLinearParentRoots,
|
||||
/// The slots of the blocks in the chain segment were not strictly increasing. I.e., a child
|
||||
/// had lower slot than a parent.
|
||||
NonLinearSlots,
|
||||
/// The block could not be applied to the state, it is invalid.
|
||||
PerBlockProcessingError(BlockProcessingError),
|
||||
}
|
||||
|
||||
impl BlockProcessingOutcome {
|
||||
pub fn shim(
|
||||
result: Result<Hash256, BlockError>,
|
||||
) -> Result<BlockProcessingOutcome, BeaconChainError> {
|
||||
match result {
|
||||
Ok(block_root) => Ok(BlockProcessingOutcome::Processed { block_root }),
|
||||
Err(BlockError::ParentUnknown(root)) => Ok(BlockProcessingOutcome::ParentUnknown(root)),
|
||||
Err(BlockError::FutureSlot {
|
||||
present_slot,
|
||||
block_slot,
|
||||
}) => Ok(BlockProcessingOutcome::FutureSlot {
|
||||
present_slot,
|
||||
block_slot,
|
||||
}),
|
||||
Err(BlockError::StateRootMismatch { block, local }) => {
|
||||
Ok(BlockProcessingOutcome::StateRootMismatch { block, local })
|
||||
}
|
||||
Err(BlockError::GenesisBlock) => Ok(BlockProcessingOutcome::GenesisBlock),
|
||||
Err(BlockError::WouldRevertFinalizedSlot {
|
||||
block_slot,
|
||||
finalized_slot,
|
||||
}) => Ok(BlockProcessingOutcome::WouldRevertFinalizedSlot {
|
||||
block_slot,
|
||||
finalized_slot,
|
||||
}),
|
||||
Err(BlockError::BlockIsAlreadyKnown) => Ok(BlockProcessingOutcome::BlockIsAlreadyKnown),
|
||||
Err(BlockError::BlockSlotLimitReached) => {
|
||||
Ok(BlockProcessingOutcome::BlockSlotLimitReached)
|
||||
}
|
||||
Err(BlockError::ProposalSignatureInvalid) => {
|
||||
Ok(BlockProcessingOutcome::ProposalSignatureInvalid)
|
||||
}
|
||||
Err(BlockError::InvalidSignature) => Ok(BlockProcessingOutcome::InvalidSignature),
|
||||
Err(BlockError::BlockIsNotLaterThanParent {
|
||||
block_slot,
|
||||
state_slot,
|
||||
}) => Ok(BlockProcessingOutcome::BlockIsNotLaterThanParent {
|
||||
block_slot,
|
||||
state_slot,
|
||||
}),
|
||||
Err(BlockError::NonLinearParentRoots) => {
|
||||
Ok(BlockProcessingOutcome::NonLinearParentRoots)
|
||||
}
|
||||
Err(BlockError::NonLinearSlots) => Ok(BlockProcessingOutcome::NonLinearSlots),
|
||||
Err(BlockError::PerBlockProcessingError(e)) => {
|
||||
Ok(BlockProcessingOutcome::PerBlockProcessingError(e))
|
||||
}
|
||||
Err(BlockError::BeaconChainError(e)) => Err(e),
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -7,10 +7,11 @@ use crate::fork_choice::SszForkChoice;
|
||||
use crate::head_tracker::HeadTracker;
|
||||
use crate::persisted_beacon_chain::PersistedBeaconChain;
|
||||
use crate::shuffling_cache::ShufflingCache;
|
||||
use crate::snapshot_cache::{SnapshotCache, DEFAULT_SNAPSHOT_CACHE_SIZE};
|
||||
use crate::timeout_rw_lock::TimeoutRwLock;
|
||||
use crate::validator_pubkey_cache::ValidatorPubkeyCache;
|
||||
use crate::{
|
||||
BeaconChain, BeaconChainTypes, CheckPoint, Eth1Chain, Eth1ChainBackend, EventHandler,
|
||||
BeaconChain, BeaconChainTypes, BeaconSnapshot, Eth1Chain, Eth1ChainBackend, EventHandler,
|
||||
ForkChoice,
|
||||
};
|
||||
use eth1::Config as Eth1Config;
|
||||
@ -71,10 +72,10 @@ where
|
||||
pub struct BeaconChainBuilder<T: BeaconChainTypes> {
|
||||
store: Option<Arc<T::Store>>,
|
||||
store_migrator: Option<T::StoreMigrator>,
|
||||
canonical_head: Option<CheckPoint<T::EthSpec>>,
|
||||
canonical_head: Option<BeaconSnapshot<T::EthSpec>>,
|
||||
/// The finalized checkpoint to anchor the chain. May be genesis or a higher
|
||||
/// checkpoint.
|
||||
pub finalized_checkpoint: Option<CheckPoint<T::EthSpec>>,
|
||||
pub finalized_snapshot: Option<BeaconSnapshot<T::EthSpec>>,
|
||||
genesis_block_root: Option<Hash256>,
|
||||
op_pool: Option<OperationPool<T::EthSpec>>,
|
||||
fork_choice: Option<ForkChoice<T>>,
|
||||
@ -110,7 +111,7 @@ where
|
||||
store: None,
|
||||
store_migrator: None,
|
||||
canonical_head: None,
|
||||
finalized_checkpoint: None,
|
||||
finalized_snapshot: None,
|
||||
genesis_block_root: None,
|
||||
op_pool: None,
|
||||
fork_choice: None,
|
||||
@ -247,14 +248,14 @@ where
|
||||
.map_err(|e| format!("DB error when reading finalized state: {:?}", e))?
|
||||
.ok_or_else(|| "Finalized state not found in store".to_string())?;
|
||||
|
||||
self.finalized_checkpoint = Some(CheckPoint {
|
||||
self.finalized_snapshot = Some(BeaconSnapshot {
|
||||
beacon_block_root: finalized_block_root,
|
||||
beacon_block: finalized_block,
|
||||
beacon_state_root: finalized_state_root,
|
||||
beacon_state: finalized_state,
|
||||
});
|
||||
|
||||
self.canonical_head = Some(CheckPoint {
|
||||
self.canonical_head = Some(BeaconSnapshot {
|
||||
beacon_block_root: head_block_root,
|
||||
beacon_block: head_block,
|
||||
beacon_state_root: head_state_root,
|
||||
@ -291,7 +292,7 @@ where
|
||||
self.genesis_block_root = Some(beacon_block_root);
|
||||
|
||||
store
|
||||
.put_state(&beacon_state_root, beacon_state.clone())
|
||||
.put_state(&beacon_state_root, &beacon_state)
|
||||
.map_err(|e| format!("Failed to store genesis state: {:?}", e))?;
|
||||
store
|
||||
.put(&beacon_block_root, &beacon_block)
|
||||
@ -305,7 +306,7 @@ where
|
||||
)
|
||||
})?;
|
||||
|
||||
self.finalized_checkpoint = Some(CheckPoint {
|
||||
self.finalized_snapshot = Some(BeaconSnapshot {
|
||||
beacon_block_root,
|
||||
beacon_block,
|
||||
beacon_state_root,
|
||||
@ -367,7 +368,7 @@ where
|
||||
let mut canonical_head = if let Some(head) = self.canonical_head {
|
||||
head
|
||||
} else {
|
||||
self.finalized_checkpoint
|
||||
self.finalized_snapshot
|
||||
.ok_or_else(|| "Cannot build without a state".to_string())?
|
||||
};
|
||||
|
||||
@ -407,7 +408,7 @@ where
|
||||
.op_pool
|
||||
.ok_or_else(|| "Cannot build without op pool".to_string())?,
|
||||
eth1_chain: self.eth1_chain,
|
||||
canonical_head: TimeoutRwLock::new(canonical_head),
|
||||
canonical_head: TimeoutRwLock::new(canonical_head.clone()),
|
||||
genesis_block_root: self
|
||||
.genesis_block_root
|
||||
.ok_or_else(|| "Cannot build without a genesis block root".to_string())?,
|
||||
@ -418,6 +419,10 @@ where
|
||||
.event_handler
|
||||
.ok_or_else(|| "Cannot build without an event handler".to_string())?,
|
||||
head_tracker: self.head_tracker.unwrap_or_default(),
|
||||
snapshot_cache: TimeoutRwLock::new(SnapshotCache::new(
|
||||
DEFAULT_SNAPSHOT_CACHE_SIZE,
|
||||
canonical_head,
|
||||
)),
|
||||
shuffling_cache: TimeoutRwLock::new(ShufflingCache::new()),
|
||||
validator_pubkey_cache: TimeoutRwLock::new(validator_pubkey_cache),
|
||||
log: log.clone(),
|
||||
@ -469,30 +474,30 @@ where
|
||||
ForkChoice::from_ssz_container(persisted)
|
||||
.map_err(|e| format!("Unable to read persisted fork choice from disk: {:?}", e))?
|
||||
} else {
|
||||
let finalized_checkpoint = &self
|
||||
.finalized_checkpoint
|
||||
let finalized_snapshot = &self
|
||||
.finalized_snapshot
|
||||
.as_ref()
|
||||
.ok_or_else(|| "fork_choice_backend requires a finalized_checkpoint")?;
|
||||
.ok_or_else(|| "fork_choice_backend requires a finalized_snapshot")?;
|
||||
let genesis_block_root = self
|
||||
.genesis_block_root
|
||||
.ok_or_else(|| "fork_choice_backend requires a genesis_block_root")?;
|
||||
|
||||
let backend = ProtoArrayForkChoice::new(
|
||||
finalized_checkpoint.beacon_block.message.slot,
|
||||
finalized_checkpoint.beacon_block.message.state_root,
|
||||
finalized_snapshot.beacon_block.message.slot,
|
||||
finalized_snapshot.beacon_block.message.state_root,
|
||||
// Note: here we set the `justified_epoch` to be the same as the epoch of the
|
||||
// finalized checkpoint. Whilst this finalized checkpoint may actually point to
|
||||
// a _later_ justified checkpoint, that checkpoint won't yet exist in the fork
|
||||
// choice.
|
||||
finalized_checkpoint.beacon_state.current_epoch(),
|
||||
finalized_checkpoint.beacon_state.current_epoch(),
|
||||
finalized_checkpoint.beacon_block_root,
|
||||
finalized_snapshot.beacon_state.current_epoch(),
|
||||
finalized_snapshot.beacon_state.current_epoch(),
|
||||
finalized_snapshot.beacon_block_root,
|
||||
)?;
|
||||
|
||||
ForkChoice::new(
|
||||
backend,
|
||||
genesis_block_root,
|
||||
&finalized_checkpoint.beacon_state,
|
||||
&finalized_snapshot.beacon_state,
|
||||
)
|
||||
};
|
||||
|
||||
@ -563,7 +568,7 @@ where
|
||||
/// Requires the state to be initialized.
|
||||
pub fn testing_slot_clock(self, slot_duration: Duration) -> Result<Self, String> {
|
||||
let genesis_time = self
|
||||
.finalized_checkpoint
|
||||
.finalized_snapshot
|
||||
.as_ref()
|
||||
.ok_or_else(|| "testing_slot_clock requires an initialized state")?
|
||||
.beacon_state
|
||||
@ -642,7 +647,7 @@ mod test {
|
||||
|
||||
#[test]
|
||||
fn recent_genesis() {
|
||||
let validator_count = 8;
|
||||
let validator_count = 1;
|
||||
let genesis_time = 13_371_337;
|
||||
|
||||
let log = get_logger();
|
||||
|
||||
@ -3,9 +3,11 @@ use crate::fork_choice::Error as ForkChoiceError;
|
||||
use operation_pool::OpPoolError;
|
||||
use ssz::DecodeError;
|
||||
use ssz_types::Error as SszTypesError;
|
||||
use state_processing::per_block_processing::errors::AttestationValidationError;
|
||||
use state_processing::BlockProcessingError;
|
||||
use state_processing::SlotProcessingError;
|
||||
use state_processing::{
|
||||
block_signature_verifier::Error as BlockSignatureVerifierError,
|
||||
per_block_processing::errors::AttestationValidationError,
|
||||
signature_sets::Error as SignatureSetError, BlockProcessingError, SlotProcessingError,
|
||||
};
|
||||
use std::time::Duration;
|
||||
use types::*;
|
||||
|
||||
@ -57,13 +59,18 @@ pub enum BeaconChainError {
|
||||
IncorrectStateForAttestation(RelativeEpochError),
|
||||
InvalidValidatorPubkeyBytes(DecodeError),
|
||||
ValidatorPubkeyCacheIncomplete(usize),
|
||||
SignatureSetError(state_processing::signature_sets::Error),
|
||||
SignatureSetError(SignatureSetError),
|
||||
BlockSignatureVerifierError(state_processing::block_signature_verifier::Error),
|
||||
DuplicateValidatorPublicKey,
|
||||
ValidatorPubkeyCacheFileError(String),
|
||||
OpPoolError(OpPoolError),
|
||||
}
|
||||
|
||||
easy_from_to!(SlotProcessingError, BeaconChainError);
|
||||
easy_from_to!(AttestationValidationError, BeaconChainError);
|
||||
easy_from_to!(SszTypesError, BeaconChainError);
|
||||
easy_from_to!(OpPoolError, BeaconChainError);
|
||||
easy_from_to!(BlockSignatureVerifierError, BeaconChainError);
|
||||
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum BlockProductionError {
|
||||
@ -84,3 +91,27 @@ easy_from_to!(BlockProcessingError, BlockProductionError);
|
||||
easy_from_to!(BeaconStateError, BlockProductionError);
|
||||
easy_from_to!(SlotProcessingError, BlockProductionError);
|
||||
easy_from_to!(Eth1ChainError, BlockProductionError);
|
||||
|
||||
/// A reason for not propagating an attestation (single or aggregate).
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum AttestationDropReason {
|
||||
SlotClockError,
|
||||
TooNew { attestation_slot: Slot, now: Slot },
|
||||
TooOld { attestation_slot: Slot, now: Slot },
|
||||
NoValidationState(BeaconChainError),
|
||||
BlockUnknown(Hash256),
|
||||
BadIndexedAttestation(AttestationValidationError),
|
||||
AggregatorNotInAttestingIndices,
|
||||
AggregatorNotSelected,
|
||||
AggregatorSignatureInvalid,
|
||||
SignatureInvalid,
|
||||
}
|
||||
|
||||
/// A reason for not propagating a block.
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum BlockDropReason {
|
||||
SlotClockError,
|
||||
TooNew { block_slot: Slot, now: Slot },
|
||||
// FIXME(sproul): add detail here
|
||||
ValidationFailure,
|
||||
}
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
use crate::metrics;
|
||||
use eth1::{Config as Eth1Config, Eth1Block, Service as HttpService};
|
||||
use eth2_hashing::hash;
|
||||
use exit_future::Exit;
|
||||
use futures::Future;
|
||||
use slog::{debug, error, trace, Logger};
|
||||
use ssz::{Decode, Encode};
|
||||
@ -279,7 +278,10 @@ impl<T: EthSpec, S: Store<T>> CachingEth1Backend<T, S> {
|
||||
}
|
||||
|
||||
/// Starts the routine which connects to the external eth1 node and updates the caches.
|
||||
pub fn start(&self, exit: Exit) -> impl Future<Item = (), Error = ()> {
|
||||
pub fn start(
|
||||
&self,
|
||||
exit: tokio::sync::oneshot::Receiver<()>,
|
||||
) -> impl Future<Item = (), Error = ()> {
|
||||
self.core.auto_update(exit)
|
||||
}
|
||||
|
||||
|
||||
@ -306,10 +306,7 @@ impl CheckpointManager {
|
||||
.ok_or_else(|| Error::UnknownJustifiedBlock(block_root))?;
|
||||
|
||||
let state = chain
|
||||
.get_state_caching_only_with_committee_caches(
|
||||
&block.state_root(),
|
||||
Some(block.slot()),
|
||||
)?
|
||||
.get_state(&block.state_root(), Some(block.slot()))?
|
||||
.ok_or_else(|| Error::UnknownJustifiedState(block.state_root()))?;
|
||||
|
||||
Ok(get_effective_balances(&state))
|
||||
|
||||
@ -3,8 +3,9 @@
|
||||
extern crate lazy_static;
|
||||
|
||||
mod beacon_chain;
|
||||
mod beacon_snapshot;
|
||||
mod block_verification;
|
||||
pub mod builder;
|
||||
mod checkpoint;
|
||||
mod errors;
|
||||
pub mod eth1_chain;
|
||||
pub mod events;
|
||||
@ -13,16 +14,17 @@ mod head_tracker;
|
||||
mod metrics;
|
||||
mod persisted_beacon_chain;
|
||||
mod shuffling_cache;
|
||||
mod snapshot_cache;
|
||||
pub mod test_utils;
|
||||
mod timeout_rw_lock;
|
||||
mod validator_pubkey_cache;
|
||||
|
||||
pub use self::beacon_chain::{
|
||||
AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, BlockProcessingOutcome,
|
||||
StateSkipConfig,
|
||||
AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, StateSkipConfig,
|
||||
};
|
||||
pub use self::checkpoint::CheckPoint;
|
||||
pub use self::beacon_snapshot::BeaconSnapshot;
|
||||
pub use self::errors::{BeaconChainError, BlockProductionError};
|
||||
pub use block_verification::{BlockError, BlockProcessingOutcome};
|
||||
pub use eth1_chain::{Eth1Chain, Eth1ChainBackend};
|
||||
pub use events::EventHandler;
|
||||
pub use fork_choice::ForkChoice;
|
||||
|
||||
@ -32,6 +32,10 @@ lazy_static! {
|
||||
"beacon_block_processing_committee_building_seconds",
|
||||
"Time spent building/obtaining committees for block processing."
|
||||
);
|
||||
pub static ref BLOCK_PROCESSING_SIGNATURE: Result<Histogram> = try_create_histogram(
|
||||
"beacon_block_processing_signature_seconds",
|
||||
"Time spent doing signature verification for a block."
|
||||
);
|
||||
pub static ref BLOCK_PROCESSING_CORE: Result<Histogram> = try_create_histogram(
|
||||
"beacon_block_processing_core_seconds",
|
||||
"Time spent doing the core per_block_processing state processing."
|
||||
|
||||
217
beacon_node/beacon_chain/src/snapshot_cache.rs
Normal file
217
beacon_node/beacon_chain/src/snapshot_cache.rs
Normal file
@ -0,0 +1,217 @@
|
||||
use crate::BeaconSnapshot;
|
||||
use std::cmp;
|
||||
use types::{Epoch, EthSpec, Hash256};
|
||||
|
||||
/// The default size of the cache.
|
||||
pub const DEFAULT_SNAPSHOT_CACHE_SIZE: usize = 4;
|
||||
|
||||
/// Provides a cache of `BeaconSnapshot` that is intended primarily for block processing.
|
||||
///
|
||||
/// ## Cache Queuing
|
||||
///
|
||||
/// The cache has a non-standard queue mechanism (specifically, it is not LRU).
|
||||
///
|
||||
/// The cache has a max number of elements (`max_len`). Until `max_len` is achieved, all snapshots
|
||||
/// are simply added to the queue. Once `max_len` is achieved, adding a new snapshot will cause an
|
||||
/// existing snapshot to be ejected. The ejected snapshot will:
|
||||
///
|
||||
/// - Never be the `head_block_root`.
|
||||
/// - Be the snapshot with the lowest `state.slot` (ties broken arbitrarily).
|
||||
pub struct SnapshotCache<T: EthSpec> {
|
||||
max_len: usize,
|
||||
head_block_root: Hash256,
|
||||
snapshots: Vec<BeaconSnapshot<T>>,
|
||||
}
|
||||
|
||||
impl<T: EthSpec> SnapshotCache<T> {
|
||||
/// Instantiate a new cache which contains the `head` snapshot.
|
||||
///
|
||||
/// Setting `max_len = 0` is equivalent to setting `max_len = 1`.
|
||||
pub fn new(max_len: usize, head: BeaconSnapshot<T>) -> Self {
|
||||
Self {
|
||||
max_len: cmp::max(max_len, 1),
|
||||
head_block_root: head.beacon_block_root,
|
||||
snapshots: vec![head],
|
||||
}
|
||||
}
|
||||
|
||||
/// Insert a snapshot, potentially removing an existing snapshot if `self` is at capacity (see
|
||||
/// struct-level documentation for more info).
|
||||
pub fn insert(&mut self, snapshot: BeaconSnapshot<T>) {
|
||||
if self.snapshots.len() < self.max_len {
|
||||
self.snapshots.push(snapshot);
|
||||
} else {
|
||||
let insert_at = self
|
||||
.snapshots
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter_map(|(i, snapshot)| {
|
||||
if snapshot.beacon_block_root != self.head_block_root {
|
||||
Some((i, snapshot.beacon_state.slot))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
})
|
||||
.min_by_key(|(_i, slot)| *slot)
|
||||
.map(|(i, _slot)| i);
|
||||
|
||||
if let Some(i) = insert_at {
|
||||
self.snapshots[i] = snapshot;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// If there is a snapshot with `block_root`, remove and return it.
|
||||
pub fn try_remove(&mut self, block_root: Hash256) -> Option<BeaconSnapshot<T>> {
|
||||
self.snapshots
|
||||
.iter()
|
||||
.position(|snapshot| snapshot.beacon_block_root == block_root)
|
||||
.map(|i| self.snapshots.remove(i))
|
||||
}
|
||||
|
||||
/// If there is a snapshot with `block_root`, clone it (with only the committee caches) and
|
||||
/// return the clone.
|
||||
pub fn get_cloned(&self, block_root: Hash256) -> Option<BeaconSnapshot<T>> {
|
||||
self.snapshots
|
||||
.iter()
|
||||
.find(|snapshot| snapshot.beacon_block_root == block_root)
|
||||
.map(|snapshot| snapshot.clone_with_only_committee_caches())
|
||||
}
|
||||
|
||||
/// Removes all snapshots from the queue that are less than or equal to the finalized epoch.
|
||||
pub fn prune(&mut self, finalized_epoch: Epoch) {
|
||||
self.snapshots.retain(|snapshot| {
|
||||
snapshot.beacon_state.slot > finalized_epoch.start_slot(T::slots_per_epoch())
|
||||
})
|
||||
}
|
||||
|
||||
/// Inform the cache that the head of the beacon chain has changed.
|
||||
///
|
||||
/// The snapshot that matches this `head_block_root` will never be ejected from the cache
|
||||
/// during `Self::insert`.
|
||||
pub fn update_head(&mut self, head_block_root: Hash256) {
|
||||
self.head_block_root = head_block_root
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod test {
|
||||
use super::*;
|
||||
use types::{
|
||||
test_utils::{generate_deterministic_keypair, TestingBeaconStateBuilder},
|
||||
BeaconBlock, Epoch, MainnetEthSpec, Signature, SignedBeaconBlock, Slot,
|
||||
};
|
||||
|
||||
const CACHE_SIZE: usize = 4;
|
||||
|
||||
fn get_snapshot(i: u64) -> BeaconSnapshot<MainnetEthSpec> {
|
||||
let spec = MainnetEthSpec::default_spec();
|
||||
|
||||
let state_builder = TestingBeaconStateBuilder::from_deterministic_keypairs(1, &spec);
|
||||
let (beacon_state, _keypairs) = state_builder.build();
|
||||
|
||||
BeaconSnapshot {
|
||||
beacon_state,
|
||||
beacon_state_root: Hash256::from_low_u64_be(i),
|
||||
beacon_block: SignedBeaconBlock {
|
||||
message: BeaconBlock::empty(&spec),
|
||||
signature: Signature::new(&[42], &generate_deterministic_keypair(0).sk),
|
||||
},
|
||||
beacon_block_root: Hash256::from_low_u64_be(i),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn insert_get_prune_update() {
|
||||
let mut cache = SnapshotCache::new(CACHE_SIZE, get_snapshot(0));
|
||||
|
||||
// Insert a bunch of entries in the cache. It should look like this:
|
||||
//
|
||||
// Index Root
|
||||
// 0 0 <--head
|
||||
// 1 1
|
||||
// 2 2
|
||||
// 3 3
|
||||
for i in 1..CACHE_SIZE as u64 {
|
||||
let mut snapshot = get_snapshot(i);
|
||||
|
||||
// Each snapshot should be one slot into an epoch, with each snapshot one epoch apart.
|
||||
snapshot.beacon_state.slot = Slot::from(i * MainnetEthSpec::slots_per_epoch() + 1);
|
||||
|
||||
cache.insert(snapshot);
|
||||
|
||||
assert_eq!(
|
||||
cache.snapshots.len(),
|
||||
i as usize + 1,
|
||||
"cache length should be as expected"
|
||||
);
|
||||
assert_eq!(cache.head_block_root, Hash256::from_low_u64_be(0));
|
||||
}
|
||||
|
||||
// Insert a new value in the cache. Afterwards it should look like:
|
||||
//
|
||||
// Index Root
|
||||
// 0 0 <--head
|
||||
// 1 42
|
||||
// 2 2
|
||||
// 3 3
|
||||
assert_eq!(cache.snapshots.len(), CACHE_SIZE);
|
||||
cache.insert(get_snapshot(42));
|
||||
assert_eq!(cache.snapshots.len(), CACHE_SIZE);
|
||||
|
||||
assert!(
|
||||
cache.try_remove(Hash256::from_low_u64_be(1)).is_none(),
|
||||
"the snapshot with the lowest slot should have been removed during the insert function"
|
||||
);
|
||||
assert!(cache.get_cloned(Hash256::from_low_u64_be(1)).is_none());
|
||||
|
||||
assert!(
|
||||
cache
|
||||
.get_cloned(Hash256::from_low_u64_be(0))
|
||||
.expect("the head should still be in the cache")
|
||||
.beacon_block_root
|
||||
== Hash256::from_low_u64_be(0),
|
||||
"get_cloned should get the correct snapshot"
|
||||
);
|
||||
assert!(
|
||||
cache
|
||||
.try_remove(Hash256::from_low_u64_be(0))
|
||||
.expect("the head should still be in the cache")
|
||||
.beacon_block_root
|
||||
== Hash256::from_low_u64_be(0),
|
||||
"try_remove should get the correct snapshot"
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
cache.snapshots.len(),
|
||||
CACHE_SIZE - 1,
|
||||
"try_remove should shorten the cache"
|
||||
);
|
||||
|
||||
// Prune the cache. Afterwards it should look like:
|
||||
//
|
||||
// Index Root
|
||||
// 0 2
|
||||
// 1 3
|
||||
cache.prune(Epoch::new(2));
|
||||
|
||||
assert_eq!(cache.snapshots.len(), 2);
|
||||
|
||||
cache.update_head(Hash256::from_low_u64_be(2));
|
||||
|
||||
// Over-fill the cache so it needs to eject some old values on insert.
|
||||
for i in 0..CACHE_SIZE as u64 {
|
||||
cache.insert(get_snapshot(u64::max_value() - i));
|
||||
}
|
||||
|
||||
// Ensure that the new head value was not removed from the cache.
|
||||
assert!(
|
||||
cache
|
||||
.try_remove(Hash256::from_low_u64_be(2))
|
||||
.expect("the new head should still be in the cache")
|
||||
.beacon_block_root
|
||||
== Hash256::from_low_u64_be(2),
|
||||
"try_remove should get the correct snapshot"
|
||||
);
|
||||
}
|
||||
}
|
||||
@ -6,8 +6,7 @@ use crate::{
|
||||
builder::{BeaconChainBuilder, Witness},
|
||||
eth1_chain::CachingEth1Backend,
|
||||
events::NullEventHandler,
|
||||
AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, BlockProcessingOutcome,
|
||||
StateSkipConfig,
|
||||
AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, StateSkipConfig,
|
||||
};
|
||||
use genesis::interop_genesis_state;
|
||||
use rayon::prelude::*;
|
||||
@ -256,20 +255,15 @@ where
|
||||
|
||||
let (block, new_state) = self.build_block(state.clone(), slot, block_strategy);
|
||||
|
||||
let outcome = self
|
||||
let block_root = self
|
||||
.chain
|
||||
.process_block(block)
|
||||
.expect("should not error during block processing");
|
||||
|
||||
self.chain.fork_choice().expect("should find head");
|
||||
|
||||
if let BlockProcessingOutcome::Processed { block_root } = outcome {
|
||||
head_block_root = Some(block_root);
|
||||
|
||||
self.add_free_attestations(&attestation_strategy, &new_state, block_root, slot);
|
||||
} else {
|
||||
panic!("block should be successfully processed: {:?}", outcome);
|
||||
}
|
||||
head_block_root = Some(block_root);
|
||||
self.add_free_attestations(&attestation_strategy, &new_state, block_root, slot);
|
||||
|
||||
state = new_state;
|
||||
slot += 1;
|
||||
@ -348,7 +342,7 @@ where
|
||||
.for_each(|attestation| {
|
||||
match self
|
||||
.chain
|
||||
.process_attestation(attestation)
|
||||
.process_attestation(attestation, Some(false))
|
||||
.expect("should not error during attestation processing")
|
||||
{
|
||||
AttestationProcessingOutcome::Processed => (),
|
||||
|
||||
@ -1,10 +1,11 @@
|
||||
use crate::errors::BeaconChainError;
|
||||
use ssz::{Decode, DecodeError, Encode};
|
||||
use std::collections::HashMap;
|
||||
use std::convert::TryInto;
|
||||
use std::fs::{File, OpenOptions};
|
||||
use std::io::{self, Read, Write};
|
||||
use std::path::Path;
|
||||
use types::{BeaconState, EthSpec, PublicKey, PublicKeyBytes};
|
||||
use types::{BeaconState, EthSpec, PublicKey, PublicKeyBytes, Validator};
|
||||
|
||||
/// Provides a mapping of `validator_index -> validator_publickey`.
|
||||
///
|
||||
@ -19,6 +20,7 @@ use types::{BeaconState, EthSpec, PublicKey, PublicKeyBytes};
|
||||
/// copy of itself. This allows it to be restored between process invocations.
|
||||
pub struct ValidatorPubkeyCache {
|
||||
pubkeys: Vec<PublicKey>,
|
||||
indices: HashMap<PublicKeyBytes, usize>,
|
||||
persitence_file: ValidatorPubkeyCacheFile,
|
||||
}
|
||||
|
||||
@ -47,6 +49,7 @@ impl ValidatorPubkeyCache {
|
||||
let mut cache = Self {
|
||||
persitence_file: ValidatorPubkeyCacheFile::create(persistence_path)?,
|
||||
pubkeys: vec![],
|
||||
indices: HashMap::new(),
|
||||
};
|
||||
|
||||
cache.import_new_pubkeys(state)?;
|
||||
@ -61,38 +64,57 @@ impl ValidatorPubkeyCache {
|
||||
&mut self,
|
||||
state: &BeaconState<T>,
|
||||
) -> Result<(), BeaconChainError> {
|
||||
state
|
||||
.validators
|
||||
.iter()
|
||||
.skip(self.pubkeys.len())
|
||||
.try_for_each(|v| {
|
||||
let i = self.pubkeys.len();
|
||||
if state.validators.len() > self.pubkeys.len() {
|
||||
self.import(&state.validators[self.pubkeys.len()..])
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
// The item is written to disk (the persistence file) _before_ it is written into
|
||||
// the local struct.
|
||||
//
|
||||
// This means that a pubkey cache read from disk will always be equivalent to or
|
||||
// _later than_ the cache that was running in the previous instance of Lighthouse.
|
||||
//
|
||||
// The motivation behind this ordering is that we do not want to have states that
|
||||
// reference a pubkey that is not in our cache. However, it's fine to have pubkeys
|
||||
// that are never referenced in a state.
|
||||
self.persitence_file.append(i, &v.pubkey)?;
|
||||
/// Adds zero or more validators to `self`.
|
||||
fn import(&mut self, validators: &[Validator]) -> Result<(), BeaconChainError> {
|
||||
self.pubkeys.reserve(validators.len());
|
||||
self.indices.reserve(validators.len());
|
||||
|
||||
self.pubkeys.push(
|
||||
(&v.pubkey)
|
||||
.try_into()
|
||||
.map_err(BeaconChainError::InvalidValidatorPubkeyBytes)?,
|
||||
);
|
||||
for v in validators.iter() {
|
||||
let i = self.pubkeys.len();
|
||||
|
||||
Ok(())
|
||||
})
|
||||
if self.indices.contains_key(&v.pubkey) {
|
||||
return Err(BeaconChainError::DuplicateValidatorPublicKey);
|
||||
}
|
||||
|
||||
// The item is written to disk (the persistence file) _before_ it is written into
|
||||
// the local struct.
|
||||
//
|
||||
// This means that a pubkey cache read from disk will always be equivalent to or
|
||||
// _later than_ the cache that was running in the previous instance of Lighthouse.
|
||||
//
|
||||
// The motivation behind this ordering is that we do not want to have states that
|
||||
// reference a pubkey that is not in our cache. However, it's fine to have pubkeys
|
||||
// that are never referenced in a state.
|
||||
self.persitence_file.append(i, &v.pubkey)?;
|
||||
|
||||
self.pubkeys.push(
|
||||
(&v.pubkey)
|
||||
.try_into()
|
||||
.map_err(BeaconChainError::InvalidValidatorPubkeyBytes)?,
|
||||
);
|
||||
|
||||
self.indices.insert(v.pubkey.clone(), i);
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Get the public key for a validator with index `i`.
|
||||
pub fn get(&self, i: usize) -> Option<&PublicKey> {
|
||||
self.pubkeys.get(i)
|
||||
}
|
||||
|
||||
/// Get the index of a validator with `pubkey`.
|
||||
pub fn get_index(&self, pubkey: &PublicKeyBytes) -> Option<usize> {
|
||||
self.indices.get(pubkey).copied()
|
||||
}
|
||||
}
|
||||
|
||||
/// Allows for maintaining an on-disk copy of the `ValidatorPubkeyCache`. The file is raw SSZ bytes
|
||||
@ -168,12 +190,14 @@ impl ValidatorPubkeyCacheFile {
|
||||
|
||||
let mut last = None;
|
||||
let mut pubkeys = Vec::with_capacity(list.len());
|
||||
let mut indices = HashMap::new();
|
||||
|
||||
for (index, pubkey) in list {
|
||||
let expected = last.map(|n| n + 1);
|
||||
if expected.map_or(true, |expected| index == expected) {
|
||||
last = Some(index);
|
||||
pubkeys.push((&pubkey).try_into().map_err(Error::SszError)?);
|
||||
indices.insert(pubkey, index);
|
||||
} else {
|
||||
return Err(Error::InconsistentIndex {
|
||||
expected,
|
||||
@ -184,6 +208,7 @@ impl ValidatorPubkeyCacheFile {
|
||||
|
||||
Ok(ValidatorPubkeyCache {
|
||||
pubkeys,
|
||||
indices,
|
||||
persitence_file: self,
|
||||
})
|
||||
}
|
||||
@ -221,6 +246,16 @@ mod test {
|
||||
if i < validator_count {
|
||||
let pubkey = cache.get(i).expect("pubkey should be present");
|
||||
assert_eq!(pubkey, &keypairs[i].pk, "pubkey should match cache");
|
||||
|
||||
let pubkey_bytes: PublicKeyBytes = pubkey.clone().into();
|
||||
|
||||
assert_eq!(
|
||||
i,
|
||||
cache
|
||||
.get_index(&pubkey_bytes)
|
||||
.expect("should resolve index"),
|
||||
"index should match cache"
|
||||
);
|
||||
} else {
|
||||
assert_eq!(
|
||||
cache.get(i),
|
||||
|
||||
572
beacon_node/beacon_chain/tests/import_chain_segment_tests.rs
Normal file
572
beacon_node/beacon_chain/tests/import_chain_segment_tests.rs
Normal file
@ -0,0 +1,572 @@
|
||||
#![cfg(not(debug_assertions))]
|
||||
|
||||
#[macro_use]
|
||||
extern crate lazy_static;
|
||||
|
||||
use beacon_chain::{
|
||||
test_utils::{AttestationStrategy, BeaconChainHarness, BlockStrategy, HarnessType},
|
||||
BeaconSnapshot, BlockError,
|
||||
};
|
||||
use types::{
|
||||
test_utils::generate_deterministic_keypair, AggregateSignature, AttestationData,
|
||||
AttesterSlashing, Checkpoint, Deposit, DepositData, Epoch, EthSpec, Hash256,
|
||||
IndexedAttestation, Keypair, MainnetEthSpec, ProposerSlashing, Signature, SignedBeaconBlock,
|
||||
SignedBeaconBlockHeader, SignedVoluntaryExit, Slot, VoluntaryExit, DEPOSIT_TREE_DEPTH,
|
||||
};
|
||||
|
||||
type E = MainnetEthSpec;
|
||||
|
||||
// Should ideally be divisible by 3.
|
||||
pub const VALIDATOR_COUNT: usize = 24;
|
||||
pub const CHAIN_SEGMENT_LENGTH: usize = 64 * 5;
|
||||
|
||||
lazy_static! {
|
||||
/// A cached set of keys.
|
||||
static ref KEYPAIRS: Vec<Keypair> = types::test_utils::generate_deterministic_keypairs(VALIDATOR_COUNT);
|
||||
|
||||
/// A cached set of valid blocks
|
||||
static ref CHAIN_SEGMENT: Vec<BeaconSnapshot<E>> = get_chain_segment();
|
||||
}
|
||||
|
||||
fn get_chain_segment() -> Vec<BeaconSnapshot<E>> {
|
||||
let harness = get_harness(VALIDATOR_COUNT);
|
||||
|
||||
harness.extend_chain(
|
||||
CHAIN_SEGMENT_LENGTH,
|
||||
BlockStrategy::OnCanonicalHead,
|
||||
AttestationStrategy::AllValidators,
|
||||
);
|
||||
|
||||
harness
|
||||
.chain
|
||||
.chain_dump()
|
||||
.expect("should dump chain")
|
||||
.into_iter()
|
||||
.skip(1)
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn get_harness(validator_count: usize) -> BeaconChainHarness<HarnessType<E>> {
|
||||
let harness = BeaconChainHarness::new(MainnetEthSpec, KEYPAIRS[0..validator_count].to_vec());
|
||||
|
||||
harness.advance_slot();
|
||||
|
||||
harness
|
||||
}
|
||||
|
||||
fn chain_segment_blocks() -> Vec<SignedBeaconBlock<E>> {
|
||||
CHAIN_SEGMENT
|
||||
.iter()
|
||||
.map(|snapshot| snapshot.beacon_block.clone())
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn junk_signature() -> Signature {
|
||||
let kp = generate_deterministic_keypair(VALIDATOR_COUNT);
|
||||
let message = &[42, 42];
|
||||
Signature::new(message, &kp.sk)
|
||||
}
|
||||
|
||||
fn junk_aggregate_signature() -> AggregateSignature {
|
||||
let mut agg_sig = AggregateSignature::new();
|
||||
agg_sig.add(&junk_signature());
|
||||
agg_sig
|
||||
}
|
||||
|
||||
fn update_proposal_signatures(
|
||||
snapshots: &mut [BeaconSnapshot<E>],
|
||||
harness: &BeaconChainHarness<HarnessType<E>>,
|
||||
) {
|
||||
for snapshot in snapshots {
|
||||
let spec = &harness.chain.spec;
|
||||
let slot = snapshot.beacon_block.slot();
|
||||
let state = &snapshot.beacon_state;
|
||||
let proposer_index = state
|
||||
.get_beacon_proposer_index(slot, spec)
|
||||
.expect("should find proposer index");
|
||||
let keypair = harness
|
||||
.keypairs
|
||||
.get(proposer_index)
|
||||
.expect("proposer keypair should be available");
|
||||
|
||||
snapshot.beacon_block =
|
||||
snapshot
|
||||
.beacon_block
|
||||
.message
|
||||
.clone()
|
||||
.sign(&keypair.sk, &state.fork, spec);
|
||||
}
|
||||
}
|
||||
|
||||
fn update_parent_roots(snapshots: &mut [BeaconSnapshot<E>]) {
|
||||
for i in 0..snapshots.len() {
|
||||
let root = snapshots[i].beacon_block.canonical_root();
|
||||
if let Some(child) = snapshots.get_mut(i + 1) {
|
||||
child.beacon_block.message.parent_root = root
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn chain_segment_full_segment() {
|
||||
let harness = get_harness(VALIDATOR_COUNT);
|
||||
let blocks = chain_segment_blocks();
|
||||
|
||||
harness
|
||||
.chain
|
||||
.slot_clock
|
||||
.set_slot(blocks.last().unwrap().slot().as_u64());
|
||||
|
||||
// Sneak in a little check to ensure we can process empty chain segments.
|
||||
harness
|
||||
.chain
|
||||
.process_chain_segment(vec![])
|
||||
.expect("should import empty chain segment");
|
||||
|
||||
harness
|
||||
.chain
|
||||
.process_chain_segment(blocks.clone())
|
||||
.expect("should import chain segment");
|
||||
|
||||
harness.chain.fork_choice().expect("should run fork choice");
|
||||
|
||||
assert_eq!(
|
||||
harness
|
||||
.chain
|
||||
.head_info()
|
||||
.expect("should get harness b head")
|
||||
.block_root,
|
||||
blocks.last().unwrap().canonical_root(),
|
||||
"harness should have last block as head"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn chain_segment_varying_chunk_size() {
|
||||
for chunk_size in &[1, 2, 3, 5, 31, 32, 33, 42] {
|
||||
let harness = get_harness(VALIDATOR_COUNT);
|
||||
let blocks = chain_segment_blocks();
|
||||
|
||||
harness
|
||||
.chain
|
||||
.slot_clock
|
||||
.set_slot(blocks.last().unwrap().slot().as_u64());
|
||||
|
||||
for chunk in blocks.chunks(*chunk_size) {
|
||||
harness
|
||||
.chain
|
||||
.process_chain_segment(chunk.to_vec())
|
||||
.expect(&format!(
|
||||
"should import chain segment of len {}",
|
||||
chunk_size
|
||||
));
|
||||
}
|
||||
|
||||
harness.chain.fork_choice().expect("should run fork choice");
|
||||
|
||||
assert_eq!(
|
||||
harness
|
||||
.chain
|
||||
.head_info()
|
||||
.expect("should get harness b head")
|
||||
.block_root,
|
||||
blocks.last().unwrap().canonical_root(),
|
||||
"harness should have last block as head"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn chain_segment_non_linear_parent_roots() {
|
||||
let harness = get_harness(VALIDATOR_COUNT);
|
||||
harness
|
||||
.chain
|
||||
.slot_clock
|
||||
.set_slot(CHAIN_SEGMENT.last().unwrap().beacon_block.slot().as_u64());
|
||||
|
||||
/*
|
||||
* Test with a block removed.
|
||||
*/
|
||||
let mut blocks = chain_segment_blocks();
|
||||
blocks.remove(2);
|
||||
|
||||
assert_eq!(
|
||||
harness.chain.process_chain_segment(blocks.clone()),
|
||||
Err(BlockError::NonLinearParentRoots),
|
||||
"should not import chain with missing parent"
|
||||
);
|
||||
|
||||
/*
|
||||
* Test with a modified parent root.
|
||||
*/
|
||||
let mut blocks = chain_segment_blocks();
|
||||
blocks[3].message.parent_root = Hash256::zero();
|
||||
|
||||
assert_eq!(
|
||||
harness.chain.process_chain_segment(blocks.clone()),
|
||||
Err(BlockError::NonLinearParentRoots),
|
||||
"should not import chain with a broken parent root link"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn chain_segment_non_linear_slots() {
|
||||
let harness = get_harness(VALIDATOR_COUNT);
|
||||
harness
|
||||
.chain
|
||||
.slot_clock
|
||||
.set_slot(CHAIN_SEGMENT.last().unwrap().beacon_block.slot().as_u64());
|
||||
|
||||
/*
|
||||
* Test where a child is lower than the parent.
|
||||
*/
|
||||
|
||||
let mut blocks = chain_segment_blocks();
|
||||
blocks[3].message.slot = Slot::new(0);
|
||||
|
||||
assert_eq!(
|
||||
harness.chain.process_chain_segment(blocks.clone()),
|
||||
Err(BlockError::NonLinearSlots),
|
||||
"should not import chain with a parent that has a lower slot than its child"
|
||||
);
|
||||
|
||||
/*
|
||||
* Test where a child is equal to the parent.
|
||||
*/
|
||||
|
||||
let mut blocks = chain_segment_blocks();
|
||||
blocks[3].message.slot = blocks[2].message.slot;
|
||||
|
||||
assert_eq!(
|
||||
harness.chain.process_chain_segment(blocks.clone()),
|
||||
Err(BlockError::NonLinearSlots),
|
||||
"should not import chain with a parent that has an equal slot to its child"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalid_signatures() {
|
||||
let mut checked_attestation = false;
|
||||
|
||||
for &block_index in &[0, 1, 32, 64, 68 + 1, 129, CHAIN_SEGMENT.len() - 1] {
|
||||
let harness = get_harness(VALIDATOR_COUNT);
|
||||
harness
|
||||
.chain
|
||||
.slot_clock
|
||||
.set_slot(CHAIN_SEGMENT.last().unwrap().beacon_block.slot().as_u64());
|
||||
|
||||
// Import all the ancestors before the `block_index` block.
|
||||
let ancestor_blocks = CHAIN_SEGMENT
|
||||
.iter()
|
||||
.take(block_index)
|
||||
.map(|snapshot| snapshot.beacon_block.clone())
|
||||
.collect();
|
||||
harness
|
||||
.chain
|
||||
.process_chain_segment(ancestor_blocks)
|
||||
.expect("should import all blocks prior to the one being tested");
|
||||
|
||||
// For the given snapshots, test the following:
|
||||
//
|
||||
// - The `process_chain_segment` function returns `InvalidSignature`.
|
||||
// - The `process_block` function returns `InvalidSignature` when importing the
|
||||
// `SignedBeaconBlock` directly.
|
||||
// - The `verify_block_for_gossip` function does _not_ return an error.
|
||||
// - The `process_block` function returns `InvalidSignature` when verifying the
|
||||
// GossipVerifiedBlock.
|
||||
let assert_invalid_signature = |snapshots: &[BeaconSnapshot<E>], item: &str| {
|
||||
let blocks = snapshots
|
||||
.iter()
|
||||
.map(|snapshot| snapshot.beacon_block.clone())
|
||||
.collect();
|
||||
|
||||
// Ensure the block will be rejected if imported in a chain segment.
|
||||
assert_eq!(
|
||||
harness.chain.process_chain_segment(blocks),
|
||||
Err(BlockError::InvalidSignature),
|
||||
"should not import chain segment with an invalid {} signature",
|
||||
item
|
||||
);
|
||||
|
||||
// Ensure the block will be rejected if imported on its own (without gossip checking).
|
||||
assert_eq!(
|
||||
harness
|
||||
.chain
|
||||
.process_block(snapshots[block_index].beacon_block.clone()),
|
||||
Err(BlockError::InvalidSignature),
|
||||
"should not import individual block with an invalid {} signature",
|
||||
item
|
||||
);
|
||||
|
||||
let gossip_verified = harness
|
||||
.chain
|
||||
.verify_block_for_gossip(snapshots[block_index].beacon_block.clone())
|
||||
.expect("should obtain gossip verified block");
|
||||
assert_eq!(
|
||||
harness.chain.process_block(gossip_verified),
|
||||
Err(BlockError::InvalidSignature),
|
||||
"should not import gossip verified block with an invalid {} signature",
|
||||
item
|
||||
);
|
||||
};
|
||||
|
||||
/*
|
||||
* Block proposal
|
||||
*/
|
||||
let mut snapshots = CHAIN_SEGMENT.clone();
|
||||
snapshots[block_index].beacon_block.signature = junk_signature();
|
||||
let blocks = snapshots
|
||||
.iter()
|
||||
.map(|snapshot| snapshot.beacon_block.clone())
|
||||
.collect();
|
||||
// Ensure the block will be rejected if imported in a chain segment.
|
||||
assert_eq!(
|
||||
harness.chain.process_chain_segment(blocks),
|
||||
Err(BlockError::InvalidSignature),
|
||||
"should not import chain segment with an invalid gossip signature",
|
||||
);
|
||||
// Ensure the block will be rejected if imported on its own (without gossip checking).
|
||||
assert_eq!(
|
||||
harness
|
||||
.chain
|
||||
.process_block(snapshots[block_index].beacon_block.clone()),
|
||||
Err(BlockError::InvalidSignature),
|
||||
"should not import individual block with an invalid gossip signature",
|
||||
);
|
||||
|
||||
/*
|
||||
* Randao reveal
|
||||
*/
|
||||
let mut snapshots = CHAIN_SEGMENT.clone();
|
||||
snapshots[block_index]
|
||||
.beacon_block
|
||||
.message
|
||||
.body
|
||||
.randao_reveal = junk_signature();
|
||||
update_parent_roots(&mut snapshots);
|
||||
update_proposal_signatures(&mut snapshots, &harness);
|
||||
assert_invalid_signature(&snapshots, "randao");
|
||||
|
||||
/*
|
||||
* Proposer slashing
|
||||
*/
|
||||
let mut snapshots = CHAIN_SEGMENT.clone();
|
||||
let proposer_slashing = ProposerSlashing {
|
||||
proposer_index: 0,
|
||||
signed_header_1: SignedBeaconBlockHeader {
|
||||
message: snapshots[block_index].beacon_block.message.block_header(),
|
||||
signature: junk_signature(),
|
||||
},
|
||||
signed_header_2: SignedBeaconBlockHeader {
|
||||
message: snapshots[block_index].beacon_block.message.block_header(),
|
||||
signature: junk_signature(),
|
||||
},
|
||||
};
|
||||
snapshots[block_index]
|
||||
.beacon_block
|
||||
.message
|
||||
.body
|
||||
.proposer_slashings
|
||||
.push(proposer_slashing)
|
||||
.expect("should update proposer slashing");
|
||||
update_parent_roots(&mut snapshots);
|
||||
update_proposal_signatures(&mut snapshots, &harness);
|
||||
assert_invalid_signature(&snapshots, "proposer slashing");
|
||||
|
||||
/*
|
||||
* Attester slashing
|
||||
*/
|
||||
let mut snapshots = CHAIN_SEGMENT.clone();
|
||||
let indexed_attestation = IndexedAttestation {
|
||||
attesting_indices: vec![0].into(),
|
||||
data: AttestationData {
|
||||
slot: Slot::new(0),
|
||||
index: 0,
|
||||
beacon_block_root: Hash256::zero(),
|
||||
source: Checkpoint {
|
||||
epoch: Epoch::new(0),
|
||||
root: Hash256::zero(),
|
||||
},
|
||||
target: Checkpoint {
|
||||
epoch: Epoch::new(0),
|
||||
root: Hash256::zero(),
|
||||
},
|
||||
},
|
||||
signature: junk_aggregate_signature(),
|
||||
};
|
||||
let attester_slashing = AttesterSlashing {
|
||||
attestation_1: indexed_attestation.clone(),
|
||||
attestation_2: indexed_attestation,
|
||||
};
|
||||
snapshots[block_index]
|
||||
.beacon_block
|
||||
.message
|
||||
.body
|
||||
.attester_slashings
|
||||
.push(attester_slashing)
|
||||
.expect("should update attester slashing");
|
||||
update_parent_roots(&mut snapshots);
|
||||
update_proposal_signatures(&mut snapshots, &harness);
|
||||
assert_invalid_signature(&snapshots, "attester slashing");
|
||||
|
||||
/*
|
||||
* Attestation
|
||||
*/
|
||||
let mut snapshots = CHAIN_SEGMENT.clone();
|
||||
if let Some(attestation) = snapshots[block_index]
|
||||
.beacon_block
|
||||
.message
|
||||
.body
|
||||
.attestations
|
||||
.get_mut(0)
|
||||
{
|
||||
attestation.signature = junk_aggregate_signature();
|
||||
update_parent_roots(&mut snapshots);
|
||||
update_proposal_signatures(&mut snapshots, &harness);
|
||||
assert_invalid_signature(&snapshots, "attestation");
|
||||
checked_attestation = true;
|
||||
}
|
||||
|
||||
/*
|
||||
* Deposit
|
||||
*
|
||||
* Note: an invalid deposit signature is permitted!
|
||||
*/
|
||||
let mut snapshots = CHAIN_SEGMENT.clone();
|
||||
let deposit = Deposit {
|
||||
proof: vec![Hash256::zero(); DEPOSIT_TREE_DEPTH + 1].into(),
|
||||
data: DepositData {
|
||||
pubkey: Keypair::random().pk.into(),
|
||||
withdrawal_credentials: Hash256::zero(),
|
||||
amount: 0,
|
||||
signature: junk_signature().into(),
|
||||
},
|
||||
};
|
||||
snapshots[block_index]
|
||||
.beacon_block
|
||||
.message
|
||||
.body
|
||||
.deposits
|
||||
.push(deposit)
|
||||
.expect("should update deposit");
|
||||
update_parent_roots(&mut snapshots);
|
||||
update_proposal_signatures(&mut snapshots, &harness);
|
||||
let blocks = snapshots
|
||||
.iter()
|
||||
.map(|snapshot| snapshot.beacon_block.clone())
|
||||
.collect();
|
||||
assert!(
|
||||
harness.chain.process_chain_segment(blocks) != Err(BlockError::InvalidSignature),
|
||||
"should not throw an invalid signature error for a bad deposit signature"
|
||||
);
|
||||
|
||||
/*
|
||||
* Voluntary exit
|
||||
*/
|
||||
let mut snapshots = CHAIN_SEGMENT.clone();
|
||||
let epoch = snapshots[block_index].beacon_state.current_epoch();
|
||||
snapshots[block_index]
|
||||
.beacon_block
|
||||
.message
|
||||
.body
|
||||
.voluntary_exits
|
||||
.push(SignedVoluntaryExit {
|
||||
message: VoluntaryExit {
|
||||
epoch,
|
||||
validator_index: 0,
|
||||
},
|
||||
signature: junk_signature(),
|
||||
})
|
||||
.expect("should update deposit");
|
||||
update_parent_roots(&mut snapshots);
|
||||
update_proposal_signatures(&mut snapshots, &harness);
|
||||
assert_invalid_signature(&snapshots, "voluntary exit");
|
||||
}
|
||||
|
||||
assert!(
|
||||
checked_attestation,
|
||||
"the test should check an attestation signature"
|
||||
)
|
||||
}
|
||||
|
||||
fn unwrap_err<T, E>(result: Result<T, E>) -> E {
|
||||
match result {
|
||||
Ok(_) => panic!("called unwrap_err on Ok"),
|
||||
Err(e) => e,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gossip_verification() {
|
||||
let harness = get_harness(VALIDATOR_COUNT);
|
||||
|
||||
let block_index = CHAIN_SEGMENT_LENGTH - 2;
|
||||
|
||||
harness
|
||||
.chain
|
||||
.slot_clock
|
||||
.set_slot(CHAIN_SEGMENT[block_index].beacon_block.slot().as_u64());
|
||||
|
||||
// Import the ancestors prior to the block we're testing.
|
||||
for snapshot in &CHAIN_SEGMENT[0..block_index] {
|
||||
let gossip_verified = harness
|
||||
.chain
|
||||
.verify_block_for_gossip(snapshot.beacon_block.clone())
|
||||
.expect("should obtain gossip verified block");
|
||||
|
||||
harness
|
||||
.chain
|
||||
.process_block(gossip_verified)
|
||||
.expect("should import valid gossip verfied block");
|
||||
}
|
||||
|
||||
/*
|
||||
* Block with invalid signature
|
||||
*/
|
||||
|
||||
let mut block = CHAIN_SEGMENT[block_index].beacon_block.clone();
|
||||
block.signature = junk_signature();
|
||||
assert_eq!(
|
||||
unwrap_err(harness.chain.verify_block_for_gossip(block)),
|
||||
BlockError::ProposalSignatureInvalid,
|
||||
"should not import a block with an invalid proposal signature"
|
||||
);
|
||||
|
||||
/*
|
||||
* Block from a future slot.
|
||||
*/
|
||||
|
||||
let mut block = CHAIN_SEGMENT[block_index].beacon_block.clone();
|
||||
let block_slot = block.message.slot + 1;
|
||||
block.message.slot = block_slot;
|
||||
assert_eq!(
|
||||
unwrap_err(harness.chain.verify_block_for_gossip(block)),
|
||||
BlockError::FutureSlot {
|
||||
present_slot: block_slot - 1,
|
||||
block_slot
|
||||
},
|
||||
"should not import a block with a future slot"
|
||||
);
|
||||
|
||||
/*
|
||||
* Block from a finalized slot.
|
||||
*/
|
||||
|
||||
let mut block = CHAIN_SEGMENT[block_index].beacon_block.clone();
|
||||
let finalized_slot = harness
|
||||
.chain
|
||||
.head_info()
|
||||
.expect("should get head info")
|
||||
.finalized_checkpoint
|
||||
.epoch
|
||||
.start_slot(E::slots_per_epoch());
|
||||
block.message.slot = finalized_slot;
|
||||
assert_eq!(
|
||||
unwrap_err(harness.chain.verify_block_for_gossip(block)),
|
||||
BlockError::WouldRevertFinalizedSlot {
|
||||
block_slot: finalized_slot,
|
||||
finalized_slot
|
||||
},
|
||||
"should not import a block with a finalized slot"
|
||||
);
|
||||
}
|
||||
@ -3,13 +3,10 @@
|
||||
#[macro_use]
|
||||
extern crate lazy_static;
|
||||
|
||||
use beacon_chain::AttestationProcessingOutcome;
|
||||
use beacon_chain::{
|
||||
test_utils::{
|
||||
AttestationStrategy, BeaconChainHarness, BlockStrategy, HarnessType, OP_POOL_DB_KEY,
|
||||
},
|
||||
BlockProcessingOutcome,
|
||||
use beacon_chain::test_utils::{
|
||||
AttestationStrategy, BeaconChainHarness, BlockStrategy, HarnessType, OP_POOL_DB_KEY,
|
||||
};
|
||||
use beacon_chain::AttestationProcessingOutcome;
|
||||
use operation_pool::PersistedOperationPool;
|
||||
use state_processing::{
|
||||
per_slot_processing, per_slot_processing::Error as SlotProcessingError, EpochProcessingError,
|
||||
@ -562,15 +559,13 @@ fn run_skip_slot_test(skip_slots: u64) {
|
||||
.head()
|
||||
.expect("should get head")
|
||||
.beacon_block
|
||||
.clone()
|
||||
.clone(),
|
||||
),
|
||||
Ok(BlockProcessingOutcome::Processed {
|
||||
block_root: harness_a
|
||||
.chain
|
||||
.head()
|
||||
.expect("should get head")
|
||||
.beacon_block_root
|
||||
})
|
||||
Ok(harness_a
|
||||
.chain
|
||||
.head()
|
||||
.expect("should get head")
|
||||
.beacon_block_root)
|
||||
);
|
||||
|
||||
harness_b
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "client"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Age Manning <Age@AgeManning.com>"]
|
||||
edition = "2018"
|
||||
|
||||
@ -12,6 +12,7 @@ toml = "^0.5"
|
||||
beacon_chain = { path = "../beacon_chain" }
|
||||
store = { path = "../store" }
|
||||
network = { path = "../network" }
|
||||
timer = { path = "../timer" }
|
||||
eth2-libp2p = { path = "../eth2-libp2p" }
|
||||
rest_api = { path = "../rest_api" }
|
||||
parking_lot = "0.9.0"
|
||||
@ -29,7 +30,6 @@ slog = { version = "2.5.2", features = ["max_level_trace"] }
|
||||
slog-async = "2.3.0"
|
||||
tokio = "0.1.22"
|
||||
dirs = "2.0.2"
|
||||
exit-future = "0.1.4"
|
||||
futures = "0.1.29"
|
||||
reqwest = "0.9.22"
|
||||
url = "2.1.0"
|
||||
@ -38,3 +38,5 @@ genesis = { path = "../genesis" }
|
||||
environment = { path = "../../lighthouse/environment" }
|
||||
lighthouse_bootstrap = { path = "../../eth2/utils/lighthouse_bootstrap" }
|
||||
eth2_ssz = { path = "../../eth2/utils/ssz" }
|
||||
lazy_static = "1.4.0"
|
||||
lighthouse_metrics = { path = "../../eth2/utils/lighthouse_metrics" }
|
||||
|
||||
@ -14,13 +14,13 @@ use beacon_chain::{
|
||||
use environment::RuntimeContext;
|
||||
use eth1::{Config as Eth1Config, Service as Eth1Service};
|
||||
use eth2_config::Eth2Config;
|
||||
use exit_future::Signal;
|
||||
use eth2_libp2p::NetworkGlobals;
|
||||
use futures::{future, Future, IntoFuture};
|
||||
use genesis::{
|
||||
generate_deterministic_keypairs, interop_genesis_state, state_from_ssz_file, Eth1GenesisService,
|
||||
};
|
||||
use lighthouse_bootstrap::Bootstrapper;
|
||||
use network::{NetworkConfig, NetworkMessage, Service as NetworkService};
|
||||
use network::{NetworkConfig, NetworkMessage, NetworkService};
|
||||
use slog::info;
|
||||
use ssz::Decode;
|
||||
use std::net::SocketAddr;
|
||||
@ -56,10 +56,10 @@ pub struct ClientBuilder<T: BeaconChainTypes> {
|
||||
beacon_chain_builder: Option<BeaconChainBuilder<T>>,
|
||||
beacon_chain: Option<Arc<BeaconChain<T>>>,
|
||||
eth1_service: Option<Eth1Service>,
|
||||
exit_signals: Vec<Signal>,
|
||||
exit_channels: Vec<tokio::sync::oneshot::Sender<()>>,
|
||||
event_handler: Option<T::EventHandler>,
|
||||
libp2p_network: Option<Arc<NetworkService<T>>>,
|
||||
libp2p_network_send: Option<UnboundedSender<NetworkMessage>>,
|
||||
network_globals: Option<Arc<NetworkGlobals<T::EthSpec>>>,
|
||||
network_send: Option<UnboundedSender<NetworkMessage<T::EthSpec>>>,
|
||||
http_listen_addr: Option<SocketAddr>,
|
||||
websocket_listen_addr: Option<SocketAddr>,
|
||||
eth_spec_instance: T::EthSpec,
|
||||
@ -90,10 +90,10 @@ where
|
||||
beacon_chain_builder: None,
|
||||
beacon_chain: None,
|
||||
eth1_service: None,
|
||||
exit_signals: vec![],
|
||||
exit_channels: vec![],
|
||||
event_handler: None,
|
||||
libp2p_network: None,
|
||||
libp2p_network_send: None,
|
||||
network_globals: None,
|
||||
network_send: None,
|
||||
http_listen_addr: None,
|
||||
websocket_listen_addr: None,
|
||||
eth_spec_instance,
|
||||
@ -249,24 +249,55 @@ where
|
||||
})
|
||||
}
|
||||
|
||||
/// Immediately starts the libp2p networking stack.
|
||||
pub fn libp2p_network(mut self, config: &NetworkConfig) -> Result<Self, String> {
|
||||
/// Immediately starts the networking stack.
|
||||
pub fn network(mut self, config: &NetworkConfig) -> Result<Self, String> {
|
||||
let beacon_chain = self
|
||||
.beacon_chain
|
||||
.clone()
|
||||
.ok_or_else(|| "libp2p_network requires a beacon chain")?;
|
||||
.ok_or_else(|| "network requires a beacon chain")?;
|
||||
let context = self
|
||||
.runtime_context
|
||||
.as_ref()
|
||||
.ok_or_else(|| "libp2p_network requires a runtime_context")?
|
||||
.ok_or_else(|| "network requires a runtime_context")?
|
||||
.service_context("network".into());
|
||||
|
||||
let (network, network_send) =
|
||||
NetworkService::new(beacon_chain, config, &context.executor, context.log)
|
||||
.map_err(|e| format!("Failed to start libp2p network: {:?}", e))?;
|
||||
let (network_globals, network_send, network_exit) =
|
||||
NetworkService::start(beacon_chain, config, &context.executor, context.log)
|
||||
.map_err(|e| format!("Failed to start network: {:?}", e))?;
|
||||
|
||||
self.libp2p_network = Some(network);
|
||||
self.libp2p_network_send = Some(network_send);
|
||||
self.network_globals = Some(network_globals);
|
||||
self.network_send = Some(network_send);
|
||||
self.exit_channels.push(network_exit);
|
||||
|
||||
Ok(self)
|
||||
}
|
||||
|
||||
/// Immediately starts the timer service.
|
||||
fn timer(mut self) -> Result<Self, String> {
|
||||
let context = self
|
||||
.runtime_context
|
||||
.as_ref()
|
||||
.ok_or_else(|| "node timer requires a runtime_context")?
|
||||
.service_context("node_timer".into());
|
||||
let beacon_chain = self
|
||||
.beacon_chain
|
||||
.clone()
|
||||
.ok_or_else(|| "node timer requires a beacon chain")?;
|
||||
let milliseconds_per_slot = self
|
||||
.chain_spec
|
||||
.as_ref()
|
||||
.ok_or_else(|| "node timer requires a chain spec".to_string())?
|
||||
.milliseconds_per_slot;
|
||||
|
||||
let timer_exit = timer::spawn(
|
||||
&context.executor,
|
||||
beacon_chain,
|
||||
milliseconds_per_slot,
|
||||
context.log,
|
||||
)
|
||||
.map_err(|e| format!("Unable to start node timer: {}", e))?;
|
||||
|
||||
self.exit_channels.push(timer_exit);
|
||||
|
||||
Ok(self)
|
||||
}
|
||||
@ -286,21 +317,21 @@ where
|
||||
.as_ref()
|
||||
.ok_or_else(|| "http_server requires a runtime_context")?
|
||||
.service_context("http".into());
|
||||
let network = self
|
||||
.libp2p_network
|
||||
let network_globals = self
|
||||
.network_globals
|
||||
.clone()
|
||||
.ok_or_else(|| "http_server requires a libp2p network")?;
|
||||
let network_send = self
|
||||
.libp2p_network_send
|
||||
.network_send
|
||||
.clone()
|
||||
.ok_or_else(|| "http_server requires a libp2p network sender")?;
|
||||
|
||||
let network_info = rest_api::NetworkInfo {
|
||||
network_service: network,
|
||||
network_globals,
|
||||
network_chan: network_send,
|
||||
};
|
||||
|
||||
let (exit_signal, listening_addr) = rest_api::start_server(
|
||||
let (exit_channel, listening_addr) = rest_api::start_server(
|
||||
&client_config.rest_api,
|
||||
&context.executor,
|
||||
beacon_chain,
|
||||
@ -316,7 +347,7 @@ where
|
||||
)
|
||||
.map_err(|e| format!("Failed to start HTTP API: {:?}", e))?;
|
||||
|
||||
self.exit_signals.push(exit_signal);
|
||||
self.exit_channels.push(exit_channel);
|
||||
self.http_listen_addr = Some(listening_addr);
|
||||
|
||||
Ok(self)
|
||||
@ -333,8 +364,8 @@ where
|
||||
.beacon_chain
|
||||
.clone()
|
||||
.ok_or_else(|| "slot_notifier requires a beacon chain")?;
|
||||
let network = self
|
||||
.libp2p_network
|
||||
let network_globals = self
|
||||
.network_globals
|
||||
.clone()
|
||||
.ok_or_else(|| "slot_notifier requires a libp2p network")?;
|
||||
let milliseconds_per_slot = self
|
||||
@ -343,10 +374,15 @@ where
|
||||
.ok_or_else(|| "slot_notifier requires a chain spec".to_string())?
|
||||
.milliseconds_per_slot;
|
||||
|
||||
let exit_signal = spawn_notifier(context, beacon_chain, network, milliseconds_per_slot)
|
||||
.map_err(|e| format!("Unable to start slot notifier: {}", e))?;
|
||||
let exit_channel = spawn_notifier(
|
||||
context,
|
||||
beacon_chain,
|
||||
network_globals,
|
||||
milliseconds_per_slot,
|
||||
)
|
||||
.map_err(|e| format!("Unable to start slot notifier: {}", e))?;
|
||||
|
||||
self.exit_signals.push(exit_signal);
|
||||
self.exit_channels.push(exit_channel);
|
||||
|
||||
Ok(self)
|
||||
}
|
||||
@ -361,10 +397,10 @@ where
|
||||
{
|
||||
Client {
|
||||
beacon_chain: self.beacon_chain,
|
||||
libp2p_network: self.libp2p_network,
|
||||
network_globals: self.network_globals,
|
||||
http_listen_addr: self.http_listen_addr,
|
||||
websocket_listen_addr: self.websocket_listen_addr,
|
||||
_exit_signals: self.exit_signals,
|
||||
_exit_channels: self.exit_channels,
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -404,7 +440,8 @@ where
|
||||
self.beacon_chain_builder = None;
|
||||
self.event_handler = None;
|
||||
|
||||
Ok(self)
|
||||
// a beacon chain requires a timer
|
||||
self.timer()
|
||||
}
|
||||
}
|
||||
|
||||
@ -434,7 +471,7 @@ where
|
||||
.ok_or_else(|| "websocket_event_handler requires a runtime_context")?
|
||||
.service_context("ws".into());
|
||||
|
||||
let (sender, exit_signal, listening_addr): (
|
||||
let (sender, exit_channel, listening_addr): (
|
||||
WebSocketSender<TEthSpec>,
|
||||
Option<_>,
|
||||
Option<_>,
|
||||
@ -446,8 +483,8 @@ where
|
||||
(WebSocketSender::dummy(), None, None)
|
||||
};
|
||||
|
||||
if let Some(signal) = exit_signal {
|
||||
self.exit_signals.push(signal);
|
||||
if let Some(channel) = exit_channel {
|
||||
self.exit_channels.push(channel);
|
||||
}
|
||||
self.event_handler = Some(sender);
|
||||
self.websocket_listen_addr = listening_addr;
|
||||
@ -648,8 +685,8 @@ where
|
||||
self.eth1_service = None;
|
||||
|
||||
let exit = {
|
||||
let (tx, rx) = exit_future::signal();
|
||||
self.exit_signals.push(tx);
|
||||
let (tx, rx) = tokio::sync::oneshot::channel();
|
||||
self.exit_channels.push(tx);
|
||||
rx
|
||||
};
|
||||
|
||||
@ -711,7 +748,7 @@ where
|
||||
.ok_or_else(|| "system_time_slot_clock requires a beacon_chain_builder")?;
|
||||
|
||||
let genesis_time = beacon_chain_builder
|
||||
.finalized_checkpoint
|
||||
.finalized_snapshot
|
||||
.as_ref()
|
||||
.ok_or_else(|| "system_time_slot_clock requires an initialized beacon state")?
|
||||
.beacon_state
|
||||
|
||||
@ -1,15 +1,14 @@
|
||||
extern crate slog;
|
||||
|
||||
pub mod config;
|
||||
mod metrics;
|
||||
mod notifier;
|
||||
|
||||
pub mod builder;
|
||||
pub mod error;
|
||||
|
||||
use beacon_chain::BeaconChain;
|
||||
use eth2_libp2p::{Enr, Multiaddr};
|
||||
use exit_future::Signal;
|
||||
use network::Service as NetworkService;
|
||||
use eth2_libp2p::{Enr, Multiaddr, NetworkGlobals};
|
||||
use std::net::SocketAddr;
|
||||
use std::sync::Arc;
|
||||
|
||||
@ -23,11 +22,11 @@ pub use eth2_config::Eth2Config;
|
||||
/// Holds references to running services, cleanly shutting them down when dropped.
|
||||
pub struct Client<T: BeaconChainTypes> {
|
||||
beacon_chain: Option<Arc<BeaconChain<T>>>,
|
||||
libp2p_network: Option<Arc<NetworkService<T>>>,
|
||||
network_globals: Option<Arc<NetworkGlobals<T::EthSpec>>>,
|
||||
http_listen_addr: Option<SocketAddr>,
|
||||
websocket_listen_addr: Option<SocketAddr>,
|
||||
/// Exit signals will "fire" when dropped, causing each service to exit gracefully.
|
||||
_exit_signals: Vec<Signal>,
|
||||
/// Exit channels will complete/error when dropped, causing each service to exit gracefully.
|
||||
_exit_channels: Vec<tokio::sync::oneshot::Sender<()>>,
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> Client<T> {
|
||||
@ -48,16 +47,16 @@ impl<T: BeaconChainTypes> Client<T> {
|
||||
|
||||
/// Returns the port of the client's libp2p stack, if it was started.
|
||||
pub fn libp2p_listen_port(&self) -> Option<u16> {
|
||||
self.libp2p_network.as_ref().map(|n| n.listen_port())
|
||||
self.network_globals.as_ref().map(|n| n.listen_port_tcp())
|
||||
}
|
||||
|
||||
/// Returns the list of libp2p addresses the client is listening to.
|
||||
pub fn libp2p_listen_addresses(&self) -> Option<Vec<Multiaddr>> {
|
||||
self.libp2p_network.as_ref().map(|n| n.listen_multiaddrs())
|
||||
self.network_globals.as_ref().map(|n| n.listen_multiaddrs())
|
||||
}
|
||||
|
||||
/// Returns the local libp2p ENR of this node, for network discovery.
|
||||
pub fn enr(&self) -> Option<Enr> {
|
||||
self.libp2p_network.as_ref()?.local_enr()
|
||||
self.network_globals.as_ref()?.local_enr()
|
||||
}
|
||||
}
|
||||
|
||||
9
beacon_node/client/src/metrics.rs
Normal file
9
beacon_node/client/src/metrics.rs
Normal file
@ -0,0 +1,9 @@
|
||||
use lazy_static::lazy_static;
|
||||
pub use lighthouse_metrics::*;
|
||||
|
||||
lazy_static! {
|
||||
pub static ref SYNC_SLOTS_PER_SECOND: Result<IntGauge> = try_create_int_gauge(
|
||||
"sync_slots_per_second",
|
||||
"The number of blocks being imported per second"
|
||||
);
|
||||
}
|
||||
@ -1,8 +1,8 @@
|
||||
use crate::metrics;
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use environment::RuntimeContext;
|
||||
use exit_future::Signal;
|
||||
use eth2_libp2p::NetworkGlobals;
|
||||
use futures::{Future, Stream};
|
||||
use network::Service as NetworkService;
|
||||
use parking_lot::Mutex;
|
||||
use slog::{debug, error, info, warn};
|
||||
use slot_clock::SlotClock;
|
||||
@ -29,9 +29,9 @@ const SPEEDO_OBSERVATIONS: usize = 4;
|
||||
pub fn spawn_notifier<T: BeaconChainTypes>(
|
||||
context: RuntimeContext<T::EthSpec>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network: Arc<NetworkService<T>>,
|
||||
network: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
milliseconds_per_slot: u64,
|
||||
) -> Result<Signal, String> {
|
||||
) -> Result<tokio::sync::oneshot::Sender<()>, String> {
|
||||
let log_1 = context.log.clone();
|
||||
let log_2 = context.log.clone();
|
||||
let log_3 = context.log.clone();
|
||||
@ -83,6 +83,8 @@ pub fn spawn_notifier<T: BeaconChainTypes>(
|
||||
let mut speedo = speedo.lock();
|
||||
speedo.observe(head_slot, Instant::now());
|
||||
|
||||
metrics::set_gauge(&metrics::SYNC_SLOTS_PER_SECOND, speedo.slots_per_second().unwrap_or_else(|| 0_f64) as i64);
|
||||
|
||||
// The next two lines take advantage of saturating subtraction on `Slot`.
|
||||
let head_distance = current_slot - head_slot;
|
||||
|
||||
@ -164,10 +166,11 @@ pub fn spawn_notifier<T: BeaconChainTypes>(
|
||||
Ok(())
|
||||
} } });
|
||||
|
||||
let (exit_signal, exit) = exit_future::signal();
|
||||
let (exit_signal, exit) = tokio::sync::oneshot::channel();
|
||||
|
||||
context
|
||||
.executor
|
||||
.spawn(exit.until(interval_future).map(|_| ()));
|
||||
.spawn(interval_future.select(exit).map(|_| ()).map_err(|_| ()));
|
||||
|
||||
Ok(exit_signal)
|
||||
}
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "eth1"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Paul Hauner <paul@paulhauner.com>"]
|
||||
edition = "2018"
|
||||
|
||||
@ -26,7 +26,6 @@ parking_lot = "0.7"
|
||||
slog = "^2.2.3"
|
||||
tokio = "0.1.22"
|
||||
state_processing = { path = "../../eth2/state_processing" }
|
||||
exit-future = "0.1.4"
|
||||
libflate = "0.1"
|
||||
lighthouse_metrics = { path = "../../eth2/utils/lighthouse_metrics"}
|
||||
lazy_static = "1.4.0"
|
||||
|
||||
@ -6,7 +6,6 @@ use crate::{
|
||||
inner::{DepositUpdater, Inner},
|
||||
DepositLog,
|
||||
};
|
||||
use exit_future::Exit;
|
||||
use futures::{
|
||||
future::{loop_fn, Loop},
|
||||
stream, Future, Stream,
|
||||
@ -314,7 +313,10 @@ impl Service {
|
||||
/// - Err(_) if there is an error.
|
||||
///
|
||||
/// Emits logs for debugging and errors.
|
||||
pub fn auto_update(&self, exit: Exit) -> impl Future<Item = (), Error = ()> {
|
||||
pub fn auto_update(
|
||||
&self,
|
||||
exit: tokio::sync::oneshot::Receiver<()>,
|
||||
) -> impl Future<Item = (), Error = ()> {
|
||||
let service = self.clone();
|
||||
let log = self.log.clone();
|
||||
let update_interval = Duration::from_millis(self.config().auto_update_interval_millis);
|
||||
@ -360,7 +362,7 @@ impl Service {
|
||||
})
|
||||
});
|
||||
|
||||
exit.until(loop_future).map(|_: Option<()>| ())
|
||||
loop_future.select(exit).map(|_| ()).map_err(|_| ())
|
||||
}
|
||||
|
||||
/// Contacts the remote eth1 node and attempts to import deposit logs up to the configured
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "eth2-libp2p"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Age Manning <Age@AgeManning.com>"]
|
||||
edition = "2018"
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
use crate::discovery::Discovery;
|
||||
use crate::rpc::{RPCEvent, RPCMessage, RPC};
|
||||
use crate::{error, GossipTopic, NetworkConfig, NetworkGlobals, Topic, TopicHash};
|
||||
use crate::{error, GossipTopic, NetworkConfig, NetworkGlobals, PubsubMessage, TopicHash};
|
||||
use enr::Enr;
|
||||
use futures::prelude::*;
|
||||
use libp2p::{
|
||||
@ -8,16 +8,14 @@ use libp2p::{
|
||||
discv5::Discv5Event,
|
||||
gossipsub::{Gossipsub, GossipsubEvent, MessageId},
|
||||
identify::{Identify, IdentifyEvent},
|
||||
ping::{Ping, PingConfig, PingEvent},
|
||||
swarm::{NetworkBehaviourAction, NetworkBehaviourEventProcess},
|
||||
tokio_io::{AsyncRead, AsyncWrite},
|
||||
NetworkBehaviour, PeerId,
|
||||
};
|
||||
use lru::LruCache;
|
||||
use slog::{debug, o};
|
||||
use std::num::NonZeroU32;
|
||||
use slog::{debug, o, warn};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use types::EthSpec;
|
||||
|
||||
const MAX_IDENTIFY_ADDRESSES: usize = 20;
|
||||
|
||||
@ -25,48 +23,43 @@ const MAX_IDENTIFY_ADDRESSES: usize = 20;
|
||||
/// This core behaviour is managed by `Behaviour` which adds peer management to all core
|
||||
/// behaviours.
|
||||
#[derive(NetworkBehaviour)]
|
||||
#[behaviour(out_event = "BehaviourEvent", poll_method = "poll")]
|
||||
pub struct Behaviour<TSubstream: AsyncRead + AsyncWrite> {
|
||||
#[behaviour(out_event = "BehaviourEvent<TSpec>", poll_method = "poll")]
|
||||
pub struct Behaviour<TSubstream: AsyncRead + AsyncWrite, TSpec: EthSpec> {
|
||||
/// The routing pub-sub mechanism for eth2.
|
||||
gossipsub: Gossipsub<TSubstream>,
|
||||
/// The Eth2 RPC specified in the wire-0 protocol.
|
||||
eth2_rpc: RPC<TSubstream>,
|
||||
eth2_rpc: RPC<TSubstream, TSpec>,
|
||||
/// Keep regular connection to peers and disconnect if absent.
|
||||
// TODO: Remove Libp2p ping in favour of discv5 ping.
|
||||
ping: Ping<TSubstream>,
|
||||
// TODO: Using id for initial interop. This will be removed by mainnet.
|
||||
/// Provides IP addresses and peer information.
|
||||
identify: Identify<TSubstream>,
|
||||
/// Discovery behaviour.
|
||||
discovery: Discovery<TSubstream>,
|
||||
discovery: Discovery<TSubstream, TSpec>,
|
||||
/// The events generated by this behaviour to be consumed in the swarm poll.
|
||||
#[behaviour(ignore)]
|
||||
events: Vec<BehaviourEvent>,
|
||||
events: Vec<BehaviourEvent<TSpec>>,
|
||||
/// A cache of recently seen gossip messages. This is used to filter out any possible
|
||||
/// duplicates that may still be seen over gossipsub.
|
||||
#[behaviour(ignore)]
|
||||
seen_gossip_messages: LruCache<MessageId, ()>,
|
||||
/// A collections of variables accessible outside the network service.
|
||||
#[behaviour(ignore)]
|
||||
network_globals: Arc<NetworkGlobals<TSpec>>,
|
||||
#[behaviour(ignore)]
|
||||
/// Logger for behaviour actions.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
impl<TSubstream: AsyncRead + AsyncWrite> Behaviour<TSubstream> {
|
||||
impl<TSubstream: AsyncRead + AsyncWrite, TSpec: EthSpec> Behaviour<TSubstream, TSpec> {
|
||||
pub fn new(
|
||||
local_key: &Keypair,
|
||||
net_conf: &NetworkConfig,
|
||||
network_globals: Arc<NetworkGlobals>,
|
||||
network_globals: Arc<NetworkGlobals<TSpec>>,
|
||||
log: &slog::Logger,
|
||||
) -> error::Result<Self> {
|
||||
let local_peer_id = local_key.public().into_peer_id();
|
||||
let behaviour_log = log.new(o!());
|
||||
|
||||
let ping_config = PingConfig::new()
|
||||
.with_timeout(Duration::from_secs(30))
|
||||
.with_interval(Duration::from_secs(20))
|
||||
.with_max_failures(NonZeroU32::new(2).expect("2 != 0"))
|
||||
.with_keep_alive(false);
|
||||
|
||||
let identify = Identify::new(
|
||||
"lighthouse/libp2p".into(),
|
||||
version::version(),
|
||||
@ -76,16 +69,16 @@ impl<TSubstream: AsyncRead + AsyncWrite> Behaviour<TSubstream> {
|
||||
Ok(Behaviour {
|
||||
eth2_rpc: RPC::new(log.clone()),
|
||||
gossipsub: Gossipsub::new(local_peer_id, net_conf.gs_config.clone()),
|
||||
discovery: Discovery::new(local_key, net_conf, network_globals, log)?,
|
||||
ping: Ping::new(ping_config),
|
||||
discovery: Discovery::new(local_key, net_conf, network_globals.clone(), log)?,
|
||||
identify,
|
||||
events: Vec::new(),
|
||||
seen_gossip_messages: LruCache::new(100_000),
|
||||
network_globals,
|
||||
log: behaviour_log,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn discovery(&self) -> &Discovery<TSubstream> {
|
||||
pub fn discovery(&self) -> &Discovery<TSubstream, TSpec> {
|
||||
&self.discovery
|
||||
}
|
||||
|
||||
@ -95,26 +88,31 @@ impl<TSubstream: AsyncRead + AsyncWrite> Behaviour<TSubstream> {
|
||||
}
|
||||
|
||||
// Implement the NetworkBehaviourEventProcess trait so that we can derive NetworkBehaviour for Behaviour
|
||||
impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<GossipsubEvent>
|
||||
for Behaviour<TSubstream>
|
||||
impl<TSubstream: AsyncRead + AsyncWrite, TSpec: EthSpec>
|
||||
NetworkBehaviourEventProcess<GossipsubEvent> for Behaviour<TSubstream, TSpec>
|
||||
{
|
||||
fn inject_event(&mut self, event: GossipsubEvent) {
|
||||
match event {
|
||||
GossipsubEvent::Message(propagation_source, id, gs_msg) => {
|
||||
let msg = PubsubMessage::from_topics(&gs_msg.topics, gs_msg.data);
|
||||
|
||||
// Note: We are keeping track here of the peer that sent us the message, not the
|
||||
// peer that originally published the message.
|
||||
if self.seen_gossip_messages.put(id.clone(), ()).is_none() {
|
||||
// if this message isn't a duplicate, notify the network
|
||||
self.events.push(BehaviourEvent::GossipMessage {
|
||||
id,
|
||||
source: propagation_source,
|
||||
topics: gs_msg.topics,
|
||||
message: msg,
|
||||
});
|
||||
match PubsubMessage::decode(&gs_msg.topics, &gs_msg.data) {
|
||||
Err(e) => {
|
||||
debug!(self.log, "Could not decode gossipsub message"; "error" => format!("{}", e))
|
||||
}
|
||||
Ok(msg) => {
|
||||
// if this message isn't a duplicate, notify the network
|
||||
self.events.push(BehaviourEvent::GossipMessage {
|
||||
id,
|
||||
source: propagation_source,
|
||||
topics: gs_msg.topics,
|
||||
message: msg,
|
||||
});
|
||||
}
|
||||
}
|
||||
} else {
|
||||
debug!(self.log, "A duplicate message was received"; "message" => format!("{:?}", msg));
|
||||
warn!(self.log, "A duplicate gossipsub message was received"; "message" => format!("{:?}", gs_msg));
|
||||
}
|
||||
}
|
||||
GossipsubEvent::Subscribed { peer_id, topic } => {
|
||||
@ -126,10 +124,10 @@ impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<GossipsubE
|
||||
}
|
||||
}
|
||||
|
||||
impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<RPCMessage>
|
||||
for Behaviour<TSubstream>
|
||||
impl<TSubstream: AsyncRead + AsyncWrite, TSpec: EthSpec>
|
||||
NetworkBehaviourEventProcess<RPCMessage<TSpec>> for Behaviour<TSubstream, TSpec>
|
||||
{
|
||||
fn inject_event(&mut self, event: RPCMessage) {
|
||||
fn inject_event(&mut self, event: RPCMessage<TSpec>) {
|
||||
match event {
|
||||
RPCMessage::PeerDialed(peer_id) => {
|
||||
self.events.push(BehaviourEvent::PeerDialed(peer_id))
|
||||
@ -144,19 +142,11 @@ impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<RPCMessage
|
||||
}
|
||||
}
|
||||
|
||||
impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<PingEvent>
|
||||
for Behaviour<TSubstream>
|
||||
{
|
||||
fn inject_event(&mut self, _event: PingEvent) {
|
||||
// not interested in ping responses at the moment.
|
||||
}
|
||||
}
|
||||
|
||||
impl<TSubstream: AsyncRead + AsyncWrite> Behaviour<TSubstream> {
|
||||
impl<TSubstream: AsyncRead + AsyncWrite, TSpec: EthSpec> Behaviour<TSubstream, TSpec> {
|
||||
/// Consumes the events list when polled.
|
||||
fn poll<TBehaviourIn>(
|
||||
&mut self,
|
||||
) -> Async<NetworkBehaviourAction<TBehaviourIn, BehaviourEvent>> {
|
||||
) -> Async<NetworkBehaviourAction<TBehaviourIn, BehaviourEvent<TSpec>>> {
|
||||
if !self.events.is_empty() {
|
||||
return Async::Ready(NetworkBehaviourAction::GenerateEvent(self.events.remove(0)));
|
||||
}
|
||||
@ -165,8 +155,8 @@ impl<TSubstream: AsyncRead + AsyncWrite> Behaviour<TSubstream> {
|
||||
}
|
||||
}
|
||||
|
||||
impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<IdentifyEvent>
|
||||
for Behaviour<TSubstream>
|
||||
impl<TSubstream: AsyncRead + AsyncWrite, TSpec: EthSpec> NetworkBehaviourEventProcess<IdentifyEvent>
|
||||
for Behaviour<TSubstream, TSpec>
|
||||
{
|
||||
fn inject_event(&mut self, event: IdentifyEvent) {
|
||||
match event {
|
||||
@ -196,8 +186,8 @@ impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<IdentifyEv
|
||||
}
|
||||
}
|
||||
|
||||
impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<Discv5Event>
|
||||
for Behaviour<TSubstream>
|
||||
impl<TSubstream: AsyncRead + AsyncWrite, TSpec: EthSpec> NetworkBehaviourEventProcess<Discv5Event>
|
||||
for Behaviour<TSubstream, TSpec>
|
||||
{
|
||||
fn inject_event(&mut self, _event: Discv5Event) {
|
||||
// discv5 has no events to inject
|
||||
@ -205,24 +195,49 @@ impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<Discv5Even
|
||||
}
|
||||
|
||||
/// Implements the combined behaviour for the libp2p service.
|
||||
impl<TSubstream: AsyncRead + AsyncWrite> Behaviour<TSubstream> {
|
||||
impl<TSubstream: AsyncRead + AsyncWrite, TSpec: EthSpec> Behaviour<TSubstream, TSpec> {
|
||||
/* Pubsub behaviour functions */
|
||||
|
||||
/// Subscribes to a gossipsub topic.
|
||||
pub fn subscribe(&mut self, topic: Topic) -> bool {
|
||||
self.gossipsub.subscribe(topic)
|
||||
pub fn subscribe(&mut self, topic: GossipTopic) -> bool {
|
||||
if !self
|
||||
.network_globals
|
||||
.gossipsub_subscriptions
|
||||
.read()
|
||||
.contains(&topic)
|
||||
{
|
||||
self.network_globals
|
||||
.gossipsub_subscriptions
|
||||
.write()
|
||||
.push(topic.clone());
|
||||
}
|
||||
self.gossipsub.subscribe(topic.into())
|
||||
}
|
||||
|
||||
/// Unsubscribe from a gossipsub topic.
|
||||
pub fn unsubscribe(&mut self, topic: Topic) -> bool {
|
||||
self.gossipsub.unsubscribe(topic)
|
||||
pub fn unsubscribe(&mut self, topic: GossipTopic) -> bool {
|
||||
let pos = self
|
||||
.network_globals
|
||||
.gossipsub_subscriptions
|
||||
.read()
|
||||
.iter()
|
||||
.position(|s| s == &topic);
|
||||
if let Some(pos) = pos {
|
||||
self.network_globals
|
||||
.gossipsub_subscriptions
|
||||
.write()
|
||||
.swap_remove(pos);
|
||||
}
|
||||
self.gossipsub.unsubscribe(topic.into())
|
||||
}
|
||||
|
||||
/// Publishes a message on the pubsub (gossipsub) behaviour.
|
||||
pub fn publish(&mut self, topics: &[Topic], message: PubsubMessage) {
|
||||
let message_data = message.into_data();
|
||||
for topic in topics {
|
||||
self.gossipsub.publish(topic, message_data.clone());
|
||||
/// Publishes a list of messages on the pubsub (gossipsub) behaviour, choosing the encoding.
|
||||
pub fn publish(&mut self, messages: Vec<PubsubMessage<TSpec>>) {
|
||||
for message in messages {
|
||||
for topic in message.topics() {
|
||||
let message_data = message.encode();
|
||||
self.gossipsub.publish(&topic.into(), message_data);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -236,14 +251,15 @@ impl<TSubstream: AsyncRead + AsyncWrite> Behaviour<TSubstream> {
|
||||
/* Eth2 RPC behaviour functions */
|
||||
|
||||
/// Sends an RPC Request/Response via the RPC protocol.
|
||||
pub fn send_rpc(&mut self, peer_id: PeerId, rpc_event: RPCEvent) {
|
||||
pub fn send_rpc(&mut self, peer_id: PeerId, rpc_event: RPCEvent<TSpec>) {
|
||||
self.eth2_rpc.send_rpc(peer_id, rpc_event);
|
||||
}
|
||||
|
||||
/* Discovery / Peer management functions */
|
||||
/// Return the list of currently connected peers.
|
||||
|
||||
/// The current number of connected libp2p peers.
|
||||
pub fn connected_peers(&self) -> usize {
|
||||
self.discovery.connected_peers()
|
||||
self.network_globals.connected_peers()
|
||||
}
|
||||
|
||||
/// Notify discovery that the peer has been banned.
|
||||
@ -268,9 +284,9 @@ impl<TSubstream: AsyncRead + AsyncWrite> Behaviour<TSubstream> {
|
||||
}
|
||||
|
||||
/// The types of events than can be obtained from polling the behaviour.
|
||||
pub enum BehaviourEvent {
|
||||
pub enum BehaviourEvent<TSpec: EthSpec> {
|
||||
/// A received RPC event and the peer that it was received from.
|
||||
RPC(PeerId, RPCEvent),
|
||||
RPC(PeerId, RPCEvent<TSpec>),
|
||||
/// We have completed an initial connection to a new peer.
|
||||
PeerDialed(PeerId),
|
||||
/// A peer has disconnected.
|
||||
@ -284,60 +300,8 @@ pub enum BehaviourEvent {
|
||||
/// The topics that this message was sent on.
|
||||
topics: Vec<TopicHash>,
|
||||
/// The message itself.
|
||||
message: PubsubMessage,
|
||||
message: PubsubMessage<TSpec>,
|
||||
},
|
||||
/// Subscribed to peer for given topic
|
||||
PeerSubscribed(PeerId, TopicHash),
|
||||
}
|
||||
|
||||
/// Messages that are passed to and from the pubsub (Gossipsub) behaviour. These are encoded and
|
||||
/// decoded upstream.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
|
||||
pub enum PubsubMessage {
|
||||
/// Gossipsub message providing notification of a new block.
|
||||
Block(Vec<u8>),
|
||||
/// Gossipsub message providing notification of a new attestation.
|
||||
Attestation(Vec<u8>),
|
||||
/// Gossipsub message providing notification of a voluntary exit.
|
||||
VoluntaryExit(Vec<u8>),
|
||||
/// Gossipsub message providing notification of a new proposer slashing.
|
||||
ProposerSlashing(Vec<u8>),
|
||||
/// Gossipsub message providing notification of a new attester slashing.
|
||||
AttesterSlashing(Vec<u8>),
|
||||
/// Gossipsub message from an unknown topic.
|
||||
Unknown(Vec<u8>),
|
||||
}
|
||||
|
||||
impl PubsubMessage {
|
||||
/* Note: This is assuming we are not hashing topics. If we choose to hash topics, these will
|
||||
* need to be modified.
|
||||
*
|
||||
* Also note that a message can be associated with many topics. As soon as one of the topics is
|
||||
* known we match. If none of the topics are known we return an unknown state.
|
||||
*/
|
||||
fn from_topics(topics: &[TopicHash], data: Vec<u8>) -> Self {
|
||||
for topic in topics {
|
||||
match GossipTopic::from(topic.as_str()) {
|
||||
GossipTopic::BeaconBlock => return PubsubMessage::Block(data),
|
||||
GossipTopic::BeaconAttestation => return PubsubMessage::Attestation(data),
|
||||
GossipTopic::VoluntaryExit => return PubsubMessage::VoluntaryExit(data),
|
||||
GossipTopic::ProposerSlashing => return PubsubMessage::ProposerSlashing(data),
|
||||
GossipTopic::AttesterSlashing => return PubsubMessage::AttesterSlashing(data),
|
||||
GossipTopic::Shard => return PubsubMessage::Unknown(data),
|
||||
GossipTopic::Unknown(_) => continue,
|
||||
}
|
||||
}
|
||||
PubsubMessage::Unknown(data)
|
||||
}
|
||||
|
||||
fn into_data(self) -> Vec<u8> {
|
||||
match self {
|
||||
PubsubMessage::Block(data)
|
||||
| PubsubMessage::Attestation(data)
|
||||
| PubsubMessage::VoluntaryExit(data)
|
||||
| PubsubMessage::ProposerSlashing(data)
|
||||
| PubsubMessage::AttesterSlashing(data)
|
||||
| PubsubMessage::Unknown(data) => data,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
use crate::topics::GossipTopic;
|
||||
use crate::types::{GossipEncoding, GossipKind, GossipTopic};
|
||||
use enr::Enr;
|
||||
use libp2p::gossipsub::{GossipsubConfig, GossipsubConfigBuilder, GossipsubMessage, MessageId};
|
||||
use libp2p::Multiaddr;
|
||||
@ -67,11 +67,11 @@ impl Default for Config {
|
||||
|
||||
// The default topics that we will initially subscribe to
|
||||
let topics = vec![
|
||||
GossipTopic::BeaconBlock,
|
||||
GossipTopic::BeaconAttestation,
|
||||
GossipTopic::VoluntaryExit,
|
||||
GossipTopic::ProposerSlashing,
|
||||
GossipTopic::AttesterSlashing,
|
||||
GossipTopic::new(GossipKind::BeaconBlock, GossipEncoding::SSZ),
|
||||
GossipTopic::new(GossipKind::BeaconAggregateAndProof, GossipEncoding::SSZ),
|
||||
GossipTopic::new(GossipKind::VoluntaryExit, GossipEncoding::SSZ),
|
||||
GossipTopic::new(GossipKind::ProposerSlashing, GossipEncoding::SSZ),
|
||||
GossipTopic::new(GossipKind::AttesterSlashing, GossipEncoding::SSZ),
|
||||
];
|
||||
|
||||
// The function used to generate a gossipsub message id
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
use crate::metrics;
|
||||
use crate::{error, NetworkConfig, NetworkGlobals};
|
||||
use crate::{error, NetworkConfig, NetworkGlobals, PeerInfo};
|
||||
/// This manages the discovery and management of peers.
|
||||
///
|
||||
/// Currently using discv5 for peer discovery.
|
||||
@ -16,10 +16,11 @@ use std::fs::File;
|
||||
use std::io::prelude::*;
|
||||
use std::path::Path;
|
||||
use std::str::FromStr;
|
||||
use std::sync::{atomic::Ordering, Arc};
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
use tokio::io::{AsyncRead, AsyncWrite};
|
||||
use tokio::timer::Delay;
|
||||
use types::EthSpec;
|
||||
|
||||
/// Maximum seconds before searching for extra peers.
|
||||
const MAX_TIME_BETWEEN_PEER_SEARCHES: u64 = 120;
|
||||
@ -30,7 +31,7 @@ const ENR_FILENAME: &str = "enr.dat";
|
||||
|
||||
/// Lighthouse discovery behaviour. This provides peer management and discovery using the Discv5
|
||||
/// libp2p protocol.
|
||||
pub struct Discovery<TSubstream> {
|
||||
pub struct Discovery<TSubstream, TSpec: EthSpec> {
|
||||
/// The currently banned peers.
|
||||
banned_peers: HashSet<PeerId>,
|
||||
|
||||
@ -56,17 +57,17 @@ pub struct Discovery<TSubstream> {
|
||||
discovery: Discv5<TSubstream>,
|
||||
|
||||
/// A collection of network constants that can be read from other threads.
|
||||
network_globals: Arc<NetworkGlobals>,
|
||||
network_globals: Arc<NetworkGlobals<TSpec>>,
|
||||
|
||||
/// Logger for the discovery behaviour.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
impl<TSubstream> Discovery<TSubstream> {
|
||||
impl<TSubstream, TSpec: EthSpec> Discovery<TSubstream, TSpec> {
|
||||
pub fn new(
|
||||
local_key: &Keypair,
|
||||
config: &NetworkConfig,
|
||||
network_globals: Arc<NetworkGlobals>,
|
||||
network_globals: Arc<NetworkGlobals<TSpec>>,
|
||||
log: &slog::Logger,
|
||||
) -> error::Result<Self> {
|
||||
let log = log.clone();
|
||||
@ -81,8 +82,7 @@ impl<TSubstream> Discovery<TSubstream> {
|
||||
None => String::from(""),
|
||||
};
|
||||
|
||||
info!(log, "ENR Initialised"; "enr" => local_enr.to_base64(), "seq" => local_enr.seq());
|
||||
debug!(log, "Discv5 Node ID Initialised"; "node_id" => format!("{}",local_enr.node_id()));
|
||||
info!(log, "ENR Initialised"; "enr" => local_enr.to_base64(), "seq" => local_enr.seq(), "id"=> format!("{}",local_enr.node_id()), "ip" => format!("{:?}", local_enr.ip()), "udp"=> local_enr.udp().unwrap_or_else(|| 0), "tcp" => local_enr.tcp().unwrap_or_else(|| 0));
|
||||
|
||||
// the last parameter enables IP limiting. 2 Nodes on the same /24 subnet per bucket and 10
|
||||
// nodes on the same /24 subnet per table.
|
||||
@ -131,21 +131,6 @@ impl<TSubstream> Discovery<TSubstream> {
|
||||
self.discovery.add_enr(enr);
|
||||
}
|
||||
|
||||
/// The current number of connected libp2p peers.
|
||||
pub fn connected_peers(&self) -> usize {
|
||||
self.network_globals.connected_peers.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
/// The current number of connected libp2p peers.
|
||||
pub fn connected_peer_set(&self) -> Vec<PeerId> {
|
||||
self.network_globals
|
||||
.connected_peer_set
|
||||
.read()
|
||||
.iter()
|
||||
.cloned()
|
||||
.collect::<Vec<_>>()
|
||||
}
|
||||
|
||||
/// The peer has been banned. Add this peer to the banned list to prevent any future
|
||||
/// re-connections.
|
||||
// TODO: Remove the peer from the DHT if present
|
||||
@ -172,7 +157,7 @@ impl<TSubstream> Discovery<TSubstream> {
|
||||
}
|
||||
|
||||
// Redirect all behaviour events to underlying discovery behaviour.
|
||||
impl<TSubstream> NetworkBehaviour for Discovery<TSubstream>
|
||||
impl<TSubstream, TSpec: EthSpec> NetworkBehaviour for Discovery<TSubstream, TSpec>
|
||||
where
|
||||
TSubstream: AsyncRead + AsyncWrite,
|
||||
{
|
||||
@ -189,18 +174,18 @@ where
|
||||
}
|
||||
|
||||
fn inject_connected(&mut self, peer_id: PeerId, _endpoint: ConnectedPoint) {
|
||||
// TODO: Search for a known ENR once discv5 is updated.
|
||||
self.network_globals
|
||||
.connected_peer_set
|
||||
.write()
|
||||
.insert(peer_id);
|
||||
self.network_globals.connected_peers.store(
|
||||
self.network_globals.connected_peer_set.read().len(),
|
||||
Ordering::Relaxed,
|
||||
);
|
||||
.insert(peer_id, PeerInfo::new());
|
||||
// TODO: Drop peers if over max_peer limit
|
||||
|
||||
metrics::inc_counter(&metrics::PEER_CONNECT_EVENT_COUNT);
|
||||
metrics::set_gauge(&metrics::PEERS_CONNECTED, self.connected_peers() as i64);
|
||||
metrics::set_gauge(
|
||||
&metrics::PEERS_CONNECTED,
|
||||
self.network_globals.connected_peers() as i64,
|
||||
);
|
||||
}
|
||||
|
||||
fn inject_disconnected(&mut self, peer_id: &PeerId, _endpoint: ConnectedPoint) {
|
||||
@ -208,13 +193,12 @@ where
|
||||
.connected_peer_set
|
||||
.write()
|
||||
.remove(peer_id);
|
||||
self.network_globals.connected_peers.store(
|
||||
self.network_globals.connected_peer_set.read().len(),
|
||||
Ordering::Relaxed,
|
||||
);
|
||||
|
||||
metrics::inc_counter(&metrics::PEER_DISCONNECT_EVENT_COUNT);
|
||||
metrics::set_gauge(&metrics::PEERS_CONNECTED, self.connected_peers() as i64);
|
||||
metrics::set_gauge(
|
||||
&metrics::PEERS_CONNECTED,
|
||||
self.network_globals.connected_peers() as i64,
|
||||
);
|
||||
}
|
||||
|
||||
fn inject_replaced(
|
||||
@ -247,8 +231,7 @@ where
|
||||
loop {
|
||||
match self.peer_discovery_delay.poll() {
|
||||
Ok(Async::Ready(_)) => {
|
||||
if self.network_globals.connected_peers.load(Ordering::Relaxed) < self.max_peers
|
||||
{
|
||||
if self.network_globals.connected_peers() < self.max_peers {
|
||||
self.find_peers();
|
||||
}
|
||||
// Set to maximum, and update to earlier, once we get our results back.
|
||||
@ -303,8 +286,7 @@ where
|
||||
for peer_id in closer_peers {
|
||||
// if we need more peers, attempt a connection
|
||||
|
||||
if self.network_globals.connected_peers.load(Ordering::Relaxed)
|
||||
< self.max_peers
|
||||
if self.network_globals.connected_peers() < self.max_peers
|
||||
&& self
|
||||
.network_globals
|
||||
.connected_peer_set
|
||||
|
||||
@ -1,30 +0,0 @@
|
||||
//! A collection of variables that are accessible outside of the network thread itself.
|
||||
use crate::{Enr, Multiaddr, PeerId};
|
||||
use parking_lot::RwLock;
|
||||
use std::collections::HashSet;
|
||||
use std::sync::atomic::AtomicUsize;
|
||||
|
||||
pub struct NetworkGlobals {
|
||||
/// The current local ENR.
|
||||
pub local_enr: RwLock<Option<Enr>>,
|
||||
/// The local peer_id.
|
||||
pub peer_id: RwLock<PeerId>,
|
||||
/// Listening multiaddrs.
|
||||
pub listen_multiaddrs: RwLock<Vec<Multiaddr>>,
|
||||
/// Current number of connected libp2p peers.
|
||||
pub connected_peers: AtomicUsize,
|
||||
/// The collection of currently connected peers.
|
||||
pub connected_peer_set: RwLock<HashSet<PeerId>>,
|
||||
}
|
||||
|
||||
impl NetworkGlobals {
|
||||
pub fn new(peer_id: PeerId) -> Self {
|
||||
NetworkGlobals {
|
||||
local_enr: RwLock::new(None),
|
||||
peer_id: RwLock::new(peer_id),
|
||||
listen_multiaddrs: RwLock::new(Vec::new()),
|
||||
connected_peers: AtomicUsize::new(0),
|
||||
connected_peer_set: RwLock::new(HashSet::new()),
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -8,25 +8,16 @@ extern crate lazy_static;
|
||||
pub mod behaviour;
|
||||
mod config;
|
||||
mod discovery;
|
||||
pub mod error;
|
||||
mod globals;
|
||||
mod metrics;
|
||||
pub mod rpc;
|
||||
mod service;
|
||||
mod topics;
|
||||
pub mod types;
|
||||
|
||||
pub use behaviour::PubsubMessage;
|
||||
pub use crate::types::{error, GossipTopic, NetworkGlobals, PeerInfo, PubsubData, PubsubMessage};
|
||||
pub use config::Config as NetworkConfig;
|
||||
pub use globals::NetworkGlobals;
|
||||
pub use libp2p::enr::Enr;
|
||||
pub use libp2p::gossipsub::{MessageId, Topic, TopicHash};
|
||||
pub use libp2p::multiaddr;
|
||||
pub use libp2p::Multiaddr;
|
||||
pub use libp2p::{
|
||||
gossipsub::{GossipsubConfig, GossipsubConfigBuilder},
|
||||
PeerId, Swarm,
|
||||
};
|
||||
pub use libp2p::{multiaddr, Multiaddr};
|
||||
pub use libp2p::{PeerId, Swarm};
|
||||
pub use rpc::RPCEvent;
|
||||
pub use service::Libp2pEvent;
|
||||
pub use service::Service;
|
||||
pub use topics::GossipTopic;
|
||||
pub use service::{Libp2pEvent, Service};
|
||||
|
||||
@ -3,7 +3,9 @@
|
||||
use crate::rpc::{ErrorMessage, RPCErrorResponse, RPCRequest, RPCResponse};
|
||||
use libp2p::bytes::BufMut;
|
||||
use libp2p::bytes::BytesMut;
|
||||
use std::marker::PhantomData;
|
||||
use tokio::codec::{Decoder, Encoder};
|
||||
use types::EthSpec;
|
||||
|
||||
pub trait OutboundCodec: Encoder + Decoder {
|
||||
type ErrorType;
|
||||
@ -17,43 +19,53 @@ pub trait OutboundCodec: Encoder + Decoder {
|
||||
/* Global Inbound Codec */
|
||||
// This deals with Decoding RPC Requests from other peers and encoding our responses
|
||||
|
||||
pub struct BaseInboundCodec<TCodec>
|
||||
pub struct BaseInboundCodec<TCodec, TSpec>
|
||||
where
|
||||
TCodec: Encoder + Decoder,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
/// Inner codec for handling various encodings
|
||||
inner: TCodec,
|
||||
phantom: PhantomData<TSpec>,
|
||||
}
|
||||
|
||||
impl<TCodec> BaseInboundCodec<TCodec>
|
||||
impl<TCodec, TSpec> BaseInboundCodec<TCodec, TSpec>
|
||||
where
|
||||
TCodec: Encoder + Decoder,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
pub fn new(codec: TCodec) -> Self {
|
||||
BaseInboundCodec { inner: codec }
|
||||
BaseInboundCodec {
|
||||
inner: codec,
|
||||
phantom: PhantomData,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Global Outbound Codec */
|
||||
// This deals with Decoding RPC Responses from other peers and encoding our requests
|
||||
pub struct BaseOutboundCodec<TOutboundCodec>
|
||||
pub struct BaseOutboundCodec<TOutboundCodec, TSpec>
|
||||
where
|
||||
TOutboundCodec: OutboundCodec,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
/// Inner codec for handling various encodings.
|
||||
inner: TOutboundCodec,
|
||||
/// Keeps track of the current response code for a chunk.
|
||||
current_response_code: Option<u8>,
|
||||
phantom: PhantomData<TSpec>,
|
||||
}
|
||||
|
||||
impl<TOutboundCodec> BaseOutboundCodec<TOutboundCodec>
|
||||
impl<TOutboundCodec, TSpec> BaseOutboundCodec<TOutboundCodec, TSpec>
|
||||
where
|
||||
TSpec: EthSpec,
|
||||
TOutboundCodec: OutboundCodec,
|
||||
{
|
||||
pub fn new(codec: TOutboundCodec) -> Self {
|
||||
BaseOutboundCodec {
|
||||
inner: codec,
|
||||
current_response_code: None,
|
||||
phantom: PhantomData,
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -63,11 +75,12 @@ where
|
||||
/* Base Inbound Codec */
|
||||
|
||||
// This Encodes RPC Responses sent to external peers
|
||||
impl<TCodec> Encoder for BaseInboundCodec<TCodec>
|
||||
impl<TCodec, TSpec> Encoder for BaseInboundCodec<TCodec, TSpec>
|
||||
where
|
||||
TCodec: Decoder + Encoder<Item = RPCErrorResponse>,
|
||||
TSpec: EthSpec,
|
||||
TCodec: Decoder + Encoder<Item = RPCErrorResponse<TSpec>>,
|
||||
{
|
||||
type Item = RPCErrorResponse;
|
||||
type Item = RPCErrorResponse<TSpec>;
|
||||
type Error = <TCodec as Encoder>::Error;
|
||||
|
||||
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
|
||||
@ -82,11 +95,12 @@ where
|
||||
}
|
||||
|
||||
// This Decodes RPC Requests from external peers
|
||||
impl<TCodec> Decoder for BaseInboundCodec<TCodec>
|
||||
impl<TCodec, TSpec> Decoder for BaseInboundCodec<TCodec, TSpec>
|
||||
where
|
||||
TCodec: Encoder + Decoder<Item = RPCRequest>,
|
||||
TSpec: EthSpec,
|
||||
TCodec: Encoder + Decoder<Item = RPCRequest<TSpec>>,
|
||||
{
|
||||
type Item = RPCRequest;
|
||||
type Item = RPCRequest<TSpec>;
|
||||
type Error = <TCodec as Decoder>::Error;
|
||||
|
||||
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
|
||||
@ -97,11 +111,12 @@ where
|
||||
/* Base Outbound Codec */
|
||||
|
||||
// This Encodes RPC Requests sent to external peers
|
||||
impl<TCodec> Encoder for BaseOutboundCodec<TCodec>
|
||||
impl<TCodec, TSpec> Encoder for BaseOutboundCodec<TCodec, TSpec>
|
||||
where
|
||||
TCodec: OutboundCodec + Encoder<Item = RPCRequest>,
|
||||
TSpec: EthSpec,
|
||||
TCodec: OutboundCodec + Encoder<Item = RPCRequest<TSpec>>,
|
||||
{
|
||||
type Item = RPCRequest;
|
||||
type Item = RPCRequest<TSpec>;
|
||||
type Error = <TCodec as Encoder>::Error;
|
||||
|
||||
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
|
||||
@ -110,11 +125,12 @@ where
|
||||
}
|
||||
|
||||
// This decodes RPC Responses received from external peers
|
||||
impl<TCodec> Decoder for BaseOutboundCodec<TCodec>
|
||||
impl<TCodec, TSpec> Decoder for BaseOutboundCodec<TCodec, TSpec>
|
||||
where
|
||||
TCodec: OutboundCodec<ErrorType = ErrorMessage> + Decoder<Item = RPCResponse>,
|
||||
TSpec: EthSpec,
|
||||
TCodec: OutboundCodec<ErrorType = ErrorMessage> + Decoder<Item = RPCResponse<TSpec>>,
|
||||
{
|
||||
type Item = RPCErrorResponse;
|
||||
type Item = RPCErrorResponse<TSpec>;
|
||||
type Error = <TCodec as Decoder>::Error;
|
||||
|
||||
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
|
||||
@ -130,7 +146,7 @@ where
|
||||
});
|
||||
|
||||
let inner_result = {
|
||||
if RPCErrorResponse::is_response(response_code) {
|
||||
if RPCErrorResponse::<TSpec>::is_response(response_code) {
|
||||
// decode an actual response and mutates the buffer if enough bytes have been read
|
||||
// returning the result.
|
||||
self.inner
|
||||
|
||||
@ -7,18 +7,19 @@ use crate::rpc::protocol::RPCError;
|
||||
use crate::rpc::{RPCErrorResponse, RPCRequest};
|
||||
use libp2p::bytes::BytesMut;
|
||||
use tokio::codec::{Decoder, Encoder};
|
||||
use types::EthSpec;
|
||||
|
||||
// Known types of codecs
|
||||
pub enum InboundCodec {
|
||||
SSZ(BaseInboundCodec<SSZInboundCodec>),
|
||||
pub enum InboundCodec<TSpec: EthSpec> {
|
||||
SSZ(BaseInboundCodec<SSZInboundCodec<TSpec>, TSpec>),
|
||||
}
|
||||
|
||||
pub enum OutboundCodec {
|
||||
SSZ(BaseOutboundCodec<SSZOutboundCodec>),
|
||||
pub enum OutboundCodec<TSpec: EthSpec> {
|
||||
SSZ(BaseOutboundCodec<SSZOutboundCodec<TSpec>, TSpec>),
|
||||
}
|
||||
|
||||
impl Encoder for InboundCodec {
|
||||
type Item = RPCErrorResponse;
|
||||
impl<T: EthSpec> Encoder for InboundCodec<T> {
|
||||
type Item = RPCErrorResponse<T>;
|
||||
type Error = RPCError;
|
||||
|
||||
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
|
||||
@ -28,8 +29,8 @@ impl Encoder for InboundCodec {
|
||||
}
|
||||
}
|
||||
|
||||
impl Decoder for InboundCodec {
|
||||
type Item = RPCRequest;
|
||||
impl<TSpec: EthSpec> Decoder for InboundCodec<TSpec> {
|
||||
type Item = RPCRequest<TSpec>;
|
||||
type Error = RPCError;
|
||||
|
||||
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
|
||||
@ -39,8 +40,8 @@ impl Decoder for InboundCodec {
|
||||
}
|
||||
}
|
||||
|
||||
impl Encoder for OutboundCodec {
|
||||
type Item = RPCRequest;
|
||||
impl<TSpec: EthSpec> Encoder for OutboundCodec<TSpec> {
|
||||
type Item = RPCRequest<TSpec>;
|
||||
type Error = RPCError;
|
||||
|
||||
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
|
||||
@ -50,8 +51,8 @@ impl Encoder for OutboundCodec {
|
||||
}
|
||||
}
|
||||
|
||||
impl Decoder for OutboundCodec {
|
||||
type Item = RPCErrorResponse;
|
||||
impl<T: EthSpec> Decoder for OutboundCodec<T> {
|
||||
type Item = RPCErrorResponse<T>;
|
||||
type Error = RPCError;
|
||||
|
||||
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
|
||||
|
||||
@ -8,17 +8,20 @@ use crate::rpc::{
|
||||
use crate::rpc::{ErrorMessage, RPCErrorResponse, RPCRequest, RPCResponse};
|
||||
use libp2p::bytes::{BufMut, Bytes, BytesMut};
|
||||
use ssz::{Decode, Encode};
|
||||
use std::marker::PhantomData;
|
||||
use tokio::codec::{Decoder, Encoder};
|
||||
use types::{EthSpec, SignedBeaconBlock};
|
||||
use unsigned_varint::codec::UviBytes;
|
||||
|
||||
/* Inbound Codec */
|
||||
|
||||
pub struct SSZInboundCodec {
|
||||
pub struct SSZInboundCodec<TSpec: EthSpec> {
|
||||
inner: UviBytes,
|
||||
protocol: ProtocolId,
|
||||
phantom: PhantomData<TSpec>,
|
||||
}
|
||||
|
||||
impl SSZInboundCodec {
|
||||
impl<T: EthSpec> SSZInboundCodec<T> {
|
||||
pub fn new(protocol: ProtocolId, max_packet_size: usize) -> Self {
|
||||
let mut uvi_codec = UviBytes::default();
|
||||
uvi_codec.set_max_len(max_packet_size);
|
||||
@ -29,24 +32,23 @@ impl SSZInboundCodec {
|
||||
SSZInboundCodec {
|
||||
inner: uvi_codec,
|
||||
protocol,
|
||||
phantom: PhantomData,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Encoder for inbound streams: Encodes RPC Responses sent to peers.
|
||||
impl Encoder for SSZInboundCodec {
|
||||
type Item = RPCErrorResponse;
|
||||
impl<TSpec: EthSpec> Encoder for SSZInboundCodec<TSpec> {
|
||||
type Item = RPCErrorResponse<TSpec>;
|
||||
type Error = RPCError;
|
||||
|
||||
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
|
||||
let bytes = match item {
|
||||
RPCErrorResponse::Success(resp) => {
|
||||
match resp {
|
||||
RPCResponse::Status(res) => res.as_ssz_bytes(),
|
||||
RPCResponse::BlocksByRange(res) => res, // already raw bytes
|
||||
RPCResponse::BlocksByRoot(res) => res, // already raw bytes
|
||||
}
|
||||
}
|
||||
RPCErrorResponse::Success(resp) => match resp {
|
||||
RPCResponse::Status(res) => res.as_ssz_bytes(),
|
||||
RPCResponse::BlocksByRange(res) => res.as_ssz_bytes(),
|
||||
RPCResponse::BlocksByRoot(res) => res.as_ssz_bytes(),
|
||||
},
|
||||
RPCErrorResponse::InvalidRequest(err) => err.as_ssz_bytes(),
|
||||
RPCErrorResponse::ServerError(err) => err.as_ssz_bytes(),
|
||||
RPCErrorResponse::Unknown(err) => err.as_ssz_bytes(),
|
||||
@ -70,8 +72,8 @@ impl Encoder for SSZInboundCodec {
|
||||
}
|
||||
|
||||
// Decoder for inbound streams: Decodes RPC requests from peers
|
||||
impl Decoder for SSZInboundCodec {
|
||||
type Item = RPCRequest;
|
||||
impl<TSpec: EthSpec> Decoder for SSZInboundCodec<TSpec> {
|
||||
type Item = RPCRequest<TSpec>;
|
||||
type Error = RPCError;
|
||||
|
||||
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
|
||||
@ -111,12 +113,13 @@ impl Decoder for SSZInboundCodec {
|
||||
|
||||
/* Outbound Codec: Codec for initiating RPC requests */
|
||||
|
||||
pub struct SSZOutboundCodec {
|
||||
pub struct SSZOutboundCodec<TSpec: EthSpec> {
|
||||
inner: UviBytes,
|
||||
protocol: ProtocolId,
|
||||
phantom: PhantomData<TSpec>,
|
||||
}
|
||||
|
||||
impl SSZOutboundCodec {
|
||||
impl<TSpec: EthSpec> SSZOutboundCodec<TSpec> {
|
||||
pub fn new(protocol: ProtocolId, max_packet_size: usize) -> Self {
|
||||
let mut uvi_codec = UviBytes::default();
|
||||
uvi_codec.set_max_len(max_packet_size);
|
||||
@ -127,13 +130,14 @@ impl SSZOutboundCodec {
|
||||
SSZOutboundCodec {
|
||||
inner: uvi_codec,
|
||||
protocol,
|
||||
phantom: PhantomData,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Encoder for outbound streams: Encodes RPC Requests to peers
|
||||
impl Encoder for SSZOutboundCodec {
|
||||
type Item = RPCRequest;
|
||||
impl<TSpec: EthSpec> Encoder for SSZOutboundCodec<TSpec> {
|
||||
type Item = RPCRequest<TSpec>;
|
||||
type Error = RPCError;
|
||||
|
||||
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
|
||||
@ -142,6 +146,7 @@ impl Encoder for SSZOutboundCodec {
|
||||
RPCRequest::Goodbye(req) => req.as_ssz_bytes(),
|
||||
RPCRequest::BlocksByRange(req) => req.as_ssz_bytes(),
|
||||
RPCRequest::BlocksByRoot(req) => req.block_roots.as_ssz_bytes(),
|
||||
RPCRequest::Phantom(_) => unreachable!("Never encode phantom data"),
|
||||
};
|
||||
// length-prefix
|
||||
self.inner
|
||||
@ -155,8 +160,8 @@ impl Encoder for SSZOutboundCodec {
|
||||
// The majority of the decoding has now been pushed upstream due to the changing specification.
|
||||
// We prefer to decode blocks and attestations with extra knowledge about the chain to perform
|
||||
// faster verification checks before decoding entire blocks/attestations.
|
||||
impl Decoder for SSZOutboundCodec {
|
||||
type Item = RPCResponse;
|
||||
impl<TSpec: EthSpec> Decoder for SSZOutboundCodec<TSpec> {
|
||||
type Item = RPCResponse<TSpec>;
|
||||
type Error = RPCError;
|
||||
|
||||
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
|
||||
@ -173,11 +178,15 @@ impl Decoder for SSZOutboundCodec {
|
||||
},
|
||||
RPC_GOODBYE => Err(RPCError::InvalidProtocol("GOODBYE doesn't have a response")),
|
||||
RPC_BLOCKS_BY_RANGE => match self.protocol.version.as_str() {
|
||||
"1" => Ok(Some(RPCResponse::BlocksByRange(Vec::new()))),
|
||||
"1" => Err(RPCError::Custom(
|
||||
"Status stream terminated unexpectedly, empty block".into(),
|
||||
)), // cannot have an empty block message.
|
||||
_ => unreachable!("Cannot negotiate an unknown version"),
|
||||
},
|
||||
RPC_BLOCKS_BY_ROOT => match self.protocol.version.as_str() {
|
||||
"1" => Ok(Some(RPCResponse::BlocksByRoot(Vec::new()))),
|
||||
"1" => Err(RPCError::Custom(
|
||||
"Status stream terminated unexpectedly, empty block".into(),
|
||||
)), // cannot have an empty block message.
|
||||
_ => unreachable!("Cannot negotiate an unknown version"),
|
||||
},
|
||||
_ => unreachable!("Cannot negotiate an unknown protocol"),
|
||||
@ -199,11 +208,15 @@ impl Decoder for SSZOutboundCodec {
|
||||
Err(RPCError::InvalidProtocol("GOODBYE doesn't have a response"))
|
||||
}
|
||||
RPC_BLOCKS_BY_RANGE => match self.protocol.version.as_str() {
|
||||
"1" => Ok(Some(RPCResponse::BlocksByRange(raw_bytes.to_vec()))),
|
||||
"1" => Ok(Some(RPCResponse::BlocksByRange(Box::new(
|
||||
SignedBeaconBlock::from_ssz_bytes(&raw_bytes)?,
|
||||
)))),
|
||||
_ => unreachable!("Cannot negotiate an unknown version"),
|
||||
},
|
||||
RPC_BLOCKS_BY_ROOT => match self.protocol.version.as_str() {
|
||||
"1" => Ok(Some(RPCResponse::BlocksByRoot(raw_bytes.to_vec()))),
|
||||
"1" => Ok(Some(RPCResponse::BlocksByRoot(Box::new(
|
||||
SignedBeaconBlock::from_ssz_bytes(&raw_bytes)?,
|
||||
)))),
|
||||
_ => unreachable!("Cannot negotiate an unknown version"),
|
||||
},
|
||||
_ => unreachable!("Cannot negotiate an unknown protocol"),
|
||||
@ -216,7 +229,7 @@ impl Decoder for SSZOutboundCodec {
|
||||
}
|
||||
}
|
||||
|
||||
impl OutboundCodec for SSZOutboundCodec {
|
||||
impl<TSpec: EthSpec> OutboundCodec for SSZOutboundCodec<TSpec> {
|
||||
type ErrorType = ErrorMessage;
|
||||
|
||||
fn decode_error(&mut self, src: &mut BytesMut) -> Result<Option<Self::ErrorType>, RPCError> {
|
||||
|
||||
@ -18,6 +18,7 @@ use std::collections::hash_map::Entry;
|
||||
use std::time::{Duration, Instant};
|
||||
use tokio::io::{AsyncRead, AsyncWrite};
|
||||
use tokio::timer::{delay_queue, DelayQueue};
|
||||
use types::EthSpec;
|
||||
|
||||
//TODO: Implement close() on the substream types to improve the poll code.
|
||||
//TODO: Implement check_timeout() on the substream types
|
||||
@ -36,42 +37,50 @@ type InboundRequestId = RequestId;
|
||||
type OutboundRequestId = RequestId;
|
||||
|
||||
/// Implementation of `ProtocolsHandler` for the RPC protocol.
|
||||
pub struct RPCHandler<TSubstream>
|
||||
pub struct RPCHandler<TSubstream, TSpec>
|
||||
where
|
||||
TSubstream: AsyncRead + AsyncWrite,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
/// The upgrade for inbound substreams.
|
||||
listen_protocol: SubstreamProtocol<RPCProtocol>,
|
||||
listen_protocol: SubstreamProtocol<RPCProtocol<TSpec>>,
|
||||
|
||||
/// If something bad happened and we should shut down the handler with an error.
|
||||
pending_error: Vec<(RequestId, ProtocolsHandlerUpgrErr<RPCError>)>,
|
||||
|
||||
/// Queue of events to produce in `poll()`.
|
||||
events_out: SmallVec<[RPCEvent; 4]>,
|
||||
events_out: SmallVec<[RPCEvent<TSpec>; 4]>,
|
||||
|
||||
/// Queue of outbound substreams to open.
|
||||
dial_queue: SmallVec<[RPCEvent; 4]>,
|
||||
dial_queue: SmallVec<[RPCEvent<TSpec>; 4]>,
|
||||
|
||||
/// Current number of concurrent outbound substreams being opened.
|
||||
dial_negotiated: u32,
|
||||
|
||||
/// Current inbound substreams awaiting processing.
|
||||
inbound_substreams:
|
||||
FnvHashMap<InboundRequestId, (InboundSubstreamState<TSubstream>, Option<delay_queue::Key>)>,
|
||||
inbound_substreams: FnvHashMap<
|
||||
InboundRequestId,
|
||||
(
|
||||
InboundSubstreamState<TSubstream, TSpec>,
|
||||
Option<delay_queue::Key>,
|
||||
),
|
||||
>,
|
||||
|
||||
/// Inbound substream `DelayQueue` which keeps track of when an inbound substream will timeout.
|
||||
inbound_substreams_delay: DelayQueue<InboundRequestId>,
|
||||
|
||||
/// Map of outbound substreams that need to be driven to completion. The `RequestId` is
|
||||
/// maintained by the application sending the request.
|
||||
outbound_substreams:
|
||||
FnvHashMap<OutboundRequestId, (OutboundSubstreamState<TSubstream>, delay_queue::Key)>,
|
||||
outbound_substreams: FnvHashMap<
|
||||
OutboundRequestId,
|
||||
(OutboundSubstreamState<TSubstream, TSpec>, delay_queue::Key),
|
||||
>,
|
||||
|
||||
/// Inbound substream `DelayQueue` which keeps track of when an inbound substream will timeout.
|
||||
outbound_substreams_delay: DelayQueue<OutboundRequestId>,
|
||||
|
||||
/// Map of outbound items that are queued as the stream processes them.
|
||||
queued_outbound_items: FnvHashMap<RequestId, Vec<RPCErrorResponse>>,
|
||||
queued_outbound_items: FnvHashMap<RequestId, Vec<RPCErrorResponse<TSpec>>>,
|
||||
|
||||
/// Sequential ID for waiting substreams. For inbound substreams, this is also the inbound request ID.
|
||||
current_inbound_substream_id: RequestId,
|
||||
@ -97,14 +106,15 @@ where
|
||||
}
|
||||
|
||||
/// State of an outbound substream. Either waiting for a response, or in the process of sending.
|
||||
pub enum InboundSubstreamState<TSubstream>
|
||||
pub enum InboundSubstreamState<TSubstream, TSpec>
|
||||
where
|
||||
TSubstream: AsyncRead + AsyncWrite,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
/// A response has been sent, pending writing and flush.
|
||||
ResponsePendingSend {
|
||||
/// The substream used to send the response
|
||||
substream: futures::sink::Send<InboundFramed<TSubstream>>,
|
||||
substream: futures::sink::Send<InboundFramed<TSubstream, TSpec>>,
|
||||
/// Whether a stream termination is requested. If true the stream will be closed after
|
||||
/// this send. Otherwise it will transition to an idle state until a stream termination is
|
||||
/// requested or a timeout is reached.
|
||||
@ -112,40 +122,41 @@ where
|
||||
},
|
||||
/// The response stream is idle and awaiting input from the application to send more chunked
|
||||
/// responses.
|
||||
ResponseIdle(InboundFramed<TSubstream>),
|
||||
ResponseIdle(InboundFramed<TSubstream, TSpec>),
|
||||
/// The substream is attempting to shutdown.
|
||||
Closing(InboundFramed<TSubstream>),
|
||||
Closing(InboundFramed<TSubstream, TSpec>),
|
||||
/// Temporary state during processing
|
||||
Poisoned,
|
||||
}
|
||||
|
||||
pub enum OutboundSubstreamState<TSubstream> {
|
||||
pub enum OutboundSubstreamState<TSubstream, TSpec: EthSpec> {
|
||||
/// A request has been sent, and we are awaiting a response. This future is driven in the
|
||||
/// handler because GOODBYE requests can be handled and responses dropped instantly.
|
||||
RequestPendingResponse {
|
||||
/// The framed negotiated substream.
|
||||
substream: OutboundFramed<TSubstream>,
|
||||
substream: OutboundFramed<TSubstream, TSpec>,
|
||||
/// Keeps track of the actual request sent.
|
||||
request: RPCRequest,
|
||||
request: RPCRequest<TSpec>,
|
||||
},
|
||||
/// Closing an outbound substream>
|
||||
Closing(OutboundFramed<TSubstream>),
|
||||
Closing(OutboundFramed<TSubstream, TSpec>),
|
||||
/// Temporary state during processing
|
||||
Poisoned,
|
||||
}
|
||||
|
||||
impl<TSubstream> InboundSubstreamState<TSubstream>
|
||||
impl<TSubstream, TSpec> InboundSubstreamState<TSubstream, TSpec>
|
||||
where
|
||||
TSubstream: AsyncRead + AsyncWrite,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
/// Moves the substream state to closing and informs the connected peer. The
|
||||
/// `queued_outbound_items` must be given as a parameter to add stream termination messages to
|
||||
/// the outbound queue.
|
||||
pub fn close(&mut self, outbound_queue: &mut Vec<RPCErrorResponse>) {
|
||||
pub fn close(&mut self, outbound_queue: &mut Vec<RPCErrorResponse<TSpec>>) {
|
||||
// When terminating a stream, report the stream termination to the requesting user via
|
||||
// an RPC error
|
||||
let error = RPCErrorResponse::ServerError(ErrorMessage {
|
||||
error_message: b"Request timed out".to_vec(),
|
||||
error_message: "Request timed out".as_bytes().to_vec(),
|
||||
});
|
||||
|
||||
// The stream termination type is irrelevant, this will terminate the
|
||||
@ -163,16 +174,11 @@ where
|
||||
|
||||
*self = InboundSubstreamState::ResponsePendingSend { substream, closing }
|
||||
}
|
||||
InboundSubstreamState::ResponseIdle(mut substream) => {
|
||||
// check if the stream is already closed
|
||||
if let Ok(Async::Ready(None)) = substream.poll() {
|
||||
*self = InboundSubstreamState::Closing(substream);
|
||||
} else {
|
||||
*self = InboundSubstreamState::ResponsePendingSend {
|
||||
substream: substream.send(error),
|
||||
closing: true,
|
||||
};
|
||||
}
|
||||
InboundSubstreamState::ResponseIdle(substream) => {
|
||||
*self = InboundSubstreamState::ResponsePendingSend {
|
||||
substream: substream.send(error),
|
||||
closing: true,
|
||||
};
|
||||
}
|
||||
InboundSubstreamState::Closing(substream) => {
|
||||
// let the stream close
|
||||
@ -185,12 +191,13 @@ where
|
||||
}
|
||||
}
|
||||
|
||||
impl<TSubstream> RPCHandler<TSubstream>
|
||||
impl<TSubstream, TSpec> RPCHandler<TSubstream, TSpec>
|
||||
where
|
||||
TSubstream: AsyncRead + AsyncWrite,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
pub fn new(
|
||||
listen_protocol: SubstreamProtocol<RPCProtocol>,
|
||||
listen_protocol: SubstreamProtocol<RPCProtocol<TSpec>>,
|
||||
inactive_timeout: Duration,
|
||||
log: &slog::Logger,
|
||||
) -> Self {
|
||||
@ -224,7 +231,7 @@ where
|
||||
///
|
||||
/// > **Note**: If you modify the protocol, modifications will only applies to future inbound
|
||||
/// > substreams, not the ones already being negotiated.
|
||||
pub fn listen_protocol_ref(&self) -> &SubstreamProtocol<RPCProtocol> {
|
||||
pub fn listen_protocol_ref(&self) -> &SubstreamProtocol<RPCProtocol<TSpec>> {
|
||||
&self.listen_protocol
|
||||
}
|
||||
|
||||
@ -232,29 +239,30 @@ where
|
||||
///
|
||||
/// > **Note**: If you modify the protocol, modifications will only applies to future inbound
|
||||
/// > substreams, not the ones already being negotiated.
|
||||
pub fn listen_protocol_mut(&mut self) -> &mut SubstreamProtocol<RPCProtocol> {
|
||||
pub fn listen_protocol_mut(&mut self) -> &mut SubstreamProtocol<RPCProtocol<TSpec>> {
|
||||
&mut self.listen_protocol
|
||||
}
|
||||
|
||||
/// Opens an outbound substream with a request.
|
||||
pub fn send_request(&mut self, rpc_event: RPCEvent) {
|
||||
pub fn send_request(&mut self, rpc_event: RPCEvent<TSpec>) {
|
||||
self.keep_alive = KeepAlive::Yes;
|
||||
|
||||
self.dial_queue.push(rpc_event);
|
||||
}
|
||||
}
|
||||
|
||||
impl<TSubstream> ProtocolsHandler for RPCHandler<TSubstream>
|
||||
impl<TSubstream, TSpec> ProtocolsHandler for RPCHandler<TSubstream, TSpec>
|
||||
where
|
||||
TSubstream: AsyncRead + AsyncWrite,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
type InEvent = RPCEvent;
|
||||
type OutEvent = RPCEvent;
|
||||
type InEvent = RPCEvent<TSpec>;
|
||||
type OutEvent = RPCEvent<TSpec>;
|
||||
type Error = ProtocolsHandlerUpgrErr<RPCError>;
|
||||
type Substream = TSubstream;
|
||||
type InboundProtocol = RPCProtocol;
|
||||
type OutboundProtocol = RPCRequest;
|
||||
type OutboundOpenInfo = RPCEvent; // Keep track of the id and the request
|
||||
type InboundProtocol = RPCProtocol<TSpec>;
|
||||
type OutboundProtocol = RPCRequest<TSpec>;
|
||||
type OutboundOpenInfo = RPCEvent<TSpec>; // Keep track of the id and the request
|
||||
|
||||
fn listen_protocol(&self) -> SubstreamProtocol<Self::InboundProtocol> {
|
||||
self.listen_protocol.clone()
|
||||
@ -262,7 +270,7 @@ where
|
||||
|
||||
fn inject_fully_negotiated_inbound(
|
||||
&mut self,
|
||||
out: <RPCProtocol as InboundUpgrade<TSubstream>>::Output,
|
||||
out: <RPCProtocol<TSpec> as InboundUpgrade<TSubstream>>::Output,
|
||||
) {
|
||||
// update the keep alive timeout if there are no more remaining outbound streams
|
||||
if let KeepAlive::Until(_) = self.keep_alive {
|
||||
@ -294,7 +302,7 @@ where
|
||||
|
||||
fn inject_fully_negotiated_outbound(
|
||||
&mut self,
|
||||
out: <RPCRequest as OutboundUpgrade<TSubstream>>::Output,
|
||||
out: <RPCRequest<TSpec> as OutboundUpgrade<TSubstream>>::Output,
|
||||
rpc_event: Self::OutboundOpenInfo,
|
||||
) {
|
||||
self.dial_negotiated -= 1;
|
||||
@ -748,11 +756,11 @@ where
|
||||
}
|
||||
|
||||
// Check for new items to send to the peer and update the underlying stream
|
||||
fn apply_queued_responses<TSubstream: AsyncRead + AsyncWrite>(
|
||||
raw_substream: InboundFramed<TSubstream>,
|
||||
queued_outbound_items: &mut Option<&mut Vec<RPCErrorResponse>>,
|
||||
fn apply_queued_responses<TSubstream: AsyncRead + AsyncWrite, TSpec: EthSpec>(
|
||||
raw_substream: InboundFramed<TSubstream, TSpec>,
|
||||
queued_outbound_items: &mut Option<&mut Vec<RPCErrorResponse<TSpec>>>,
|
||||
new_items_to_send: &mut bool,
|
||||
) -> InboundSubstreamState<TSubstream> {
|
||||
) -> InboundSubstreamState<TSubstream, TSpec> {
|
||||
match queued_outbound_items {
|
||||
Some(ref mut queue) if !queue.is_empty() => {
|
||||
*new_items_to_send = true;
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
//! Available RPC methods types and ids.
|
||||
|
||||
use ssz_derive::{Decode, Encode};
|
||||
use types::{Epoch, Hash256, Slot};
|
||||
use types::{Epoch, EthSpec, Hash256, SignedBeaconBlock, Slot};
|
||||
|
||||
/* Request/Response data structures for RPC methods */
|
||||
|
||||
@ -129,16 +129,16 @@ pub struct BlocksByRootRequest {
|
||||
// Collection of enums and structs used by the Codecs to encode/decode RPC messages
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum RPCResponse {
|
||||
pub enum RPCResponse<T: EthSpec> {
|
||||
/// A HELLO message.
|
||||
Status(StatusMessage),
|
||||
|
||||
/// A response to a get BLOCKS_BY_RANGE request. A None response signifies the end of the
|
||||
/// batch.
|
||||
BlocksByRange(Vec<u8>),
|
||||
BlocksByRange(Box<SignedBeaconBlock<T>>),
|
||||
|
||||
/// A response to a get BLOCKS_BY_ROOT request.
|
||||
BlocksByRoot(Vec<u8>),
|
||||
BlocksByRoot(Box<SignedBeaconBlock<T>>),
|
||||
}
|
||||
|
||||
/// Indicates which response is being terminated by a stream termination response.
|
||||
@ -152,9 +152,9 @@ pub enum ResponseTermination {
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum RPCErrorResponse {
|
||||
pub enum RPCErrorResponse<T: EthSpec> {
|
||||
/// The response is a successful.
|
||||
Success(RPCResponse),
|
||||
Success(RPCResponse<T>),
|
||||
|
||||
/// The response was invalid.
|
||||
InvalidRequest(ErrorMessage),
|
||||
@ -169,7 +169,7 @@ pub enum RPCErrorResponse {
|
||||
StreamTermination(ResponseTermination),
|
||||
}
|
||||
|
||||
impl RPCErrorResponse {
|
||||
impl<T: EthSpec> RPCErrorResponse<T> {
|
||||
/// Used to encode the response in the codec.
|
||||
pub fn as_u8(&self) -> Option<u8> {
|
||||
match self {
|
||||
@ -242,17 +242,21 @@ impl std::fmt::Display for StatusMessage {
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for RPCResponse {
|
||||
impl<T: EthSpec> std::fmt::Display for RPCResponse<T> {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
RPCResponse::Status(status) => write!(f, "{}", status),
|
||||
RPCResponse::BlocksByRange(_) => write!(f, "<BlocksByRange>"),
|
||||
RPCResponse::BlocksByRoot(_) => write!(f, "<BlocksByRoot>"),
|
||||
RPCResponse::BlocksByRange(block) => {
|
||||
write!(f, "BlocksByRange: Block slot: {}", block.message.slot)
|
||||
}
|
||||
RPCResponse::BlocksByRoot(block) => {
|
||||
write!(f, "BlocksByRoot: BLock slot: {}", block.message.slot)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for RPCErrorResponse {
|
||||
impl<T: EthSpec> std::fmt::Display for RPCErrorResponse<T> {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
RPCErrorResponse::Success(res) => write!(f, "{}", res),
|
||||
|
||||
@ -20,6 +20,7 @@ use slog::o;
|
||||
use std::marker::PhantomData;
|
||||
use std::time::Duration;
|
||||
use tokio::io::{AsyncRead, AsyncWrite};
|
||||
use types::EthSpec;
|
||||
|
||||
pub(crate) mod codec;
|
||||
mod handler;
|
||||
@ -28,19 +29,19 @@ mod protocol;
|
||||
|
||||
/// The return type used in the behaviour and the resultant event from the protocols handler.
|
||||
#[derive(Debug)]
|
||||
pub enum RPCEvent {
|
||||
pub enum RPCEvent<T: EthSpec> {
|
||||
/// An inbound/outbound request for RPC protocol. The first parameter is a sequential
|
||||
/// id which tracks an awaiting substream for the response.
|
||||
Request(RequestId, RPCRequest),
|
||||
Request(RequestId, RPCRequest<T>),
|
||||
/// A response that is being sent or has been received from the RPC protocol. The first parameter returns
|
||||
/// that which was sent with the corresponding request, the second is a single chunk of a
|
||||
/// response.
|
||||
Response(RequestId, RPCErrorResponse),
|
||||
Response(RequestId, RPCErrorResponse<T>),
|
||||
/// An Error occurred.
|
||||
Error(RequestId, RPCError),
|
||||
}
|
||||
|
||||
impl RPCEvent {
|
||||
impl<T: EthSpec> RPCEvent<T> {
|
||||
pub fn id(&self) -> usize {
|
||||
match *self {
|
||||
RPCEvent::Request(id, _) => id,
|
||||
@ -50,7 +51,7 @@ impl RPCEvent {
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for RPCEvent {
|
||||
impl<T: EthSpec> std::fmt::Display for RPCEvent<T> {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
RPCEvent::Request(id, req) => write!(f, "RPC Request(id: {}, {})", id, req),
|
||||
@ -62,16 +63,16 @@ impl std::fmt::Display for RPCEvent {
|
||||
|
||||
/// Implements the libp2p `NetworkBehaviour` trait and therefore manages network-level
|
||||
/// logic.
|
||||
pub struct RPC<TSubstream> {
|
||||
pub struct RPC<TSubstream, TSpec: EthSpec> {
|
||||
/// Queue of events to processed.
|
||||
events: Vec<NetworkBehaviourAction<RPCEvent, RPCMessage>>,
|
||||
events: Vec<NetworkBehaviourAction<RPCEvent<TSpec>, RPCMessage<TSpec>>>,
|
||||
/// Pins the generic substream.
|
||||
marker: PhantomData<TSubstream>,
|
||||
/// Slog logger for RPC behaviour.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
impl<TSubstream> RPC<TSubstream> {
|
||||
impl<TSubstream, TSpec: EthSpec> RPC<TSubstream, TSpec> {
|
||||
pub fn new(log: slog::Logger) -> Self {
|
||||
let log = log.new(o!("service" => "libp2p_rpc"));
|
||||
RPC {
|
||||
@ -84,7 +85,7 @@ impl<TSubstream> RPC<TSubstream> {
|
||||
/// Submits an RPC request.
|
||||
///
|
||||
/// The peer must be connected for this to succeed.
|
||||
pub fn send_rpc(&mut self, peer_id: PeerId, rpc_event: RPCEvent) {
|
||||
pub fn send_rpc(&mut self, peer_id: PeerId, rpc_event: RPCEvent<TSpec>) {
|
||||
self.events.push(NetworkBehaviourAction::SendEvent {
|
||||
peer_id,
|
||||
event: rpc_event,
|
||||
@ -92,16 +93,19 @@ impl<TSubstream> RPC<TSubstream> {
|
||||
}
|
||||
}
|
||||
|
||||
impl<TSubstream> NetworkBehaviour for RPC<TSubstream>
|
||||
impl<TSubstream, TSpec> NetworkBehaviour for RPC<TSubstream, TSpec>
|
||||
where
|
||||
TSubstream: AsyncRead + AsyncWrite,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
type ProtocolsHandler = RPCHandler<TSubstream>;
|
||||
type OutEvent = RPCMessage;
|
||||
type ProtocolsHandler = RPCHandler<TSubstream, TSpec>;
|
||||
type OutEvent = RPCMessage<TSpec>;
|
||||
|
||||
fn new_handler(&mut self) -> Self::ProtocolsHandler {
|
||||
RPCHandler::new(
|
||||
SubstreamProtocol::new(RPCProtocol),
|
||||
SubstreamProtocol::new(RPCProtocol {
|
||||
phantom: PhantomData,
|
||||
}),
|
||||
Duration::from_secs(30),
|
||||
&self.log,
|
||||
)
|
||||
@ -157,8 +161,8 @@ where
|
||||
}
|
||||
|
||||
/// Messages sent to the user from the RPC protocol.
|
||||
pub enum RPCMessage {
|
||||
RPC(PeerId, RPCEvent),
|
||||
pub enum RPCMessage<TSpec: EthSpec> {
|
||||
RPC(PeerId, RPCEvent<TSpec>),
|
||||
PeerDialed(PeerId),
|
||||
PeerDisconnected(PeerId),
|
||||
}
|
||||
|
||||
@ -15,6 +15,7 @@ use futures::{
|
||||
};
|
||||
use libp2p::core::{upgrade, InboundUpgrade, OutboundUpgrade, ProtocolName, UpgradeInfo};
|
||||
use std::io;
|
||||
use std::marker::PhantomData;
|
||||
use std::time::Duration;
|
||||
use tokio::codec::Framed;
|
||||
use tokio::io::{AsyncRead, AsyncWrite};
|
||||
@ -22,6 +23,7 @@ use tokio::prelude::*;
|
||||
use tokio::timer::timeout;
|
||||
use tokio::util::FutureExt;
|
||||
use tokio_io_timeout::TimeoutStream;
|
||||
use types::EthSpec;
|
||||
|
||||
/// The maximum bytes that can be sent across the RPC.
|
||||
const MAX_RPC_SIZE: usize = 4_194_304; // 4M
|
||||
@ -44,9 +46,11 @@ pub const RPC_BLOCKS_BY_RANGE: &str = "beacon_blocks_by_range";
|
||||
pub const RPC_BLOCKS_BY_ROOT: &str = "beacon_blocks_by_root";
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RPCProtocol;
|
||||
pub struct RPCProtocol<TSpec: EthSpec> {
|
||||
pub phantom: PhantomData<TSpec>,
|
||||
}
|
||||
|
||||
impl UpgradeInfo for RPCProtocol {
|
||||
impl<TSpec: EthSpec> UpgradeInfo for RPCProtocol<TSpec> {
|
||||
type Info = ProtocolId;
|
||||
type InfoIter = Vec<Self::Info>;
|
||||
|
||||
@ -104,27 +108,30 @@ impl ProtocolName for ProtocolId {
|
||||
// The inbound protocol reads the request, decodes it and returns the stream to the protocol
|
||||
// handler to respond to once ready.
|
||||
|
||||
pub type InboundOutput<TSocket> = (RPCRequest, InboundFramed<TSocket>);
|
||||
pub type InboundFramed<TSocket> = Framed<TimeoutStream<upgrade::Negotiated<TSocket>>, InboundCodec>;
|
||||
type FnAndThen<TSocket> = fn(
|
||||
(Option<RPCRequest>, InboundFramed<TSocket>),
|
||||
) -> FutureResult<InboundOutput<TSocket>, RPCError>;
|
||||
type FnMapErr<TSocket> = fn(timeout::Error<(RPCError, InboundFramed<TSocket>)>) -> RPCError;
|
||||
pub type InboundOutput<TSocket, TSpec> = (RPCRequest<TSpec>, InboundFramed<TSocket, TSpec>);
|
||||
pub type InboundFramed<TSocket, TSpec> =
|
||||
Framed<TimeoutStream<upgrade::Negotiated<TSocket>>, InboundCodec<TSpec>>;
|
||||
type FnAndThen<TSocket, TSpec> = fn(
|
||||
(Option<RPCRequest<TSpec>>, InboundFramed<TSocket, TSpec>),
|
||||
) -> FutureResult<InboundOutput<TSocket, TSpec>, RPCError>;
|
||||
type FnMapErr<TSocket, TSpec> =
|
||||
fn(timeout::Error<(RPCError, InboundFramed<TSocket, TSpec>)>) -> RPCError;
|
||||
|
||||
impl<TSocket> InboundUpgrade<TSocket> for RPCProtocol
|
||||
impl<TSocket, TSpec> InboundUpgrade<TSocket> for RPCProtocol<TSpec>
|
||||
where
|
||||
TSocket: AsyncRead + AsyncWrite,
|
||||
TSpec: EthSpec,
|
||||
{
|
||||
type Output = InboundOutput<TSocket>;
|
||||
type Output = InboundOutput<TSocket, TSpec>;
|
||||
type Error = RPCError;
|
||||
|
||||
type Future = future::AndThen<
|
||||
future::MapErr<
|
||||
timeout::Timeout<stream::StreamFuture<InboundFramed<TSocket>>>,
|
||||
FnMapErr<TSocket>,
|
||||
timeout::Timeout<stream::StreamFuture<InboundFramed<TSocket, TSpec>>>,
|
||||
FnMapErr<TSocket, TSpec>,
|
||||
>,
|
||||
FutureResult<InboundOutput<TSocket>, RPCError>,
|
||||
FnAndThen<TSocket>,
|
||||
FutureResult<InboundOutput<TSocket, TSpec>, RPCError>,
|
||||
FnAndThen<TSocket, TSpec>,
|
||||
>;
|
||||
|
||||
fn upgrade_inbound(
|
||||
@ -141,7 +148,7 @@ where
|
||||
Framed::new(timed_socket, codec)
|
||||
.into_future()
|
||||
.timeout(Duration::from_secs(REQUEST_TIMEOUT))
|
||||
.map_err(RPCError::from as FnMapErr<TSocket>)
|
||||
.map_err(RPCError::from as FnMapErr<TSocket, TSpec>)
|
||||
.and_then({
|
||||
|(req, stream)| match req {
|
||||
Some(req) => futures::future::ok((req, stream)),
|
||||
@ -149,7 +156,7 @@ where
|
||||
"Stream terminated early".into(),
|
||||
)),
|
||||
}
|
||||
} as FnAndThen<TSocket>)
|
||||
} as FnAndThen<TSocket, TSpec>)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -161,14 +168,15 @@ where
|
||||
// `OutboundUpgrade`
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum RPCRequest {
|
||||
pub enum RPCRequest<TSpec: EthSpec> {
|
||||
Status(StatusMessage),
|
||||
Goodbye(GoodbyeReason),
|
||||
BlocksByRange(BlocksByRangeRequest),
|
||||
BlocksByRoot(BlocksByRootRequest),
|
||||
Phantom(PhantomData<TSpec>),
|
||||
}
|
||||
|
||||
impl UpgradeInfo for RPCRequest {
|
||||
impl<TSpec: EthSpec> UpgradeInfo for RPCRequest<TSpec> {
|
||||
type Info = ProtocolId;
|
||||
type InfoIter = Vec<Self::Info>;
|
||||
|
||||
@ -179,7 +187,7 @@ impl UpgradeInfo for RPCRequest {
|
||||
}
|
||||
|
||||
/// Implements the encoding per supported protocol for RPCRequest.
|
||||
impl RPCRequest {
|
||||
impl<TSpec: EthSpec> RPCRequest<TSpec> {
|
||||
pub fn supported_protocols(&self) -> Vec<ProtocolId> {
|
||||
match self {
|
||||
// add more protocols when versions/encodings are supported
|
||||
@ -187,6 +195,7 @@ impl RPCRequest {
|
||||
RPCRequest::Goodbye(_) => vec![ProtocolId::new(RPC_GOODBYE, "1", "ssz")],
|
||||
RPCRequest::BlocksByRange(_) => vec![ProtocolId::new(RPC_BLOCKS_BY_RANGE, "1", "ssz")],
|
||||
RPCRequest::BlocksByRoot(_) => vec![ProtocolId::new(RPC_BLOCKS_BY_ROOT, "1", "ssz")],
|
||||
RPCRequest::Phantom(_) => Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
@ -200,6 +209,7 @@ impl RPCRequest {
|
||||
RPCRequest::Goodbye(_) => false,
|
||||
RPCRequest::BlocksByRange(_) => true,
|
||||
RPCRequest::BlocksByRoot(_) => true,
|
||||
RPCRequest::Phantom(_) => unreachable!("Phantom should never be initialised"),
|
||||
}
|
||||
}
|
||||
|
||||
@ -211,6 +221,7 @@ impl RPCRequest {
|
||||
RPCRequest::Goodbye(_) => false,
|
||||
RPCRequest::BlocksByRange(_) => true,
|
||||
RPCRequest::BlocksByRoot(_) => true,
|
||||
RPCRequest::Phantom(_) => unreachable!("Phantom should never be initialised"),
|
||||
}
|
||||
}
|
||||
|
||||
@ -224,6 +235,7 @@ impl RPCRequest {
|
||||
RPCRequest::BlocksByRoot(_) => ResponseTermination::BlocksByRoot,
|
||||
RPCRequest::Status(_) => unreachable!(),
|
||||
RPCRequest::Goodbye(_) => unreachable!(),
|
||||
RPCRequest::Phantom(_) => unreachable!("Phantom should never be initialised"),
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -232,15 +244,17 @@ impl RPCRequest {
|
||||
|
||||
/* Outbound upgrades */
|
||||
|
||||
pub type OutboundFramed<TSocket> = Framed<upgrade::Negotiated<TSocket>, OutboundCodec>;
|
||||
pub type OutboundFramed<TSocket, TSpec> =
|
||||
Framed<upgrade::Negotiated<TSocket>, OutboundCodec<TSpec>>;
|
||||
|
||||
impl<TSocket> OutboundUpgrade<TSocket> for RPCRequest
|
||||
impl<TSocket, TSpec> OutboundUpgrade<TSocket> for RPCRequest<TSpec>
|
||||
where
|
||||
TSpec: EthSpec,
|
||||
TSocket: AsyncRead + AsyncWrite,
|
||||
{
|
||||
type Output = OutboundFramed<TSocket>;
|
||||
type Output = OutboundFramed<TSocket, TSpec>;
|
||||
type Error = RPCError;
|
||||
type Future = sink::Send<OutboundFramed<TSocket>>;
|
||||
type Future = sink::Send<OutboundFramed<TSocket, TSpec>>;
|
||||
fn upgrade_outbound(
|
||||
self,
|
||||
socket: upgrade::Negotiated<TSocket>,
|
||||
@ -340,13 +354,14 @@ impl std::error::Error for RPCError {
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for RPCRequest {
|
||||
impl<TSpec: EthSpec> std::fmt::Display for RPCRequest<TSpec> {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
RPCRequest::Status(status) => write!(f, "Status Message: {}", status),
|
||||
RPCRequest::Goodbye(reason) => write!(f, "Goodbye: {}", reason),
|
||||
RPCRequest::BlocksByRange(req) => write!(f, "Blocks by range: {}", req),
|
||||
RPCRequest::BlocksByRoot(req) => write!(f, "Blocks by root: {:?}", req),
|
||||
RPCRequest::Phantom(_) => unreachable!("Phantom should never be initialised"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,9 +1,9 @@
|
||||
use crate::behaviour::{Behaviour, BehaviourEvent, PubsubMessage};
|
||||
use crate::error;
|
||||
use crate::behaviour::{Behaviour, BehaviourEvent};
|
||||
use crate::multiaddr::Protocol;
|
||||
use crate::rpc::RPCEvent;
|
||||
use crate::types::error;
|
||||
use crate::NetworkConfig;
|
||||
use crate::{NetworkGlobals, Topic, TopicHash};
|
||||
use crate::{NetworkGlobals, PubsubMessage, TopicHash};
|
||||
use futures::prelude::*;
|
||||
use futures::Stream;
|
||||
use libp2p::core::{
|
||||
@ -24,9 +24,10 @@ use std::io::{Error, ErrorKind};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use tokio::timer::DelayQueue;
|
||||
use types::EthSpec;
|
||||
|
||||
type Libp2pStream = Boxed<(PeerId, StreamMuxerBox), Error>;
|
||||
type Libp2pBehaviour = Behaviour<Substream<StreamMuxerBox>>;
|
||||
type Libp2pBehaviour<TSpec> = Behaviour<Substream<StreamMuxerBox>, TSpec>;
|
||||
|
||||
const NETWORK_KEY_FILENAME: &str = "key";
|
||||
/// The time in milliseconds to wait before banning a peer. This allows for any Goodbye messages to be
|
||||
@ -34,10 +35,10 @@ const NETWORK_KEY_FILENAME: &str = "key";
|
||||
const BAN_PEER_WAIT_TIMEOUT: u64 = 200;
|
||||
|
||||
/// The configuration and state of the libp2p components for the beacon node.
|
||||
pub struct Service {
|
||||
pub struct Service<TSpec: EthSpec> {
|
||||
/// The libp2p Swarm handler.
|
||||
//TODO: Make this private
|
||||
pub swarm: Swarm<Libp2pStream, Libp2pBehaviour>,
|
||||
pub swarm: Swarm<Libp2pStream, Libp2pBehaviour<TSpec>>,
|
||||
|
||||
/// This node's PeerId.
|
||||
pub local_peer_id: PeerId,
|
||||
@ -52,11 +53,11 @@ pub struct Service {
|
||||
pub log: slog::Logger,
|
||||
}
|
||||
|
||||
impl Service {
|
||||
impl<TSpec: EthSpec> Service<TSpec> {
|
||||
pub fn new(
|
||||
config: &NetworkConfig,
|
||||
log: slog::Logger,
|
||||
) -> error::Result<(Arc<NetworkGlobals>, Self)> {
|
||||
) -> error::Result<(Arc<NetworkGlobals<TSpec>>, Self)> {
|
||||
trace!(log, "Libp2p Service starting");
|
||||
|
||||
let local_keypair = if let Some(hex_bytes) = &config.secret_key_hex {
|
||||
@ -70,7 +71,11 @@ impl Service {
|
||||
info!(log, "Libp2p Service"; "peer_id" => format!("{:?}", local_peer_id));
|
||||
|
||||
// set up a collection of variables accessible outside of the network crate
|
||||
let network_globals = Arc::new(NetworkGlobals::new(local_peer_id.clone()));
|
||||
let network_globals = Arc::new(NetworkGlobals::new(
|
||||
local_peer_id.clone(),
|
||||
config.libp2p_port,
|
||||
config.discovery_port,
|
||||
));
|
||||
|
||||
let mut swarm = {
|
||||
// Set up the transport - tcp/ws with noise/secio and mplex/yamux
|
||||
@ -133,12 +138,15 @@ impl Service {
|
||||
}
|
||||
|
||||
let mut subscribed_topics: Vec<String> = vec![];
|
||||
for topic in config.topics.clone() {
|
||||
let raw_topic: Topic = topic.into();
|
||||
let topic_string = raw_topic.no_hash();
|
||||
if swarm.subscribe(raw_topic.clone()) {
|
||||
for topic in &config.topics {
|
||||
let topic_string: String = topic.clone().into();
|
||||
if swarm.subscribe(topic.clone()) {
|
||||
trace!(log, "Subscribed to topic"; "topic" => format!("{}", topic_string));
|
||||
subscribed_topics.push(topic_string.as_str().into());
|
||||
subscribed_topics.push(topic_string);
|
||||
network_globals
|
||||
.gossipsub_subscriptions
|
||||
.write()
|
||||
.push(topic.clone());
|
||||
} else {
|
||||
warn!(log, "Could not subscribe to topic"; "topic" => format!("{}",topic_string));
|
||||
}
|
||||
@ -167,9 +175,9 @@ impl Service {
|
||||
}
|
||||
}
|
||||
|
||||
impl Stream for Service {
|
||||
type Item = Libp2pEvent;
|
||||
type Error = crate::error::Error;
|
||||
impl<TSpec: EthSpec> Stream for Service<TSpec> {
|
||||
type Item = Libp2pEvent<TSpec>;
|
||||
type Error = error::Error;
|
||||
|
||||
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
|
||||
loop {
|
||||
@ -313,9 +321,9 @@ fn build_transport(local_private_key: Keypair) -> Boxed<(PeerId, StreamMuxerBox)
|
||||
|
||||
#[derive(Debug)]
|
||||
/// Events that can be obtained from polling the Libp2p Service.
|
||||
pub enum Libp2pEvent {
|
||||
pub enum Libp2pEvent<TSpec: EthSpec> {
|
||||
/// An RPC response request has been received on the swarm.
|
||||
RPC(PeerId, RPCEvent),
|
||||
RPC(PeerId, RPCEvent<TSpec>),
|
||||
/// Initiated the connection to a new peer.
|
||||
PeerDialed(PeerId),
|
||||
/// A peer has disconnected.
|
||||
@ -325,7 +333,7 @@ pub enum Libp2pEvent {
|
||||
id: MessageId,
|
||||
source: PeerId,
|
||||
topics: Vec<TopicHash>,
|
||||
message: PubsubMessage,
|
||||
message: PubsubMessage<TSpec>,
|
||||
},
|
||||
/// Subscribed to peer for a topic hash.
|
||||
PeerSubscribed(PeerId, TopicHash),
|
||||
|
||||
@ -1,71 +0,0 @@
|
||||
use libp2p::gossipsub::Topic;
|
||||
use serde_derive::{Deserialize, Serialize};
|
||||
|
||||
/// The gossipsub topic names.
|
||||
// These constants form a topic name of the form /TOPIC_PREFIX/TOPIC/ENCODING_POSTFIX
|
||||
// For example /eth2/beacon_block/ssz
|
||||
pub const TOPIC_PREFIX: &str = "eth2";
|
||||
pub const TOPIC_ENCODING_POSTFIX: &str = "ssz";
|
||||
pub const BEACON_BLOCK_TOPIC: &str = "beacon_block";
|
||||
pub const BEACON_ATTESTATION_TOPIC: &str = "beacon_attestation";
|
||||
pub const VOLUNTARY_EXIT_TOPIC: &str = "voluntary_exit";
|
||||
pub const PROPOSER_SLASHING_TOPIC: &str = "proposer_slashing";
|
||||
pub const ATTESTER_SLASHING_TOPIC: &str = "attester_slashing";
|
||||
pub const SHARD_TOPIC_PREFIX: &str = "shard";
|
||||
|
||||
/// Enum that brings these topics into the rust type system.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub enum GossipTopic {
|
||||
BeaconBlock,
|
||||
BeaconAttestation,
|
||||
VoluntaryExit,
|
||||
ProposerSlashing,
|
||||
AttesterSlashing,
|
||||
Shard,
|
||||
Unknown(String),
|
||||
}
|
||||
|
||||
impl From<&str> for GossipTopic {
|
||||
fn from(topic: &str) -> GossipTopic {
|
||||
let topic_parts: Vec<&str> = topic.split('/').collect();
|
||||
if topic_parts.len() == 4
|
||||
&& topic_parts[1] == TOPIC_PREFIX
|
||||
&& topic_parts[3] == TOPIC_ENCODING_POSTFIX
|
||||
{
|
||||
match topic_parts[2] {
|
||||
BEACON_BLOCK_TOPIC => GossipTopic::BeaconBlock,
|
||||
BEACON_ATTESTATION_TOPIC => GossipTopic::BeaconAttestation,
|
||||
VOLUNTARY_EXIT_TOPIC => GossipTopic::VoluntaryExit,
|
||||
PROPOSER_SLASHING_TOPIC => GossipTopic::ProposerSlashing,
|
||||
ATTESTER_SLASHING_TOPIC => GossipTopic::AttesterSlashing,
|
||||
unknown_topic => GossipTopic::Unknown(unknown_topic.into()),
|
||||
}
|
||||
} else {
|
||||
GossipTopic::Unknown(topic.into())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Into<Topic> for GossipTopic {
|
||||
fn into(self) -> Topic {
|
||||
Topic::new(self.into())
|
||||
}
|
||||
}
|
||||
|
||||
impl Into<String> for GossipTopic {
|
||||
fn into(self) -> String {
|
||||
match self {
|
||||
GossipTopic::BeaconBlock => topic_builder(BEACON_BLOCK_TOPIC),
|
||||
GossipTopic::BeaconAttestation => topic_builder(BEACON_ATTESTATION_TOPIC),
|
||||
GossipTopic::VoluntaryExit => topic_builder(VOLUNTARY_EXIT_TOPIC),
|
||||
GossipTopic::ProposerSlashing => topic_builder(PROPOSER_SLASHING_TOPIC),
|
||||
GossipTopic::AttesterSlashing => topic_builder(ATTESTER_SLASHING_TOPIC),
|
||||
GossipTopic::Shard => topic_builder(SHARD_TOPIC_PREFIX),
|
||||
GossipTopic::Unknown(topic) => topic,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn topic_builder(topic: &'static str) -> String {
|
||||
format!("/{}/{}/{}", TOPIC_PREFIX, topic, TOPIC_ENCODING_POSTFIX,)
|
||||
}
|
||||
68
beacon_node/eth2-libp2p/src/types/globals.rs
Normal file
68
beacon_node/eth2-libp2p/src/types/globals.rs
Normal file
@ -0,0 +1,68 @@
|
||||
//! A collection of variables that are accessible outside of the network thread itself.
|
||||
use crate::{Enr, GossipTopic, Multiaddr, PeerId, PeerInfo};
|
||||
use parking_lot::RwLock;
|
||||
use std::collections::HashMap;
|
||||
use std::sync::atomic::{AtomicU16, Ordering};
|
||||
use types::EthSpec;
|
||||
|
||||
pub struct NetworkGlobals<TSpec: EthSpec> {
|
||||
/// The current local ENR.
|
||||
pub local_enr: RwLock<Option<Enr>>,
|
||||
/// The local peer_id.
|
||||
pub peer_id: RwLock<PeerId>,
|
||||
/// Listening multiaddrs.
|
||||
pub listen_multiaddrs: RwLock<Vec<Multiaddr>>,
|
||||
/// The tcp port that the libp2p service is listening on
|
||||
pub listen_port_tcp: AtomicU16,
|
||||
/// The udp port that the discovery service is listening on
|
||||
pub listen_port_udp: AtomicU16,
|
||||
/// The collection of currently connected peers.
|
||||
pub connected_peer_set: RwLock<HashMap<PeerId, PeerInfo<TSpec>>>,
|
||||
/// The current gossipsub topic subscriptions.
|
||||
pub gossipsub_subscriptions: RwLock<Vec<GossipTopic>>,
|
||||
}
|
||||
|
||||
impl<TSpec: EthSpec> NetworkGlobals<TSpec> {
|
||||
pub fn new(peer_id: PeerId, tcp_port: u16, udp_port: u16) -> Self {
|
||||
NetworkGlobals {
|
||||
local_enr: RwLock::new(None),
|
||||
peer_id: RwLock::new(peer_id),
|
||||
listen_multiaddrs: RwLock::new(Vec::new()),
|
||||
listen_port_tcp: AtomicU16::new(tcp_port),
|
||||
listen_port_udp: AtomicU16::new(udp_port),
|
||||
connected_peer_set: RwLock::new(HashMap::new()),
|
||||
gossipsub_subscriptions: RwLock::new(Vec::new()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns the local ENR from the underlying Discv5 behaviour that external peers may connect
|
||||
/// to.
|
||||
pub fn local_enr(&self) -> Option<Enr> {
|
||||
self.local_enr.read().clone()
|
||||
}
|
||||
|
||||
/// Returns the local libp2p PeerID.
|
||||
pub fn local_peer_id(&self) -> PeerId {
|
||||
self.peer_id.read().clone()
|
||||
}
|
||||
|
||||
/// Returns the list of `Multiaddr` that the underlying libp2p instance is listening on.
|
||||
pub fn listen_multiaddrs(&self) -> Vec<Multiaddr> {
|
||||
self.listen_multiaddrs.read().clone()
|
||||
}
|
||||
|
||||
/// Returns the libp2p TCP port that this node has been configured to listen on.
|
||||
pub fn listen_port_tcp(&self) -> u16 {
|
||||
self.listen_port_tcp.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
/// Returns the UDP discovery port that this node has been configured to listen on.
|
||||
pub fn listen_port_udp(&self) -> u16 {
|
||||
self.listen_port_udp.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
/// Returns the number of libp2p connected peers.
|
||||
pub fn connected_peers(&self) -> usize {
|
||||
self.connected_peer_set.read().len()
|
||||
}
|
||||
}
|
||||
10
beacon_node/eth2-libp2p/src/types/mod.rs
Normal file
10
beacon_node/eth2-libp2p/src/types/mod.rs
Normal file
@ -0,0 +1,10 @@
|
||||
pub mod error;
|
||||
mod globals;
|
||||
mod peer_info;
|
||||
mod pubsub;
|
||||
mod topics;
|
||||
|
||||
pub use globals::NetworkGlobals;
|
||||
pub use peer_info::{EnrBitfield, PeerInfo};
|
||||
pub use pubsub::{PubsubData, PubsubMessage};
|
||||
pub use topics::{GossipEncoding, GossipKind, GossipTopic};
|
||||
45
beacon_node/eth2-libp2p/src/types/peer_info.rs
Normal file
45
beacon_node/eth2-libp2p/src/types/peer_info.rs
Normal file
@ -0,0 +1,45 @@
|
||||
//NOTE: This should be removed in favour of the PeerManager PeerInfo, once built.
|
||||
|
||||
use types::{BitVector, EthSpec, SubnetId};
|
||||
|
||||
#[allow(type_alias_bounds)]
|
||||
pub type EnrBitfield<T: EthSpec> = BitVector<T::SubnetBitfieldLength>;
|
||||
|
||||
/// Information about a given connected peer.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PeerInfo<T: EthSpec> {
|
||||
/// The current syncing state of the peer. The state may be determined after it's initial
|
||||
/// connection.
|
||||
pub syncing_state: Option<PeerSyncingState>,
|
||||
/// The ENR subnet bitfield of the peer. This may be determined after it's initial
|
||||
/// connection.
|
||||
pub enr_bitfield: Option<EnrBitfield<T>>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum PeerSyncingState {
|
||||
/// At the current state as our node.
|
||||
Synced,
|
||||
/// The peer is further ahead than our node and useful for block downloads.
|
||||
Ahead,
|
||||
/// Is behind our current head and not useful for block downloads.
|
||||
Behind,
|
||||
}
|
||||
|
||||
impl<T: EthSpec> PeerInfo<T> {
|
||||
/// Creates a new PeerInfo, specifying it's
|
||||
pub fn new() -> Self {
|
||||
PeerInfo {
|
||||
syncing_state: None,
|
||||
enr_bitfield: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns if the peer is subscribed to a given `SubnetId`
|
||||
pub fn on_subnet(&self, subnet_id: SubnetId) -> bool {
|
||||
if let Some(bitfield) = &self.enr_bitfield {
|
||||
return bitfield.get(*subnet_id as usize).unwrap_or_else(|_| false);
|
||||
}
|
||||
false
|
||||
}
|
||||
}
|
||||
170
beacon_node/eth2-libp2p/src/types/pubsub.rs
Normal file
170
beacon_node/eth2-libp2p/src/types/pubsub.rs
Normal file
@ -0,0 +1,170 @@
|
||||
//! Handles the encoding and decoding of pubsub messages.
|
||||
|
||||
use crate::types::{GossipEncoding, GossipKind, GossipTopic};
|
||||
use crate::TopicHash;
|
||||
use ssz::{Decode, Encode};
|
||||
use std::boxed::Box;
|
||||
use types::SubnetId;
|
||||
use types::{
|
||||
Attestation, AttesterSlashing, EthSpec, ProposerSlashing, SignedAggregateAndProof,
|
||||
SignedBeaconBlock, VoluntaryExit,
|
||||
};
|
||||
|
||||
/// Messages that are passed to and from the pubsub (Gossipsub) behaviour.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct PubsubMessage<T: EthSpec> {
|
||||
/// The encoding to be used to encode/decode the message
|
||||
pub encoding: GossipEncoding,
|
||||
/// The actual message being sent.
|
||||
pub data: PubsubData<T>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum PubsubData<T: EthSpec> {
|
||||
/// Gossipsub message providing notification of a new block.
|
||||
BeaconBlock(Box<SignedBeaconBlock<T>>),
|
||||
/// Gossipsub message providing notification of a Aggregate attestation and associated proof.
|
||||
AggregateAndProofAttestation(Box<SignedAggregateAndProof<T>>),
|
||||
/// Gossipsub message providing notification of a raw un-aggregated attestation with its shard id.
|
||||
Attestation(Box<(SubnetId, Attestation<T>)>),
|
||||
/// Gossipsub message providing notification of a voluntary exit.
|
||||
VoluntaryExit(Box<VoluntaryExit>),
|
||||
/// Gossipsub message providing notification of a new proposer slashing.
|
||||
ProposerSlashing(Box<ProposerSlashing>),
|
||||
/// Gossipsub message providing notification of a new attester slashing.
|
||||
AttesterSlashing(Box<AttesterSlashing<T>>),
|
||||
}
|
||||
|
||||
impl<T: EthSpec> PubsubMessage<T> {
|
||||
pub fn new(encoding: GossipEncoding, data: PubsubData<T>) -> Self {
|
||||
PubsubMessage { encoding, data }
|
||||
}
|
||||
|
||||
/// Returns the topics that each pubsub message will be sent across, given a supported
|
||||
/// gossipsub encoding.
|
||||
pub fn topics(&self) -> Vec<GossipTopic> {
|
||||
let encoding = self.encoding.clone();
|
||||
match &self.data {
|
||||
PubsubData::BeaconBlock(_) => vec![GossipTopic::new(GossipKind::BeaconBlock, encoding)],
|
||||
PubsubData::AggregateAndProofAttestation(_) => vec![GossipTopic::new(
|
||||
GossipKind::BeaconAggregateAndProof,
|
||||
encoding,
|
||||
)],
|
||||
PubsubData::Attestation(attestation_data) => vec![GossipTopic::new(
|
||||
GossipKind::CommitteeIndex(attestation_data.0),
|
||||
encoding,
|
||||
)],
|
||||
PubsubData::VoluntaryExit(_) => {
|
||||
vec![GossipTopic::new(GossipKind::VoluntaryExit, encoding)]
|
||||
}
|
||||
PubsubData::ProposerSlashing(_) => {
|
||||
vec![GossipTopic::new(GossipKind::ProposerSlashing, encoding)]
|
||||
}
|
||||
PubsubData::AttesterSlashing(_) => {
|
||||
vec![GossipTopic::new(GossipKind::AttesterSlashing, encoding)]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Note: This is assuming we are not hashing topics. If we choose to hash topics, these will
|
||||
* need to be modified.
|
||||
*
|
||||
* Also note that a message can be associated with many topics. As soon as one of the topics is
|
||||
* known we match. If none of the topics are known we return an unknown state.
|
||||
*/
|
||||
pub fn decode(topics: &[TopicHash], data: &[u8]) -> Result<Self, String> {
|
||||
let mut unknown_topics = Vec::new();
|
||||
for topic in topics {
|
||||
match GossipTopic::decode(topic.as_str()) {
|
||||
Err(_) => {
|
||||
unknown_topics.push(topic);
|
||||
continue;
|
||||
}
|
||||
Ok(gossip_topic) => {
|
||||
match gossip_topic.encoding() {
|
||||
// group each part by encoding type
|
||||
GossipEncoding::SSZ => {
|
||||
// the ssz decoders
|
||||
let encoding = GossipEncoding::SSZ;
|
||||
match gossip_topic.kind() {
|
||||
GossipKind::BeaconAggregateAndProof => {
|
||||
let agg_and_proof =
|
||||
SignedAggregateAndProof::from_ssz_bytes(data)
|
||||
.map_err(|e| format!("{:?}", e))?;
|
||||
return Ok(PubsubMessage::new(
|
||||
encoding,
|
||||
PubsubData::AggregateAndProofAttestation(Box::new(
|
||||
agg_and_proof,
|
||||
)),
|
||||
));
|
||||
}
|
||||
GossipKind::CommitteeIndex(subnet_id) => {
|
||||
let attestation = Attestation::from_ssz_bytes(data)
|
||||
.map_err(|e| format!("{:?}", e))?;
|
||||
return Ok(PubsubMessage::new(
|
||||
encoding,
|
||||
PubsubData::Attestation(Box::new((
|
||||
*subnet_id,
|
||||
attestation,
|
||||
))),
|
||||
));
|
||||
}
|
||||
GossipKind::BeaconBlock => {
|
||||
let beacon_block = SignedBeaconBlock::from_ssz_bytes(data)
|
||||
.map_err(|e| format!("{:?}", e))?;
|
||||
return Ok(PubsubMessage::new(
|
||||
encoding,
|
||||
PubsubData::BeaconBlock(Box::new(beacon_block)),
|
||||
));
|
||||
}
|
||||
GossipKind::VoluntaryExit => {
|
||||
let voluntary_exit = VoluntaryExit::from_ssz_bytes(data)
|
||||
.map_err(|e| format!("{:?}", e))?;
|
||||
return Ok(PubsubMessage::new(
|
||||
encoding,
|
||||
PubsubData::VoluntaryExit(Box::new(voluntary_exit)),
|
||||
));
|
||||
}
|
||||
GossipKind::ProposerSlashing => {
|
||||
let proposer_slashing = ProposerSlashing::from_ssz_bytes(data)
|
||||
.map_err(|e| format!("{:?}", e))?;
|
||||
return Ok(PubsubMessage::new(
|
||||
encoding,
|
||||
PubsubData::ProposerSlashing(Box::new(proposer_slashing)),
|
||||
));
|
||||
}
|
||||
GossipKind::AttesterSlashing => {
|
||||
let attester_slashing = AttesterSlashing::from_ssz_bytes(data)
|
||||
.map_err(|e| format!("{:?}", e))?;
|
||||
return Ok(PubsubMessage::new(
|
||||
encoding,
|
||||
PubsubData::AttesterSlashing(Box::new(attester_slashing)),
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(format!("Unknown gossipsub topics: {:?}", unknown_topics))
|
||||
}
|
||||
|
||||
/// Encodes a pubsub message based on the topic encodings. The first known encoding is used. If
|
||||
/// no encoding is known, and error is returned.
|
||||
pub fn encode(&self) -> Vec<u8> {
|
||||
match self.encoding {
|
||||
GossipEncoding::SSZ => {
|
||||
// SSZ Encodings
|
||||
return match &self.data {
|
||||
PubsubData::BeaconBlock(data) => data.as_ssz_bytes(),
|
||||
PubsubData::AggregateAndProofAttestation(data) => data.as_ssz_bytes(),
|
||||
PubsubData::VoluntaryExit(data) => data.as_ssz_bytes(),
|
||||
PubsubData::ProposerSlashing(data) => data.as_ssz_bytes(),
|
||||
PubsubData::AttesterSlashing(data) => data.as_ssz_bytes(),
|
||||
PubsubData::Attestation(data) => data.1.as_ssz_bytes(),
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
140
beacon_node/eth2-libp2p/src/types/topics.rs
Normal file
140
beacon_node/eth2-libp2p/src/types/topics.rs
Normal file
@ -0,0 +1,140 @@
|
||||
use libp2p::gossipsub::Topic;
|
||||
use serde_derive::{Deserialize, Serialize};
|
||||
use types::SubnetId;
|
||||
|
||||
/// The gossipsub topic names.
|
||||
// These constants form a topic name of the form /TOPIC_PREFIX/TOPIC/ENCODING_POSTFIX
|
||||
// For example /eth2/beacon_block/ssz
|
||||
pub const TOPIC_PREFIX: &str = "eth2";
|
||||
pub const SSZ_ENCODING_POSTFIX: &str = "ssz";
|
||||
pub const BEACON_BLOCK_TOPIC: &str = "beacon_block";
|
||||
pub const BEACON_AGGREGATE_AND_PROOF_TOPIC: &str = "beacon_aggregate_and_proof";
|
||||
// for speed and easier string manipulation, committee topic index is split into a prefix and a
|
||||
// postfix. The topic is committee_index{}_beacon_attestation where {} is an integer.
|
||||
pub const COMMITEE_INDEX_TOPIC_PREFIX: &str = "committee_index";
|
||||
pub const COMMITEE_INDEX_TOPIC_POSTFIX: &str = "_beacon_attestation";
|
||||
pub const VOLUNTARY_EXIT_TOPIC: &str = "voluntary_exit";
|
||||
pub const PROPOSER_SLASHING_TOPIC: &str = "proposer_slashing";
|
||||
pub const ATTESTER_SLASHING_TOPIC: &str = "attester_slashing";
|
||||
|
||||
/// A gossipsub topic which encapsulates the type of messages that should be sent and received over
|
||||
/// the pubsub protocol and the way the messages should be encoded.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
pub struct GossipTopic {
|
||||
/// The encoding of the topic.
|
||||
encoding: GossipEncoding,
|
||||
/// The kind of topic.
|
||||
kind: GossipKind,
|
||||
}
|
||||
|
||||
/// Enum that brings these topics into the rust type system.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
pub enum GossipKind {
|
||||
/// Topic for publishing beacon blocks.
|
||||
BeaconBlock,
|
||||
/// Topic for publishing aggregate attestations and proofs.
|
||||
BeaconAggregateAndProof,
|
||||
/// Topic for publishing raw attestations on a particular subnet.
|
||||
CommitteeIndex(SubnetId),
|
||||
/// Topic for publishing voluntary exits.
|
||||
VoluntaryExit,
|
||||
/// Topic for publishing block proposer slashings.
|
||||
ProposerSlashing,
|
||||
/// Topic for publishing attester slashings.
|
||||
AttesterSlashing,
|
||||
}
|
||||
|
||||
/// The known encoding types for gossipsub messages.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
pub enum GossipEncoding {
|
||||
/// Messages are encoded with SSZ.
|
||||
SSZ,
|
||||
}
|
||||
|
||||
impl GossipTopic {
|
||||
pub fn new(kind: GossipKind, encoding: GossipEncoding) -> Self {
|
||||
GossipTopic { encoding, kind }
|
||||
}
|
||||
|
||||
/// Returns the encoding type for the gossipsub topic.
|
||||
pub fn encoding(&self) -> &GossipEncoding {
|
||||
&self.encoding
|
||||
}
|
||||
|
||||
/// Returns the kind of message expected on the gossipsub topic.
|
||||
pub fn kind(&self) -> &GossipKind {
|
||||
&self.kind
|
||||
}
|
||||
|
||||
pub fn decode(topic: &str) -> Result<Self, String> {
|
||||
let topic_parts: Vec<&str> = topic.split('/').collect();
|
||||
if topic_parts.len() == 4 && topic_parts[1] == TOPIC_PREFIX {
|
||||
let encoding = match topic_parts[3] {
|
||||
SSZ_ENCODING_POSTFIX => GossipEncoding::SSZ,
|
||||
_ => return Err(format!("Unknown encoding: {}", topic)),
|
||||
};
|
||||
let kind = match topic_parts[2] {
|
||||
BEACON_BLOCK_TOPIC => GossipKind::BeaconBlock,
|
||||
BEACON_AGGREGATE_AND_PROOF_TOPIC => GossipKind::BeaconAggregateAndProof,
|
||||
VOLUNTARY_EXIT_TOPIC => GossipKind::VoluntaryExit,
|
||||
PROPOSER_SLASHING_TOPIC => GossipKind::ProposerSlashing,
|
||||
ATTESTER_SLASHING_TOPIC => GossipKind::AttesterSlashing,
|
||||
topic => match committee_topic_index(topic) {
|
||||
Some(subnet_id) => GossipKind::CommitteeIndex(subnet_id),
|
||||
None => return Err(format!("Unknown topic: {}", topic)),
|
||||
},
|
||||
};
|
||||
|
||||
return Ok(GossipTopic { encoding, kind });
|
||||
}
|
||||
|
||||
Err(format!("Unknown topic: {}", topic))
|
||||
}
|
||||
}
|
||||
|
||||
impl Into<Topic> for GossipTopic {
|
||||
fn into(self) -> Topic {
|
||||
Topic::new(self.into())
|
||||
}
|
||||
}
|
||||
|
||||
impl Into<String> for GossipTopic {
|
||||
fn into(self) -> String {
|
||||
let encoding = match self.encoding {
|
||||
GossipEncoding::SSZ => SSZ_ENCODING_POSTFIX,
|
||||
};
|
||||
|
||||
let kind = match self.kind {
|
||||
GossipKind::BeaconBlock => BEACON_BLOCK_TOPIC.into(),
|
||||
GossipKind::BeaconAggregateAndProof => BEACON_AGGREGATE_AND_PROOF_TOPIC.into(),
|
||||
GossipKind::VoluntaryExit => VOLUNTARY_EXIT_TOPIC.into(),
|
||||
GossipKind::ProposerSlashing => PROPOSER_SLASHING_TOPIC.into(),
|
||||
GossipKind::AttesterSlashing => ATTESTER_SLASHING_TOPIC.into(),
|
||||
GossipKind::CommitteeIndex(index) => format!(
|
||||
"{}{}{}",
|
||||
COMMITEE_INDEX_TOPIC_PREFIX, *index, COMMITEE_INDEX_TOPIC_POSTFIX
|
||||
),
|
||||
};
|
||||
format!("/{}/{}/{}", TOPIC_PREFIX, kind, encoding)
|
||||
}
|
||||
}
|
||||
|
||||
// helper functions
|
||||
|
||||
// Determines if a string is a committee topic.
|
||||
fn committee_topic_index(topic: &str) -> Option<SubnetId> {
|
||||
if topic.starts_with(COMMITEE_INDEX_TOPIC_PREFIX)
|
||||
&& topic.ends_with(COMMITEE_INDEX_TOPIC_POSTFIX)
|
||||
{
|
||||
return Some(SubnetId::new(
|
||||
u64::from_str_radix(
|
||||
topic
|
||||
.trim_start_matches(COMMITEE_INDEX_TOPIC_PREFIX)
|
||||
.trim_end_matches(COMMITEE_INDEX_TOPIC_POSTFIX),
|
||||
10,
|
||||
)
|
||||
.ok()?,
|
||||
));
|
||||
}
|
||||
None
|
||||
}
|
||||
@ -5,6 +5,9 @@ use eth2_libp2p::NetworkConfig;
|
||||
use eth2_libp2p::Service as LibP2PService;
|
||||
use slog::{debug, error, o, Drain};
|
||||
use std::time::Duration;
|
||||
use types::MinimalEthSpec;
|
||||
|
||||
type E = MinimalEthSpec;
|
||||
use tempdir::TempDir;
|
||||
|
||||
pub fn build_log(level: slog::Level, enabled: bool) -> slog::Logger {
|
||||
@ -43,7 +46,7 @@ pub fn build_libp2p_instance(
|
||||
boot_nodes: Vec<Enr>,
|
||||
secret_key: Option<String>,
|
||||
log: slog::Logger,
|
||||
) -> LibP2PService {
|
||||
) -> LibP2PService<E> {
|
||||
let config = build_config(port, boot_nodes, secret_key);
|
||||
// launch libp2p service
|
||||
LibP2PService::new(&config, log.clone())
|
||||
@ -52,15 +55,19 @@ pub fn build_libp2p_instance(
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub fn get_enr(node: &LibP2PService) -> Enr {
|
||||
pub fn get_enr(node: &LibP2PService<E>) -> Enr {
|
||||
node.swarm.discovery().local_enr().clone()
|
||||
}
|
||||
|
||||
// Returns `n` libp2p peers in fully connected topology.
|
||||
#[allow(dead_code)]
|
||||
pub fn build_full_mesh(log: slog::Logger, n: usize, start_port: Option<u16>) -> Vec<LibP2PService> {
|
||||
pub fn build_full_mesh(
|
||||
log: slog::Logger,
|
||||
n: usize,
|
||||
start_port: Option<u16>,
|
||||
) -> Vec<LibP2PService<E>> {
|
||||
let base_port = start_port.unwrap_or(9000);
|
||||
let mut nodes: Vec<LibP2PService> = (base_port..base_port + n as u16)
|
||||
let mut nodes: Vec<LibP2PService<E>> = (base_port..base_port + n as u16)
|
||||
.map(|p| build_libp2p_instance(p, vec![], None, log.clone()))
|
||||
.collect();
|
||||
let multiaddrs: Vec<Multiaddr> = nodes
|
||||
@ -84,7 +91,10 @@ pub fn build_full_mesh(log: slog::Logger, n: usize, start_port: Option<u16>) ->
|
||||
// Constructs a pair of nodes with seperate loggers. The sender dials the receiver.
|
||||
// This returns a (sender, receiver) pair.
|
||||
#[allow(dead_code)]
|
||||
pub fn build_node_pair(log: &slog::Logger, start_port: u16) -> (LibP2PService, LibP2PService) {
|
||||
pub fn build_node_pair(
|
||||
log: &slog::Logger,
|
||||
start_port: u16,
|
||||
) -> (LibP2PService<E>, LibP2PService<E>) {
|
||||
let sender_log = log.new(o!("who" => "sender"));
|
||||
let receiver_log = log.new(o!("who" => "receiver"));
|
||||
|
||||
@ -101,9 +111,9 @@ pub fn build_node_pair(log: &slog::Logger, start_port: u16) -> (LibP2PService, L
|
||||
|
||||
// Returns `n` peers in a linear topology
|
||||
#[allow(dead_code)]
|
||||
pub fn build_linear(log: slog::Logger, n: usize, start_port: Option<u16>) -> Vec<LibP2PService> {
|
||||
pub fn build_linear(log: slog::Logger, n: usize, start_port: Option<u16>) -> Vec<LibP2PService<E>> {
|
||||
let base_port = start_port.unwrap_or(9000);
|
||||
let mut nodes: Vec<LibP2PService> = (base_port..base_port + n as u16)
|
||||
let mut nodes: Vec<LibP2PService<E>> = (base_port..base_port + n as u16)
|
||||
.map(|p| build_libp2p_instance(p, vec![], None, log.clone()))
|
||||
.collect();
|
||||
let multiaddrs: Vec<Multiaddr> = nodes
|
||||
|
||||
@ -1,8 +1,12 @@
|
||||
#![cfg(test)]
|
||||
use crate::types::GossipEncoding;
|
||||
use ::types::{BeaconBlock, EthSpec, MinimalEthSpec, Signature, SignedBeaconBlock};
|
||||
use eth2_libp2p::*;
|
||||
use futures::prelude::*;
|
||||
use slog::{debug, Level};
|
||||
|
||||
type E = MinimalEthSpec;
|
||||
|
||||
mod common;
|
||||
|
||||
/* Gossipsub tests */
|
||||
@ -23,7 +27,14 @@ fn test_gossipsub_forward() {
|
||||
let num_nodes = 20;
|
||||
let mut nodes = common::build_linear(log.clone(), num_nodes, Some(19000));
|
||||
let mut received_count = 0;
|
||||
let pubsub_message = PubsubMessage::Block(vec![0; 4]);
|
||||
let spec = E::default_spec();
|
||||
let empty_block = BeaconBlock::empty(&spec);
|
||||
let signed_block = SignedBeaconBlock {
|
||||
message: empty_block,
|
||||
signature: Signature::empty_signature(),
|
||||
};
|
||||
let data = PubsubData::BeaconBlock(Box::new(signed_block));
|
||||
let pubsub_message = PubsubMessage::new(GossipEncoding::SSZ, data);
|
||||
let publishing_topic: String = "/eth2/beacon_block/ssz".into();
|
||||
let mut subscribed_count = 0;
|
||||
tokio::run(futures::future::poll_fn(move || -> Result<_, ()> {
|
||||
@ -61,10 +72,7 @@ fn test_gossipsub_forward() {
|
||||
subscribed_count += 1;
|
||||
// Every node except the corner nodes are connected to 2 nodes.
|
||||
if subscribed_count == (num_nodes * 2) - 2 {
|
||||
node.swarm.publish(
|
||||
&[Topic::new(topic.into_string())],
|
||||
pubsub_message.clone(),
|
||||
);
|
||||
node.swarm.publish(vec![pubsub_message.clone()]);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -90,7 +98,14 @@ fn test_gossipsub_full_mesh_publish() {
|
||||
let num_nodes = 12;
|
||||
let mut nodes = common::build_full_mesh(log, num_nodes, Some(11320));
|
||||
let mut publishing_node = nodes.pop().unwrap();
|
||||
let pubsub_message = PubsubMessage::Block(vec![0; 4]);
|
||||
let spec = E::default_spec();
|
||||
let empty_block = BeaconBlock::empty(&spec);
|
||||
let signed_block = SignedBeaconBlock {
|
||||
message: empty_block,
|
||||
signature: Signature::empty_signature(),
|
||||
};
|
||||
let data = PubsubData::BeaconBlock(Box::new(signed_block));
|
||||
let pubsub_message = PubsubMessage::new(GossipEncoding::SSZ, data);
|
||||
let publishing_topic: String = "/eth2/beacon_block/ssz".into();
|
||||
let mut subscribed_count = 0;
|
||||
let mut received_count = 0;
|
||||
@ -123,9 +138,7 @@ fn test_gossipsub_full_mesh_publish() {
|
||||
if topic == TopicHash::from_raw("/eth2/beacon_block/ssz") {
|
||||
subscribed_count += 1;
|
||||
if subscribed_count == num_nodes - 1 {
|
||||
publishing_node
|
||||
.swarm
|
||||
.publish(&[Topic::new(topic.into_string())], pubsub_message.clone());
|
||||
publishing_node.swarm.publish(vec![pubsub_message.clone()]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
#![cfg(test)]
|
||||
use crate::behaviour::{Behaviour, BehaviourEvent};
|
||||
use crate::multiaddr::Protocol;
|
||||
use ::types::MinimalEthSpec;
|
||||
use eth2_libp2p::*;
|
||||
use futures::prelude::*;
|
||||
use libp2p::core::identity::Keypair;
|
||||
@ -16,10 +17,12 @@ use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use tokio::prelude::*;
|
||||
|
||||
type TSpec = MinimalEthSpec;
|
||||
|
||||
mod common;
|
||||
|
||||
type Libp2pStream = Boxed<(PeerId, StreamMuxerBox), Error>;
|
||||
type Libp2pBehaviour = Behaviour<Substream<StreamMuxerBox>>;
|
||||
type Libp2pBehaviour = Behaviour<Substream<StreamMuxerBox>, TSpec>;
|
||||
|
||||
/// Build and return a eth2_libp2p Swarm with only secio support.
|
||||
fn build_secio_swarm(
|
||||
@ -29,7 +32,11 @@ fn build_secio_swarm(
|
||||
let local_keypair = Keypair::generate_secp256k1();
|
||||
let local_peer_id = PeerId::from(local_keypair.public());
|
||||
|
||||
let network_globals = Arc::new(NetworkGlobals::new(local_peer_id.clone()));
|
||||
let network_globals = Arc::new(NetworkGlobals::new(
|
||||
local_peer_id.clone(),
|
||||
config.libp2p_port,
|
||||
config.discovery_port,
|
||||
));
|
||||
|
||||
let mut swarm = {
|
||||
// Set up the transport - tcp/ws with secio and mplex/yamux
|
||||
|
||||
@ -7,10 +7,14 @@ use std::sync::atomic::{AtomicBool, Ordering::Relaxed};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::Duration;
|
||||
use tokio::prelude::*;
|
||||
use types::{Epoch, Hash256, Slot};
|
||||
use types::{
|
||||
BeaconBlock, Epoch, EthSpec, Hash256, MinimalEthSpec, Signature, SignedBeaconBlock, Slot,
|
||||
};
|
||||
|
||||
mod common;
|
||||
|
||||
type E = MinimalEthSpec;
|
||||
|
||||
#[test]
|
||||
// Tests the STATUS RPC message
|
||||
fn test_status_rpc() {
|
||||
@ -73,7 +77,7 @@ fn test_status_rpc() {
|
||||
warn!(sender_log, "Sender Completed");
|
||||
return Ok(Async::Ready(true));
|
||||
}
|
||||
_ => panic!("Received invalid RPC message"),
|
||||
e => panic!("Received invalid RPC message {}", e),
|
||||
},
|
||||
Async::Ready(Some(_)) => (),
|
||||
Async::Ready(None) | Async::NotReady => return Ok(Async::NotReady),
|
||||
@ -98,7 +102,7 @@ fn test_status_rpc() {
|
||||
RPCEvent::Response(id, RPCErrorResponse::Success(rpc_response.clone())),
|
||||
);
|
||||
}
|
||||
_ => panic!("Received invalid RPC message"),
|
||||
e => panic!("Received invalid RPC message {}", e),
|
||||
},
|
||||
Async::Ready(Some(_)) => (),
|
||||
Async::Ready(None) | Async::NotReady => return Ok(Async::NotReady),
|
||||
@ -145,7 +149,13 @@ fn test_blocks_by_range_chunked_rpc() {
|
||||
});
|
||||
|
||||
// BlocksByRange Response
|
||||
let rpc_response = RPCResponse::BlocksByRange(vec![13, 13, 13]);
|
||||
let spec = E::default_spec();
|
||||
let empty_block = BeaconBlock::empty(&spec);
|
||||
let empty_signed = SignedBeaconBlock {
|
||||
message: empty_block,
|
||||
signature: Signature::empty_signature(),
|
||||
};
|
||||
let rpc_response = RPCResponse::BlocksByRange(Box::new(empty_signed));
|
||||
|
||||
let sender_request = rpc_request.clone();
|
||||
let sender_log = log.clone();
|
||||
@ -272,7 +282,13 @@ fn test_blocks_by_range_single_empty_rpc() {
|
||||
});
|
||||
|
||||
// BlocksByRange Response
|
||||
let rpc_response = RPCResponse::BlocksByRange(vec![]);
|
||||
let spec = E::default_spec();
|
||||
let empty_block = BeaconBlock::empty(&spec);
|
||||
let empty_signed = SignedBeaconBlock {
|
||||
message: empty_block,
|
||||
signature: Signature::empty_signature(),
|
||||
};
|
||||
let rpc_response = RPCResponse::BlocksByRange(Box::new(empty_signed));
|
||||
|
||||
let sender_request = rpc_request.clone();
|
||||
let sender_log = log.clone();
|
||||
@ -373,132 +389,6 @@ fn test_blocks_by_range_single_empty_rpc() {
|
||||
assert!(test_result.load(Relaxed));
|
||||
}
|
||||
|
||||
#[test]
|
||||
// Tests a streamed, chunked BlocksByRoot RPC Message
|
||||
fn test_blocks_by_root_chunked_rpc() {
|
||||
// set up the logging. The level and enabled logging or not
|
||||
let log_level = Level::Trace;
|
||||
let enable_logging = false;
|
||||
|
||||
let messages_to_send = 3;
|
||||
|
||||
let log = common::build_log(log_level, enable_logging);
|
||||
|
||||
// get sender/receiver
|
||||
let (mut sender, mut receiver) = common::build_node_pair(&log, 10515);
|
||||
|
||||
// BlocksByRoot Request
|
||||
let rpc_request = RPCRequest::BlocksByRoot(BlocksByRootRequest {
|
||||
block_roots: vec![Hash256::from_low_u64_be(0), Hash256::from_low_u64_be(0)],
|
||||
});
|
||||
|
||||
// BlocksByRoot Response
|
||||
let rpc_response = RPCResponse::BlocksByRoot(vec![13, 13, 13]);
|
||||
|
||||
let sender_request = rpc_request.clone();
|
||||
let sender_log = log.clone();
|
||||
let sender_response = rpc_response.clone();
|
||||
|
||||
// keep count of the number of messages received
|
||||
let messages_received = Arc::new(Mutex::new(0));
|
||||
// build the sender future
|
||||
let sender_future = future::poll_fn(move || -> Poll<bool, ()> {
|
||||
loop {
|
||||
match sender.poll().unwrap() {
|
||||
Async::Ready(Some(Libp2pEvent::PeerDialed(peer_id))) => {
|
||||
// Send a BlocksByRoot request
|
||||
warn!(sender_log, "Sender sending RPC request");
|
||||
sender
|
||||
.swarm
|
||||
.send_rpc(peer_id, RPCEvent::Request(1, sender_request.clone()));
|
||||
}
|
||||
Async::Ready(Some(Libp2pEvent::RPC(_, event))) => match event {
|
||||
// Should receive the RPC response
|
||||
RPCEvent::Response(id, response) => {
|
||||
warn!(sender_log, "Sender received a response");
|
||||
assert_eq!(id, 1);
|
||||
match response {
|
||||
RPCErrorResponse::Success(res) => {
|
||||
assert_eq!(res, sender_response.clone());
|
||||
*messages_received.lock().unwrap() += 1;
|
||||
warn!(sender_log, "Chunk received");
|
||||
}
|
||||
RPCErrorResponse::StreamTermination(
|
||||
ResponseTermination::BlocksByRoot,
|
||||
) => {
|
||||
// should be exactly 10 messages before terminating
|
||||
assert_eq!(*messages_received.lock().unwrap(), messages_to_send);
|
||||
// end the test
|
||||
return Ok(Async::Ready(true));
|
||||
}
|
||||
m => panic!("Invalid RPC received: {}", m),
|
||||
}
|
||||
}
|
||||
m => panic!("Received invalid RPC message: {}", m),
|
||||
},
|
||||
Async::Ready(Some(_)) => {}
|
||||
Async::Ready(None) | Async::NotReady => return Ok(Async::NotReady),
|
||||
};
|
||||
}
|
||||
});
|
||||
|
||||
// build the receiver future
|
||||
let receiver_future = future::poll_fn(move || -> Poll<bool, ()> {
|
||||
loop {
|
||||
match receiver.poll().unwrap() {
|
||||
Async::Ready(Some(Libp2pEvent::RPC(peer_id, event))) => match event {
|
||||
// Should receive the sent RPC request
|
||||
RPCEvent::Request(id, request) => {
|
||||
assert_eq!(id, 1);
|
||||
assert_eq!(rpc_request.clone(), request);
|
||||
|
||||
// send the response
|
||||
warn!(log, "Receiver got request");
|
||||
|
||||
for _ in 1..=messages_to_send {
|
||||
receiver.swarm.send_rpc(
|
||||
peer_id.clone(),
|
||||
RPCEvent::Response(
|
||||
id,
|
||||
RPCErrorResponse::Success(rpc_response.clone()),
|
||||
),
|
||||
);
|
||||
}
|
||||
// send the stream termination
|
||||
receiver.swarm.send_rpc(
|
||||
peer_id,
|
||||
RPCEvent::Response(
|
||||
id,
|
||||
RPCErrorResponse::StreamTermination(
|
||||
ResponseTermination::BlocksByRoot,
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
_ => panic!("Received invalid RPC message"),
|
||||
},
|
||||
Async::Ready(Some(_)) => (),
|
||||
Async::Ready(None) | Async::NotReady => return Ok(Async::NotReady),
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// execute the futures and check the result
|
||||
let test_result = Arc::new(AtomicBool::new(false));
|
||||
let error_result = test_result.clone();
|
||||
let thread_result = test_result.clone();
|
||||
tokio::run(
|
||||
sender_future
|
||||
.select(receiver_future)
|
||||
.timeout(Duration::from_millis(1000))
|
||||
.map_err(move |_| error_result.store(false, Relaxed))
|
||||
.map(move |result| {
|
||||
thread_result.store(result.0, Relaxed);
|
||||
}),
|
||||
);
|
||||
assert!(test_result.load(Relaxed));
|
||||
}
|
||||
|
||||
#[test]
|
||||
// Tests a Goodbye RPC message
|
||||
fn test_goodbye_rpc() {
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "genesis"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Paul Hauner <paul@paulhauner.com>"]
|
||||
edition = "2018"
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "network"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Age Manning <Age@AgeManning.com>"]
|
||||
edition = "2018"
|
||||
|
||||
@ -13,7 +13,10 @@ tempdir = "0.3"
|
||||
beacon_chain = { path = "../beacon_chain" }
|
||||
store = { path = "../store" }
|
||||
eth2-libp2p = { path = "../eth2-libp2p" }
|
||||
hashmap_delay = { path = "../../eth2/utils/hashmap_delay" }
|
||||
rest_types = { path = "../../eth2/utils/rest_types" }
|
||||
types = { path = "../../eth2/types" }
|
||||
slot_clock = { path = "../../eth2/utils/slot_clock" }
|
||||
slog = { version = "2.5.2", features = ["max_level_trace"] }
|
||||
hex = "0.3"
|
||||
eth2_ssz = "0.1.2"
|
||||
|
||||
575
beacon_node/network/src/attestation_service/mod.rs
Normal file
575
beacon_node/network/src/attestation_service/mod.rs
Normal file
@ -0,0 +1,575 @@
|
||||
//! This service keeps track of which shard subnet the beacon node should be subscribed to at any
|
||||
//! given time. It schedules subscriptions to shard subnets, requests peer discoveries and
|
||||
//! determines whether attestations should be aggregated and/or passed to the beacon node.
|
||||
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use eth2_libp2p::{types::GossipKind, NetworkGlobals};
|
||||
use futures::prelude::*;
|
||||
use hashmap_delay::HashSetDelay;
|
||||
use rand::seq::SliceRandom;
|
||||
use rest_types::ValidatorSubscription;
|
||||
use slog::{crit, debug, error, o, warn};
|
||||
use slot_clock::SlotClock;
|
||||
use std::boxed::Box;
|
||||
use std::collections::VecDeque;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
use types::{Attestation, SubnetId};
|
||||
use types::{EthSpec, Slot};
|
||||
|
||||
/// The minimum number of slots ahead that we attempt to discover peers for a subscription. If the
|
||||
/// slot is less than this number, skip the peer discovery process.
|
||||
const MIN_PEER_DISCOVERY_SLOT_LOOK_AHEAD: u64 = 1;
|
||||
/// The number of slots ahead that we attempt to discover peers for a subscription. If the slot to
|
||||
/// attest to is greater than this, we queue a discovery request for this many slots prior to
|
||||
/// subscribing.
|
||||
const TARGET_PEER_DISCOVERY_SLOT_LOOK_AHEAD: u64 = 6;
|
||||
/// The time (in seconds) before a last seen validator is considered absent and we unsubscribe from the random
|
||||
/// gossip topics that we subscribed to due to the validator connection.
|
||||
const LAST_SEEN_VALIDATOR_TIMEOUT: u64 = 1800; // 30 mins
|
||||
/// The number of seconds in advance that we subscribe to a subnet before the required slot.
|
||||
const ADVANCE_SUBSCRIBE_SECS: u64 = 3;
|
||||
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum AttServiceMessage {
|
||||
/// Subscribe to the specified subnet id.
|
||||
Subscribe(SubnetId),
|
||||
/// Unsubscribe to the specified subnet id.
|
||||
Unsubscribe(SubnetId),
|
||||
/// Add the `SubnetId` to the ENR bitfield.
|
||||
EnrAdd(SubnetId),
|
||||
/// Remove the `SubnetId` from the ENR bitfield.
|
||||
EnrRemove(SubnetId),
|
||||
/// Discover peers for a particular subnet.
|
||||
DiscoverPeers(SubnetId),
|
||||
}
|
||||
|
||||
pub struct AttestationService<T: BeaconChainTypes> {
|
||||
/// Queued events to return to the driving service.
|
||||
events: VecDeque<AttServiceMessage>,
|
||||
|
||||
/// A collection of public network variables.
|
||||
network_globals: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
|
||||
/// A reference to the beacon chain to process received attestations.
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
|
||||
/// The collection of currently subscribed random subnets mapped to their expiry deadline.
|
||||
random_subnets: HashSetDelay<SubnetId>,
|
||||
|
||||
/// A collection of timeouts for when to start searching for peers for a particular shard.
|
||||
discover_peers: HashSetDelay<(SubnetId, Slot)>,
|
||||
|
||||
/// A collection of timeouts for when to subscribe to a shard subnet.
|
||||
subscriptions: HashSetDelay<(SubnetId, Slot)>,
|
||||
|
||||
/// A collection of timeouts for when to unsubscribe from a shard subnet.
|
||||
unsubscriptions: HashSetDelay<(SubnetId, Slot)>,
|
||||
|
||||
/// A collection of seen validators. These dictate how many random subnets we should be
|
||||
/// subscribed to. As these time out, we unsubscribe for the required random subnets and update
|
||||
/// our ENR.
|
||||
/// This is a set of validator indices.
|
||||
known_validators: HashSetDelay<u64>,
|
||||
|
||||
/// The logger for the attestation service.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> AttestationService<T> {
|
||||
/* Public functions */
|
||||
|
||||
pub fn new(
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network_globals: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
log: &slog::Logger,
|
||||
) -> Self {
|
||||
let log = log.new(o!("service" => "attestation_service"));
|
||||
|
||||
// calculate the random subnet duration from the spec constants
|
||||
let spec = &beacon_chain.spec;
|
||||
let random_subnet_duration_millis = spec
|
||||
.epochs_per_random_subnet_subscription
|
||||
.saturating_mul(T::EthSpec::slots_per_epoch())
|
||||
.saturating_mul(spec.milliseconds_per_slot);
|
||||
|
||||
AttestationService {
|
||||
events: VecDeque::with_capacity(10),
|
||||
network_globals,
|
||||
beacon_chain,
|
||||
random_subnets: HashSetDelay::new(Duration::from_millis(random_subnet_duration_millis)),
|
||||
discover_peers: HashSetDelay::default(),
|
||||
subscriptions: HashSetDelay::default(),
|
||||
unsubscriptions: HashSetDelay::default(),
|
||||
known_validators: HashSetDelay::new(Duration::from_secs(LAST_SEEN_VALIDATOR_TIMEOUT)),
|
||||
log,
|
||||
}
|
||||
}
|
||||
|
||||
/// Processes a list of validator subscriptions.
|
||||
///
|
||||
/// This will:
|
||||
/// - Register new validators as being known.
|
||||
/// - Subscribe to the required number of random subnets.
|
||||
/// - Update the local ENR for new random subnets due to seeing new validators.
|
||||
/// - Search for peers for required subnets.
|
||||
/// - Request subscriptions for subnets on specific slots when required.
|
||||
/// - Build the timeouts for each of these events.
|
||||
///
|
||||
/// This returns a result simply for the ergonomics of using ?. The result can be
|
||||
/// safely dropped.
|
||||
pub fn validator_subscriptions(
|
||||
&mut self,
|
||||
subscriptions: Vec<ValidatorSubscription>,
|
||||
) -> Result<(), ()> {
|
||||
for subscription in subscriptions {
|
||||
//NOTE: We assume all subscriptions have been verified before reaching this service
|
||||
|
||||
// Registers the validator with the attestation service.
|
||||
// This will subscribe to long-lived random subnets if required.
|
||||
self.add_known_validator(subscription.validator_index);
|
||||
|
||||
let subnet_id = SubnetId::new(
|
||||
subscription.attestation_committee_index
|
||||
% self.beacon_chain.spec.attestation_subnet_count,
|
||||
);
|
||||
// determine if we should run a discovery lookup request and request it if required
|
||||
let _ = self.discover_peers_request(subnet_id, subscription.slot);
|
||||
|
||||
// set the subscription timer to subscribe to the next subnet if required
|
||||
let _ = self.subscribe_to_subnet(subnet_id, subscription.slot);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn handle_attestation(
|
||||
&mut self,
|
||||
subnet: SubnetId,
|
||||
attestation: Box<Attestation<T::EthSpec>>,
|
||||
) {
|
||||
}
|
||||
|
||||
/* Internal private functions */
|
||||
|
||||
/// Checks if there are currently queued discovery requests and the time required to make the
|
||||
/// request.
|
||||
///
|
||||
/// If there is sufficient time and no other request exists, queues a peer discovery request
|
||||
/// for the required subnet.
|
||||
fn discover_peers_request(
|
||||
&mut self,
|
||||
subnet_id: SubnetId,
|
||||
subscription_slot: Slot,
|
||||
) -> Result<(), ()> {
|
||||
let current_slot = self.beacon_chain.slot_clock.now().ok_or_else(|| {
|
||||
warn!(self.log, "Could not get the current slot");
|
||||
})?;
|
||||
let slot_duration = Duration::from_millis(self.beacon_chain.spec.milliseconds_per_slot);
|
||||
|
||||
// if there is enough time to perform a discovery lookup
|
||||
if subscription_slot >= current_slot.saturating_add(MIN_PEER_DISCOVERY_SLOT_LOOK_AHEAD) {
|
||||
// check if a discovery request already exists
|
||||
if self
|
||||
.discover_peers
|
||||
.get(&(subnet_id, subscription_slot))
|
||||
.is_some()
|
||||
{
|
||||
// already a request queued, end
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// check current event log to see if there is a discovery event queued
|
||||
if self
|
||||
.events
|
||||
.iter()
|
||||
.find(|event| event == &&AttServiceMessage::DiscoverPeers(subnet_id))
|
||||
.is_some()
|
||||
{
|
||||
// already queued a discovery event
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// if the slot is more than epoch away, add an event to start looking for peers
|
||||
if subscription_slot
|
||||
< current_slot.saturating_add(TARGET_PEER_DISCOVERY_SLOT_LOOK_AHEAD)
|
||||
{
|
||||
// then instantly add a discovery request
|
||||
self.events
|
||||
.push_back(AttServiceMessage::DiscoverPeers(subnet_id));
|
||||
} else {
|
||||
// Queue the discovery event to be executed for
|
||||
// TARGET_PEER_DISCOVERY_SLOT_LOOK_AHEAD
|
||||
|
||||
let duration_to_discover = {
|
||||
let duration_to_next_slot = self
|
||||
.beacon_chain
|
||||
.slot_clock
|
||||
.duration_to_next_slot()
|
||||
.ok_or_else(|| {
|
||||
warn!(self.log, "Unable to determine duration to next slot");
|
||||
})?;
|
||||
// The -1 is done here to exclude the current slot duration, as we will use
|
||||
// `duration_to_next_slot`.
|
||||
let slots_until_discover = subscription_slot
|
||||
.saturating_sub(current_slot)
|
||||
.saturating_sub(1u64)
|
||||
.saturating_sub(TARGET_PEER_DISCOVERY_SLOT_LOOK_AHEAD);
|
||||
|
||||
duration_to_next_slot + slot_duration * (slots_until_discover.as_u64() as u32)
|
||||
};
|
||||
|
||||
self.discover_peers
|
||||
.insert_at((subnet_id, subscription_slot), duration_to_discover);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Checks the current random subnets and subscriptions to determine if a new subscription for this
|
||||
/// subnet is required for the given slot.
|
||||
///
|
||||
/// If required, adds a subscription event and an associated unsubscription event.
|
||||
fn subscribe_to_subnet(
|
||||
&mut self,
|
||||
subnet_id: SubnetId,
|
||||
subscription_slot: Slot,
|
||||
) -> Result<(), ()> {
|
||||
// initialise timing variables
|
||||
let current_slot = self.beacon_chain.slot_clock.now().ok_or_else(|| {
|
||||
warn!(self.log, "Could not get the current slot");
|
||||
})?;
|
||||
let slot_duration = Duration::from_millis(self.beacon_chain.spec.milliseconds_per_slot);
|
||||
let advance_subscription_duration = Duration::from_secs(ADVANCE_SUBSCRIBE_SECS);
|
||||
|
||||
// calculate the time to subscribe to the subnet
|
||||
let duration_to_subscribe = {
|
||||
let duration_to_next_slot = self
|
||||
.beacon_chain
|
||||
.slot_clock
|
||||
.duration_to_next_slot()
|
||||
.ok_or_else(|| {
|
||||
warn!(self.log, "Unable to determine duration to next slot");
|
||||
})?;
|
||||
// The -1 is done here to exclude the current slot duration, as we will use
|
||||
// `duration_to_next_slot`.
|
||||
let slots_until_subscribe = subscription_slot
|
||||
.saturating_sub(current_slot)
|
||||
.saturating_sub(1u64);
|
||||
|
||||
duration_to_next_slot + slot_duration * (slots_until_subscribe.as_u64() as u32)
|
||||
- advance_subscription_duration
|
||||
};
|
||||
// the duration until we no longer need this subscription. We assume a single slot is
|
||||
// sufficient.
|
||||
let expected_end_subscription_duration =
|
||||
duration_to_subscribe + slot_duration + advance_subscription_duration;
|
||||
|
||||
// Checks on current subscriptions
|
||||
// Note: We may be connected to a long-lived random subnet. In this case we still add the
|
||||
// subscription timeout and check this case when the timeout fires. This is because a
|
||||
// long-lived random subnet can be unsubscribed at any time when a validator becomes
|
||||
// in-active. This case is checked on the subscription event (see `handle_subscriptions`).
|
||||
|
||||
// Return if we already have a subscription for this subnet_id and slot
|
||||
if self.subscriptions.contains(&(subnet_id, subscription_slot)) {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// We are not currently subscribed and have no waiting subscription, create one
|
||||
self.subscriptions
|
||||
.insert_at((subnet_id, subscription_slot), duration_to_subscribe);
|
||||
|
||||
// if there is an unsubscription event for the slot prior, we remove it to prevent
|
||||
// unsubscriptions immediately after the subscription. We also want to minimize
|
||||
// subscription churn and maintain a consecutive subnet subscriptions.
|
||||
self.unsubscriptions
|
||||
.remove(&(subnet_id, subscription_slot.saturating_sub(1u64)));
|
||||
// add an unsubscription event to remove ourselves from the subnet once completed
|
||||
self.unsubscriptions.insert_at(
|
||||
(subnet_id, subscription_slot),
|
||||
expected_end_subscription_duration,
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Updates the `known_validators` mapping and subscribes to a set of random subnets if required.
|
||||
///
|
||||
/// This also updates the ENR to indicate our long-lived subscription to the subnet
|
||||
fn add_known_validator(&mut self, validator_index: u64) {
|
||||
if self.known_validators.get(&validator_index).is_none() {
|
||||
// New validator has subscribed
|
||||
// Subscribe to random topics and update the ENR if needed.
|
||||
|
||||
let spec = &self.beacon_chain.spec;
|
||||
|
||||
if self.random_subnets.len() < spec.attestation_subnet_count as usize {
|
||||
// Still room for subscriptions
|
||||
self.subscribe_to_random_subnets(
|
||||
self.beacon_chain.spec.random_subnets_per_validator as usize,
|
||||
);
|
||||
}
|
||||
}
|
||||
// add the new validator or update the current timeout for a known validator
|
||||
self.known_validators.insert(validator_index);
|
||||
}
|
||||
|
||||
/// Subscribe to long-lived random subnets and update the local ENR bitfield.
|
||||
fn subscribe_to_random_subnets(&mut self, no_subnets_to_subscribe: usize) {
|
||||
let subnet_count = self.beacon_chain.spec.attestation_subnet_count;
|
||||
|
||||
// Build a list of random subnets that we are not currently subscribed to.
|
||||
let available_subnets = (0..subnet_count)
|
||||
.map(SubnetId::new)
|
||||
.filter(|subnet_id| self.random_subnets.get(subnet_id).is_none())
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
let to_subscribe_subnets = {
|
||||
if available_subnets.len() < no_subnets_to_subscribe {
|
||||
debug!(self.log, "Reached maximum random subnet subscriptions");
|
||||
available_subnets
|
||||
} else {
|
||||
// select a random sample of available subnets
|
||||
available_subnets
|
||||
.choose_multiple(&mut rand::thread_rng(), no_subnets_to_subscribe)
|
||||
.cloned()
|
||||
.collect::<Vec<_>>()
|
||||
}
|
||||
};
|
||||
|
||||
for subnet_id in to_subscribe_subnets {
|
||||
// remove this subnet from any immediate subscription/un-subscription events
|
||||
self.subscriptions
|
||||
.retain(|(map_subnet_id, _)| map_subnet_id != &subnet_id);
|
||||
self.unsubscriptions
|
||||
.retain(|(map_subnet_id, _)| map_subnet_id != &subnet_id);
|
||||
|
||||
// insert a new random subnet
|
||||
self.random_subnets.insert(subnet_id);
|
||||
|
||||
// if we are not already subscribed, then subscribe
|
||||
let topic_kind = &GossipKind::CommitteeIndex(subnet_id);
|
||||
|
||||
if let None = self
|
||||
.network_globals
|
||||
.gossipsub_subscriptions
|
||||
.read()
|
||||
.iter()
|
||||
.find(|topic| topic.kind() == topic_kind)
|
||||
{
|
||||
// not already subscribed to the topic
|
||||
self.events
|
||||
.push_back(AttServiceMessage::Subscribe(subnet_id));
|
||||
}
|
||||
// add the subnet to the ENR bitfield
|
||||
self.events.push_back(AttServiceMessage::EnrAdd(subnet_id));
|
||||
}
|
||||
}
|
||||
|
||||
/* A collection of functions that handle the various timeouts */
|
||||
|
||||
/// Request a discovery query to find peers for a particular subnet.
|
||||
fn handle_discover_peers(&mut self, subnet_id: SubnetId, target_slot: Slot) {
|
||||
debug!(self.log, "Searching for peers for subnet"; "subnet" => *subnet_id, "target_slot" => target_slot);
|
||||
self.events
|
||||
.push_back(AttServiceMessage::DiscoverPeers(subnet_id));
|
||||
}
|
||||
|
||||
/// A queued subscription is ready.
|
||||
///
|
||||
/// We add subscriptions events even if we are already subscribed to a random subnet (as these
|
||||
/// can be unsubscribed at any time by inactive validators). If we are
|
||||
/// still subscribed at the time the event fires, we don't re-subscribe.
|
||||
fn handle_subscriptions(&mut self, subnet_id: SubnetId, target_slot: Slot) {
|
||||
// Check if the subnet currently exists as a long-lasting random subnet
|
||||
if let Some(expiry) = self.random_subnets.get(&subnet_id) {
|
||||
// we are subscribed via a random subnet, if this is to expire during the time we need
|
||||
// to be subscribed, just extend the expiry
|
||||
let slot_duration = Duration::from_millis(self.beacon_chain.spec.milliseconds_per_slot);
|
||||
let advance_subscription_duration = Duration::from_secs(ADVANCE_SUBSCRIBE_SECS);
|
||||
// we require the subnet subscription for at least a slot on top of the initial
|
||||
// subscription time
|
||||
let expected_end_subscription_duration = slot_duration + advance_subscription_duration;
|
||||
|
||||
if expiry < &(Instant::now() + expected_end_subscription_duration) {
|
||||
self.random_subnets
|
||||
.update_timeout(&subnet_id, expected_end_subscription_duration);
|
||||
}
|
||||
} else {
|
||||
// we are also not un-subscribing from a subnet if the next slot requires us to be
|
||||
// subscribed. Therefore there could be the case that we are already still subscribed
|
||||
// to the required subnet. In which case we do not issue another subscription request.
|
||||
let topic_kind = &GossipKind::CommitteeIndex(subnet_id);
|
||||
if self
|
||||
.network_globals
|
||||
.gossipsub_subscriptions
|
||||
.read()
|
||||
.iter()
|
||||
.find(|topic| topic.kind() == topic_kind)
|
||||
.is_none()
|
||||
{
|
||||
// we are not already subscribed
|
||||
debug!(self.log, "Subscribing to subnet"; "subnet" => *subnet_id, "target_slot" => target_slot.as_u64());
|
||||
self.events
|
||||
.push_back(AttServiceMessage::Subscribe(subnet_id));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A queued unsubscription is ready.
|
||||
///
|
||||
/// Unsubscription events are added, even if we are subscribed to long-lived random subnets. If
|
||||
/// a random subnet is present, we do not unsubscribe from it.
|
||||
fn handle_unsubscriptions(&mut self, subnet_id: SubnetId, target_slot: Slot) {
|
||||
// Check if the subnet currently exists as a long-lasting random subnet
|
||||
if self.random_subnets.contains(&subnet_id) {
|
||||
return;
|
||||
}
|
||||
|
||||
debug!(self.log, "Unsubscribing from subnet"; "subnet" => *subnet_id, "processed_slot" => target_slot.as_u64());
|
||||
|
||||
// various logic checks
|
||||
if self.subscriptions.contains(&(subnet_id, target_slot)) {
|
||||
crit!(self.log, "Unsubscribing from a subnet in subscriptions");
|
||||
}
|
||||
self.events
|
||||
.push_back(AttServiceMessage::Unsubscribe(subnet_id));
|
||||
}
|
||||
|
||||
/// A random subnet has expired.
|
||||
///
|
||||
/// This function selects a new subnet to join, or extends the expiry if there are no more
|
||||
/// available subnets to choose from.
|
||||
fn handle_random_subnet_expiry(&mut self, subnet_id: SubnetId) {
|
||||
let subnet_count = self.beacon_chain.spec.attestation_subnet_count;
|
||||
if self.random_subnets.len() == (subnet_count - 1) as usize {
|
||||
// We are at capacity, simply increase the timeout of the current subnet
|
||||
self.random_subnets.insert(subnet_id);
|
||||
return;
|
||||
}
|
||||
|
||||
// we are not at capacity, unsubscribe from the current subnet, remove the ENR bitfield bit and choose a new random one
|
||||
// from the available subnets
|
||||
// Note: This should not occur during a required subnet as subscriptions update the timeout
|
||||
// to last as long as they are needed.
|
||||
|
||||
debug!(self.log, "Unsubscribing from random subnet"; "subnet_id" => *subnet_id);
|
||||
self.events
|
||||
.push_back(AttServiceMessage::Unsubscribe(subnet_id));
|
||||
self.events
|
||||
.push_back(AttServiceMessage::EnrRemove(subnet_id));
|
||||
self.subscribe_to_random_subnets(1);
|
||||
}
|
||||
|
||||
/// A known validator has not sent a subscription in a while. They are considered offline and the
|
||||
/// beacon node no longer needs to be subscribed to the allocated random subnets.
|
||||
///
|
||||
/// We don't keep track of a specific validator to random subnet, rather the ratio of active
|
||||
/// validators to random subnets. So when a validator goes offline, we can simply remove the
|
||||
/// allocated amount of random subnets.
|
||||
fn handle_known_validator_expiry(&mut self) -> Result<(), ()> {
|
||||
let spec = &self.beacon_chain.spec;
|
||||
let subnet_count = spec.attestation_subnet_count;
|
||||
let random_subnets_per_validator = spec.random_subnets_per_validator;
|
||||
if self.known_validators.len() as u64 * random_subnets_per_validator >= subnet_count {
|
||||
// have too many validators, ignore
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let subscribed_subnets = self.random_subnets.keys_vec();
|
||||
let to_remove_subnets = subscribed_subnets.choose_multiple(
|
||||
&mut rand::thread_rng(),
|
||||
random_subnets_per_validator as usize,
|
||||
);
|
||||
let current_slot = self.beacon_chain.slot_clock.now().ok_or_else(|| {
|
||||
warn!(self.log, "Could not get the current slot");
|
||||
})?;
|
||||
|
||||
for subnet_id in to_remove_subnets {
|
||||
// If a subscription is queued for two slots in the future, it's associated unsubscription
|
||||
// will unsubscribe from the expired subnet.
|
||||
// If there is no subscription for this subnet,slot it is safe to add one, without
|
||||
// unsubscribing early from a required subnet
|
||||
if self
|
||||
.subscriptions
|
||||
.get(&(**subnet_id, current_slot + 2))
|
||||
.is_none()
|
||||
{
|
||||
// set an unsubscribe event
|
||||
let duration_to_next_slot = self
|
||||
.beacon_chain
|
||||
.slot_clock
|
||||
.duration_to_next_slot()
|
||||
.ok_or_else(|| {
|
||||
warn!(self.log, "Unable to determine duration to next slot");
|
||||
})?;
|
||||
let slot_duration =
|
||||
Duration::from_millis(self.beacon_chain.spec.milliseconds_per_slot);
|
||||
// Set the unsubscription timeout
|
||||
let unsubscription_duration = duration_to_next_slot + slot_duration * 2;
|
||||
self.unsubscriptions
|
||||
.insert_at((**subnet_id, current_slot + 2), unsubscription_duration);
|
||||
}
|
||||
|
||||
// as the long lasting subnet subscription is being removed, remove the subnet_id from
|
||||
// the ENR bitfield
|
||||
self.events
|
||||
.push_back(AttServiceMessage::EnrRemove(**subnet_id));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> Stream for AttestationService<T> {
|
||||
type Item = AttServiceMessage;
|
||||
type Error = ();
|
||||
|
||||
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
|
||||
// process any peer discovery events
|
||||
while let Async::Ready(Some((subnet_id, target_slot))) =
|
||||
self.discover_peers.poll().map_err(|e| {
|
||||
error!(self.log, "Failed to check for peer discovery requests"; "error"=> format!("{}", e));
|
||||
})?
|
||||
{
|
||||
self.handle_discover_peers(subnet_id, target_slot);
|
||||
}
|
||||
|
||||
// process any subscription events
|
||||
while let Async::Ready(Some((subnet_id, target_slot))) = self.subscriptions.poll().map_err(|e| {
|
||||
error!(self.log, "Failed to check for subnet subscription times"; "error"=> format!("{}", e));
|
||||
})?
|
||||
{
|
||||
self.handle_subscriptions(subnet_id, target_slot);
|
||||
}
|
||||
|
||||
// process any un-subscription events
|
||||
while let Async::Ready(Some((subnet_id, target_slot))) = self.unsubscriptions.poll().map_err(|e| {
|
||||
error!(self.log, "Failed to check for subnet unsubscription times"; "error"=> format!("{}", e));
|
||||
})?
|
||||
{
|
||||
self.handle_unsubscriptions(subnet_id, target_slot);
|
||||
}
|
||||
|
||||
// process any random subnet expiries
|
||||
while let Async::Ready(Some(subnet)) = self.random_subnets.poll().map_err(|e| {
|
||||
error!(self.log, "Failed to check for random subnet cycles"; "error"=> format!("{}", e));
|
||||
})?
|
||||
{
|
||||
self.handle_random_subnet_expiry(subnet);
|
||||
}
|
||||
|
||||
// process any known validator expiries
|
||||
while let Async::Ready(Some(_validator_index)) = self.known_validators.poll().map_err(|e| {
|
||||
error!(self.log, "Failed to check for random subnet cycles"; "error"=> format!("{}", e));
|
||||
})?
|
||||
{
|
||||
let _ = self.handle_known_validator_expiry();
|
||||
}
|
||||
|
||||
// process any generated events
|
||||
if let Some(event) = self.events.pop_front() {
|
||||
return Ok(Async::Ready(Some(event)));
|
||||
}
|
||||
|
||||
Ok(Async::NotReady)
|
||||
}
|
||||
}
|
||||
@ -1,6 +1,4 @@
|
||||
// generates error types
|
||||
use eth2_libp2p;
|
||||
|
||||
use error_chain::error_chain;
|
||||
|
||||
error_chain! {
|
||||
|
||||
@ -1,12 +1,11 @@
|
||||
/// This crate provides the network server for Lighthouse.
|
||||
pub mod error;
|
||||
pub mod message_handler;
|
||||
pub mod message_processor;
|
||||
pub mod persisted_dht;
|
||||
pub mod service;
|
||||
pub mod sync;
|
||||
|
||||
mod attestation_service;
|
||||
mod persisted_dht;
|
||||
mod router;
|
||||
mod sync;
|
||||
|
||||
pub use eth2_libp2p::NetworkConfig;
|
||||
pub use message_processor::MessageProcessor;
|
||||
pub use service::NetworkMessage;
|
||||
pub use service::Service;
|
||||
pub use service::{NetworkMessage, NetworkService};
|
||||
|
||||
@ -1,367 +0,0 @@
|
||||
#![allow(clippy::unit_arg)]
|
||||
use crate::error;
|
||||
use crate::service::NetworkMessage;
|
||||
use crate::MessageProcessor;
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use eth2_libp2p::{
|
||||
behaviour::PubsubMessage,
|
||||
rpc::{RPCError, RPCErrorResponse, RPCRequest, RPCResponse, RequestId, ResponseTermination},
|
||||
MessageId, PeerId, RPCEvent,
|
||||
};
|
||||
use futures::future::Future;
|
||||
use futures::stream::Stream;
|
||||
use slog::{debug, o, trace, warn};
|
||||
use ssz::{Decode, DecodeError};
|
||||
use std::sync::Arc;
|
||||
use tokio::sync::mpsc;
|
||||
use types::{Attestation, AttesterSlashing, ProposerSlashing, SignedBeaconBlock, VoluntaryExit};
|
||||
|
||||
/// Handles messages received from the network and client and organises syncing. This
|
||||
/// functionality of this struct is to validate an decode messages from the network before
|
||||
/// passing them to the internal message processor. The message processor spawns a syncing thread
|
||||
/// which manages which blocks need to be requested and processed.
|
||||
pub struct MessageHandler<T: BeaconChainTypes> {
|
||||
/// A channel to the network service to allow for gossip propagation.
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage>,
|
||||
/// Processes validated and decoded messages from the network. Has direct access to the
|
||||
/// sync manager.
|
||||
message_processor: MessageProcessor<T>,
|
||||
/// The `MessageHandler` logger.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
/// Types of messages the handler can receive.
|
||||
#[derive(Debug)]
|
||||
pub enum HandlerMessage {
|
||||
/// We have initiated a connection to a new peer.
|
||||
PeerDialed(PeerId),
|
||||
/// Peer has disconnected,
|
||||
PeerDisconnected(PeerId),
|
||||
/// An RPC response/request has been received.
|
||||
RPC(PeerId, RPCEvent),
|
||||
/// A gossip message has been received. The fields are: message id, the peer that sent us this
|
||||
/// message and the message itself.
|
||||
PubsubMessage(MessageId, PeerId, PubsubMessage),
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> MessageHandler<T> {
|
||||
/// Initializes and runs the MessageHandler.
|
||||
pub fn spawn(
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage>,
|
||||
executor: &tokio::runtime::TaskExecutor,
|
||||
log: slog::Logger,
|
||||
) -> error::Result<mpsc::UnboundedSender<HandlerMessage>> {
|
||||
let message_handler_log = log.new(o!("service"=> "msg_handler"));
|
||||
trace!(message_handler_log, "Service starting");
|
||||
|
||||
let (handler_send, handler_recv) = mpsc::unbounded_channel();
|
||||
|
||||
// Initialise a message instance, which itself spawns the syncing thread.
|
||||
let message_processor =
|
||||
MessageProcessor::new(executor, beacon_chain, network_send.clone(), &log);
|
||||
|
||||
// generate the Message handler
|
||||
let mut handler = MessageHandler {
|
||||
network_send,
|
||||
message_processor,
|
||||
log: message_handler_log,
|
||||
};
|
||||
|
||||
// spawn handler task and move the message handler instance into the spawned thread
|
||||
executor.spawn(
|
||||
handler_recv
|
||||
.for_each(move |msg| Ok(handler.handle_message(msg)))
|
||||
.map_err(move |_| {
|
||||
debug!(log, "Network message handler terminated.");
|
||||
}),
|
||||
);
|
||||
|
||||
Ok(handler_send)
|
||||
}
|
||||
|
||||
/// Handle all messages incoming from the network service.
|
||||
fn handle_message(&mut self, message: HandlerMessage) {
|
||||
match message {
|
||||
// we have initiated a connection to a peer
|
||||
HandlerMessage::PeerDialed(peer_id) => {
|
||||
self.message_processor.on_connect(peer_id);
|
||||
}
|
||||
// A peer has disconnected
|
||||
HandlerMessage::PeerDisconnected(peer_id) => {
|
||||
self.message_processor.on_disconnect(peer_id);
|
||||
}
|
||||
// An RPC message request/response has been received
|
||||
HandlerMessage::RPC(peer_id, rpc_event) => {
|
||||
self.handle_rpc_message(peer_id, rpc_event);
|
||||
}
|
||||
// An RPC message request/response has been received
|
||||
HandlerMessage::PubsubMessage(id, peer_id, gossip) => {
|
||||
self.handle_gossip(id, peer_id, gossip);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* RPC - Related functionality */
|
||||
|
||||
/// Handle RPC messages
|
||||
fn handle_rpc_message(&mut self, peer_id: PeerId, rpc_message: RPCEvent) {
|
||||
match rpc_message {
|
||||
RPCEvent::Request(id, req) => self.handle_rpc_request(peer_id, id, req),
|
||||
RPCEvent::Response(id, resp) => self.handle_rpc_response(peer_id, id, resp),
|
||||
RPCEvent::Error(id, error) => self.handle_rpc_error(peer_id, id, error),
|
||||
}
|
||||
}
|
||||
|
||||
/// A new RPC request has been received from the network.
|
||||
fn handle_rpc_request(&mut self, peer_id: PeerId, request_id: RequestId, request: RPCRequest) {
|
||||
match request {
|
||||
RPCRequest::Status(status_message) => {
|
||||
self.message_processor
|
||||
.on_status_request(peer_id, request_id, status_message)
|
||||
}
|
||||
RPCRequest::Goodbye(goodbye_reason) => {
|
||||
debug!(
|
||||
self.log, "PeerGoodbye";
|
||||
"peer" => format!("{:?}", peer_id),
|
||||
"reason" => format!("{:?}", goodbye_reason),
|
||||
);
|
||||
self.message_processor.on_disconnect(peer_id);
|
||||
}
|
||||
RPCRequest::BlocksByRange(request) => self
|
||||
.message_processor
|
||||
.on_blocks_by_range_request(peer_id, request_id, request),
|
||||
RPCRequest::BlocksByRoot(request) => self
|
||||
.message_processor
|
||||
.on_blocks_by_root_request(peer_id, request_id, request),
|
||||
}
|
||||
}
|
||||
|
||||
/// An RPC response has been received from the network.
|
||||
// we match on id and ignore responses past the timeout.
|
||||
fn handle_rpc_response(
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
request_id: RequestId,
|
||||
error_response: RPCErrorResponse,
|
||||
) {
|
||||
// an error could have occurred.
|
||||
match error_response {
|
||||
RPCErrorResponse::InvalidRequest(error) => {
|
||||
warn!(self.log, "Peer indicated invalid request";"peer_id" => format!("{:?}", peer_id), "error" => error.as_string());
|
||||
self.handle_rpc_error(peer_id, request_id, RPCError::RPCErrorResponse);
|
||||
}
|
||||
RPCErrorResponse::ServerError(error) => {
|
||||
warn!(self.log, "Peer internal server error";"peer_id" => format!("{:?}", peer_id), "error" => error.as_string());
|
||||
self.handle_rpc_error(peer_id, request_id, RPCError::RPCErrorResponse);
|
||||
}
|
||||
RPCErrorResponse::Unknown(error) => {
|
||||
warn!(self.log, "Unknown peer error";"peer" => format!("{:?}", peer_id), "error" => error.as_string());
|
||||
self.handle_rpc_error(peer_id, request_id, RPCError::RPCErrorResponse);
|
||||
}
|
||||
RPCErrorResponse::Success(response) => {
|
||||
match response {
|
||||
RPCResponse::Status(status_message) => {
|
||||
self.message_processor
|
||||
.on_status_response(peer_id, status_message);
|
||||
}
|
||||
RPCResponse::BlocksByRange(response) => {
|
||||
match self.decode_beacon_block(response) {
|
||||
Ok(beacon_block) => {
|
||||
self.message_processor.on_blocks_by_range_response(
|
||||
peer_id,
|
||||
request_id,
|
||||
Some(beacon_block),
|
||||
);
|
||||
}
|
||||
Err(e) => {
|
||||
// TODO: Down-vote Peer
|
||||
warn!(self.log, "Peer sent invalid BEACON_BLOCKS response";"peer" => format!("{:?}", peer_id), "error" => format!("{:?}", e));
|
||||
}
|
||||
}
|
||||
}
|
||||
RPCResponse::BlocksByRoot(response) => {
|
||||
match self.decode_beacon_block(response) {
|
||||
Ok(beacon_block) => {
|
||||
self.message_processor.on_blocks_by_root_response(
|
||||
peer_id,
|
||||
request_id,
|
||||
Some(beacon_block),
|
||||
);
|
||||
}
|
||||
Err(e) => {
|
||||
// TODO: Down-vote Peer
|
||||
warn!(self.log, "Peer sent invalid BEACON_BLOCKS response";"peer" => format!("{:?}", peer_id), "error" => format!("{:?}", e));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
RPCErrorResponse::StreamTermination(response_type) => {
|
||||
// have received a stream termination, notify the processing functions
|
||||
match response_type {
|
||||
ResponseTermination::BlocksByRange => {
|
||||
self.message_processor
|
||||
.on_blocks_by_range_response(peer_id, request_id, None);
|
||||
}
|
||||
ResponseTermination::BlocksByRoot => {
|
||||
self.message_processor
|
||||
.on_blocks_by_root_response(peer_id, request_id, None);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Handle various RPC errors
|
||||
fn handle_rpc_error(&mut self, peer_id: PeerId, request_id: RequestId, error: RPCError) {
|
||||
warn!(self.log, "RPC Error"; "Peer" => format!("{:?}", peer_id), "request_id" => format!("{}", request_id), "Error" => format!("{:?}", error));
|
||||
self.message_processor.on_rpc_error(peer_id, request_id);
|
||||
}
|
||||
|
||||
/// Handle RPC messages
|
||||
fn handle_gossip(&mut self, id: MessageId, peer_id: PeerId, gossip_message: PubsubMessage) {
|
||||
match gossip_message {
|
||||
PubsubMessage::Block(message) => match self.decode_gossip_block(message) {
|
||||
Ok(block) => {
|
||||
let should_forward_on = self
|
||||
.message_processor
|
||||
.on_block_gossip(peer_id.clone(), block);
|
||||
// TODO: Apply more sophisticated validation and decoding logic
|
||||
if should_forward_on {
|
||||
self.propagate_message(id, peer_id);
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
debug!(self.log, "Invalid gossiped beacon block"; "peer_id" => format!("{}", peer_id), "Error" => format!("{:?}", e));
|
||||
}
|
||||
},
|
||||
PubsubMessage::Attestation(message) => match self.decode_gossip_attestation(message) {
|
||||
Ok(attestation) => {
|
||||
// TODO: Apply more sophisticated validation and decoding logic
|
||||
self.propagate_message(id, peer_id.clone());
|
||||
self.message_processor
|
||||
.on_attestation_gossip(peer_id, attestation);
|
||||
}
|
||||
Err(e) => {
|
||||
debug!(self.log, "Invalid gossiped attestation"; "peer_id" => format!("{}", peer_id), "Error" => format!("{:?}", e));
|
||||
}
|
||||
},
|
||||
PubsubMessage::VoluntaryExit(message) => match self.decode_gossip_exit(message) {
|
||||
Ok(_exit) => {
|
||||
// TODO: Apply more sophisticated validation and decoding logic
|
||||
self.propagate_message(id, peer_id.clone());
|
||||
// TODO: Handle exits
|
||||
debug!(self.log, "Received a voluntary exit"; "peer_id" => format!("{}", peer_id) );
|
||||
}
|
||||
Err(e) => {
|
||||
debug!(self.log, "Invalid gossiped exit"; "peer_id" => format!("{}", peer_id), "Error" => format!("{:?}", e));
|
||||
}
|
||||
},
|
||||
PubsubMessage::ProposerSlashing(message) => {
|
||||
match self.decode_gossip_proposer_slashing(message) {
|
||||
Ok(_slashing) => {
|
||||
// TODO: Apply more sophisticated validation and decoding logic
|
||||
self.propagate_message(id, peer_id.clone());
|
||||
// TODO: Handle proposer slashings
|
||||
debug!(self.log, "Received a proposer slashing"; "peer_id" => format!("{}", peer_id) );
|
||||
}
|
||||
Err(e) => {
|
||||
debug!(self.log, "Invalid gossiped proposer slashing"; "peer_id" => format!("{}", peer_id), "Error" => format!("{:?}", e));
|
||||
}
|
||||
}
|
||||
}
|
||||
PubsubMessage::AttesterSlashing(message) => {
|
||||
match self.decode_gossip_attestation_slashing(message) {
|
||||
Ok(_slashing) => {
|
||||
// TODO: Apply more sophisticated validation and decoding logic
|
||||
self.propagate_message(id, peer_id.clone());
|
||||
// TODO: Handle attester slashings
|
||||
debug!(self.log, "Received an attester slashing"; "peer_id" => format!("{}", peer_id) );
|
||||
}
|
||||
Err(e) => {
|
||||
debug!(self.log, "Invalid gossiped attester slashing"; "peer_id" => format!("{}", peer_id), "Error" => format!("{:?}", e));
|
||||
}
|
||||
}
|
||||
}
|
||||
PubsubMessage::Unknown(message) => {
|
||||
// Received a message from an unknown topic. Ignore for now
|
||||
debug!(self.log, "Unknown Gossip Message"; "peer_id" => format!("{}", peer_id), "Message" => format!("{:?}", message));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Informs the network service that the message should be forwarded to other peers.
|
||||
fn propagate_message(&mut self, message_id: MessageId, propagation_source: PeerId) {
|
||||
self.network_send
|
||||
.try_send(NetworkMessage::Propagate {
|
||||
propagation_source,
|
||||
message_id,
|
||||
})
|
||||
.unwrap_or_else(|_| {
|
||||
warn!(
|
||||
self.log,
|
||||
"Could not send propagation request to the network service"
|
||||
)
|
||||
});
|
||||
}
|
||||
|
||||
/* Decoding of gossipsub objects from the network.
|
||||
*
|
||||
* The decoding is done in the message handler as it has access to to a `BeaconChain` and can
|
||||
* therefore apply more efficient logic in decoding and verification.
|
||||
*
|
||||
* TODO: Apply efficient decoding/verification of these objects
|
||||
*/
|
||||
|
||||
/* Gossipsub Domain Decoding */
|
||||
// Note: These are not generics as type-specific verification will need to be applied.
|
||||
fn decode_gossip_block(
|
||||
&self,
|
||||
beacon_block: Vec<u8>,
|
||||
) -> Result<SignedBeaconBlock<T::EthSpec>, DecodeError> {
|
||||
//TODO: Apply verification before decoding.
|
||||
SignedBeaconBlock::from_ssz_bytes(&beacon_block)
|
||||
}
|
||||
|
||||
fn decode_gossip_attestation(
|
||||
&self,
|
||||
beacon_block: Vec<u8>,
|
||||
) -> Result<Attestation<T::EthSpec>, DecodeError> {
|
||||
//TODO: Apply verification before decoding.
|
||||
Attestation::from_ssz_bytes(&beacon_block)
|
||||
}
|
||||
|
||||
fn decode_gossip_exit(&self, voluntary_exit: Vec<u8>) -> Result<VoluntaryExit, DecodeError> {
|
||||
//TODO: Apply verification before decoding.
|
||||
VoluntaryExit::from_ssz_bytes(&voluntary_exit)
|
||||
}
|
||||
|
||||
fn decode_gossip_proposer_slashing(
|
||||
&self,
|
||||
proposer_slashing: Vec<u8>,
|
||||
) -> Result<ProposerSlashing, DecodeError> {
|
||||
//TODO: Apply verification before decoding.
|
||||
ProposerSlashing::from_ssz_bytes(&proposer_slashing)
|
||||
}
|
||||
|
||||
fn decode_gossip_attestation_slashing(
|
||||
&self,
|
||||
attester_slashing: Vec<u8>,
|
||||
) -> Result<AttesterSlashing<T::EthSpec>, DecodeError> {
|
||||
//TODO: Apply verification before decoding.
|
||||
AttesterSlashing::from_ssz_bytes(&attester_slashing)
|
||||
}
|
||||
|
||||
/* Req/Resp Domain Decoding */
|
||||
|
||||
/// Verifies and decodes an ssz-encoded `SignedBeaconBlock`. If `None` is passed, this represents a
|
||||
/// stream termination.
|
||||
fn decode_beacon_block(
|
||||
&self,
|
||||
beacon_block: Vec<u8>,
|
||||
) -> Result<SignedBeaconBlock<T::EthSpec>, DecodeError> {
|
||||
//TODO: Implement faster block verification before decoding entirely
|
||||
SignedBeaconBlock::from_ssz_bytes(&beacon_block)
|
||||
}
|
||||
}
|
||||
@ -1,13 +1,15 @@
|
||||
use beacon_chain::BeaconChainTypes;
|
||||
use eth2_libp2p::Enr;
|
||||
use rlp;
|
||||
use std::sync::Arc;
|
||||
use store::{DBColumn, Error as StoreError, SimpleStoreItem, Store};
|
||||
use types::{EthSpec, Hash256};
|
||||
use store::Store;
|
||||
use store::{DBColumn, Error as StoreError, SimpleStoreItem};
|
||||
use types::Hash256;
|
||||
|
||||
/// 32-byte key for accessing the `DhtEnrs`.
|
||||
pub const DHT_DB_KEY: &str = "PERSISTEDDHTPERSISTEDDHTPERSISTE";
|
||||
|
||||
pub fn load_dht<T: Store<E>, E: EthSpec>(store: Arc<T>) -> Vec<Enr> {
|
||||
pub fn load_dht<T: BeaconChainTypes>(store: Arc<T::Store>) -> Vec<Enr> {
|
||||
// Load DHT from store
|
||||
let key = Hash256::from_slice(&DHT_DB_KEY.as_bytes());
|
||||
match store.get(&key) {
|
||||
@ -20,8 +22,8 @@ pub fn load_dht<T: Store<E>, E: EthSpec>(store: Arc<T>) -> Vec<Enr> {
|
||||
}
|
||||
|
||||
/// Attempt to persist the ENR's in the DHT to `self.store`.
|
||||
pub fn persist_dht<T: Store<E>, E: EthSpec>(
|
||||
store: Arc<T>,
|
||||
pub fn persist_dht<T: BeaconChainTypes>(
|
||||
store: Arc<T::Store>,
|
||||
enrs: Vec<Enr>,
|
||||
) -> Result<(), store::Error> {
|
||||
let key = Hash256::from_slice(&DHT_DB_KEY.as_bytes());
|
||||
|
||||
275
beacon_node/network/src/router/mod.rs
Normal file
275
beacon_node/network/src/router/mod.rs
Normal file
@ -0,0 +1,275 @@
|
||||
//! This module handles incoming network messages.
|
||||
//!
|
||||
//! It routes the messages to appropriate services, such as the Sync
|
||||
//! and processes those that are
|
||||
#![allow(clippy::unit_arg)]
|
||||
|
||||
pub mod processor;
|
||||
|
||||
use crate::error;
|
||||
use crate::service::NetworkMessage;
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use eth2_libp2p::{
|
||||
rpc::{RPCError, RPCErrorResponse, RPCRequest, RPCResponse, RequestId, ResponseTermination},
|
||||
MessageId, PeerId, PubsubData, PubsubMessage, RPCEvent,
|
||||
};
|
||||
use futures::future::Future;
|
||||
use futures::stream::Stream;
|
||||
use processor::Processor;
|
||||
use slog::{debug, o, trace, warn};
|
||||
use std::sync::Arc;
|
||||
use tokio::sync::mpsc;
|
||||
use types::EthSpec;
|
||||
|
||||
/// Handles messages received from the network and client and organises syncing. This
|
||||
/// functionality of this struct is to validate an decode messages from the network before
|
||||
/// passing them to the internal message processor. The message processor spawns a syncing thread
|
||||
/// which manages which blocks need to be requested and processed.
|
||||
pub struct Router<T: BeaconChainTypes> {
|
||||
/// A channel to the network service to allow for gossip propagation.
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage<T::EthSpec>>,
|
||||
/// Processes validated and decoded messages from the network. Has direct access to the
|
||||
/// sync manager.
|
||||
processor: Processor<T>,
|
||||
/// The `Router` logger.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
/// Types of messages the handler can receive.
|
||||
#[derive(Debug)]
|
||||
pub enum RouterMessage<T: EthSpec> {
|
||||
/// We have initiated a connection to a new peer.
|
||||
PeerDialed(PeerId),
|
||||
/// Peer has disconnected,
|
||||
PeerDisconnected(PeerId),
|
||||
/// An RPC response/request has been received.
|
||||
RPC(PeerId, RPCEvent<T>),
|
||||
/// A gossip message has been received. The fields are: message id, the peer that sent us this
|
||||
/// message and the message itself.
|
||||
PubsubMessage(MessageId, PeerId, PubsubMessage<T>),
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> Router<T> {
|
||||
/// Initializes and runs the Router.
|
||||
pub fn spawn(
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage<T::EthSpec>>,
|
||||
executor: &tokio::runtime::TaskExecutor,
|
||||
log: slog::Logger,
|
||||
) -> error::Result<mpsc::UnboundedSender<RouterMessage<T::EthSpec>>> {
|
||||
let message_handler_log = log.new(o!("service"=> "msg_handler"));
|
||||
trace!(message_handler_log, "Service starting");
|
||||
|
||||
let (handler_send, handler_recv) = mpsc::unbounded_channel();
|
||||
|
||||
// Initialise a message instance, which itself spawns the syncing thread.
|
||||
let processor = Processor::new(executor, beacon_chain, network_send.clone(), &log);
|
||||
|
||||
// generate the Message handler
|
||||
let mut handler = Router {
|
||||
network_send,
|
||||
processor,
|
||||
log: message_handler_log,
|
||||
};
|
||||
|
||||
// spawn handler task and move the message handler instance into the spawned thread
|
||||
executor.spawn(
|
||||
handler_recv
|
||||
.for_each(move |msg| Ok(handler.handle_message(msg)))
|
||||
.map_err(move |_| {
|
||||
debug!(log, "Network message handler terminated.");
|
||||
}),
|
||||
);
|
||||
|
||||
Ok(handler_send)
|
||||
}
|
||||
|
||||
/// Handle all messages incoming from the network service.
|
||||
fn handle_message(&mut self, message: RouterMessage<T::EthSpec>) {
|
||||
match message {
|
||||
// we have initiated a connection to a peer
|
||||
RouterMessage::PeerDialed(peer_id) => {
|
||||
self.processor.on_connect(peer_id);
|
||||
}
|
||||
// A peer has disconnected
|
||||
RouterMessage::PeerDisconnected(peer_id) => {
|
||||
self.processor.on_disconnect(peer_id);
|
||||
}
|
||||
// An RPC message request/response has been received
|
||||
RouterMessage::RPC(peer_id, rpc_event) => {
|
||||
self.handle_rpc_message(peer_id, rpc_event);
|
||||
}
|
||||
// An RPC message request/response has been received
|
||||
RouterMessage::PubsubMessage(id, peer_id, gossip) => {
|
||||
self.handle_gossip(id, peer_id, gossip);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* RPC - Related functionality */
|
||||
|
||||
/// Handle RPC messages
|
||||
fn handle_rpc_message(&mut self, peer_id: PeerId, rpc_message: RPCEvent<T::EthSpec>) {
|
||||
match rpc_message {
|
||||
RPCEvent::Request(id, req) => self.handle_rpc_request(peer_id, id, req),
|
||||
RPCEvent::Response(id, resp) => self.handle_rpc_response(peer_id, id, resp),
|
||||
RPCEvent::Error(id, error) => self.handle_rpc_error(peer_id, id, error),
|
||||
}
|
||||
}
|
||||
|
||||
/// A new RPC request has been received from the network.
|
||||
fn handle_rpc_request(
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
request_id: RequestId,
|
||||
request: RPCRequest<T::EthSpec>,
|
||||
) {
|
||||
match request {
|
||||
RPCRequest::Status(status_message) => {
|
||||
self.processor
|
||||
.on_status_request(peer_id, request_id, status_message)
|
||||
}
|
||||
RPCRequest::Goodbye(goodbye_reason) => {
|
||||
debug!(
|
||||
self.log, "PeerGoodbye";
|
||||
"peer" => format!("{:?}", peer_id),
|
||||
"reason" => format!("{:?}", goodbye_reason),
|
||||
);
|
||||
self.processor.on_disconnect(peer_id);
|
||||
}
|
||||
RPCRequest::BlocksByRange(request) => self
|
||||
.processor
|
||||
.on_blocks_by_range_request(peer_id, request_id, request),
|
||||
RPCRequest::BlocksByRoot(request) => self
|
||||
.processor
|
||||
.on_blocks_by_root_request(peer_id, request_id, request),
|
||||
RPCRequest::Phantom(_) => unreachable!("Phantom never initialised"),
|
||||
}
|
||||
}
|
||||
|
||||
/// An RPC response has been received from the network.
|
||||
// we match on id and ignore responses past the timeout.
|
||||
fn handle_rpc_response(
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
request_id: RequestId,
|
||||
error_response: RPCErrorResponse<T::EthSpec>,
|
||||
) {
|
||||
// an error could have occurred.
|
||||
match error_response {
|
||||
RPCErrorResponse::InvalidRequest(error) => {
|
||||
warn!(self.log, "Peer indicated invalid request";"peer_id" => format!("{:?}", peer_id), "error" => error.as_string());
|
||||
self.handle_rpc_error(peer_id, request_id, RPCError::RPCErrorResponse);
|
||||
}
|
||||
RPCErrorResponse::ServerError(error) => {
|
||||
warn!(self.log, "Peer internal server error";"peer_id" => format!("{:?}", peer_id), "error" => error.as_string());
|
||||
self.handle_rpc_error(peer_id, request_id, RPCError::RPCErrorResponse);
|
||||
}
|
||||
RPCErrorResponse::Unknown(error) => {
|
||||
warn!(self.log, "Unknown peer error";"peer" => format!("{:?}", peer_id), "error" => error.as_string());
|
||||
self.handle_rpc_error(peer_id, request_id, RPCError::RPCErrorResponse);
|
||||
}
|
||||
RPCErrorResponse::Success(response) => match response {
|
||||
RPCResponse::Status(status_message) => {
|
||||
self.processor.on_status_response(peer_id, status_message);
|
||||
}
|
||||
RPCResponse::BlocksByRange(beacon_block) => {
|
||||
self.processor.on_blocks_by_range_response(
|
||||
peer_id,
|
||||
request_id,
|
||||
Some(beacon_block),
|
||||
);
|
||||
}
|
||||
RPCResponse::BlocksByRoot(beacon_block) => {
|
||||
self.processor.on_blocks_by_root_response(
|
||||
peer_id,
|
||||
request_id,
|
||||
Some(beacon_block),
|
||||
);
|
||||
}
|
||||
},
|
||||
RPCErrorResponse::StreamTermination(response_type) => {
|
||||
// have received a stream termination, notify the processing functions
|
||||
match response_type {
|
||||
ResponseTermination::BlocksByRange => {
|
||||
self.processor
|
||||
.on_blocks_by_range_response(peer_id, request_id, None);
|
||||
}
|
||||
ResponseTermination::BlocksByRoot => {
|
||||
self.processor
|
||||
.on_blocks_by_root_response(peer_id, request_id, None);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Handle various RPC errors
|
||||
fn handle_rpc_error(&mut self, peer_id: PeerId, request_id: RequestId, error: RPCError) {
|
||||
warn!(self.log, "RPC Error"; "Peer" => format!("{:?}", peer_id), "request_id" => format!("{}", request_id), "Error" => format!("{:?}", error));
|
||||
self.processor.on_rpc_error(peer_id, request_id);
|
||||
}
|
||||
|
||||
/// Handle RPC messages
|
||||
fn handle_gossip(
|
||||
&mut self,
|
||||
id: MessageId,
|
||||
peer_id: PeerId,
|
||||
gossip_message: PubsubMessage<T::EthSpec>,
|
||||
) {
|
||||
match gossip_message.data {
|
||||
PubsubData::BeaconBlock(block) => {
|
||||
if self.processor.should_forward_block(&block) {
|
||||
self.propagate_message(id, peer_id.clone());
|
||||
}
|
||||
self.processor.on_block_gossip(peer_id, block);
|
||||
}
|
||||
PubsubData::AggregateAndProofAttestation(_agg_attestation) => {
|
||||
// TODO: Handle propagation conditions
|
||||
self.propagate_message(id, peer_id);
|
||||
// TODO Handle aggregate attestion
|
||||
// self.processor
|
||||
// .on_attestation_gossip(peer_id.clone(), &agg_attestation);
|
||||
}
|
||||
PubsubData::Attestation(boxed_shard_attestation) => {
|
||||
// TODO: Handle propagation conditions
|
||||
self.propagate_message(id, peer_id.clone());
|
||||
self.processor
|
||||
.on_attestation_gossip(peer_id, boxed_shard_attestation.1);
|
||||
}
|
||||
PubsubData::VoluntaryExit(_exit) => {
|
||||
// TODO: Apply more sophisticated validation
|
||||
self.propagate_message(id, peer_id.clone());
|
||||
// TODO: Handle exits
|
||||
debug!(self.log, "Received a voluntary exit"; "peer_id" => format!("{}", peer_id) );
|
||||
}
|
||||
PubsubData::ProposerSlashing(_proposer_slashing) => {
|
||||
// TODO: Apply more sophisticated validation
|
||||
self.propagate_message(id, peer_id.clone());
|
||||
// TODO: Handle proposer slashings
|
||||
debug!(self.log, "Received a proposer slashing"; "peer_id" => format!("{}", peer_id) );
|
||||
}
|
||||
PubsubData::AttesterSlashing(_attester_slashing) => {
|
||||
// TODO: Apply more sophisticated validation
|
||||
self.propagate_message(id, peer_id.clone());
|
||||
// TODO: Handle attester slashings
|
||||
debug!(self.log, "Received an attester slashing"; "peer_id" => format!("{}", peer_id) );
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Informs the network service that the message should be forwarded to other peers.
|
||||
fn propagate_message(&mut self, message_id: MessageId, propagation_source: PeerId) {
|
||||
self.network_send
|
||||
.try_send(NetworkMessage::Propagate {
|
||||
propagation_source,
|
||||
message_id,
|
||||
})
|
||||
.unwrap_or_else(|_| {
|
||||
warn!(
|
||||
self.log,
|
||||
"Could not send propagation request to the network service"
|
||||
)
|
||||
});
|
||||
}
|
||||
}
|
||||
@ -19,9 +19,6 @@ use types::{Attestation, Epoch, EthSpec, Hash256, SignedBeaconBlock, Slot};
|
||||
/// Otherwise we queue it.
|
||||
pub(crate) const FUTURE_SLOT_TOLERANCE: u64 = 1;
|
||||
|
||||
const SHOULD_FORWARD_GOSSIP_BLOCK: bool = true;
|
||||
const SHOULD_NOT_FORWARD_GOSSIP_BLOCK: bool = false;
|
||||
|
||||
/// Keeps track of syncing information for known connected peers.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct PeerSyncInfo {
|
||||
@ -52,7 +49,7 @@ impl PeerSyncInfo {
|
||||
|
||||
/// Processes validated messages from the network. It relays necessary data to the syncing thread
|
||||
/// and processes blocks from the pubsub network.
|
||||
pub struct MessageProcessor<T: BeaconChainTypes> {
|
||||
pub struct Processor<T: BeaconChainTypes> {
|
||||
/// A reference to the underlying beacon chain.
|
||||
chain: Arc<BeaconChain<T>>,
|
||||
/// A channel to the syncing thread.
|
||||
@ -60,17 +57,17 @@ pub struct MessageProcessor<T: BeaconChainTypes> {
|
||||
/// A oneshot channel for destroying the sync thread.
|
||||
_sync_exit: oneshot::Sender<()>,
|
||||
/// A network context to return and handle RPC requests.
|
||||
network: HandlerNetworkContext,
|
||||
network: HandlerNetworkContext<T::EthSpec>,
|
||||
/// The `RPCHandler` logger.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
/// Instantiate a `MessageProcessor` instance
|
||||
impl<T: BeaconChainTypes> Processor<T> {
|
||||
/// Instantiate a `Processor` instance
|
||||
pub fn new(
|
||||
executor: &tokio::runtime::TaskExecutor,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage>,
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage<T::EthSpec>>,
|
||||
log: &slog::Logger,
|
||||
) -> Self {
|
||||
let sync_logger = log.new(o!("service"=> "sync"));
|
||||
@ -83,7 +80,7 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
sync_logger,
|
||||
);
|
||||
|
||||
MessageProcessor {
|
||||
Processor {
|
||||
chain: beacon_chain,
|
||||
sync_send,
|
||||
_sync_exit,
|
||||
@ -303,7 +300,7 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
self.network.send_rpc_response(
|
||||
peer_id.clone(),
|
||||
request_id,
|
||||
RPCResponse::BlocksByRoot(block.as_ssz_bytes()),
|
||||
RPCResponse::BlocksByRoot(Box::new(block)),
|
||||
);
|
||||
send_block_count += 1;
|
||||
} else {
|
||||
@ -389,7 +386,7 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
self.network.send_rpc_response(
|
||||
peer_id.clone(),
|
||||
request_id,
|
||||
RPCResponse::BlocksByRange(block.as_ssz_bytes()),
|
||||
RPCResponse::BlocksByRange(Box::new(block)),
|
||||
);
|
||||
}
|
||||
} else {
|
||||
@ -436,9 +433,8 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
request_id: RequestId,
|
||||
beacon_block: Option<SignedBeaconBlock<T::EthSpec>>,
|
||||
beacon_block: Option<Box<SignedBeaconBlock<T::EthSpec>>>,
|
||||
) {
|
||||
let beacon_block = beacon_block.map(Box::new);
|
||||
trace!(
|
||||
self.log,
|
||||
"Received BlocksByRange Response";
|
||||
@ -457,9 +453,8 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
request_id: RequestId,
|
||||
beacon_block: Option<SignedBeaconBlock<T::EthSpec>>,
|
||||
beacon_block: Option<Box<SignedBeaconBlock<T::EthSpec>>>,
|
||||
) {
|
||||
let beacon_block = beacon_block.map(Box::new);
|
||||
trace!(
|
||||
self.log,
|
||||
"Received BlocksByRoot Response";
|
||||
@ -473,6 +468,22 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
});
|
||||
}
|
||||
|
||||
/// Template function to be called on a block to determine if the block should be propagated
|
||||
/// across the network.
|
||||
pub fn should_forward_block(&mut self, _block: &Box<SignedBeaconBlock<T::EthSpec>>) -> bool {
|
||||
// TODO: Propagate error once complete
|
||||
// self.chain.should_forward_block(block).is_ok()
|
||||
true
|
||||
}
|
||||
|
||||
/// Template function to be called on an attestation to determine if the attestation should be propagated
|
||||
/// across the network.
|
||||
pub fn _should_forward_attestation(&mut self, _attestation: &Attestation<T::EthSpec>) -> bool {
|
||||
// TODO: Propagate error once complete
|
||||
//self.chain.should_forward_attestation(attestation).is_ok()
|
||||
true
|
||||
}
|
||||
|
||||
/// Process a gossip message declaring a new block.
|
||||
///
|
||||
/// Attempts to apply to block to the beacon chain. May queue the block for later processing.
|
||||
@ -481,9 +492,9 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
pub fn on_block_gossip(
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
block: SignedBeaconBlock<T::EthSpec>,
|
||||
block: Box<SignedBeaconBlock<T::EthSpec>>,
|
||||
) -> bool {
|
||||
match self.chain.process_block(block.clone()) {
|
||||
match BlockProcessingOutcome::shim(self.chain.process_block(*block.clone())) {
|
||||
Ok(outcome) => match outcome {
|
||||
BlockProcessingOutcome::Processed { .. } => {
|
||||
trace!(self.log, "Gossipsub block processed";
|
||||
@ -508,24 +519,13 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
"location" => "block gossip"
|
||||
),
|
||||
}
|
||||
|
||||
SHOULD_FORWARD_GOSSIP_BLOCK
|
||||
}
|
||||
BlockProcessingOutcome::ParentUnknown { .. } => {
|
||||
// Inform the sync manager to find parents for this block
|
||||
trace!(self.log, "Block with unknown parent received";
|
||||
"peer_id" => format!("{:?}",peer_id));
|
||||
self.send_to_sync(SyncMessage::UnknownBlock(peer_id, Box::new(block)));
|
||||
SHOULD_FORWARD_GOSSIP_BLOCK
|
||||
self.send_to_sync(SyncMessage::UnknownBlock(peer_id, block));
|
||||
}
|
||||
BlockProcessingOutcome::FutureSlot {
|
||||
present_slot,
|
||||
block_slot,
|
||||
} if present_slot + FUTURE_SLOT_TOLERANCE >= block_slot => {
|
||||
//TODO: Decide the logic here
|
||||
SHOULD_FORWARD_GOSSIP_BLOCK
|
||||
}
|
||||
BlockProcessingOutcome::BlockIsAlreadyKnown => SHOULD_FORWARD_GOSSIP_BLOCK,
|
||||
other => {
|
||||
warn!(
|
||||
self.log,
|
||||
@ -539,7 +539,6 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
"Invalid gossip beacon block ssz";
|
||||
"ssz" => format!("0x{}", hex::encode(block.as_ssz_bytes())),
|
||||
);
|
||||
SHOULD_NOT_FORWARD_GOSSIP_BLOCK //TODO: Decide if we want to forward these
|
||||
}
|
||||
},
|
||||
Err(_) => {
|
||||
@ -549,15 +548,18 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
"Erroneous gossip beacon block ssz";
|
||||
"ssz" => format!("0x{}", hex::encode(block.as_ssz_bytes())),
|
||||
);
|
||||
SHOULD_NOT_FORWARD_GOSSIP_BLOCK
|
||||
}
|
||||
}
|
||||
// TODO: Update with correct block gossip checking
|
||||
true
|
||||
}
|
||||
|
||||
/// Process a gossip message declaring a new attestation.
|
||||
///
|
||||
/// Not currently implemented.
|
||||
pub fn on_attestation_gossip(&mut self, peer_id: PeerId, msg: Attestation<T::EthSpec>) {
|
||||
pub fn on_attestation_gossip(&mut self, _peer_id: PeerId, _msg: Attestation<T::EthSpec>) {
|
||||
// TODO: Handle subnet gossip
|
||||
/*
|
||||
match self.chain.process_attestation(msg.clone()) {
|
||||
Ok(outcome) => match outcome {
|
||||
AttestationProcessingOutcome::Processed => {
|
||||
@ -603,7 +605,8 @@ impl<T: BeaconChainTypes> MessageProcessor<T> {
|
||||
"ssz" => format!("0x{}", hex::encode(msg.as_ssz_bytes())),
|
||||
);
|
||||
}
|
||||
}
|
||||
};
|
||||
*/
|
||||
}
|
||||
}
|
||||
|
||||
@ -625,15 +628,15 @@ pub(crate) fn status_message<T: BeaconChainTypes>(
|
||||
/// Wraps a Network Channel to employ various RPC related network functionality for the message
|
||||
/// handler. The handler doesn't manage it's own request Id's and can therefore only send
|
||||
/// responses or requests with 0 request Ids.
|
||||
pub struct HandlerNetworkContext {
|
||||
pub struct HandlerNetworkContext<T: EthSpec> {
|
||||
/// The network channel to relay messages to the Network service.
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage>,
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage<T>>,
|
||||
/// Logger for the `NetworkContext`.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
impl HandlerNetworkContext {
|
||||
pub fn new(network_send: mpsc::UnboundedSender<NetworkMessage>, log: slog::Logger) -> Self {
|
||||
impl<T: EthSpec> HandlerNetworkContext<T> {
|
||||
pub fn new(network_send: mpsc::UnboundedSender<NetworkMessage<T>>, log: slog::Logger) -> Self {
|
||||
Self { network_send, log }
|
||||
}
|
||||
|
||||
@ -655,7 +658,7 @@ impl HandlerNetworkContext {
|
||||
});
|
||||
}
|
||||
|
||||
pub fn send_rpc_request(&mut self, peer_id: PeerId, rpc_request: RPCRequest) {
|
||||
pub fn send_rpc_request(&mut self, peer_id: PeerId, rpc_request: RPCRequest<T>) {
|
||||
// the message handler cannot send requests with ids. Id's are managed by the sync
|
||||
// manager.
|
||||
let request_id = 0;
|
||||
@ -667,7 +670,7 @@ impl HandlerNetworkContext {
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
request_id: RequestId,
|
||||
rpc_response: RPCResponse,
|
||||
rpc_response: RPCResponse<T>,
|
||||
) {
|
||||
self.send_rpc_event(
|
||||
peer_id,
|
||||
@ -680,12 +683,12 @@ impl HandlerNetworkContext {
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
request_id: RequestId,
|
||||
rpc_error_response: RPCErrorResponse,
|
||||
rpc_error_response: RPCErrorResponse<T>,
|
||||
) {
|
||||
self.send_rpc_event(peer_id, RPCEvent::Response(request_id, rpc_error_response));
|
||||
}
|
||||
|
||||
fn send_rpc_event(&mut self, peer_id: PeerId, rpc_event: RPCEvent) {
|
||||
fn send_rpc_event(&mut self, peer_id: PeerId, rpc_event: RPCEvent<T>) {
|
||||
self.network_send
|
||||
.try_send(NetworkMessage::RPC(peer_id, rpc_event))
|
||||
.unwrap_or_else(|_| {
|
||||
@ -1,23 +1,24 @@
|
||||
use crate::error;
|
||||
use crate::message_handler::{HandlerMessage, MessageHandler};
|
||||
use crate::persisted_dht::{load_dht, persist_dht};
|
||||
use crate::NetworkConfig;
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use core::marker::PhantomData;
|
||||
use eth2_libp2p::Service as LibP2PService;
|
||||
use eth2_libp2p::{
|
||||
rpc::RPCRequest, Enr, Libp2pEvent, MessageId, Multiaddr, NetworkGlobals, PeerId, Swarm, Topic,
|
||||
use crate::router::{Router, RouterMessage};
|
||||
use crate::{
|
||||
attestation_service::{AttServiceMessage, AttestationService},
|
||||
NetworkConfig,
|
||||
};
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use eth2_libp2p::Service as LibP2PService;
|
||||
use eth2_libp2p::{rpc::RPCRequest, Enr, Libp2pEvent, MessageId, NetworkGlobals, PeerId, Swarm};
|
||||
use eth2_libp2p::{PubsubMessage, RPCEvent};
|
||||
use futures::prelude::*;
|
||||
use futures::Stream;
|
||||
use rest_types::ValidatorSubscription;
|
||||
use slog::{debug, error, info, trace};
|
||||
use std::collections::HashSet;
|
||||
use std::sync::{atomic::Ordering, Arc};
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
use tokio::runtime::TaskExecutor;
|
||||
use tokio::sync::{mpsc, oneshot};
|
||||
use tokio::timer::Delay;
|
||||
use types::EthSpec;
|
||||
|
||||
mod tests;
|
||||
|
||||
@ -25,27 +26,46 @@ mod tests;
|
||||
const BAN_PEER_TIMEOUT: u64 = 30;
|
||||
|
||||
/// Service that handles communication between internal services and the `eth2_libp2p` network service.
|
||||
pub struct Service<T: BeaconChainTypes> {
|
||||
libp2p_port: u16,
|
||||
network_globals: Arc<NetworkGlobals>,
|
||||
_libp2p_exit: oneshot::Sender<()>,
|
||||
_network_send: mpsc::UnboundedSender<NetworkMessage>,
|
||||
_phantom: PhantomData<T>,
|
||||
pub struct NetworkService<T: BeaconChainTypes> {
|
||||
/// The underlying libp2p service that drives all the network interactions.
|
||||
libp2p: LibP2PService<T::EthSpec>,
|
||||
/// An attestation and subnet manager service.
|
||||
attestation_service: AttestationService<T>,
|
||||
/// The receiver channel for lighthouse to communicate with the network service.
|
||||
network_recv: mpsc::UnboundedReceiver<NetworkMessage<T::EthSpec>>,
|
||||
/// The sending channel for the network service to send messages to be routed throughout
|
||||
/// lighthouse.
|
||||
router_send: mpsc::UnboundedSender<RouterMessage<T::EthSpec>>,
|
||||
/// A reference to lighthouse's database to persist the DHT.
|
||||
store: Arc<T::Store>,
|
||||
/// A collection of global variables, accessible outside of the network service.
|
||||
network_globals: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
/// An initial delay to update variables after the libp2p service has started.
|
||||
initial_delay: Delay,
|
||||
/// The logger for the network service.
|
||||
log: slog::Logger,
|
||||
/// A probability of propagation.
|
||||
propagation_percentage: Option<u8>,
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> Service<T> {
|
||||
pub fn new(
|
||||
impl<T: BeaconChainTypes> NetworkService<T> {
|
||||
pub fn start(
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
config: &NetworkConfig,
|
||||
executor: &TaskExecutor,
|
||||
network_log: slog::Logger,
|
||||
) -> error::Result<(Arc<Self>, mpsc::UnboundedSender<NetworkMessage>)> {
|
||||
) -> error::Result<(
|
||||
Arc<NetworkGlobals<T::EthSpec>>,
|
||||
mpsc::UnboundedSender<NetworkMessage<T::EthSpec>>,
|
||||
oneshot::Sender<()>,
|
||||
)> {
|
||||
// build the network channel
|
||||
let (network_send, network_recv) = mpsc::unbounded_channel::<NetworkMessage>();
|
||||
// launch message handler thread
|
||||
let (network_send, network_recv) = mpsc::unbounded_channel::<NetworkMessage<T::EthSpec>>();
|
||||
// Get a reference to the beacon chain store
|
||||
let store = beacon_chain.store.clone();
|
||||
let message_handler_send = MessageHandler::spawn(
|
||||
beacon_chain,
|
||||
// launch the router task
|
||||
let router_send = Router::spawn(
|
||||
beacon_chain.clone(),
|
||||
network_send.clone(),
|
||||
executor,
|
||||
network_log.clone(),
|
||||
@ -53,82 +73,42 @@ impl<T: BeaconChainTypes> Service<T> {
|
||||
|
||||
let propagation_percentage = config.propagation_percentage;
|
||||
// launch libp2p service
|
||||
let (network_globals, mut libp2p_service) =
|
||||
LibP2PService::new(config, network_log.clone())?;
|
||||
let (network_globals, mut libp2p) = LibP2PService::new(config, network_log.clone())?;
|
||||
|
||||
for enr in load_dht::<T::Store, T::EthSpec>(store.clone()) {
|
||||
libp2p_service.swarm.add_enr(enr);
|
||||
for enr in load_dht::<T>(store.clone()) {
|
||||
libp2p.swarm.add_enr(enr);
|
||||
}
|
||||
|
||||
// A delay used to initialise code after the network has started
|
||||
// This is currently used to obtain the listening addresses from the libp2p service.
|
||||
let initial_delay = Delay::new(Instant::now() + Duration::from_secs(1));
|
||||
|
||||
let libp2p_exit = spawn_service::<T>(
|
||||
libp2p_service,
|
||||
network_recv,
|
||||
message_handler_send,
|
||||
executor,
|
||||
store,
|
||||
network_globals.clone(),
|
||||
initial_delay,
|
||||
network_log.clone(),
|
||||
propagation_percentage,
|
||||
)?;
|
||||
// create the attestation service
|
||||
let attestation_service =
|
||||
AttestationService::new(beacon_chain, network_globals.clone(), &network_log);
|
||||
|
||||
let network_service = Service {
|
||||
libp2p_port: config.libp2p_port,
|
||||
network_globals,
|
||||
_libp2p_exit: libp2p_exit,
|
||||
_network_send: network_send.clone(),
|
||||
_phantom: PhantomData,
|
||||
// create the network service and spawn the task
|
||||
let network_service = NetworkService {
|
||||
libp2p,
|
||||
attestation_service,
|
||||
network_recv,
|
||||
router_send,
|
||||
store,
|
||||
network_globals: network_globals.clone(),
|
||||
initial_delay,
|
||||
log: network_log,
|
||||
propagation_percentage,
|
||||
};
|
||||
|
||||
Ok((Arc::new(network_service), network_send))
|
||||
}
|
||||
let network_exit = spawn_service(network_service, &executor)?;
|
||||
|
||||
/// Returns the local ENR from the underlying Discv5 behaviour that external peers may connect
|
||||
/// to.
|
||||
pub fn local_enr(&self) -> Option<Enr> {
|
||||
self.network_globals.local_enr.read().clone()
|
||||
}
|
||||
|
||||
/// Returns the local libp2p PeerID.
|
||||
pub fn local_peer_id(&self) -> PeerId {
|
||||
self.network_globals.peer_id.read().clone()
|
||||
}
|
||||
|
||||
/// Returns the list of `Multiaddr` that the underlying libp2p instance is listening on.
|
||||
pub fn listen_multiaddrs(&self) -> Vec<Multiaddr> {
|
||||
self.network_globals.listen_multiaddrs.read().clone()
|
||||
}
|
||||
|
||||
/// Returns the libp2p port that this node has been configured to listen using.
|
||||
pub fn listen_port(&self) -> u16 {
|
||||
self.libp2p_port
|
||||
}
|
||||
|
||||
/// Returns the number of libp2p connected peers.
|
||||
pub fn connected_peers(&self) -> usize {
|
||||
self.network_globals.connected_peers.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
/// Returns the set of `PeerId` that are connected via libp2p.
|
||||
pub fn connected_peer_set(&self) -> HashSet<PeerId> {
|
||||
self.network_globals.connected_peer_set.read().clone()
|
||||
Ok((network_globals, network_send, network_exit))
|
||||
}
|
||||
}
|
||||
|
||||
fn spawn_service<T: BeaconChainTypes>(
|
||||
mut libp2p_service: LibP2PService,
|
||||
mut network_recv: mpsc::UnboundedReceiver<NetworkMessage>,
|
||||
mut message_handler_send: mpsc::UnboundedSender<HandlerMessage>,
|
||||
mut service: NetworkService<T>,
|
||||
executor: &TaskExecutor,
|
||||
store: Arc<T::Store>,
|
||||
network_globals: Arc<NetworkGlobals>,
|
||||
mut initial_delay: Delay,
|
||||
log: slog::Logger,
|
||||
propagation_percentage: Option<u8>,
|
||||
) -> error::Result<tokio::sync::oneshot::Sender<()>> {
|
||||
let (network_exit, mut exit_rx) = tokio::sync::oneshot::channel();
|
||||
|
||||
@ -136,25 +116,26 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
executor.spawn(
|
||||
futures::future::poll_fn(move || -> Result<_, ()> {
|
||||
|
||||
let log = &service.log;
|
||||
|
||||
if !initial_delay.is_elapsed() {
|
||||
if let Ok(Async::Ready(_)) = initial_delay.poll() {
|
||||
let multi_addrs = Swarm::listeners(&libp2p_service.swarm).cloned().collect();
|
||||
*network_globals.listen_multiaddrs.write() = multi_addrs;
|
||||
if !service.initial_delay.is_elapsed() {
|
||||
if let Ok(Async::Ready(_)) = service.initial_delay.poll() {
|
||||
let multi_addrs = Swarm::listeners(&service.libp2p.swarm).cloned().collect();
|
||||
*service.network_globals.listen_multiaddrs.write() = multi_addrs;
|
||||
}
|
||||
}
|
||||
|
||||
// perform termination tasks when the network is being shutdown
|
||||
if let Ok(Async::Ready(_)) | Err(_) = exit_rx.poll() {
|
||||
// network thread is terminating
|
||||
let enrs: Vec<Enr> = libp2p_service.swarm.enr_entries().cloned().collect();
|
||||
let enrs: Vec<Enr> = service.libp2p.swarm.enr_entries().cloned().collect();
|
||||
debug!(
|
||||
log,
|
||||
"Persisting DHT to store";
|
||||
"Number of peers" => format!("{}", enrs.len()),
|
||||
);
|
||||
|
||||
match persist_dht::<T::Store, T::EthSpec>(store.clone(), enrs) {
|
||||
match persist_dht::<T>(service.store.clone(), enrs) {
|
||||
Err(e) => error!(
|
||||
log,
|
||||
"Failed to persist DHT on drop";
|
||||
@ -173,11 +154,11 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
// processes the network channel before processing the libp2p swarm
|
||||
loop {
|
||||
// poll the network channel
|
||||
match network_recv.poll() {
|
||||
match service.network_recv.poll() {
|
||||
Ok(Async::Ready(Some(message))) => match message {
|
||||
NetworkMessage::RPC(peer_id, rpc_event) => {
|
||||
trace!(log, "Sending RPC"; "rpc" => format!("{}", rpc_event));
|
||||
libp2p_service.swarm.send_rpc(peer_id, rpc_event);
|
||||
service.libp2p.swarm.send_rpc(peer_id, rpc_event);
|
||||
}
|
||||
NetworkMessage::Propagate {
|
||||
propagation_source,
|
||||
@ -186,7 +167,7 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
// TODO: Remove this for mainnet
|
||||
// randomly prevents propagation
|
||||
let mut should_send = true;
|
||||
if let Some(percentage) = propagation_percentage {
|
||||
if let Some(percentage) = service.propagation_percentage {
|
||||
// not exact percentage but close enough
|
||||
let rand = rand::random::<u8>() % 100;
|
||||
if rand > percentage {
|
||||
@ -201,16 +182,16 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
"propagation_peer" => format!("{:?}", propagation_source),
|
||||
"message_id" => message_id.to_string(),
|
||||
);
|
||||
libp2p_service
|
||||
service.libp2p
|
||||
.swarm
|
||||
.propagate_message(&propagation_source, message_id);
|
||||
}
|
||||
}
|
||||
NetworkMessage::Publish { topics, message } => {
|
||||
NetworkMessage::Publish { messages } => {
|
||||
// TODO: Remove this for mainnet
|
||||
// randomly prevents propagation
|
||||
let mut should_send = true;
|
||||
if let Some(percentage) = propagation_percentage {
|
||||
if let Some(percentage) = service.propagation_percentage {
|
||||
// not exact percentage but close enough
|
||||
let rand = rand::random::<u8>() % 100;
|
||||
if rand > percentage {
|
||||
@ -219,18 +200,31 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
}
|
||||
}
|
||||
if !should_send {
|
||||
info!(log, "Random filter did not publish message");
|
||||
info!(log, "Random filter did not publish messages");
|
||||
} else {
|
||||
debug!(log, "Sending pubsub message"; "topics" => format!("{:?}",topics));
|
||||
libp2p_service.swarm.publish(&topics, message);
|
||||
let mut unique_topics = Vec::new();
|
||||
for message in &messages {
|
||||
for topic in message.topics() {
|
||||
if !unique_topics.contains(&topic) {
|
||||
unique_topics.push(topic);
|
||||
}
|
||||
}
|
||||
}
|
||||
debug!(log, "Sending pubsub messages"; "count" => messages.len(), "topics" => format!("{:?}", unique_topics));
|
||||
service.libp2p.swarm.publish(messages);
|
||||
}
|
||||
}
|
||||
NetworkMessage::Disconnect { peer_id } => {
|
||||
libp2p_service.disconnect_and_ban_peer(
|
||||
service.libp2p.disconnect_and_ban_peer(
|
||||
peer_id,
|
||||
std::time::Duration::from_secs(BAN_PEER_TIMEOUT),
|
||||
);
|
||||
}
|
||||
NetworkMessage::Subscribe { subscriptions } =>
|
||||
{
|
||||
// the result is dropped as it used solely for ergonomics
|
||||
let _ = service.attestation_service.validator_subscriptions(subscriptions);
|
||||
}
|
||||
},
|
||||
Ok(Async::NotReady) => break,
|
||||
Ok(Async::Ready(None)) => {
|
||||
@ -244,10 +238,24 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
}
|
||||
}
|
||||
|
||||
// process any attestation service events
|
||||
// NOTE: This must come after the network message processing as that may trigger events in
|
||||
// the attestation service.
|
||||
while let Ok(Async::Ready(Some(attestation_service_message))) = service.attestation_service.poll() {
|
||||
match attestation_service_message {
|
||||
// TODO: Implement
|
||||
AttServiceMessage::Subscribe(_subnet) => { },
|
||||
AttServiceMessage::Unsubscribe(_subnet) => { },
|
||||
AttServiceMessage::EnrAdd(_subnet) => { },
|
||||
AttServiceMessage::EnrRemove(_subnet) => { },
|
||||
AttServiceMessage::DiscoverPeers(_subnet) => { },
|
||||
}
|
||||
}
|
||||
|
||||
let mut peers_to_ban = Vec::new();
|
||||
// poll the swarm
|
||||
loop {
|
||||
match libp2p_service.poll() {
|
||||
match service.libp2p.poll() {
|
||||
Ok(Async::Ready(Some(event))) => match event {
|
||||
Libp2pEvent::RPC(peer_id, rpc_event) => {
|
||||
// trace!(log, "Received RPC"; "rpc" => format!("{}", rpc_event));
|
||||
@ -256,21 +264,21 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
if let RPCEvent::Request(_, RPCRequest::Goodbye(_)) = rpc_event {
|
||||
peers_to_ban.push(peer_id.clone());
|
||||
};
|
||||
message_handler_send
|
||||
.try_send(HandlerMessage::RPC(peer_id, rpc_event))
|
||||
.map_err(|_| { debug!(log, "Failed to send RPC to handler");} )?;
|
||||
service.router_send
|
||||
.try_send(RouterMessage::RPC(peer_id, rpc_event))
|
||||
.map_err(|_| { debug!(log, "Failed to send RPC to router");} )?;
|
||||
}
|
||||
Libp2pEvent::PeerDialed(peer_id) => {
|
||||
debug!(log, "Peer Dialed"; "peer_id" => format!("{:?}", peer_id));
|
||||
message_handler_send
|
||||
.try_send(HandlerMessage::PeerDialed(peer_id))
|
||||
.map_err(|_| { debug!(log, "Failed to send peer dialed to handler");})?;
|
||||
service.router_send
|
||||
.try_send(RouterMessage::PeerDialed(peer_id))
|
||||
.map_err(|_| { debug!(log, "Failed to send peer dialed to router");})?;
|
||||
}
|
||||
Libp2pEvent::PeerDisconnected(peer_id) => {
|
||||
debug!(log, "Peer Disconnected"; "peer_id" => format!("{:?}", peer_id));
|
||||
message_handler_send
|
||||
.try_send(HandlerMessage::PeerDisconnected(peer_id))
|
||||
.map_err(|_| { debug!(log, "Failed to send peer disconnect to handler");})?;
|
||||
service.router_send
|
||||
.try_send(RouterMessage::PeerDisconnected(peer_id))
|
||||
.map_err(|_| { debug!(log, "Failed to send peer disconnect to router");})?;
|
||||
}
|
||||
Libp2pEvent::PubsubMessage {
|
||||
id,
|
||||
@ -278,9 +286,9 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
message,
|
||||
..
|
||||
} => {
|
||||
message_handler_send
|
||||
.try_send(HandlerMessage::PubsubMessage(id, source, message))
|
||||
.map_err(|_| { debug!(log, "Failed to send pubsub message to handler");})?;
|
||||
service.router_send
|
||||
.try_send(RouterMessage::PubsubMessage(id, source, message))
|
||||
.map_err(|_| { debug!(log, "Failed to send pubsub message to router");})?;
|
||||
}
|
||||
Libp2pEvent::PeerSubscribed(_, _) => {}
|
||||
},
|
||||
@ -292,7 +300,7 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
|
||||
// ban and disconnect any peers that sent Goodbye requests
|
||||
while let Some(peer_id) = peers_to_ban.pop() {
|
||||
libp2p_service.disconnect_and_ban_peer(
|
||||
service.libp2p.disconnect_and_ban_peer(
|
||||
peer_id.clone(),
|
||||
std::time::Duration::from_secs(BAN_PEER_TIMEOUT),
|
||||
);
|
||||
@ -308,14 +316,15 @@ fn spawn_service<T: BeaconChainTypes>(
|
||||
|
||||
/// Types of messages that the network service can receive.
|
||||
#[derive(Debug)]
|
||||
pub enum NetworkMessage {
|
||||
/// Send an RPC message to the libp2p service.
|
||||
RPC(PeerId, RPCEvent),
|
||||
/// Publish a message to gossipsub.
|
||||
Publish {
|
||||
topics: Vec<Topic>,
|
||||
message: PubsubMessage,
|
||||
pub enum NetworkMessage<T: EthSpec> {
|
||||
/// Subscribes a list of validators to specific slots for attestation duties.
|
||||
Subscribe {
|
||||
subscriptions: Vec<ValidatorSubscription>,
|
||||
},
|
||||
/// Send an RPC message to the libp2p service.
|
||||
RPC(PeerId, RPCEvent<T>),
|
||||
/// Publish a list of messages to the gossipsub protocol.
|
||||
Publish { messages: Vec<PubsubMessage<T>> },
|
||||
/// Propagate a received gossipsub message.
|
||||
Propagate {
|
||||
propagation_source: PeerId,
|
||||
|
||||
@ -35,7 +35,7 @@
|
||||
|
||||
use super::network_context::SyncNetworkContext;
|
||||
use super::range_sync::{Batch, BatchProcessResult, RangeSync};
|
||||
use crate::message_processor::PeerSyncInfo;
|
||||
use crate::router::processor::PeerSyncInfo;
|
||||
use crate::service::NetworkMessage;
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes, BlockProcessingOutcome};
|
||||
use eth2_libp2p::rpc::methods::*;
|
||||
@ -153,7 +153,7 @@ pub struct SyncManager<T: BeaconChainTypes> {
|
||||
input_channel: mpsc::UnboundedReceiver<SyncMessage<T::EthSpec>>,
|
||||
|
||||
/// A network context to contact the network service.
|
||||
network: SyncNetworkContext,
|
||||
network: SyncNetworkContext<T::EthSpec>,
|
||||
|
||||
/// The object handling long-range batch load-balanced syncing.
|
||||
range_sync: RangeSync<T>,
|
||||
@ -180,7 +180,7 @@ pub struct SyncManager<T: BeaconChainTypes> {
|
||||
pub fn spawn<T: BeaconChainTypes>(
|
||||
executor: &tokio::runtime::TaskExecutor,
|
||||
beacon_chain: Weak<BeaconChain<T>>,
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage>,
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage<T::EthSpec>>,
|
||||
log: slog::Logger,
|
||||
) -> (
|
||||
mpsc::UnboundedSender<SyncMessage<T::EthSpec>>,
|
||||
@ -391,7 +391,7 @@ impl<T: BeaconChainTypes> SyncManager<T> {
|
||||
|
||||
// we have the correct block, try and process it
|
||||
if let Some(chain) = self.chain.upgrade() {
|
||||
match chain.process_block(block.clone()) {
|
||||
match BlockProcessingOutcome::shim(chain.process_block(block.clone())) {
|
||||
Ok(outcome) => {
|
||||
match outcome {
|
||||
BlockProcessingOutcome::Processed { block_root } => {
|
||||
@ -597,7 +597,7 @@ impl<T: BeaconChainTypes> SyncManager<T> {
|
||||
.downloaded_blocks
|
||||
.pop()
|
||||
.expect("There is always at least one block in the queue");
|
||||
match chain.process_block(newest_block.clone()) {
|
||||
match BlockProcessingOutcome::shim(chain.process_block(newest_block.clone())) {
|
||||
Ok(BlockProcessingOutcome::ParentUnknown { .. }) => {
|
||||
// need to keep looking for parents
|
||||
// add the block back to the queue and continue the search
|
||||
@ -642,7 +642,7 @@ impl<T: BeaconChainTypes> SyncManager<T> {
|
||||
while let Some(block) = parent_request.downloaded_blocks.pop() {
|
||||
// check if the chain exists
|
||||
if let Some(chain) = self.chain.upgrade() {
|
||||
match chain.process_block(block) {
|
||||
match BlockProcessingOutcome::shim(chain.process_block(block)) {
|
||||
Ok(BlockProcessingOutcome::Processed { .. })
|
||||
| Ok(BlockProcessingOutcome::BlockIsAlreadyKnown { .. }) => {} // continue to the next block
|
||||
|
||||
|
||||
@ -5,9 +5,4 @@ pub mod manager;
|
||||
mod network_context;
|
||||
mod range_sync;
|
||||
|
||||
/// Currently implemented sync methods.
|
||||
pub enum SyncMethod {
|
||||
SimpleSync,
|
||||
}
|
||||
|
||||
pub use manager::SyncMessage;
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
//! Provides network functionality for the Syncing thread. This fundamentally wraps a network
|
||||
//! channel and stores a global RPC ID to perform requests.
|
||||
|
||||
use crate::message_processor::status_message;
|
||||
use crate::router::processor::status_message;
|
||||
use crate::service::NetworkMessage;
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use eth2_libp2p::rpc::methods::*;
|
||||
@ -10,20 +10,21 @@ use eth2_libp2p::PeerId;
|
||||
use slog::{debug, trace, warn};
|
||||
use std::sync::Weak;
|
||||
use tokio::sync::mpsc;
|
||||
use types::EthSpec;
|
||||
|
||||
/// Wraps a Network channel to employ various RPC related network functionality for the Sync manager. This includes management of a global RPC request Id.
|
||||
|
||||
pub struct SyncNetworkContext {
|
||||
pub struct SyncNetworkContext<T: EthSpec> {
|
||||
/// The network channel to relay messages to the Network service.
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage>,
|
||||
network_send: mpsc::UnboundedSender<NetworkMessage<T>>,
|
||||
|
||||
request_id: RequestId,
|
||||
/// Logger for the `SyncNetworkContext`.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
impl SyncNetworkContext {
|
||||
pub fn new(network_send: mpsc::UnboundedSender<NetworkMessage>, log: slog::Logger) -> Self {
|
||||
impl<T: EthSpec> SyncNetworkContext<T> {
|
||||
pub fn new(network_send: mpsc::UnboundedSender<NetworkMessage<T>>, log: slog::Logger) -> Self {
|
||||
Self {
|
||||
network_send,
|
||||
request_id: 0,
|
||||
@ -31,9 +32,9 @@ impl SyncNetworkContext {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn status_peer<T: BeaconChainTypes>(
|
||||
pub fn status_peer<U: BeaconChainTypes>(
|
||||
&mut self,
|
||||
chain: Weak<BeaconChain<T>>,
|
||||
chain: Weak<BeaconChain<U>>,
|
||||
peer_id: PeerId,
|
||||
) {
|
||||
if let Some(chain) = chain.upgrade() {
|
||||
@ -117,7 +118,7 @@ impl SyncNetworkContext {
|
||||
pub fn send_rpc_request(
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
rpc_request: RPCRequest,
|
||||
rpc_request: RPCRequest<T>,
|
||||
) -> Result<RequestId, &'static str> {
|
||||
let request_id = self.request_id;
|
||||
self.request_id += 1;
|
||||
@ -125,7 +126,11 @@ impl SyncNetworkContext {
|
||||
Ok(request_id)
|
||||
}
|
||||
|
||||
fn send_rpc_event(&mut self, peer_id: PeerId, rpc_event: RPCEvent) -> Result<(), &'static str> {
|
||||
fn send_rpc_event(
|
||||
&mut self,
|
||||
peer_id: PeerId,
|
||||
rpc_event: RPCEvent<T>,
|
||||
) -> Result<(), &'static str> {
|
||||
self.network_send
|
||||
.try_send(NetworkMessage::RPC(peer_id, rpc_event))
|
||||
.map_err(|_| {
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
use super::batch::Batch;
|
||||
use crate::message_processor::FUTURE_SLOT_TOLERANCE;
|
||||
use crate::router::processor::FUTURE_SLOT_TOLERANCE;
|
||||
use crate::sync::manager::SyncMessage;
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes, BlockProcessingOutcome};
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes, BlockError};
|
||||
use slog::{debug, error, trace, warn};
|
||||
use std::sync::{Arc, Weak};
|
||||
use tokio::sync::mpsc;
|
||||
@ -54,116 +54,78 @@ fn process_batch<T: BeaconChainTypes>(
|
||||
batch: &Batch<T::EthSpec>,
|
||||
log: &slog::Logger,
|
||||
) -> Result<(), String> {
|
||||
let mut successful_block_import = false;
|
||||
for block in &batch.downloaded_blocks {
|
||||
if let Some(chain) = chain.upgrade() {
|
||||
let processing_result = chain.process_block(block.clone());
|
||||
|
||||
if let Ok(outcome) = processing_result {
|
||||
match outcome {
|
||||
BlockProcessingOutcome::Processed { block_root } => {
|
||||
// The block was valid and we processed it successfully.
|
||||
trace!(
|
||||
log, "Imported block from network";
|
||||
"slot" => block.slot(),
|
||||
"block_root" => format!("{}", block_root),
|
||||
);
|
||||
successful_block_import = true;
|
||||
}
|
||||
BlockProcessingOutcome::ParentUnknown { parent, .. } => {
|
||||
// blocks should be sequential and all parents should exist
|
||||
warn!(
|
||||
log, "Parent block is unknown";
|
||||
"parent_root" => format!("{}", parent),
|
||||
"baby_block_slot" => block.slot(),
|
||||
);
|
||||
if successful_block_import {
|
||||
run_fork_choice(chain, log);
|
||||
}
|
||||
return Err(format!(
|
||||
"Block at slot {} has an unknown parent.",
|
||||
block.slot()
|
||||
));
|
||||
}
|
||||
BlockProcessingOutcome::BlockIsAlreadyKnown => {
|
||||
// this block is already known to us, move to the next
|
||||
debug!(
|
||||
log, "Imported a block that is already known";
|
||||
"block_slot" => block.slot(),
|
||||
);
|
||||
}
|
||||
BlockProcessingOutcome::FutureSlot {
|
||||
present_slot,
|
||||
block_slot,
|
||||
} => {
|
||||
if present_slot + FUTURE_SLOT_TOLERANCE >= block_slot {
|
||||
// The block is too far in the future, drop it.
|
||||
warn!(
|
||||
log, "Block is ahead of our slot clock";
|
||||
"msg" => "block for future slot rejected, check your time",
|
||||
"present_slot" => present_slot,
|
||||
"block_slot" => block_slot,
|
||||
"FUTURE_SLOT_TOLERANCE" => FUTURE_SLOT_TOLERANCE,
|
||||
);
|
||||
if successful_block_import {
|
||||
run_fork_choice(chain, log);
|
||||
}
|
||||
return Err(format!(
|
||||
"Block at slot {} is too far in the future",
|
||||
block.slot()
|
||||
));
|
||||
} else {
|
||||
// The block is in the future, but not too far.
|
||||
debug!(
|
||||
log, "Block is slightly ahead of our slot clock, ignoring.";
|
||||
"present_slot" => present_slot,
|
||||
"block_slot" => block_slot,
|
||||
"FUTURE_SLOT_TOLERANCE" => FUTURE_SLOT_TOLERANCE,
|
||||
);
|
||||
}
|
||||
}
|
||||
BlockProcessingOutcome::WouldRevertFinalizedSlot { .. } => {
|
||||
debug!(
|
||||
log, "Finalized or earlier block processed";
|
||||
"outcome" => format!("{:?}", outcome),
|
||||
);
|
||||
// block reached our finalized slot or was earlier, move to the next block
|
||||
}
|
||||
BlockProcessingOutcome::GenesisBlock => {
|
||||
debug!(
|
||||
log, "Genesis block was processed";
|
||||
"outcome" => format!("{:?}", outcome),
|
||||
);
|
||||
}
|
||||
_ => {
|
||||
warn!(
|
||||
log, "Invalid block received";
|
||||
"msg" => "peer sent invalid block",
|
||||
"outcome" => format!("{:?}", outcome),
|
||||
);
|
||||
if successful_block_import {
|
||||
run_fork_choice(chain, log);
|
||||
}
|
||||
return Err(format!("Invalid block at slot {}", block.slot()));
|
||||
}
|
||||
if let Some(chain) = chain.upgrade() {
|
||||
match chain.process_chain_segment(batch.downloaded_blocks.clone()) {
|
||||
Ok(roots) => {
|
||||
trace!(
|
||||
log, "Imported blocks from network";
|
||||
"count" => roots.len(),
|
||||
);
|
||||
}
|
||||
Err(BlockError::ParentUnknown(parent)) => {
|
||||
// blocks should be sequential and all parents should exist
|
||||
warn!(
|
||||
log, "Parent block is unknown";
|
||||
"parent_root" => format!("{}", parent),
|
||||
);
|
||||
}
|
||||
Err(BlockError::BlockIsAlreadyKnown) => {
|
||||
// this block is already known to us, move to the next
|
||||
debug!(
|
||||
log, "Imported a block that is already known";
|
||||
);
|
||||
}
|
||||
Err(BlockError::FutureSlot {
|
||||
present_slot,
|
||||
block_slot,
|
||||
}) => {
|
||||
if present_slot + FUTURE_SLOT_TOLERANCE >= block_slot {
|
||||
// The block is too far in the future, drop it.
|
||||
warn!(
|
||||
log, "Block is ahead of our slot clock";
|
||||
"msg" => "block for future slot rejected, check your time",
|
||||
"present_slot" => present_slot,
|
||||
"block_slot" => block_slot,
|
||||
"FUTURE_SLOT_TOLERANCE" => FUTURE_SLOT_TOLERANCE,
|
||||
);
|
||||
} else {
|
||||
// The block is in the future, but not too far.
|
||||
debug!(
|
||||
log, "Block is slightly ahead of our slot clock, ignoring.";
|
||||
"present_slot" => present_slot,
|
||||
"block_slot" => block_slot,
|
||||
"FUTURE_SLOT_TOLERANCE" => FUTURE_SLOT_TOLERANCE,
|
||||
);
|
||||
}
|
||||
} else {
|
||||
}
|
||||
Err(BlockError::WouldRevertFinalizedSlot { .. }) => {
|
||||
debug!(
|
||||
log, "Finalized or earlier block processed";
|
||||
);
|
||||
// block reached our finalized slot or was earlier, move to the next block
|
||||
}
|
||||
Err(BlockError::GenesisBlock) => {
|
||||
debug!(
|
||||
log, "Genesis block was processed";
|
||||
);
|
||||
}
|
||||
Err(BlockError::BeaconChainError(e)) => {
|
||||
warn!(
|
||||
log, "BlockProcessingFailure";
|
||||
"msg" => "unexpected condition in processing block.",
|
||||
"outcome" => format!("{:?}", processing_result)
|
||||
"outcome" => format!("{:?}", e)
|
||||
);
|
||||
}
|
||||
other => {
|
||||
warn!(
|
||||
log, "Invalid block received";
|
||||
"msg" => "peer sent invalid block",
|
||||
"outcome" => format!("{:?}", other),
|
||||
);
|
||||
if successful_block_import {
|
||||
run_fork_choice(chain, log);
|
||||
}
|
||||
return Err(format!(
|
||||
"Unexpected block processing error: {:?}",
|
||||
processing_result
|
||||
));
|
||||
}
|
||||
} else {
|
||||
return Ok(()); // terminate early due to dropped beacon chain
|
||||
}
|
||||
} else {
|
||||
return Ok(()); // terminate early due to dropped beacon chain
|
||||
}
|
||||
|
||||
// Batch completed successfully, run fork choice.
|
||||
|
||||
@ -17,7 +17,7 @@ use types::{Hash256, SignedBeaconBlock, Slot};
|
||||
/// downvote peers with poor bandwidth. This can be set arbitrarily high, in which case the
|
||||
/// responder will fill the response up to the max request size, assuming they have the bandwidth
|
||||
/// to do so.
|
||||
pub const BLOCKS_PER_BATCH: u64 = 50;
|
||||
pub const BLOCKS_PER_BATCH: u64 = 64;
|
||||
|
||||
/// The number of times to retry a batch before the chain is considered failed and removed.
|
||||
const MAX_BATCH_RETRIES: u8 = 5;
|
||||
@ -141,7 +141,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
/// batch.
|
||||
pub fn on_block_response(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
request_id: RequestId,
|
||||
beacon_block: &Option<SignedBeaconBlock<T::EthSpec>>,
|
||||
) -> Option<()> {
|
||||
@ -161,7 +161,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
/// failed indicating that further batches are required.
|
||||
fn handle_completed_batch(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
batch: Batch<T::EthSpec>,
|
||||
) {
|
||||
// An entire batch of blocks has been received. This functions checks to see if it can be processed,
|
||||
@ -255,7 +255,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
/// of the batch processor.
|
||||
pub fn on_batch_process_result(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
processing_id: u64,
|
||||
batch: &mut Option<Batch<T::EthSpec>>,
|
||||
result: &BatchProcessResult,
|
||||
@ -385,7 +385,11 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
// TODO: Batches could have been partially downloaded due to RPC size-limit restrictions. We
|
||||
// need to add logic for partial batch downloads. Potentially, if another peer returns the same
|
||||
// batch, we try a partial download.
|
||||
fn handle_invalid_batch(&mut self, network: &mut SyncNetworkContext, batch: Batch<T::EthSpec>) {
|
||||
fn handle_invalid_batch(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
batch: Batch<T::EthSpec>,
|
||||
) {
|
||||
// The current batch could not be processed, indicating either the current or previous
|
||||
// batches are invalid
|
||||
|
||||
@ -415,7 +419,11 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
///
|
||||
/// If the re-downloaded batch is different to the original and can be processed, the original
|
||||
/// peer will be downvoted.
|
||||
fn reprocess_batch(&mut self, network: &mut SyncNetworkContext, mut batch: Batch<T::EthSpec>) {
|
||||
fn reprocess_batch(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
mut batch: Batch<T::EthSpec>,
|
||||
) {
|
||||
// marks the batch as attempting to be reprocessed by hashing the downloaded blocks
|
||||
batch.original_hash = Some(batch.hash());
|
||||
|
||||
@ -455,7 +463,11 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
/// This chain has been requested to start syncing.
|
||||
///
|
||||
/// This could be new chain, or an old chain that is being resumed.
|
||||
pub fn start_syncing(&mut self, network: &mut SyncNetworkContext, local_finalized_slot: Slot) {
|
||||
pub fn start_syncing(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
local_finalized_slot: Slot,
|
||||
) {
|
||||
// A local finalized slot is provided as other chains may have made
|
||||
// progress whilst this chain was Stopped or paused. If so, update the `processed_batch_id` to
|
||||
// accommodate potentially downloaded batches from other chains. Also prune any old batches
|
||||
@ -490,7 +502,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
/// Add a peer to the chain.
|
||||
///
|
||||
/// If the chain is active, this starts requesting batches from this peer.
|
||||
pub fn add_peer(&mut self, network: &mut SyncNetworkContext, peer_id: PeerId) {
|
||||
pub fn add_peer(&mut self, network: &mut SyncNetworkContext<T::EthSpec>, peer_id: PeerId) {
|
||||
self.peer_pool.insert(peer_id.clone());
|
||||
// do not request blocks if the chain is not syncing
|
||||
if let ChainSyncingState::Stopped = self.state {
|
||||
@ -503,7 +515,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
}
|
||||
|
||||
/// Sends a STATUS message to all peers in the peer pool.
|
||||
pub fn status_peers(&self, network: &mut SyncNetworkContext) {
|
||||
pub fn status_peers(&self, network: &mut SyncNetworkContext<T::EthSpec>) {
|
||||
for peer_id in self.peer_pool.iter() {
|
||||
network.status_peer(self.chain.clone(), peer_id.clone());
|
||||
}
|
||||
@ -517,7 +529,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
/// this chain.
|
||||
pub fn inject_error(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
peer_id: &PeerId,
|
||||
request_id: RequestId,
|
||||
) -> Option<ProcessingResult> {
|
||||
@ -541,7 +553,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
/// `MAX_BATCH_RETRIES`.
|
||||
pub fn failed_batch(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
mut batch: Batch<T::EthSpec>,
|
||||
) -> ProcessingResult {
|
||||
batch.retries += 1;
|
||||
@ -575,7 +587,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
|
||||
/// Attempts to request the next required batches from the peer pool if the chain is syncing. It will exhaust the peer
|
||||
/// pool and left over batches until the batch buffer is reached or all peers are exhausted.
|
||||
fn request_batches(&mut self, network: &mut SyncNetworkContext) {
|
||||
fn request_batches(&mut self, network: &mut SyncNetworkContext<T::EthSpec>) {
|
||||
if let ChainSyncingState::Syncing = self.state {
|
||||
while self.send_range_request(network) {}
|
||||
}
|
||||
@ -583,7 +595,7 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
|
||||
/// Requests the next required batch from a peer. Returns true, if there was a peer available
|
||||
/// to send a request and there are batches to request, false otherwise.
|
||||
fn send_range_request(&mut self, network: &mut SyncNetworkContext) -> bool {
|
||||
fn send_range_request(&mut self, network: &mut SyncNetworkContext<T::EthSpec>) -> bool {
|
||||
// find the next pending batch and request it from the peer
|
||||
if let Some(peer_id) = self.get_next_peer() {
|
||||
if let Some(batch) = self.get_next_batch(peer_id) {
|
||||
@ -669,7 +681,11 @@ impl<T: BeaconChainTypes> SyncingChain<T> {
|
||||
}
|
||||
|
||||
/// Requests the provided batch from the provided peer.
|
||||
fn send_batch(&mut self, network: &mut SyncNetworkContext, batch: Batch<T::EthSpec>) {
|
||||
fn send_batch(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
batch: Batch<T::EthSpec>,
|
||||
) {
|
||||
let request = batch.to_blocks_by_range_request();
|
||||
if let Ok(request_id) = network.blocks_by_range_request(batch.current_peer.clone(), request)
|
||||
{
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
//! with this struct to to simplify the logic of the other layers of sync.
|
||||
|
||||
use super::chain::{ChainSyncingState, SyncingChain};
|
||||
use crate::message_processor::PeerSyncInfo;
|
||||
use crate::router::processor::PeerSyncInfo;
|
||||
use crate::sync::manager::SyncMessage;
|
||||
use crate::sync::network_context::SyncNetworkContext;
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
@ -103,7 +103,11 @@ impl<T: BeaconChainTypes> ChainCollection<T> {
|
||||
///
|
||||
/// This removes any out-dated chains, swaps to any higher priority finalized chains and
|
||||
/// updates the state of the collection.
|
||||
pub fn update_finalized(&mut self, network: &mut SyncNetworkContext, log: &slog::Logger) {
|
||||
pub fn update_finalized(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
log: &slog::Logger,
|
||||
) {
|
||||
let local_slot = match self.beacon_chain.upgrade() {
|
||||
Some(chain) => {
|
||||
let local = match PeerSyncInfo::from_chain(&chain) {
|
||||
@ -197,7 +201,7 @@ impl<T: BeaconChainTypes> ChainCollection<T> {
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn new_head_chain(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
remote_finalized_slot: Slot,
|
||||
target_head: Hash256,
|
||||
target_slot: Slot,
|
||||
@ -277,7 +281,11 @@ impl<T: BeaconChainTypes> ChainCollection<T> {
|
||||
///
|
||||
/// This removes chains with no peers, or chains whose start block slot is less than our current
|
||||
/// finalized block slot.
|
||||
pub fn purge_outdated_chains(&mut self, network: &mut SyncNetworkContext, log: &slog::Logger) {
|
||||
pub fn purge_outdated_chains(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
log: &slog::Logger,
|
||||
) {
|
||||
// Remove any chains that have no peers
|
||||
self.finalized_chains
|
||||
.retain(|chain| !chain.peer_pool.is_empty());
|
||||
@ -349,7 +357,7 @@ impl<T: BeaconChainTypes> ChainCollection<T> {
|
||||
/// This will re-status the chains peers on removal. The index must exist.
|
||||
pub fn remove_chain(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
index: usize,
|
||||
log: &slog::Logger,
|
||||
) {
|
||||
|
||||
@ -42,7 +42,7 @@
|
||||
use super::chain::ProcessingResult;
|
||||
use super::chain_collection::{ChainCollection, SyncState};
|
||||
use super::{Batch, BatchProcessResult};
|
||||
use crate::message_processor::PeerSyncInfo;
|
||||
use crate::router::processor::PeerSyncInfo;
|
||||
use crate::sync::manager::SyncMessage;
|
||||
use crate::sync::network_context::SyncNetworkContext;
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
@ -108,7 +108,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {
|
||||
/// prioritised by peer-pool size.
|
||||
pub fn add_peer(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
peer_id: PeerId,
|
||||
remote: PeerSyncInfo,
|
||||
) {
|
||||
@ -228,7 +228,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {
|
||||
/// This request could complete a chain or simply add to its progress.
|
||||
pub fn blocks_by_range_response(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
peer_id: PeerId,
|
||||
request_id: RequestId,
|
||||
beacon_block: Option<SignedBeaconBlock<T::EthSpec>>,
|
||||
@ -255,7 +255,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {
|
||||
|
||||
pub fn handle_block_process_result(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
processing_id: u64,
|
||||
batch: Batch<T::EthSpec>,
|
||||
result: BatchProcessResult,
|
||||
@ -326,7 +326,11 @@ impl<T: BeaconChainTypes> RangeSync<T> {
|
||||
|
||||
/// A peer has disconnected. This removes the peer from any ongoing chains and mappings. A
|
||||
/// disconnected peer could remove a chain
|
||||
pub fn peer_disconnect(&mut self, network: &mut SyncNetworkContext, peer_id: &PeerId) {
|
||||
pub fn peer_disconnect(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
peer_id: &PeerId,
|
||||
) {
|
||||
// if the peer is in the awaiting head mapping, remove it
|
||||
self.awaiting_head_peers.remove(&peer_id);
|
||||
|
||||
@ -340,7 +344,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {
|
||||
/// When a peer gets removed, both the head and finalized chains need to be searched to check which pool the peer is in. The chain may also have a batch or batches awaiting
|
||||
/// for this peer. If so we mark the batch as failed. The batch may then hit it's maximum
|
||||
/// retries. In this case, we need to remove the chain and re-status all the peers.
|
||||
fn remove_peer(&mut self, network: &mut SyncNetworkContext, peer_id: &PeerId) {
|
||||
fn remove_peer(&mut self, network: &mut SyncNetworkContext<T::EthSpec>, peer_id: &PeerId) {
|
||||
if let Some((index, ProcessingResult::RemoveChain)) =
|
||||
self.chains.head_finalized_request(|chain| {
|
||||
if chain.peer_pool.remove(peer_id) {
|
||||
@ -370,7 +374,7 @@ impl<T: BeaconChainTypes> RangeSync<T> {
|
||||
/// been too many failed attempts for the batch, remove the chain.
|
||||
pub fn inject_error(
|
||||
&mut self,
|
||||
network: &mut SyncNetworkContext,
|
||||
network: &mut SyncNetworkContext<T::EthSpec>,
|
||||
peer_id: PeerId,
|
||||
request_id: RequestId,
|
||||
) {
|
||||
|
||||
@ -1,12 +1,13 @@
|
||||
[package]
|
||||
name = "rest_api"
|
||||
version = "0.1.0"
|
||||
authors = ["Paul Hauner <paul@paulhauner.com>", "Luke Anderson <luke@sigmaprime.io>"]
|
||||
version = "0.2.0"
|
||||
authors = ["Paul Hauner <paul@paulhauner.com>", "Age Manning <Age@AgeManning.com>", "Luke Anderson <luke@sigmaprime.io>"]
|
||||
edition = "2018"
|
||||
|
||||
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
||||
[dependencies]
|
||||
bls = { path = "../../eth2/utils/bls" }
|
||||
rest_types = { path = "../../eth2/utils/rest_types" }
|
||||
beacon_chain = { path = "../beacon_chain" }
|
||||
network = { path = "../network" }
|
||||
eth2-libp2p = { path = "../eth2-libp2p" }
|
||||
@ -24,7 +25,6 @@ state_processing = { path = "../../eth2/state_processing" }
|
||||
types = { path = "../../eth2/types" }
|
||||
http = "0.1"
|
||||
hyper = "0.12"
|
||||
exit-future = "0.1.4"
|
||||
tokio = "0.1.22"
|
||||
url = "2.1"
|
||||
lazy_static = "1.3.0"
|
||||
@ -35,6 +35,7 @@ hex = "0.3"
|
||||
parking_lot = "0.9"
|
||||
futures = "0.1.29"
|
||||
operation_pool = { path = "../../eth2/operation_pool" }
|
||||
rayon = "1.3.0"
|
||||
|
||||
[dev-dependencies]
|
||||
remote_beacon_node = { path = "../../eth2/utils/remote_beacon_node" }
|
||||
|
||||
@ -5,29 +5,17 @@ use crate::{ApiError, ApiResult, BoxFut, UrlQuery};
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes, StateSkipConfig};
|
||||
use futures::{Future, Stream};
|
||||
use hyper::{Body, Request};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use ssz_derive::{Decode, Encode};
|
||||
use rest_types::{
|
||||
BlockResponse, CanonicalHeadResponse, Committee, HeadBeaconBlock, StateResponse,
|
||||
ValidatorRequest, ValidatorResponse,
|
||||
};
|
||||
use std::sync::Arc;
|
||||
use store::Store;
|
||||
use types::{
|
||||
AttesterSlashing, BeaconState, CommitteeIndex, EthSpec, Hash256, ProposerSlashing,
|
||||
PublicKeyBytes, RelativeEpoch, SignedBeaconBlock, Slot, Validator,
|
||||
AttesterSlashing, BeaconState, EthSpec, Hash256, ProposerSlashing, PublicKeyBytes,
|
||||
RelativeEpoch, Slot,
|
||||
};
|
||||
|
||||
/// Information about the block and state that are at head of the beacon chain.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
|
||||
pub struct CanonicalHeadResponse {
|
||||
pub slot: Slot,
|
||||
pub block_root: Hash256,
|
||||
pub state_root: Hash256,
|
||||
pub finalized_slot: Slot,
|
||||
pub finalized_block_root: Hash256,
|
||||
pub justified_slot: Slot,
|
||||
pub justified_block_root: Hash256,
|
||||
pub previous_justified_slot: Slot,
|
||||
pub previous_justified_block_root: Hash256,
|
||||
}
|
||||
|
||||
/// HTTP handler to return a `BeaconBlock` at a given `root` or `slot`.
|
||||
pub fn get_head<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
@ -62,15 +50,7 @@ pub fn get_head<T: BeaconChainTypes>(
|
||||
ResponseBuilder::new(&req)?.body(&head)
|
||||
}
|
||||
|
||||
/// Information about a block that is at the head of a chain. May or may not represent the
|
||||
/// canonical head.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
|
||||
pub struct HeadBeaconBlock {
|
||||
pub beacon_block_root: Hash256,
|
||||
pub beacon_block_slot: Slot,
|
||||
}
|
||||
|
||||
/// HTTP handler to return a list of head block roots.
|
||||
/// HTTP handler to return a list of head BeaconBlocks.
|
||||
pub fn get_heads<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
@ -87,14 +67,7 @@ pub fn get_heads<T: BeaconChainTypes>(
|
||||
ResponseBuilder::new(&req)?.body(&heads)
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
|
||||
#[serde(bound = "T: EthSpec")]
|
||||
pub struct BlockResponse<T: EthSpec> {
|
||||
pub root: Hash256,
|
||||
pub beacon_block: SignedBeaconBlock<T>,
|
||||
}
|
||||
|
||||
/// HTTP handler to return a `SignedBeaconBlock` at a given `root` or `slot`.
|
||||
/// HTTP handler to return a `BeaconBlock` at a given `root` or `slot`.
|
||||
pub fn get_block<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
@ -158,14 +131,6 @@ pub fn get_fork<T: BeaconChainTypes>(
|
||||
ResponseBuilder::new(&req)?.body(&beacon_chain.head()?.beacon_state.fork)
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
|
||||
pub struct ValidatorResponse {
|
||||
pub pubkey: PublicKeyBytes,
|
||||
pub validator_index: Option<usize>,
|
||||
pub balance: Option<u64>,
|
||||
pub validator: Option<Validator>,
|
||||
}
|
||||
|
||||
/// HTTP handler to which accepts a query string of a list of validator pubkeys and maps it to a
|
||||
/// `ValidatorResponse`.
|
||||
///
|
||||
@ -246,13 +211,6 @@ pub fn get_active_validators<T: BeaconChainTypes>(
|
||||
ResponseBuilder::new(&req)?.body(&validators)
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
|
||||
pub struct ValidatorRequest {
|
||||
/// If set to `None`, uses the canonical head state.
|
||||
pub state_root: Option<Hash256>,
|
||||
pub pubkeys: Vec<PublicKeyBytes>,
|
||||
}
|
||||
|
||||
/// HTTP handler to which accepts a `ValidatorRequest` and returns a `ValidatorResponse` for
|
||||
/// each of the given `pubkeys`. When `state_root` is `None`, the canonical head is used.
|
||||
///
|
||||
@ -365,13 +323,6 @@ fn validator_response_by_pubkey<E: EthSpec>(
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
|
||||
pub struct Committee {
|
||||
pub slot: Slot,
|
||||
pub index: CommitteeIndex,
|
||||
pub committee: Vec<usize>,
|
||||
}
|
||||
|
||||
/// HTTP handler
|
||||
pub fn get_committees<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
@ -405,13 +356,6 @@ pub fn get_committees<T: BeaconChainTypes>(
|
||||
ResponseBuilder::new(&req)?.body(&committees)
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
|
||||
#[serde(bound = "T: EthSpec")]
|
||||
pub struct StateResponse<T: EthSpec> {
|
||||
pub root: Hash256,
|
||||
pub beacon_state: BeaconState<T>,
|
||||
}
|
||||
|
||||
/// HTTP handler to return a `BeaconState` at a given `root` or `slot`.
|
||||
///
|
||||
/// Will not return a state if the request slot is in the future. Will return states higher than
|
||||
|
||||
@ -1,20 +1,17 @@
|
||||
use crate::{ApiError, ApiResult};
|
||||
use crate::{ApiError, ApiResult, NetworkChannel};
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes, StateSkipConfig};
|
||||
use bls::PublicKeyBytes;
|
||||
use eth2_libp2p::GossipTopic;
|
||||
use eth2_libp2p::PubsubMessage;
|
||||
use eth2_libp2p::types::GossipEncoding;
|
||||
use eth2_libp2p::{PubsubData, PubsubMessage};
|
||||
use hex;
|
||||
use http::header;
|
||||
use hyper::{Body, Request};
|
||||
use network::NetworkMessage;
|
||||
use parking_lot::RwLock;
|
||||
use ssz::{Decode, Encode};
|
||||
use std::sync::Arc;
|
||||
use ssz::Decode;
|
||||
use store::{iter::AncestorIter, Store};
|
||||
use tokio::sync::mpsc;
|
||||
use types::{
|
||||
Attestation, BeaconState, CommitteeIndex, Epoch, EthSpec, Hash256, RelativeEpoch, Signature,
|
||||
SignedBeaconBlock, Slot,
|
||||
SignedAggregateAndProof, SignedBeaconBlock, Slot,
|
||||
};
|
||||
|
||||
/// Parse a slot.
|
||||
@ -49,7 +46,7 @@ pub fn parse_committee_index(string: &str) -> Result<CommitteeIndex, ApiError> {
|
||||
/// Checks the provided request to ensure that the `content-type` header.
|
||||
///
|
||||
/// The content-type header should either be omitted, in which case JSON is assumed, or it should
|
||||
/// explicity specify `application/json`. If anything else is provided, an error is returned.
|
||||
/// explicitly specify `application/json`. If anything else is provided, an error is returned.
|
||||
pub fn check_content_type_for_json(req: &Request<Body>) -> Result<(), ApiError> {
|
||||
match req.headers().get(header::CONTENT_TYPE) {
|
||||
Some(h) if h == "application/json" => Ok(()),
|
||||
@ -61,7 +58,7 @@ pub fn check_content_type_for_json(req: &Request<Body>) -> Result<(), ApiError>
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse a signature from a `0x` preixed string.
|
||||
/// Parse a signature from a `0x` prefixed string.
|
||||
pub fn parse_signature(string: &str) -> Result<Signature, ApiError> {
|
||||
const PREFIX: &str = "0x";
|
||||
|
||||
@ -78,7 +75,7 @@ pub fn parse_signature(string: &str) -> Result<Signature, ApiError> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse a root from a `0x` preixed string.
|
||||
/// Parse a root from a `0x` prefixed string.
|
||||
///
|
||||
/// E.g., `"0x0000000000000000000000000000000000000000000000000000000000000000"`
|
||||
pub fn parse_root(string: &str) -> Result<Hash256, ApiError> {
|
||||
@ -232,18 +229,17 @@ pub fn implementation_pending_response(_req: Request<Body>) -> ApiResult {
|
||||
}
|
||||
|
||||
pub fn publish_beacon_block_to_network<T: BeaconChainTypes + 'static>(
|
||||
chan: Arc<RwLock<mpsc::UnboundedSender<NetworkMessage>>>,
|
||||
mut chan: NetworkChannel<T::EthSpec>,
|
||||
block: SignedBeaconBlock<T::EthSpec>,
|
||||
) -> Result<(), ApiError> {
|
||||
// create the network topic to send on
|
||||
let topic = GossipTopic::BeaconBlock;
|
||||
let message = PubsubMessage::Block(block.as_ssz_bytes());
|
||||
// send the block via SSZ encoding
|
||||
let messages = vec![PubsubMessage::new(
|
||||
GossipEncoding::SSZ,
|
||||
PubsubData::BeaconBlock(Box::new(block)),
|
||||
)];
|
||||
|
||||
// Publish the block to the p2p network via gossipsub.
|
||||
if let Err(e) = chan.write().try_send(NetworkMessage::Publish {
|
||||
topics: vec![topic.into()],
|
||||
message,
|
||||
}) {
|
||||
if let Err(e) = chan.try_send(NetworkMessage::Publish { messages }) {
|
||||
return Err(ApiError::ServerError(format!(
|
||||
"Unable to send new block to network: {:?}",
|
||||
e
|
||||
@ -253,19 +249,51 @@ pub fn publish_beacon_block_to_network<T: BeaconChainTypes + 'static>(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn publish_attestation_to_network<T: BeaconChainTypes + 'static>(
|
||||
chan: Arc<RwLock<mpsc::UnboundedSender<NetworkMessage>>>,
|
||||
attestation: Attestation<T::EthSpec>,
|
||||
/// Publishes a raw un-aggregated attestation to the network.
|
||||
pub fn publish_raw_attestations_to_network<T: BeaconChainTypes + 'static>(
|
||||
mut chan: NetworkChannel<T::EthSpec>,
|
||||
attestations: Vec<Attestation<T::EthSpec>>,
|
||||
) -> Result<(), ApiError> {
|
||||
// create the network topic to send on
|
||||
let topic = GossipTopic::BeaconAttestation;
|
||||
let message = PubsubMessage::Attestation(attestation.as_ssz_bytes());
|
||||
let messages = attestations
|
||||
.into_iter()
|
||||
.map(|attestation| {
|
||||
// create the gossip message to send to the network
|
||||
let subnet_id = attestation.subnet_id();
|
||||
PubsubMessage::new(
|
||||
GossipEncoding::SSZ,
|
||||
PubsubData::Attestation(Box::new((subnet_id, attestation))),
|
||||
)
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
// Publish the attestation to the p2p network via gossipsub.
|
||||
if let Err(e) = chan.write().try_send(NetworkMessage::Publish {
|
||||
topics: vec![topic.into()],
|
||||
message,
|
||||
}) {
|
||||
// Publish the attestations to the p2p network via gossipsub.
|
||||
if let Err(e) = chan.try_send(NetworkMessage::Publish { messages }) {
|
||||
return Err(ApiError::ServerError(format!(
|
||||
"Unable to send new attestation to network: {:?}",
|
||||
e
|
||||
)));
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Publishes an aggregated attestation to the network.
|
||||
pub fn publish_aggregate_attestations_to_network<T: BeaconChainTypes + 'static>(
|
||||
mut chan: NetworkChannel<T::EthSpec>,
|
||||
signed_proofs: Vec<SignedAggregateAndProof<T::EthSpec>>,
|
||||
) -> Result<(), ApiError> {
|
||||
let messages = signed_proofs
|
||||
.into_iter()
|
||||
.map(|signed_proof| {
|
||||
PubsubMessage::new(
|
||||
GossipEncoding::SSZ,
|
||||
PubsubData::AggregateAndProofAttestation(Box::new(signed_proof)),
|
||||
)
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
// Publish the attestations to the p2p network via gossipsub.
|
||||
if let Err(e) = chan.try_send(NetworkMessage::Publish { messages }) {
|
||||
return Err(ApiError::ServerError(format!(
|
||||
"Unable to send new attestation to network: {:?}",
|
||||
e
|
||||
|
||||
@ -21,38 +21,32 @@ mod validator;
|
||||
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use client_network::NetworkMessage;
|
||||
use client_network::Service as NetworkService;
|
||||
pub use config::ApiEncodingFormat;
|
||||
use error::{ApiError, ApiResult};
|
||||
use eth2_config::Eth2Config;
|
||||
use eth2_libp2p::NetworkGlobals;
|
||||
use hyper::rt::Future;
|
||||
use hyper::server::conn::AddrStream;
|
||||
use hyper::service::{make_service_fn, service_fn};
|
||||
use hyper::{Body, Request, Response, Server};
|
||||
use parking_lot::RwLock;
|
||||
use slog::{info, warn};
|
||||
use std::net::SocketAddr;
|
||||
use std::ops::Deref;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
use tokio::runtime::TaskExecutor;
|
||||
use tokio::sync::mpsc;
|
||||
use tokio::sync::{mpsc, oneshot};
|
||||
use url_query::UrlQuery;
|
||||
|
||||
pub use crate::helpers::parse_pubkey_bytes;
|
||||
pub use beacon::{
|
||||
BlockResponse, CanonicalHeadResponse, Committee, HeadBeaconBlock, StateResponse,
|
||||
ValidatorRequest, ValidatorResponse,
|
||||
};
|
||||
pub use config::Config;
|
||||
pub use validator::{ValidatorDutiesRequest, ValidatorDuty};
|
||||
|
||||
pub type BoxFut = Box<dyn Future<Item = Response<Body>, Error = ApiError> + Send>;
|
||||
pub type NetworkChannel = Arc<RwLock<mpsc::UnboundedSender<NetworkMessage>>>;
|
||||
pub type NetworkChannel<T> = mpsc::UnboundedSender<NetworkMessage<T>>;
|
||||
|
||||
pub struct NetworkInfo<T: BeaconChainTypes> {
|
||||
pub network_service: Arc<NetworkService<T>>,
|
||||
pub network_chan: mpsc::UnboundedSender<NetworkMessage>,
|
||||
pub network_globals: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
pub network_chan: NetworkChannel<T::EthSpec>,
|
||||
}
|
||||
|
||||
// Allowing more than 7 arguments.
|
||||
@ -66,7 +60,7 @@ pub fn start_server<T: BeaconChainTypes>(
|
||||
freezer_db_path: PathBuf,
|
||||
eth2_config: Eth2Config,
|
||||
log: slog::Logger,
|
||||
) -> Result<(exit_future::Signal, SocketAddr), hyper::Error> {
|
||||
) -> Result<(oneshot::Sender<()>, SocketAddr), hyper::Error> {
|
||||
let inner_log = log.clone();
|
||||
let eth2_config = Arc::new(eth2_config);
|
||||
|
||||
@ -75,8 +69,8 @@ pub fn start_server<T: BeaconChainTypes>(
|
||||
let beacon_chain = beacon_chain.clone();
|
||||
let log = inner_log.clone();
|
||||
let eth2_config = eth2_config.clone();
|
||||
let network_service = network_info.network_service.clone();
|
||||
let network_channel = Arc::new(RwLock::new(network_info.network_chan.clone()));
|
||||
let network_globals = network_info.network_globals.clone();
|
||||
let network_channel = network_info.network_chan.clone();
|
||||
let db_path = db_path.clone();
|
||||
let freezer_db_path = freezer_db_path.clone();
|
||||
|
||||
@ -84,7 +78,7 @@ pub fn start_server<T: BeaconChainTypes>(
|
||||
router::route(
|
||||
req,
|
||||
beacon_chain.clone(),
|
||||
network_service.clone(),
|
||||
network_globals.clone(),
|
||||
network_channel.clone(),
|
||||
eth2_config.clone(),
|
||||
log.clone(),
|
||||
@ -104,7 +98,7 @@ pub fn start_server<T: BeaconChainTypes>(
|
||||
let actual_listen_addr = server.local_addr();
|
||||
|
||||
// Build a channel to kill the HTTP server.
|
||||
let (exit_signal, exit) = exit_future::signal();
|
||||
let (exit_signal, exit) = oneshot::channel();
|
||||
let inner_log = log.clone();
|
||||
let server_exit = exit.and_then(move |_| {
|
||||
info!(inner_log, "HTTP service shutdown");
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
use crate::error::ApiResult;
|
||||
use crate::response_builder::ResponseBuilder;
|
||||
use crate::NetworkService;
|
||||
use crate::NetworkGlobals;
|
||||
use beacon_chain::BeaconChainTypes;
|
||||
use eth2_libp2p::{Multiaddr, PeerId};
|
||||
use hyper::{Body, Request};
|
||||
@ -11,7 +11,7 @@ use std::sync::Arc;
|
||||
/// Returns a list of `Multiaddr`, serialized according to their `serde` impl.
|
||||
pub fn get_listen_addresses<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
network: Arc<NetworkService<T>>,
|
||||
network: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
) -> ApiResult {
|
||||
let multiaddresses: Vec<Multiaddr> = network.listen_multiaddrs();
|
||||
ResponseBuilder::new(&req)?.body_no_ssz(&multiaddresses)
|
||||
@ -22,9 +22,9 @@ pub fn get_listen_addresses<T: BeaconChainTypes>(
|
||||
/// Returns the TCP port number in its plain form (which is also valid JSON serialization)
|
||||
pub fn get_listen_port<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
network: Arc<NetworkService<T>>,
|
||||
network: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
) -> ApiResult {
|
||||
ResponseBuilder::new(&req)?.body(&network.listen_port())
|
||||
ResponseBuilder::new(&req)?.body(&network.listen_port_tcp())
|
||||
}
|
||||
|
||||
/// HTTP handler to return the Discv5 ENR from the client's libp2p service.
|
||||
@ -32,7 +32,7 @@ pub fn get_listen_port<T: BeaconChainTypes>(
|
||||
/// ENR is encoded as base64 string.
|
||||
pub fn get_enr<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
network: Arc<NetworkService<T>>,
|
||||
network: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
) -> ApiResult {
|
||||
ResponseBuilder::new(&req)?.body_no_ssz(
|
||||
&network
|
||||
@ -47,7 +47,7 @@ pub fn get_enr<T: BeaconChainTypes>(
|
||||
/// PeerId is encoded as base58 string.
|
||||
pub fn get_peer_id<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
network: Arc<NetworkService<T>>,
|
||||
network: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
) -> ApiResult {
|
||||
ResponseBuilder::new(&req)?.body_no_ssz(&network.local_peer_id().to_base58())
|
||||
}
|
||||
@ -55,7 +55,7 @@ pub fn get_peer_id<T: BeaconChainTypes>(
|
||||
/// HTTP handler to return the number of peers connected in the client's libp2p service.
|
||||
pub fn get_peer_count<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
network: Arc<NetworkService<T>>,
|
||||
network: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
) -> ApiResult {
|
||||
ResponseBuilder::new(&req)?.body(&network.connected_peers())
|
||||
}
|
||||
@ -65,11 +65,12 @@ pub fn get_peer_count<T: BeaconChainTypes>(
|
||||
/// Peers are presented as a list of `PeerId::to_string()`.
|
||||
pub fn get_peer_list<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
network: Arc<NetworkService<T>>,
|
||||
network: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
) -> ApiResult {
|
||||
let connected_peers: Vec<String> = network
|
||||
.connected_peer_set()
|
||||
.iter()
|
||||
.connected_peer_set
|
||||
.read()
|
||||
.keys()
|
||||
.map(PeerId::to_string)
|
||||
.collect();
|
||||
ResponseBuilder::new(&req)?.body_no_ssz(&connected_peers)
|
||||
|
||||
@ -3,8 +3,8 @@ use crate::{
|
||||
BoxFut, NetworkChannel,
|
||||
};
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use client_network::Service as NetworkService;
|
||||
use eth2_config::Eth2Config;
|
||||
use eth2_libp2p::NetworkGlobals;
|
||||
use futures::{Future, IntoFuture};
|
||||
use hyper::{Body, Error, Method, Request, Response};
|
||||
use slog::debug;
|
||||
@ -25,8 +25,8 @@ where
|
||||
pub fn route<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network_service: Arc<NetworkService<T>>,
|
||||
network_channel: NetworkChannel,
|
||||
network_globals: Arc<NetworkGlobals<T::EthSpec>>,
|
||||
network_channel: NetworkChannel<T::EthSpec>,
|
||||
eth2_config: Arc<Eth2Config>,
|
||||
local_log: slog::Logger,
|
||||
db_path: PathBuf,
|
||||
@ -49,22 +49,22 @@ pub fn route<T: BeaconChainTypes>(
|
||||
|
||||
// Methods for Network
|
||||
(&Method::GET, "/network/enr") => {
|
||||
into_boxfut(network::get_enr::<T>(req, network_service))
|
||||
into_boxfut(network::get_enr::<T>(req, network_globals))
|
||||
}
|
||||
(&Method::GET, "/network/peer_count") => {
|
||||
into_boxfut(network::get_peer_count::<T>(req, network_service))
|
||||
into_boxfut(network::get_peer_count::<T>(req, network_globals))
|
||||
}
|
||||
(&Method::GET, "/network/peer_id") => {
|
||||
into_boxfut(network::get_peer_id::<T>(req, network_service))
|
||||
into_boxfut(network::get_peer_id::<T>(req, network_globals))
|
||||
}
|
||||
(&Method::GET, "/network/peers") => {
|
||||
into_boxfut(network::get_peer_list::<T>(req, network_service))
|
||||
into_boxfut(network::get_peer_list::<T>(req, network_globals))
|
||||
}
|
||||
(&Method::GET, "/network/listen_port") => {
|
||||
into_boxfut(network::get_listen_port::<T>(req, network_service))
|
||||
into_boxfut(network::get_listen_port::<T>(req, network_globals))
|
||||
}
|
||||
(&Method::GET, "/network/listen_addresses") => {
|
||||
into_boxfut(network::get_listen_addresses::<T>(req, network_service))
|
||||
into_boxfut(network::get_listen_addresses::<T>(req, network_globals))
|
||||
}
|
||||
|
||||
// Methods for Beacon Node
|
||||
@ -121,6 +121,14 @@ pub fn route<T: BeaconChainTypes>(
|
||||
drop(timer);
|
||||
into_boxfut(response)
|
||||
}
|
||||
(&Method::POST, "/validator/subscribe") => {
|
||||
validator::post_validator_subscriptions::<T>(
|
||||
req,
|
||||
beacon_chain,
|
||||
network_channel,
|
||||
log,
|
||||
)
|
||||
}
|
||||
(&Method::GET, "/validator/duties/all") => {
|
||||
into_boxfut(validator::get_all_validator_duties::<T>(req, beacon_chain))
|
||||
}
|
||||
@ -144,10 +152,22 @@ pub fn route<T: BeaconChainTypes>(
|
||||
drop(timer);
|
||||
into_boxfut(response)
|
||||
}
|
||||
(&Method::POST, "/validator/attestation") => {
|
||||
validator::publish_attestation::<T>(req, beacon_chain, network_channel, log)
|
||||
(&Method::GET, "/validator/aggregate_attestation") => {
|
||||
into_boxfut(validator::get_aggregate_attestation::<T>(req, beacon_chain))
|
||||
}
|
||||
(&Method::POST, "/validator/attestations") => {
|
||||
validator::publish_attestations::<T>(req, beacon_chain, network_channel, log)
|
||||
}
|
||||
(&Method::POST, "/validator/aggregate_and_proofs") => {
|
||||
validator::publish_aggregate_and_proofs::<T>(
|
||||
req,
|
||||
beacon_chain,
|
||||
network_channel,
|
||||
log,
|
||||
)
|
||||
}
|
||||
|
||||
// Methods for consensus
|
||||
(&Method::GET, "/consensus/global_votes") => {
|
||||
into_boxfut(consensus::get_vote_count::<T>(req, beacon_chain))
|
||||
}
|
||||
|
||||
@ -1,47 +1,27 @@
|
||||
use crate::helpers::{
|
||||
check_content_type_for_json, publish_attestation_to_network, publish_beacon_block_to_network,
|
||||
check_content_type_for_json, publish_aggregate_attestations_to_network,
|
||||
publish_beacon_block_to_network, publish_raw_attestations_to_network,
|
||||
};
|
||||
use crate::response_builder::ResponseBuilder;
|
||||
use crate::{ApiError, ApiResult, BoxFut, NetworkChannel, UrlQuery};
|
||||
use beacon_chain::{
|
||||
AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, BlockProcessingOutcome,
|
||||
StateSkipConfig,
|
||||
AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, BlockError, StateSkipConfig,
|
||||
};
|
||||
use bls::PublicKeyBytes;
|
||||
use futures::{Future, Stream};
|
||||
use hyper::{Body, Request};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use network::NetworkMessage;
|
||||
use rayon::prelude::*;
|
||||
use rest_types::{ValidatorDutiesRequest, ValidatorDutyBytes, ValidatorSubscription};
|
||||
use slog::{error, info, warn, Logger};
|
||||
use ssz_derive::{Decode, Encode};
|
||||
use std::sync::Arc;
|
||||
use types::beacon_state::EthSpec;
|
||||
use types::{
|
||||
Attestation, BeaconState, CommitteeIndex, Epoch, RelativeEpoch, SignedBeaconBlock, Slot,
|
||||
Attestation, BeaconState, Epoch, RelativeEpoch, SignedAggregateAndProof, SignedBeaconBlock,
|
||||
Slot,
|
||||
};
|
||||
|
||||
#[derive(PartialEq, Debug, Serialize, Deserialize, Clone)]
|
||||
pub struct ValidatorDuty {
|
||||
/// The validator's BLS public key, uniquely identifying them. _48-bytes, hex encoded with 0x prefix, case insensitive._
|
||||
pub validator_pubkey: PublicKeyBytes,
|
||||
/// The validator's index in `state.validators`
|
||||
pub validator_index: Option<usize>,
|
||||
/// The slot at which the validator must attest.
|
||||
pub attestation_slot: Option<Slot>,
|
||||
/// The index of the committee within `slot` of which the validator is a member.
|
||||
pub attestation_committee_index: Option<CommitteeIndex>,
|
||||
/// The position of the validator in the committee.
|
||||
pub attestation_committee_position: Option<usize>,
|
||||
/// The slots in which a validator must propose a block (can be empty).
|
||||
pub block_proposal_slots: Vec<Slot>,
|
||||
}
|
||||
|
||||
#[derive(PartialEq, Debug, Serialize, Deserialize, Clone, Encode, Decode)]
|
||||
pub struct ValidatorDutiesRequest {
|
||||
pub epoch: Epoch,
|
||||
pub pubkeys: Vec<PublicKeyBytes>,
|
||||
}
|
||||
|
||||
/// HTTP Handler to retrieve a the duties for a set of validators during a particular epoch. This
|
||||
/// HTTP Handler to retrieve the duties for a set of validators during a particular epoch. This
|
||||
/// method allows for collecting bulk sets of validator duties without risking exceeding the max
|
||||
/// URL length with query pairs.
|
||||
pub fn post_validator_duties<T: BeaconChainTypes>(
|
||||
@ -74,6 +54,79 @@ pub fn post_validator_duties<T: BeaconChainTypes>(
|
||||
Box::new(future)
|
||||
}
|
||||
|
||||
/// HTTP Handler to retrieve subscriptions for a set of validators. This allows the node to
|
||||
/// organise peer discovery and topic subscription for known validators.
|
||||
pub fn post_validator_subscriptions<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
mut network_chan: NetworkChannel<T::EthSpec>,
|
||||
log: Logger,
|
||||
) -> BoxFut {
|
||||
try_future!(check_content_type_for_json(&req));
|
||||
let response_builder = ResponseBuilder::new(&req);
|
||||
|
||||
let body = req.into_body();
|
||||
Box::new(
|
||||
body.concat2()
|
||||
.map_err(|e| ApiError::ServerError(format!("Unable to get request body: {:?}", e)))
|
||||
.and_then(|chunks| {
|
||||
serde_json::from_slice(&chunks).map_err(|e| {
|
||||
ApiError::BadRequest(format!(
|
||||
"Unable to parse JSON into ValidatorSubscriptions: {:?}",
|
||||
e
|
||||
))
|
||||
})
|
||||
})
|
||||
.and_then(move |subscriptions: Vec<ValidatorSubscription>| {
|
||||
let fork = beacon_chain
|
||||
.wall_clock_state()
|
||||
.map(|state| state.fork.clone())
|
||||
.map_err(|e| {
|
||||
error!(log, "Unable to get current beacon state");
|
||||
ApiError::ServerError(format!("Error getting current beacon state {:?}", e))
|
||||
})?;
|
||||
|
||||
// verify the signatures in parallel
|
||||
subscriptions.par_iter().try_for_each(|subscription| {
|
||||
if let Some(pubkey) =
|
||||
&beacon_chain.validator_pubkey(subscription.validator_index as usize)?
|
||||
{
|
||||
if subscription.verify(
|
||||
pubkey,
|
||||
&beacon_chain.spec,
|
||||
&fork,
|
||||
T::EthSpec::slots_per_epoch(),
|
||||
) {
|
||||
Ok(())
|
||||
} else {
|
||||
error!(log, "HTTP RPC sent invalid signatures");
|
||||
Err(ApiError::ProcessingError(format!(
|
||||
"Could not verify signatures"
|
||||
)))
|
||||
}
|
||||
} else {
|
||||
error!(log, "HTTP RPC sent unknown validator");
|
||||
Err(ApiError::ProcessingError(format!(
|
||||
"Could not verify signatures"
|
||||
)))
|
||||
}
|
||||
})?;
|
||||
|
||||
// subscriptions are verified, send them to the network thread
|
||||
network_chan
|
||||
.try_send(NetworkMessage::Subscribe { subscriptions })
|
||||
.map_err(|e| {
|
||||
ApiError::ServerError(format!(
|
||||
"Unable to subscriptions to the network: {:?}",
|
||||
e
|
||||
))
|
||||
})?;
|
||||
Ok(())
|
||||
})
|
||||
.and_then(|_| response_builder?.body_no_ssz(&())),
|
||||
)
|
||||
}
|
||||
|
||||
/// HTTP Handler to retrieve all validator duties for the given epoch.
|
||||
pub fn get_all_validator_duties<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
@ -154,7 +207,7 @@ fn return_validator_duties<T: BeaconChainTypes>(
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
epoch: Epoch,
|
||||
validator_pubkeys: Vec<PublicKeyBytes>,
|
||||
) -> Result<Vec<ValidatorDuty>, ApiError> {
|
||||
) -> Result<Vec<ValidatorDutyBytes>, ApiError> {
|
||||
let mut state = get_state_for_epoch(&beacon_chain, epoch, StateSkipConfig::WithoutStateRoots)?;
|
||||
|
||||
let relative_epoch = RelativeEpoch::from_epoch(state.current_epoch(), epoch)
|
||||
@ -189,11 +242,24 @@ fn return_validator_duties<T: BeaconChainTypes>(
|
||||
validator_pubkeys
|
||||
.into_iter()
|
||||
.map(|validator_pubkey| {
|
||||
if let Some(validator_index) =
|
||||
state.get_validator_index(&validator_pubkey).map_err(|e| {
|
||||
ApiError::ServerError(format!("Unable to read pubkey cache: {:?}", e))
|
||||
})?
|
||||
{
|
||||
// The `beacon_chain` can return a validator index that does not exist in all states.
|
||||
// Therefore, we must check to ensure that the validator index is valid for our
|
||||
// `state`.
|
||||
let validator_index = if let Some(i) = beacon_chain
|
||||
.validator_index(&validator_pubkey)
|
||||
.map_err(|e| {
|
||||
ApiError::ServerError(format!("Unable to get validator index: {:?}", e))
|
||||
})? {
|
||||
if i < state.validators.len() {
|
||||
Some(i)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
if let Some(validator_index) = validator_index {
|
||||
let duties = state
|
||||
.get_attestation_duties(validator_index, relative_epoch)
|
||||
.map_err(|e| {
|
||||
@ -203,28 +269,39 @@ fn return_validator_duties<T: BeaconChainTypes>(
|
||||
))
|
||||
})?;
|
||||
|
||||
// Obtain the aggregator modulo
|
||||
let aggregator_modulo = duties.map(|d| {
|
||||
std::cmp::max(
|
||||
1,
|
||||
d.committee_len as u64
|
||||
/ &beacon_chain.spec.target_aggregators_per_committee,
|
||||
)
|
||||
});
|
||||
|
||||
let block_proposal_slots = validator_proposers
|
||||
.iter()
|
||||
.filter(|(i, _slot)| validator_index == *i)
|
||||
.map(|(_i, slot)| *slot)
|
||||
.collect();
|
||||
|
||||
Ok(ValidatorDuty {
|
||||
Ok(ValidatorDutyBytes {
|
||||
validator_pubkey,
|
||||
validator_index: Some(validator_index),
|
||||
validator_index: Some(validator_index as u64),
|
||||
attestation_slot: duties.map(|d| d.slot),
|
||||
attestation_committee_index: duties.map(|d| d.index),
|
||||
attestation_committee_position: duties.map(|d| d.committee_position),
|
||||
block_proposal_slots,
|
||||
aggregator_modulo,
|
||||
})
|
||||
} else {
|
||||
Ok(ValidatorDuty {
|
||||
Ok(ValidatorDutyBytes {
|
||||
validator_pubkey,
|
||||
validator_index: None,
|
||||
attestation_slot: None,
|
||||
attestation_committee_index: None,
|
||||
attestation_committee_position: None,
|
||||
block_proposal_slots: vec![],
|
||||
aggregator_modulo: None,
|
||||
})
|
||||
}
|
||||
})
|
||||
@ -264,7 +341,7 @@ pub fn get_new_beacon_block<T: BeaconChainTypes>(
|
||||
pub fn publish_beacon_block<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network_chan: NetworkChannel,
|
||||
network_chan: NetworkChannel<T::EthSpec>,
|
||||
log: Logger,
|
||||
) -> BoxFut {
|
||||
try_future!(check_content_type_for_json(&req));
|
||||
@ -282,7 +359,7 @@ pub fn publish_beacon_block<T: BeaconChainTypes>(
|
||||
.and_then(move |block: SignedBeaconBlock<T::EthSpec>| {
|
||||
let slot = block.slot();
|
||||
match beacon_chain.process_block(block.clone()) {
|
||||
Ok(BlockProcessingOutcome::Processed { block_root }) => {
|
||||
Ok(block_root) => {
|
||||
// Block was processed, publish via gossipsub
|
||||
info!(
|
||||
log,
|
||||
@ -325,19 +402,7 @@ pub fn publish_beacon_block<T: BeaconChainTypes>(
|
||||
|
||||
Ok(())
|
||||
}
|
||||
Ok(outcome) => {
|
||||
warn!(
|
||||
log,
|
||||
"Invalid block from local validator";
|
||||
"outcome" => format!("{:?}", outcome)
|
||||
);
|
||||
|
||||
Err(ApiError::ProcessingError(format!(
|
||||
"The SignedBeaconBlock could not be processed and has not been published: {:?}",
|
||||
outcome
|
||||
)))
|
||||
}
|
||||
Err(e) => {
|
||||
Err(BlockError::BeaconChainError(e)) => {
|
||||
error!(
|
||||
log,
|
||||
"Error whilst processing block";
|
||||
@ -349,6 +414,18 @@ pub fn publish_beacon_block<T: BeaconChainTypes>(
|
||||
e
|
||||
)))
|
||||
}
|
||||
Err(other) => {
|
||||
warn!(
|
||||
log,
|
||||
"Invalid block from local validator";
|
||||
"outcome" => format!("{:?}", other)
|
||||
);
|
||||
|
||||
Err(ApiError::ProcessingError(format!(
|
||||
"The SignedBeaconBlock could not be processed and has not been published: {:?}",
|
||||
other
|
||||
)))
|
||||
}
|
||||
}
|
||||
})
|
||||
.and_then(|_| response_builder?.body_no_ssz(&()))
|
||||
@ -372,11 +449,28 @@ pub fn get_new_attestation<T: BeaconChainTypes>(
|
||||
ResponseBuilder::new(&req)?.body(&attestation)
|
||||
}
|
||||
|
||||
/// HTTP Handler to publish an Attestation, which has been signed by a validator.
|
||||
pub fn publish_attestation<T: BeaconChainTypes>(
|
||||
/// HTTP Handler to retrieve the aggregate attestation for a slot
|
||||
pub fn get_aggregate_attestation<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network_chan: NetworkChannel,
|
||||
) -> ApiResult {
|
||||
let query = UrlQuery::from_request(&req)?;
|
||||
|
||||
let slot = query.slot()?;
|
||||
let index = query.committee_index()?;
|
||||
|
||||
let aggregate_attestation = beacon_chain
|
||||
.return_aggregate_attestation(slot, index)
|
||||
.map_err(|e| ApiError::BadRequest(format!("Unable to produce attestation: {:?}", e)))?;
|
||||
|
||||
ResponseBuilder::new(&req)?.body(&aggregate_attestation)
|
||||
}
|
||||
|
||||
/// HTTP Handler to publish a list of Attestations, which have been signed by a number of validators.
|
||||
pub fn publish_attestations<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network_chan: NetworkChannel<T::EthSpec>,
|
||||
log: Logger,
|
||||
) -> BoxFut {
|
||||
try_future!(check_content_type_for_json(&req));
|
||||
@ -390,13 +484,20 @@ pub fn publish_attestation<T: BeaconChainTypes>(
|
||||
.and_then(|chunks| {
|
||||
serde_json::from_slice(&chunks.as_slice()).map_err(|e| {
|
||||
ApiError::BadRequest(format!(
|
||||
"Unable to deserialize JSON into a SignedBeaconBlock: {:?}",
|
||||
"Unable to deserialize JSON into a list of attestations: {:?}",
|
||||
e
|
||||
))
|
||||
})
|
||||
})
|
||||
.and_then(move |attestation: Attestation<T::EthSpec>| {
|
||||
match beacon_chain.process_attestation(attestation.clone()) {
|
||||
.and_then(move |attestations: Vec<Attestation<T::EthSpec>>| {
|
||||
// Note: This is a new attestation from a validator. We want to process this and
|
||||
// inform the validator whether the attestation was valid. In doing so, we store
|
||||
// this un-aggregated raw attestation in the op_pool by default. This is
|
||||
// sub-optimal as if we have no validators needing to aggregate, these don't need
|
||||
// to be stored in the op-pool. This is minimal however as the op_pool gets pruned
|
||||
// every slot
|
||||
attestations.par_iter().try_for_each(|attestation| {
|
||||
match beacon_chain.process_attestation(attestation.clone(), Some(true)) {
|
||||
Ok(AttestationProcessingOutcome::Processed) => {
|
||||
// Block was processed, publish via gossipsub
|
||||
info!(
|
||||
@ -407,7 +508,99 @@ pub fn publish_attestation<T: BeaconChainTypes>(
|
||||
"index" => attestation.data.index,
|
||||
"slot" => attestation.data.slot,
|
||||
);
|
||||
publish_attestation_to_network::<T>(network_chan, attestation)
|
||||
Ok(())
|
||||
}
|
||||
Ok(outcome) => {
|
||||
warn!(
|
||||
log,
|
||||
"Invalid attestation from local validator";
|
||||
"outcome" => format!("{:?}", outcome)
|
||||
);
|
||||
|
||||
Err(ApiError::ProcessingError(format!(
|
||||
"An Attestation could not be processed and has not been published: {:?}",
|
||||
outcome
|
||||
)))
|
||||
}
|
||||
Err(e) => {
|
||||
error!(
|
||||
log,
|
||||
"Error whilst processing attestation";
|
||||
"error" => format!("{:?}", e)
|
||||
);
|
||||
|
||||
Err(ApiError::ServerError(format!(
|
||||
"Error while processing attestation: {:?}",
|
||||
e
|
||||
)))
|
||||
}
|
||||
}
|
||||
})?;
|
||||
Ok(attestations)
|
||||
})
|
||||
.and_then(|attestations| {
|
||||
publish_raw_attestations_to_network::<T>(network_chan, attestations)
|
||||
})
|
||||
.and_then(|_| response_builder?.body_no_ssz(&())),
|
||||
)
|
||||
}
|
||||
|
||||
/// HTTP Handler to publish an Attestation, which has been signed by a validator.
|
||||
pub fn publish_aggregate_and_proofs<T: BeaconChainTypes>(
|
||||
req: Request<Body>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
network_chan: NetworkChannel<T::EthSpec>,
|
||||
log: Logger,
|
||||
) -> BoxFut {
|
||||
try_future!(check_content_type_for_json(&req));
|
||||
let response_builder = ResponseBuilder::new(&req);
|
||||
|
||||
Box::new(
|
||||
req.into_body()
|
||||
.concat2()
|
||||
.map_err(|e| ApiError::ServerError(format!("Unable to get request body: {:?}", e)))
|
||||
.map(|chunk| chunk.iter().cloned().collect::<Vec<u8>>())
|
||||
.and_then(|chunks| {
|
||||
serde_json::from_slice(&chunks.as_slice()).map_err(|e| {
|
||||
ApiError::BadRequest(format!(
|
||||
"Unable to deserialize JSON into a list of SignedAggregateAndProof: {:?}",
|
||||
e
|
||||
))
|
||||
})
|
||||
})
|
||||
.and_then(move |signed_proofs: Vec<SignedAggregateAndProof<T::EthSpec>>| {
|
||||
|
||||
// Verify the signatures for the aggregate and proof and if valid process the
|
||||
// aggregate
|
||||
// TODO: Double check speed and logic consistency of handling current fork vs
|
||||
// validator fork for signatures.
|
||||
// TODO: More efficient way of getting a fork?
|
||||
let fork = &beacon_chain.head()?.beacon_state.fork;
|
||||
|
||||
signed_proofs.par_iter().try_for_each(|signed_proof| {
|
||||
let agg_proof = &signed_proof.message;
|
||||
let validator_pubkey = &beacon_chain.validator_pubkey(agg_proof.aggregator_index as usize)?.ok_or_else(|| {
|
||||
warn!(
|
||||
log,
|
||||
"Unknown validator from local validator client";
|
||||
);
|
||||
|
||||
ApiError::ProcessingError(format!("The validator is known"))
|
||||
})?;
|
||||
if signed_proof.is_valid(validator_pubkey, fork) {
|
||||
let attestation = &agg_proof.aggregate;
|
||||
match beacon_chain.process_attestation(attestation.clone(), Some(false)) {
|
||||
Ok(AttestationProcessingOutcome::Processed) => {
|
||||
// Block was processed, publish via gossipsub
|
||||
info!(
|
||||
log,
|
||||
"Attestation from local validator";
|
||||
"target" => attestation.data.source.epoch,
|
||||
"source" => attestation.data.source.epoch,
|
||||
"index" => attestation.data.index,
|
||||
"slot" => attestation.data.slot,
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
Ok(outcome) => {
|
||||
warn!(
|
||||
@ -434,6 +627,21 @@ pub fn publish_attestation<T: BeaconChainTypes>(
|
||||
)))
|
||||
}
|
||||
}
|
||||
|
||||
} else {
|
||||
error!(
|
||||
log,
|
||||
"Invalid AggregateAndProof Signature"
|
||||
);
|
||||
Err(ApiError::ServerError(format!(
|
||||
"Invalid AggregateAndProof Signature"
|
||||
)))
|
||||
}
|
||||
})?;
|
||||
Ok(signed_proofs)
|
||||
})
|
||||
.and_then(move |signed_proofs| {
|
||||
publish_aggregate_attestations_to_network::<T>(network_chan, signed_proofs)
|
||||
})
|
||||
.and_then(|_| response_builder?.body_no_ssz(&())),
|
||||
)
|
||||
|
||||
@ -6,9 +6,9 @@ use node_test_rig::{
|
||||
testing_client_config, ClientConfig, ClientGenesis, LocalBeaconNode,
|
||||
};
|
||||
use remote_beacon_node::{
|
||||
Committee, HeadBeaconBlock, PersistedOperationPool, PublishStatus, ValidatorDuty,
|
||||
ValidatorResponse,
|
||||
Committee, HeadBeaconBlock, PersistedOperationPool, PublishStatus, ValidatorResponse,
|
||||
};
|
||||
use rest_types::ValidatorDutyBytes;
|
||||
use std::convert::TryInto;
|
||||
use std::sync::Arc;
|
||||
use types::{
|
||||
@ -141,7 +141,7 @@ fn validator_produce_attestation() {
|
||||
remote_node
|
||||
.http
|
||||
.validator()
|
||||
.publish_attestation(attestation.clone()),
|
||||
.publish_attestations(vec![attestation.clone()]),
|
||||
)
|
||||
.expect("should publish attestation");
|
||||
assert!(
|
||||
@ -167,7 +167,7 @@ fn validator_produce_attestation() {
|
||||
remote_node
|
||||
.http
|
||||
.validator()
|
||||
.publish_attestation(attestation),
|
||||
.publish_attestations(vec![attestation]),
|
||||
)
|
||||
.expect("should publish attestation");
|
||||
assert!(
|
||||
@ -229,7 +229,7 @@ fn validator_duties() {
|
||||
}
|
||||
|
||||
fn check_duties<T: BeaconChainTypes>(
|
||||
duties: Vec<ValidatorDuty>,
|
||||
duties: Vec<ValidatorDutyBytes>,
|
||||
epoch: Epoch,
|
||||
validators: Vec<PublicKey>,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
use clap::ArgMatches;
|
||||
use client::{config::DEFAULT_DATADIR, ClientConfig, ClientGenesis, Eth2Config};
|
||||
use eth2_config::{read_from_file, write_to_file};
|
||||
use eth2_libp2p::{Enr, Multiaddr};
|
||||
use eth2_libp2p::{Enr, GossipTopic, Multiaddr};
|
||||
use eth2_testnet_config::Eth2TestnetConfig;
|
||||
use genesis::recent_genesis_time;
|
||||
use rand::{distributions::Alphanumeric, Rng};
|
||||
@ -135,7 +135,12 @@ pub fn get_configs<E: EthSpec>(
|
||||
}
|
||||
|
||||
if let Some(topics_str) = cli_args.value_of("topics") {
|
||||
client_config.network.topics = topics_str.split(',').map(|s| s.into()).collect();
|
||||
let mut topics = Vec::new();
|
||||
let topic_list = topics_str.split(',').collect::<Vec<_>>();
|
||||
for topic_str in topic_list {
|
||||
topics.push(GossipTopic::decode(topic_str)?);
|
||||
}
|
||||
client_config.network.topics = topics;
|
||||
}
|
||||
|
||||
if let Some(discovery_address_str) = cli_args.value_of("discovery-address") {
|
||||
|
||||
@ -124,7 +124,7 @@ impl<E: EthSpec> ProductionBeaconNode<E> {
|
||||
.system_time_slot_clock()?
|
||||
.websocket_event_handler(client_config.websocket_server.clone())?
|
||||
.build_beacon_chain()?
|
||||
.libp2p_network(&client_config.network)?
|
||||
.network(&client_config.network)?
|
||||
.notifier()?;
|
||||
|
||||
let builder = if client_config.rest_api.enabled {
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "store"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Paul Hauner <paul@paulhauner.com>"]
|
||||
edition = "2018"
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@ use crate::{
|
||||
leveldb_store::LevelDB, DBColumn, Error, PartialBeaconState, SimpleStoreItem, Store, StoreItem,
|
||||
};
|
||||
use lru::LruCache;
|
||||
use parking_lot::{Mutex, RwLock};
|
||||
use parking_lot::RwLock;
|
||||
use slog::{debug, trace, warn, Logger};
|
||||
use ssz::{Decode, Encode};
|
||||
use ssz_derive::{Decode, Encode};
|
||||
@ -22,7 +22,6 @@ use std::convert::TryInto;
|
||||
use std::marker::PhantomData;
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
use types::beacon_state::CloneConfig;
|
||||
use types::*;
|
||||
|
||||
/// 32-byte key for accessing the `split` of the freezer DB.
|
||||
@ -46,9 +45,7 @@ pub struct HotColdDB<E: EthSpec> {
|
||||
/// The hot database also contains all blocks.
|
||||
pub(crate) hot_db: LevelDB<E>,
|
||||
/// LRU cache of deserialized blocks. Updated whenever a block is loaded.
|
||||
block_cache: Mutex<LruCache<Hash256, SignedBeaconBlock<E>>>,
|
||||
/// LRU cache of deserialized states. Updated whenever a state is loaded.
|
||||
state_cache: Mutex<LruCache<Hash256, BeaconState<E>>>,
|
||||
block_cache: RwLock<LruCache<Hash256, SignedBeaconBlock<E>>>,
|
||||
/// Chain spec.
|
||||
spec: ChainSpec,
|
||||
/// Logger.
|
||||
@ -112,7 +109,7 @@ impl<E: EthSpec> Store<E> for HotColdDB<E> {
|
||||
self.put(block_root, &block)?;
|
||||
|
||||
// Update cache.
|
||||
self.block_cache.lock().put(*block_root, block);
|
||||
self.block_cache.write().put(*block_root, block);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@ -122,7 +119,7 @@ impl<E: EthSpec> Store<E> for HotColdDB<E> {
|
||||
metrics::inc_counter(&metrics::BEACON_BLOCK_GET_COUNT);
|
||||
|
||||
// Check the cache.
|
||||
if let Some(block) = self.block_cache.lock().get(block_root) {
|
||||
if let Some(block) = self.block_cache.write().get(block_root) {
|
||||
metrics::inc_counter(&metrics::BEACON_BLOCK_CACHE_HIT_COUNT);
|
||||
return Ok(Some(block.clone()));
|
||||
}
|
||||
@ -131,7 +128,7 @@ impl<E: EthSpec> Store<E> for HotColdDB<E> {
|
||||
match self.get::<SignedBeaconBlock<E>>(block_root)? {
|
||||
Some(block) => {
|
||||
// Add to cache.
|
||||
self.block_cache.lock().put(*block_root, block.clone());
|
||||
self.block_cache.write().put(*block_root, block.clone());
|
||||
Ok(Some(block))
|
||||
}
|
||||
None => Ok(None),
|
||||
@ -140,12 +137,12 @@ impl<E: EthSpec> Store<E> for HotColdDB<E> {
|
||||
|
||||
/// Delete a block from the store and the block cache.
|
||||
fn delete_block(&self, block_root: &Hash256) -> Result<(), Error> {
|
||||
self.block_cache.lock().pop(block_root);
|
||||
self.block_cache.write().pop(block_root);
|
||||
self.delete::<SignedBeaconBlock<E>>(block_root)
|
||||
}
|
||||
|
||||
/// Store a state in the store.
|
||||
fn put_state(&self, state_root: &Hash256, state: BeaconState<E>) -> Result<(), Error> {
|
||||
fn put_state(&self, state_root: &Hash256, state: &BeaconState<E>) -> Result<(), Error> {
|
||||
if state.slot < self.get_split_slot() {
|
||||
self.store_cold_state(state_root, &state)
|
||||
} else {
|
||||
@ -159,7 +156,7 @@ impl<E: EthSpec> Store<E> for HotColdDB<E> {
|
||||
state_root: &Hash256,
|
||||
slot: Option<Slot>,
|
||||
) -> Result<Option<BeaconState<E>>, Error> {
|
||||
self.get_state_with(state_root, slot, CloneConfig::all())
|
||||
self.get_state_with(state_root, slot)
|
||||
}
|
||||
|
||||
/// Get a state from the store.
|
||||
@ -169,7 +166,6 @@ impl<E: EthSpec> Store<E> for HotColdDB<E> {
|
||||
&self,
|
||||
state_root: &Hash256,
|
||||
slot: Option<Slot>,
|
||||
clone_config: CloneConfig,
|
||||
) -> Result<Option<BeaconState<E>>, Error> {
|
||||
metrics::inc_counter(&metrics::BEACON_STATE_GET_COUNT);
|
||||
|
||||
@ -177,10 +173,10 @@ impl<E: EthSpec> Store<E> for HotColdDB<E> {
|
||||
if slot < self.get_split_slot() {
|
||||
self.load_cold_state_by_slot(slot).map(Some)
|
||||
} else {
|
||||
self.load_hot_state(state_root, clone_config)
|
||||
self.load_hot_state(state_root)
|
||||
}
|
||||
} else {
|
||||
match self.load_hot_state(state_root, clone_config)? {
|
||||
match self.load_hot_state(state_root)? {
|
||||
Some(state) => Ok(Some(state)),
|
||||
None => self.load_cold_state(state_root),
|
||||
}
|
||||
@ -204,9 +200,6 @@ impl<E: EthSpec> Store<E> for HotColdDB<E> {
|
||||
.key_delete(DBColumn::BeaconState.into(), state_root.as_bytes())?;
|
||||
}
|
||||
|
||||
// Delete from the cache.
|
||||
self.state_cache.lock().pop(state_root);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@ -309,10 +302,7 @@ impl<E: EthSpec> Store<E> for HotColdDB<E> {
|
||||
{
|
||||
// NOTE: minor inefficiency here because we load an unnecessary hot state summary
|
||||
let state = self
|
||||
.load_hot_state(
|
||||
&epoch_boundary_state_root,
|
||||
CloneConfig::committee_caches_only(),
|
||||
)?
|
||||
.load_hot_state(&epoch_boundary_state_root)?
|
||||
.ok_or_else(|| {
|
||||
HotColdDBError::MissingEpochBoundaryState(epoch_boundary_state_root)
|
||||
})?;
|
||||
@ -348,8 +338,7 @@ impl<E: EthSpec> HotColdDB<E> {
|
||||
split: RwLock::new(Split::default()),
|
||||
cold_db: LevelDB::open(cold_path)?,
|
||||
hot_db: LevelDB::open(hot_path)?,
|
||||
block_cache: Mutex::new(LruCache::new(config.block_cache_size)),
|
||||
state_cache: Mutex::new(LruCache::new(config.state_cache_size)),
|
||||
block_cache: RwLock::new(LruCache::new(config.block_cache_size)),
|
||||
config,
|
||||
spec,
|
||||
log,
|
||||
@ -371,7 +360,7 @@ impl<E: EthSpec> HotColdDB<E> {
|
||||
pub fn store_hot_state(
|
||||
&self,
|
||||
state_root: &Hash256,
|
||||
state: BeaconState<E>,
|
||||
state: &BeaconState<E>,
|
||||
) -> Result<(), Error> {
|
||||
// On the epoch boundary, store the full state.
|
||||
if state.slot % E::slots_per_epoch() == 0 {
|
||||
@ -387,10 +376,7 @@ impl<E: EthSpec> HotColdDB<E> {
|
||||
// Store a summary of the state.
|
||||
// We store one even for the epoch boundary states, as we may need their slots
|
||||
// when doing a look up by state root.
|
||||
self.put_state_summary(state_root, HotStateSummary::new(state_root, &state)?)?;
|
||||
|
||||
// Store the state in the cache.
|
||||
self.state_cache.lock().put(*state_root, state);
|
||||
self.put_state_summary(state_root, HotStateSummary::new(state_root, state)?)?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@ -398,24 +384,9 @@ impl<E: EthSpec> HotColdDB<E> {
|
||||
/// Load a post-finalization state from the hot database.
|
||||
///
|
||||
/// Will replay blocks from the nearest epoch boundary.
|
||||
pub fn load_hot_state(
|
||||
&self,
|
||||
state_root: &Hash256,
|
||||
clone_config: CloneConfig,
|
||||
) -> Result<Option<BeaconState<E>>, Error> {
|
||||
pub fn load_hot_state(&self, state_root: &Hash256) -> Result<Option<BeaconState<E>>, Error> {
|
||||
metrics::inc_counter(&metrics::BEACON_STATE_HOT_GET_COUNT);
|
||||
|
||||
// Check the cache.
|
||||
if let Some(state) = self.state_cache.lock().get(state_root) {
|
||||
metrics::inc_counter(&metrics::BEACON_STATE_CACHE_HIT_COUNT);
|
||||
|
||||
let timer = metrics::start_timer(&metrics::BEACON_STATE_CACHE_CLONE_TIME);
|
||||
let state = state.clone_with(clone_config);
|
||||
metrics::stop_timer(timer);
|
||||
|
||||
return Ok(Some(state));
|
||||
}
|
||||
|
||||
if let Some(HotStateSummary {
|
||||
slot,
|
||||
latest_block_root,
|
||||
@ -439,9 +410,6 @@ impl<E: EthSpec> HotColdDB<E> {
|
||||
self.replay_blocks(boundary_state, blocks, slot)?
|
||||
};
|
||||
|
||||
// Update the LRU cache.
|
||||
self.state_cache.lock().put(*state_root, state.clone());
|
||||
|
||||
Ok(Some(state))
|
||||
} else {
|
||||
Ok(None)
|
||||
|
||||
@ -345,7 +345,7 @@ mod test {
|
||||
|
||||
let state_a_root = hashes.next().unwrap();
|
||||
state_b.state_roots[0] = state_a_root;
|
||||
store.put_state(&state_a_root, state_a).unwrap();
|
||||
store.put_state(&state_a_root, &state_a).unwrap();
|
||||
|
||||
let iter = BlockRootsIterator::new(store, &state_b);
|
||||
|
||||
@ -393,8 +393,8 @@ mod test {
|
||||
let state_a_root = Hash256::from_low_u64_be(slots_per_historical_root as u64);
|
||||
let state_b_root = Hash256::from_low_u64_be(slots_per_historical_root as u64 * 2);
|
||||
|
||||
store.put_state(&state_a_root, state_a).unwrap();
|
||||
store.put_state(&state_b_root, state_b.clone()).unwrap();
|
||||
store.put_state(&state_a_root, &state_a).unwrap();
|
||||
store.put_state(&state_b_root, &state_b.clone()).unwrap();
|
||||
|
||||
let iter = StateRootsIterator::new(store, &state_b);
|
||||
|
||||
|
||||
@ -123,7 +123,7 @@ impl<E: EthSpec> Store<E> for LevelDB<E> {
|
||||
}
|
||||
|
||||
/// Store a state in the store.
|
||||
fn put_state(&self, state_root: &Hash256, state: BeaconState<E>) -> Result<(), Error> {
|
||||
fn put_state(&self, state_root: &Hash256, state: &BeaconState<E>) -> Result<(), Error> {
|
||||
store_full_state(self, state_root, &state)
|
||||
}
|
||||
|
||||
|
||||
@ -38,7 +38,6 @@ pub use errors::Error;
|
||||
pub use impls::beacon_state::StorageContainer as BeaconStateStorageContainer;
|
||||
pub use metrics::scrape_for_metrics;
|
||||
pub use state_batch::StateBatch;
|
||||
pub use types::beacon_state::CloneConfig;
|
||||
pub use types::*;
|
||||
|
||||
/// An object capable of storing and retrieving objects implementing `StoreItem`.
|
||||
@ -97,7 +96,7 @@ pub trait Store<E: EthSpec>: Sync + Send + Sized + 'static {
|
||||
}
|
||||
|
||||
/// Store a state in the store.
|
||||
fn put_state(&self, state_root: &Hash256, state: BeaconState<E>) -> Result<(), Error>;
|
||||
fn put_state(&self, state_root: &Hash256, state: &BeaconState<E>) -> Result<(), Error>;
|
||||
|
||||
/// Store a state summary in the store.
|
||||
// NOTE: this is a hack for the HotColdDb, we could consider splitting this
|
||||
@ -122,7 +121,6 @@ pub trait Store<E: EthSpec>: Sync + Send + Sized + 'static {
|
||||
&self,
|
||||
state_root: &Hash256,
|
||||
slot: Option<Slot>,
|
||||
_clone_config: CloneConfig,
|
||||
) -> Result<Option<BeaconState<E>>, Error> {
|
||||
// Default impl ignores config. Overriden in `HotColdDb`.
|
||||
self.get_state(state_root, slot)
|
||||
|
||||
@ -76,7 +76,7 @@ impl<E: EthSpec> Store<E> for MemoryStore<E> {
|
||||
}
|
||||
|
||||
/// Store a state in the store.
|
||||
fn put_state(&self, state_root: &Hash256, state: BeaconState<E>) -> Result<(), Error> {
|
||||
fn put_state(&self, state_root: &Hash256, state: &BeaconState<E>) -> Result<(), Error> {
|
||||
store_full_state(self, state_root, &state)
|
||||
}
|
||||
|
||||
|
||||
@ -38,7 +38,7 @@ impl<E: EthSpec> StateBatch<E> {
|
||||
/// May fail to write the full batch if any of the items error (i.e. not atomic!)
|
||||
pub fn commit<S: Store<E>>(self, store: &S) -> Result<(), Error> {
|
||||
self.items.into_iter().try_for_each(|item| match item {
|
||||
BatchItem::Full(state_root, state) => store.put_state(&state_root, state),
|
||||
BatchItem::Full(state_root, state) => store.put_state(&state_root, &state),
|
||||
BatchItem::Summary(state_root, summary) => {
|
||||
store.put_state_summary(&state_root, summary)
|
||||
}
|
||||
|
||||
14
beacon_node/timer/Cargo.toml
Normal file
14
beacon_node/timer/Cargo.toml
Normal file
@ -0,0 +1,14 @@
|
||||
[package]
|
||||
name = "timer"
|
||||
version = "0.2.0"
|
||||
authors = ["Age Manning <Age@AgeManning.com>"]
|
||||
edition = "2018"
|
||||
|
||||
[dependencies]
|
||||
beacon_chain = { path = "../beacon_chain" }
|
||||
types = { path = "../../eth2/types" }
|
||||
slot_clock = { path = "../../eth2/utils/slot_clock" }
|
||||
tokio = "0.1.22"
|
||||
slog = "2.5.2"
|
||||
parking_lot = "0.10.0"
|
||||
futures = "0.1.29"
|
||||
97
beacon_node/timer/src/lib.rs
Normal file
97
beacon_node/timer/src/lib.rs
Normal file
@ -0,0 +1,97 @@
|
||||
//! A timer service for the beacon node.
|
||||
//!
|
||||
//! This service allows task execution on the beacon node for various functionality.
|
||||
|
||||
use beacon_chain::{BeaconChain, BeaconChainTypes};
|
||||
use futures::prelude::*;
|
||||
use slog::warn;
|
||||
use slot_clock::SlotClock;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
use tokio::runtime::TaskExecutor;
|
||||
use tokio::timer::Interval;
|
||||
use types::EthSpec;
|
||||
|
||||
/// A collection of timers that can execute actions on the beacon node.
|
||||
///
|
||||
/// This currently only has a per-slot timer, although others may be added in the future
|
||||
struct Timer<T: BeaconChainTypes> {
|
||||
/// Beacon chain associated.
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
/// A timer that fires every slot.
|
||||
per_slot_timer: Interval,
|
||||
/// The logger for the timer.
|
||||
log: slog::Logger,
|
||||
}
|
||||
|
||||
impl<T: BeaconChainTypes> Timer<T> {
|
||||
pub fn new(
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
milliseconds_per_slot: u64,
|
||||
log: slog::Logger,
|
||||
) -> Result<Self, &'static str> {
|
||||
let duration_to_next_slot = beacon_chain
|
||||
.slot_clock
|
||||
.duration_to_next_slot()
|
||||
.ok_or_else(|| "slot_notifier unable to determine time to next slot")?;
|
||||
|
||||
let slot_duration = Duration::from_millis(milliseconds_per_slot);
|
||||
// A per-slot timer
|
||||
let start_instant = Instant::now() + duration_to_next_slot;
|
||||
let per_slot_timer = Interval::new(start_instant, slot_duration);
|
||||
|
||||
Ok(Timer {
|
||||
beacon_chain,
|
||||
per_slot_timer,
|
||||
log,
|
||||
})
|
||||
}
|
||||
|
||||
/// Tasks that occur on a per-slot basis.
|
||||
pub fn per_slot_task(&self) {
|
||||
self.beacon_chain.per_slot_task();
|
||||
}
|
||||
|
||||
pub fn per_epoch_task(&self) {
|
||||
self.beacon_chain.per_epoch_task();
|
||||
}
|
||||
}
|
||||
|
||||
/// Spawns a timer service which periodically executes tasks for the beacon chain
|
||||
pub fn spawn<T: BeaconChainTypes>(
|
||||
executor: &TaskExecutor,
|
||||
beacon_chain: Arc<BeaconChain<T>>,
|
||||
milliseconds_per_slot: u64,
|
||||
log: slog::Logger,
|
||||
) -> Result<tokio::sync::oneshot::Sender<()>, &'static str> {
|
||||
//let thread_log = log.clone();
|
||||
let mut timer = Timer::new(beacon_chain, milliseconds_per_slot, log)?;
|
||||
let (exit_signal, mut exit) = tokio::sync::oneshot::channel();
|
||||
|
||||
executor.spawn(futures::future::poll_fn(move || -> Result<_, ()> {
|
||||
if let Ok(Async::Ready(_)) | Err(_) = exit.poll() {
|
||||
// notifier is terminating, end the process
|
||||
return Ok(Async::Ready(()));
|
||||
}
|
||||
|
||||
while let Async::Ready(_) = timer
|
||||
.per_slot_timer
|
||||
.poll()
|
||||
.map_err(|e| warn!(timer.log, "Per slot timer error"; "error" => format!("{:?}", e)))?
|
||||
{
|
||||
timer.per_slot_task();
|
||||
match timer
|
||||
.beacon_chain
|
||||
.slot_clock
|
||||
.now()
|
||||
.map(|slot| (slot % T::EthSpec::slots_per_epoch()).as_u64())
|
||||
{
|
||||
Some(0) => timer.per_epoch_task(),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
Ok(Async::NotReady)
|
||||
}));
|
||||
|
||||
Ok(exit_signal)
|
||||
}
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "version"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Age Manning <Age@AgeManning.com>"]
|
||||
edition = "2018"
|
||||
|
||||
|
||||
@ -1,13 +1,12 @@
|
||||
[package]
|
||||
name = "websocket_server"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Paul Hauner <paul@paulhauner.com>"]
|
||||
edition = "2018"
|
||||
|
||||
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
||||
|
||||
[dependencies]
|
||||
exit-future = "0.1.4"
|
||||
futures = "0.1.29"
|
||||
serde = "1.0.102"
|
||||
serde_derive = "1.0.102"
|
||||
|
||||
@ -40,7 +40,14 @@ pub fn start_server<T: EthSpec>(
|
||||
config: &Config,
|
||||
executor: &TaskExecutor,
|
||||
log: &Logger,
|
||||
) -> Result<(WebSocketSender<T>, exit_future::Signal, SocketAddr), String> {
|
||||
) -> Result<
|
||||
(
|
||||
WebSocketSender<T>,
|
||||
tokio::sync::oneshot::Sender<()>,
|
||||
SocketAddr,
|
||||
),
|
||||
String,
|
||||
> {
|
||||
let server_string = format!("{}:{}", config.listen_address, config.port);
|
||||
|
||||
// Create a server that simply ignores any incoming messages.
|
||||
@ -64,29 +71,31 @@ pub fn start_server<T: EthSpec>(
|
||||
let broadcaster = server.broadcaster();
|
||||
|
||||
// Produce a signal/channel that can gracefully shutdown the websocket server.
|
||||
let exit_signal = {
|
||||
let (exit_signal, exit) = exit_future::signal();
|
||||
let exit_channel = {
|
||||
let (exit_channel, exit) = tokio::sync::oneshot::channel();
|
||||
|
||||
let log_inner = log.clone();
|
||||
let broadcaster_inner = server.broadcaster();
|
||||
let exit_future = exit.and_then(move |_| {
|
||||
if let Err(e) = broadcaster_inner.shutdown() {
|
||||
warn!(
|
||||
log_inner,
|
||||
"Websocket server errored on shutdown";
|
||||
"error" => format!("{:?}", e)
|
||||
);
|
||||
} else {
|
||||
info!(log_inner, "Websocket server shutdown");
|
||||
}
|
||||
Ok(())
|
||||
});
|
||||
let exit_future = exit
|
||||
.and_then(move |_| {
|
||||
if let Err(e) = broadcaster_inner.shutdown() {
|
||||
warn!(
|
||||
log_inner,
|
||||
"Websocket server errored on shutdown";
|
||||
"error" => format!("{:?}", e)
|
||||
);
|
||||
} else {
|
||||
info!(log_inner, "Websocket server shutdown");
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.map_err(|_| ());
|
||||
|
||||
// Place a future on the executor that will shutdown the websocket server when the
|
||||
// application exits.
|
||||
executor.spawn(exit_future);
|
||||
|
||||
exit_signal
|
||||
exit_channel
|
||||
};
|
||||
|
||||
let log_inner = log.clone();
|
||||
@ -118,7 +127,7 @@ pub fn start_server<T: EthSpec>(
|
||||
sender: Some(broadcaster),
|
||||
_phantom: PhantomData,
|
||||
},
|
||||
exit_signal,
|
||||
exit_channel,
|
||||
actual_listen_addr,
|
||||
))
|
||||
}
|
||||
|
||||
@ -5,16 +5,23 @@ client to connect to the beacon node and produce blocks and attestations.
|
||||
|
||||
## Endpoints
|
||||
|
||||
HTTP Path | Description |
|
||||
HTTP Path | HTTP Method | Description |
|
||||
| --- | -- |
|
||||
[`/validator/duties`](#validatorduties) | Provides block and attestation production information for validators.
|
||||
[`/validator/duties/all`](#validatordutiesall) | Provides block and attestation production information for all validators.
|
||||
[`/validator/duties/active`](#validatordutiesactive) | Provides block and attestation production information for all active validators.
|
||||
[`/validator/block`](#validatorblock) | Produces a `BeaconBlock` object from current state.
|
||||
[`/validator/attestation`](#validatorattestation) | Produces an unsigned `Attestation` object from current state.
|
||||
[`/validator/block`](#validatorblock) | Processes a `SignedBeaconBlock` object and publishes it to the network.
|
||||
[`/validator/attestation`](#validatorattestation) | Processes a signed `Attestation` and publishes it to the network.
|
||||
|
||||
[`/validator/duties`](#validatorduties) | GET | Provides block and attestation production information for validators.
|
||||
[`/validator/duties/all`](#validatordutiesall) | GET |Provides block and attestation production information for all validators.
|
||||
[`/validator/duties/active`](#validatordutiesactive) | GET | Provides block and attestation production information for all active validators.
|
||||
[`/validator/block`](#validatorblockget) | GET | Retrieves the current beacon
|
||||
block for the validator to publish.
|
||||
[`/validator/block`](#validatorblockpost) | POST | Publishes a signed block to the
|
||||
network.
|
||||
[`/validator/attestation`](#validatorattestation) | GET | Retrieves the current best attestation for a validator to publish.
|
||||
[`/validator/attestations`](#validatorattestations) | POST | Publishes a list
|
||||
of raw unaggregated attestations to their appropriate subnets
|
||||
[`/validator/aggregate_attestation`](#validatoraggregateattestation) | GET | Gets an aggregate attestation for validators to sign and publish.
|
||||
[`/validator/aggregate_attestations`](#validatoraggregateattestation) | POST |
|
||||
Publishes a list of aggregated attestations for validators who are aggregators
|
||||
[`/validator/subscribe`](#validatorsubscribe) | POST | Subscribes a list of
|
||||
validators to the beacon node for a particular duty/slot.
|
||||
|
||||
## `/validator/duties`
|
||||
|
||||
@ -81,7 +88,8 @@ _Note: for demonstration purposes the second pubkey is some unknown pubkey._
|
||||
"attestation_slot": 38511,
|
||||
"attestation_committee_index": 3,
|
||||
"attestation_committee_position": 39,
|
||||
"block_proposal_slots": []
|
||||
"block_proposal_slots": [],
|
||||
"aggregator_modulo": 5,
|
||||
},
|
||||
{
|
||||
"validator_pubkey": "0x42f87bc7c8fa10408425bbeeeb3dc3874242b4bd92f57775b60b39142426f9ec80b273a64269332d97bdb7d93ae05a42",
|
||||
@ -90,6 +98,7 @@ _Note: for demonstration purposes the second pubkey is some unknown pubkey._
|
||||
"attestation_committee_index": null,
|
||||
"attestation_committee_position": null,
|
||||
"block_proposal_slots": []
|
||||
"aggregator_modulo": null,
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "operation_pool"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Michael Sproul <michael@sigmaprime.io>"]
|
||||
edition = "2018"
|
||||
|
||||
|
||||
@ -10,8 +10,8 @@ use attestation_id::AttestationId;
|
||||
use max_cover::maximum_cover;
|
||||
use parking_lot::RwLock;
|
||||
use state_processing::per_block_processing::errors::{
|
||||
AttestationValidationError, AttesterSlashingValidationError, ExitValidationError,
|
||||
ProposerSlashingValidationError,
|
||||
AttestationInvalid, AttestationValidationError, AttesterSlashingValidationError,
|
||||
ExitValidationError, ProposerSlashingValidationError,
|
||||
};
|
||||
use state_processing::per_block_processing::{
|
||||
get_slashable_indices_modular, verify_attestation_for_block_inclusion,
|
||||
@ -22,25 +22,43 @@ use std::collections::{hash_map, HashMap, HashSet};
|
||||
use std::marker::PhantomData;
|
||||
use types::{
|
||||
typenum::Unsigned, Attestation, AttesterSlashing, BeaconState, BeaconStateError, ChainSpec,
|
||||
EthSpec, Fork, ProposerSlashing, RelativeEpoch, SignedVoluntaryExit, Validator,
|
||||
CommitteeIndex, Epoch, EthSpec, Fork, ProposerSlashing, RelativeEpoch, SignedVoluntaryExit,
|
||||
Slot, Validator,
|
||||
};
|
||||
|
||||
/// The number of slots we keep shard subnet attestations in the operation pool for. A value of 0
|
||||
/// means we remove the attestation pool as soon as the slot ends.
|
||||
const ATTESTATION_SUBNET_SLOT_DURATION: u64 = 1;
|
||||
|
||||
#[derive(Default, Debug)]
|
||||
pub struct OperationPool<T: EthSpec + Default> {
|
||||
/// Map from attestation ID (see below) to vectors of attestations.
|
||||
attestations: RwLock<HashMap<AttestationId, Vec<Attestation<T>>>>,
|
||||
/// Map from attestation ID (see `attestation_id`) to vectors of attestations.
|
||||
///
|
||||
/// These are collected from the aggregate channel. They should already be aggregated but we
|
||||
/// check for disjoint attestations in the unlikely event we receive disjoint attestations.
|
||||
aggregate_attestations: RwLock<HashMap<AttestationId, Vec<Attestation<T>>>>,
|
||||
/// A collection of aggregated attestations for a particular slot and committee index.
|
||||
///
|
||||
/// Un-aggregated attestations are collected on a shard subnet and if a connected validator is
|
||||
/// required to aggregate these attestations they are aggregated and stored here until the
|
||||
/// validator is required to publish the aggregate attestation.
|
||||
/// This segregates attestations into (slot,committee_index) then by `AttestationId`.
|
||||
committee_attestations:
|
||||
RwLock<HashMap<(Slot, CommitteeIndex), HashMap<AttestationId, Attestation<T>>>>,
|
||||
/// Map from two attestation IDs to a slashing for those IDs.
|
||||
attester_slashings: RwLock<HashMap<(AttestationId, AttestationId), AttesterSlashing<T>>>,
|
||||
/// Map from proposer index to slashing.
|
||||
proposer_slashings: RwLock<HashMap<u64, ProposerSlashing>>,
|
||||
/// Map from exiting validator to their exit data.
|
||||
voluntary_exits: RwLock<HashMap<u64, SignedVoluntaryExit>>,
|
||||
/// Marker to pin the generics.
|
||||
_phantom: PhantomData<T>,
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum OpPoolError {
|
||||
GetAttestationsTotalBalanceError(BeaconStateError),
|
||||
NoAttestationsForSlotCommittee,
|
||||
}
|
||||
|
||||
impl<T: EthSpec> OperationPool<T> {
|
||||
@ -49,12 +67,13 @@ impl<T: EthSpec> OperationPool<T> {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Insert an attestation into the pool, aggregating it with existing attestations if possible.
|
||||
/// Insert an attestation from the aggregate channel into the pool, checking if the
|
||||
/// aggregate can be further aggregated
|
||||
///
|
||||
/// ## Note
|
||||
///
|
||||
/// This function assumes the given `attestation` is valid.
|
||||
pub fn insert_attestation(
|
||||
pub fn insert_aggregate_attestation(
|
||||
&self,
|
||||
attestation: Attestation<T>,
|
||||
fork: &Fork,
|
||||
@ -63,7 +82,7 @@ impl<T: EthSpec> OperationPool<T> {
|
||||
let id = AttestationId::from_data(&attestation.data, fork, spec);
|
||||
|
||||
// Take a write lock on the attestations map.
|
||||
let mut attestations = self.attestations.write();
|
||||
let mut attestations = self.aggregate_attestations.write();
|
||||
|
||||
let existing_attestations = match attestations.entry(id) {
|
||||
hash_map::Entry::Vacant(entry) => {
|
||||
@ -90,9 +109,90 @@ impl<T: EthSpec> OperationPool<T> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Total number of attestations in the pool, including attestations for the same data.
|
||||
/// Insert a raw un-aggregated attestation into the pool, for a given (slot, committee_index).
|
||||
///
|
||||
/// ## Note
|
||||
///
|
||||
/// It would be a fair assumption that all attestations here are unaggregated and we
|
||||
/// therefore do not need to check if `signers_disjoint_form`. However the cost of doing
|
||||
/// so is low, so we perform this check for added safety.
|
||||
pub fn insert_raw_attestation(
|
||||
&self,
|
||||
attestation: Attestation<T>,
|
||||
fork: &Fork,
|
||||
spec: &ChainSpec,
|
||||
) -> Result<(), AttestationValidationError> {
|
||||
let id = AttestationId::from_data(&attestation.data, fork, spec);
|
||||
|
||||
let slot = attestation.data.slot.clone();
|
||||
let committee_index = attestation.data.index.clone();
|
||||
|
||||
// Take a write lock on the attestations map.
|
||||
let mut attestations = self.committee_attestations.write();
|
||||
|
||||
let slot_index_map = attestations
|
||||
.entry((slot, committee_index))
|
||||
.or_insert_with(|| HashMap::new());
|
||||
|
||||
let existing_attestation = match slot_index_map.entry(id) {
|
||||
hash_map::Entry::Vacant(entry) => {
|
||||
entry.insert(attestation);
|
||||
return Ok(());
|
||||
}
|
||||
hash_map::Entry::Occupied(entry) => entry.into_mut(),
|
||||
};
|
||||
|
||||
if existing_attestation.signers_disjoint_from(&attestation) {
|
||||
existing_attestation.aggregate(&attestation);
|
||||
} else if *existing_attestation != attestation {
|
||||
return Err(AttestationValidationError::Invalid(
|
||||
AttestationInvalid::NotDisjoint,
|
||||
));
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Total number of aggregate attestations in the pool from the aggregate channel, including attestations for the same data.
|
||||
pub fn num_attestations(&self) -> usize {
|
||||
self.attestations.read().values().map(Vec::len).sum()
|
||||
self.aggregate_attestations
|
||||
.read()
|
||||
.values()
|
||||
.map(Vec::len)
|
||||
.sum()
|
||||
}
|
||||
|
||||
/// Total number of attestations in the pool, including attestations for the same data.
|
||||
pub fn total_num_attestations(&self) -> usize {
|
||||
self.num_attestations().saturating_add(
|
||||
self.committee_attestations
|
||||
.read()
|
||||
.values()
|
||||
.map(|map| map.values().len())
|
||||
.sum(),
|
||||
)
|
||||
}
|
||||
|
||||
/// Get the aggregated raw attestations for a (slot, committee)
|
||||
//TODO: Check this logic and optimize
|
||||
pub fn get_raw_aggregated_attestations(
|
||||
&self,
|
||||
slot: &Slot,
|
||||
index: &CommitteeIndex,
|
||||
state: &BeaconState<T>,
|
||||
spec: &ChainSpec,
|
||||
) -> Result<Attestation<T>, OpPoolError> {
|
||||
let curr_domain_bytes =
|
||||
AttestationId::compute_domain_bytes(state.current_epoch(), &state.fork, spec);
|
||||
self.committee_attestations
|
||||
.read()
|
||||
.get(&(*slot, *index))
|
||||
.ok_or_else(|| OpPoolError::NoAttestationsForSlotCommittee)?
|
||||
.iter()
|
||||
.filter(|(key, _)| key.domain_bytes_match(&curr_domain_bytes))
|
||||
.next()
|
||||
.map(|(_key, attestation)| attestation.clone())
|
||||
.ok_or_else(|| OpPoolError::NoAttestationsForSlotCommittee)
|
||||
}
|
||||
|
||||
/// Get a list of attestations for inclusion in a block.
|
||||
@ -109,7 +209,7 @@ impl<T: EthSpec> OperationPool<T> {
|
||||
let prev_domain_bytes = AttestationId::compute_domain_bytes(prev_epoch, &state.fork, spec);
|
||||
let curr_domain_bytes =
|
||||
AttestationId::compute_domain_bytes(current_epoch, &state.fork, spec);
|
||||
let reader = self.attestations.read();
|
||||
let reader = self.aggregate_attestations.read();
|
||||
let active_indices = state
|
||||
.get_cached_active_validator_indices(RelativeEpoch::Current)
|
||||
.map_err(OpPoolError::GetAttestationsTotalBalanceError)?;
|
||||
@ -141,19 +241,38 @@ impl<T: EthSpec> OperationPool<T> {
|
||||
))
|
||||
}
|
||||
|
||||
/// Remove attestations which are too old to be included in a block.
|
||||
pub fn prune_attestations(&self, finalized_state: &BeaconState<T>) {
|
||||
/// Removes aggregate attestations which are too old to be included in a block.
|
||||
///
|
||||
/// This leaves the committee_attestations intact. The committee attestations have their own
|
||||
/// prune function as these are not for block inclusion and can be pruned more frequently.
|
||||
/// See `prune_committee_attestations`.
|
||||
//TODO: Michael to check this before merge
|
||||
pub fn prune_attestations(&self, current_epoch: &Epoch) {
|
||||
// We know we can include an attestation if:
|
||||
// state.slot <= attestation_slot + SLOTS_PER_EPOCH
|
||||
// We approximate this check using the attestation's epoch, to avoid computing
|
||||
// the slot or relying on the committee cache of the finalized state.
|
||||
self.attestations.write().retain(|_, attestations| {
|
||||
// All the attestations in this bucket have the same data, so we only need to
|
||||
// check the first one.
|
||||
attestations.first().map_or(false, |att| {
|
||||
finalized_state.current_epoch() <= att.data.target.epoch + 1
|
||||
})
|
||||
});
|
||||
self.aggregate_attestations
|
||||
.write()
|
||||
.retain(|_, attestations| {
|
||||
// All the attestations in this bucket have the same data, so we only need to
|
||||
// check the first one.
|
||||
attestations
|
||||
.first()
|
||||
.map_or(false, |att| *current_epoch <= att.data.target.epoch + 1)
|
||||
});
|
||||
}
|
||||
|
||||
/// Removes old committee attestations. These should be used in the slot that they are
|
||||
/// collected. We keep these around for one extra slot (i.e current_slot + 1) to account for
|
||||
/// potential delays.
|
||||
///
|
||||
/// The beacon chain should call this function every slot with the current slot as the
|
||||
/// parameter.
|
||||
pub fn prune_committee_attestations(&self, current_slot: &Slot) {
|
||||
self.committee_attestations
|
||||
.write()
|
||||
.retain(|(slot, _), _| *slot + ATTESTATION_SUBNET_SLOT_DURATION >= *current_slot)
|
||||
}
|
||||
|
||||
/// Insert a proposer slashing into the pool.
|
||||
@ -332,8 +451,8 @@ impl<T: EthSpec> OperationPool<T> {
|
||||
}
|
||||
|
||||
/// Prune all types of transactions given the latest finalized state.
|
||||
// TODO: Michael - Can we shift these to per-epoch?
|
||||
pub fn prune_all(&self, finalized_state: &BeaconState<T>, spec: &ChainSpec) {
|
||||
self.prune_attestations(finalized_state);
|
||||
self.prune_proposer_slashings(finalized_state);
|
||||
self.prune_attester_slashings(finalized_state, spec);
|
||||
self.prune_voluntary_exits(finalized_state);
|
||||
@ -383,7 +502,8 @@ fn prune_validator_hash_map<T, F, E: EthSpec>(
|
||||
/// Compare two operation pools.
|
||||
impl<T: EthSpec + Default> PartialEq for OperationPool<T> {
|
||||
fn eq(&self, other: &Self) -> bool {
|
||||
*self.attestations.read() == *other.attestations.read()
|
||||
*self.aggregate_attestations.read() == *other.aggregate_attestations.read()
|
||||
&& *self.committee_attestations.read() == *other.committee_attestations.read()
|
||||
&& *self.attester_slashings.read() == *other.attester_slashings.read()
|
||||
&& *self.proposer_slashings.read() == *other.proposer_slashings.read()
|
||||
&& *self.voluntary_exits.read() == *other.voluntary_exits.read()
|
||||
@ -397,6 +517,7 @@ mod release_tests {
|
||||
use super::*;
|
||||
use state_processing::common::{get_attesting_indices, get_base_reward};
|
||||
use std::collections::BTreeSet;
|
||||
use std::iter::FromIterator;
|
||||
use types::test_utils::*;
|
||||
use types::*;
|
||||
|
||||
@ -820,11 +941,15 @@ mod release_tests {
|
||||
let committee = state
|
||||
.get_beacon_committee(att.data.slot, att.data.index)
|
||||
.expect("should get beacon committee");
|
||||
let att_indices = get_attesting_indices::<MainnetEthSpec>(
|
||||
committee.committee,
|
||||
&fresh_validators_bitlist,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let att_indices = BTreeSet::from_iter(
|
||||
get_attesting_indices::<MainnetEthSpec>(
|
||||
committee.committee,
|
||||
&fresh_validators_bitlist,
|
||||
)
|
||||
.unwrap(),
|
||||
);
|
||||
|
||||
let fresh_indices = &att_indices - &seen_indices;
|
||||
|
||||
let rewards = fresh_indices
|
||||
|
||||
@ -17,7 +17,9 @@ pub struct PersistedOperationPool<T: EthSpec> {
|
||||
/// Mapping from attestation ID to attestation mappings.
|
||||
// We could save space by not storing the attestation ID, but it might
|
||||
// be difficult to make that roundtrip due to eager aggregation.
|
||||
attestations: Vec<(AttestationId, Vec<Attestation<T>>)>,
|
||||
// Note: That we don't store the committee attestations as these are short lived and not worth
|
||||
// persisting
|
||||
aggregate_attestations: Vec<(AttestationId, Vec<Attestation<T>>)>,
|
||||
/// Attester slashings.
|
||||
attester_slashings: Vec<AttesterSlashing<T>>,
|
||||
/// Proposer slashings.
|
||||
@ -29,8 +31,8 @@ pub struct PersistedOperationPool<T: EthSpec> {
|
||||
impl<T: EthSpec> PersistedOperationPool<T> {
|
||||
/// Convert an `OperationPool` into serializable form.
|
||||
pub fn from_operation_pool(operation_pool: &OperationPool<T>) -> Self {
|
||||
let attestations = operation_pool
|
||||
.attestations
|
||||
let aggregate_attestations = operation_pool
|
||||
.aggregate_attestations
|
||||
.read()
|
||||
.iter()
|
||||
.map(|(att_id, att)| (att_id.clone(), att.clone()))
|
||||
@ -58,7 +60,7 @@ impl<T: EthSpec> PersistedOperationPool<T> {
|
||||
.collect();
|
||||
|
||||
Self {
|
||||
attestations,
|
||||
aggregate_attestations,
|
||||
attester_slashings,
|
||||
proposer_slashings,
|
||||
voluntary_exits,
|
||||
@ -67,7 +69,7 @@ impl<T: EthSpec> PersistedOperationPool<T> {
|
||||
|
||||
/// Reconstruct an `OperationPool`.
|
||||
pub fn into_operation_pool(self, state: &BeaconState<T>, spec: &ChainSpec) -> OperationPool<T> {
|
||||
let attestations = RwLock::new(self.attestations.into_iter().collect());
|
||||
let aggregate_attestations = RwLock::new(self.aggregate_attestations.into_iter().collect());
|
||||
let attester_slashings = RwLock::new(
|
||||
self.attester_slashings
|
||||
.into_iter()
|
||||
@ -93,7 +95,8 @@ impl<T: EthSpec> PersistedOperationPool<T> {
|
||||
);
|
||||
|
||||
OperationPool {
|
||||
attestations,
|
||||
aggregate_attestations,
|
||||
committee_attestations: Default::default(),
|
||||
attester_slashings,
|
||||
proposer_slashings,
|
||||
voluntary_exits,
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "proto_array_fork_choice"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Paul Hauner <paul@sigmaprime.io>"]
|
||||
edition = "2018"
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "state_processing"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
authors = ["Paul Hauner <paul@paulhauner.com>"]
|
||||
edition = "2018"
|
||||
|
||||
@ -15,7 +15,6 @@ serde = "1.0.102"
|
||||
serde_derive = "1.0.102"
|
||||
lazy_static = "1.4.0"
|
||||
serde_yaml = "0.8.11"
|
||||
eth2_ssz = "0.1.2"
|
||||
beacon_chain = { path = "../../beacon_node/beacon_chain" }
|
||||
store = { path = "../../beacon_node/store" }
|
||||
|
||||
@ -24,6 +23,7 @@ store = { path = "../../beacon_node/store" }
|
||||
bls = { path = "../utils/bls" }
|
||||
integer-sqrt = "0.1.2"
|
||||
itertools = "0.8.1"
|
||||
eth2_ssz = "0.1.2"
|
||||
eth2_ssz_types = { path = "../utils/ssz_types" }
|
||||
merkle_proof = { path = "../utils/merkle_proof" }
|
||||
log = "0.4.8"
|
||||
|
||||
@ -1,4 +1,3 @@
|
||||
use std::collections::BTreeSet;
|
||||
use types::*;
|
||||
|
||||
/// Returns validator indices which participated in the attestation, sorted by increasing index.
|
||||
@ -7,17 +6,20 @@ use types::*;
|
||||
pub fn get_attesting_indices<T: EthSpec>(
|
||||
committee: &[usize],
|
||||
bitlist: &BitList<T::MaxValidatorsPerCommittee>,
|
||||
) -> Result<BTreeSet<usize>, BeaconStateError> {
|
||||
) -> Result<Vec<usize>, BeaconStateError> {
|
||||
if bitlist.len() != committee.len() {
|
||||
return Err(BeaconStateError::InvalidBitfield);
|
||||
}
|
||||
|
||||
Ok(committee
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter_map(|(i, validator_index)| match bitlist.get(i) {
|
||||
Ok(true) => Some(*validator_index),
|
||||
_ => None,
|
||||
})
|
||||
.collect())
|
||||
let mut indices = Vec::with_capacity(bitlist.num_set_bits());
|
||||
|
||||
for (i, validator_index) in committee.iter().enumerate() {
|
||||
if let Ok(true) = bitlist.get(i) {
|
||||
indices.push(*validator_index)
|
||||
}
|
||||
}
|
||||
|
||||
indices.sort_unstable();
|
||||
|
||||
Ok(indices)
|
||||
}
|
||||
|
||||
@ -10,7 +10,7 @@ pub fn initiate_validator_exit<T: EthSpec>(
|
||||
spec: &ChainSpec,
|
||||
) -> Result<(), Error> {
|
||||
if index >= state.validators.len() {
|
||||
return Err(Error::UnknownValidator);
|
||||
return Err(Error::UnknownValidator(index as u64));
|
||||
}
|
||||
|
||||
// Return if the validator already initiated exit
|
||||
|
||||
@ -12,7 +12,7 @@ pub fn slash_validator<T: EthSpec>(
|
||||
spec: &ChainSpec,
|
||||
) -> Result<(), Error> {
|
||||
if slashed_index >= state.validators.len() || slashed_index >= state.balances.len() {
|
||||
return Err(BeaconStateError::UnknownValidator);
|
||||
return Err(BeaconStateError::UnknownValidator(slashed_index as u64));
|
||||
}
|
||||
|
||||
let epoch = state.current_epoch();
|
||||
|
||||
@ -10,8 +10,8 @@ pub mod test_utils;
|
||||
|
||||
pub use genesis::{initialize_beacon_state_from_eth1, is_valid_genesis_state, process_activations};
|
||||
pub use per_block_processing::{
|
||||
errors::BlockProcessingError, per_block_processing, signature_sets, BlockSignatureStrategy,
|
||||
VerifySignatures,
|
||||
block_signature_verifier, errors::BlockProcessingError, per_block_processing, signature_sets,
|
||||
BlockSignatureStrategy, BlockSignatureVerifier, VerifySignatures,
|
||||
};
|
||||
pub use per_epoch_processing::{errors::EpochProcessingError, per_epoch_processing};
|
||||
pub use per_slot_processing::{per_slot_processing, Error as SlotProcessingError};
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user