cerc-io/lighthouse
11
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity

Implement standard eth2.0 API (#1569)

Browse Source 
- Resolves #1550
- Resolves #824
- Resolves #825
- Resolves #1131
- Resolves #1411
- Resolves #1256
- Resolve #1177

- Includes the `ShufflingId` struct initially defined in #1492. That PR is now closed and the changes are included here, with significant bug fixes.
- Implement the https://github.com/ethereum/eth2.0-APIs in a new `http_api` crate using `warp`. This replaces the `rest_api` crate.
- Add a new `common/eth2` crate which provides a wrapper around `reqwest`, providing the HTTP client that is used by the validator client and for testing. This replaces the `common/remote_beacon_node` crate.
- Create a `http_metrics` crate which is a dedicated server for Prometheus metrics (they are no longer served on the same port as the REST API). We now have flags for `--metrics`, `--metrics-address`, etc.
- Allow the `subnet_id` to be an optional parameter for `VerifiedUnaggregatedAttestation::verify`. This means it does not need to be provided unnecessarily by the validator client.
- Move `fn map_attestation_committee` in `mod beacon_chain::attestation_verification` to a new `fn with_committee_cache` on the `BeaconChain` so the same cache can be used for obtaining validator duties.
- Add some other helpers to `BeaconChain` to assist with common API duties (e.g., `block_root_at_slot`, `head_beacon_block_root`).
- Change the `NaiveAggregationPool` so it can index attestations by `hash_tree_root(attestation.data)`. This is a requirement of the API.
- Add functions to `BeaconChainHarness` to allow it to create slashings and exits.
- Allow for `eth1::Eth1NetworkId` to go to/from a `String`.
- Add functions to the `OperationPool` to allow getting all objects in the pool.
- Add function to `BeaconState` to check if a committee cache is initialized.
- Fix bug where `seconds_per_eth1_block` was not transferring over from `YamlConfig` to `ChainSpec`.
- Add the `deposit_contract_address` to `YamlConfig` and `ChainSpec`. We needed to be able to return it in an API response.
- Change some uses of serde `serialize_with` and `deserialize_with` to a single use of `with` (code quality).
- Impl `Display` and `FromStr` for several BLS fields.
- Check for clock discrepancy when VC polls BN for sync state (with +/- 1 slot tolerance). This is not intended to be comprehensive, it was just easy to do.

- See #1434 for a per-endpoint overview.
- Seeking clarity here: https://github.com/ethereum/eth2.0-APIs/issues/75

- [x] Add docs for prom port to close #1256
- [x] Follow up on this #1177
- [x] ~~Follow up with #1424~~ Will fix in future PR.
- [x] Follow up with #1411
- [x] ~~Follow up with  #1260~~ Will fix in future PR.
- [x] Add quotes to all integers.
- [x] Remove `rest_types`
- [x] Address missing beacon block error. (#1629)
- [x] ~~Add tests for lighthouse/peers endpoints~~ Wontfix
- [x] ~~Follow up with validator status proposal~~ Tracked in #1434
- [x] Unify graffiti structs
- [x] ~~Start server when waiting for genesis?~~ Will fix in future PR.
- [x] TODO in http_api tests
- [x] Move lighthouse endpoints off /eth/v1
- [x] Update docs to link to standard

- ~~Blocked on #1586~~

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
This commit is contained in:
Paul Hauner
2020-10-01 11:12:36 +10:00
co-authored by Michael Sproul
parent 8e20176337
commit cdec3cec18
156 changed files with 8862 additions and 8916 deletions
Download Patch File Download Diff File Expand all files Collapse all files
consensus/fork_choice/src/fork_choice.rs
+15 -1
View File
@@ -4,7 +4,7 @@ use proto_array::{Block as ProtoBlock, ProtoArrayForkChoice};
use ssz_derive::{Decode, Encode};
use types::{
BeaconBlock, BeaconState, BeaconStateError, Checkpoint, Epoch, EthSpec, Hash256,
IndexedAttestation, Slot,
IndexedAttestation, RelativeEpoch, ShufflingId, Slot,
};
use crate::ForkChoiceStore;
@@ -240,10 +240,18 @@ where
/// Instantiates `Self` from the genesis parameters.
pub fn from_genesis(
fc_store: T,
genesis_block_root: Hash256,
genesis_block: &BeaconBlock<E>,
genesis_state: &BeaconState<E>,
) -> Result<Self, Error<T::Error>> {
let finalized_block_slot = genesis_block.slot;
let finalized_block_state_root = genesis_block.state_root;
let current_epoch_shuffling_id =
ShufflingId::new(genesis_block_root, genesis_state, RelativeEpoch::Current)
.map_err(Error::BeaconStateError)?;
let next_epoch_shuffling_id =
ShufflingId::new(genesis_block_root, genesis_state, RelativeEpoch::Next)
.map_err(Error::BeaconStateError)?;
let proto_array = ProtoArrayForkChoice::new(
finalized_block_slot,
@@ -251,6 +259,8 @@ where
fc_store.justified_checkpoint().epoch,
fc_store.finalized_checkpoint().epoch,
fc_store.finalized_checkpoint().root,
current_epoch_shuffling_id,
next_epoch_shuffling_id,
)?;
Ok(Self {
@@ -534,6 +544,10 @@ where
root: block_root,
parent_root: Some(block.parent_root),
target_root,
current_epoch_shuffling_id: ShufflingId::new(block_root, state, RelativeEpoch::Current)
.map_err(Error::BeaconStateError)?,
next_epoch_shuffling_id: ShufflingId::new(block_root, state, RelativeEpoch::Next)
.map_err(Error::BeaconStateError)?,
state_root: block.state_root,
justified_epoch: state.current_justified_checkpoint.epoch,
finalized_epoch: state.finalized_checkpoint.epoch,
consensus/fork_choice/src/lib.rs
+1
View File
@@ -6,3 +6,4 @@ pub use crate::fork_choice::{
SAFE_SLOTS_TO_UPDATE_JUSTIFIED,
};
pub use fork_choice_store::ForkChoiceStore;
pub use proto_array::Block as ProtoBlock;
consensus/fork_choice/tests/tests.rs
+1 -1
View File
@@ -351,7 +351,7 @@ impl ForkChoiceTest {
let mut verified_attestation = self
.harness
.chain
.verify_unaggregated_attestation_for_gossip(attestation, subnet_id)
.verify_unaggregated_attestation_for_gossip(attestation, Some(subnet_id))
.expect("precondition: should gossip verify attestation");
if let MutationDelay::Blocks(slots) = delay {
consensus/proto_array/src/fork_choice_test_definition.rs
+12 -1
View File
@@ -4,7 +4,7 @@ mod votes;
use crate::proto_array_fork_choice::{Block, ProtoArrayForkChoice};
use serde_derive::{Deserialize, Serialize};
use types::{Epoch, Hash256, Slot};
use types::{Epoch, Hash256, ShufflingId, Slot};
pub use ffg_updates::*;
pub use no_votes::*;
@@ -55,12 +55,15 @@ pub struct ForkChoiceTestDefinition {
impl ForkChoiceTestDefinition {
pub fn run(self) {
let junk_shuffling_id = ShufflingId::from_components(Epoch::new(0), Hash256::zero());
let mut fork_choice = ProtoArrayForkChoice::new(
self.finalized_block_slot,
Hash256::zero(),
self.justified_epoch,
self.finalized_epoch,
self.finalized_root,
junk_shuffling_id.clone(),
junk_shuffling_id,
)
.expect("should create fork choice struct");
@@ -125,6 +128,14 @@ impl ForkChoiceTestDefinition {
parent_root: Some(parent_root),
state_root: Hash256::zero(),
target_root: Hash256::zero(),
current_epoch_shuffling_id: ShufflingId::from_components(
Epoch::new(0),
Hash256::zero(),
),
next_epoch_shuffling_id: ShufflingId::from_components(
Epoch::new(0),
Hash256::zero(),
),
justified_epoch,
finalized_epoch,
};
consensus/proto_array/src/proto_array.rs
+5 -1
View File
@@ -2,7 +2,7 @@ use crate::{error::Error, Block};
use serde_derive::{Deserialize, Serialize};
use ssz_derive::{Decode, Encode};
use std::collections::HashMap;
use types::{Epoch, Hash256, Slot};
use types::{Epoch, Hash256, ShufflingId, Slot};
#[derive(Clone, PartialEq, Debug, Encode, Decode, Serialize, Deserialize)]
pub struct ProtoNode {
@@ -18,6 +18,8 @@ pub struct ProtoNode {
/// The `target_root` is not necessary for `ProtoArray` either, it also just exists for upstream
/// components (namely fork choice attestation verification).
pub target_root: Hash256,
pub current_epoch_shuffling_id: ShufflingId,
pub next_epoch_shuffling_id: ShufflingId,
pub root: Hash256,
pub parent: Option<usize>,
pub justified_epoch: Epoch,
@@ -142,6 +144,8 @@ impl ProtoArray {
slot: block.slot,
root: block.root,
target_root: block.target_root,
current_epoch_shuffling_id: block.current_epoch_shuffling_id,
next_epoch_shuffling_id: block.next_epoch_shuffling_id,
state_root: block.state_root,
parent: block
.parent_root
consensus/proto_array/src/proto_array_fork_choice.rs
+16 -1
View File
@@ -4,7 +4,7 @@ use crate::ssz_container::SszContainer;
use ssz::{Decode, Encode};
use ssz_derive::{Decode, Encode};
use std::collections::HashMap;
use types::{Epoch, Hash256, Slot};
use types::{Epoch, Hash256, ShufflingId, Slot};
pub const DEFAULT_PRUNE_THRESHOLD: usize = 256;
@@ -25,6 +25,8 @@ pub struct Block {
pub parent_root: Option<Hash256>,
pub state_root: Hash256,
pub target_root: Hash256,
pub current_epoch_shuffling_id: ShufflingId,
pub next_epoch_shuffling_id: ShufflingId,
pub justified_epoch: Epoch,
pub finalized_epoch: Epoch,
}
@@ -70,6 +72,8 @@ impl ProtoArrayForkChoice {
justified_epoch: Epoch,
finalized_epoch: Epoch,
finalized_root: Hash256,
current_epoch_shuffling_id: ShufflingId,
next_epoch_shuffling_id: ShufflingId,
) -> Result<Self, String> {
let mut proto_array = ProtoArray {
prune_threshold: DEFAULT_PRUNE_THRESHOLD,
@@ -87,6 +91,8 @@ impl ProtoArrayForkChoice {
// We are using the finalized_root as the target_root, since it always lies on an
// epoch boundary.
target_root: finalized_root,
current_epoch_shuffling_id,
next_epoch_shuffling_id,
justified_epoch,
finalized_epoch,
};
@@ -194,6 +200,8 @@ impl ProtoArrayForkChoice {
parent_root,
state_root: block.state_root,
target_root: block.target_root,
current_epoch_shuffling_id: block.current_epoch_shuffling_id.clone(),
next_epoch_shuffling_id: block.next_epoch_shuffling_id.clone(),
justified_epoch: block.justified_epoch,
finalized_epoch: block.finalized_epoch,
})
@@ -341,6 +349,7 @@ mod test_compute_deltas {
let finalized_desc = Hash256::from_low_u64_be(2);
let not_finalized_desc = Hash256::from_low_u64_be(3);
let unknown = Hash256::from_low_u64_be(4);
let junk_shuffling_id = ShufflingId::from_components(Epoch::new(0), Hash256::zero());
let mut fc = ProtoArrayForkChoice::new(
genesis_slot,
@@ -348,6 +357,8 @@ mod test_compute_deltas {
genesis_epoch,
genesis_epoch,
finalized_root,
junk_shuffling_id.clone(),
junk_shuffling_id.clone(),
)
.unwrap();
@@ -359,6 +370,8 @@ mod test_compute_deltas {
parent_root: Some(finalized_root),
state_root,
target_root: finalized_root,
current_epoch_shuffling_id: junk_shuffling_id.clone(),
next_epoch_shuffling_id: junk_shuffling_id.clone(),
justified_epoch: genesis_epoch,
finalized_epoch: genesis_epoch,
})
@@ -372,6 +385,8 @@ mod test_compute_deltas {
parent_root: None,
state_root,
target_root: finalized_root,
current_epoch_shuffling_id: junk_shuffling_id.clone(),
next_epoch_shuffling_id: junk_shuffling_id.clone(),
justified_epoch: genesis_epoch,
finalized_epoch: genesis_epoch,
})
consensus/serde_hex/Cargo.toml
-9
View File
@@ -1,9 +0,0 @@
[package]
name = "serde_hex"
version = "0.2.0"
authors = ["Paul Hauner <paul@paulhauner.com>"]
edition = "2018"
[dependencies]
serde = "1.0.110"
hex = "0.4.2"
consensus/serde_utils/Cargo.toml
+1
View File
@@ -7,6 +7,7 @@ edition = "2018"
[dependencies]
serde = { version = "1.0.110", features = ["derive"] }
serde_derive = "1.0.110"
hex = "0.4.2"
[dev-dependencies]
serde_json = "1.0.52"
consensus/serde_utils/src/bytes_4_hex.rs
+38
View File
@@ -0,0 +1,38 @@
//! Formats `[u8; 4]` as a 0x-prefixed hex string.
//!
//! E.g., `[0, 1, 2, 3]` serializes as `"0x00010203"`.
use crate::hex::PrefixedHexVisitor;
use serde::de::Error;
use serde::{Deserializer, Serializer};
const BYTES_LEN: usize = 4;
pub fn serialize<S>(bytes: &[u8; BYTES_LEN], serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut hex_string: String = "0x".to_string();
hex_string.push_str(&hex::encode(&bytes));
serializer.serialize_str(&hex_string)
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<[u8; BYTES_LEN], D::Error>
where
D: Deserializer<'de>,
{
let decoded = deserializer.deserialize_str(PrefixedHexVisitor)?;
if decoded.len() != BYTES_LEN {
return Err(D::Error::custom(format!(
"expected {} bytes for array, got {}",
BYTES_LEN,
decoded.len()
)));
}
let mut array = [0; BYTES_LEN];
array.copy_from_slice(&decoded);
Ok(array)
}
consensus/serde_hex/src/lib.rs → consensus/serde_utils/src/hex.rs
+12
View File
@@ -1,6 +1,9 @@
//! Provides utilities for parsing 0x-prefixed hex strings.
use serde::de::{self, Visitor};
use std::fmt;
/// Encode `data` as a 0x-prefixed hex string.
pub fn encode<T: AsRef<[u8]>>(data: T) -> String {
let hex = hex::encode(data);
let mut s = "0x".to_string();
@@ -8,6 +11,15 @@ pub fn encode<T: AsRef<[u8]>>(data: T) -> String {
s
}
/// Decode `data` from a 0x-prefixed hex string.
pub fn decode(s: &str) -> Result<Vec<u8>, String> {
if s.starts_with("0x") {
hex::decode(&s[2..]).map_err(|e| format!("invalid hex: {:?}", e))
} else {
Err("hex must have 0x prefix".to_string())
}
}
pub struct PrefixedHexVisitor;
impl<'de> Visitor<'de> for PrefixedHexVisitor {
consensus/serde_utils/src/lib.rs
+8 -1
View File
@@ -1,2 +1,9 @@
pub mod quoted_u64;
mod quoted_int;
pub mod bytes_4_hex;
pub mod hex;
pub mod quoted_u64_vec;
pub mod u32_hex;
pub mod u8_hex;
pub use quoted_int::{quoted_u32, quoted_u64, quoted_u8};
consensus/serde_utils/src/quoted_int.rs
+144
View File
@@ -0,0 +1,144 @@
//! Formats some integer types using quotes.
//!
//! E.g., `1` serializes as `"1"`.
//!
//! Quotes can be optional during decoding.
use serde::{Deserializer, Serializer};
use serde_derive::{Deserialize, Serialize};
use std::convert::TryFrom;
use std::marker::PhantomData;
macro_rules! define_mod {
($int: ty, $visit_fn: ident) => {
/// Serde support for deserializing quoted integers.
///
/// Configurable so that quotes are either required or optional.
pub struct QuotedIntVisitor<T> {
require_quotes: bool,
_phantom: PhantomData<T>,
}
impl<'a, T> serde::de::Visitor<'a> for QuotedIntVisitor<T>
where
T: From<$int> + Into<$int> + Copy + TryFrom<u64>,
{
type Value = T;
fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
if self.require_quotes {
write!(formatter, "a quoted integer")
} else {
write!(formatter, "a quoted or unquoted integer")
}
}
fn visit_str<E>(self, s: &str) -> Result<Self::Value, E>
where
E: serde::de::Error,
{
s.parse::<$int>()
.map(T::from)
.map_err(serde::de::Error::custom)
}
fn visit_u64<E>(self, v: u64) -> Result<Self::Value, E>
where
E: serde::de::Error,
{
if self.require_quotes {
Err(serde::de::Error::custom(
"received unquoted integer when quotes are required",
))
} else {
T::try_from(v).map_err(|_| serde::de::Error::custom("invalid integer"))
}
}
}
/// Wrapper type for requiring quotes on a `$int`-like type.
///
/// Unlike using `serde(with = "quoted_$int::require_quotes")` this is composable, and can be nested
/// inside types like `Option`, `Result` and `Vec`.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Deserialize, Serialize)]
#[serde(transparent)]
pub struct Quoted<T>
where
T: From<$int> + Into<$int> + Copy + TryFrom<u64>,
{
#[serde(with = "require_quotes")]
pub value: T,
}
/// Serialize with quotes.
pub fn serialize<S, T>(value: &T, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
T: From<$int> + Into<$int> + Copy,
{
let v: $int = (*value).into();
serializer.serialize_str(&format!("{}", v))
}
/// Deserialize with or without quotes.
pub fn deserialize<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
D: Deserializer<'de>,
T: From<$int> + Into<$int> + Copy + TryFrom<u64>,
{
deserializer.deserialize_any(QuotedIntVisitor {
require_quotes: false,
_phantom: PhantomData,
})
}
/// Requires quotes when deserializing.
///
/// Usage: `#[serde(with = "quoted_u64::require_quotes")]`.
pub mod require_quotes {
pub use super::serialize;
use super::*;
pub fn deserialize<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
D: Deserializer<'de>,
T: From<$int> + Into<$int> + Copy + TryFrom<u64>,
{
deserializer.deserialize_any(QuotedIntVisitor {
require_quotes: true,
_phantom: PhantomData,
})
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn require_quotes() {
let x = serde_json::from_str::<Quoted<$int>>("\"8\"").unwrap();
assert_eq!(x.value, 8);
serde_json::from_str::<Quoted<$int>>("8").unwrap_err();
}
}
};
}
pub mod quoted_u8 {
use super::*;
define_mod!(u8, visit_u8);
}
pub mod quoted_u32 {
use super::*;
define_mod!(u32, visit_u32);
}
pub mod quoted_u64 {
use super::*;
define_mod!(u64, visit_u64);
}
consensus/serde_utils/src/quoted_u64.rs
-115
View File
@@ -1,115 +0,0 @@
use serde::{Deserializer, Serializer};
use serde_derive::{Deserialize, Serialize};
use std::marker::PhantomData;
/// Serde support for deserializing quoted integers.
///
/// Configurable so that quotes are either required or optional.
pub struct QuotedIntVisitor<T> {
require_quotes: bool,
_phantom: PhantomData<T>,
}
impl<'a, T> serde::de::Visitor<'a> for QuotedIntVisitor<T>
where
T: From<u64> + Into<u64> + Copy,
{
type Value = T;
fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
if self.require_quotes {
write!(formatter, "a quoted integer")
} else {
write!(formatter, "a quoted or unquoted integer")
}
}
fn visit_str<E>(self, s: &str) -> Result<Self::Value, E>
where
E: serde::de::Error,
{
s.parse::<u64>()
.map(T::from)
.map_err(serde::de::Error::custom)
}
fn visit_u64<E>(self, v: u64) -> Result<Self::Value, E>
where
E: serde::de::Error,
{
if self.require_quotes {
Err(serde::de::Error::custom(
"received unquoted integer when quotes are required",
))
} else {
Ok(T::from(v))
}
}
}
/// Wrapper type for requiring quotes on a `u64`-like type.
///
/// Unlike using `serde(with = "quoted_u64::require_quotes")` this is composable, and can be nested
/// inside types like `Option`, `Result` and `Vec`.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Deserialize, Serialize)]
#[serde(transparent)]
pub struct Quoted<T>
where
T: From<u64> + Into<u64> + Copy,
{
#[serde(with = "require_quotes")]
pub value: T,
}
/// Serialize with quotes.
pub fn serialize<S, T>(value: &T, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
T: From<u64> + Into<u64> + Copy,
{
let v: u64 = (*value).into();
serializer.serialize_str(&format!("{}", v))
}
/// Deserialize with or without quotes.
pub fn deserialize<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
D: Deserializer<'de>,
T: From<u64> + Into<u64> + Copy,
{
deserializer.deserialize_any(QuotedIntVisitor {
require_quotes: false,
_phantom: PhantomData,
})
}
/// Requires quotes when deserializing.
///
/// Usage: `#[serde(with = "quoted_u64::require_quotes")]`.
pub mod require_quotes {
pub use super::serialize;
use super::*;
pub fn deserialize<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
D: Deserializer<'de>,
T: From<u64> + Into<u64> + Copy,
{
deserializer.deserialize_any(QuotedIntVisitor {
require_quotes: true,
_phantom: PhantomData,
})
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn require_quotes() {
let x = serde_json::from_str::<Quoted<u64>>("\"8\"").unwrap();
assert_eq!(x.value, 8);
serde_json::from_str::<Quoted<u64>>("8").unwrap_err();
}
}
consensus/serde_utils/src/quoted_u64_vec.rs
+7 -1
View File
@@ -1,3 +1,9 @@
//! Formats `Vec<u64>` using quotes.
//!
//! E.g., `vec![0, 1, 2]` serializes as `["0", "1", "2"]`.
//!
//! Quotes can be optional during decoding.
use serde::ser::SerializeSeq;
use serde::{Deserializer, Serializer};
use serde_derive::{Deserialize, Serialize};
@@ -6,7 +12,7 @@ use serde_derive::{Deserialize, Serialize};
#[serde(transparent)]
pub struct QuotedIntWrapper {
#[serde(with = "crate::quoted_u64")]
int: u64,
pub int: u64,
}
pub struct QuotedIntVecVisitor;
consensus/serde_utils/src/u32_hex.rs
+21
View File
@@ -0,0 +1,21 @@
//! Formats `u32` as a 0x-prefixed, little-endian hex string.
//!
//! E.g., `0` serializes as `"0x00000000"`.
use crate::bytes_4_hex;
use serde::{Deserializer, Serializer};
pub fn serialize<S>(num: &u32, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let hex = format!("0x{}", hex::encode(num.to_le_bytes()));
serializer.serialize_str(&hex)
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<u32, D::Error>
where
D: Deserializer<'de>,
{
bytes_4_hex::deserialize(deserializer).map(u32::from_le_bytes)
}
consensus/serde_utils/src/u8_hex.rs
+29
View File
@@ -0,0 +1,29 @@
//! Formats `u8` as a 0x-prefixed hex string.
//!
//! E.g., `0` serializes as `"0x00"`.
use crate::hex::PrefixedHexVisitor;
use serde::de::Error;
use serde::{Deserializer, Serializer};
pub fn serialize<S>(byte: &u8, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let hex = format!("0x{}", hex::encode([*byte]));
serializer.serialize_str(&hex)
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<u8, D::Error>
where
D: Deserializer<'de>,
{
let bytes = deserializer.deserialize_str(PrefixedHexVisitor)?;
if bytes.len() != 1 {
return Err(D::Error::custom(format!(
"expected 1 byte for u8, got {}",
bytes.len()
)));
}
Ok(bytes[0])
}
consensus/ssz_types/Cargo.toml
+1 -1
View File
@@ -11,7 +11,7 @@ name = "ssz_types"
tree_hash = "0.1.0"
serde = "1.0.110"
serde_derive = "1.0.110"
serde_hex = { path = "../serde_hex" }
serde_utils = { path = "../serde_utils" }
eth2_ssz = "0.1.2"
typenum = "1.12.0"
arbitrary = { version = "0.4.4", features = ["derive"], optional = true }
consensus/ssz_types/src/bitfield.rs
+1 -1
View File
@@ -3,7 +3,7 @@ use crate::Error;
use core::marker::PhantomData;
use serde::de::{Deserialize, Deserializer};
use serde::ser::{Serialize, Serializer};
use serde_hex::{encode as hex_encode, PrefixedHexVisitor};
use serde_utils::hex::{encode as hex_encode, PrefixedHexVisitor};
use ssz::{Decode, Encode};
use tree_hash::Hash256;
use typenum::Unsigned;
consensus/types/Cargo.toml
+2
View File
@@ -39,6 +39,8 @@ tempfile = "3.1.0"
derivative = "2.1.1"
rusqlite = { version = "0.23.1", features = ["bundled"], optional = true }
arbitrary = { version = "0.4.4", features = ["derive"], optional = true }
serde_utils = { path = "../serde_utils" }
regex = "1.3.9"
[dev-dependencies]
serde_json = "1.0.52"
consensus/types/src/aggregate_and_proof.rs
+1
View File
@@ -16,6 +16,7 @@ use tree_hash_derive::TreeHash;
#[serde(bound = "T: EthSpec")]
pub struct AggregateAndProof<T: EthSpec> {
/// The index of the validator that created the attestation.
#[serde(with = "serde_utils::quoted_u64")]
pub aggregator_index: u64,
/// The aggregate attestation.
pub aggregate: Attestation<T>,
consensus/types/src/attestation_data.rs
+1
View File
@@ -26,6 +26,7 @@ use tree_hash_derive::TreeHash;
)]
pub struct AttestationData {
pub slot: Slot,
#[serde(with = "serde_utils::quoted_u64")]
pub index: u64,
// LMD GHOST vote
consensus/types/src/attestation_duty.rs
+3
View File
@@ -12,4 +12,7 @@ pub struct AttestationDuty {
pub committee_position: usize,
/// The total number of attesters in the committee.
pub committee_len: usize,
/// The committee count at `attestation_slot`.
#[serde(with = "serde_utils::quoted_u64")]
pub committees_at_slot: u64,
}
consensus/types/src/beacon_block.rs
+1
View File
@@ -16,6 +16,7 @@ use tree_hash_derive::TreeHash;
#[serde(bound = "T: EthSpec")]
pub struct BeaconBlock<T: EthSpec> {
pub slot: Slot,
#[serde(with = "serde_utils::quoted_u64")]
pub proposer_index: u64,
pub parent_root: Hash256,
pub state_root: Hash256,
consensus/types/src/beacon_block_body.rs
-5
View File
@@ -1,5 +1,4 @@
use crate::test_utils::TestRandom;
use crate::utils::{graffiti_from_hex_str, graffiti_to_hex_str, Graffiti};
use crate::*;
use serde_derive::{Deserialize, Serialize};
@@ -17,10 +16,6 @@ use tree_hash_derive::TreeHash;
pub struct BeaconBlockBody<T: EthSpec> {
pub randao_reveal: Signature,
pub eth1_data: Eth1Data,
#[serde(
serialize_with = "graffiti_to_hex_str",
deserialize_with = "graffiti_from_hex_str"
)]
pub graffiti: Graffiti,
pub proposer_slashings: VariableList<ProposerSlashing, T::MaxProposerSlashings>,
pub attester_slashings: VariableList<AttesterSlashing<T>, T::MaxAttesterSlashings>,
consensus/types/src/beacon_block_header.rs
+1
View File
@@ -14,6 +14,7 @@ use tree_hash_derive::TreeHash;
#[derive(Debug, PartialEq, Clone, Serialize, Deserialize, Encode, Decode, TreeHash, TestRandom)]
pub struct BeaconBlockHeader {
pub slot: Slot,
#[serde(with = "serde_utils::quoted_u64")]
pub proposer_index: u64,
pub parent_root: Hash256,
pub state_root: Hash256,
consensus/types/src/beacon_state.rs
+9
View File
@@ -157,6 +157,7 @@ where
T: EthSpec,
{
// Versioning
#[serde(with = "serde_utils::quoted_u64")]
pub genesis_time: u64,
pub genesis_validators_root: Hash256,
pub slot: Slot,
@@ -173,6 +174,7 @@ where
// Ethereum 1.0 chain data
pub eth1_data: Eth1Data,
pub eth1_data_votes: VariableList<Eth1Data, T::SlotsPerEth1VotingPeriod>,
#[serde(with = "serde_utils::quoted_u64")]
pub eth1_deposit_index: u64,
// Registry
@@ -913,6 +915,13 @@ impl<T: EthSpec> BeaconState<T> {
self.exit_cache = ExitCache::default();
}
/// Returns `true` if the committee cache for `relative_epoch` is built and ready to use.
pub fn committee_cache_is_initialized(&self, relative_epoch: RelativeEpoch) -> bool {
let i = Self::committee_cache_index(relative_epoch);
self.committee_caches[i].is_initialized_at(relative_epoch.into_epoch(self.current_epoch()))
}
/// Build an epoch cache, unless it is has already been built.
pub fn build_committee_cache(
&mut self,
consensus/types/src/beacon_state/committee_cache.rs
+1
View File
@@ -186,6 +186,7 @@ impl CommitteeCache {
index,
committee_position,
committee_len,
committees_at_slot: self.committees_per_slot(),
})
})
}
consensus/types/src/chain_spec.rs
+80 -58
View File
@@ -4,10 +4,6 @@ use serde_derive::{Deserialize, Serialize};
use std::fs::File;
use std::path::Path;
use tree_hash::TreeHash;
use utils::{
fork_from_hex_str, fork_to_hex_str, u32_from_hex_str, u32_to_hex_str, u8_from_hex_str,
u8_to_hex_str,
};
/// Each of the BLS signature domains.
///
@@ -65,12 +61,9 @@ pub struct ChainSpec {
/*
* Initial Values
*/
#[serde(
serialize_with = "fork_to_hex_str",
deserialize_with = "fork_from_hex_str"
)]
#[serde(with = "serde_utils::bytes_4_hex")]
pub genesis_fork_version: [u8; 4],
#[serde(deserialize_with = "u8_from_hex_str", serialize_with = "u8_to_hex_str")]
#[serde(with = "serde_utils::u8_hex")]
pub bls_withdrawal_prefix_byte: u8,
/*
@@ -115,6 +108,7 @@ pub struct ChainSpec {
*/
pub eth1_follow_distance: u64,
pub seconds_per_eth1_block: u64,
pub deposit_contract_address: Address,
/*
* Networking
@@ -326,6 +320,9 @@ impl ChainSpec {
*/
eth1_follow_distance: 1_024,
seconds_per_eth1_block: 14,
deposit_contract_address: "1234567890123456789012345678901234567890"
.parse()
.expect("chain spec deposit contract address"),
/*
* Network specific
@@ -448,104 +445,127 @@ pub struct YamlConfig {
#[serde(default)]
config_name: String,
// ChainSpec
max_committees_per_slot: usize,
target_committee_size: usize,
#[serde(with = "serde_utils::quoted_u64")]
max_committees_per_slot: u64,
#[serde(with = "serde_utils::quoted_u64")]
target_committee_size: u64,
#[serde(with = "serde_utils::quoted_u64")]
min_per_epoch_churn_limit: u64,
#[serde(with = "serde_utils::quoted_u64")]
churn_limit_quotient: u64,
#[serde(with = "serde_utils::quoted_u8")]
shuffle_round_count: u8,
#[serde(with = "serde_utils::quoted_u64")]
min_genesis_active_validator_count: u64,
#[serde(with = "serde_utils::quoted_u64")]
min_genesis_time: u64,
#[serde(with = "serde_utils::quoted_u64")]
genesis_delay: u64,
#[serde(with = "serde_utils::quoted_u64")]
min_deposit_amount: u64,
#[serde(with = "serde_utils::quoted_u64")]
max_effective_balance: u64,
#[serde(with = "serde_utils::quoted_u64")]
ejection_balance: u64,
#[serde(with = "serde_utils::quoted_u64")]
effective_balance_increment: u64,
#[serde(with = "serde_utils::quoted_u64")]
hysteresis_quotient: u64,
#[serde(with = "serde_utils::quoted_u64")]
hysteresis_downward_multiplier: u64,
#[serde(with = "serde_utils::quoted_u64")]
hysteresis_upward_multiplier: u64,
// Proportional slashing multiplier defaults to 3 for compatibility with Altona and Medalla.
#[serde(default = "default_proportional_slashing_multiplier")]
#[serde(with = "serde_utils::quoted_u64")]
proportional_slashing_multiplier: u64,
#[serde(
serialize_with = "fork_to_hex_str",
deserialize_with = "fork_from_hex_str"
)]
#[serde(with = "serde_utils::bytes_4_hex")]
genesis_fork_version: [u8; 4],
#[serde(deserialize_with = "u8_from_hex_str", serialize_with = "u8_to_hex_str")]
#[serde(with = "serde_utils::u8_hex")]
bls_withdrawal_prefix: u8,
#[serde(with = "serde_utils::quoted_u64")]
seconds_per_slot: u64,
#[serde(with = "serde_utils::quoted_u64")]
min_attestation_inclusion_delay: u64,
#[serde(with = "serde_utils::quoted_u64")]
min_seed_lookahead: u64,
#[serde(with = "serde_utils::quoted_u64")]
max_seed_lookahead: u64,
#[serde(with = "serde_utils::quoted_u64")]
min_epochs_to_inactivity_penalty: u64,
#[serde(with = "serde_utils::quoted_u64")]
min_validator_withdrawability_delay: u64,
#[serde(with = "serde_utils::quoted_u64")]
shard_committee_period: u64,
#[serde(with = "serde_utils::quoted_u64")]
base_reward_factor: u64,
#[serde(with = "serde_utils::quoted_u64")]
whistleblower_reward_quotient: u64,
#[serde(with = "serde_utils::quoted_u64")]
proposer_reward_quotient: u64,
#[serde(with = "serde_utils::quoted_u64")]
inactivity_penalty_quotient: u64,
#[serde(with = "serde_utils::quoted_u64")]
min_slashing_penalty_quotient: u64,
#[serde(with = "serde_utils::quoted_u64")]
safe_slots_to_update_justified: u64,
#[serde(
deserialize_with = "u32_from_hex_str",
serialize_with = "u32_to_hex_str"
)]
#[serde(with = "serde_utils::u32_hex")]
domain_beacon_proposer: u32,
#[serde(
deserialize_with = "u32_from_hex_str",
serialize_with = "u32_to_hex_str"
)]
#[serde(with = "serde_utils::u32_hex")]
domain_beacon_attester: u32,
#[serde(
deserialize_with = "u32_from_hex_str",
serialize_with = "u32_to_hex_str"
)]
#[serde(with = "serde_utils::u32_hex")]
domain_randao: u32,
#[serde(
deserialize_with = "u32_from_hex_str",
serialize_with = "u32_to_hex_str"
)]
#[serde(with = "serde_utils::u32_hex")]
domain_deposit: u32,
#[serde(
deserialize_with = "u32_from_hex_str",
serialize_with = "u32_to_hex_str"
)]
#[serde(with = "serde_utils::u32_hex")]
domain_voluntary_exit: u32,
#[serde(
deserialize_with = "u32_from_hex_str",
serialize_with = "u32_to_hex_str"
)]
#[serde(with = "serde_utils::u32_hex")]
domain_selection_proof: u32,
#[serde(
deserialize_with = "u32_from_hex_str",
serialize_with = "u32_to_hex_str"
)]
#[serde(with = "serde_utils::u32_hex")]
domain_aggregate_and_proof: u32,
// EthSpec
#[serde(with = "serde_utils::quoted_u32")]
max_validators_per_committee: u32,
#[serde(with = "serde_utils::quoted_u64")]
slots_per_epoch: u64,
#[serde(with = "serde_utils::quoted_u64")]
epochs_per_eth1_voting_period: u64,
slots_per_historical_root: usize,
epochs_per_historical_vector: usize,
epochs_per_slashings_vector: usize,
#[serde(with = "serde_utils::quoted_u64")]
slots_per_historical_root: u64,
#[serde(with = "serde_utils::quoted_u64")]
epochs_per_historical_vector: u64,
#[serde(with = "serde_utils::quoted_u64")]
epochs_per_slashings_vector: u64,
#[serde(with = "serde_utils::quoted_u64")]
historical_roots_limit: u64,
#[serde(with = "serde_utils::quoted_u64")]
validator_registry_limit: u64,
#[serde(with = "serde_utils::quoted_u32")]
max_proposer_slashings: u32,
#[serde(with = "serde_utils::quoted_u32")]
max_attester_slashings: u32,
#[serde(with = "serde_utils::quoted_u32")]
max_attestations: u32,
#[serde(with = "serde_utils::quoted_u32")]
max_deposits: u32,
#[serde(with = "serde_utils::quoted_u32")]
max_voluntary_exits: u32,
// Validator
#[serde(with = "serde_utils::quoted_u64")]
eth1_follow_distance: u64,
#[serde(with = "serde_utils::quoted_u64")]
target_aggregators_per_committee: u64,
#[serde(with = "serde_utils::quoted_u64")]
random_subnets_per_validator: u64,
#[serde(with = "serde_utils::quoted_u64")]
epochs_per_random_subnet_subscription: u64,
#[serde(with = "serde_utils::quoted_u64")]
seconds_per_eth1_block: u64,
deposit_contract_address: Address,
/* TODO: incorporate these into ChainSpec and turn on `serde(deny_unknown_fields)`
deposit_chain_id: u64,
deposit_network_id: u64,
deposit_contract_address: String,
*/
}
@@ -568,8 +588,8 @@ impl YamlConfig {
Self {
config_name: T::spec_name().to_string(),
// ChainSpec
max_committees_per_slot: spec.max_committees_per_slot,
target_committee_size: spec.target_committee_size,
max_committees_per_slot: spec.max_committees_per_slot as u64,
target_committee_size: spec.target_committee_size as u64,
min_per_epoch_churn_limit: spec.min_per_epoch_churn_limit,
churn_limit_quotient: spec.churn_limit_quotient,
shuffle_round_count: spec.shuffle_round_count,
@@ -611,9 +631,9 @@ impl YamlConfig {
max_validators_per_committee: T::MaxValidatorsPerCommittee::to_u32(),
slots_per_epoch: T::slots_per_epoch(),
epochs_per_eth1_voting_period: T::EpochsPerEth1VotingPeriod::to_u64(),
slots_per_historical_root: T::slots_per_historical_root(),
epochs_per_historical_vector: T::epochs_per_historical_vector(),
epochs_per_slashings_vector: T::EpochsPerSlashingsVector::to_usize(),
slots_per_historical_root: T::slots_per_historical_root() as u64,
epochs_per_historical_vector: T::epochs_per_historical_vector() as u64,
epochs_per_slashings_vector: T::EpochsPerSlashingsVector::to_u64(),
historical_roots_limit: T::HistoricalRootsLimit::to_u64(),
validator_registry_limit: T::ValidatorRegistryLimit::to_u64(),
max_proposer_slashings: T::MaxProposerSlashings::to_u32(),
@@ -628,6 +648,7 @@ impl YamlConfig {
random_subnets_per_validator: spec.random_subnets_per_validator,
epochs_per_random_subnet_subscription: spec.epochs_per_random_subnet_subscription,
seconds_per_eth1_block: spec.seconds_per_eth1_block,
deposit_contract_address: spec.deposit_contract_address,
}
}
@@ -643,9 +664,9 @@ impl YamlConfig {
if self.max_validators_per_committee != T::MaxValidatorsPerCommittee::to_u32()
|| self.slots_per_epoch != T::slots_per_epoch()
|| self.epochs_per_eth1_voting_period != T::EpochsPerEth1VotingPeriod::to_u64()
|| self.slots_per_historical_root != T::slots_per_historical_root()
|| self.epochs_per_historical_vector != T::epochs_per_historical_vector()
|| self.epochs_per_slashings_vector != T::EpochsPerSlashingsVector::to_usize()
|| self.slots_per_historical_root != T::slots_per_historical_root() as u64
|| self.epochs_per_historical_vector != T::epochs_per_historical_vector() as u64
|| self.epochs_per_slashings_vector != T::EpochsPerSlashingsVector::to_u64()
|| self.historical_roots_limit != T::HistoricalRootsLimit::to_u64()
|| self.validator_registry_limit != T::ValidatorRegistryLimit::to_u64()
|| self.max_proposer_slashings != T::MaxProposerSlashings::to_u32()
@@ -662,8 +683,8 @@ impl YamlConfig {
/*
* Misc
*/
max_committees_per_slot: self.max_committees_per_slot,
target_committee_size: self.target_committee_size,
max_committees_per_slot: self.max_committees_per_slot as usize,
target_committee_size: self.target_committee_size as usize,
min_per_epoch_churn_limit: self.min_per_epoch_churn_limit,
churn_limit_quotient: self.churn_limit_quotient,
shuffle_round_count: self.shuffle_round_count,
@@ -685,6 +706,7 @@ impl YamlConfig {
random_subnets_per_validator: self.random_subnets_per_validator,
epochs_per_random_subnet_subscription: self.epochs_per_random_subnet_subscription,
seconds_per_eth1_block: self.seconds_per_eth1_block,
deposit_contract_address: self.deposit_contract_address,
/*
* Gwei values
*/
consensus/types/src/deposit_data.rs
+1
View File
@@ -15,6 +15,7 @@ use tree_hash_derive::TreeHash;
pub struct DepositData {
pub pubkey: PublicKeyBytes,
pub withdrawal_credentials: Hash256,
#[serde(with = "serde_utils::quoted_u64")]
pub amount: u64,
pub signature: SignatureBytes,
}
consensus/types/src/deposit_message.rs
+1
View File
@@ -15,6 +15,7 @@ use tree_hash_derive::TreeHash;
pub struct DepositMessage {
pub pubkey: PublicKeyBytes,
pub withdrawal_credentials: Hash256,
#[serde(with = "serde_utils::quoted_u64")]
pub amount: u64,
}
consensus/types/src/enr_fork_id.rs
+2 -9
View File
@@ -1,5 +1,4 @@
use crate::test_utils::TestRandom;
use crate::utils::{fork_from_hex_str, fork_to_hex_str};
use crate::Epoch;
use serde_derive::{Deserialize, Serialize};
@@ -16,15 +15,9 @@ use tree_hash_derive::TreeHash;
Debug, Clone, PartialEq, Default, Serialize, Deserialize, Encode, Decode, TreeHash, TestRandom,
)]
pub struct EnrForkId {
#[serde(
serialize_with = "fork_to_hex_str",
deserialize_with = "fork_from_hex_str"
)]
#[serde(with = "serde_utils::bytes_4_hex")]
pub fork_digest: [u8; 4],
#[serde(
serialize_with = "fork_to_hex_str",
deserialize_with = "fork_from_hex_str"
)]
#[serde(with = "serde_utils::bytes_4_hex")]
pub next_fork_version: [u8; 4],
pub next_fork_epoch: Epoch,
}
consensus/types/src/eth1_data.rs
+1
View File
@@ -26,6 +26,7 @@ use tree_hash_derive::TreeHash;
)]
pub struct Eth1Data {
pub deposit_root: Hash256,
#[serde(with = "serde_utils::quoted_u64")]
pub deposit_count: u64,
pub block_hash: Hash256,
}
consensus/types/src/fork.rs
+2 -9
View File
@@ -1,5 +1,4 @@
use crate::test_utils::TestRandom;
use crate::utils::{fork_from_hex_str, fork_to_hex_str};
use crate::Epoch;
use serde_derive::{Deserialize, Serialize};
@@ -25,15 +24,9 @@ use tree_hash_derive::TreeHash;
TestRandom,
)]
pub struct Fork {
#[serde(
serialize_with = "fork_to_hex_str",
deserialize_with = "fork_from_hex_str"
)]
#[serde(with = "serde_utils::bytes_4_hex")]
pub previous_version: [u8; 4],
#[serde(
serialize_with = "fork_to_hex_str",
deserialize_with = "fork_from_hex_str"
)]
#[serde(with = "serde_utils::bytes_4_hex")]
pub current_version: [u8; 4],
pub epoch: Epoch,
}
consensus/types/src/fork_data.rs
+1 -5
View File
@@ -1,5 +1,4 @@
use crate::test_utils::TestRandom;
use crate::utils::{fork_from_hex_str, fork_to_hex_str};
use crate::{Hash256, SignedRoot};
use serde_derive::{Deserialize, Serialize};
@@ -15,10 +14,7 @@ use tree_hash_derive::TreeHash;
Debug, Clone, PartialEq, Default, Serialize, Deserialize, Encode, Decode, TreeHash, TestRandom,
)]
pub struct ForkData {
#[serde(
serialize_with = "fork_to_hex_str",
deserialize_with = "fork_from_hex_str"
)]
#[serde(with = "serde_utils::bytes_4_hex")]
pub current_version: [u8; 4],
pub genesis_validators_root: Hash256,
}
consensus/types/src/free_attestation.rs
+1
View File
@@ -9,5 +9,6 @@ use serde_derive::Serialize;
pub struct FreeAttestation {
pub data: AttestationData,
pub signature: Signature,
#[serde(with = "serde_utils::quoted_u64")]
pub validator_index: u64,
}
consensus/types/src/graffiti.rs
+132
View File
@@ -0,0 +1,132 @@
use crate::{
test_utils::{RngCore, TestRandom},
Hash256,
};
use regex::bytes::Regex;
use serde::{de::Error, Deserialize, Deserializer, Serialize, Serializer};
use ssz::{Decode, DecodeError, Encode};
use std::fmt;
use tree_hash::TreeHash;
pub const GRAFFITI_BYTES_LEN: usize = 32;
/// The 32-byte `graffiti` field on a beacon block.
#[derive(Default, Debug, PartialEq, Clone, Copy, Serialize, Deserialize)]
#[serde(transparent)]
#[cfg_attr(feature = "arbitrary-fuzz", derive(arbitrary::Arbitrary))]
pub struct Graffiti(#[serde(with = "serde_graffiti")] pub [u8; GRAFFITI_BYTES_LEN]);
impl Graffiti {
pub fn as_utf8_lossy(&self) -> String {
#[allow(clippy::invalid_regex)]
let re = Regex::new("\\p{C}").expect("graffiti regex is valid");
String::from_utf8_lossy(&re.replace_all(&self.0[..], &b""[..])).to_string()
}
}
impl fmt::Display for Graffiti {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", serde_utils::hex::encode(&self.0))
}
}
impl From<[u8; GRAFFITI_BYTES_LEN]> for Graffiti {
fn from(bytes: [u8; GRAFFITI_BYTES_LEN]) -> Self {
Self(bytes)
}
}
impl Into<[u8; GRAFFITI_BYTES_LEN]> for Graffiti {
fn into(self) -> [u8; GRAFFITI_BYTES_LEN] {
self.0
}
}
pub mod serde_graffiti {
use super::*;
pub fn serialize<S>(bytes: &[u8; GRAFFITI_BYTES_LEN], serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
serializer.serialize_str(&serde_utils::hex::encode(bytes))
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<[u8; GRAFFITI_BYTES_LEN], D::Error>
where
D: Deserializer<'de>,
{
let s: String = Deserialize::deserialize(deserializer)?;
let bytes = serde_utils::hex::decode(&s).map_err(D::Error::custom)?;
if bytes.len() != GRAFFITI_BYTES_LEN {
return Err(D::Error::custom(format!(
"incorrect byte length {}, expected {}",
bytes.len(),
GRAFFITI_BYTES_LEN
)));
}
let mut array = [0; GRAFFITI_BYTES_LEN];
array[..].copy_from_slice(&bytes);
Ok(array)
}
}
impl Encode for Graffiti {
fn is_ssz_fixed_len() -> bool {
<[u8; GRAFFITI_BYTES_LEN] as Encode>::is_ssz_fixed_len()
}
fn ssz_fixed_len() -> usize {
<[u8; GRAFFITI_BYTES_LEN] as Encode>::ssz_fixed_len()
}
fn ssz_bytes_len(&self) -> usize {
self.0.ssz_bytes_len()
}
fn ssz_append(&self, buf: &mut Vec<u8>) {
self.0.ssz_append(buf)
}
}
impl Decode for Graffiti {
fn is_ssz_fixed_len() -> bool {
<[u8; GRAFFITI_BYTES_LEN] as Decode>::is_ssz_fixed_len()
}
fn ssz_fixed_len() -> usize {
<[u8; GRAFFITI_BYTES_LEN] as Decode>::ssz_fixed_len()
}
fn from_ssz_bytes(bytes: &[u8]) -> Result<Self, DecodeError> {
<[u8; GRAFFITI_BYTES_LEN]>::from_ssz_bytes(bytes).map(Self)
}
}
impl TreeHash for Graffiti {
fn tree_hash_type() -> tree_hash::TreeHashType {
<[u8; GRAFFITI_BYTES_LEN]>::tree_hash_type()
}
fn tree_hash_packed_encoding(&self) -> Vec<u8> {
self.0.tree_hash_packed_encoding()
}
fn tree_hash_packing_factor() -> usize {
<[u8; GRAFFITI_BYTES_LEN]>::tree_hash_packing_factor()
}
fn tree_hash_root(&self) -> tree_hash::Hash256 {
self.0.tree_hash_root()
}
}
impl TestRandom for Graffiti {
fn random_for_test(rng: &mut impl RngCore) -> Self {
Self::from(Hash256::random_for_test(rng).to_fixed_bytes())
}
}
consensus/types/src/indexed_attestation.rs
+38
View File
@@ -18,6 +18,7 @@ use tree_hash_derive::TreeHash;
#[serde(bound = "T: EthSpec")]
pub struct IndexedAttestation<T: EthSpec> {
/// Lists validator registry indices, not committee indices.
#[serde(with = "quoted_variable_list_u64")]
pub attesting_indices: VariableList<u64, T::MaxValidatorsPerCommittee>,
pub data: AttestationData,
pub signature: AggregateSignature,
@@ -53,6 +54,43 @@ impl<T: EthSpec> Hash for IndexedAttestation<T> {
}
}
/// Serialize a variable list of `u64` such that each int is quoted. Deserialize a variable
/// list supporting both quoted and un-quoted ints.
///
/// E.g.,`["0", "1", "2"]`
mod quoted_variable_list_u64 {
use super::*;
use crate::Unsigned;
use serde::ser::SerializeSeq;
use serde::{Deserializer, Serializer};
use serde_utils::quoted_u64_vec::{QuotedIntVecVisitor, QuotedIntWrapper};
pub fn serialize<S, T>(value: &VariableList<u64, T>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
T: Unsigned,
{
let mut seq = serializer.serialize_seq(Some(value.len()))?;
for &int in value.iter() {
seq.serialize_element(&QuotedIntWrapper { int })?;
}
seq.end()
}
pub fn deserialize<'de, D, T>(deserializer: D) -> Result<VariableList<u64, T>, D::Error>
where
D: Deserializer<'de>,
T: Unsigned,
{
deserializer
.deserialize_any(QuotedIntVecVisitor)
.and_then(|vec| {
VariableList::new(vec)
.map_err(|e| serde::de::Error::custom(format!("invalid length: {:?}", e)))
})
}
}
#[cfg(test)]
mod tests {
use super::*;
consensus/types/src/lib.rs
+6 -2
View File
@@ -29,19 +29,21 @@ pub mod eth_spec;
pub mod fork;
pub mod fork_data;
pub mod free_attestation;
pub mod graffiti;
pub mod historical_batch;
pub mod indexed_attestation;
pub mod pending_attestation;
pub mod proposer_slashing;
pub mod relative_epoch;
pub mod selection_proof;
pub mod shuffling_id;
pub mod signed_aggregate_and_proof;
pub mod signed_beacon_block;
pub mod signed_beacon_block_header;
pub mod signed_voluntary_exit;
pub mod signing_data;
pub mod utils;
pub mod validator;
pub mod validator_subscription;
pub mod voluntary_exit;
#[macro_use]
pub mod slot_epoch_macros;
@@ -74,12 +76,14 @@ pub use crate::eth1_data::Eth1Data;
pub use crate::fork::Fork;
pub use crate::fork_data::ForkData;
pub use crate::free_attestation::FreeAttestation;
pub use crate::graffiti::{Graffiti, GRAFFITI_BYTES_LEN};
pub use crate::historical_batch::HistoricalBatch;
pub use crate::indexed_attestation::IndexedAttestation;
pub use crate::pending_attestation::PendingAttestation;
pub use crate::proposer_slashing::ProposerSlashing;
pub use crate::relative_epoch::{Error as RelativeEpochError, RelativeEpoch};
pub use crate::selection_proof::SelectionProof;
pub use crate::shuffling_id::ShufflingId;
pub use crate::signed_aggregate_and_proof::SignedAggregateAndProof;
pub use crate::signed_beacon_block::{SignedBeaconBlock, SignedBeaconBlockHash};
pub use crate::signed_beacon_block_header::SignedBeaconBlockHeader;
@@ -88,6 +92,7 @@ pub use crate::signing_data::{SignedRoot, SigningData};
pub use crate::slot_epoch::{Epoch, Slot};
pub use crate::subnet_id::SubnetId;
pub use crate::validator::Validator;
pub use crate::validator_subscription::ValidatorSubscription;
pub use crate::voluntary_exit::VoluntaryExit;
pub type CommitteeIndex = u64;
@@ -99,4 +104,3 @@ pub use bls::{
AggregateSignature, Keypair, PublicKey, PublicKeyBytes, SecretKey, Signature, SignatureBytes,
};
pub use ssz_types::{typenum, typenum::Unsigned, BitList, BitVector, FixedVector, VariableList};
pub use utils::{Graffiti, GRAFFITI_BYTES_LEN};
consensus/types/src/pending_attestation.rs
+2
View File
@@ -13,7 +13,9 @@ use tree_hash_derive::TreeHash;
pub struct PendingAttestation<T: EthSpec> {
pub aggregation_bits: BitList<T::MaxValidatorsPerCommittee>,
pub data: AttestationData,
#[serde(with = "serde_utils::quoted_u64")]
pub inclusion_delay: u64,
#[serde(with = "serde_utils::quoted_u64")]
pub proposer_index: u64,
}
consensus/types/src/shuffling_id.rs
+61
View File
@@ -0,0 +1,61 @@
use crate::*;
use serde_derive::{Deserialize, Serialize};
use ssz_derive::{Decode, Encode};
use std::hash::Hash;
/// Can be used to key (ID) the shuffling in some chain, in some epoch.
///
/// ## Reasoning
///
/// We say that the ID of some shuffling is always equal to a 2-tuple:
///
/// - The epoch for which the shuffling should be effective.
/// - A block root, where this is the root at the *last* slot of the penultimate epoch. I.e., the
/// final block which contributed a randao reveal to the seed for the shuffling.
///
/// The struct stores exactly that 2-tuple.
#[derive(Debug, PartialEq, Eq, Clone, Hash, Serialize, Deserialize, Encode, Decode)]
pub struct ShufflingId {
pub shuffling_epoch: Epoch,
shuffling_decision_block: Hash256,
}
impl ShufflingId {
/// Using the given `state`, return the shuffling id for the shuffling at the given
/// `relative_epoch`.
///
/// The `block_root` provided should be either:
///
/// - The root of the block which produced this state.
/// - If the state is from a skip slot, the root of the latest block in that state.
pub fn new<E: EthSpec>(
block_root: Hash256,
state: &BeaconState<E>,
relative_epoch: RelativeEpoch,
) -> Result<Self, BeaconStateError> {
let shuffling_epoch = relative_epoch.into_epoch(state.current_epoch());
let shuffling_decision_slot = shuffling_epoch
.saturating_sub(1_u64)
.start_slot(E::slots_per_epoch())
.saturating_sub(1_u64);
let shuffling_decision_block = if state.slot == shuffling_decision_slot {
block_root
} else {
*state.get_block_root(shuffling_decision_slot)?
};
Ok(Self {
shuffling_epoch,
shuffling_decision_block,
})
}
pub fn from_components(shuffling_epoch: Epoch, shuffling_decision_block: Hash256) -> Self {
Self {
shuffling_epoch,
shuffling_decision_block,
}
}
}
consensus/types/src/slot_epoch_macros.rs
+13
View File
@@ -313,6 +313,18 @@ macro_rules! impl_ssz {
};
}
macro_rules! impl_from_str {
($type: ident) => {
impl std::str::FromStr for $type {
type Err = std::num::ParseIntError;
fn from_str(s: &str) -> Result<$type, Self::Err> {
u64::from_str(s).map($type)
}
}
};
}
macro_rules! impl_common {
($type: ident) => {
impl_from_into_u64!($type);
@@ -328,6 +340,7 @@ macro_rules! impl_common {
impl_display!($type);
impl_debug!($type);
impl_ssz!($type);
impl_from_str!($type);
};
}
consensus/types/src/subnet_id.rs
+2 -1
View File
@@ -6,7 +6,8 @@ use std::ops::{Deref, DerefMut};
#[cfg_attr(feature = "arbitrary-fuzz", derive(arbitrary::Arbitrary))]
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct SubnetId(u64);
#[serde(transparent)]
pub struct SubnetId(#[serde(with = "serde_utils::quoted_u64")] u64);
impl SubnetId {
pub fn new(id: u64) -> Self {
consensus/types/src/utils.rs
-3
View File
@@ -1,3 +0,0 @@
mod serde_utils;
pub use self::serde_utils::*;
consensus/types/src/utils/serde_utils.rs
-134
View File
@@ -1,134 +0,0 @@
use serde::de::Error;
use serde::{Deserialize, Deserializer, Serializer};
pub const FORK_BYTES_LEN: usize = 4;
pub const GRAFFITI_BYTES_LEN: usize = 32;
/// Type for a slice of `GRAFFITI_BYTES_LEN` bytes.
///
/// Gets included inside each `BeaconBlockBody`.
pub type Graffiti = [u8; GRAFFITI_BYTES_LEN];
pub fn u8_from_hex_str<'de, D>(deserializer: D) -> Result<u8, D::Error>
where
D: Deserializer<'de>,
{
let s: String = Deserialize::deserialize(deserializer)?;
let start = match s.as_str().get(2..) {
Some(start) => start,
None => return Err(D::Error::custom("string length too small")),
};
u8::from_str_radix(&start, 16).map_err(D::Error::custom)
}
#[allow(clippy::trivially_copy_pass_by_ref)] // Serde requires the `byte` to be a ref.
pub fn u8_to_hex_str<S>(byte: &u8, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut hex: String = "0x".to_string();
hex.push_str(&hex::encode(&[*byte]));
serializer.serialize_str(&hex)
}
pub fn u32_from_hex_str<'de, D>(deserializer: D) -> Result<u32, D::Error>
where
D: Deserializer<'de>,
{
let s: String = Deserialize::deserialize(deserializer)?;
let start = s
.as_str()
.get(2..)
.ok_or_else(|| D::Error::custom("string length too small"))?;
u32::from_str_radix(&start, 16)
.map_err(D::Error::custom)
.map(u32::from_be)
}
#[allow(clippy::trivially_copy_pass_by_ref)] // Serde requires the `num` to be a ref.
pub fn u32_to_hex_str<S>(num: &u32, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut hex: String = "0x".to_string();
let bytes = num.to_le_bytes();
hex.push_str(&hex::encode(&bytes));
serializer.serialize_str(&hex)
}
pub fn fork_from_hex_str<'de, D>(deserializer: D) -> Result<[u8; FORK_BYTES_LEN], D::Error>
where
D: Deserializer<'de>,
{
let s: String = Deserialize::deserialize(deserializer)?;
let mut array = [0 as u8; FORK_BYTES_LEN];
let start = s
.as_str()
.get(2..)
.ok_or_else(|| D::Error::custom("string length too small"))?;
let decoded: Vec<u8> = hex::decode(&start).map_err(D::Error::custom)?;
if decoded.len() != FORK_BYTES_LEN {
return Err(D::Error::custom("Fork length too long"));
}
for (i, item) in array.iter_mut().enumerate() {
if i > decoded.len() {
break;
}
*item = decoded[i];
}
Ok(array)
}
#[allow(clippy::trivially_copy_pass_by_ref)]
pub fn fork_to_hex_str<S>(bytes: &[u8; FORK_BYTES_LEN], serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut hex_string: String = "0x".to_string();
hex_string.push_str(&hex::encode(&bytes));
serializer.serialize_str(&hex_string)
}
pub fn graffiti_to_hex_str<S>(bytes: &Graffiti, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut hex_string: String = "0x".to_string();
hex_string.push_str(&hex::encode(&bytes));
serializer.serialize_str(&hex_string)
}
pub fn graffiti_from_hex_str<'de, D>(deserializer: D) -> Result<Graffiti, D::Error>
where
D: Deserializer<'de>,
{
let s: String = Deserialize::deserialize(deserializer)?;
let mut array = Graffiti::default();
let start = s
.as_str()
.get(2..)
.ok_or_else(|| D::Error::custom("string length too small"))?;
let decoded: Vec<u8> = hex::decode(&start).map_err(D::Error::custom)?;
if decoded.len() > GRAFFITI_BYTES_LEN {
return Err(D::Error::custom("Fork length too long"));
}
for (i, item) in array.iter_mut().enumerate() {
if i > decoded.len() {
break;
}
*item = decoded[i];
}
Ok(array)
}
consensus/types/src/validator_subscription.rs
+21
View File
@@ -0,0 +1,21 @@
use crate::*;
use serde::{Deserialize, Serialize};
use ssz_derive::{Decode, Encode};
/// A validator subscription, created when a validator subscribes to a slot to perform optional aggregation
/// duties.
#[derive(PartialEq, Debug, Serialize, Deserialize, Clone, Encode, Decode)]
pub struct ValidatorSubscription {
/// The validators index.
pub validator_index: u64,
/// The index of the committee within `slot` of which the validator is a member. Used by the
/// beacon node to quickly evaluate the associated `SubnetId`.
pub attestation_committee_index: CommitteeIndex,
/// The slot in which to subscribe.
pub slot: Slot,
/// Committee count at slot to subscribe.
pub committee_count_at_slot: u64,
/// If true, the validator is an aggregator and the beacon node should aggregate attestations
/// for this slot.
pub is_aggregator: bool,
}
consensus/types/src/voluntary_exit.rs
+1
View File
@@ -16,6 +16,7 @@ use tree_hash_derive::TreeHash;
pub struct VoluntaryExit {
/// Earliest epoch when voluntary exit can be processed.
pub epoch: Epoch,
#[serde(with = "serde_utils::quoted_u64")]
pub validator_index: u64,
}