From 55f977902151b6a56260dc63bbad54421716d8ee Mon Sep 17 00:00:00 2001 From: Chris Natoli Date: Mon, 7 Jan 2019 11:52:55 +1100 Subject: [PATCH 01/35] Rewrite goals --- README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index e28855504..669d26a8b 100644 --- a/README.md +++ b/README.md @@ -33,9 +33,10 @@ present-Ethereum functionality. ### Goals -The purpose of this project is to further research and development towards a -secure, efficient, and decentralized Ethereum protocol, facilitated by a new -open-source Ethereum Serenity client. +The purpose of this project is to work alongside the Ethereum community and +implement a secure, open-source Ethereum Serenity client in Rust. With the +focus on security, the goal for Lighthouse is to provide a trustworthy, well +tested, secure client that is resilient to any errors. In addition to implementing a new client, the project seeks to maintain and improve the Ethereum protocol wherever possible. From 0343cd1cbfa61961d43a67e5df6172e709a3129f Mon Sep 17 00:00:00 2001 From: Chris Natoli Date: Mon, 7 Jan 2019 11:52:15 +1100 Subject: [PATCH 02/35] Rework goals; Add categories and more content --- README.md | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 669d26a8b..d4ad0957a 100644 --- a/README.md +++ b/README.md @@ -33,10 +33,28 @@ present-Ethereum functionality. ### Goals -The purpose of this project is to work alongside the Ethereum community and -implement a secure, open-source Ethereum Serenity client in Rust. With the -focus on security, the goal for Lighthouse is to provide a trustworthy, well -tested, secure client that is resilient to any errors. +The purpose of this project is to work alongside the Ethereum community to +implement a secure, trustworthy, open-source Ethereum Serenity client in Rust. + +* **Security**: Lighthouse's main goal is to implement everything with a +security-first mindset. The goal is to ensure that all components of lighthouse +are thoroughly tested, checked and secure. + +* **Trust** : As Ethereum Serenity is a Proof-of-Stake system, which +involves the interaction of the Ethereum protocol and user funds. Thus, a goal +of Lighthouse is to provide a client that is trustworthy. + + All code can be tested and verified the goal of Lighthouse is to provide code +that is trusted. + +* **Transparency**: Lighthouse aims at being as transparent as possible. This goal is for +Lighthouse to embrace the open-source community and allow for all to understand +the decisions, direction and changes in all aspects. + +* **Error Resilience**: As Lighthouse embraces the "never `panic`" mindset, the +goal is to be resilient to errors that may occur. Providing a client that has +tolerance against errors provides further properties for a secure, trustworthy +client that Lighthouse aims to provide. In addition to implementing a new client, the project seeks to maintain and improve the Ethereum protocol wherever possible. From 450ac2b7f9a0d2efc1317ee81389165a836b7d71 Mon Sep 17 00:00:00 2001 From: Chris Natoli Date: Mon, 7 Jan 2019 13:56:07 +1100 Subject: [PATCH 03/35] Minor structure change; Nitpicks; --- README.md | 69 ++++++++++++++++++++++++++++--------------------------- 1 file changed, 35 insertions(+), 34 deletions(-) diff --git a/README.md b/README.md index d4ad0957a..8e85c9cf4 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ If you'd like some background on Sigma Prime, please see the [Lighthouse Update ## Lighthouse Client Lighthouse is an open-source Ethereum Serenity client that is currently under -development. Designed as a Serenity-only client, Lighthouse will not +development. Designed as a Serenity-only client, Lighthouse will not re-implement the existing proof-of-work protocol. Maintaining a forward-focus on Ethereum Serenity ensures that Lighthouse avoids reproducing the high-quality work already undertaken by existing projects. As such, Lighthouse will connect @@ -76,12 +76,11 @@ by the team: is more resource intensive than proof-of-work. As such, clients need to ensure that bad blocks can be rejected as efficiently as possible. At present, blocks having 10 million ETH staked can be processed in 0.006 - seconds, and invalid blocks are rejected even more quickly. See [issue - #103](https://github.com/ethereum/beacon_chain/issues/103) on + seconds, and invalid blocks are rejected even more quickly. See + [issue #103](https://github.com/ethereum/beacon_chain/issues/103) on [ethereum/beacon_chain](https://github.com/ethereum/beacon_chain). -. - **P2P networking**: Serenity will likely use the [libp2p - framework](https://libp2p.io/). Lighthouse aims to work alongside + framework](https://libp2p.io/). Lighthouse is working alongside [Parity](https://www.parity.io/) to ensure [libp2p-rust](https://github.com/libp2p/rust-libp2p) is fit-for-purpose. - **Validator duties** : The project involves development of "validator @@ -96,9 +95,10 @@ implementation](https://github.com/sigp/lighthouse/tree/master/beacon_chain/util and this [research](https://github.com/sigp/serialization_sandbox/blob/report/report/serialization_report.md) on serialization formats for more information. -- **Casper FFG fork-choice**: The [Casper - FFG](https://arxiv.org/abs/1710.09437) fork-choice rules allow the chain to -select a canonical chain in the case of a fork. +- **Fork-choice**: The current fork choice rule is +[*LMD Ghost*](https://vitalik.ca/general/2018/12/05/cbc_casper.html#lmd-ghost), +which effectively takes the latest messages and forms the canonical chain using +the [GHOST](https://eprint.iacr.org/2013/881.pdf) mechanism. - **Efficient state transition logic**: State transition logic governs updates to the validator set as validators log in/out, penalizes/rewards validators, rotates validators across shards, and implements other core tasks. @@ -109,32 +109,6 @@ In addition to these components we are also working on database schemas, RPC frameworks, specification development, database optimizations (e.g., bloom-filters), and tons of other interesting stuff (at least we think so). -### Contributing - -**Lighthouse welcomes contributors with open-arms.** - -Layer-1 infrastructure is a critical component for the ecosystem and relies -heavily on contributions from the community. Building Ethereum Serenity is a huge -task and we refuse to conduct an inappropriate ICO or charge licensing fees. -Instead, we fund development through grants and support from Sigma Prime. - -If you would like to learn more about Ethereum Serenity and/or -[Rust](https://www.rust-lang.org/), we are more than happy to on-board you -and assign you some tasks. We aim to be as accepting and understanding as -possible; we are more than happy to up-skill contributors in exchange for their -assistance with the project. - -Alternatively, if you are an ETH/Rust veteran, we'd love your input. We're -always looking for the best way to implement things and welcome all -respectful criticisms. - -If you'd like to contribute, try having a look through the [open -issues](https://github.com/sigp/lighthouse/issues) (tip: look for the [good -first -issue](https://github.com/sigp/lighthouse/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) -tag) and ping us on the [gitter](https://gitter.im/sigp/lighthouse) channel. We need -your support! - ### Running **NOTE: The cryptography libraries used in this implementation are @@ -167,6 +141,33 @@ A few basic steps are needed to get set up: Lighthouse presently runs on Rust `stable`, however, benchmarks currently require the `nightly` version. + +### Contributing + +**Lighthouse welcomes contributors with open-arms.** + +Layer-1 infrastructure is a critical component for the ecosystem and relies +heavily on contributions from the community. Building Ethereum Serenity is a huge +task and we refuse to conduct an inappropriate ICO or charge licensing fees. +Instead, we fund development through grants and support from Sigma Prime. + +If you would like to learn more about Ethereum Serenity and/or +[Rust](https://www.rust-lang.org/), we are more than happy to on-board you +and assign you some tasks. We aim to be as accepting and understanding as +possible; we are more than happy to up-skill contributors in exchange for their +assistance with the project. + +Alternatively, if you are an ETH/Rust veteran, we'd love your input. We're +always looking for the best way to implement things and welcome all +respectful criticisms. + +If you'd like to contribute, try having a look through the [open +issues](https://github.com/sigp/lighthouse/issues) (tip: look for the [good +first +issue](https://github.com/sigp/lighthouse/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) +tag) and ping us on the [gitter](https://gitter.im/sigp/lighthouse) channel. We need +your support! + ### Engineering Ethos Lighthouse aims to produce many small easily-tested components, each separated From 27c6d14a612a7865a35191aabada54868424380b Mon Sep 17 00:00:00 2001 From: Chris Natoli Date: Mon, 7 Jan 2019 15:14:48 +1100 Subject: [PATCH 04/35] Add serenity doc; Move info from readme/onboarding into serenity doc; --- docs/Serenity.md | 161 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 161 insertions(+) create mode 100644 docs/Serenity.md diff --git a/docs/Serenity.md b/docs/Serenity.md new file mode 100644 index 000000000..9a190449f --- /dev/null +++ b/docs/Serenity.md @@ -0,0 +1,161 @@ +# Ethereum Serenity + +This document aims at providing a high level understanding of Ethereum and the +Serenity phase of the Ethereum roadmap. + +## The Blockchain + +A blockchain can be seen as a decentralized, distributed ledger. The ledger of +transactions is replicated onto all nodes in the network. When a transaction +occurs, it is first propagated to the nodes. Once the nodes receive the +transaction, and verifies the correctness, the nodes attempt to batch the +transactions into a block and append the block to the ledger. Once the ledger +has been successfully appended onto, they propagate the block to the network. +If accepted, this block now becomes the latest block in the chain. If two people +propose a block at the same time, the one canonical blockchain forks. At this +point it must be resolved, and each system has it's own way of resolving these +forks. + +![Blockchain](http://yuml.me/b0d6b30a.jpg) +
Figure 1. Example blockchain with a resolved fork.
+ +
+ +The idea of the blockchain was first proposed in the seminal [Bitcoin +whitepaper](https://bitcoin.org/bitcoin.pdf) by Satoshi Nakamoto. Since then, a +vast number of updates and blockchains have taken shape providing different +functionality or properties to the original blockchain. + +## What is Ethereum? + +Ethereum is an open blockchain protocol, allowing for the building and use of +decentralized applications that run on blockchain technology. Ethereum was one +of the initial platforms providing turing-complete code to be run on the +blockchain, allowing for conditional payments to occur through the use of this +code. Since then, Ethereum has advanced to allow for a number of Decentralized +Applications (DApps) to be developed and run completely with the blockchain as +the backbone. + +General Ethereum Introduction: + +* [What is Ethereum](http://ethdocs.org/en/latest/introduction/what-is-ethereum.html) +* [Ethereum Introduction](https://github.com/ethereum/wiki/wiki/Ethereum-introduction) + + +### Proof-of-Work and the current state of Ethereum. + +Currently, Ethereum is based on the Proof-of-Work model, a Sybil resilient +mechanism to allow nodes to propose blocks to the network. Although it provides +properties that allow the blockchain to operate in an open, public +(permissionless) network, it faces it's challenges and as a result impacts +the operation of the blockchain. + +The main goals to advance Ethereum is to (1) increase the scalability and +overall transaction processing power of the Ethereum world computer and (2) +find a suitable replacement for Proof-of-Work that still provides the necessary +properties that we need. + +* [Proof-of-Work in Cryptocurrencies: an accessible introduction](https://blog.sigmaprime.io/what-is-proof-of-work.html) + +## Serenity + +Ethereum Serenity refers to a new blockchain system currently under development +by the Ethereum Foundation and the Ethereum community. + +As part of the original Ethereum roadmap +[\[1\]](https://blog.ethereum.org/2015/03/03/ethereum-launch-process/) +[\[2\]](http://ethdocs.org/en/latest/introduction/the-homestead-release.html), +the Proof-of-Stake integration falls under **Release Step 4: *Serenity***. With +this, a number of changes are to be made to the current Ethereum protocol to +incorporate some of the new Proof-of-Stake mechanisms as well as improve on +some of the hindrances faced by the current Proof-of-Work chain. + +To now advance the current Ethereum, the decision is made to move to a sharded +Beacon chain structure where multiple shard-chains will be operating and +interacting with a central beacon chain.The Serenity blockchain consists of +1,025 proof-of-stake blockchains. This includes the "beacon chain" and 1,024 +"shard chains". + +Ethereum Serenity is also known as "Ethereum 2.0" and "Shasper". We prefer +Serenity as it more accurately reflects the established Ethereum roadmap (plus +we think it's a nice name). + +(Be mindful, the specifications change occasionally, so check these to keep up +to date) + +* Current Specifications: + * [Danny Ryan's "State of the Spec"](https://notes.ethereum.org/s/BJEZWNoyE) (A nice summary of the current specifications) + * [Ethereum Serenity - Phase 0: Beacon Chain Spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md) + * [Ethereum Serenity - Phase 1: Sharded Data Chains](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/1_shard-data-chains.md) + * [Beacon Chain - Vitalik Buterin and Justin Drake explain](https://www.youtube.com/watch?v=GAywmwGToUI) +* Understanding Sharding: + * [Prysmatic Labs: Sharding Explained](https://medium.com/prysmatic-labs/how-to-scale-ethereum-sharding-explained-ba2e283b7fce) +* Other relevant resources + * [Proof of Stake - Casper FFG](https://www.youtube.com/watch?v=uQ3IqLDf-oo) + * [Justin Drake VDF Devcon4 Talk](https://www.youtube.com/watch?v=zqL_cMlPjOI) + + +### Beacon Chain + +The concept of a beacon chain differs from existing blockchains, such as +Bitcoin and Ethereum, in that it doesn't process transactions per se. Instead, +it maintains a set of bonded (staked) validators and coordinates these to +provide services to a static set of *sub-blockchains* (i.e. shards). Each of +these shard blockchains processes normal transactions (e.g. "Transfer 5 ETH +from A to B") in parallel whilst deferring consensus mechanisms to the beacon +chain. + +Major services provided by the beacon chain to its shards include the following: + +- A source of entropy, likely using a [RANDAO + VDF + scheme](https://ethresear.ch/t/minimal-vdf-randomness-beacon/3566). +- Validator management, including: + - Inducting and ejecting validators. + - Assigning randomly-shuffled subsets of validators to particular shards. + - Penalizing and rewarding validators. +- Proof-of-stake consensus for shard chain blocks. + +### Shard Chains + +Shards are analogous to CPU cores - they're a resource where transactions can +execute in series (one-after-another). Presently, Ethereum is single-core and +can only _fully_ process one transaction at a time. Sharding allows processing +of multiple transactions simultaneously, greatly increasing the per-second +transaction capacity of Ethereum. + +Each shard uses a proof-of-stake consensus mechanism and shares its validators +(stakers) with other shards. The beacon chain rotates validators +pseudo-randomly between different shards. Shards will likely be the basis of +layer-2 transaction processing schemes, however, that is not in scope of this +discussion. + +### The Proof-of-Work Chain + +The present-Ethereum proof-of-work (PoW) chain will host a smart contract that +enables accounts to deposit 32 ETH, a BLS public key, and some [other +parameters](https://github.com/ethereum/eth2.0-specs/blob/master/specs/casper_sharding_v2.1.md#pow-chain-changes), +allowing them to become beacon chain validators. Each beacon chain will +reference a PoW block hash allowing PoW clients to use the beacon chain as a +source of [Casper FFG finality](https://arxiv.org/abs/1710.09437), if desired. + +It is a requirement that ETH can move freely between shard chains, as well as between +Serenity and present-Ethereum blockchains. The exact mechanics of these transfers remain +an active topic of research and their details are yet to be confirmed. + +## Serenity Progress + +Ethereum Serenity is not fully specified and a working implementation does not +yet exist. Some teams have demos available which indicate progress, but do not +constitute a complete product. We look forward to providing user functionality +once we are ready to provide a minimum-viable user experience. + +The work-in-progress specifications live in the +[ethereum/eth2.0-specs](https://github.com/ethereum/eth2.0-specs) repository. +There is active discussion about the specification in the +[ethereum/sharding](https://gitter.im/ethereum/sharding) gitter channel. A +proof-of-concept implementation in Python is available at +[ethereum/beacon_chain](https://github.com/ethereum/beacon_chain). + +Presently, the specification focuses almost exclusively on the beacon chain, +as it is the focus of current development efforts. Progress on shard chain +specification will soon follow. From 94a0725032136ddc32768ba2c62207929b8ab1f1 Mon Sep 17 00:00:00 2001 From: Chris Natoli Date: Mon, 7 Jan 2019 16:57:09 +1100 Subject: [PATCH 05/35] Add 'lighthouse.md'; Update structure; Move info into relevant files. --- README.md | 158 +++--------------------------- docs/lighthouse.md | 83 ++++++++++++++++ docs/onboarding.md | 102 +++---------------- docs/{Serenity.md => serenity.md} | 0 4 files changed, 112 insertions(+), 231 deletions(-) create mode 100644 docs/lighthouse.md rename docs/{Serenity.md => serenity.md} (100%) diff --git a/README.md b/README.md index 8e85c9cf4..de0193705 100644 --- a/README.md +++ b/README.md @@ -31,33 +31,6 @@ to existing clients, such as [Parity-Ethereum](https://github.com/paritytech/parity-ethereum), via RPC to enable present-Ethereum functionality. -### Goals - -The purpose of this project is to work alongside the Ethereum community to -implement a secure, trustworthy, open-source Ethereum Serenity client in Rust. - -* **Security**: Lighthouse's main goal is to implement everything with a -security-first mindset. The goal is to ensure that all components of lighthouse -are thoroughly tested, checked and secure. - -* **Trust** : As Ethereum Serenity is a Proof-of-Stake system, which -involves the interaction of the Ethereum protocol and user funds. Thus, a goal -of Lighthouse is to provide a client that is trustworthy. - - All code can be tested and verified the goal of Lighthouse is to provide code -that is trusted. - -* **Transparency**: Lighthouse aims at being as transparent as possible. This goal is for -Lighthouse to embrace the open-source community and allow for all to understand -the decisions, direction and changes in all aspects. - -* **Error Resilience**: As Lighthouse embraces the "never `panic`" mindset, the -goal is to be resilient to errors that may occur. Providing a client that has -tolerance against errors provides further properties for a secure, trustworthy -client that Lighthouse aims to provide. - -In addition to implementing a new client, the project seeks to maintain and -improve the Ethereum protocol wherever possible. ### Components @@ -109,6 +82,16 @@ In addition to these components we are also working on database schemas, RPC frameworks, specification development, database optimizations (e.g., bloom-filters), and tons of other interesting stuff (at least we think so). +### Directory Structure + +Here we provide an overview of the directory structure: + +- `/beacon_chain`: contains logic derived directly from the specification. + E.g., shuffling algorithms, state transition logic and structs, block +validation, BLS crypto, etc. +- `/lighthouse`: contains logic specific to this client implementation. E.g., + CLI parsing, RPC end-points, databases, etc. + ### Running **NOTE: The cryptography libraries used in this implementation are @@ -141,16 +124,10 @@ A few basic steps are needed to get set up: Lighthouse presently runs on Rust `stable`, however, benchmarks currently require the `nightly` version. - ### Contributing **Lighthouse welcomes contributors with open-arms.** -Layer-1 infrastructure is a critical component for the ecosystem and relies -heavily on contributions from the community. Building Ethereum Serenity is a huge -task and we refuse to conduct an inappropriate ICO or charge licensing fees. -Instead, we fund development through grants and support from Sigma Prime. - If you would like to learn more about Ethereum Serenity and/or [Rust](https://www.rust-lang.org/), we are more than happy to on-board you and assign you some tasks. We aim to be as accepting and understanding as @@ -161,6 +138,9 @@ Alternatively, if you are an ETH/Rust veteran, we'd love your input. We're always looking for the best way to implement things and welcome all respectful criticisms. +If you are looking to contribute, please head to our +[onboarding documentation](https://github.com/sigp/lighthouse/blob/master/docs/onboarding.md). + If you'd like to contribute, try having a look through the [open issues](https://github.com/sigp/lighthouse/issues) (tip: look for the [good first @@ -168,123 +148,11 @@ issue](https://github.com/sigp/lighthouse/issues?q=is%3Aissue+is%3Aopen+label%3A tag) and ping us on the [gitter](https://gitter.im/sigp/lighthouse) channel. We need your support! -### Engineering Ethos - -Lighthouse aims to produce many small easily-tested components, each separated -into individual crates wherever possible. - -Generally, tests can be kept in the same file, as is typical in Rust. -Integration tests should be placed in the `tests` directory in the crate's -root. Particularity large (line-count) tests should be placed into a separate -file. - -A function is not considered complete until a test exists for it. We produce -tests to protect against regression (accidentally breaking things) and to -provide examples that help readers of the code base understand how functions -should (or should not) be used. - -Each pull request is to be reviewed by at least one "core developer" (i.e., -someone with write-access to the repository). This helps to ensure bugs are -detected, consistency is maintained, and responsibility of errors is dispersed. - -Discussion must be respectful and intellectual. Have fun and make jokes, but -always respect the limits of other people. - -### Directory Structure - -Here we provide an overview of the directory structure: - -- `/beacon_chain`: contains logic derived directly from the specification. - E.g., shuffling algorithms, state transition logic and structs, block -validation, BLS crypto, etc. -- `/lighthouse`: contains logic specific to this client implementation. E.g., - CLI parsing, RPC end-points, databases, etc. - ## Contact The best place for discussion is the [sigp/lighthouse gitter](https://gitter.im/sigp/lighthouse). Ping @paulhauner or @AgeManning to get the quickest response. - -# What is Ethereum Serenity - -Ethereum Serenity refers to a new blockchain system currently under development by -the Ethereum Foundation and the Ethereum community. The Serenity blockchain -consists of 1,025 proof-of-stake blockchains. This includes the "beacon chain" -and 1,024 "shard chains". - -Ethereum Serenity is also known as "Ethereum 2.0" and "Shasper". We prefer -Serenity as it more accurately reflects the established Ethereum roadmap (plus -we think it's a nice name). - -## Beacon Chain - -The concept of a beacon chain differs from existing blockchains, such as -Bitcoin and Ethereum, in that it doesn't process transactions per se. Instead, -it maintains a set of bonded (staked) validators and coordinates these to -provide services to a static set of *sub-blockchains* (i.e. shards). Each of -these shard blockchains processes normal transactions (e.g. "Transfer 5 ETH -from A to B") in parallel whilst deferring consensus mechanisms to the beacon -chain. - -Major services provided by the beacon chain to its shards include the following: - -- A source of entropy, likely using a [RANDAO + VDF - scheme](https://ethresear.ch/t/minimal-vdf-randomness-beacon/3566). -- Validator management, including: - - Inducting and ejecting validators. - - Assigning randomly-shuffled subsets of validators to particular shards. - - Penalizing and rewarding validators. -- Proof-of-stake consensus for shard chain blocks. - -## Shard Chains - -Shards are analogous to CPU cores - they're a resource where transactions can -execute in series (one-after-another). Presently, Ethereum is single-core and -can only _fully_ process one transaction at a time. Sharding allows processing -of multiple transactions simultaneously, greatly increasing the per-second -transaction capacity of Ethereum. - -Each shard uses a proof-of-stake consensus mechanism and shares its validators -(stakers) with other shards. The beacon chain rotates validators -pseudo-randomly between different shards. Shards will likely be the basis of -layer-2 transaction processing schemes, however, that is not in scope of this -discussion. - -## The Proof-of-Work Chain - -The present-Ethereum proof-of-work (PoW) chain will host a smart contract that -enables accounts to deposit 32 ETH, a BLS public key, and some [other -parameters](https://github.com/ethereum/eth2.0-specs/blob/master/specs/casper_sharding_v2.1.md#pow-chain-changes), -allowing them to become beacon chain validators. Each beacon chain will -reference a PoW block hash allowing PoW clients to use the beacon chain as a -source of [Casper FFG finality](https://arxiv.org/abs/1710.09437), if desired. - -It is a requirement that ETH can move freely between shard chains, as well as between -Serenity and present-Ethereum blockchains. The exact mechanics of these transfers remain -an active topic of research and their details are yet to be confirmed. - -## Ethereum Serenity Progress - -Ethereum Serenity is not fully specified and a working implementation does not yet -exist. Some teams have demos available which indicate progress, but do not -constitute a complete product. We look forward to providing user functionality -once we are ready to provide a minimum-viable user experience. - -The work-in-progress Serenity specification lives -[here](https://github.com/ethereum/eth2.0-specs/blob/master/specs/casper_sharding_v2.1.md) -in the [ethereum/eth2.0-specs](https://github.com/ethereum/eth2.0-specs) -repository. The spec is still in a draft phase, however there are several teams -basing their Serenity implementations upon it while the Ethereum Foundation research -team continue to fill in the gaps. There is active discussion about the specification in the -[ethereum/sharding](https://gitter.im/ethereum/sharding) gitter channel. A -proof-of-concept implementation in Python is available at -[ethereum/beacon_chain](https://github.com/ethereum/beacon_chain). - -Presently, the specification focuses almost exclusively on the beacon chain, -as it is the focus of current development efforts. Progress on shard chain -specification will soon follow. - # Donations If you support the cause, we could certainly use donations to help fund development: diff --git a/docs/lighthouse.md b/docs/lighthouse.md new file mode 100644 index 000000000..8ca2387f8 --- /dev/null +++ b/docs/lighthouse.md @@ -0,0 +1,83 @@ +# About Lighthouse + +## Goals + +The purpose of this project is to work alongside the Ethereum community to +implement a secure, trustworthy, open-source Ethereum Serenity client in Rust. + +* **Security**: Lighthouse's main goal is to implement everything with a +security-first mindset. The goal is to ensure that all components of lighthouse +are thoroughly tested, checked and secure. + +* **Trust** : As Ethereum Serenity is a Proof-of-Stake system, which +involves the interaction of the Ethereum protocol and user funds. Thus, a goal +of Lighthouse is to provide a client that is trustworthy. + + All code can be tested and verified the goal of Lighthouse is to provide code +that is trusted. + +* **Transparency**: Lighthouse aims at being as transparent as possible. This +goal is for Lighthouse to embrace the open-source community and allow for all +to understand the decisions, direction and changes in all aspects. + +* **Error Resilience**: As Lighthouse embraces the "never `panic`" mindset, the +goal is to be resilient to errors that may occur. Providing a client that has +tolerance against errors provides further properties for a secure, trustworthy +client that Lighthouse aims to provide. + +In addition to implementing a new client, the project seeks to maintain and +improve the Ethereum protocol wherever possible. + +## Ideology + +### Never Panic + +Lighthouse will be the gateway interacting with the Proof-of-Stake system +employed by Ethereum. This requires the validation and proposal of blocks +and extremely timely responses. As part of this, Lighthouse aims to ensure +the most uptime as possible, meaning minimising the amount of +exceptions and gracefully handling any issues. + +Rust's `panic` provides the ability to throw an exception and exit, this +will terminate the running processes. Thus, Lighthouse aims to use `panic` +as little as possible to minimise the possible termination cases. + +### Security First Mindset + +Lighthouse aims to provide a safe, secure Serenity client for the Ethereum +ecosystem. At each step of development, the aim is to have a security-first +mindset and always ensure you are following the safe, secure mindset. When +contributing to any part of the Lighthouse client, through any development, +always ensure you understand each aspect thoroughly and cover all potential +security considerations of your code. + +### Functions aren't completed until they are tested + +As part of the Security First mindset, we want to aim to cover as many distinct +cases. A function being developed is not considered "completed" until tests +exist for that function. The tests not only help show the correctness of the +function, but also provide a way for new developers to understand how the +function is to be called and how it works. + + +## Engineering Ethos + +Lighthouse aims to produce many small easily-tested components, each separated +into individual crates wherever possible. + +Generally, tests can be kept in the same file, as is typical in Rust. +Integration tests should be placed in the `tests` directory in the crate's +root. Particularity large (line-count) tests should be placed into a separate +file. + +A function is not considered complete until a test exists for it. We produce +tests to protect against regression (accidentally breaking things) and to +provide examples that help readers of the code base understand how functions +should (or should not) be used. + +Each pull request is to be reviewed by at least one "core developer" (i.e., +someone with write-access to the repository). This helps to ensure bugs are +detected, consistency is maintained, and responsibility of errors is dispersed. + +Discussion must be respectful and intellectual. Have fun and make jokes, but +always respect the limits of other people. diff --git a/docs/onboarding.md b/docs/onboarding.md index 90b38df97..1d0557f29 100644 --- a/docs/onboarding.md +++ b/docs/onboarding.md @@ -1,5 +1,7 @@ # Contributing to Lighthouse +[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/sigp/lighthouse?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) + Lighthouse is an open-source Ethereum Serenity client built in [Rust](https://www.rust-lang.org/). @@ -13,40 +15,15 @@ documentation, writing extra tests or developing components, all help is appreciated and your contributions will help not only the community but all the contributors. +Layer-1 infrastructure is a critical component for the ecosystem and relies +heavily on contributions from the community. Building Ethereum Serenity is a +huge task and we refuse to conduct an inappropriate ICO or charge licensing +fees. Instead, we fund development through grants and support from Sigma +Prime. + If you have any additional questions, please feel free to jump on the [gitter](https://gitter.im/sigp/lighthouse) and have a chat with all of us. -## Ideology - -### Never Panic - -Lighthouse will be the gateway interacting with the Proof-of-Stake system -employed by Ethereum. This requires the validation and proposal of blocks -and extremely timely responses. As part of this, Lighthouse aims to ensure -the most uptime as possible, meaning minimising the amount of -exceptions and gracefully handling any issues. - -Rust's `panic` provides the ability to throw an exception and exit, this -will terminate the running processes. Thus, Lighthouse aims to use `panic` -as little as possible to minimise the possible termination cases. - -### Security First Mindset - -Lighthouse aims to provide a safe, secure Serenity client for the Ethereum -ecosystem. At each step of development, the aim is to have a security-first -mindset and always ensure you are following the safe, secure mindset. When -contributing to any part of the Lighthouse client, through any development, -always ensure you understand each aspect thoroughly and cover all potential -security considerations of your code. - -### Functions aren't completed until they are tested - -As part of the Security First mindset, we want to aim to cover as many distinct -cases. A function being developed is not considered "completed" until tests -exist for that function. The tests not only help show the correctness of the -function, but also provide a way for new developers to understand how the -function is to be called and how it works. - ## Understanding Serenity Ethereum's Serenity is based on a Proof-of-Stake based sharded beacon chain. @@ -54,59 +31,12 @@ Ethereum's Serenity is based on a Proof-of-Stake based sharded beacon chain. (*If you don't know what that is, don't `panic`, that's what this documentation is for!* :smile:) -### Ethereum +Read through our [Understanding +Serenity](https://github.com/sigp/lighthouse/blob/master/docs/serenity.md) docs +to learn more! :smile: -Ethereum is an open blockchain protocol, allowing for the building and use of -decentralized applications that run on blockchain technology. The blockchain can -be seen as a decentralized, distributed ledger of transactions. - -General Ethereum Introduction: - -* [What is Ethereum](http://ethdocs.org/en/latest/introduction/what-is-ethereum.html) -* [Ethereum Introduction](https://github.com/ethereum/wiki/wiki/Ethereum-introduction) - - -### Proof-of-Work and the current state of Ethereum. - -Currently, Ethereum is based on the Proof-of-Work model, a Sybil resilient -mechanism to allow nodes to propose blocks to the network. Although it provides -properties that allow the blockchain to operate in an open, public -(permissionless) network, it faces it's challenges and as a result impacts -the operation of the blockchain. - -The main goals to advance Ethereum is to (1) increase the scalability and -overall transaction processing power of the Ethereum world computer and (2) -find a suitable replacement for Proof-of-Work that still provides the necessary -properties that we need. - -* [Proof-of-Work in Cryptocurrencies: an accessible introduction](https://blog.sigmaprime.io/what-is-proof-of-work.html) - -### Serenity - -As part of the original Ethereum roadmap -[\[1\]](https://blog.ethereum.org/2015/03/03/ethereum-launch-process/) -[\[2\]](http://ethdocs.org/en/latest/introduction/the-homestead-release.html), -the Proof-of-Stake integration falls under **Release Step 4:*Serenity***. With -this, a number of changes are to be made to the current Ethereum protocol to -incorporate some of the new Proof-of-Stake mechanisms as well as improve on -some of the hindrances faced by the current Proof-of-Work chain. - -To now advance the current Ethereum, the decision is made to move to a sharded -Beacon chain structure where multiple shard-chains will be operating and -interacting with a central beacon chain. - -(Be mindful, the specifications change occasionally, so check these to keep up -to date) - -* Current Specifications: - * [Danny Ryan's "State of the Spec"](https://notes.ethereum.org/s/BJEZWNoyE) (A nice summary of the current specifications) - * [Ethereum Serenity - Phase 0: Beacon Chain Spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md) - * [Ethereum Serenity - Phase 1: Sharded Data Chains](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/1_shard-data-chains.md) - * [Beacon Chain - Vitalik Buterin and Justin Drake explain](https://www.youtube.com/watch?v=GAywmwGToUI) -* Understanding Sharding: - * [Prysmatic Labs: Sharding Explained](https://medium.com/prysmatic-labs/how-to-scale-ethereum-sharding-explained-ba2e283b7fce) -* Other relevant resources - * [Proof of Stake - Casper FFG](https://www.youtube.com/watch?v=uQ3IqLDf-oo) +The document explains the necessary fundamentals for understanding Ethereum, +Proof-of-Stake and the Serenity we are working towards. ## Development Onboarding @@ -142,8 +72,7 @@ allows you to easily install versions of rust. $ curl https://sh.rustup.rs -sSf | sh ``` -**Windows (You need a bit more):** - +**Windows (You need a bit more):**J * Install the Visual Studio 2015 with C++ support * Install Rustup using: https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe * You can then use the ``VS2015 x64 Native Tools Command Prompt`` and run: @@ -161,7 +90,6 @@ handy for handling dependencies and helping to modularise your project better. *Note: If you've installed rust through rustup, you should have ``cargo`` installed.* - #### Rust Terminology When developing rust, you'll come across some terminology that differs to @@ -217,6 +145,8 @@ and intellectual. Have fun, but always respect the limits of other people. **Testing** +*"A function is not considered complete until tests exist for it."* + Generally, tests can be self-contained in the same file. Integration tests should be added into the ``tests/`` directory in the crate's **root**. diff --git a/docs/Serenity.md b/docs/serenity.md similarity index 100% rename from docs/Serenity.md rename to docs/serenity.md From cfb9a1a3ae36b28ba0630c278a0c94866eb76fd6 Mon Sep 17 00:00:00 2001 From: Chris Natoli Date: Mon, 7 Jan 2019 17:06:07 +1100 Subject: [PATCH 06/35] Update onboarding docs to point to new docs --- docs/onboarding.md | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/onboarding.md b/docs/onboarding.md index 1d0557f29..8af3b0a83 100644 --- a/docs/onboarding.md +++ b/docs/onboarding.md @@ -15,6 +15,9 @@ documentation, writing extra tests or developing components, all help is appreciated and your contributions will help not only the community but all the contributors. +We've bundled up our Goals, Ethos and Ideology into one document for you to +read through, please read our [About Lighthouse](lighthouse.md) docs. :smile: + Layer-1 infrastructure is a critical component for the ecosystem and relies heavily on contributions from the community. Building Ethereum Serenity is a huge task and we refuse to conduct an inappropriate ICO or charge licensing @@ -24,6 +27,20 @@ Prime. If you have any additional questions, please feel free to jump on the [gitter](https://gitter.im/sigp/lighthouse) and have a chat with all of us. +**Pre-reading Materials:** + +* [About Lighthouse](lighthouse.md) +* [Ethereum Serenity](serenity.md) + +**Repository** + +If you'd like to contribute, try having a look through the [open +issues](https://github.com/sigp/lighthouse/issues) (tip: look for the [good +first +issue](https://github.com/sigp/lighthouse/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) +tag) and ping us on the [gitter](https://gitter.im/sigp/lighthouse) channel. We need +your support! + ## Understanding Serenity Ethereum's Serenity is based on a Proof-of-Stake based sharded beacon chain. @@ -33,7 +50,7 @@ is for!* :smile:) Read through our [Understanding Serenity](https://github.com/sigp/lighthouse/blob/master/docs/serenity.md) docs -to learn more! :smile: +to learn more! :smile: (*unless you've already read it.*) The document explains the necessary fundamentals for understanding Ethereum, Proof-of-Stake and the Serenity we are working towards. @@ -72,7 +89,7 @@ allows you to easily install versions of rust. $ curl https://sh.rustup.rs -sSf | sh ``` -**Windows (You need a bit more):**J +**Windows (You need a bit more):** * Install the Visual Studio 2015 with C++ support * Install Rustup using: https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe * You can then use the ``VS2015 x64 Native Tools Command Prompt`` and run: From 973a3c1469513bd57a601285f0fe01e3427e2b2f Mon Sep 17 00:00:00 2001 From: Chris Natoli Date: Mon, 7 Jan 2019 17:13:34 +1100 Subject: [PATCH 07/35] Update readme with links to new docs; --- README.md | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index de0193705..bd52ef506 100644 --- a/README.md +++ b/README.md @@ -7,18 +7,6 @@ Chain, maintained by Sigma Prime. The "Serenity" project is also known as "Ethereum 2.0" or "Shasper". -## Introduction - -This readme is split into two major sections: - -- [Lighthouse Client](#lighthouse-client): information about this - implementation. -- [What is Ethereum Serenity](#what-is-ethereum-serenity): an introduction to Ethereum Serenity. - -If you'd like some background on Sigma Prime, please see the [Lighthouse Update -\#00](https://lighthouse.sigmaprime.io/update-00.html) blog post or the -[company website](https://sigmaprime.io). - ## Lighthouse Client Lighthouse is an open-source Ethereum Serenity client that is currently under @@ -31,6 +19,15 @@ to existing clients, such as [Parity-Ethereum](https://github.com/paritytech/parity-ethereum), via RPC to enable present-Ethereum functionality. +### Further Reading + +- [About Lighthouse](docs/lighthouse.md): Goals, Ideology and Ethos surrounding +this implementation. +- [What is Ethereum Serenity](docs/serenity.md): an introduction to Ethereum Serenity. + +If you'd like some background on Sigma Prime, please see the [Lighthouse Update +\#00](https://lighthouse.sigmaprime.io/update-00.html) blog post or the +[company website](https://sigmaprime.io). ### Components @@ -86,10 +83,10 @@ bloom-filters), and tons of other interesting stuff (at least we think so). Here we provide an overview of the directory structure: -- `/beacon_chain`: contains logic derived directly from the specification. +- `beacon_chain/`: contains logic derived directly from the specification. E.g., shuffling algorithms, state transition logic and structs, block validation, BLS crypto, etc. -- `/lighthouse`: contains logic specific to this client implementation. E.g., +- `lighthouse/'``: contains logic specific to this client implementation. E.g., CLI parsing, RPC end-points, databases, etc. ### Running From a4d202558f9c4f39c598b04cd5b232f85355e5f3 Mon Sep 17 00:00:00 2001 From: Chris Natoli Date: Mon, 7 Jan 2019 17:37:49 +1100 Subject: [PATCH 08/35] Fix bad syntax --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index bd52ef506..01ebf6853 100644 --- a/README.md +++ b/README.md @@ -86,7 +86,7 @@ Here we provide an overview of the directory structure: - `beacon_chain/`: contains logic derived directly from the specification. E.g., shuffling algorithms, state transition logic and structs, block validation, BLS crypto, etc. -- `lighthouse/'``: contains logic specific to this client implementation. E.g., +- `lighthouse/`: contains logic specific to this client implementation. E.g., CLI parsing, RPC end-points, databases, etc. ### Running From a2b23d6ae2051ee7d3ad0a1bf167ee4861907d99 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 14:57:34 -0600 Subject: [PATCH 09/35] run cargo fmt --- .../attestation_validation/src/lib.rs | 6 +- beacon_chain/genesis/src/lib.rs | 2 +- beacon_chain/types/src/test_utils/mod.rs | 10 +-- beacon_chain/types/src/validator_record.rs | 6 +- beacon_chain/utils/bls/src/lib.rs | 2 +- .../validator_induction/src/inductor.rs | 69 ++++++++++++------- beacon_chain/validator_induction/src/lib.rs | 4 +- lighthouse/beacon_chain/tests/chain_test.rs | 2 +- lighthouse/db/src/disk_db.rs | 3 +- .../db/src/stores/beacon_state_store.rs | 4 +- lighthouse/main.rs | 8 ++- 11 files changed, 72 insertions(+), 44 deletions(-) diff --git a/beacon_chain/attestation_validation/src/lib.rs b/beacon_chain/attestation_validation/src/lib.rs index 254d6134e..825371ed0 100644 --- a/beacon_chain/attestation_validation/src/lib.rs +++ b/beacon_chain/attestation_validation/src/lib.rs @@ -14,9 +14,9 @@ mod justified_slot; mod shard_block; mod signature; -pub use crate::enums::{Invalid, Outcome, Error}; pub use crate::block_inclusion::validate_attestation_for_block; -pub use crate::justified_slot::validate_attestation_justified_slot; +pub use crate::enums::{Error, Invalid, Outcome}; pub use crate::justified_block::validate_attestation_justified_block_hash; -pub use crate::signature::validate_attestation_signature; +pub use crate::justified_slot::validate_attestation_justified_slot; pub use crate::shard_block::validate_attestation_data_shard_block_hash; +pub use crate::signature::validate_attestation_signature; diff --git a/beacon_chain/genesis/src/lib.rs b/beacon_chain/genesis/src/lib.rs index e590fa8be..6f45863aa 100644 --- a/beacon_chain/genesis/src/lib.rs +++ b/beacon_chain/genesis/src/lib.rs @@ -3,8 +3,8 @@ extern crate types; extern crate validator_induction; extern crate validator_shuffling; -mod beacon_state; mod beacon_block; +mod beacon_state; pub use crate::beacon_block::genesis_beacon_block; pub use crate::beacon_state::{genesis_beacon_state, Error as GenesisError}; diff --git a/beacon_chain/types/src/test_utils/mod.rs b/beacon_chain/types/src/test_utils/mod.rs index f3e382e6e..eb54f2a53 100644 --- a/beacon_chain/types/src/test_utils/mod.rs +++ b/beacon_chain/types/src/test_utils/mod.rs @@ -6,12 +6,13 @@ pub mod address; pub mod aggregate_signature; pub mod bitfield; pub mod hash256; -pub mod signature; -pub mod secret_key; pub mod public_key; +pub mod secret_key; +pub mod signature; pub trait TestRandom -where T: RngCore +where + T: RngCore, { fn random_for_test(rng: &mut T) -> Self; } @@ -35,7 +36,8 @@ impl TestRandom for usize { } impl TestRandom for Vec -where U: TestRandom +where + U: TestRandom, { fn random_for_test(rng: &mut T) -> Self { vec![ diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index 5f74d28d2..3f4a43261 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -1,5 +1,5 @@ use super::bls::PublicKey; -use super::{Hash256}; +use super::Hash256; use crate::test_utils::TestRandom; use rand::RngCore; use ssz::{Decodable, DecodeError, Encodable, SszStream}; @@ -40,7 +40,7 @@ pub struct ValidatorRecord { pub exit_count: u64, pub custody_commitment: Hash256, pub latest_custody_reseed_slot: u64, - pub penultimate_custody_reseed_slot: u64 + pub penultimate_custody_reseed_slot: u64, } impl ValidatorRecord { @@ -132,7 +132,7 @@ impl Decodable for ValidatorRecord { exit_count, custody_commitment, latest_custody_reseed_slot, - penultimate_custody_reseed_slot + penultimate_custody_reseed_slot, }, i, )) diff --git a/beacon_chain/utils/bls/src/lib.rs b/beacon_chain/utils/bls/src/lib.rs index 9d27d870b..d7a3ff15d 100644 --- a/beacon_chain/utils/bls/src/lib.rs +++ b/beacon_chain/utils/bls/src/lib.rs @@ -19,8 +19,8 @@ pub use self::bls_aggregates::AggregatePublicKey; pub const BLS_AGG_SIG_BYTE_SIZE: usize = 97; use hashing::canonical_hash; -use std::default::Default; use ssz::ssz_encode; +use std::default::Default; fn extend_if_needed(hash: &mut Vec) { // NOTE: bls_aggregates crate demands 48 bytes, this may be removed as we get closer to production diff --git a/beacon_chain/validator_induction/src/inductor.rs b/beacon_chain/validator_induction/src/inductor.rs index 720e38fa4..f00c9ec55 100644 --- a/beacon_chain/validator_induction/src/inductor.rs +++ b/beacon_chain/validator_induction/src/inductor.rs @@ -1,19 +1,19 @@ -use bls::{verify_proof_of_possession}; -use types::{BeaconState, Deposit, ValidatorRecord, ValidatorStatus}; +use bls::verify_proof_of_possession; use spec::ChainSpec; +use types::{BeaconState, Deposit, ValidatorRecord, ValidatorStatus}; #[derive(Debug, PartialEq, Clone)] pub enum ValidatorInductionError { InvalidShard, InvaidProofOfPossession, - InvalidWithdrawalCredentials + InvalidWithdrawalCredentials, } pub fn process_deposit( state: &mut BeaconState, deposit: &Deposit, - spec: &ChainSpec) --> Result { + spec: &ChainSpec, +) -> Result { let deposit_input = &deposit.deposit_data.deposit_input; let deposit_data = &deposit.deposit_data; @@ -22,18 +22,22 @@ pub fn process_deposit( return Err(ValidatorInductionError::InvaidProofOfPossession); } - let validator_index = state.validator_registry.iter() + let validator_index = state + .validator_registry + .iter() .position(|validator| validator.pubkey == deposit_input.pubkey); match validator_index { Some(i) => { - if state.validator_registry[i].withdrawal_credentials == deposit_input.withdrawal_credentials { + if state.validator_registry[i].withdrawal_credentials + == deposit_input.withdrawal_credentials + { state.validator_balances[i] += deposit_data.value; return Ok(i); } Err(ValidatorInductionError::InvalidWithdrawalCredentials) - }, + } None => { let validator = ValidatorRecord { pubkey: deposit_input.pubkey.clone(), @@ -45,7 +49,7 @@ pub fn process_deposit( exit_count: 0, custody_commitment: deposit_input.custody_commitment, latest_custody_reseed_slot: 0, - penultimate_custody_reseed_slot: 0 + penultimate_custody_reseed_slot: 0, }; match min_empty_validator_index(state, spec) { @@ -53,7 +57,7 @@ pub fn process_deposit( state.validator_registry[i] = validator; state.validator_balances[i] = deposit_data.value; Ok(i) - }, + } None => { state.validator_registry.push(validator); state.validator_balances.push(deposit_data.value); @@ -64,14 +68,13 @@ pub fn process_deposit( } } -fn min_empty_validator_index( - state: &BeaconState, - spec: &ChainSpec -) -> Option { +fn min_empty_validator_index(state: &BeaconState, spec: &ChainSpec) -> Option { for i in 0..state.validator_registry.len() { if state.validator_balances[i] == 0 && state.validator_registry[i].latest_status_change_slot - + spec.zero_balance_validator_ttl <= state.slot { + + spec.zero_balance_validator_ttl + <= state.slot + { return Some(i); } } @@ -107,7 +110,10 @@ mod tests { (dep.deposit_data.deposit_input.pubkey == val.pubkey) & (dep.deposit_data.deposit_input.withdrawal_credentials == val.withdrawal_credentials) & (dep.deposit_data.deposit_input.randao_commitment == val.randao_commitment) - & (verify_proof_of_possession(&dep.deposit_data.deposit_input.proof_of_possession, &val.pubkey)) + & (verify_proof_of_possession( + &dep.deposit_data.deposit_input.proof_of_possession, + &val.pubkey, + )) } #[test] @@ -120,7 +126,10 @@ mod tests { let result = process_deposit(&mut state, &deposit, &spec); assert_eq!(result.unwrap(), 0); - assert!(deposit_equals_record(&deposit, &state.validator_registry[0])); + assert!(deposit_equals_record( + &deposit, + &state.validator_registry[0] + )); assert_eq!(state.validator_registry.len(), 1); assert_eq!(state.validator_balances.len(), 1); } @@ -135,7 +144,10 @@ mod tests { let result = process_deposit(&mut state, &deposit, &spec); deposit.deposit_data.value = DEPOSIT_GWEI; assert_eq!(result.unwrap(), i); - assert!(deposit_equals_record(&deposit, &state.validator_registry[i])); + assert!(deposit_equals_record( + &deposit, + &state.validator_registry[i] + )); assert_eq!(state.validator_registry.len(), i + 1); assert_eq!(state.validator_balances.len(), i + 1); } @@ -151,7 +163,8 @@ mod tests { deposit.deposit_data.value = DEPOSIT_GWEI; validator.pubkey = deposit.deposit_data.deposit_input.pubkey.clone(); - validator.withdrawal_credentials = deposit.deposit_data.deposit_input.withdrawal_credentials; + validator.withdrawal_credentials = + deposit.deposit_data.deposit_input.withdrawal_credentials; validator.randao_commitment = deposit.deposit_data.deposit_input.randao_commitment; state.validator_registry.push(validator); @@ -160,7 +173,10 @@ mod tests { let result = process_deposit(&mut state, &deposit, &spec); assert_eq!(result.unwrap(), 0); - assert!(deposit_equals_record(&deposit, &state.validator_registry[0])); + assert!(deposit_equals_record( + &deposit, + &state.validator_registry[0] + )); assert_eq!(state.validator_balances[0], DEPOSIT_GWEI * 2); assert_eq!(state.validator_registry.len(), 1); assert_eq!(state.validator_balances.len(), 1); @@ -183,7 +199,10 @@ mod tests { let result = process_deposit(&mut state, &deposit, &spec); assert_eq!(result.unwrap(), 0); - assert!(deposit_equals_record(&deposit, &state.validator_registry[0])); + assert!(deposit_equals_record( + &deposit, + &state.validator_registry[0] + )); assert_eq!(state.validator_balances[0], DEPOSIT_GWEI); assert_eq!(state.validator_registry.len(), 1); assert_eq!(state.validator_balances.len(), 1); @@ -195,11 +214,15 @@ mod tests { let mut deposit = get_deposit(); let spec = ChainSpec::foundation(); deposit.deposit_data.value = DEPOSIT_GWEI; - deposit.deposit_data.deposit_input.proof_of_possession = create_proof_of_possession(&Keypair::random()); + deposit.deposit_data.deposit_input.proof_of_possession = + create_proof_of_possession(&Keypair::random()); let result = process_deposit(&mut state, &deposit, &spec); - assert_eq!(result, Err(ValidatorInductionError::InvaidProofOfPossession)); + assert_eq!( + result, + Err(ValidatorInductionError::InvaidProofOfPossession) + ); assert_eq!(state.validator_registry.len(), 0); assert_eq!(state.validator_balances.len(), 0); } diff --git a/beacon_chain/validator_induction/src/lib.rs b/beacon_chain/validator_induction/src/lib.rs index f6dec3cfa..7119d6c2e 100644 --- a/beacon_chain/validator_induction/src/lib.rs +++ b/beacon_chain/validator_induction/src/lib.rs @@ -1,8 +1,8 @@ extern crate bls; extern crate hashing; -extern crate types; extern crate spec; +extern crate types; mod inductor; -pub use crate::inductor::{ValidatorInductionError, process_deposit}; +pub use crate::inductor::{process_deposit, ValidatorInductionError}; diff --git a/lighthouse/beacon_chain/tests/chain_test.rs b/lighthouse/beacon_chain/tests/chain_test.rs index aec6c08e6..8d2cfa501 100644 --- a/lighthouse/beacon_chain/tests/chain_test.rs +++ b/lighthouse/beacon_chain/tests/chain_test.rs @@ -1,4 +1,4 @@ -use chain::{BlockProcessingOutcome, BeaconChain}; +use chain::{BeaconChain, BlockProcessingOutcome}; use db::{ stores::{BeaconBlockStore, BeaconStateStore}, MemoryDB, diff --git a/lighthouse/db/src/disk_db.rs b/lighthouse/db/src/disk_db.rs index b084f483f..9d8a71bc4 100644 --- a/lighthouse/db/src/disk_db.rs +++ b/lighthouse/db/src/disk_db.rs @@ -44,7 +44,8 @@ impl DiskDB { let db = match columns { None => DB::open(&options, db_path), Some(columns) => DB::open_cf(&options, db_path, columns), - }.expect("Unable to open local database");; + } + .expect("Unable to open local database");; Self { db } } diff --git a/lighthouse/db/src/stores/beacon_state_store.rs b/lighthouse/db/src/stores/beacon_state_store.rs index 122b22509..a54e19249 100644 --- a/lighthouse/db/src/stores/beacon_state_store.rs +++ b/lighthouse/db/src/stores/beacon_state_store.rs @@ -42,10 +42,10 @@ mod tests { use super::super::super::MemoryDB; use super::*; - use std::sync::Arc; use ssz::ssz_encode; - use types::Hash256; + use std::sync::Arc; use types::test_utils::{SeedableRng, TestRandom, XorShiftRng}; + use types::Hash256; test_crud_for_store!(BeaconStateStore, DB_COLUMN); diff --git a/lighthouse/main.rs b/lighthouse/main.rs index b2c199db9..3102feaf7 100644 --- a/lighthouse/main.rs +++ b/lighthouse/main.rs @@ -12,8 +12,8 @@ mod config; use std::path::PathBuf; -use clap::{App, Arg}; use crate::config::LighthouseConfig; +use clap::{App, Arg}; use slog::Drain; fn main() { @@ -32,13 +32,15 @@ fn main() { .value_name("DIR") .help("Data directory for keys and databases.") .takes_value(true), - ).arg( + ) + .arg( Arg::with_name("port") .long("port") .value_name("PORT") .help("Network listen port for p2p connections.") .takes_value(true), - ).get_matches(); + ) + .get_matches(); let mut config = LighthouseConfig::default(); From 32d96995419dd1e0655fba308d6f2608a8157848 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 14:59:15 -0600 Subject: [PATCH 10/35] Add `cargo fmt` as a build step to pass CI --- .travis.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.travis.yml b/.travis.yml index 3337de420..0357e1c11 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,7 @@ language: rust +cache: cargo script: + - cargo fmt --all -- --check - cargo build --verbose --all - cargo test --verbose --all rust: @@ -10,3 +12,6 @@ matrix: allow_failures: - rust: nightly fast_finish: true +install: + - (cargo install rustfmt || true) + - PATH=$PATH:/home/travis/.cargo/bin From bc053ba41535fbd8a56795ab76988e43d5ea70c1 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 15:05:50 -0600 Subject: [PATCH 11/35] Tweaks for travis --- .travis.yml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 0357e1c11..8518c5d45 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,4 @@ language: rust -cache: cargo script: - cargo fmt --all -- --check - cargo build --verbose --all @@ -13,5 +12,4 @@ matrix: - rust: nightly fast_finish: true install: - - (cargo install rustfmt || true) - - PATH=$PATH:/home/travis/.cargo/bin + - rustup component add rustfmt From d12b5249e0496c6b87b5f6e38c8831c4ea258c66 Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Mon, 14 Jan 2019 16:32:55 +1100 Subject: [PATCH 12/35] Add genesis tests pre modifying the beancon structs --- beacon_chain/genesis/Cargo.toml | 1 + beacon_chain/genesis/src/beacon_block.rs | 54 ++++++++++++ beacon_chain/genesis/src/beacon_state.rs | 105 ++++++++++++++++++++++- 3 files changed, 159 insertions(+), 1 deletion(-) diff --git a/beacon_chain/genesis/Cargo.toml b/beacon_chain/genesis/Cargo.toml index 499333979..dbfd7e1fc 100644 --- a/beacon_chain/genesis/Cargo.toml +++ b/beacon_chain/genesis/Cargo.toml @@ -6,6 +6,7 @@ edition = "2018" [dependencies] bls = { path = "../utils/bls" } +bls-aggregates = { git = "https://github.com/sigp/signature-schemes" } spec = { path = "../spec" } ssz = { path = "../utils/ssz" } types = { path = "../types" } diff --git a/beacon_chain/genesis/src/beacon_block.rs b/beacon_chain/genesis/src/beacon_block.rs index 696ac6499..3ec321237 100644 --- a/beacon_chain/genesis/src/beacon_block.rs +++ b/beacon_chain/genesis/src/beacon_block.rs @@ -47,4 +47,58 @@ mod tests { // This only checks that the function runs without panic. genesis_beacon_block(state_root, &spec); } + + // Tests parent_root, randao_reveal, deposit_root are the zero hash after creation and slot == 0 + #[test] + fn test_zero_items() { + let spec = ChainSpec::foundation(); + + // Note: state_root will not be available without a state (test in beacon_state) + let state_root = Hash256::zero(); + + let genesis_block = genesis_beacon_block(state_root, &spec); + + assert!(genesis_block.parent_root.is_zero()); + assert!(genesis_block.randao_reveal.is_zero()); + assert!(genesis_block.slot == 0); + // assert!(genesis_block.depsoit_root.is_zero()); + + } + + #[test] + fn test_beacon_body() { + let spec = ChainSpec::foundation(); + + // Note: state_root will not be available without a state (test in beacon_state) + let state_root = Hash256::zero(); + + let genesis_block = genesis_beacon_block(state_root, &spec); + + assert!(genesis_block.body.proposer_slashings.is_empty()); + assert!(genesis_block.body.casper_slashings.is_empty()); + assert!(genesis_block.body.attestations.is_empty()); + assert!(genesis_block.body.deposits.is_empty()); + assert!(genesis_block.body.exits.is_empty()); + // Specs have changed to include 3 more variables in BeaconBody to be added later + } + + #[test] + fn test_signature() { + let spec = ChainSpec::foundation(); + + // Note: state_root will not be available without a state (test in beacon_state) + let state_root = Hash256::zero(); + + let genesis_block = genesis_beacon_block(state_root, &spec); + + // Signature should consist of [bytes48(0), bytes48(0)] + // Note this is implemented using Apache Milagro BLS which requires one extra byte -> 97bytes + let raw_sig = genesis_block.signature.as_raw(); + let raw_sig_bytes = raw_sig.as_bytes(); + + assert!(raw_sig_bytes.len() == 97); + for item in raw_sig_bytes.iter() { + assert!(*item == 0); + } + } } diff --git a/beacon_chain/genesis/src/beacon_state.rs b/beacon_chain/genesis/src/beacon_state.rs index f9c2ef327..3ebd3dba7 100644 --- a/beacon_chain/genesis/src/beacon_state.rs +++ b/beacon_chain/genesis/src/beacon_state.rs @@ -86,12 +86,13 @@ mod tests { extern crate validator_induction; use super::*; + use types::Hash256; // TODO: enhance these tests. // https://github.com/sigp/lighthouse/issues/117 #[test] - fn test_genesis() { + fn test_gen_state() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); @@ -101,4 +102,106 @@ mod tests { spec.initial_validators.len() ); } + + #[test] + fn test_gen_state_misc() { + let spec = ChainSpec::foundation(); + + let state = genesis_beacon_state(&spec).unwrap(); + + assert_eq!(state.slot, 0); + assert_eq!(state.genesis_time, spec.genesis_time); + assert_eq!(state.fork_data.pre_fork_version, 0); + assert_eq!(state.fork_data.post_fork_version, 0); + assert_eq!(state.fork_data.fork_slot, 0); + } + + #[test] + fn test_gen_state_validators() { + let spec = ChainSpec::foundation(); + + let state = genesis_beacon_state(&spec).unwrap(); + + assert_eq!(state.validator_registry, spec.initial_validators); + assert_eq!(state.validator_balances, spec.initial_balances); + assert!(state.validator_registry_latest_change_slot == 0); + assert!(state.validator_registry_exit_count == 0); + assert_eq!(state.validator_registry_delta_chain_tip, Hash256::zero()); + } + + #[test] + fn test_gen_state_randomness_committees() { + let spec = ChainSpec::foundation(); + + let state = genesis_beacon_state(&spec).unwrap(); + + // Note: specs now have randao_mixes containing 8,192 zero hashes + assert_eq!(state.randao_mix, Hash256::zero()); + + // Note: next_seed has changed to latest_vdf_outputs[8,192]8,192] + assert_eq!(state.next_seed, Hash256::zero()); + + // TODO: Check shard and committee shuffling requires solving issue: + // https://github.com/sigp/lighthouse/issues/151 + + // initial_shuffling = get_shuffling(Hash256::zero(), &state.validator_registry, 0, 0) + // initial_shuffling = initial_shuffling.append(initial_shuffling.clone()); + } + + #[test] + fn test_gen_state_custody_finanilty() { + let spec = ChainSpec::foundation(); + + let state = genesis_beacon_state(&spec).unwrap(); + + // Note: custody_challenges are not included yet but are in Eth2.0 specs + + assert_eq!(state.previous_justified_slot, 0); + assert_eq!(state.justified_slot, 0); + assert_eq!(state.justification_bitfield, 0); + assert_eq!(state.finalized_slot, 0); + } + + #[test] + fn test_gen_state_recent_state() { + let spec = ChainSpec::foundation(); + + let state = genesis_beacon_state(&spec).unwrap(); + + + // Test latest_crosslinks + assert_eq!(state.latest_crosslinks.len(), 1024); + for link in state.latest_crosslinks.iter() { + assert_eq!(link.slot, 0); + assert_eq!(link.shard_block_root, Hash256::zero()); + } + + // Test latest_block_roots + assert_eq!(state.latest_block_roots.len(), 64); + for block in state.latest_block_roots.iter() { + assert_eq!(*block, Hash256::zero()); + } + + // Test latest_penalized_exit_balances + // Note: Eth2.0 specs says this should be an array of length LATEST_PENALIZE_EXIT_LENGTH + // = (8,192) + assert!(state.latest_penalized_exit_balances.is_empty()); + + // Test latest_attestations + assert!(state.latest_attestations.is_empty()); + + // Note: missing batched_block_roots in new spec + + } + + // Note: here we refer to it as pow_reciept in the Eth2.0 specs it is called deposit + #[test] + fn test_gen_state_deposit_root() { + let spec = ChainSpec::foundation(); + + let state = genesis_beacon_state(&spec).unwrap(); + + assert_eq!(state.processed_pow_receipt_root, spec.processed_pow_receipt_root); + assert!(state.candidate_pow_receipt_roots.is_empty()); + } } From 7979ec1635367e13ce3f7417ec37c9b4ec770e42 Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Mon, 14 Jan 2019 19:28:57 +1100 Subject: [PATCH 13/35] Modify BeaconState and BeaconBlockBody to reflect changes in the specs. Also add new variable LATEST_RANDAO_MIXES_LENGTH to foundation --- beacon_chain/genesis/src/beacon_block.rs | 3 ++ beacon_chain/genesis/src/beacon_state.rs | 11 +++-- beacon_chain/spec/src/foundation.rs | 1 + beacon_chain/spec/src/lib.rs | 1 + beacon_chain/types/src/beacon_block_body.rs | 21 +++++++++ beacon_chain/types/src/beacon_state.rs | 48 +++++++++++---------- 6 files changed, 59 insertions(+), 26 deletions(-) diff --git a/beacon_chain/genesis/src/beacon_block.rs b/beacon_chain/genesis/src/beacon_block.rs index 696ac6499..a1bbdd3a5 100644 --- a/beacon_chain/genesis/src/beacon_block.rs +++ b/beacon_chain/genesis/src/beacon_block.rs @@ -16,6 +16,9 @@ pub fn genesis_beacon_block(state_root: Hash256, spec: &ChainSpec) -> BeaconBloc proposer_slashings: vec![], casper_slashings: vec![], attestations: vec![], + custody_reseeds: vec![], + custody_challenges: vec![], + custody_responses: vec![], deposits: vec![], exits: vec![], }, diff --git a/beacon_chain/genesis/src/beacon_state.rs b/beacon_chain/genesis/src/beacon_state.rs index f9c2ef327..f8006a3c6 100644 --- a/beacon_chain/genesis/src/beacon_state.rs +++ b/beacon_chain/genesis/src/beacon_state.rs @@ -47,11 +47,13 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { /* * Randomness and committees */ - randao_mix: spec.zero_hash, - next_seed: spec.zero_hash, + latest_randao_mixes: vec![spec.zero_hash; spec.latest_randao_mixes_length as usize], + latest_vdf_outputs: vec![spec.zero_hash; (spec.latest_randao_mixes_length / spec.epoch_length) as usize], shard_committees_at_slots: vec![], - persistent_committees: vec![], - persistent_committee_reassignments: vec![], + /* + * Custody challenges + */ + custody_challenges: vec![], /* * Finality */ @@ -66,6 +68,7 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { latest_block_roots: vec![spec.zero_hash; spec.epoch_length as usize], latest_penalized_exit_balances: vec![], latest_attestations: vec![], + batched_block_roots: vec![], /* * PoW receipt root */ diff --git a/beacon_chain/spec/src/foundation.rs b/beacon_chain/spec/src/foundation.rs index 31207f058..1a493d72c 100644 --- a/beacon_chain/spec/src/foundation.rs +++ b/beacon_chain/spec/src/foundation.rs @@ -24,6 +24,7 @@ impl ChainSpec { beacon_chain_shard_number: u64::max_value(), bls_withdrawal_prefix_byte: 0x00, max_casper_votes: 1_024, + latest_randao_mixes_length: 8_192, /* * Deposit contract */ diff --git a/beacon_chain/spec/src/lib.rs b/beacon_chain/spec/src/lib.rs index 786cf326b..eb024cb6c 100644 --- a/beacon_chain/spec/src/lib.rs +++ b/beacon_chain/spec/src/lib.rs @@ -18,6 +18,7 @@ pub struct ChainSpec { pub beacon_chain_shard_number: u64, pub bls_withdrawal_prefix_byte: u8, pub max_casper_votes: u64, + pub latest_randao_mixes_length: u64, /* * Deposit contract */ diff --git a/beacon_chain/types/src/beacon_block_body.rs b/beacon_chain/types/src/beacon_block_body.rs index 8bcf1af33..b5891d592 100644 --- a/beacon_chain/types/src/beacon_block_body.rs +++ b/beacon_chain/types/src/beacon_block_body.rs @@ -3,11 +3,20 @@ use super::{Attestation, CasperSlashing, Deposit, Exit, ProposerSlashing}; use crate::test_utils::TestRandom; use rand::RngCore; +// The following types are just dummy classes as they will not be defined until +// Phase 1 (Sharding phase) +type CustodyReseed = usize; +type CustodyChallenge = usize; +type CustodyResponse = usize; + #[derive(Debug, PartialEq, Clone, Default)] pub struct BeaconBlockBody { pub proposer_slashings: Vec, pub casper_slashings: Vec, pub attestations: Vec, + pub custody_reseeds: Vec, + pub custody_challenges: Vec, + pub custody_responses: Vec, pub deposits: Vec, pub exits: Vec, } @@ -17,6 +26,9 @@ impl Encodable for BeaconBlockBody { s.append_vec(&self.proposer_slashings); s.append_vec(&self.casper_slashings); s.append_vec(&self.attestations); + s.append_vec(&self.custody_reseeds); + s.append_vec(&self.custody_challenges); + s.append_vec(&self.custody_responses); s.append_vec(&self.deposits); s.append_vec(&self.exits); } @@ -27,6 +39,9 @@ impl Decodable for BeaconBlockBody { let (proposer_slashings, i) = <_>::ssz_decode(bytes, i)?; let (casper_slashings, i) = <_>::ssz_decode(bytes, i)?; let (attestations, i) = <_>::ssz_decode(bytes, i)?; + let (custody_reseeds, i) = <_>::ssz_decode(bytes, i)?; + let (custody_challenges, i) = <_>::ssz_decode(bytes, i)?; + let (custody_responses, i) = <_>::ssz_decode(bytes, i)?; let (deposits, i) = <_>::ssz_decode(bytes, i)?; let (exits, i) = <_>::ssz_decode(bytes, i)?; @@ -35,6 +50,9 @@ impl Decodable for BeaconBlockBody { proposer_slashings, casper_slashings, attestations, + custody_reseeds, + custody_challenges, + custody_responses, deposits, exits, }, @@ -49,6 +67,9 @@ impl TestRandom for BeaconBlockBody { proposer_slashings: <_>::random_for_test(rng), casper_slashings: <_>::random_for_test(rng), attestations: <_>::random_for_test(rng), + custody_reseeds: <_>::random_for_test(rng), + custody_challenges: <_>::random_for_test(rng), + custody_responses: <_>::random_for_test(rng), deposits: <_>::random_for_test(rng), exits: <_>::random_for_test(rng), } diff --git a/beacon_chain/types/src/beacon_state.rs b/beacon_chain/types/src/beacon_state.rs index 588cfb678..b9994bddd 100644 --- a/beacon_chain/types/src/beacon_state.rs +++ b/beacon_chain/types/src/beacon_state.rs @@ -3,7 +3,6 @@ use super::crosslink_record::CrosslinkRecord; use super::fork_data::ForkData; use super::pending_attestation_record::PendingAttestationRecord; use super::shard_committee::ShardCommittee; -use super::shard_reassignment_record::ShardReassignmentRecord; use super::validator_record::ValidatorRecord; use super::Hash256; use crate::test_utils::TestRandom; @@ -11,6 +10,9 @@ use hashing::canonical_hash; use rand::RngCore; use ssz::{ssz_encode, Decodable, DecodeError, Encodable, SszStream}; +// Custody will not be added to the specs until Phase 1 (Sharding Phase) so dummay class used. +type CustodyChallenge = usize; + #[derive(Debug, PartialEq, Clone, Default)] pub struct BeaconState { // Misc @@ -26,11 +28,12 @@ pub struct BeaconState { pub validator_registry_delta_chain_tip: Hash256, // Randomness and committees - pub randao_mix: Hash256, - pub next_seed: Hash256, + pub latest_randao_mixes: Vec, + pub latest_vdf_outputs: Vec, pub shard_committees_at_slots: Vec>, - pub persistent_committees: Vec>, - pub persistent_committee_reassignments: Vec, + + // Custody challenges + pub custody_challenges: Vec, // Finality pub previous_justified_slot: u64, @@ -43,8 +46,9 @@ pub struct BeaconState { pub latest_block_roots: Vec, pub latest_penalized_exit_balances: Vec, pub latest_attestations: Vec, + pub batched_block_roots: Vec, - // PoW receipt root + // PoW receipt root (a.k.a. deposit root) pub processed_pow_receipt_root: Hash256, pub candidate_pow_receipt_roots: Vec, } @@ -67,11 +71,10 @@ impl Encodable for BeaconState { s.append(&self.validator_registry_latest_change_slot); s.append(&self.validator_registry_exit_count); s.append(&self.validator_registry_delta_chain_tip); - s.append(&self.randao_mix); - s.append(&self.next_seed); + s.append(&self.latest_randao_mixes); + s.append(&self.latest_vdf_outputs); s.append(&self.shard_committees_at_slots); - s.append(&self.persistent_committees); - s.append(&self.persistent_committee_reassignments); + s.append(&self.custody_challenges); s.append(&self.previous_justified_slot); s.append(&self.justified_slot); s.append(&self.justification_bitfield); @@ -80,6 +83,7 @@ impl Encodable for BeaconState { s.append(&self.latest_block_roots); s.append(&self.latest_penalized_exit_balances); s.append(&self.latest_attestations); + s.append(&self.batched_block_roots); s.append(&self.processed_pow_receipt_root); s.append(&self.candidate_pow_receipt_roots); } @@ -95,11 +99,10 @@ impl Decodable for BeaconState { let (validator_registry_latest_change_slot, i) = <_>::ssz_decode(bytes, i)?; let (validator_registry_exit_count, i) = <_>::ssz_decode(bytes, i)?; let (validator_registry_delta_chain_tip, i) = <_>::ssz_decode(bytes, i)?; - let (randao_mix, i) = <_>::ssz_decode(bytes, i)?; - let (next_seed, i) = <_>::ssz_decode(bytes, i)?; + let (latest_randao_mixes, i) = <_>::ssz_decode(bytes, i)?; + let (latest_vdf_outputs, i) = <_>::ssz_decode(bytes, i)?; let (shard_committees_at_slots, i) = <_>::ssz_decode(bytes, i)?; - let (persistent_committees, i) = <_>::ssz_decode(bytes, i)?; - let (persistent_committee_reassignments, i) = <_>::ssz_decode(bytes, i)?; + let (custody_challenges, i) = <_>::ssz_decode(bytes, i)?; let (previous_justified_slot, i) = <_>::ssz_decode(bytes, i)?; let (justified_slot, i) = <_>::ssz_decode(bytes, i)?; let (justification_bitfield, i) = <_>::ssz_decode(bytes, i)?; @@ -108,6 +111,7 @@ impl Decodable for BeaconState { let (latest_block_roots, i) = <_>::ssz_decode(bytes, i)?; let (latest_penalized_exit_balances, i) = <_>::ssz_decode(bytes, i)?; let (latest_attestations, i) = <_>::ssz_decode(bytes, i)?; + let (batched_block_roots, i) = <_>::ssz_decode(bytes, i)?; let (processed_pow_receipt_root, i) = <_>::ssz_decode(bytes, i)?; let (candidate_pow_receipt_roots, i) = <_>::ssz_decode(bytes, i)?; @@ -121,11 +125,10 @@ impl Decodable for BeaconState { validator_registry_latest_change_slot, validator_registry_exit_count, validator_registry_delta_chain_tip, - randao_mix, - next_seed, + latest_randao_mixes, + latest_vdf_outputs, shard_committees_at_slots, - persistent_committees, - persistent_committee_reassignments, + custody_challenges, previous_justified_slot, justified_slot, justification_bitfield, @@ -134,6 +137,7 @@ impl Decodable for BeaconState { latest_block_roots, latest_penalized_exit_balances, latest_attestations, + batched_block_roots, processed_pow_receipt_root, candidate_pow_receipt_roots, }, @@ -153,11 +157,10 @@ impl TestRandom for BeaconState { validator_registry_latest_change_slot: <_>::random_for_test(rng), validator_registry_exit_count: <_>::random_for_test(rng), validator_registry_delta_chain_tip: <_>::random_for_test(rng), - randao_mix: <_>::random_for_test(rng), - next_seed: <_>::random_for_test(rng), + latest_randao_mixes: <_>::random_for_test(rng), + latest_vdf_outputs: <_>::random_for_test(rng), shard_committees_at_slots: <_>::random_for_test(rng), - persistent_committees: <_>::random_for_test(rng), - persistent_committee_reassignments: <_>::random_for_test(rng), + custody_challenges: <_>::random_for_test(rng), previous_justified_slot: <_>::random_for_test(rng), justified_slot: <_>::random_for_test(rng), justification_bitfield: <_>::random_for_test(rng), @@ -166,6 +169,7 @@ impl TestRandom for BeaconState { latest_block_roots: <_>::random_for_test(rng), latest_penalized_exit_balances: <_>::random_for_test(rng), latest_attestations: <_>::random_for_test(rng), + batched_block_roots: <_>::random_for_test(rng), processed_pow_receipt_root: <_>::random_for_test(rng), candidate_pow_receipt_roots: <_>::random_for_test(rng), } From ca28c187097d4bbb56f22a629048c84f97047363 Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Mon, 14 Jan 2019 19:28:57 +1100 Subject: [PATCH 14/35] Modify BeaconState and BeaconBlockBody to reflect changes in the specs. Also add new variable LATEST_RANDAO_MIXES_LENGTH to foundation Signed-off-by: Kirk Baird --- beacon_chain/genesis/src/beacon_block.rs | 3 ++ beacon_chain/genesis/src/beacon_state.rs | 11 +++-- beacon_chain/spec/src/foundation.rs | 1 + beacon_chain/spec/src/lib.rs | 1 + beacon_chain/types/src/beacon_block_body.rs | 21 +++++++++ beacon_chain/types/src/beacon_state.rs | 48 +++++++++++---------- 6 files changed, 59 insertions(+), 26 deletions(-) diff --git a/beacon_chain/genesis/src/beacon_block.rs b/beacon_chain/genesis/src/beacon_block.rs index 696ac6499..a1bbdd3a5 100644 --- a/beacon_chain/genesis/src/beacon_block.rs +++ b/beacon_chain/genesis/src/beacon_block.rs @@ -16,6 +16,9 @@ pub fn genesis_beacon_block(state_root: Hash256, spec: &ChainSpec) -> BeaconBloc proposer_slashings: vec![], casper_slashings: vec![], attestations: vec![], + custody_reseeds: vec![], + custody_challenges: vec![], + custody_responses: vec![], deposits: vec![], exits: vec![], }, diff --git a/beacon_chain/genesis/src/beacon_state.rs b/beacon_chain/genesis/src/beacon_state.rs index f9c2ef327..f8006a3c6 100644 --- a/beacon_chain/genesis/src/beacon_state.rs +++ b/beacon_chain/genesis/src/beacon_state.rs @@ -47,11 +47,13 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { /* * Randomness and committees */ - randao_mix: spec.zero_hash, - next_seed: spec.zero_hash, + latest_randao_mixes: vec![spec.zero_hash; spec.latest_randao_mixes_length as usize], + latest_vdf_outputs: vec![spec.zero_hash; (spec.latest_randao_mixes_length / spec.epoch_length) as usize], shard_committees_at_slots: vec![], - persistent_committees: vec![], - persistent_committee_reassignments: vec![], + /* + * Custody challenges + */ + custody_challenges: vec![], /* * Finality */ @@ -66,6 +68,7 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { latest_block_roots: vec![spec.zero_hash; spec.epoch_length as usize], latest_penalized_exit_balances: vec![], latest_attestations: vec![], + batched_block_roots: vec![], /* * PoW receipt root */ diff --git a/beacon_chain/spec/src/foundation.rs b/beacon_chain/spec/src/foundation.rs index 31207f058..1a493d72c 100644 --- a/beacon_chain/spec/src/foundation.rs +++ b/beacon_chain/spec/src/foundation.rs @@ -24,6 +24,7 @@ impl ChainSpec { beacon_chain_shard_number: u64::max_value(), bls_withdrawal_prefix_byte: 0x00, max_casper_votes: 1_024, + latest_randao_mixes_length: 8_192, /* * Deposit contract */ diff --git a/beacon_chain/spec/src/lib.rs b/beacon_chain/spec/src/lib.rs index 786cf326b..eb024cb6c 100644 --- a/beacon_chain/spec/src/lib.rs +++ b/beacon_chain/spec/src/lib.rs @@ -18,6 +18,7 @@ pub struct ChainSpec { pub beacon_chain_shard_number: u64, pub bls_withdrawal_prefix_byte: u8, pub max_casper_votes: u64, + pub latest_randao_mixes_length: u64, /* * Deposit contract */ diff --git a/beacon_chain/types/src/beacon_block_body.rs b/beacon_chain/types/src/beacon_block_body.rs index 8bcf1af33..b5891d592 100644 --- a/beacon_chain/types/src/beacon_block_body.rs +++ b/beacon_chain/types/src/beacon_block_body.rs @@ -3,11 +3,20 @@ use super::{Attestation, CasperSlashing, Deposit, Exit, ProposerSlashing}; use crate::test_utils::TestRandom; use rand::RngCore; +// The following types are just dummy classes as they will not be defined until +// Phase 1 (Sharding phase) +type CustodyReseed = usize; +type CustodyChallenge = usize; +type CustodyResponse = usize; + #[derive(Debug, PartialEq, Clone, Default)] pub struct BeaconBlockBody { pub proposer_slashings: Vec, pub casper_slashings: Vec, pub attestations: Vec, + pub custody_reseeds: Vec, + pub custody_challenges: Vec, + pub custody_responses: Vec, pub deposits: Vec, pub exits: Vec, } @@ -17,6 +26,9 @@ impl Encodable for BeaconBlockBody { s.append_vec(&self.proposer_slashings); s.append_vec(&self.casper_slashings); s.append_vec(&self.attestations); + s.append_vec(&self.custody_reseeds); + s.append_vec(&self.custody_challenges); + s.append_vec(&self.custody_responses); s.append_vec(&self.deposits); s.append_vec(&self.exits); } @@ -27,6 +39,9 @@ impl Decodable for BeaconBlockBody { let (proposer_slashings, i) = <_>::ssz_decode(bytes, i)?; let (casper_slashings, i) = <_>::ssz_decode(bytes, i)?; let (attestations, i) = <_>::ssz_decode(bytes, i)?; + let (custody_reseeds, i) = <_>::ssz_decode(bytes, i)?; + let (custody_challenges, i) = <_>::ssz_decode(bytes, i)?; + let (custody_responses, i) = <_>::ssz_decode(bytes, i)?; let (deposits, i) = <_>::ssz_decode(bytes, i)?; let (exits, i) = <_>::ssz_decode(bytes, i)?; @@ -35,6 +50,9 @@ impl Decodable for BeaconBlockBody { proposer_slashings, casper_slashings, attestations, + custody_reseeds, + custody_challenges, + custody_responses, deposits, exits, }, @@ -49,6 +67,9 @@ impl TestRandom for BeaconBlockBody { proposer_slashings: <_>::random_for_test(rng), casper_slashings: <_>::random_for_test(rng), attestations: <_>::random_for_test(rng), + custody_reseeds: <_>::random_for_test(rng), + custody_challenges: <_>::random_for_test(rng), + custody_responses: <_>::random_for_test(rng), deposits: <_>::random_for_test(rng), exits: <_>::random_for_test(rng), } diff --git a/beacon_chain/types/src/beacon_state.rs b/beacon_chain/types/src/beacon_state.rs index 588cfb678..b9994bddd 100644 --- a/beacon_chain/types/src/beacon_state.rs +++ b/beacon_chain/types/src/beacon_state.rs @@ -3,7 +3,6 @@ use super::crosslink_record::CrosslinkRecord; use super::fork_data::ForkData; use super::pending_attestation_record::PendingAttestationRecord; use super::shard_committee::ShardCommittee; -use super::shard_reassignment_record::ShardReassignmentRecord; use super::validator_record::ValidatorRecord; use super::Hash256; use crate::test_utils::TestRandom; @@ -11,6 +10,9 @@ use hashing::canonical_hash; use rand::RngCore; use ssz::{ssz_encode, Decodable, DecodeError, Encodable, SszStream}; +// Custody will not be added to the specs until Phase 1 (Sharding Phase) so dummay class used. +type CustodyChallenge = usize; + #[derive(Debug, PartialEq, Clone, Default)] pub struct BeaconState { // Misc @@ -26,11 +28,12 @@ pub struct BeaconState { pub validator_registry_delta_chain_tip: Hash256, // Randomness and committees - pub randao_mix: Hash256, - pub next_seed: Hash256, + pub latest_randao_mixes: Vec, + pub latest_vdf_outputs: Vec, pub shard_committees_at_slots: Vec>, - pub persistent_committees: Vec>, - pub persistent_committee_reassignments: Vec, + + // Custody challenges + pub custody_challenges: Vec, // Finality pub previous_justified_slot: u64, @@ -43,8 +46,9 @@ pub struct BeaconState { pub latest_block_roots: Vec, pub latest_penalized_exit_balances: Vec, pub latest_attestations: Vec, + pub batched_block_roots: Vec, - // PoW receipt root + // PoW receipt root (a.k.a. deposit root) pub processed_pow_receipt_root: Hash256, pub candidate_pow_receipt_roots: Vec, } @@ -67,11 +71,10 @@ impl Encodable for BeaconState { s.append(&self.validator_registry_latest_change_slot); s.append(&self.validator_registry_exit_count); s.append(&self.validator_registry_delta_chain_tip); - s.append(&self.randao_mix); - s.append(&self.next_seed); + s.append(&self.latest_randao_mixes); + s.append(&self.latest_vdf_outputs); s.append(&self.shard_committees_at_slots); - s.append(&self.persistent_committees); - s.append(&self.persistent_committee_reassignments); + s.append(&self.custody_challenges); s.append(&self.previous_justified_slot); s.append(&self.justified_slot); s.append(&self.justification_bitfield); @@ -80,6 +83,7 @@ impl Encodable for BeaconState { s.append(&self.latest_block_roots); s.append(&self.latest_penalized_exit_balances); s.append(&self.latest_attestations); + s.append(&self.batched_block_roots); s.append(&self.processed_pow_receipt_root); s.append(&self.candidate_pow_receipt_roots); } @@ -95,11 +99,10 @@ impl Decodable for BeaconState { let (validator_registry_latest_change_slot, i) = <_>::ssz_decode(bytes, i)?; let (validator_registry_exit_count, i) = <_>::ssz_decode(bytes, i)?; let (validator_registry_delta_chain_tip, i) = <_>::ssz_decode(bytes, i)?; - let (randao_mix, i) = <_>::ssz_decode(bytes, i)?; - let (next_seed, i) = <_>::ssz_decode(bytes, i)?; + let (latest_randao_mixes, i) = <_>::ssz_decode(bytes, i)?; + let (latest_vdf_outputs, i) = <_>::ssz_decode(bytes, i)?; let (shard_committees_at_slots, i) = <_>::ssz_decode(bytes, i)?; - let (persistent_committees, i) = <_>::ssz_decode(bytes, i)?; - let (persistent_committee_reassignments, i) = <_>::ssz_decode(bytes, i)?; + let (custody_challenges, i) = <_>::ssz_decode(bytes, i)?; let (previous_justified_slot, i) = <_>::ssz_decode(bytes, i)?; let (justified_slot, i) = <_>::ssz_decode(bytes, i)?; let (justification_bitfield, i) = <_>::ssz_decode(bytes, i)?; @@ -108,6 +111,7 @@ impl Decodable for BeaconState { let (latest_block_roots, i) = <_>::ssz_decode(bytes, i)?; let (latest_penalized_exit_balances, i) = <_>::ssz_decode(bytes, i)?; let (latest_attestations, i) = <_>::ssz_decode(bytes, i)?; + let (batched_block_roots, i) = <_>::ssz_decode(bytes, i)?; let (processed_pow_receipt_root, i) = <_>::ssz_decode(bytes, i)?; let (candidate_pow_receipt_roots, i) = <_>::ssz_decode(bytes, i)?; @@ -121,11 +125,10 @@ impl Decodable for BeaconState { validator_registry_latest_change_slot, validator_registry_exit_count, validator_registry_delta_chain_tip, - randao_mix, - next_seed, + latest_randao_mixes, + latest_vdf_outputs, shard_committees_at_slots, - persistent_committees, - persistent_committee_reassignments, + custody_challenges, previous_justified_slot, justified_slot, justification_bitfield, @@ -134,6 +137,7 @@ impl Decodable for BeaconState { latest_block_roots, latest_penalized_exit_balances, latest_attestations, + batched_block_roots, processed_pow_receipt_root, candidate_pow_receipt_roots, }, @@ -153,11 +157,10 @@ impl TestRandom for BeaconState { validator_registry_latest_change_slot: <_>::random_for_test(rng), validator_registry_exit_count: <_>::random_for_test(rng), validator_registry_delta_chain_tip: <_>::random_for_test(rng), - randao_mix: <_>::random_for_test(rng), - next_seed: <_>::random_for_test(rng), + latest_randao_mixes: <_>::random_for_test(rng), + latest_vdf_outputs: <_>::random_for_test(rng), shard_committees_at_slots: <_>::random_for_test(rng), - persistent_committees: <_>::random_for_test(rng), - persistent_committee_reassignments: <_>::random_for_test(rng), + custody_challenges: <_>::random_for_test(rng), previous_justified_slot: <_>::random_for_test(rng), justified_slot: <_>::random_for_test(rng), justification_bitfield: <_>::random_for_test(rng), @@ -166,6 +169,7 @@ impl TestRandom for BeaconState { latest_block_roots: <_>::random_for_test(rng), latest_penalized_exit_balances: <_>::random_for_test(rng), latest_attestations: <_>::random_for_test(rng), + batched_block_roots: <_>::random_for_test(rng), processed_pow_receipt_root: <_>::random_for_test(rng), candidate_pow_receipt_roots: <_>::random_for_test(rng), } From 730281ed7d7fd69373cf954874c473098793c8e7 Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Tue, 15 Jan 2019 10:37:08 +1100 Subject: [PATCH 15/35] modify formatting --- beacon_chain/genesis/src/beacon_state.rs | 5 ++++- beacon_chain/types/src/beacon_block_body.rs | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/beacon_chain/genesis/src/beacon_state.rs b/beacon_chain/genesis/src/beacon_state.rs index f8006a3c6..f8c7eb539 100644 --- a/beacon_chain/genesis/src/beacon_state.rs +++ b/beacon_chain/genesis/src/beacon_state.rs @@ -48,7 +48,10 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { * Randomness and committees */ latest_randao_mixes: vec![spec.zero_hash; spec.latest_randao_mixes_length as usize], - latest_vdf_outputs: vec![spec.zero_hash; (spec.latest_randao_mixes_length / spec.epoch_length) as usize], + latest_vdf_outputs: vec![ + spec.zero_hash; + (spec.latest_randao_mixes_length / spec.epoch_length) as usize + ], shard_committees_at_slots: vec![], /* * Custody challenges diff --git a/beacon_chain/types/src/beacon_block_body.rs b/beacon_chain/types/src/beacon_block_body.rs index b5891d592..67fa34d91 100644 --- a/beacon_chain/types/src/beacon_block_body.rs +++ b/beacon_chain/types/src/beacon_block_body.rs @@ -3,7 +3,7 @@ use super::{Attestation, CasperSlashing, Deposit, Exit, ProposerSlashing}; use crate::test_utils::TestRandom; use rand::RngCore; -// The following types are just dummy classes as they will not be defined until +// The following types are just dummy classes as they will not be defined until // Phase 1 (Sharding phase) type CustodyReseed = usize; type CustodyChallenge = usize; From 0b47b81a6c5ecc09b7672c4378ad5b24d528d9ca Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Tue, 15 Jan 2019 12:43:14 +1100 Subject: [PATCH 16/35] Add tests for genesis and modify genesis to match new specs --- beacon_chain/genesis/src/beacon_block.rs | 16 +++---- beacon_chain/genesis/src/beacon_state.rs | 60 +++++++++++++++--------- beacon_chain/spec/src/foundation.rs | 3 ++ beacon_chain/spec/src/lib.rs | 3 ++ 4 files changed, 50 insertions(+), 32 deletions(-) diff --git a/beacon_chain/genesis/src/beacon_block.rs b/beacon_chain/genesis/src/beacon_block.rs index 423a1c64a..6caa0c80c 100644 --- a/beacon_chain/genesis/src/beacon_block.rs +++ b/beacon_chain/genesis/src/beacon_block.rs @@ -39,9 +39,6 @@ fn genesis_signature() -> Signature { mod tests { use super::*; - // TODO: enhance these tests. - // https://github.com/sigp/lighthouse/issues/117 - #[test] fn test_genesis() { let spec = ChainSpec::foundation(); @@ -51,7 +48,7 @@ mod tests { genesis_beacon_block(state_root, &spec); } - // Tests parent_root, randao_reveal, deposit_root are the zero hash after creation and slot == 0 + // Tests items that are 0 or zero_hash #[test] fn test_zero_items() { let spec = ChainSpec::foundation(); @@ -61,13 +58,13 @@ mod tests { let genesis_block = genesis_beacon_block(state_root, &spec); + assert!(genesis_block.slot == 0); assert!(genesis_block.parent_root.is_zero()); assert!(genesis_block.randao_reveal.is_zero()); - assert!(genesis_block.slot == 0); - // assert!(genesis_block.depsoit_root.is_zero()); - + assert!(genesis_block.candidate_pow_receipt_root.is_zero()); // aka deposit_root } + // Tests the BeaconBlockBody inside BeaconBlock #[test] fn test_beacon_body() { let spec = ChainSpec::foundation(); @@ -77,12 +74,13 @@ mod tests { let genesis_block = genesis_beacon_block(state_root, &spec); + // Custody items are not being implemented until phase 1 so tests to be added later + assert!(genesis_block.body.proposer_slashings.is_empty()); assert!(genesis_block.body.casper_slashings.is_empty()); assert!(genesis_block.body.attestations.is_empty()); assert!(genesis_block.body.deposits.is_empty()); assert!(genesis_block.body.exits.is_empty()); - // Specs have changed to include 3 more variables in BeaconBody to be added later } #[test] @@ -102,6 +100,6 @@ mod tests { assert!(raw_sig_bytes.len() == 97); for item in raw_sig_bytes.iter() { assert!(*item == 0); - } + } } } diff --git a/beacon_chain/genesis/src/beacon_state.rs b/beacon_chain/genesis/src/beacon_state.rs index 94207473c..7326bbf1d 100644 --- a/beacon_chain/genesis/src/beacon_state.rs +++ b/beacon_chain/genesis/src/beacon_state.rs @@ -68,8 +68,8 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { * Recent state */ latest_crosslinks: vec![initial_crosslink; spec.shard_count as usize], - latest_block_roots: vec![spec.zero_hash; spec.epoch_length as usize], - latest_penalized_exit_balances: vec![], + latest_block_roots: vec![spec.zero_hash; spec.latest_block_roots_length as usize], + latest_penalized_exit_balances: vec![0; spec.latest_penalized_exit_length as usize], latest_attestations: vec![], batched_block_roots: vec![], /* @@ -94,9 +94,6 @@ mod tests { use super::*; use types::Hash256; - // TODO: enhance these tests. - // https://github.com/sigp/lighthouse/issues/117 - #[test] fn test_gen_state() { let spec = ChainSpec::foundation(); @@ -109,6 +106,7 @@ mod tests { ); } + // Test Misc #[test] fn test_gen_state_misc() { let spec = ChainSpec::foundation(); @@ -116,12 +114,13 @@ mod tests { let state = genesis_beacon_state(&spec).unwrap(); assert_eq!(state.slot, 0); - assert_eq!(state.genesis_time, spec.genesis_time); + assert_eq!(state.genesis_time, spec.genesis_time); assert_eq!(state.fork_data.pre_fork_version, 0); assert_eq!(state.fork_data.post_fork_version, 0); assert_eq!(state.fork_data.fork_slot, 0); } + // Test validators #[test] fn test_gen_state_validators() { let spec = ChainSpec::foundation(); @@ -135,17 +134,24 @@ mod tests { assert_eq!(state.validator_registry_delta_chain_tip, Hash256::zero()); } + // Test randomness and committees #[test] fn test_gen_state_randomness_committees() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); - // Note: specs now have randao_mixes containing 8,192 zero hashes - assert_eq!(state.randao_mix, Hash256::zero()); + // Array of size 8,192 each being zero_hash + assert_eq!(state.latest_randao_mixes.len(), 8_192); + for item in state.latest_randao_mixes.iter() { + assert_eq!(*item, Hash256::zero()); + } - // Note: next_seed has changed to latest_vdf_outputs[8,192]8,192] - assert_eq!(state.next_seed, Hash256::zero()); + // Array of size 8,192 each being a zero hash + assert_eq!(state.latest_vdf_outputs.len(), (8_192 / 64)); + for item in state.latest_vdf_outputs.iter() { + assert_eq!(*item, Hash256::zero()); + } // TODO: Check shard and committee shuffling requires solving issue: // https://github.com/sigp/lighthouse/issues/151 @@ -154,60 +160,68 @@ mod tests { // initial_shuffling = initial_shuffling.append(initial_shuffling.clone()); } + // Custody not implemented until Phase 1 + // This test will always pass until Phase 1 #[test] - fn test_gen_state_custody_finanilty() { + fn test_gen_state_custody() {} + + // Test finality + #[test] + fn test_gen_state_finanilty() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); - // Note: custody_challenges are not included yet but are in Eth2.0 specs - assert_eq!(state.previous_justified_slot, 0); assert_eq!(state.justified_slot, 0); assert_eq!(state.justification_bitfield, 0); assert_eq!(state.finalized_slot, 0); } + // Test recent state #[test] fn test_gen_state_recent_state() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); - // Test latest_crosslinks - assert_eq!(state.latest_crosslinks.len(), 1024); + assert_eq!(state.latest_crosslinks.len(), 1_024); for link in state.latest_crosslinks.iter() { assert_eq!(link.slot, 0); assert_eq!(link.shard_block_root, Hash256::zero()); } // Test latest_block_roots - assert_eq!(state.latest_block_roots.len(), 64); + assert_eq!(state.latest_block_roots.len(), 8_192); for block in state.latest_block_roots.iter() { assert_eq!(*block, Hash256::zero()); } // Test latest_penalized_exit_balances - // Note: Eth2.0 specs says this should be an array of length LATEST_PENALIZE_EXIT_LENGTH - // = (8,192) - assert!(state.latest_penalized_exit_balances.is_empty()); + assert_eq!(state.latest_penalized_exit_balances.len(), 8_192); + for item in state.latest_penalized_exit_balances.iter() { + assert!(*item == 0); + } // Test latest_attestations assert!(state.latest_attestations.is_empty()); - // Note: missing batched_block_roots in new spec - + // batched_block_roots + assert!(state.batched_block_roots.is_empty()); } - // Note: here we refer to it as pow_reciept in the Eth2.0 specs it is called deposit + // Test PoW Receipts a.k.a. deposits #[test] fn test_gen_state_deposit_root() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); - assert_eq!(state.processed_pow_receipt_root, spec.processed_pow_receipt_root); + assert_eq!( + state.processed_pow_receipt_root, + spec.processed_pow_receipt_root + ); assert!(state.candidate_pow_receipt_roots.is_empty()); } } diff --git a/beacon_chain/spec/src/foundation.rs b/beacon_chain/spec/src/foundation.rs index 1a493d72c..8e3146ed2 100644 --- a/beacon_chain/spec/src/foundation.rs +++ b/beacon_chain/spec/src/foundation.rs @@ -24,7 +24,10 @@ impl ChainSpec { beacon_chain_shard_number: u64::max_value(), bls_withdrawal_prefix_byte: 0x00, max_casper_votes: 1_024, + latest_block_roots_length: 8_192, latest_randao_mixes_length: 8_192, + latest_penalized_exit_length: 8_192, + max_withdrawals_per_epoch: 4, /* * Deposit contract */ diff --git a/beacon_chain/spec/src/lib.rs b/beacon_chain/spec/src/lib.rs index eb024cb6c..5b7bdfae3 100644 --- a/beacon_chain/spec/src/lib.rs +++ b/beacon_chain/spec/src/lib.rs @@ -18,7 +18,10 @@ pub struct ChainSpec { pub beacon_chain_shard_number: u64, pub bls_withdrawal_prefix_byte: u8, pub max_casper_votes: u64, + pub latest_block_roots_length: u64, pub latest_randao_mixes_length: u64, + pub latest_penalized_exit_length: u64, + pub max_withdrawals_per_epoch: u64, /* * Deposit contract */ From 8dec3c1dc7b90701cef6438e52cc0fe11b80aa31 Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Wed, 16 Jan 2019 10:39:02 +1100 Subject: [PATCH 17/35] Added/modified types and constants according to specs --- beacon_chain/genesis/src/beacon_block.rs | 17 ++----- beacon_chain/genesis/src/beacon_state.rs | 25 ++++++---- beacon_chain/spec/src/foundation.rs | 48 ++++++++++++++----- beacon_chain/spec/src/lib.rs | 39 +++++++++++---- beacon_chain/types/src/beacon_state.rs | 36 +++++++++++--- .../src/candidate_pow_receipt_root_record.rs | 1 + beacon_chain/types/src/validator_record.rs | 20 ++++++++ beacon_chain/utils/bls/src/signature.rs | 18 +++++++ .../validator_induction/src/inductor.rs | 4 ++ 9 files changed, 160 insertions(+), 48 deletions(-) diff --git a/beacon_chain/genesis/src/beacon_block.rs b/beacon_chain/genesis/src/beacon_block.rs index 6caa0c80c..3d0b8f512 100644 --- a/beacon_chain/genesis/src/beacon_block.rs +++ b/beacon_chain/genesis/src/beacon_block.rs @@ -1,17 +1,15 @@ -use bls::{Signature, BLS_AGG_SIG_BYTE_SIZE}; use spec::ChainSpec; -use ssz::{encode::encode_length, Decodable, LENGTH_BYTES}; use types::{BeaconBlock, BeaconBlockBody, Hash256}; /// Generate a genesis BeaconBlock. pub fn genesis_beacon_block(state_root: Hash256, spec: &ChainSpec) -> BeaconBlock { BeaconBlock { - slot: spec.initial_slot_number, + slot: spec.genesis_slot_number, parent_root: spec.zero_hash, state_root, randao_reveal: spec.zero_hash, candidate_pow_receipt_root: spec.zero_hash, - signature: genesis_signature(), + signature: spec.empty_signature.clone(), body: BeaconBlockBody { proposer_slashings: vec![], casper_slashings: vec![], @@ -25,19 +23,11 @@ pub fn genesis_beacon_block(state_root: Hash256, spec: &ChainSpec) -> BeaconBloc } } -fn genesis_signature() -> Signature { - let mut bytes = encode_length(BLS_AGG_SIG_BYTE_SIZE, LENGTH_BYTES); - bytes.append(&mut vec![0; BLS_AGG_SIG_BYTE_SIZE]); - let (signature, _) = match Signature::ssz_decode(&bytes, 0) { - Ok(sig) => sig, - Err(_) => unreachable!(), - }; - signature -} #[cfg(test)] mod tests { use super::*; + use bls::{Signature}; #[test] fn test_genesis() { @@ -101,5 +91,6 @@ mod tests { for item in raw_sig_bytes.iter() { assert!(*item == 0); } + assert_eq!(genesis_block.signature, Signature::empty_sig()); } } diff --git a/beacon_chain/genesis/src/beacon_state.rs b/beacon_chain/genesis/src/beacon_state.rs index 7326bbf1d..09404275d 100644 --- a/beacon_chain/genesis/src/beacon_state.rs +++ b/beacon_chain/genesis/src/beacon_state.rs @@ -21,7 +21,7 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { }; let initial_crosslink = CrosslinkRecord { - slot: spec.initial_slot_number, + slot: spec.genesis_slot_number, shard_block_root: spec.zero_hash, }; @@ -29,19 +29,19 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { /* * Misc */ - slot: spec.initial_slot_number, + slot: spec.genesis_slot_number, genesis_time: spec.genesis_time, fork_data: ForkData { - pre_fork_version: spec.initial_fork_version, - post_fork_version: spec.initial_fork_version, - fork_slot: spec.initial_slot_number, + pre_fork_version: spec.genesis_fork_version, + post_fork_version: spec.genesis_fork_version, + fork_slot: spec.genesis_slot_number, }, /* * Validator registry */ validator_registry: spec.initial_validators.clone(), validator_balances: spec.initial_balances.clone(), - validator_registry_latest_change_slot: spec.initial_slot_number, + validator_registry_latest_change_slot: spec.genesis_slot_number, validator_registry_exit_count: 0, validator_registry_delta_chain_tip: spec.zero_hash, /* @@ -52,7 +52,12 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { spec.zero_hash; (spec.latest_randao_mixes_length / spec.epoch_length) as usize ], - shard_committees_at_slots: vec![], + previous_epoch_start_shard: spec.genesis_start_shard, + current_epoch_start_shard: spec.genesis_start_shard, + previous_epoch_calculation_slot: spec.genesis_slot_number, + current_epoch_calculation_slot: spec.genesis_slot_number, + previous_epoch_randao_mix: spec.zero_hash, + current_epoch_randao_mix: spec.zero_hash, /* * Custody challenges */ @@ -60,10 +65,10 @@ pub fn genesis_beacon_state(spec: &ChainSpec) -> Result { /* * Finality */ - previous_justified_slot: spec.initial_slot_number, - justified_slot: spec.initial_slot_number, + previous_justified_slot: spec.genesis_slot_number, + justified_slot: spec.genesis_slot_number, justification_bitfield: 0, - finalized_slot: spec.initial_slot_number, + finalized_slot: spec.genesis_slot_number, /* * Recent state */ diff --git a/beacon_chain/spec/src/foundation.rs b/beacon_chain/spec/src/foundation.rs index 8e3146ed2..c622bd4a0 100644 --- a/beacon_chain/spec/src/foundation.rs +++ b/beacon_chain/spec/src/foundation.rs @@ -1,5 +1,5 @@ use super::ChainSpec; -use bls::{Keypair, PublicKey, SecretKey}; +use bls::{Keypair, PublicKey, SecretKey, Signature}; use types::{Address, Hash256, ValidatorRecord}; @@ -17,12 +17,11 @@ impl ChainSpec { * Misc */ shard_count: 1_024, - target_committee_size: 256, + target_committee_size: 128, ejection_balance: 16, max_balance_churn_quotient: 32, gwei_per_eth: u64::pow(10, 9), beacon_chain_shard_number: u64::max_value(), - bls_withdrawal_prefix_byte: 0x00, max_casper_votes: 1_024, latest_block_roots_length: 8_192, latest_randao_mixes_length: 8_192, @@ -38,35 +37,58 @@ impl ChainSpec { /* * Initial Values */ - initial_fork_version: 0, - initial_slot_number: 0, + genesis_fork_version: 0, + genesis_slot_number: 0, + genesis_start_shard: 0, + far_future_slot: u64::max_value(), zero_hash: Hash256::zero(), + empty_signature: Signature::empty_sig(), + bls_withdrawal_prefix_byte: 0x00, /* * Time parameters */ slot_duration: 6, min_attestation_inclusion_delay: 4, epoch_length: 64, + seed_lookahead: 64, min_validator_registry_change_interval: 256, pow_receipt_root_voting_period: 1_024, - shard_persistent_committee_change_period: u64::pow(2, 17), - collective_penalty_calculation_period: u64::pow(2, 20), - zero_balance_validator_ttl: u64::pow(2, 22), + min_validator_withdrawal_time: u64::pow(2, 14), + shard_persistent_committee_change_period: u64::pow(2, 17), // old + collective_penalty_calculation_period: u64::pow(2, 20), // old + zero_balance_validator_ttl: u64::pow(2, 22), // old /* * Reward and penalty quotients */ - base_reward_quotient: 2_048, + base_reward_quotient: 1_024, whistleblower_reward_quotient: 512, includer_reward_quotient: 8, - inactivity_penalty_quotient: u64::pow(2, 34), + inactivity_penalty_quotient: u64::pow(2, 24), + /* + * Status flags + */ + initiated_exit: 1, + withdrawable: 2, /* * Max operations per block */ max_proposer_slashings: 16, - max_casper_slashings: 15, + max_casper_slashings: 16, max_attestations: 128, max_deposits: 16, max_exits: 16, + /* + * Validator registry delta flags + */ + activation: 0, + exit: 1, + /* + * Signature domains + */ + domain_deposit: 0, + domain_attestation: 1, + domain_proposal: 2, + domain_exit: 3, /* * Intialization parameters */ @@ -107,6 +129,10 @@ fn initial_validators_for_testing() -> Vec { withdrawal_credentials: Hash256::zero(), randao_commitment: Hash256::zero(), randao_layers: 0, + activation_slot: u64::max_value(), + exit_slot: u64::max_value(), + withdrawal_slot: u64::max_value(), + penalized_slot: u64::max_value(), status: From::from(0), latest_status_change_slot: 0, exit_count: 0, diff --git a/beacon_chain/spec/src/lib.rs b/beacon_chain/spec/src/lib.rs index 5b7bdfae3..cc4ce7380 100644 --- a/beacon_chain/spec/src/lib.rs +++ b/beacon_chain/spec/src/lib.rs @@ -3,6 +3,7 @@ extern crate types; mod foundation; +use bls::Signature; use types::{Address, Hash256, ValidatorRecord}; #[derive(PartialEq, Debug)] @@ -16,7 +17,6 @@ pub struct ChainSpec { pub max_balance_churn_quotient: u64, pub gwei_per_eth: u64, pub beacon_chain_shard_number: u64, - pub bls_withdrawal_prefix_byte: u8, pub max_casper_votes: u64, pub latest_block_roots_length: u64, pub latest_randao_mixes_length: u64, @@ -32,20 +32,26 @@ pub struct ChainSpec { /* * Initial Values */ - pub initial_fork_version: u64, - pub initial_slot_number: u64, + pub genesis_fork_version: u64, + pub genesis_slot_number: u64, + pub genesis_start_shard: u64, + pub far_future_slot: u64, pub zero_hash: Hash256, + pub empty_signature: Signature, + pub bls_withdrawal_prefix_byte: u8, /* * Time parameters */ pub slot_duration: u64, pub min_attestation_inclusion_delay: u64, pub epoch_length: u64, - pub min_validator_registry_change_interval: u64, - pub pow_receipt_root_voting_period: u64, - pub shard_persistent_committee_change_period: u64, - pub collective_penalty_calculation_period: u64, - pub zero_balance_validator_ttl: u64, + pub seed_lookahead: u64, + pub min_validator_registry_change_interval: u64, // a.k.a. entry_exit_delay + pub pow_receipt_root_voting_period: u64, // a.k. deposit_root_voting_period + pub min_validator_withdrawal_time: u64, + pub shard_persistent_committee_change_period: u64, //old + pub collective_penalty_calculation_period: u64, // old + pub zero_balance_validator_ttl: u64, // old /* * Reward and penalty quotients */ @@ -53,6 +59,11 @@ pub struct ChainSpec { pub whistleblower_reward_quotient: u64, pub includer_reward_quotient: u64, pub inactivity_penalty_quotient: u64, + /* + * Status flags + */ + pub initiated_exit: u64, + pub withdrawable: u64, /* * Max operations per block */ @@ -61,6 +72,18 @@ pub struct ChainSpec { pub max_attestations: u64, pub max_deposits: u64, pub max_exits: u64, + /* + * Validator registry delta flags + */ + pub activation: u64, + pub exit: u64, + /* + * Signature domains + */ + pub domain_deposit: u64, + pub domain_attestation: u64, + pub domain_proposal: u64, + pub domain_exit: u64, /* * Intialization parameters */ diff --git a/beacon_chain/types/src/beacon_state.rs b/beacon_chain/types/src/beacon_state.rs index b9994bddd..10d367f19 100644 --- a/beacon_chain/types/src/beacon_state.rs +++ b/beacon_chain/types/src/beacon_state.rs @@ -2,7 +2,6 @@ use super::candidate_pow_receipt_root_record::CandidatePoWReceiptRootRecord; use super::crosslink_record::CrosslinkRecord; use super::fork_data::ForkData; use super::pending_attestation_record::PendingAttestationRecord; -use super::shard_committee::ShardCommittee; use super::validator_record::ValidatorRecord; use super::Hash256; use crate::test_utils::TestRandom; @@ -30,7 +29,12 @@ pub struct BeaconState { // Randomness and committees pub latest_randao_mixes: Vec, pub latest_vdf_outputs: Vec, - pub shard_committees_at_slots: Vec>, + pub previous_epoch_start_shard: u64, + pub current_epoch_start_shard: u64, + pub previous_epoch_calculation_slot: u64, + pub current_epoch_calculation_slot: u64, + pub previous_epoch_randao_mix: Hash256, + pub current_epoch_randao_mix: Hash256, // Custody challenges pub custody_challenges: Vec, @@ -73,7 +77,12 @@ impl Encodable for BeaconState { s.append(&self.validator_registry_delta_chain_tip); s.append(&self.latest_randao_mixes); s.append(&self.latest_vdf_outputs); - s.append(&self.shard_committees_at_slots); + s.append(&self.previous_epoch_start_shard); + s.append(&self.current_epoch_start_shard); + s.append(&self.previous_epoch_calculation_slot); + s.append(&self.current_epoch_calculation_slot); + s.append(&self.previous_epoch_randao_mix); + s.append(&self.current_epoch_randao_mix); s.append(&self.custody_challenges); s.append(&self.previous_justified_slot); s.append(&self.justified_slot); @@ -101,7 +110,12 @@ impl Decodable for BeaconState { let (validator_registry_delta_chain_tip, i) = <_>::ssz_decode(bytes, i)?; let (latest_randao_mixes, i) = <_>::ssz_decode(bytes, i)?; let (latest_vdf_outputs, i) = <_>::ssz_decode(bytes, i)?; - let (shard_committees_at_slots, i) = <_>::ssz_decode(bytes, i)?; + let (previous_epoch_start_shard, i) = <_>::ssz_decode(bytes, i)?; + let (current_epoch_start_shard, i) = <_>::ssz_decode(bytes, i)?; + let (previous_epoch_calculation_slot, i) = <_>::ssz_decode(bytes, i)?; + let (current_epoch_calculation_slot, i) = <_>::ssz_decode(bytes, i)?; + let (previous_epoch_randao_mix, i) = <_>::ssz_decode(bytes, i)?; + let (current_epoch_randao_mix, i) = <_>::ssz_decode(bytes, i)?; let (custody_challenges, i) = <_>::ssz_decode(bytes, i)?; let (previous_justified_slot, i) = <_>::ssz_decode(bytes, i)?; let (justified_slot, i) = <_>::ssz_decode(bytes, i)?; @@ -127,7 +141,12 @@ impl Decodable for BeaconState { validator_registry_delta_chain_tip, latest_randao_mixes, latest_vdf_outputs, - shard_committees_at_slots, + previous_epoch_start_shard, + current_epoch_start_shard, + previous_epoch_calculation_slot, + current_epoch_calculation_slot, + previous_epoch_randao_mix, + current_epoch_randao_mix, custody_challenges, previous_justified_slot, justified_slot, @@ -159,7 +178,12 @@ impl TestRandom for BeaconState { validator_registry_delta_chain_tip: <_>::random_for_test(rng), latest_randao_mixes: <_>::random_for_test(rng), latest_vdf_outputs: <_>::random_for_test(rng), - shard_committees_at_slots: <_>::random_for_test(rng), + previous_epoch_start_shard: <_>::random_for_test(rng), + current_epoch_start_shard: <_>::random_for_test(rng), + previous_epoch_calculation_slot: <_>::random_for_test(rng), + current_epoch_calculation_slot: <_>::random_for_test(rng), + previous_epoch_randao_mix: <_>::random_for_test(rng), + current_epoch_randao_mix: <_>::random_for_test(rng), custody_challenges: <_>::random_for_test(rng), previous_justified_slot: <_>::random_for_test(rng), justified_slot: <_>::random_for_test(rng), diff --git a/beacon_chain/types/src/candidate_pow_receipt_root_record.rs b/beacon_chain/types/src/candidate_pow_receipt_root_record.rs index 5f260c423..9648763f3 100644 --- a/beacon_chain/types/src/candidate_pow_receipt_root_record.rs +++ b/beacon_chain/types/src/candidate_pow_receipt_root_record.rs @@ -3,6 +3,7 @@ use super::Hash256; use crate::test_utils::TestRandom; use rand::RngCore; +// Note: this is refer to as DepositRootVote in specs #[derive(Debug, PartialEq, Clone)] pub struct CandidatePoWReceiptRootRecord { pub candidate_pow_receipt_root: Hash256, diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index 3f4a43261..00e29df64 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -35,6 +35,10 @@ pub struct ValidatorRecord { pub withdrawal_credentials: Hash256, pub randao_commitment: Hash256, pub randao_layers: u64, + pub activation_slot: u64, + pub exit_slot: u64, + pub withdrawal_slot: u64, + pub penalized_slot: u64, pub status: ValidatorStatus, pub latest_status_change_slot: u64, pub exit_count: u64, @@ -99,6 +103,10 @@ impl Encodable for ValidatorRecord { s.append(&self.withdrawal_credentials); s.append(&self.randao_commitment); s.append(&self.randao_layers); + s.append(&self.activation_slot); + s.append(&self.exit_slot); + s.append(&self.withdrawal_slot); + s.append(&self.penalized_slot); s.append(&self.status); s.append(&self.latest_status_change_slot); s.append(&self.exit_count); @@ -114,6 +122,10 @@ impl Decodable for ValidatorRecord { let (withdrawal_credentials, i) = <_>::ssz_decode(bytes, i)?; let (randao_commitment, i) = <_>::ssz_decode(bytes, i)?; let (randao_layers, i) = <_>::ssz_decode(bytes, i)?; + let (activation_slot, i) = <_>::ssz_decode(bytes, i)?; + let (exit_slot, i) = <_>::ssz_decode(bytes, i)?; + let (withdrawal_slot, i) = <_>::ssz_decode(bytes, i)?; + let (penalized_slot, i) = <_>::ssz_decode(bytes, i)?; let (status, i) = <_>::ssz_decode(bytes, i)?; let (latest_status_change_slot, i) = <_>::ssz_decode(bytes, i)?; let (exit_count, i) = <_>::ssz_decode(bytes, i)?; @@ -127,6 +139,10 @@ impl Decodable for ValidatorRecord { withdrawal_credentials, randao_commitment, randao_layers, + activation_slot, + exit_slot, + withdrawal_slot, + penalized_slot, status, latest_status_change_slot, exit_count, @@ -146,6 +162,10 @@ impl TestRandom for ValidatorRecord { withdrawal_credentials: <_>::random_for_test(rng), randao_commitment: <_>::random_for_test(rng), randao_layers: <_>::random_for_test(rng), + activation_slot: <_>::random_for_test(rng), + exit_slot: <_>::random_for_test(rng), + withdrawal_slot: <_>::random_for_test(rng), + penalized_slot: <_>::random_for_test(rng), status: <_>::random_for_test(rng), latest_status_change_slot: <_>::random_for_test(rng), exit_count: <_>::random_for_test(rng), diff --git a/beacon_chain/utils/bls/src/signature.rs b/beacon_chain/utils/bls/src/signature.rs index 242908e21..bd363e34a 100644 --- a/beacon_chain/utils/bls/src/signature.rs +++ b/beacon_chain/utils/bls/src/signature.rs @@ -35,6 +35,12 @@ impl Signature { pub fn as_raw(&self) -> &RawSignature { &self.0 } + + /// Returns a new empty signature. + pub fn empty_sig() -> Self { + let empty: Vec = vec![0; 97]; + Signature(RawSignature::from_bytes(&empty).unwrap()) + } } impl Encodable for Signature { @@ -68,4 +74,16 @@ mod tests { assert_eq!(original, decoded); } + + #[test] + pub fn test_empty_sig() { + let sig = Signature::empty_sig(); + + let sig_as_bytes: Vec = sig.as_raw().as_bytes(); + + assert_eq!(sig_as_bytes.len(), 97); + for one_byte in sig_as_bytes.iter() { + assert_eq!(*one_byte, 0); + } + } } diff --git a/beacon_chain/validator_induction/src/inductor.rs b/beacon_chain/validator_induction/src/inductor.rs index f00c9ec55..fd8768cd2 100644 --- a/beacon_chain/validator_induction/src/inductor.rs +++ b/beacon_chain/validator_induction/src/inductor.rs @@ -44,6 +44,10 @@ pub fn process_deposit( withdrawal_credentials: deposit_input.withdrawal_credentials, randao_commitment: deposit_input.randao_commitment, randao_layers: 0, + activation_slot: spec.far_future_slot, + exit_slot: spec.far_future_slot, + withdrawal_slot: spec.far_future_slot, + penalized_slot: spec.far_future_slot, status: ValidatorStatus::PendingActivation, latest_status_change_slot: state.validator_registry_latest_change_slot, exit_count: 0, From b39f7ac6c2bddb15d9b884ce5a1f36f686c6d8de Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Wed, 16 Jan 2019 11:11:41 +1100 Subject: [PATCH 18/35] Formating previous commit --- beacon_chain/genesis/src/beacon_block.rs | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/beacon_chain/genesis/src/beacon_block.rs b/beacon_chain/genesis/src/beacon_block.rs index 3d0b8f512..b57ffd510 100644 --- a/beacon_chain/genesis/src/beacon_block.rs +++ b/beacon_chain/genesis/src/beacon_block.rs @@ -23,11 +23,10 @@ pub fn genesis_beacon_block(state_root: Hash256, spec: &ChainSpec) -> BeaconBloc } } - #[cfg(test)] mod tests { use super::*; - use bls::{Signature}; + use bls::Signature; #[test] fn test_genesis() { From fe107616ce6ab18fbc8b674b95eff98cee10b7f0 Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Wed, 16 Jan 2019 13:32:03 +1100 Subject: [PATCH 19/35] Finailized updating structs / types to match specs as at 16.1.19 --- beacon_chain/spec/src/foundation.rs | 25 +------ beacon_chain/spec/src/lib.rs | 24 +------ beacon_chain/types/src/validator_record.rs | 15 ++-- .../validator_induction/src/inductor.rs | 68 +++---------------- 4 files changed, 19 insertions(+), 113 deletions(-) diff --git a/beacon_chain/spec/src/foundation.rs b/beacon_chain/spec/src/foundation.rs index c622bd4a0..5583957b2 100644 --- a/beacon_chain/spec/src/foundation.rs +++ b/beacon_chain/spec/src/foundation.rs @@ -51,12 +51,9 @@ impl ChainSpec { min_attestation_inclusion_delay: 4, epoch_length: 64, seed_lookahead: 64, - min_validator_registry_change_interval: 256, + entry_exit_delay: 256, pow_receipt_root_voting_period: 1_024, min_validator_withdrawal_time: u64::pow(2, 14), - shard_persistent_committee_change_period: u64::pow(2, 17), // old - collective_penalty_calculation_period: u64::pow(2, 20), // old - zero_balance_validator_ttl: u64::pow(2, 22), // old /* * Reward and penalty quotients */ @@ -64,11 +61,6 @@ impl ChainSpec { whistleblower_reward_quotient: 512, includer_reward_quotient: 8, inactivity_penalty_quotient: u64::pow(2, 24), - /* - * Status flags - */ - initiated_exit: 1, - withdrawable: 2, /* * Max operations per block */ @@ -77,18 +69,6 @@ impl ChainSpec { max_attestations: 128, max_deposits: 16, max_exits: 16, - /* - * Validator registry delta flags - */ - activation: 0, - exit: 1, - /* - * Signature domains - */ - domain_deposit: 0, - domain_attestation: 1, - domain_proposal: 2, - domain_exit: 3, /* * Intialization parameters */ @@ -133,9 +113,8 @@ fn initial_validators_for_testing() -> Vec { exit_slot: u64::max_value(), withdrawal_slot: u64::max_value(), penalized_slot: u64::max_value(), - status: From::from(0), - latest_status_change_slot: 0, exit_count: 0, + status: From::from(0), custody_commitment: Hash256::zero(), latest_custody_reseed_slot: 0, penultimate_custody_reseed_slot: 0, diff --git a/beacon_chain/spec/src/lib.rs b/beacon_chain/spec/src/lib.rs index cc4ce7380..fbe82794d 100644 --- a/beacon_chain/spec/src/lib.rs +++ b/beacon_chain/spec/src/lib.rs @@ -46,12 +46,9 @@ pub struct ChainSpec { pub min_attestation_inclusion_delay: u64, pub epoch_length: u64, pub seed_lookahead: u64, - pub min_validator_registry_change_interval: u64, // a.k.a. entry_exit_delay - pub pow_receipt_root_voting_period: u64, // a.k. deposit_root_voting_period + pub entry_exit_delay: u64, + pub pow_receipt_root_voting_period: u64, // a.k.a. deposit_root_voting_period pub min_validator_withdrawal_time: u64, - pub shard_persistent_committee_change_period: u64, //old - pub collective_penalty_calculation_period: u64, // old - pub zero_balance_validator_ttl: u64, // old /* * Reward and penalty quotients */ @@ -59,11 +56,6 @@ pub struct ChainSpec { pub whistleblower_reward_quotient: u64, pub includer_reward_quotient: u64, pub inactivity_penalty_quotient: u64, - /* - * Status flags - */ - pub initiated_exit: u64, - pub withdrawable: u64, /* * Max operations per block */ @@ -72,18 +64,6 @@ pub struct ChainSpec { pub max_attestations: u64, pub max_deposits: u64, pub max_exits: u64, - /* - * Validator registry delta flags - */ - pub activation: u64, - pub exit: u64, - /* - * Signature domains - */ - pub domain_deposit: u64, - pub domain_attestation: u64, - pub domain_proposal: u64, - pub domain_exit: u64, /* * Intialization parameters */ diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index 00e29df64..7622946f7 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -39,9 +39,8 @@ pub struct ValidatorRecord { pub exit_slot: u64, pub withdrawal_slot: u64, pub penalized_slot: u64, - pub status: ValidatorStatus, - pub latest_status_change_slot: u64, pub exit_count: u64, + pub status: ValidatorStatus, pub custody_commitment: Hash256, pub latest_custody_reseed_slot: u64, pub penultimate_custody_reseed_slot: u64, @@ -107,9 +106,8 @@ impl Encodable for ValidatorRecord { s.append(&self.exit_slot); s.append(&self.withdrawal_slot); s.append(&self.penalized_slot); - s.append(&self.status); - s.append(&self.latest_status_change_slot); s.append(&self.exit_count); + s.append(&self.status); s.append(&self.custody_commitment); s.append(&self.latest_custody_reseed_slot); s.append(&self.penultimate_custody_reseed_slot); @@ -126,9 +124,8 @@ impl Decodable for ValidatorRecord { let (exit_slot, i) = <_>::ssz_decode(bytes, i)?; let (withdrawal_slot, i) = <_>::ssz_decode(bytes, i)?; let (penalized_slot, i) = <_>::ssz_decode(bytes, i)?; - let (status, i) = <_>::ssz_decode(bytes, i)?; - let (latest_status_change_slot, i) = <_>::ssz_decode(bytes, i)?; let (exit_count, i) = <_>::ssz_decode(bytes, i)?; + let (status, i) = <_>::ssz_decode(bytes, i)?; let (custody_commitment, i) = <_>::ssz_decode(bytes, i)?; let (latest_custody_reseed_slot, i) = <_>::ssz_decode(bytes, i)?; let (penultimate_custody_reseed_slot, i) = <_>::ssz_decode(bytes, i)?; @@ -143,9 +140,8 @@ impl Decodable for ValidatorRecord { exit_slot, withdrawal_slot, penalized_slot, - status, - latest_status_change_slot, exit_count, + status, custody_commitment, latest_custody_reseed_slot, penultimate_custody_reseed_slot, @@ -166,9 +162,8 @@ impl TestRandom for ValidatorRecord { exit_slot: <_>::random_for_test(rng), withdrawal_slot: <_>::random_for_test(rng), penalized_slot: <_>::random_for_test(rng), - status: <_>::random_for_test(rng), - latest_status_change_slot: <_>::random_for_test(rng), exit_count: <_>::random_for_test(rng), + status: <_>::random_for_test(rng), custody_commitment: <_>::random_for_test(rng), latest_custody_reseed_slot: <_>::random_for_test(rng), penultimate_custody_reseed_slot: <_>::random_for_test(rng), diff --git a/beacon_chain/validator_induction/src/inductor.rs b/beacon_chain/validator_induction/src/inductor.rs index fd8768cd2..871217ffe 100644 --- a/beacon_chain/validator_induction/src/inductor.rs +++ b/beacon_chain/validator_induction/src/inductor.rs @@ -13,7 +13,7 @@ pub fn process_deposit( state: &mut BeaconState, deposit: &Deposit, spec: &ChainSpec, -) -> Result { +) -> Result<(), ValidatorInductionError> { let deposit_input = &deposit.deposit_data.deposit_input; let deposit_data = &deposit.deposit_data; @@ -33,7 +33,7 @@ pub fn process_deposit( == deposit_input.withdrawal_credentials { state.validator_balances[i] += deposit_data.value; - return Ok(i); + return Ok(()); } Err(ValidatorInductionError::InvalidWithdrawalCredentials) @@ -48,43 +48,21 @@ pub fn process_deposit( exit_slot: spec.far_future_slot, withdrawal_slot: spec.far_future_slot, penalized_slot: spec.far_future_slot, - status: ValidatorStatus::PendingActivation, - latest_status_change_slot: state.validator_registry_latest_change_slot, exit_count: 0, + status: ValidatorStatus::PendingActivation, custody_commitment: deposit_input.custody_commitment, latest_custody_reseed_slot: 0, penultimate_custody_reseed_slot: 0, }; - match min_empty_validator_index(state, spec) { - Some(i) => { - state.validator_registry[i] = validator; - state.validator_balances[i] = deposit_data.value; - Ok(i) - } - None => { - state.validator_registry.push(validator); - state.validator_balances.push(deposit_data.value); - Ok(state.validator_registry.len() - 1) - } - } + let _index = state.validator_registry.len(); + state.validator_registry.push(validator); + state.validator_balances.push(deposit_data.value); + Ok(()) } } } -fn min_empty_validator_index(state: &BeaconState, spec: &ChainSpec) -> Option { - for i in 0..state.validator_registry.len() { - if state.validator_balances[i] == 0 - && state.validator_registry[i].latest_status_change_slot - + spec.zero_balance_validator_ttl - <= state.slot - { - return Some(i); - } - } - None -} - #[cfg(test)] mod tests { use super::*; @@ -129,7 +107,7 @@ mod tests { let result = process_deposit(&mut state, &deposit, &spec); - assert_eq!(result.unwrap(), 0); + assert_eq!(result.unwrap(), ()); assert!(deposit_equals_record( &deposit, &state.validator_registry[0] @@ -147,7 +125,7 @@ mod tests { let mut deposit = get_deposit(); let result = process_deposit(&mut state, &deposit, &spec); deposit.deposit_data.value = DEPOSIT_GWEI; - assert_eq!(result.unwrap(), i); + assert_eq!(result.unwrap(), ()); assert!(deposit_equals_record( &deposit, &state.validator_registry[i] @@ -176,7 +154,7 @@ mod tests { let result = process_deposit(&mut state, &deposit, &spec); - assert_eq!(result.unwrap(), 0); + assert_eq!(result.unwrap(), ()); assert!(deposit_equals_record( &deposit, &state.validator_registry[0] @@ -186,32 +164,6 @@ mod tests { assert_eq!(state.validator_balances.len(), 1); } - #[test] - fn test_process_deposit_replace_validator() { - let mut state = BeaconState::default(); - let spec = ChainSpec::foundation(); - - let mut validator = get_validator(); - validator.latest_status_change_slot = 0; - state.validator_registry.push(validator); - state.validator_balances.push(0); - - let mut deposit = get_deposit(); - deposit.deposit_data.value = DEPOSIT_GWEI; - state.slot = spec.zero_balance_validator_ttl; - - let result = process_deposit(&mut state, &deposit, &spec); - - assert_eq!(result.unwrap(), 0); - assert!(deposit_equals_record( - &deposit, - &state.validator_registry[0] - )); - assert_eq!(state.validator_balances[0], DEPOSIT_GWEI); - assert_eq!(state.validator_registry.len(), 1); - assert_eq!(state.validator_balances.len(), 1); - } - #[test] fn test_process_deposit_invalid_proof_of_possession() { let mut state = BeaconState::default(); From f74bf597dd3c17455cb45a26d3ba2f897874ef29 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 09:52:38 -0600 Subject: [PATCH 20/35] Adds some new fields from the latest spec --- beacon_chain/types/src/validator_record.rs | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index 3f4a43261..8a562ceb4 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -35,9 +35,12 @@ pub struct ValidatorRecord { pub withdrawal_credentials: Hash256, pub randao_commitment: Hash256, pub randao_layers: u64, - pub status: ValidatorStatus, - pub latest_status_change_slot: u64, + pub activation_slot: u64, + pub exit_slot: u64, + pub withdrawal_slot: u64, + pub penalized_slot: u64, pub exit_count: u64, + pub status: ValidatorStatus, pub custody_commitment: Hash256, pub latest_custody_reseed_slot: u64, pub penultimate_custody_reseed_slot: u64, From 6928301b91d6a605519ac936da365a51f7e2695e Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 8 Jan 2019 16:55:04 -0600 Subject: [PATCH 21/35] Updates `ValidatorRecord` to match the latest spec --- beacon_chain/types/src/lib.rs | 2 +- beacon_chain/types/src/validator_record.rs | 98 ++++++++++------------ 2 files changed, 43 insertions(+), 57 deletions(-) diff --git a/beacon_chain/types/src/lib.rs b/beacon_chain/types/src/lib.rs index 42bea9e0a..f6847356a 100644 --- a/beacon_chain/types/src/lib.rs +++ b/beacon_chain/types/src/lib.rs @@ -50,7 +50,7 @@ pub use crate::proposer_slashing::ProposerSlashing; pub use crate::shard_committee::ShardCommittee; pub use crate::slashable_vote_data::SlashableVoteData; pub use crate::special_record::{SpecialRecord, SpecialRecordKind}; -pub use crate::validator_record::{ValidatorRecord, ValidatorStatus}; +pub use crate::validator_record::{StatusFlags as ValidatorStatusFlags, ValidatorRecord}; pub type Hash256 = H256; pub type Address = H160; diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index 8a562ceb4..05980559e 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -5,25 +5,20 @@ use rand::RngCore; use ssz::{Decodable, DecodeError, Encodable, SszStream}; use std::convert; +const STATUS_FLAG_INITIATED_EXIT: u8 = 1; +const STATUS_FLAG_WITHDRAWABLE: u8 = 2; + #[derive(Debug, PartialEq, Clone, Copy)] -pub enum ValidatorStatus { - PendingActivation, - Active, - PendingExit, - PendingWithdraw, - Withdrawn, - Penalized, +pub enum StatusFlags { + InitiatedExit, + Withdrawable, } -impl convert::From for ValidatorStatus { - fn from(status: u8) -> Self { - match status { - 0 => ValidatorStatus::PendingActivation, - 1 => ValidatorStatus::Active, - 2 => ValidatorStatus::PendingExit, - 3 => ValidatorStatus::PendingWithdraw, - 5 => ValidatorStatus::Withdrawn, - 127 => ValidatorStatus::Penalized, +impl convert::From for StatusFlags { + fn from(status_flag: u8) -> Self { + match status_flag { + STATUS_FLAG_INITIATED_EXIT => StatusFlags::InitiatedExit, + STATUS_FLAG_WITHDRAWABLE => StatusFlags::Withdrawable, _ => unreachable!(), } } @@ -40,59 +35,38 @@ pub struct ValidatorRecord { pub withdrawal_slot: u64, pub penalized_slot: u64, pub exit_count: u64, - pub status: ValidatorStatus, + pub status_flags: StatusFlags, pub custody_commitment: Hash256, pub latest_custody_reseed_slot: u64, pub penultimate_custody_reseed_slot: u64, } -impl ValidatorRecord { - pub fn status_is(&self, status: ValidatorStatus) -> bool { - self.status == status - } -} - -impl Encodable for ValidatorStatus { +impl Encodable for StatusFlags { fn ssz_append(&self, s: &mut SszStream) { let byte: u8 = match self { - ValidatorStatus::PendingActivation => 0, - ValidatorStatus::Active => 1, - ValidatorStatus::PendingExit => 2, - ValidatorStatus::PendingWithdraw => 3, - ValidatorStatus::Withdrawn => 5, - ValidatorStatus::Penalized => 127, + StatusFlags::InitiatedExit => STATUS_FLAG_INITIATED_EXIT, + StatusFlags::Withdrawable => STATUS_FLAG_WITHDRAWABLE, }; s.append(&byte); } } -impl Decodable for ValidatorStatus { +impl Decodable for StatusFlags { fn ssz_decode(bytes: &[u8], i: usize) -> Result<(Self, usize), DecodeError> { let (byte, i) = u8::ssz_decode(bytes, i)?; let status = match byte { - 0 => ValidatorStatus::PendingActivation, - 1 => ValidatorStatus::Active, - 2 => ValidatorStatus::PendingExit, - 3 => ValidatorStatus::PendingWithdraw, - 5 => ValidatorStatus::Withdrawn, - 127 => ValidatorStatus::Penalized, + 1 => StatusFlags::InitiatedExit, + 2 => StatusFlags::Withdrawable, _ => return Err(DecodeError::Invalid), }; Ok((status, i)) } } -impl TestRandom for ValidatorStatus { +impl TestRandom for StatusFlags { fn random_for_test(rng: &mut T) -> Self { - let options = vec![ - ValidatorStatus::PendingActivation, - ValidatorStatus::Active, - ValidatorStatus::PendingExit, - ValidatorStatus::PendingWithdraw, - ValidatorStatus::Withdrawn, - ValidatorStatus::Penalized, - ]; - options[(rng.next_u32() as usize) % options.len()] + let options = vec![StatusFlags::InitiatedExit, StatusFlags::Withdrawable]; + options[(rng.next_u32() as usize) % options.len()].clone() } } @@ -102,9 +76,12 @@ impl Encodable for ValidatorRecord { s.append(&self.withdrawal_credentials); s.append(&self.randao_commitment); s.append(&self.randao_layers); - s.append(&self.status); - s.append(&self.latest_status_change_slot); + s.append(&self.activation_slot); + s.append(&self.exit_slot); + s.append(&self.withdrawal_slot); + s.append(&self.penalized_slot); s.append(&self.exit_count); + s.append(&self.status_flags); s.append(&self.custody_commitment); s.append(&self.latest_custody_reseed_slot); s.append(&self.penultimate_custody_reseed_slot); @@ -117,9 +94,12 @@ impl Decodable for ValidatorRecord { let (withdrawal_credentials, i) = <_>::ssz_decode(bytes, i)?; let (randao_commitment, i) = <_>::ssz_decode(bytes, i)?; let (randao_layers, i) = <_>::ssz_decode(bytes, i)?; - let (status, i) = <_>::ssz_decode(bytes, i)?; - let (latest_status_change_slot, i) = <_>::ssz_decode(bytes, i)?; + let (activation_slot, i) = <_>::ssz_decode(bytes, i)?; + let (exit_slot, i) = <_>::ssz_decode(bytes, i)?; + let (withdrawal_slot, i) = <_>::ssz_decode(bytes, i)?; + let (penalized_slot, i) = <_>::ssz_decode(bytes, i)?; let (exit_count, i) = <_>::ssz_decode(bytes, i)?; + let (status_flags, i) = <_>::ssz_decode(bytes, i)?; let (custody_commitment, i) = <_>::ssz_decode(bytes, i)?; let (latest_custody_reseed_slot, i) = <_>::ssz_decode(bytes, i)?; let (penultimate_custody_reseed_slot, i) = <_>::ssz_decode(bytes, i)?; @@ -130,9 +110,12 @@ impl Decodable for ValidatorRecord { withdrawal_credentials, randao_commitment, randao_layers, - status, - latest_status_change_slot, + activation_slot, + exit_slot, + withdrawal_slot, + penalized_slot, exit_count, + status_flags, custody_commitment, latest_custody_reseed_slot, penultimate_custody_reseed_slot, @@ -149,9 +132,12 @@ impl TestRandom for ValidatorRecord { withdrawal_credentials: <_>::random_for_test(rng), randao_commitment: <_>::random_for_test(rng), randao_layers: <_>::random_for_test(rng), - status: <_>::random_for_test(rng), - latest_status_change_slot: <_>::random_for_test(rng), + activation_slot: <_>::random_for_test(rng), + exit_slot: <_>::random_for_test(rng), + withdrawal_slot: <_>::random_for_test(rng), + penalized_slot: <_>::random_for_test(rng), exit_count: <_>::random_for_test(rng), + status_flags: <_>::random_for_test(rng), custody_commitment: <_>::random_for_test(rng), latest_custody_reseed_slot: <_>::random_for_test(rng), penultimate_custody_reseed_slot: <_>::random_for_test(rng), @@ -179,7 +165,7 @@ mod tests { #[test] pub fn test_validator_status_ssz_round_trip() { let mut rng = XorShiftRng::from_seed([42; 16]); - let original = ValidatorStatus::random_for_test(&mut rng); + let original = StatusFlags::random_for_test(&mut rng); let bytes = ssz_encode(&original); let (decoded, _) = <_>::ssz_decode(&bytes, 0).unwrap(); From 16cc8556e83b2a1ec59836871fec849d3e502059 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 11:30:33 -0600 Subject: [PATCH 22/35] Covers the 'missing' `StatusFlag` when the value is `0`. Strongly types the field by using Option<_>. Updates the serialization and test generation to account for the type. --- beacon_chain/types/src/validator_record.rs | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index 05980559e..bc0b6b6b5 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -35,7 +35,7 @@ pub struct ValidatorRecord { pub withdrawal_slot: u64, pub penalized_slot: u64, pub exit_count: u64, - pub status_flags: StatusFlags, + pub status_flags: Option, pub custody_commitment: Hash256, pub latest_custody_reseed_slot: u64, pub penultimate_custody_reseed_slot: u64, @@ -81,7 +81,11 @@ impl Encodable for ValidatorRecord { s.append(&self.withdrawal_slot); s.append(&self.penalized_slot); s.append(&self.exit_count); - s.append(&self.status_flags); + if let Some(status_flags) = self.status_flags { + s.append(&status_flags); + } else { + s.append(&(0 as u8)); + } s.append(&self.custody_commitment); s.append(&self.latest_custody_reseed_slot); s.append(&self.penultimate_custody_reseed_slot); @@ -99,11 +103,17 @@ impl Decodable for ValidatorRecord { let (withdrawal_slot, i) = <_>::ssz_decode(bytes, i)?; let (penalized_slot, i) = <_>::ssz_decode(bytes, i)?; let (exit_count, i) = <_>::ssz_decode(bytes, i)?; - let (status_flags, i) = <_>::ssz_decode(bytes, i)?; + let (status_flags_byte, i): (u8, usize) = <_>::ssz_decode(bytes, i)?; let (custody_commitment, i) = <_>::ssz_decode(bytes, i)?; let (latest_custody_reseed_slot, i) = <_>::ssz_decode(bytes, i)?; let (penultimate_custody_reseed_slot, i) = <_>::ssz_decode(bytes, i)?; + let status_flags = if status_flags_byte == 0u8 { + None + } else { + Some(StatusFlags::from(status_flags_byte)) + }; + Ok(( Self { pubkey, @@ -137,7 +147,7 @@ impl TestRandom for ValidatorRecord { withdrawal_slot: <_>::random_for_test(rng), penalized_slot: <_>::random_for_test(rng), exit_count: <_>::random_for_test(rng), - status_flags: <_>::random_for_test(rng), + status_flags: Some(<_>::random_for_test(rng)), custody_commitment: <_>::random_for_test(rng), latest_custody_reseed_slot: <_>::random_for_test(rng), penultimate_custody_reseed_slot: <_>::random_for_test(rng), From 4d3d351b673562321059cd917b3b8db17c3e67cc Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 11:32:54 -0600 Subject: [PATCH 23/35] Adds a `std::default::Default` implementation for `ValidatorRecord` Updates the test generation so that it uses sane values for some marker values like `FAR_FUTURE_SLOT` --- beacon_chain/types/src/validator_record.rs | 26 ++++++++++++++++++---- beacon_chain/utils/bls/src/public_key.rs | 8 +++++++ 2 files changed, 30 insertions(+), 4 deletions(-) diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index bc0b6b6b5..7077d87e2 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -4,6 +4,7 @@ use crate::test_utils::TestRandom; use rand::RngCore; use ssz::{Decodable, DecodeError, Encodable, SszStream}; use std::convert; +use std::default; const STATUS_FLAG_INITIATED_EXIT: u8 = 1; const STATUS_FLAG_WITHDRAWABLE: u8 = 2; @@ -41,6 +42,26 @@ pub struct ValidatorRecord { pub penultimate_custody_reseed_slot: u64, } +impl default::Default for ValidatorRecord { + fn default() -> Self { + Self { + pubkey: PublicKey::default(), + withdrawal_credentials: Hash256::default(), + randao_commitment: Hash256::default(), + randao_layers: 0, + activation_slot: std::u64::MAX, + exit_slot: std::u64::MAX, + withdrawal_slot: std::u64::MAX, + penalized_slot: std::u64::MAX, + exit_count: 0, + status_flags: None, + custody_commitment: Hash256::default(), + latest_custody_reseed_slot: 0, // NOTE: is `GENESIS_SLOT` + penultimate_custody_reseed_slot: 0, // NOTE: is `GENESIS_SLOT` + } + } +} + impl Encodable for StatusFlags { fn ssz_append(&self, s: &mut SszStream) { let byte: u8 = match self { @@ -142,15 +163,12 @@ impl TestRandom for ValidatorRecord { withdrawal_credentials: <_>::random_for_test(rng), randao_commitment: <_>::random_for_test(rng), randao_layers: <_>::random_for_test(rng), - activation_slot: <_>::random_for_test(rng), - exit_slot: <_>::random_for_test(rng), - withdrawal_slot: <_>::random_for_test(rng), - penalized_slot: <_>::random_for_test(rng), exit_count: <_>::random_for_test(rng), status_flags: Some(<_>::random_for_test(rng)), custody_commitment: <_>::random_for_test(rng), latest_custody_reseed_slot: <_>::random_for_test(rng), penultimate_custody_reseed_slot: <_>::random_for_test(rng), + ..Self::default() } } } diff --git a/beacon_chain/utils/bls/src/public_key.rs b/beacon_chain/utils/bls/src/public_key.rs index 49dbc9e4b..e7950969e 100644 --- a/beacon_chain/utils/bls/src/public_key.rs +++ b/beacon_chain/utils/bls/src/public_key.rs @@ -1,6 +1,7 @@ use super::SecretKey; use bls_aggregates::PublicKey as RawPublicKey; use ssz::{decode_ssz_list, Decodable, DecodeError, Encodable, SszStream}; +use std::default; /// A single BLS signature. /// @@ -20,6 +21,13 @@ impl PublicKey { } } +impl default::Default for PublicKey { + fn default() -> Self { + let secret_key = SecretKey::random(); + PublicKey::from_secret_key(&secret_key) + } +} + impl Encodable for PublicKey { fn ssz_append(&self, s: &mut SszStream) { s.append_vec(&self.0.as_bytes()); From 42d950eb88f27605c505710a1ecea213ac8989af Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 11:33:45 -0600 Subject: [PATCH 24/35] Implements `is_active_validator` helper --- beacon_chain/types/src/validator_record.rs | 28 ++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index 7077d87e2..0e1102a59 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -42,6 +42,12 @@ pub struct ValidatorRecord { pub penultimate_custody_reseed_slot: u64, } +impl ValidatorRecord { + pub fn is_active_at(&self, slot: u64) -> bool { + self.activation_slot <= slot && slot < self.exit_slot + } +} + impl default::Default for ValidatorRecord { fn default() -> Self { Self { @@ -200,4 +206,26 @@ mod tests { assert_eq!(original, decoded); } + + #[test] + fn test_validator_can_be_active() { + let mut rng = XorShiftRng::from_seed([42; 16]); + let mut validator = ValidatorRecord::random_for_test(&mut rng); + + let activation_slot = u64::random_for_test(&mut rng); + let exit_slot = activation_slot + 234; + + validator.activation_slot = activation_slot; + validator.exit_slot = exit_slot; + + for slot in (activation_slot - 100)..(exit_slot + 100) { + if slot < activation_slot { + assert!(!validator.is_active_at(slot)); + } else if slot >= exit_slot { + assert!(!validator.is_active_at(slot)); + } else { + assert!(validator.is_active_at(slot)); + } + } + } } From 9720c548afea1a3b004569f4e3c5cecd4cf2b039 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 12:56:13 -0600 Subject: [PATCH 25/35] Adds `get_active_validator_indices` helper --- beacon_chain/types/src/lib.rs | 1 + beacon_chain/types/src/validator_registry.rs | 170 +++++++++++++++++++ 2 files changed, 171 insertions(+) create mode 100644 beacon_chain/types/src/validator_registry.rs diff --git a/beacon_chain/types/src/lib.rs b/beacon_chain/types/src/lib.rs index f6847356a..b89cbc50f 100644 --- a/beacon_chain/types/src/lib.rs +++ b/beacon_chain/types/src/lib.rs @@ -26,6 +26,7 @@ pub mod shard_reassignment_record; pub mod slashable_vote_data; pub mod special_record; pub mod validator_record; +pub mod validator_registry; pub mod readers; diff --git a/beacon_chain/types/src/validator_registry.rs b/beacon_chain/types/src/validator_registry.rs new file mode 100644 index 000000000..1171f7acf --- /dev/null +++ b/beacon_chain/types/src/validator_registry.rs @@ -0,0 +1,170 @@ +/// Contains logic to manipulate a `&[ValidatorRecord]`. +/// For now, we avoid defining a newtype and just have flat functions here. +use super::validator_record::*; + +pub fn get_active_validator_indices(validators: &[ValidatorRecord], slot: u64) -> Vec { + validators + .iter() + .enumerate() + .filter_map(|(index, validator)| { + if validator.is_active_at(slot) { + Some(index) + } else { + None + } + }) + .collect::>() +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::test_utils::{SeedableRng, TestRandom, XorShiftRng}; + + #[test] + fn can_get_empty_active_validator_indices() { + let mut rng = XorShiftRng::from_seed([42; 16]); + + let validators = vec![]; + let some_slot = u64::random_for_test(&mut rng); + let indices = get_active_validator_indices(&validators, some_slot); + assert_eq!(indices, vec![]); + } + + #[test] + fn can_get_no_active_validator_indices() { + let mut rng = XorShiftRng::from_seed([42; 16]); + let mut validators = vec![]; + let count_validators = 10; + for _ in 0..count_validators { + validators.push(ValidatorRecord::default()) + } + + let some_slot = u64::random_for_test(&mut rng); + let indices = get_active_validator_indices(&validators, some_slot); + assert_eq!(indices, vec![]); + } + + #[test] + fn can_get_all_active_validator_indices() { + let mut rng = XorShiftRng::from_seed([42; 16]); + let count_validators = 10; + let some_slot = u64::random_for_test(&mut rng); + + let mut validators = (0..count_validators) + .into_iter() + .map(|_| { + let mut validator = ValidatorRecord::default(); + + let activation_offset = u64::random_for_test(&mut rng); + let exit_offset = u64::random_for_test(&mut rng); + + validator.activation_slot = some_slot.checked_sub(activation_offset).unwrap_or(0); + validator.exit_slot = some_slot.checked_add(exit_offset).unwrap_or(std::u64::MAX); + + validator + }) + .collect::>(); + + // test boundary condition by ensuring that at least one validator in the list just activated + if let Some(validator) = validators.get_mut(0) { + validator.activation_slot = some_slot; + } + + let indices = get_active_validator_indices(&validators, some_slot); + assert_eq!( + indices, + (0..count_validators).into_iter().collect::>() + ); + } + + fn set_validators_to_default_entry_exit(validators: &mut [ValidatorRecord]) { + for validator in validators.iter_mut() { + validator.activation_slot = std::u64::MAX; + validator.exit_slot = std::u64::MAX; + } + } + + // sets all `validators` to be active as of some slot prior to `slot`. returns the activation slot. + fn set_validators_to_activated(validators: &mut [ValidatorRecord], slot: u64) -> u64 { + let activation_slot = slot - 10; + for validator in validators.iter_mut() { + validator.activation_slot = activation_slot; + } + activation_slot + } + + // sets all `validators` to be exited as of some slot before `slot`. + fn set_validators_to_exited( + validators: &mut [ValidatorRecord], + slot: u64, + activation_slot: u64, + ) { + assert!(activation_slot < slot); + let mut exit_slot = activation_slot + 10; + while exit_slot >= slot { + exit_slot -= 1; + } + assert!(activation_slot < exit_slot && exit_slot < slot); + + for validator in validators.iter_mut() { + validator.exit_slot = exit_slot; + } + } + + #[test] + fn can_get_some_active_validator_indices() { + let mut rng = XorShiftRng::from_seed([42; 16]); + const COUNT_PARTITIONS: usize = 3; + const COUNT_VALIDATORS: usize = 3 * COUNT_PARTITIONS; + let some_slot: u64 = u64::random_for_test(&mut rng); + + let mut validators = (0..COUNT_VALIDATORS) + .into_iter() + .map(|_| { + let mut validator = ValidatorRecord::default(); + + let activation_offset = u64::random_for_test(&mut rng); + let exit_offset = u64::random_for_test(&mut rng); + + validator.activation_slot = some_slot.checked_sub(activation_offset).unwrap_or(0); + validator.exit_slot = some_slot.checked_add(exit_offset).unwrap_or(std::u64::MAX); + + validator + }) + .collect::>(); + + // we partition the set into partitions based on lifecycle: + for (i, chunk) in validators.chunks_exact_mut(COUNT_PARTITIONS).enumerate() { + match i { + 0 => { + // 1. not activated (Default::default()) + set_validators_to_default_entry_exit(chunk); + } + 1 => { + // 2. activated, but not exited + set_validators_to_activated(chunk, some_slot); + // test boundary condition by ensuring that at least one validator in the list just activated + if let Some(validator) = chunk.get_mut(0) { + validator.activation_slot = some_slot; + } + } + 2 => { + // 3. exited + let activation_slot = set_validators_to_activated(chunk, some_slot); + set_validators_to_exited(chunk, some_slot, activation_slot); + // test boundary condition by ensuring that at least one validator in the list just exited + if let Some(validator) = chunk.get_mut(0) { + validator.exit_slot = some_slot; + } + } + _ => unreachable!( + "constants local to this test not in sync with generation of test case" + ), + } + } + + let indices = get_active_validator_indices(&validators, some_slot); + assert_eq!(indices, vec![3, 4, 5]); + } +} From 869f36ffbec78233a91f501ae3d03eedf8a3c71a Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 14:46:56 -0600 Subject: [PATCH 26/35] Use the `Default::default` validator for testing --- beacon_chain/spec/src/foundation.rs | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/beacon_chain/spec/src/foundation.rs b/beacon_chain/spec/src/foundation.rs index 1a493d72c..de433ea6b 100644 --- a/beacon_chain/spec/src/foundation.rs +++ b/beacon_chain/spec/src/foundation.rs @@ -101,15 +101,7 @@ fn initial_validators_for_testing() -> Vec { }; let validator_record = ValidatorRecord { pubkey: keypair.pk.clone(), - withdrawal_credentials: Hash256::zero(), - randao_commitment: Hash256::zero(), - randao_layers: 0, - status: From::from(0), - latest_status_change_slot: 0, - exit_count: 0, - custody_commitment: Hash256::zero(), - latest_custody_reseed_slot: 0, - penultimate_custody_reseed_slot: 0, + ..std::default::Default::default() }; initial_validators.push(validator_record); } From 2d1b61522b8e66461d61c1f6c6764d78f0cc44ef Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 14:47:25 -0600 Subject: [PATCH 27/35] Uses the new `get_active_validator_indices` implementation --- beacon_chain/validator_shuffling/src/shuffle.rs | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/beacon_chain/validator_shuffling/src/shuffle.rs b/beacon_chain/validator_shuffling/src/shuffle.rs index 5b31f8e03..13bdf97fe 100644 --- a/beacon_chain/validator_shuffling/src/shuffle.rs +++ b/beacon_chain/validator_shuffling/src/shuffle.rs @@ -2,7 +2,8 @@ use std::cmp::min; use honey_badger_split::SplitExt; use spec::ChainSpec; -use types::{ShardCommittee, ValidatorRecord, ValidatorStatus}; +use types::validator_registry::get_active_validator_indices; +use types::{ShardCommittee, ValidatorRecord}; use vec_shuffle::{shuffle, ShuffleErr}; type DelegatedCycle = Vec>; @@ -24,17 +25,7 @@ pub fn shard_and_committees_for_cycle( spec: &ChainSpec, ) -> Result { let shuffled_validator_indices = { - let validator_indices = validators - .iter() - .enumerate() - .filter_map(|(i, validator)| { - if validator.status_is(ValidatorStatus::Active) { - Some(i) - } else { - None - } - }) - .collect(); + let validator_indices = get_active_validator_indices(validators, 0); shuffle(seed, validator_indices)? }; let shard_indices: Vec = (0_usize..spec.shard_count as usize).into_iter().collect(); From b09d44c235f1e65aa630345e211f823409bc06fa Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Thu, 10 Jan 2019 14:49:06 -0600 Subject: [PATCH 28/35] Gets this module to compile, at expense of deviating from old spec Some changes to integrate with the newer `ValidatorRecord` type. Deviates from the old spec but should be updated shortly with newer logic so the breaking change is only temporary. --- .../validator_induction/src/inductor.rs | 20 +++++++------------ 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/beacon_chain/validator_induction/src/inductor.rs b/beacon_chain/validator_induction/src/inductor.rs index f00c9ec55..4ac9d7924 100644 --- a/beacon_chain/validator_induction/src/inductor.rs +++ b/beacon_chain/validator_induction/src/inductor.rs @@ -1,6 +1,6 @@ use bls::verify_proof_of_possession; use spec::ChainSpec; -use types::{BeaconState, Deposit, ValidatorRecord, ValidatorStatus}; +use types::{BeaconState, Deposit, ValidatorRecord}; #[derive(Debug, PartialEq, Clone)] pub enum ValidatorInductionError { @@ -43,13 +43,10 @@ pub fn process_deposit( pubkey: deposit_input.pubkey.clone(), withdrawal_credentials: deposit_input.withdrawal_credentials, randao_commitment: deposit_input.randao_commitment, - randao_layers: 0, - status: ValidatorStatus::PendingActivation, - latest_status_change_slot: state.validator_registry_latest_change_slot, - exit_count: 0, custody_commitment: deposit_input.custody_commitment, latest_custody_reseed_slot: 0, penultimate_custody_reseed_slot: 0, + ..std::default::Default::default() }; match min_empty_validator_index(state, spec) { @@ -68,13 +65,11 @@ pub fn process_deposit( } } -fn min_empty_validator_index(state: &BeaconState, spec: &ChainSpec) -> Option { +// NOTE: this has been modified from the spec to get tests working +// this function is no longer used in the latest spec so this is simply a transition step +fn min_empty_validator_index(state: &BeaconState, _spec: &ChainSpec) -> Option { for i in 0..state.validator_registry.len() { - if state.validator_balances[i] == 0 - && state.validator_registry[i].latest_status_change_slot - + spec.zero_balance_validator_ttl - <= state.slot - { + if state.validator_balances[i] == 0 { return Some(i); } } @@ -187,8 +182,7 @@ mod tests { let mut state = BeaconState::default(); let spec = ChainSpec::foundation(); - let mut validator = get_validator(); - validator.latest_status_change_slot = 0; + let validator = get_validator(); state.validator_registry.push(validator); state.validator_balances.push(0); From c9a9f0e305f8d1fb4e488225335d130bc038496b Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 15 Jan 2019 17:03:45 -0800 Subject: [PATCH 29/35] Remove imports already in the prelude --- beacon_chain/types/src/validator_record.rs | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index 0e1102a59..4f2ada4ad 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -3,8 +3,6 @@ use super::Hash256; use crate::test_utils::TestRandom; use rand::RngCore; use ssz::{Decodable, DecodeError, Encodable, SszStream}; -use std::convert; -use std::default; const STATUS_FLAG_INITIATED_EXIT: u8 = 1; const STATUS_FLAG_WITHDRAWABLE: u8 = 2; @@ -15,7 +13,7 @@ pub enum StatusFlags { Withdrawable, } -impl convert::From for StatusFlags { +impl From for StatusFlags { fn from(status_flag: u8) -> Self { match status_flag { STATUS_FLAG_INITIATED_EXIT => StatusFlags::InitiatedExit, @@ -48,7 +46,7 @@ impl ValidatorRecord { } } -impl default::Default for ValidatorRecord { +impl Default for ValidatorRecord { fn default() -> Self { Self { pubkey: PublicKey::default(), From 1f32da81e733a117c122cc77a16d11b8f4284885 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 15 Jan 2019 17:23:29 -0800 Subject: [PATCH 30/35] Simplify how the `StatusFlags` are handled wrt serialization --- beacon_chain/types/src/validator_record.rs | 75 ++++++++-------------- 1 file changed, 26 insertions(+), 49 deletions(-) diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index 4f2ada4ad..e63e3ea2c 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -13,13 +13,31 @@ pub enum StatusFlags { Withdrawable, } -impl From for StatusFlags { - fn from(status_flag: u8) -> Self { - match status_flag { - STATUS_FLAG_INITIATED_EXIT => StatusFlags::InitiatedExit, - STATUS_FLAG_WITHDRAWABLE => StatusFlags::Withdrawable, - _ => unreachable!(), +struct StatusFlagsDecodeError; + +impl From for DecodeError { + fn from(_: StatusFlagsDecodeError) -> DecodeError { + DecodeError::Invalid + } +} + +fn status_flag_to_byte(flag: Option) -> u8 { + if let Some(flag) = flag { + match flag { + StatusFlags::InitiatedExit => STATUS_FLAG_INITIATED_EXIT, + StatusFlags::Withdrawable => STATUS_FLAG_WITHDRAWABLE, } + } else { + 0 + } +} + +fn status_flag_from_byte(flag: u8) -> Result, StatusFlagsDecodeError> { + match flag { + 0 => Ok(None), + 1 => Ok(Some(StatusFlags::InitiatedExit)), + 2 => Ok(Some(StatusFlags::Withdrawable)), + _ => Err(StatusFlagsDecodeError), } } @@ -66,28 +84,6 @@ impl Default for ValidatorRecord { } } -impl Encodable for StatusFlags { - fn ssz_append(&self, s: &mut SszStream) { - let byte: u8 = match self { - StatusFlags::InitiatedExit => STATUS_FLAG_INITIATED_EXIT, - StatusFlags::Withdrawable => STATUS_FLAG_WITHDRAWABLE, - }; - s.append(&byte); - } -} - -impl Decodable for StatusFlags { - fn ssz_decode(bytes: &[u8], i: usize) -> Result<(Self, usize), DecodeError> { - let (byte, i) = u8::ssz_decode(bytes, i)?; - let status = match byte { - 1 => StatusFlags::InitiatedExit, - 2 => StatusFlags::Withdrawable, - _ => return Err(DecodeError::Invalid), - }; - Ok((status, i)) - } -} - impl TestRandom for StatusFlags { fn random_for_test(rng: &mut T) -> Self { let options = vec![StatusFlags::InitiatedExit, StatusFlags::Withdrawable]; @@ -106,11 +102,7 @@ impl Encodable for ValidatorRecord { s.append(&self.withdrawal_slot); s.append(&self.penalized_slot); s.append(&self.exit_count); - if let Some(status_flags) = self.status_flags { - s.append(&status_flags); - } else { - s.append(&(0 as u8)); - } + s.append(&status_flag_to_byte(self.status_flags)); s.append(&self.custody_commitment); s.append(&self.latest_custody_reseed_slot); s.append(&self.penultimate_custody_reseed_slot); @@ -133,11 +125,7 @@ impl Decodable for ValidatorRecord { let (latest_custody_reseed_slot, i) = <_>::ssz_decode(bytes, i)?; let (penultimate_custody_reseed_slot, i) = <_>::ssz_decode(bytes, i)?; - let status_flags = if status_flags_byte == 0u8 { - None - } else { - Some(StatusFlags::from(status_flags_byte)) - }; + let status_flags = status_flag_from_byte(status_flags_byte)?; Ok(( Self { @@ -194,17 +182,6 @@ mod tests { assert_eq!(original, decoded); } - #[test] - pub fn test_validator_status_ssz_round_trip() { - let mut rng = XorShiftRng::from_seed([42; 16]); - let original = StatusFlags::random_for_test(&mut rng); - - let bytes = ssz_encode(&original); - let (decoded, _) = <_>::ssz_decode(&bytes, 0).unwrap(); - - assert_eq!(original, decoded); - } - #[test] fn test_validator_can_be_active() { let mut rng = XorShiftRng::from_seed([42; 16]); From 01a20acb12c9ca682aedd736bbc86f23750ce606 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 15 Jan 2019 18:19:26 -0800 Subject: [PATCH 31/35] Remove import already in prelude --- beacon_chain/spec/src/foundation.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/beacon_chain/spec/src/foundation.rs b/beacon_chain/spec/src/foundation.rs index de433ea6b..315be168e 100644 --- a/beacon_chain/spec/src/foundation.rs +++ b/beacon_chain/spec/src/foundation.rs @@ -101,7 +101,7 @@ fn initial_validators_for_testing() -> Vec { }; let validator_record = ValidatorRecord { pubkey: keypair.pk.clone(), - ..std::default::Default::default() + ..Default::default() }; initial_validators.push(validator_record); } From 306bcd6f8d235d637ef038f593f98014ccec20c4 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 15 Jan 2019 21:20:09 -0800 Subject: [PATCH 32/35] Add some clarifying doc comments --- beacon_chain/types/src/validator_record.rs | 4 ++++ beacon_chain/types/src/validator_registry.rs | 1 + 2 files changed, 5 insertions(+) diff --git a/beacon_chain/types/src/validator_record.rs b/beacon_chain/types/src/validator_record.rs index e63e3ea2c..df0ce4d4c 100644 --- a/beacon_chain/types/src/validator_record.rs +++ b/beacon_chain/types/src/validator_record.rs @@ -21,6 +21,7 @@ impl From for DecodeError { } } +/// Handles the serialization logic for the `status_flags` field of the `ValidatorRecord`. fn status_flag_to_byte(flag: Option) -> u8 { if let Some(flag) = flag { match flag { @@ -32,6 +33,7 @@ fn status_flag_to_byte(flag: Option) -> u8 { } } +/// Handles the deserialization logic for the `status_flags` field of the `ValidatorRecord`. fn status_flag_from_byte(flag: u8) -> Result, StatusFlagsDecodeError> { match flag { 0 => Ok(None), @@ -59,12 +61,14 @@ pub struct ValidatorRecord { } impl ValidatorRecord { + /// This predicate indicates if the validator represented by this record is considered "active" at `slot`. pub fn is_active_at(&self, slot: u64) -> bool { self.activation_slot <= slot && slot < self.exit_slot } } impl Default for ValidatorRecord { + /// Yields a "default" `ValidatorRecord`. Primarily used for testing. fn default() -> Self { Self { pubkey: PublicKey::default(), diff --git a/beacon_chain/types/src/validator_registry.rs b/beacon_chain/types/src/validator_registry.rs index 1171f7acf..abb2e6b3f 100644 --- a/beacon_chain/types/src/validator_registry.rs +++ b/beacon_chain/types/src/validator_registry.rs @@ -2,6 +2,7 @@ /// For now, we avoid defining a newtype and just have flat functions here. use super::validator_record::*; +/// Given an indexed sequence of `validators`, return the indices corresponding to validators that are active at `slot`. pub fn get_active_validator_indices(validators: &[ValidatorRecord], slot: u64) -> Vec { validators .iter() From 0e3d0848acd82450762c3ed5b599ef3cb2c7b96c Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Thu, 17 Jan 2019 17:59:33 +1100 Subject: [PATCH 33/35] Relove formatting and other issues raise in PR --- beacon_chain/genesis/Cargo.toml | 1 - beacon_chain/genesis/src/beacon_block.rs | 4 ---- beacon_chain/genesis/src/beacon_state.rs | 26 ++++++++---------------- beacon_chain/types/src/beacon_state.rs | 2 +- 4 files changed, 9 insertions(+), 24 deletions(-) diff --git a/beacon_chain/genesis/Cargo.toml b/beacon_chain/genesis/Cargo.toml index dbfd7e1fc..499333979 100644 --- a/beacon_chain/genesis/Cargo.toml +++ b/beacon_chain/genesis/Cargo.toml @@ -6,7 +6,6 @@ edition = "2018" [dependencies] bls = { path = "../utils/bls" } -bls-aggregates = { git = "https://github.com/sigp/signature-schemes" } spec = { path = "../spec" } ssz = { path = "../utils/ssz" } types = { path = "../types" } diff --git a/beacon_chain/genesis/src/beacon_block.rs b/beacon_chain/genesis/src/beacon_block.rs index 6caa0c80c..92230cd98 100644 --- a/beacon_chain/genesis/src/beacon_block.rs +++ b/beacon_chain/genesis/src/beacon_block.rs @@ -53,7 +53,6 @@ mod tests { fn test_zero_items() { let spec = ChainSpec::foundation(); - // Note: state_root will not be available without a state (test in beacon_state) let state_root = Hash256::zero(); let genesis_block = genesis_beacon_block(state_root, &spec); @@ -69,7 +68,6 @@ mod tests { fn test_beacon_body() { let spec = ChainSpec::foundation(); - // Note: state_root will not be available without a state (test in beacon_state) let state_root = Hash256::zero(); let genesis_block = genesis_beacon_block(state_root, &spec); @@ -87,7 +85,6 @@ mod tests { fn test_signature() { let spec = ChainSpec::foundation(); - // Note: state_root will not be available without a state (test in beacon_state) let state_root = Hash256::zero(); let genesis_block = genesis_beacon_block(state_root, &spec); @@ -97,7 +94,6 @@ mod tests { let raw_sig = genesis_block.signature.as_raw(); let raw_sig_bytes = raw_sig.as_bytes(); - assert!(raw_sig_bytes.len() == 97); for item in raw_sig_bytes.iter() { assert!(*item == 0); } diff --git a/beacon_chain/genesis/src/beacon_state.rs b/beacon_chain/genesis/src/beacon_state.rs index 7326bbf1d..58feca840 100644 --- a/beacon_chain/genesis/src/beacon_state.rs +++ b/beacon_chain/genesis/src/beacon_state.rs @@ -88,14 +88,11 @@ impl From for Error { #[cfg(test)] mod tests { - extern crate bls; - extern crate validator_induction; - use super::*; use types::Hash256; #[test] - fn test_gen_state() { + fn test_genesis_state() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); @@ -106,9 +103,8 @@ mod tests { ); } - // Test Misc #[test] - fn test_gen_state_misc() { + fn test_genesis_state_misc() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); @@ -120,9 +116,8 @@ mod tests { assert_eq!(state.fork_data.fork_slot, 0); } - // Test validators #[test] - fn test_gen_state_validators() { + fn test_genesis_state_validators() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); @@ -134,9 +129,8 @@ mod tests { assert_eq!(state.validator_registry_delta_chain_tip, Hash256::zero()); } - // Test randomness and committees #[test] - fn test_gen_state_randomness_committees() { + fn test_genesis_state_randomness_committees() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); @@ -161,13 +155,11 @@ mod tests { } // Custody not implemented until Phase 1 - // This test will always pass until Phase 1 #[test] - fn test_gen_state_custody() {} + fn test_genesis_state_custody() {} - // Test finality #[test] - fn test_gen_state_finanilty() { + fn test_genesis_state_finanilty() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); @@ -178,9 +170,8 @@ mod tests { assert_eq!(state.finalized_slot, 0); } - // Test recent state #[test] - fn test_gen_state_recent_state() { + fn test_genesis_state_recent_state() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); @@ -211,9 +202,8 @@ mod tests { assert!(state.batched_block_roots.is_empty()); } - // Test PoW Receipts a.k.a. deposits #[test] - fn test_gen_state_deposit_root() { + fn test_genesis_state_deposit_root() { let spec = ChainSpec::foundation(); let state = genesis_beacon_state(&spec).unwrap(); diff --git a/beacon_chain/types/src/beacon_state.rs b/beacon_chain/types/src/beacon_state.rs index b9994bddd..cbcb00d19 100644 --- a/beacon_chain/types/src/beacon_state.rs +++ b/beacon_chain/types/src/beacon_state.rs @@ -10,7 +10,7 @@ use hashing::canonical_hash; use rand::RngCore; use ssz::{ssz_encode, Decodable, DecodeError, Encodable, SszStream}; -// Custody will not be added to the specs until Phase 1 (Sharding Phase) so dummay class used. +// Custody will not be added to the specs until Phase 1 (Sharding Phase) so dummy class used. type CustodyChallenge = usize; #[derive(Debug, PartialEq, Clone, Default)] From dc5986b8c07d0a054f53de9b1e7d50e1bc72f6dc Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Fri, 18 Jan 2019 09:20:28 +1100 Subject: [PATCH 34/35] Minor formatting --- beacon_chain/genesis/src/beacon_block.rs | 2 -- 1 file changed, 2 deletions(-) diff --git a/beacon_chain/genesis/src/beacon_block.rs b/beacon_chain/genesis/src/beacon_block.rs index 92230cd98..78968b7a6 100644 --- a/beacon_chain/genesis/src/beacon_block.rs +++ b/beacon_chain/genesis/src/beacon_block.rs @@ -48,7 +48,6 @@ mod tests { genesis_beacon_block(state_root, &spec); } - // Tests items that are 0 or zero_hash #[test] fn test_zero_items() { let spec = ChainSpec::foundation(); @@ -63,7 +62,6 @@ mod tests { assert!(genesis_block.candidate_pow_receipt_root.is_zero()); // aka deposit_root } - // Tests the BeaconBlockBody inside BeaconBlock #[test] fn test_beacon_body() { let spec = ChainSpec::foundation(); From a0e7f8830336b7c0391e675a419fce918d4fef16 Mon Sep 17 00:00:00 2001 From: Kirk Baird Date: Fri, 18 Jan 2019 09:59:07 +1100 Subject: [PATCH 35/35] Merge changes from genesis_test and minor formatting --- beacon_chain/genesis/src/beacon_block.rs | 2 +- beacon_chain/spec/src/foundation.rs | 2 +- beacon_chain/utils/bls/src/signature.rs | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/beacon_chain/genesis/src/beacon_block.rs b/beacon_chain/genesis/src/beacon_block.rs index 5c2124ccc..3fefff6a0 100644 --- a/beacon_chain/genesis/src/beacon_block.rs +++ b/beacon_chain/genesis/src/beacon_block.rs @@ -84,6 +84,6 @@ mod tests { for item in raw_sig_bytes.iter() { assert!(*item == 0); } - assert_eq!(genesis_block.signature, Signature::empty_sig()); + assert_eq!(genesis_block.signature, Signature::empty_signature()); } } diff --git a/beacon_chain/spec/src/foundation.rs b/beacon_chain/spec/src/foundation.rs index 5583957b2..2e3f92df6 100644 --- a/beacon_chain/spec/src/foundation.rs +++ b/beacon_chain/spec/src/foundation.rs @@ -42,7 +42,7 @@ impl ChainSpec { genesis_start_shard: 0, far_future_slot: u64::max_value(), zero_hash: Hash256::zero(), - empty_signature: Signature::empty_sig(), + empty_signature: Signature::empty_signature(), bls_withdrawal_prefix_byte: 0x00, /* * Time parameters diff --git a/beacon_chain/utils/bls/src/signature.rs b/beacon_chain/utils/bls/src/signature.rs index bd363e34a..07fc2baff 100644 --- a/beacon_chain/utils/bls/src/signature.rs +++ b/beacon_chain/utils/bls/src/signature.rs @@ -37,7 +37,7 @@ impl Signature { } /// Returns a new empty signature. - pub fn empty_sig() -> Self { + pub fn empty_signature() -> Self { let empty: Vec = vec![0; 97]; Signature(RawSignature::from_bytes(&empty).unwrap()) } @@ -76,8 +76,8 @@ mod tests { } #[test] - pub fn test_empty_sig() { - let sig = Signature::empty_sig(); + pub fn test_empty_signature() { + let sig = Signature::empty_signature(); let sig_as_bytes: Vec = sig.as_raw().as_bytes();