Commit Graph

1227 Commits

Author SHA1 Message Date
Paul Hauner
2c6b40be78
Allow slot clock to work on genesis (#573)
* Allow slot clock to work on genesis

* Loose over-strict requirements for slot clock tests
2019-10-29 12:51:32 +11:00
gnattishness
a488c4dccd Rename get_permutated_index to compute_shuffled_index. (#564)
To match name of equivalent function in v0.8.3 spec.
2019-10-29 12:07:12 +11:00
Paul Hauner
dd370b2e33
Revert "Implement push_leaf for MerkleTree (#563)" (#572)
This reverts commit 13faa47da1.
2019-10-26 15:17:29 +11:00
pscott
13faa47da1 Implement push_leaf for MerkleTree (#563)
* Prototype for far_right push

* Add push method and tests

* Modify beacon_chain_builder for interop to use push instead of create

* Add Push method to MerkleTree

* Cargo fmt

* Remove redundant tests

* Fix typo

* Rename push to push_leaf

* Fix clippy warnings

* Add DepthTooSmall enum variant

* Avoid cloning in MerkleTree::push_leaf

* Add quickcheck test for push_leaf

* Cargo fmt updated

* Return err instead of using unwrap()

* Use enumerate instead of hard indexing

* Use if let and return string on error

* Fix typo in deposit_leave
2019-10-25 16:07:03 +11:00
Michael Sproul
6c2871637d Delete unused validator_change crate (#569)
Hopefully for the last time...
2019-10-25 11:52:19 +11:00
Michael Sproul
96a327fe50
Update Milagro BLS to v0.11.0 (#554) 2019-10-02 12:04:17 +10:00
Paul Hauner
9aedb12bfc
Refactor docs into mdbook (#547)
* Refactor documentation, ef_tests makefile

* Add makefile to root

* Tidy readme

* Fix readme badges

* Tidy logs

* Add terminalize gif

* Update readme image

* Update readme image

* Tidy logs

* Update readme image

* Update readme

* Update readme

* Fix book link

* Update makefiles

* Update book

* Fix link in book

* Add readme for book

* Remove old docs, move api spec YAML

* Fix eth2/ dir readme

* Add readme for lcli

* Add about this book section

* Minor formatting improvements

* Address mehdi's comments
2019-10-02 11:04:56 +10:00
pscott
7eb82125ef Clippy clean (#536)
* Change into_iter to iter

* Fix clippy 'easy' warnings

* Clippy eth2/utils

* Add struct NetworkInfo

* Clippy for types, utils, and beacon_node/store/src/iters.rs

* Cargo fmt

* Change foo to my_foo

* Remove complex signature

* suppress clippy warning for unit_value in benches

* Use enumerate instead of iterating over range

* Allow trivially_copy_pass_by_ref in serde_utils
2019-09-30 13:58:45 +10:00
Michael Sproul
392b4203d1
Consolidate lazy_static versions 2019-09-30 12:36:42 +10:00
Michael Sproul
01db3b6f4e
Fix SignedRoot macro attribute parsing 2019-09-30 12:32:44 +10:00
Michael Sproul
81325a0e05
Run rustfmt 2019-09-30 11:30:47 +10:00
Michael Sproul
3d559d8b41
Robustify derive macro attribute parsing (#544)
Missing whitespace in the implementation of `TokenStream::to_string` on
beta and nightly was breaking our parsing of derive macro attributes.

This change makes the parser ignore whitespace, and should make the beta
and nightly builds succeed again.
2019-09-28 14:29:14 +10:00
Paul Hauner
97a45cdcb8
Address Michael's comments 2019-09-26 13:00:31 +10:00
Paul Hauner
8ceb2e3d95
Refactor slot clock to remove underflow
Previously I had used `Instant` to refer to the genesis time.
2019-09-21 11:21:47 +10:00
Paul Hauner
879f5bfdd0
Fix signature serialization bug 2019-09-14 22:25:53 -04:00
Paul Hauner
8c5a8034b6
Add whiteblock script, CLI options to support it 2019-09-10 12:13:54 -04:00
Paul Hauner
d9a4dbd912
Add nimbus-specific test 2019-09-09 12:22:09 -04:00
Paul Hauner
15220ae565
Fix minor vec access panic opportunity 2019-09-09 01:55:43 -04:00
Paul Hauner
e1d6e187d1
Fix bug in crosslink rewards during per-epoch 2019-09-09 01:54:32 -04:00
Paul Hauner
0e5f33d928
Removes testin dbg!'s
Hopefully this appeases @gregthegreek
2019-09-08 18:37:30 -04:00
Paul Hauner
1040c80cd8
Add noisey debugging to attestation processing 2019-09-08 16:28:01 -04:00
Paul Hauner
6311b13169
Add ssz benches to state processing 2019-09-08 13:42:38 -04:00
Paul Hauner
6a870d468c
Add ssz_fixed_len method to ssz::Encode 2019-09-08 12:23:37 -04:00
Paul Hauner
bef21d9218
Merge branch 'v0.8.3' into interop 2019-09-06 22:04:13 +10:00
Michael Sproul
1c9e4bc09c
Fix bug in SSZ encoding of FixedVector 2019-09-06 17:05:40 +10:00
Paul Hauner
8b69a48fc5
Allow validator client to start before genesis 2019-09-06 10:03:45 +10:00
Paul Hauner
940ddd0d13
Use michael's milagro in interop keypairs 2019-09-05 20:57:48 +10:00
Paul Hauner
3bc62ef411
Merge branch 'v0.8.3' into interop-v0.8.3 2019-09-05 17:39:23 +10:00
Michael Sproul
24ebad9168
Merge remote-tracking branch 'origin/master' into v0.8.3 2019-09-05 12:27:58 +10:00
Michael Sproul
d511c939eb
SSZ generic tests for big uints 2019-09-04 14:32:04 +10:00
Paul Hauner
572df4f37e
Make bootstrapper block til connection established 2019-09-04 13:56:30 +10:00
Paul Hauner
7edc5f37b9
Move BeaconChainHarness to interop spec 2019-09-04 10:25:30 +10:00
Michael Sproul
f47eaf5730
Parallel tests and SSZ generic 2019-09-03 16:46:10 +10:00
Michael Sproul
8d5a579aa6
Fix BeaconChain tests 2019-09-03 10:20:51 +10:00
Paul Hauner
5a8c31e6bf
Remove JSON epoch test 2019-09-02 16:29:05 +10:00
Paul Hauner
d05c2d4110
Start working on u64 json issue 2019-09-02 13:21:53 +10:00
Paul Hauner
d4bf1390c9
Add interop eth1 data stub 2019-09-02 11:39:28 +10:00
Paul Hauner
543e9457b7
Move bootstrapper into own crate 2019-09-01 19:32:57 +10:00
Paul Hauner
96fb3be2c7
Swap endianness of test keys 2019-08-30 17:02:57 +10:00
Paul Hauner
a6e6827337
Remove pub const from interop keypairs 2019-08-30 16:48:22 +10:00
Paul Hauner
9ffb6d0fe1
Fix fake_crypto test fails 2019-08-30 15:49:06 +10:00
Paul Hauner
6234adc0d6
Add interop-spec genesis procedure 2019-08-30 15:33:34 +10:00
Michael Sproul
e154b30232
merkle_proof: implement tree construction
Plus QuickCheck tests!
2019-08-30 14:10:44 +10:00
Paul Hauner
25f2e212c3
Update to latest interop keypair spec 2019-08-30 13:30:07 +10:00
Michael Sproul
6cf9b3c1a4
Epoch processing tests 2019-08-30 13:29:26 +10:00
Paul Hauner
2e11faf763
Re-enable signature verification on attn incl 2019-08-30 11:19:52 +10:00
Paul Hauner
a474061ec7
Disable sig verification when filling blocks 2019-08-30 11:14:13 +10:00
Michael Sproul
81cafdc804
Shuffling and sanity tests 2019-08-29 17:41:20 +10:00
Paul Hauner
29c8477242
Merge branch 'master' into interop 2019-08-29 16:00:22 +10:00
Paul Hauner
3acd75f876
Merge branch 'millsecond-slots' into interop 2019-08-29 14:33:40 +10:00
Paul Hauner
8cfbe8bbfb
Change seconds_per_slot to milliseconds_per_slot 2019-08-29 14:32:21 +10:00
Paul Hauner
7d03806107
Upgrade codebase to new SlotClock API 2019-08-29 14:26:30 +10:00
Paul Hauner
e9e912323e
Restrict fork choice iterators to the root 2019-08-29 13:56:00 +10:00
Paul Hauner
bcd53a8b10
Migrate codebase across to new SlotClock API 2019-08-29 13:25:55 +10:00
Paul Hauner
7bfe02be1c
Refactor slot clock. 2019-08-29 12:46:18 +10:00
Paul Hauner
bcffe42712
Bulk signature verification (#507)
* Add basic block processing benches

* Start reviving state processing benches

* Fix old block builders

* Add optimization for faster pubkey add

* Tidy benches, add another

* Add extra block processing bench

* Start working on faster BLS scheme

* Add partially complete sig verify optimization

* Add .gitignore to state processing

* Add progress on faster signature verification

* Fix SignatureSet for fake_crypto

* Tidy attester slashings sig set

* Tidy bulk signature verifier

* Refactor signature sets to be cleaner

* Start threading SignatureStrategy through code

* Add (empty) test dir

* Move BenchingBlockBuilder

* Add initial block signature verification tests

* Add tests for bulk signature verification

* Start threading SignatureStrategy in block proc.

* Refactor per_block_processing errors

* Use sig set tuples instead of lists of two

* Remove dead code

* Thread VerifySignatures through per_block_processing

* Add bulk signature verification

* Introduce parallel bulk signature verification

* Expand state processing benches

* Fix additional compile errors

* Fix issue where par iter chunks is 0

* Update milagro_bls dep

* Remove debugs, code fragment in beacon chain

* Tidy, add comments to block sig verifier

* Fix various PR comments

* Add block_root option to per_block_processing

* Fix comment in block signature verifier

* Fix comments from PR review

* Remove old comment

* Fix comment
2019-08-29 11:34:25 +10:00
Michael Sproul
23a308e595
BLS and SSZ static tests 2019-08-28 18:46:16 +10:00
Kyle Liu
74af13a372 Fix this link per @blacktemplar to link to the actual crate (#518) 2019-08-28 09:32:54 +10:00
Luke Anderson
328f11d564
Validator API (#504)
* Implemented more REST API endpoints.
 - Added many of the endpoints, which return 501 - Not Implemented
 - Created helper function to return a not implemented error
 - Created a parse_pubkey function to get a PublicKey from a hex string
 - Created a HTTP handler for the validator endpoints
 - Started implementing validator/duties endpoint.

* Fleshed out get validator duties.
 - Re-implemented the get validator duties function for the REST API
 - Added an 'as_hex_string' function to FakePublicKey, beacuse it was missing.

* Fixed small caching/state bug.

* Extended to array of API inputs.
 - Created  function for getting arrays from GET parameters.
 - Extended get validator duties function to support array of validator duties.

* Tidy API to be more consistent with recent decisions

* Addressing Paul's comments.
 - Cleaning up function to get list of proposers.
 - Removing unnecessary serde annotations
 - Clarifying error messages
 - Only accept pubkeys if they are '0x' prefixed.

* Fixed formatting with rustfmt.
2019-08-28 00:40:35 +10:00
Paul Hauner
6bb3a65189
Guard reduced tree from errors 2019-08-27 18:09:31 +10:00
Michael Sproul
4d2cdc9492
Update to spec v0.8.3 2019-08-27 16:59:53 +10:00
Kyle Liu
82666e932e Removing old SSZ readme and replacing it with cargo badge. (#514)
* Removing old ssz readme per paulhauner, replacing with cargo badge.

Co-Authored-by: Paul Hauner <paul@paulhauner.com>

* Replacing my shitty text badge with an actual badge per Paul.

Co-Authored-by: Paul Hauner <paul@paulhauner.com>
2019-08-27 12:02:36 +10:00
Paul Hauner
ed6c39e25a
Add log for fork choice integrity in beacon chain 2019-08-27 11:19:50 +10:00
Michael Sproul
5a7903a377
Improve BeaconState safe accessors
And fix a bug in the compact committees accessor.
2019-08-26 11:39:48 +10:00
Paul Hauner
0c3fdcd57c
Bootstrap (#501)
* Renamed fork_choice::process_attestation_from_block

* Processing attestation in fork choice

* Retrieving state from store and checking signature

* Looser check on beacon state validity.

* Cleaned up get_attestation_state

* Expanded fork choice api to provide latest validator message.

* Checking if the an attestation contains a latest message

* Correct process_attestation error handling.

* Copy paste error in comment fixed.

* Tidy ancestor iterators

* Getting attestation slot via helper method

* Refactored attestation creation in test utils

* Revert "Refactored attestation creation in test utils"

This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306.

* Integration tests for free attestation processing

* Implicit conflicts resolved.

* formatting

* Do first pass on Grants code

* Add another attestation processing test

* Tidy attestation processing

* Remove old code fragment

* Add non-compiling half finished changes

* Simplify, fix bugs, add tests for chain iters

* Remove attestation processing from op pool

* Fix bug with fork choice, tidy

* Fix overly restrictive check in fork choice.

* Ensure committee cache is build during attn proc

* Ignore unknown blocks at fork choice

* Various minor fixes

* Make fork choice write lock in to read lock

* Remove unused method

* Tidy comments

* Fix attestation prod. target roots change

* Fix compile error in store iters

* Reject any attestation prior to finalization

* Begin metrics refactor

* Move beacon_chain to new metrics structure.

* Make metrics not panic if already defined

* Use global prometheus gather at rest api

* Unify common metric fns into a crate

* Add heavy metering to block processing

* Remove hypen from prometheus metric name

* Add more beacon chain metrics

* Add beacon chain persistence metric

* Prune op pool on finalization

* Add extra prom beacon chain metrics

* Prefix BeaconChain metrics with "beacon_"

* Add more store metrics

* Add basic metrics to libp2p

* Add metrics to HTTP server

* Remove old `http_server` crate

* Update metrics names to be more like standard

* Fix broken beacon chain metrics, add slot clock metrics

* Add lighthouse_metrics gather fn

* Remove http args

* Fix wrong state given to op pool prune

* Make prom metric names more consistent

* Add more metrics, tidy existing metrics

* Fix store block read metrics

* Tidy attestation metrics

* Fix minor PR comments

* Fix minor PR comments

* Remove duplicated attestation finalization check

* Remove awkward `let` statement

* Add first attempts at HTTP bootstrap

* Add beacon_block methods to rest api

* Fix serde for block.body.grafitti

* Allow travis failures on beta (see desc)

There's a non-backward compatible change in `cargo fmt`. Stable and beta
do not agree.

* Add network routes to API

* Fix rustc warnings

* Add best_slot method

* Add --bootstrap arg to beacon node

* Get bootstrapper working for ENR address

* Store intermediate states during block processing

* Allow bootstrapper to scrape libp2p address

* Update bootstrapper libp2p address finding

* Add comments

* Tidy API to be more consistent with recent decisions

* Address some review comments

* Make BeaconChainTypes Send + Sync + 'static

* Add `/network/listen_port` API endpoint

* Abandon starting the node if libp2p doesn't start

* Update bootstrapper for API changes

* Remove unnecessary trait bounds
2019-08-23 15:53:53 +10:00
Paul Hauner
c4ced3e0d2
Fix block processing blowup, upgrade metrics (#500)
* Renamed fork_choice::process_attestation_from_block

* Processing attestation in fork choice

* Retrieving state from store and checking signature

* Looser check on beacon state validity.

* Cleaned up get_attestation_state

* Expanded fork choice api to provide latest validator message.

* Checking if the an attestation contains a latest message

* Correct process_attestation error handling.

* Copy paste error in comment fixed.

* Tidy ancestor iterators

* Getting attestation slot via helper method

* Refactored attestation creation in test utils

* Revert "Refactored attestation creation in test utils"

This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306.

* Integration tests for free attestation processing

* Implicit conflicts resolved.

* formatting

* Do first pass on Grants code

* Add another attestation processing test

* Tidy attestation processing

* Remove old code fragment

* Add non-compiling half finished changes

* Simplify, fix bugs, add tests for chain iters

* Remove attestation processing from op pool

* Fix bug with fork choice, tidy

* Fix overly restrictive check in fork choice.

* Ensure committee cache is build during attn proc

* Ignore unknown blocks at fork choice

* Various minor fixes

* Make fork choice write lock in to read lock

* Remove unused method

* Tidy comments

* Fix attestation prod. target roots change

* Fix compile error in store iters

* Reject any attestation prior to finalization

* Begin metrics refactor

* Move beacon_chain to new metrics structure.

* Make metrics not panic if already defined

* Use global prometheus gather at rest api

* Unify common metric fns into a crate

* Add heavy metering to block processing

* Remove hypen from prometheus metric name

* Add more beacon chain metrics

* Add beacon chain persistence metric

* Prune op pool on finalization

* Add extra prom beacon chain metrics

* Prefix BeaconChain metrics with "beacon_"

* Add more store metrics

* Add basic metrics to libp2p

* Add metrics to HTTP server

* Remove old `http_server` crate

* Update metrics names to be more like standard

* Fix broken beacon chain metrics, add slot clock metrics

* Add lighthouse_metrics gather fn

* Remove http args

* Fix wrong state given to op pool prune

* Make prom metric names more consistent

* Add more metrics, tidy existing metrics

* Fix store block read metrics

* Tidy attestation metrics

* Fix minor PR comments

* Allow travis failures on beta (see desc)

There's a non-backward compatible change in `cargo fmt`. Stable and beta
do not agree.

* Tidy `lighthouse_metrics` docs

* Fix typo
2019-08-19 21:02:34 +10:00
Paul Hauner
cd26a19a70
Attestation processing (#497)
* Renamed fork_choice::process_attestation_from_block

* Processing attestation in fork choice

* Retrieving state from store and checking signature

* Looser check on beacon state validity.

* Cleaned up get_attestation_state

* Expanded fork choice api to provide latest validator message.

* Checking if the an attestation contains a latest message

* Correct process_attestation error handling.

* Copy paste error in comment fixed.

* Tidy ancestor iterators

* Getting attestation slot via helper method

* Refactored attestation creation in test utils

* Revert "Refactored attestation creation in test utils"

This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306.

* Integration tests for free attestation processing

* Implicit conflicts resolved.

* formatting

* Do first pass on Grants code

* Add another attestation processing test

* Tidy attestation processing

* Remove old code fragment

* Add non-compiling half finished changes

* Simplify, fix bugs, add tests for chain iters

* Remove attestation processing from op pool

* Fix bug with fork choice, tidy

* Fix overly restrictive check in fork choice.

* Ensure committee cache is build during attn proc

* Ignore unknown blocks at fork choice

* Various minor fixes

* Make fork choice write lock in to read lock

* Remove unused method

* Tidy comments

* Fix attestation prod. target roots change

* Fix compile error in store iters

* Reject any attestation prior to finalization

* Fix minor PR comments

* Remove duplicated attestation finalization check

* Remove awkward `let` statement
2019-08-14 10:55:24 +10:00
Age Manning
468015f9bb Initial Interop Updates (#492)
* Add interop chain spec and rename chain_id

* Add ability to connect to raw libp2p nodes

* Adds Identify protocol, clean up RPC protocol name handling

* Update to latest libp2p, gossipsub improvements

* Updates to latest interop branch.

- Shifts decoding of objects into message handler.
- Updates to latest interop gossipsub.
- Adds interop spec constant.

* Configuration updates allow for verbosity CLI flag and spec constants

* Update submodules to master

* Correct minimal chainspec modifications

* Duplication of validator polls are no longer fatal

* Apply PR suggestions
2019-08-10 11:44:17 +10:00
Paul Hauner
3a1f56a42e
Tidy ancestor iterators (#490)
* Tidy ancestor iterators

* Improve comments, remove code fragement
2019-08-08 12:28:10 +10:00
Paul Hauner
8cfa36fedd
Publish ssz_types (and deps) to crates.io (#468)
* Rename `hashing` crate to `eth2_hashing`

* Add license, desc to eth2_hashing Cargo.toml

* Remove merkle root from eth2 hashing

* Remove old benches folder (zombied from old branch)

* Add docs to eth2_hashing

* Prepare tree_hash for publishing on crates.io

* Update deps to use crates.io instead of paths

* Update all crates to pull ssz from crates.io

* Remove cached_tree_hash, add patches to manifest

* Fix compile error in benches

* Remove unused code

* Fix fake_crypto compile error
2019-08-08 11:39:47 +10:00
Pawan Dhananjay
88e89f9ab2 Update ethereum types (#489)
* Update ethereum-types to version 0.6

* Fix tests

* Run rustfmt
2019-08-06 14:41:42 +10:00
blacktemplar
01054ecf2f Use SignatureBytes and PublicKeyBytes for deposits (#472)
* Replace deposit signatures with SignatureBytes, a struct which lazyly parsers signatures only on demand.

* check byte length when parsing SignatureBytes

* add comment to struct

* distinguish BadSignature and BadSignatureBytes in verify_deposit_signature

* add test for valid signature

* Implements TryInto<Signature> for &SignatureBytes and From<Signature> for &SignatureBytes

* add and use PublicKeyBytes + fix formatting

* fix compiler warning + docs for macro generated structs

* adds tests to ensure correct byte lengths

* small style improvement as suggested by michaelsproul
2019-08-06 13:49:11 +10:00
Paul Hauner
845f336a59
Interop chain start strategies (#479)
* Implement more flexible beacon chain genesis

* Fix compile issues from rebase on master

* Rename CLI flag

* Adds initial documentation for TOML files

* Update docs readme

* Add first version of cli_util

* Dont write cache fields in serde

* Tidy cli_util

* Add code to load genesis YAML file

* Move serde_utils out of tests in `types`

* Update logging text

* Fix serde YAML for Fork

* Make yaml hex decoding more strict

* Update deterministic key generate for interop

* Set deposit count on testing genesis state

* Make some fixes for deposit count

* Remove code fragements

* Large restructure of docs

* Tidy docs

* Fix readme link

* Add interop docs

* Tidy README
2019-08-06 13:29:27 +10:00
Michael Sproul
0374e31907 Disallow extra bytes in Bitfield from_bytes (#488) 2019-08-06 11:05:35 +10:00
Paul Hauner
4f45bf2255
Tree hash benches (#486)
* Add initial tree hash benches

* Add tree hash example

* Use lazy static in tree hash benches
2019-08-05 18:06:50 +10:00
Michael Sproul
65c18ddc60 Fix Bitfield from_bytes empty vec bug (#487)
Credit to @kirk-baird for finding the bug with the fuzzer.

Co-authored-by: Kirk Baird <baird.k@outlook.com>
2019-08-05 18:06:24 +10:00
Paul Hauner
e21d3fed05
Revert "Tidy ancestor iterators"
This reverts commit 5079c25bb2.

Accidental push to master.. my bad!
2019-08-05 16:27:55 +10:00
Paul Hauner
5079c25bb2
Tidy ancestor iterators 2019-08-05 16:25:21 +10:00
Paul Hauner
7738d51a72
Add cargo fmt to .travis.yml (#474)
* Run cargo fmt

* Add cargo fmt to travis.yml
2019-07-31 14:45:09 +10:00
blacktemplar
309b10c4a8 add logging functionality for aligning key value pairs after message (#461)
* add logging functionality for aligning key value pairs after the main messages

* move to own crate, change default message width to 40

* use FullFormat in validator_client (CompactFormat is not compatible with aligning)

* move logging to eth2/utils/logging
2019-07-31 09:06:53 +10:00
Paul Hauner
81a089aa8b
Remove old benches (#465)
* Remove cached_tree_hash benches

* Remove state_processing benches
2019-07-30 17:19:03 +10:00
Michael Sproul
eb669ab40f
Add v0.8 genesis tests (#466)
Closes #452
2019-07-30 17:02:38 +10:00
Michael Sproul
a236003a7b Update to frozen spec ❄️ (v0.8.1) (#444)
* types: first updates for v0.8

* state_processing: epoch processing v0.8.0

* state_processing: block processing v0.8.0

* tree_hash_derive: support generics in SignedRoot

* types v0.8: update to use ssz_types

* state_processing v0.8: use ssz_types

* ssz_types: add bitwise methods and from_elem

* types: fix v0.8 FIXMEs

* ssz_types: add bitfield shift_up

* ssz_types: iterators and DerefMut for VariableList

* types,state_processing: use VariableList

* ssz_types: fix BitVector Decode impl

Fixed a typo in the implementation of ssz::Decode for BitVector, which caused it
to be considered variable length!

* types: fix test modules for v0.8 update

* types: remove slow type-level arithmetic

* state_processing: fix tests for v0.8

* op_pool: update for v0.8

* ssz_types: Bitfield difference length-independent

Allow computing the difference of two bitfields of different lengths.

* Implement compact committee support

* epoch_processing: committee & active index roots

* state_processing: genesis state builder v0.8

* state_processing: implement v0.8.1

* Further improve tree_hash

* Strip examples, tests from cached_tree_hash

* Update TreeHash, un-impl CachedTreeHash

* Update bitfield TreeHash, un-impl CachedTreeHash

* Update FixedLenVec TreeHash, unimpl CachedTreeHash

* Update update tree_hash_derive for new TreeHash

* Fix TreeHash, un-impl CachedTreeHash for ssz_types

* Remove fixed_len_vec, ssz benches

SSZ benches relied upon fixed_len_vec -- it is easier to just delete
them and rebuild them later (when necessary)

* Remove boolean_bitfield crate

* Fix fake_crypto BLS compile errors

* Update ef_tests for new v.8 type params

* Update ef_tests submodule to v0.8.1 tag

* Make fixes to support parsing ssz ef_tests

* `compact_committee...` to `compact_committees...`

* Derive more traits for `CompactCommittee`

* Flip bitfield byte-endianness

* Fix tree_hash for bitfields

* Modify CLI output for ef_tests

* Bump ssz crate version

* Update ssz_types doc comment

* Del cached tree hash tests from ssz_static tests

* Tidy SSZ dependencies

* Rename ssz_types crate to eth2_ssz_types

* validator_client: update for v0.8

* ssz_types: update union/difference for bit order swap

* beacon_node: update for v0.8, EthSpec

* types: disable cached tree hash, update min spec

* state_processing: fix slot bug in committee update

* tests: temporarily disable fork choice harness test

See #447

* committee cache: prevent out-of-bounds access

In the case where we tried to access the committee of a shard that didn't have a committee in the
current epoch, we were accessing elements beyond the end of the shuffling vector and panicking! This
commit adds a check to make the failure safe and explicit.

* fix bug in get_indexed_attestation and simplify

There was a bug in our implementation of get_indexed_attestation whereby
incorrect "committee indices" were used to index into the custody bitfield. The
bug was only observable in the case where some bits of the custody bitfield were
set to 1. The implementation has been simplified to remove the bug, and a test
added.

* state_proc: workaround for compact committees bug

https://github.com/ethereum/eth2.0-specs/issues/1315

* v0.8: updates to make the EF tests pass

* Remove redundant max operation checks.
* Always supply both messages when checking attestation signatures -- allowing
  verification of an attestation with no signatures.
* Swap the order of the fork and domain constant in `get_domain`, to match
  the spec.

* rustfmt

* ef_tests: add new epoch processing tests

* Integrate v0.8 into master (compiles)

* Remove unused crates, fix clippy lints

* Replace v0.6.3 tags w/ v0.8.1

* Remove old comment

* Ensure lmd ghost tests only run in release

* Update readme
2019-07-30 12:44:51 +10:00
John Adler
1b26a36ebc Verify transfer now checks for amount + fees. (#457) 2019-07-29 12:17:04 +10:00
Paul Hauner
7458022fcf
Fork choice bug fixes (#449)
* Change reduced tree for adding weightless node

* Add more comments for reduced tree fork choice

* Small refactor on reduced tree for readability

* Move test_harness forking logic into itself

* Add new `AncestorIter` trait to store

* Add unfinished tests to fork choice

* Make `beacon_state.genesis_block_root` public

* Add failing lmd_ghost fork choice tests

* Extend fork_choice tests, create failing test

* Implement Debug for generic ReducedTree

* Add lazy_static to fork choice tests

* Add verify_integrity fn to reduced tree

* Fix bugs in reduced tree

* Ensure all reduced tree tests verify integrity

* Slightly alter reduce tree test params

* Add (failing) reduced tree test

* Fix bug in fork choice

Iter ancestors was not working well with skip slots

* Put maximum depth for common ancestor search

Ensures that we don't search back past the finalized root.

* Add basic finalization tests for reduced tree

* Change fork choice to use beacon_block_root

Previously it was using target_root, which was wrong

* Make ancestor iter return option

* Disable fork choice test when !debug_assertions

* Fix type, removed code fragment

* Tidy some borrow-checker evading

* Lower reduced tree random test iterations
2019-07-29 12:08:52 +10:00
Pawan Dhananjay
db094022b9 Remove unused dependencies (#456) 2019-07-29 09:55:57 +10:00
John Adler
fec7168512
Fix lots of typos. 2019-07-26 15:26:06 -04:00
Kirk Baird
0513559252 Fix syncing bugs by recursively attempting to process parents in the … (#429)
* Fix syncing bugs by recursively attempting to process parents in the import queue, change BlockRootsIterator

* Swap from crossbeam channel to tokio mpsc

* Recursion fix

* Remove exess block processing

* Fix network lag, correct attestation topic

* Correct network poll logic

* Overhaul of SimpleSync and modify BlockRootsIterator to return start_slot

* Fix bug in tests relating to StateRootsIterator

* Remove old, commented-out heartbeat code.

* Tidy docs on import queue enum

* Change source logging msg in simple sync

* Rename function parameter in simple sync

* Use `BestBlockRootsIterator` in `reduced_tree`

* Update comments for `BestBlockRootsIterator`

* Fix duplicate dep in cargo.toml
2019-07-16 17:28:15 +10:00
Paul Hauner
88c6d15c32
Padding efficent merkle root algo (#436)
* Add initial work on padding efficent merkle roots

* Improve merklize_padded

* Improve tree_hash crate -- fix bugs, docs

* Update codebase for tree_hash API change

* Remove dbg statements, fix import error

* Fix clippy lints, doc error

* Tidy tree hash comments

* Increase tree_hash max tree height

* Fix PR review comments

* Fix typos

* Fix cache access off-by-one in tree hash

* Set max tree depth to 48 (from 64)
2019-07-16 14:40:56 +10:00
Paul Hauner
fcd3da81bf
Merge pull request #426 from sigp/ssz-types
Add ssz_types crate
2019-07-11 16:53:32 +10:00
Paul Hauner
561cec0bf6
Move many bitfield Options to Results 2019-07-11 13:19:38 +10:00
Paul Hauner
61406b34bc
Resolve issues raised from @michaelsproul review 2019-07-11 12:45:34 +10:00
Paul Hauner
2c1afcc2d6
Rename marker structs for Bitfield 2019-07-11 12:40:37 +10:00
Paul Hauner
08868a6e7a
Merge pull request #425 from sigp/ssz-v0.8
Changes for publishing ssz to crates.io
2019-07-11 09:22:07 +10:00
Paul Hauner
5e1a2ebf25
Uncomment broken tree hash impl for variable list 2019-07-09 20:30:29 +10:00
Paul Hauner
bb3b3fd8b9
Fix bug around single-byte for 0-bits 2019-07-09 20:29:10 +10:00
Paul Hauner
daa8916e6b
Add comments after self-review 2019-07-09 20:28:19 +10:00
Paul Hauner
090133b088
Add more comments to bitfield 2019-07-09 17:31:34 +10:00
Paul Hauner
734aa3b8bd
Satisfy clippy lint in SSZ 2019-07-09 17:01:37 +10:00
Paul Hauner
7283fdff15
Fix clippy lints in ssz_types 2019-07-09 16:58:53 +10:00
Paul Hauner
57cc946018
Add bitvector tests 2019-07-09 16:37:09 +10:00
Paul Hauner
000d941e2e
Add tests for BitList 2019-07-09 16:03:02 +10:00
Paul Hauner
e4ef0fc9d4
Add crate-level docs to ssz_types 2019-07-09 11:00:40 +10:00
Paul Hauner
94265272b4
Tidy bitfield docs 2019-07-09 09:28:22 +10:00
Paul Hauner
5a7c44ed37
Add failing doc tests 2019-07-08 18:53:25 +10:00
Paul Hauner
1484773cd1
Refactor to use Bitfield struct with type variants 2019-07-08 18:41:43 +10:00
Paul Hauner
93cd38da55
Progress further on Bitfield struct 2019-07-08 16:27:08 +10:00
Paul Hauner
bbcc58dca3
Start building new bitfield struct 2019-07-08 16:07:40 +10:00
Paul Hauner
636ebb0d4e
Add progress on bitfields 2019-07-08 11:54:47 +10:00
Paul Hauner
ecb0bf11c7
Move bitlist and bitvector into own files 2019-07-08 09:36:52 +10:00
Paul Hauner
c8c5c8ff16
Collect errors into a single error 2019-07-06 15:57:11 +10:00
Paul Hauner
2b7d5560ad
Refactor SSZ types bitfield
Removes superfulous `Bitfield` struct
2019-07-06 15:51:15 +10:00
Paul Hauner
5943e176cf
Add ssz_types crate 2019-07-05 17:33:20 +10:00
Age Manning
af28d5e20c
Add persistent network identification 2019-07-04 14:14:27 +10:00
Age Manning
6ee2b4df34
Complete merging of network addition branch 2019-07-04 14:14:26 +10:00
Age Manning
c7e17c8641
Updates for latest master 2019-07-04 14:14:26 +10:00
Age Manning
64abd0bc5b
Removes network parameters from chain spec 2019-07-04 14:07:26 +10:00
Age Manning
4e24c8e651
Add topics to chain id 2019-07-04 14:07:26 +10:00
Paul Hauner
4dc274858e
Update SSZ for publishing to crates.io 2019-07-04 13:34:54 +10:00
Paul Hauner
54bda210e2
Update ssz_derive for publishing to crates.io 2019-07-04 13:22:33 +10:00
Paul Hauner
027f0a539d
Prepare ssz for publishing on crates.io 2019-07-03 16:06:20 +10:00
Michael Sproul
a04b1f981e
op_pool: remove SszPair 2019-07-01 16:25:57 +10:00
Michael Sproul
44ed3228b9
ssz: implement Encode and Decode for tuples 2019-07-01 14:54:34 +10:00
Michael Sproul
73c4171b52
op_pool: finish persistence support 2019-07-01 10:55:55 +10:00
Michael Sproul
7fe458af45
op_pool: re-jig deposit handling (needs more work) 2019-07-01 10:55:55 +10:00
Michael Sproul
604fe2d97f
op_pool: partial persistence support 2019-07-01 10:55:55 +10:00
Michael Sproul
38d2d03e3a
op_pool: use max cover algorithm, refactor 2019-07-01 10:55:54 +10:00
Paul Hauner
9a356a00c2
Merge pull request #405 from c-o-l-o-r/wasm-ssz
Make SSZ compatible with `wasm32` targets
2019-06-27 13:31:18 +10:00
Matt Garnett
e93fb94e7a calculate MAX_LENGTH_VALUE for 32-bit and 64-bit targets 2019-06-25 10:12:49 -04:00
Matt Garnett
db9dd3dffe fix encoding impl for usize on 32-bit architectures 2019-06-25 09:59:50 -04:00
Paul Hauner
8afe8b3569
Implement fixes from PR review 2019-06-24 15:31:36 +10:00
Paul Hauner
3a196f3fdc
Tidy, fix clippy lints 2019-06-24 09:34:56 +10:00
Paul Hauner
075c989375
Remove old fork_choice crate 2019-06-24 09:34:56 +10:00
Paul Hauner
77fba0b98e
Fix bugs in fork choice, add more tests 2019-06-24 09:34:56 +10:00
Paul Hauner
7a4c3e26ac
Fix bug in reduced tree fork choice 2019-06-24 09:34:54 +10:00
Paul Hauner
0b2ad4d0a1
Mark reduced_tree fork choice as incomplete 2019-06-24 09:34:54 +10:00
Paul Hauner
d0037f49d8
Add progress on debugging fork choice 2019-06-24 09:34:54 +10:00
Paul Hauner
5a8cde0598
Change "canonical_root" of block to be signed root 2019-06-24 09:34:54 +10:00
Paul Hauner
e485f3ee75
Fix bug in reduced tree fork choice 2019-06-24 09:34:54 +10:00
Paul Hauner
85b23f9f1b
Add incomplete progress on fork choice 2019-06-24 09:34:54 +10:00
Paul Hauner
5a98502ad6
Fix bug in epoch trans. finalization 2019-06-24 09:34:53 +10:00
Paul Hauner
55196dff64
Remove iter mod from beacon chain
Now the iter mod in store is the only implementation
2019-06-24 09:34:53 +10:00
Paul Hauner
952e08ba38
Add state roots iter to store 2019-06-24 09:34:53 +10:00
Paul Hauner
2b5c70711d
Cover edge case in reduced tree 2019-06-24 09:34:53 +10:00
Paul Hauner
fd384e54f4
Add weightless blocks to reduced tree, fix bugs 2019-06-24 09:34:53 +10:00
Paul Hauner
9c2bbb6c05
Add stubbed-out block processing to fork choice 2019-06-24 09:34:52 +10:00
Paul Hauner
8fb6ffffe2
Add new beacon chain test harness 2019-06-24 09:34:52 +10:00
Paul Hauner
f4621a9f1a
Improve reduced tree fork choice 2019-06-24 09:34:52 +10:00
Paul Hauner
7756a658a7
Update fork choice find head fn 2019-06-24 09:34:51 +10:00
Paul Hauner
2ee71aa808
Add new fork choice struct to beacon chain 2019-06-24 09:34:51 +10:00