Commit Graph

54 Commits

Author SHA1 Message Date
Paul Hauner
7f6ae4c2f5
Refactor tree hashing (#861)
* Pre-allocated tree hash caches

* Add SmallVec to tree hash cache

* Avoid allocation for validator.pubkey

* Avoid iterator which seems to be doing heap alloc

* Add more smallvecs

* MOAR SMALLVEC

* Move non-test code to Hash256 tree hash

* Fix byte ordering error

* Add incomplete but working merkle stream impl

* Fix zero hash error

* Add zero hash fn

* Add MerkleStream comments

* Add smallvec, tidy

* Integrate into tree hash derive

* Update ssz_types tree hash

* Don't heap alloc for mix in length

* Add byte-level streaming to MerkleStream

* Avoid recursion in write method

* Update BLS to MerkleStream

* Fix some not-compiling tests

* Remove debug profiling

* Remove code duplication

* Move beacon state tree hash to new hasher

* Fix failing tests

* Update comments

* Add some fast-paths to tree_hash::merkle_root

* Remove unncessary test

* Rename MerkleStream -> MerkleHasher

* Rename new_with_leaf_count -> with_leaves

* Tidy

* Remove NonZeroUsize

* Remove todo

* Update smallvec
2020-03-05 08:07:27 +11:00
Michael Sproul
371e5adcf8
Update to Spec v0.10 (#817)
* Start updating types

* WIP

* Signature hacking

* Existing EF tests passing with fake_crypto

* Updates

* Delete outdated API spec

* The refactor continues

* It compiles

* WIP test fixes

* All release tests passing bar genesis state parsing

* Update and test YamlConfig

* Update to spec v0.10 compatible BLS

* Updates to BLS EF tests

* Add EF test for AggregateVerify

And delete unused hash2curve tests for uncompressed points

* Update EF tests to v0.10.1

* Use optional block root correctly in block proc

* Use genesis fork in deposit domain. All tests pass

* Cargo fmt

* Fast aggregate verify test

* Update REST API docs

* Cargo fmt

* Fix unused import

* Bump spec tags to v0.10.1

* Add `seconds_per_eth1_block` to chainspec

* Update to timestamp based eth1 voting scheme

* Return None from `get_votes_to_consider` if block cache is empty

* Handle overflows in `is_candidate_block`

* Revert to failing tests

* Fix eth1 data sets test

* Choose default vote according to spec

* Fix collect_valid_votes tests

* Fix `get_votes_to_consider` to choose all eligible blocks

* Uncomment winning_vote tests

* Add comments; remove unused code

* Reduce seconds_per_eth1_block for simulation

* Addressed review comments

* Add test for default vote case

* Fix logs

* Remove unused functions

* Meter default eth1 votes

* Fix comments

* Address review comments; remove unused dependency

* Disable/delete two outdated tests

* Bump eth1 default vote warn to error

* Delete outdated eth1 test

Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
2020-02-11 10:19:36 +11:00
Paul Hauner
251aea645c
Make API friendly to block explorers (#702)
* Add validator index to duties response

* Add `get_state` method to beacon chain

* Improve /beacon/validators endpoint

* Add validators/all and validators/active endpoints

* Start refactor of HTTP docs

* Document /beacon/heads endpoint

* Remove some unused API endpoints

* Improve API docs

* Add methods to get all validator duties

* Improve docs

* Remove dead links

* Make tables left-justified

* Add /consensus/vote_count endpoint

* Add /consensus/individual_votes endpoint

* Update formatting

* Tidy

* Add committees endpoint

* Strictly require 0x prefix for serde in BLS

* Update docs to have 0x prefix

* Fix failing tests

* Add unfinished code

* Improve testing, fix bugs

* Tidy, ensure all beacon endpoints smoke tested

* Fix pubkey cache error

* Address comments with docs
2019-12-19 11:45:28 +11:00
Michael Sproul
24e941d175
Update to spec v0.9.1 (#597)
* Update to spec v0.9.0

* Update to v0.9.1

* Bump spec tags for v0.9.1

* Formatting, fix CI failures

* Resolve accidental KeyPair merge conflict

* Document new BeaconState functions

* Fix incorrect cache drops in `advance_caches`

* Update fork choice for v0.9.1

* Clean up some FIXMEs

* Fix a few docs/logs
2019-11-21 11:47:30 +11:00
Paul Hauner
e1d6e187d1
Fix bug in crosslink rewards during per-epoch 2019-09-09 01:54:32 -04:00
Michael Sproul
6cf9b3c1a4
Epoch processing tests 2019-08-30 13:29:26 +10:00
Michael Sproul
4d2cdc9492
Update to spec v0.8.3 2019-08-27 16:59:53 +10:00
Michael Sproul
5a7903a377
Improve BeaconState safe accessors
And fix a bug in the compact committees accessor.
2019-08-26 11:39:48 +10:00
Michael Sproul
a236003a7b Update to frozen spec ❄️ (v0.8.1) (#444)
* types: first updates for v0.8

* state_processing: epoch processing v0.8.0

* state_processing: block processing v0.8.0

* tree_hash_derive: support generics in SignedRoot

* types v0.8: update to use ssz_types

* state_processing v0.8: use ssz_types

* ssz_types: add bitwise methods and from_elem

* types: fix v0.8 FIXMEs

* ssz_types: add bitfield shift_up

* ssz_types: iterators and DerefMut for VariableList

* types,state_processing: use VariableList

* ssz_types: fix BitVector Decode impl

Fixed a typo in the implementation of ssz::Decode for BitVector, which caused it
to be considered variable length!

* types: fix test modules for v0.8 update

* types: remove slow type-level arithmetic

* state_processing: fix tests for v0.8

* op_pool: update for v0.8

* ssz_types: Bitfield difference length-independent

Allow computing the difference of two bitfields of different lengths.

* Implement compact committee support

* epoch_processing: committee & active index roots

* state_processing: genesis state builder v0.8

* state_processing: implement v0.8.1

* Further improve tree_hash

* Strip examples, tests from cached_tree_hash

* Update TreeHash, un-impl CachedTreeHash

* Update bitfield TreeHash, un-impl CachedTreeHash

* Update FixedLenVec TreeHash, unimpl CachedTreeHash

* Update update tree_hash_derive for new TreeHash

* Fix TreeHash, un-impl CachedTreeHash for ssz_types

* Remove fixed_len_vec, ssz benches

SSZ benches relied upon fixed_len_vec -- it is easier to just delete
them and rebuild them later (when necessary)

* Remove boolean_bitfield crate

* Fix fake_crypto BLS compile errors

* Update ef_tests for new v.8 type params

* Update ef_tests submodule to v0.8.1 tag

* Make fixes to support parsing ssz ef_tests

* `compact_committee...` to `compact_committees...`

* Derive more traits for `CompactCommittee`

* Flip bitfield byte-endianness

* Fix tree_hash for bitfields

* Modify CLI output for ef_tests

* Bump ssz crate version

* Update ssz_types doc comment

* Del cached tree hash tests from ssz_static tests

* Tidy SSZ dependencies

* Rename ssz_types crate to eth2_ssz_types

* validator_client: update for v0.8

* ssz_types: update union/difference for bit order swap

* beacon_node: update for v0.8, EthSpec

* types: disable cached tree hash, update min spec

* state_processing: fix slot bug in committee update

* tests: temporarily disable fork choice harness test

See #447

* committee cache: prevent out-of-bounds access

In the case where we tried to access the committee of a shard that didn't have a committee in the
current epoch, we were accessing elements beyond the end of the shuffling vector and panicking! This
commit adds a check to make the failure safe and explicit.

* fix bug in get_indexed_attestation and simplify

There was a bug in our implementation of get_indexed_attestation whereby
incorrect "committee indices" were used to index into the custody bitfield. The
bug was only observable in the case where some bits of the custody bitfield were
set to 1. The implementation has been simplified to remove the bug, and a test
added.

* state_proc: workaround for compact committees bug

https://github.com/ethereum/eth2.0-specs/issues/1315

* v0.8: updates to make the EF tests pass

* Remove redundant max operation checks.
* Always supply both messages when checking attestation signatures -- allowing
  verification of an attestation with no signatures.
* Swap the order of the fork and domain constant in `get_domain`, to match
  the spec.

* rustfmt

* ef_tests: add new epoch processing tests

* Integrate v0.8 into master (compiles)

* Remove unused crates, fix clippy lints

* Replace v0.6.3 tags w/ v0.8.1

* Remove old comment

* Ensure lmd ghost tests only run in release

* Update readme
2019-07-30 12:44:51 +10:00
John Adler
fec7168512
Fix lots of typos. 2019-07-26 15:26:06 -04:00
Paul Hauner
5a98502ad6
Fix bug in epoch trans. finalization 2019-06-24 09:34:53 +10:00
Michael Sproul
03c50354f4
v0.6: update all spec version comments 2019-06-17 17:22:48 +10:00
Paul Hauner
a71f05066b
Merge branch 'docker-env' into v0.6.1 2019-06-13 10:37:35 -04:00
Michael Sproul
44bccda4b3
epoch proc: set correct active index root 2019-06-12 15:39:48 +10:00
Paul Hauner
e550c0218f
Add various fixes to clippy lints
Thou shalt appease clippy
2019-06-10 11:01:25 -04:00
Paul Hauner
a662c3a940
Fix various compile errors and warnings 2019-06-09 06:26:34 -04:00
Paul Hauner
caddeba81b
Remove genesis_epoch from ChainSpec 2019-06-08 08:17:42 -04:00
Paul Hauner
e74d49fc8a
Remove dupe info between ChainSpec and EthSpec 2019-06-08 07:57:25 -04:00
Paul Hauner
9790968378
Remove old inclusion_distance file 2019-05-22 12:19:13 +10:00
Michael Sproul
56424d94c6
spec v0.6.1: update per-epoch processing 2019-05-21 16:43:54 +10:00
Michael Sproul
d3d2900a6a
state_processing: get_attesting_indices in common 2019-05-21 16:43:54 +10:00
Paul Hauner
6660311b2b
Rename EpochCache to CommitteeCache 2019-05-20 14:36:54 +10:00
Paul Hauner
9264ec1aa9
Remove state.get_crosslink_committee fn
Replaced by state.get_crosslink_committee_for_shard
2019-05-19 17:08:09 +10:00
Paul Hauner
ea96d24420
Make #stateprocessingcompileagain 2019-05-19 16:56:39 +10:00
Paul Hauner
03849de319
Fix some BeaconState API changes in state proc. 2019-05-19 15:56:24 +10:00
Michael Sproul
137afa9131
state_processing: nicer attestation swap 2019-05-15 17:14:07 +10:00
Michael Sproul
79de966d3a
spec: top-level per-epoch processing v0.6.1 2019-05-14 15:00:18 +10:00
Michael Sproul
e80264b8c8
Merge remote-tracking branch 'origin/master' into spec-0.6 2019-05-14 12:09:59 +10:00
Michael Sproul
13ec3d125e
WIP trash changes 2019-05-13 17:32:06 +10:00
Paul Hauner
ce8ebeccbc
Rename BeaconStateTypes to EthSpec 2019-05-10 14:47:09 +10:00
Paul Hauner
8cefd20e9d
Add progress on state_processing fixed-len update 2019-05-08 15:36:02 +10:00
Michael Sproul
839ef0119b
spec: justification and finalization v0.6.1 2019-05-07 18:27:36 +10:00
Michael Sproul
d95ae95ce8
spec: update tags to v0.5.1 2019-04-17 12:00:23 +10:00
Paul Hauner
b8c4c3308a
Update types to new tree_hash crate 2019-04-16 14:14:38 +10:00
Paul Hauner
baca2c90ab
Add last of 0.5.0 upgrades. Woo! 2019-03-19 19:43:31 +11:00
Paul Hauner
35b90728c7
Push more epoch processing fns to 0.5.0 2019-03-19 19:27:10 +11:00
Paul Hauner
d20fb93f0c
Update rewards processing to v0.5.0 2019-03-19 17:16:51 +11:00
Paul Hauner
37b8e9f39a
Move get_active_validator_indices to state 2019-03-19 09:09:57 +11:00
Paul Hauner
1028acf3f1
Move state trans fns into state_processing 2019-03-18 21:34:42 +11:00
Paul Hauner
816c2c651b
Modify genesis processing process.
- Removed BeaconStateBuilder
- Added genesis code to `state_processing`.
2019-03-17 23:11:07 +11:00
Paul Hauner
3a384d93f8
Allow state processing to compile under v0.5.0 2019-03-17 17:47:12 +11:00
Paul Hauner
1c1c15a122
Tidy per epoch processing
- Add comments to ValidatorStatuses
- Add some checks to guard against a bad statuses list
- Remove unused attester_sets.rs file.
2019-03-14 16:00:22 +11:00
Paul Hauner
10aee6214c
Tidy per_epoch_processing
Mainly renaming variables and files for readability.
2019-03-14 14:59:30 +11:00
Paul Hauner
95599ddc66
Update Attesters struct
- Renames variables
- Moves total balance calculation into struct
2019-03-14 12:49:48 +11:00
Paul Hauner
a319144835
Improve crosslink reward processing 2019-03-14 12:17:43 +11:00
Paul Hauner
6f919e6f7d
Add first iteration on faster rewards processing. 2019-03-14 11:53:50 +11:00
Paul Hauner
f949919b9b
Add comments to epoch_processing 2019-03-12 18:02:53 +11:00
Paul Hauner
53456a6c79
Remove last inclusion_slot(..) call 2019-03-10 21:06:40 +11:00
Paul Hauner
f27b62d410
Add optimisation for epoch processing 2019-03-10 17:49:06 +11:00
Paul Hauner
63743a962c
Add per-epoch benchmarks, optimise function. 2019-03-09 10:38:53 +11:00