Commit Graph

149 Commits

Author SHA1 Message Date
Paul Hauner
b374ead24b
Protect against OOB offset in variable list SSZ decoding (#974)
* Add "pretty-ssz" tool to lcli

* Protect against OOB SSZ offset

* Add more work on decoding

* Fix benches

* Add more decode fixes

* Rename fixed_ptr

* Add, fix tests

* Add extra test

* Increase SSZ decode error granularity

* Ripples new error types across ssz crate

* Add comment to `sanitize_offset`

* Introduce max_len to SSZ list decoding

* Restrict FixedVector, check for zero-len items

* Double check for empty list

* Address Michael's comment
2020-04-20 15:35:47 +10:00
Michael Sproul
26bdc2927b
Update to spec v0.11 (#959)
* Update process_final_updates() hysteresis computation

* Update core to v0.11.1

* Bump tags to v0.11.1

* Update docs and deposit contract

* Add compute_fork_digest

* Address review comments

Co-authored-by: Herman Alonso Junge <alonso.junge@gmail.com>
2020-04-01 22:03:03 +11:00
Paul Hauner
7f6ae4c2f5
Refactor tree hashing (#861)
* Pre-allocated tree hash caches

* Add SmallVec to tree hash cache

* Avoid allocation for validator.pubkey

* Avoid iterator which seems to be doing heap alloc

* Add more smallvecs

* MOAR SMALLVEC

* Move non-test code to Hash256 tree hash

* Fix byte ordering error

* Add incomplete but working merkle stream impl

* Fix zero hash error

* Add zero hash fn

* Add MerkleStream comments

* Add smallvec, tidy

* Integrate into tree hash derive

* Update ssz_types tree hash

* Don't heap alloc for mix in length

* Add byte-level streaming to MerkleStream

* Avoid recursion in write method

* Update BLS to MerkleStream

* Fix some not-compiling tests

* Remove debug profiling

* Remove code duplication

* Move beacon state tree hash to new hasher

* Fix failing tests

* Update comments

* Add some fast-paths to tree_hash::merkle_root

* Remove unncessary test

* Rename MerkleStream -> MerkleHasher

* Rename new_with_leaf_count -> with_leaves

* Tidy

* Remove NonZeroUsize

* Remove todo

* Update smallvec
2020-03-05 08:07:27 +11:00
Paul Hauner
871163aecc
Add optimized SSZ decoding for fixed-len items (#865)
* Add custom SSZ decode for Validator

* Move efficient decode into macro

* Don't allocate SSZ offset to heap

* Use smallvec in SszDecoder

* Fix test compile error
2020-03-04 11:45:01 +11:00
Michael Sproul
371e5adcf8
Update to Spec v0.10 (#817)
* Start updating types

* WIP

* Signature hacking

* Existing EF tests passing with fake_crypto

* Updates

* Delete outdated API spec

* The refactor continues

* It compiles

* WIP test fixes

* All release tests passing bar genesis state parsing

* Update and test YamlConfig

* Update to spec v0.10 compatible BLS

* Updates to BLS EF tests

* Add EF test for AggregateVerify

And delete unused hash2curve tests for uncompressed points

* Update EF tests to v0.10.1

* Use optional block root correctly in block proc

* Use genesis fork in deposit domain. All tests pass

* Cargo fmt

* Fast aggregate verify test

* Update REST API docs

* Cargo fmt

* Fix unused import

* Bump spec tags to v0.10.1

* Add `seconds_per_eth1_block` to chainspec

* Update to timestamp based eth1 voting scheme

* Return None from `get_votes_to_consider` if block cache is empty

* Handle overflows in `is_candidate_block`

* Revert to failing tests

* Fix eth1 data sets test

* Choose default vote according to spec

* Fix collect_valid_votes tests

* Fix `get_votes_to_consider` to choose all eligible blocks

* Uncomment winning_vote tests

* Add comments; remove unused code

* Reduce seconds_per_eth1_block for simulation

* Addressed review comments

* Add test for default vote case

* Fix logs

* Remove unused functions

* Meter default eth1 votes

* Fix comments

* Address review comments; remove unused dependency

* Disable/delete two outdated tests

* Bump eth1 default vote warn to error

* Delete outdated eth1 test

Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
2020-02-11 10:19:36 +11:00
pscott
7396cd2cab Fix clippy warnings (#813)
* Clippy account manager

* Clippy account_manager

* Clippy beacon_node/beacon_chain

* Clippy beacon_node/client

* Clippy beacon_node/eth1

* Clippy beacon_node/eth2-libp2p

* Clippy beacon_node/genesis

* Clippy beacon_node/network

* Clippy beacon_node/rest_api

* Clippy beacon_node/src

* Clippy beacon_node/store

* Clippy eth2/lmd_ghost

* Clippy eth2/operation_pool

* Clippy eth2/state_processing

* Clippy eth2/types

* Clippy eth2/utils/bls

* Clippy eth2/utils/cahced_tree_hash

* Clippy eth2/utils/deposit_contract

* Clippy eth2/utils/eth2_interop_keypairs

* Clippy eth2/utils/eth2_testnet_config

* Clippy eth2/utils/lighthouse_metrics

* Clippy eth2/utils/ssz

* Clippy eth2/utils/ssz_types

* Clippy eth2/utils/tree_hash_derive

* Clippy lcli

* Clippy tests/beacon_chain_sim

* Clippy validator_client

* Cargo fmt
2020-01-21 18:38:56 +11:00
Paul Hauner
370c658c7c
Optimization: reduce BLS decompression (#766)
* Add RwLock-style caching for BLS pubkeys

* Tidy docker ignore

* Remove RwLocks

* Merge in master
2020-01-10 15:32:10 +11:00
Mehdi Zerouali
83c571d6eb Delete fuzzing targets (#586) 2019-10-31 13:13:21 +11:00
Age Manning
5c97ed3562
Updates external dependencies (#577)
* Updates external dependencies

* Correct fmt formatting
2019-10-30 12:22:18 +11:00
pscott
7eb82125ef Clippy clean (#536)
* Change into_iter to iter

* Fix clippy 'easy' warnings

* Clippy eth2/utils

* Add struct NetworkInfo

* Clippy for types, utils, and beacon_node/store/src/iters.rs

* Cargo fmt

* Change foo to my_foo

* Remove complex signature

* suppress clippy warning for unit_value in benches

* Use enumerate instead of iterating over range

* Allow trivially_copy_pass_by_ref in serde_utils
2019-09-30 13:58:45 +10:00
Paul Hauner
97a45cdcb8
Address Michael's comments 2019-09-26 13:00:31 +10:00
Paul Hauner
6a870d468c
Add ssz_fixed_len method to ssz::Encode 2019-09-08 12:23:37 -04:00
Kyle Liu
74af13a372 Fix this link per @blacktemplar to link to the actual crate (#518) 2019-08-28 09:32:54 +10:00
Kyle Liu
82666e932e Removing old SSZ readme and replacing it with cargo badge. (#514)
* Removing old ssz readme per paulhauner, replacing with cargo badge.

Co-Authored-by: Paul Hauner <paul@paulhauner.com>

* Replacing my shitty text badge with an actual badge per Paul.

Co-Authored-by: Paul Hauner <paul@paulhauner.com>
2019-08-27 12:02:36 +10:00
Pawan Dhananjay
88e89f9ab2 Update ethereum types (#489)
* Update ethereum-types to version 0.6

* Fix tests

* Run rustfmt
2019-08-06 14:41:42 +10:00
Michael Sproul
a236003a7b Update to frozen spec ❄️ (v0.8.1) (#444)
* types: first updates for v0.8

* state_processing: epoch processing v0.8.0

* state_processing: block processing v0.8.0

* tree_hash_derive: support generics in SignedRoot

* types v0.8: update to use ssz_types

* state_processing v0.8: use ssz_types

* ssz_types: add bitwise methods and from_elem

* types: fix v0.8 FIXMEs

* ssz_types: add bitfield shift_up

* ssz_types: iterators and DerefMut for VariableList

* types,state_processing: use VariableList

* ssz_types: fix BitVector Decode impl

Fixed a typo in the implementation of ssz::Decode for BitVector, which caused it
to be considered variable length!

* types: fix test modules for v0.8 update

* types: remove slow type-level arithmetic

* state_processing: fix tests for v0.8

* op_pool: update for v0.8

* ssz_types: Bitfield difference length-independent

Allow computing the difference of two bitfields of different lengths.

* Implement compact committee support

* epoch_processing: committee & active index roots

* state_processing: genesis state builder v0.8

* state_processing: implement v0.8.1

* Further improve tree_hash

* Strip examples, tests from cached_tree_hash

* Update TreeHash, un-impl CachedTreeHash

* Update bitfield TreeHash, un-impl CachedTreeHash

* Update FixedLenVec TreeHash, unimpl CachedTreeHash

* Update update tree_hash_derive for new TreeHash

* Fix TreeHash, un-impl CachedTreeHash for ssz_types

* Remove fixed_len_vec, ssz benches

SSZ benches relied upon fixed_len_vec -- it is easier to just delete
them and rebuild them later (when necessary)

* Remove boolean_bitfield crate

* Fix fake_crypto BLS compile errors

* Update ef_tests for new v.8 type params

* Update ef_tests submodule to v0.8.1 tag

* Make fixes to support parsing ssz ef_tests

* `compact_committee...` to `compact_committees...`

* Derive more traits for `CompactCommittee`

* Flip bitfield byte-endianness

* Fix tree_hash for bitfields

* Modify CLI output for ef_tests

* Bump ssz crate version

* Update ssz_types doc comment

* Del cached tree hash tests from ssz_static tests

* Tidy SSZ dependencies

* Rename ssz_types crate to eth2_ssz_types

* validator_client: update for v0.8

* ssz_types: update union/difference for bit order swap

* beacon_node: update for v0.8, EthSpec

* types: disable cached tree hash, update min spec

* state_processing: fix slot bug in committee update

* tests: temporarily disable fork choice harness test

See #447

* committee cache: prevent out-of-bounds access

In the case where we tried to access the committee of a shard that didn't have a committee in the
current epoch, we were accessing elements beyond the end of the shuffling vector and panicking! This
commit adds a check to make the failure safe and explicit.

* fix bug in get_indexed_attestation and simplify

There was a bug in our implementation of get_indexed_attestation whereby
incorrect "committee indices" were used to index into the custody bitfield. The
bug was only observable in the case where some bits of the custody bitfield were
set to 1. The implementation has been simplified to remove the bug, and a test
added.

* state_proc: workaround for compact committees bug

https://github.com/ethereum/eth2.0-specs/issues/1315

* v0.8: updates to make the EF tests pass

* Remove redundant max operation checks.
* Always supply both messages when checking attestation signatures -- allowing
  verification of an attestation with no signatures.
* Swap the order of the fork and domain constant in `get_domain`, to match
  the spec.

* rustfmt

* ef_tests: add new epoch processing tests

* Integrate v0.8 into master (compiles)

* Remove unused crates, fix clippy lints

* Replace v0.6.3 tags w/ v0.8.1

* Remove old comment

* Ensure lmd ghost tests only run in release

* Update readme
2019-07-30 12:44:51 +10:00
Pawan Dhananjay
db094022b9 Remove unused dependencies (#456) 2019-07-29 09:55:57 +10:00
John Adler
fec7168512
Fix lots of typos. 2019-07-26 15:26:06 -04:00
Paul Hauner
fcd3da81bf
Merge pull request #426 from sigp/ssz-types
Add ssz_types crate
2019-07-11 16:53:32 +10:00
Paul Hauner
734aa3b8bd
Satisfy clippy lint in SSZ 2019-07-09 17:01:37 +10:00
Paul Hauner
4dc274858e
Update SSZ for publishing to crates.io 2019-07-04 13:34:54 +10:00
Paul Hauner
027f0a539d
Prepare ssz for publishing on crates.io 2019-07-03 16:06:20 +10:00
Michael Sproul
44ed3228b9
ssz: implement Encode and Decode for tuples 2019-07-01 14:54:34 +10:00
Paul Hauner
9a356a00c2
Merge pull request #405 from c-o-l-o-r/wasm-ssz
Make SSZ compatible with `wasm32` targets
2019-06-27 13:31:18 +10:00
Matt Garnett
e93fb94e7a calculate MAX_LENGTH_VALUE for 32-bit and 64-bit targets 2019-06-25 10:12:49 -04:00
Matt Garnett
db9dd3dffe fix encoding impl for usize on 32-bit architectures 2019-06-25 09:59:50 -04:00
Paul Hauner
e6747094c8
Merge pull request #386 from vlopes11/ssz-ambiguous
`ssz` is ambiguous (name vs any other name during import resolution)
2019-06-24 07:56:50 +10:00
Matt Garnett
87e681c617 make ssz crate wasm compatible 2019-06-22 14:37:07 -04:00
Paul Hauner
39ec96ad82
Run rustfmt 2019-06-07 02:55:43 -04:00
Paul Hauner
45fb11b208
Impl ssz enc/dec for NonZeroUsize 2019-06-04 12:24:13 +10:00
Paul Hauner
7a2ab2e9aa
Add support for SSZ union type via Option 2019-06-04 12:03:54 +10:00
Paul Hauner
39cb63e01e
Merge branch 'v0.6.1' into docker-env 2019-06-04 09:25:00 +10:00
Victor Lopez
f5bf4a861e
Fmt fix
Signed-off-by: Victor Lopez <vhrlopes@gmail.com>
2019-05-28 01:58:16 +00:00
Paul Hauner
21ecaddac1
Fix various clippy lints 2019-05-28 10:56:05 +10:00
Victor Lopez
ce106caa88
ssz is ambiguous (name vs any other name during import resolution) 2019-05-22 14:23:19 -03:00
Paul Hauner
519ee81c68
Merge branch '368' into ef-tests 2019-05-15 12:33:40 +10:00
Michael Sproul
3a0f00215d
spec: ensure types up to date with v0.6.1 2019-05-15 11:48:09 +10:00
Paul Hauner
cf509bea9b
Improve ef_tests crate 2019-05-13 22:10:23 +10:00
Paul Hauner
a0148b5aae
Rename SSZ traits (Encodable -> Encode) 2019-05-13 15:12:19 +10:00
Paul Hauner
94ab2479fe
Add docs to SSZ 2019-05-13 14:13:15 +10:00
Paul Hauner
16a8cdb714
Add tests for excess bytes in SSZ 2019-05-13 13:51:58 +10:00
Paul Hauner
9ccaec5b91
Move SSZ macros into own file 2019-05-13 12:33:59 +10:00
Paul Hauner
eb62b5842d
Update beacon node crates for new SSZ 2019-05-13 12:07:32 +10:00
Paul Hauner
d9bfbc2657
Add macros for easy SSZ impl via From trait 2019-05-13 09:25:37 +10:00
Paul Hauner
f96a6268e5
Remove old dbg! statements from tests 2019-05-12 09:34:01 +10:00
Paul Hauner
02afc6ef24
Tidy ssz decoding code 2019-05-11 22:52:24 +10:00
Paul Hauner
fc2a406edf
Fix bug in ssz decoding 2019-05-11 22:25:28 +10:00
Paul Hauner
1f7b7ec498
Add tests for SSZ offset attacks 2019-05-11 19:09:27 +10:00
Paul Hauner
3cd112d42d
Encapsulate ssz vec decoding in public fn 2019-05-10 16:37:00 +10:00
Paul Hauner
1f098ecbde
Remove antiquated test vectors 2019-05-06 11:52:46 +10:00