Commit Graph

33 Commits

Author SHA1 Message Date
Michael Sproul
32074f0d09
Use checked arithmetic in types and state proc (#1009) 2020-04-20 12:35:11 +10:00
Paul Hauner
7f6ae4c2f5
Refactor tree hashing (#861)
* Pre-allocated tree hash caches

* Add SmallVec to tree hash cache

* Avoid allocation for validator.pubkey

* Avoid iterator which seems to be doing heap alloc

* Add more smallvecs

* MOAR SMALLVEC

* Move non-test code to Hash256 tree hash

* Fix byte ordering error

* Add incomplete but working merkle stream impl

* Fix zero hash error

* Add zero hash fn

* Add MerkleStream comments

* Add smallvec, tidy

* Integrate into tree hash derive

* Update ssz_types tree hash

* Don't heap alloc for mix in length

* Add byte-level streaming to MerkleStream

* Avoid recursion in write method

* Update BLS to MerkleStream

* Fix some not-compiling tests

* Remove debug profiling

* Remove code duplication

* Move beacon state tree hash to new hasher

* Fix failing tests

* Update comments

* Add some fast-paths to tree_hash::merkle_root

* Remove unncessary test

* Rename MerkleStream -> MerkleHasher

* Rename new_with_leaf_count -> with_leaves

* Tidy

* Remove NonZeroUsize

* Remove todo

* Update smallvec
2020-03-05 08:07:27 +11:00
Paul Hauner
c3182e3c1c
Tree hash cache arena (#836)
* Start adding interop genesis state to lcli

* Use more efficient method to generate genesis state

* Remove duplicate int_to_bytes32

* Add lcli command to change state genesis time

* Add option to allow VC to start with unsynced BN

* Set VC to do parallel key loading

* Don't default to dummy eth1 backend

* Add endpoint to dump operation pool

* Add metrics for op pool

* Remove state clone for slot notifier

* Add mem size approximation for tree hash cache

* Avoid cloning tree hash when getting head

* Avoid cloning tree hash when getting head

* Add working arena-based cached tree hash

* Add another benchmark

* Add pre-allocation for caches

* Make cache nullable

* Fix bugs in cache tree hash

* Add validator tree hash optimization

* Optimize hash_concat

* Make hash32_concat return fixed-len array

* Fix failing API tests

* Add new beacon state cache struct

* Add validator-specific cache

* Separate list and values arenas

* Add parallel validator registry hashing

* Remove MultiTreeHashCache

* Remove cached tree hash macro

* Fix failing tree hash test

* Address Michael's comments

* Add CachedTreeHash impl for ef tests

* Fix messy merge conflict

* Rename cache struct, add comments

* Rename cache struct, add comments

* Remove unnecessary mutability

* Wrap iter in result

* Tidy cached tree hash

* Address Michael comments

* Address more comments

* Use ring::Context
2020-02-07 12:42:49 +11:00
Paul Hauner
f267bf2afe
Optimizations, disable val client sync check & additional lcli tools (#834)
* Start adding interop genesis state to lcli

* Use more efficient method to generate genesis state

* Remove duplicate int_to_bytes32

* Add lcli command to change state genesis time

* Add option to allow VC to start with unsynced BN

* Set VC to do parallel key loading

* Don't default to dummy eth1 backend

* Add endpoint to dump operation pool

* Add metrics for op pool

* Remove state clone for slot notifier

* Add mem size approximation for tree hash cache

* Avoid cloning tree hash when getting head

* Fix failing API tests

* Address Michael's comments

* Add HashMap::from_par_iter
2020-02-04 12:43:04 +11:00
pscott
7396cd2cab Fix clippy warnings (#813)
* Clippy account manager

* Clippy account_manager

* Clippy beacon_node/beacon_chain

* Clippy beacon_node/client

* Clippy beacon_node/eth1

* Clippy beacon_node/eth2-libp2p

* Clippy beacon_node/genesis

* Clippy beacon_node/network

* Clippy beacon_node/rest_api

* Clippy beacon_node/src

* Clippy beacon_node/store

* Clippy eth2/lmd_ghost

* Clippy eth2/operation_pool

* Clippy eth2/state_processing

* Clippy eth2/types

* Clippy eth2/utils/bls

* Clippy eth2/utils/cahced_tree_hash

* Clippy eth2/utils/deposit_contract

* Clippy eth2/utils/eth2_interop_keypairs

* Clippy eth2/utils/eth2_testnet_config

* Clippy eth2/utils/lighthouse_metrics

* Clippy eth2/utils/ssz

* Clippy eth2/utils/ssz_types

* Clippy eth2/utils/tree_hash_derive

* Clippy lcli

* Clippy tests/beacon_chain_sim

* Clippy validator_client

* Cargo fmt
2020-01-21 18:38:56 +11:00
pscott
8e1e6838d2 Add individual processing for ef tests (#745)
* Add Individual signature verification for ef tests

* Add clone state to avoid errors in ef tests

* Add state verification for VerifyIndividual in ef tests

* Add Clone derive for errors

* Update comments
2020-01-08 10:24:44 +11:00
Michael Sproul
c1a2238f1a
Implement tree hash caching (#584)
* Implement basic tree hash caching

* Use spaces to indent top-level Cargo.toml

* Optimize BLS tree hash by hashing bytes directly

* Implement tree hash caching for validator registry

* Persist BeaconState tree hash cache to disk

* Address Paul's review comments
2019-11-05 15:46:52 +11:00
Paul Hauner
8cfa36fedd
Publish ssz_types (and deps) to crates.io (#468)
* Rename `hashing` crate to `eth2_hashing`

* Add license, desc to eth2_hashing Cargo.toml

* Remove merkle root from eth2 hashing

* Remove old benches folder (zombied from old branch)

* Add docs to eth2_hashing

* Prepare tree_hash for publishing on crates.io

* Update deps to use crates.io instead of paths

* Update all crates to pull ssz from crates.io

* Remove cached_tree_hash, add patches to manifest

* Fix compile error in benches

* Remove unused code

* Fix fake_crypto compile error
2019-08-08 11:39:47 +10:00
John Adler
fec7168512
Fix lots of typos. 2019-07-26 15:26:06 -04:00
Michael Sproul
3a0f00215d
spec: ensure types up to date with v0.6.1 2019-05-15 11:48:09 +10:00
Luke Anderson
93a4bef41c
Very small wording/typo fixes for docs. 2019-05-10 13:49:01 +10:00
Paul Hauner
3be80ba9a9
Marginally improve cached_tree_hash docs 2019-05-10 13:29:43 +10:00
Paul Hauner
84d72cfed6
Tidy and add docs for cached tree hash 2019-04-29 17:46:01 +10:00
Paul Hauner
f622aa0b65
Add doc comments to vec impl 2019-04-29 15:45:02 +10:00
Paul Hauner
f20314bd87
Fix clippy lints, impl treehash for slices 2019-04-29 15:32:41 +10:00
Paul Hauner
240d1e197a
Ignore clippy range_plus_one lints 2019-04-29 14:24:57 +10:00
Paul Hauner
a90bbbfd82
Address various clippy lints, improve comments 2019-04-29 14:04:52 +10:00
Paul Hauner
fbf8fad4f1
Add counter-resets to reset_modifications 2019-04-29 10:57:19 +10:00
Paul Hauner
0599d3f1f8
Remove redundant type param fom CachedTreeHash 2019-04-29 07:48:48 +10:00
Paul Hauner
68b36787e2
Move leaf node creation into separate fn 2019-04-28 20:30:48 +10:00
Paul Hauner
c58723350c
Improve cached hashing performance 2019-04-28 11:33:29 +10:00
Paul Hauner
89d64b007f
Run cargofmt --all 2019-04-27 19:04:26 +10:00
Paul Hauner
6c9be1a73c
Add tree hash cache as field to BeaconState. 2019-04-27 19:02:52 +10:00
Paul Hauner
80fa5d08c5
Fix bug with cached tree hash, passes tests 2019-04-27 16:24:29 +10:00
Paul Hauner
a425beb42a
Fix chunk index bug with structs 2019-04-26 17:39:38 +10:00
Paul Hauner
d3309b9f7e
Fix bug with bitfield tree hash caching 2019-04-26 17:27:39 +10:00
Paul Hauner
dddcc91ef3
Fix bug with shrinking list. 2019-04-26 16:56:01 +10:00
Paul Hauner
f65e981f6f
Begin implementing cached hashing in types 2019-04-26 15:24:18 +10:00
Paul Hauner
ecff8f0007
Update CachedTreeHash trait to return schema 2019-04-26 12:27:04 +10:00
Paul Hauner
8976e652d2
Introduce BTreeSchema 2019-04-26 11:55:26 +10:00
Paul Hauner
794b48078c
Remove num_items from BTreeOverlay 2019-04-26 11:39:33 +10:00
Paul Hauner
15f81c0907
Implement CachedTreeHash for TreeHashVector 2019-04-26 11:15:17 +10:00
Paul Hauner
b213a5ade4
Create cached_tree_hash crate. 2019-04-26 09:55:03 +10:00