6656cb00e4
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Paul Hauner
|
6656cb00e4
|
Optimize attestation processing (#841)
* Start updating types * WIP * Signature hacking * Existing EF tests passing with fake_crypto * Updates * Delete outdated API spec * The refactor continues * It compiles * WIP test fixes * All release tests passing bar genesis state parsing * Update and test YamlConfig * Update to spec v0.10 compatible BLS * Updates to BLS EF tests * Add EF test for AggregateVerify And delete unused hash2curve tests for uncompressed points * Update EF tests to v0.10.1 * Use optional block root correctly in block proc * Use genesis fork in deposit domain. All tests pass * Cargo fmt * Fast aggregate verify test * Update REST API docs * Cargo fmt * Fix unused import * Bump spec tags to v0.10.1 * Add `seconds_per_eth1_block` to chainspec * Update to timestamp based eth1 voting scheme * Return None from `get_votes_to_consider` if block cache is empty * Handle overflows in `is_candidate_block` * Revert to failing tests * Fix eth1 data sets test * Choose default vote according to spec * Fix collect_valid_votes tests * Fix `get_votes_to_consider` to choose all eligible blocks * Uncomment winning_vote tests * Add comments; remove unused code * Reduce seconds_per_eth1_block for simulation * Addressed review comments * Add test for default vote case * Fix logs * Remove unused functions * Meter default eth1 votes * Fix comments * Address review comments; remove unused dependency * Add first attempt at attestation proc. re-write * Add version 2 of attestation processing * Minor fixes * Add validator pubkey cache * Make get_indexed_attestation take a committee * Link signature processing into new attn verification * First working version * Ensure pubkey cache is updated * Add more metrics, slight optimizations * Clone committee cache during attestation processing * Update shuffling cache during block processing * Remove old commented-out code * Fix shuffling cache insert bug * Used indexed attestation in fork choice * Restructure attn processing, add metrics * Add more detailed metrics * Tidy, fix failing tests * Fix failing tests, tidy * Disable/delete two outdated tests * Tidy * Add pubkey cache persistence file * Add more comments * Integrate persistence file into builder * Add pubkey cache tests * Add data_dir to beacon chain builder * Remove Option in pubkey cache persistence file * Ensure consistency between datadir/data_dir * Fix failing network test * Tidy * Fix todos * Add attestation processing tests * Add another test * Only run attestation tests in release * Make attestation tests MainnetEthSpec * Address Michael's comments * Remove redundant check * Fix warning * Fix failing test Co-authored-by: Michael Sproul <micsproul@gmail.com> Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com> |
||
Michael Sproul
|
371e5adcf8
|
Update to Spec v0.10 (#817)
* Start updating types * WIP * Signature hacking * Existing EF tests passing with fake_crypto * Updates * Delete outdated API spec * The refactor continues * It compiles * WIP test fixes * All release tests passing bar genesis state parsing * Update and test YamlConfig * Update to spec v0.10 compatible BLS * Updates to BLS EF tests * Add EF test for AggregateVerify And delete unused hash2curve tests for uncompressed points * Update EF tests to v0.10.1 * Use optional block root correctly in block proc * Use genesis fork in deposit domain. All tests pass * Cargo fmt * Fast aggregate verify test * Update REST API docs * Cargo fmt * Fix unused import * Bump spec tags to v0.10.1 * Add `seconds_per_eth1_block` to chainspec * Update to timestamp based eth1 voting scheme * Return None from `get_votes_to_consider` if block cache is empty * Handle overflows in `is_candidate_block` * Revert to failing tests * Fix eth1 data sets test * Choose default vote according to spec * Fix collect_valid_votes tests * Fix `get_votes_to_consider` to choose all eligible blocks * Uncomment winning_vote tests * Add comments; remove unused code * Reduce seconds_per_eth1_block for simulation * Addressed review comments * Add test for default vote case * Fix logs * Remove unused functions * Meter default eth1 votes * Fix comments * Address review comments; remove unused dependency * Disable/delete two outdated tests * Bump eth1 default vote warn to error * Delete outdated eth1 test Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com> |
||
|
Michael Sproul
|
24e941d175
|
Update to spec v0.9.1 (#597)
* Update to spec v0.9.0 * Update to v0.9.1 * Bump spec tags for v0.9.1 * Formatting, fix CI failures * Resolve accidental KeyPair merge conflict * Document new BeaconState functions * Fix incorrect cache drops in `advance_caches` * Update fork choice for v0.9.1 * Clean up some FIXMEs * Fix a few docs/logs |
||
pscott
|
c7b3a7abd8 |
BlockProcessing testing (#559)
* Add valid_deposit test and build_with_deposit method * Insert_deposit takes a num_deposit param * Deposit with spec.max_effective_balance * Copy int_to_bytes32 implem from beacon_chain_builder * Add debug information to insert_deposit * Remove length-proof assertion * Insert_deposit displays error * Batch deposits now pass tests * Optimize insert_deposit * Rename insert_deposits and set num_deposits to 2 in valid_deposit test * update test_utils to pass tests * fix typo in test_utils * update cast in test_utils * Add DepositCountInvalid tests * Add tests for bad deposit signature * Add tests and test utils in test_builder * Return error instead of ok on bad signature * Update DepositTestTask enum * Add comment about manually setting deposit_count and deposit_index * add badblsbytes test * add bad_index var for clarity ; remove underflow test * cargo fmt * Add insert 3 exits tests * Add validator_unknwon test * Add far_future_epoch test and already exited * Add MaxVoluntaryExits + 1 test * Add exit_already_initiated test * Add exit_not_active test * Add too_young_to_leave test * Cargo fmt * Confirm already_anitiated test * Fix typo in enum variant * Adjust some tests to return ok(()) and revert changes for early return in per_block_processing.rs * cargo fmt * Adjust AlreadyIniated test to expect Ok(()) and revert changes in per_block_processing.rs * Remove extraneous newline * Add insert_valid_attester_slashing * Initial cargo fmt * Add NotSlashable test * Cargo fmt * Remove AttestationDataIdentical * Make test_task pass through reference ; fix max_attester_slashing_plus_one test * Initial cargo fmt * Add InvalidIndexedAttestation1 and 2 * Add comments * Add ProposalsIdenticalTest * Add ProposalsIdentical test * Cargo fmt * Add ProposerUnknown test * Add ProposalEpochMismatch test * Add BadProposal1Signature and Badproposal2Signature tests * Add ProposerNotSlashable test * Derive PartialEq and use if instead of match * Merge attestation tests * Remove useless AlreadyInitiated variant in beacon_state * Remove MaxOperations plus one tests for each operation * Clean comments * add IncludedTooLate and BadTargetEpoch tests * Update AttestationDataBuilder call in operation_pool testing * Cargo fmt * Remove BadIndex enum variant, unused in the code * Cargo fmt * Cargo fmt updated * simply increment deposit_count instead of hardsetting deposit_index in insert_deposits * Fix bad_merkle_proof when calling insert_deposits |
||
|
pscott
|
7eb82125ef |
Clippy clean (#536)
* Change into_iter to iter * Fix clippy 'easy' warnings * Clippy eth2/utils * Add struct NetworkInfo * Clippy for types, utils, and beacon_node/store/src/iters.rs * Cargo fmt * Change foo to my_foo * Remove complex signature * suppress clippy warning for unit_value in benches * Use enumerate instead of iterating over range * Allow trivially_copy_pass_by_ref in serde_utils |
||
|
Paul Hauner
|
bcffe42712
|
Bulk signature verification (#507)
* Add basic block processing benches * Start reviving state processing benches * Fix old block builders * Add optimization for faster pubkey add * Tidy benches, add another * Add extra block processing bench * Start working on faster BLS scheme * Add partially complete sig verify optimization * Add .gitignore to state processing * Add progress on faster signature verification * Fix SignatureSet for fake_crypto * Tidy attester slashings sig set * Tidy bulk signature verifier * Refactor signature sets to be cleaner * Start threading SignatureStrategy through code * Add (empty) test dir * Move BenchingBlockBuilder * Add initial block signature verification tests * Add tests for bulk signature verification * Start threading SignatureStrategy in block proc. * Refactor per_block_processing errors * Use sig set tuples instead of lists of two * Remove dead code * Thread VerifySignatures through per_block_processing * Add bulk signature verification * Introduce parallel bulk signature verification * Expand state processing benches * Fix additional compile errors * Fix issue where par iter chunks is 0 * Update milagro_bls dep * Remove debugs, code fragment in beacon chain * Tidy, add comments to block sig verifier * Fix various PR comments * Add block_root option to per_block_processing * Fix comment in block signature verifier * Fix comments from PR review * Remove old comment * Fix comment |
||
|
Paul Hauner
|
cd26a19a70
|
Attestation processing (#497)
* Renamed fork_choice::process_attestation_from_block * Processing attestation in fork choice * Retrieving state from store and checking signature * Looser check on beacon state validity. * Cleaned up get_attestation_state * Expanded fork choice api to provide latest validator message. * Checking if the an attestation contains a latest message * Correct process_attestation error handling. * Copy paste error in comment fixed. * Tidy ancestor iterators * Getting attestation slot via helper method * Refactored attestation creation in test utils * Revert "Refactored attestation creation in test utils" This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306. * Integration tests for free attestation processing * Implicit conflicts resolved. * formatting * Do first pass on Grants code * Add another attestation processing test * Tidy attestation processing * Remove old code fragment * Add non-compiling half finished changes * Simplify, fix bugs, add tests for chain iters * Remove attestation processing from op pool * Fix bug with fork choice, tidy * Fix overly restrictive check in fork choice. * Ensure committee cache is build during attn proc * Ignore unknown blocks at fork choice * Various minor fixes * Make fork choice write lock in to read lock * Remove unused method * Tidy comments * Fix attestation prod. target roots change * Fix compile error in store iters * Reject any attestation prior to finalization * Fix minor PR comments * Remove duplicated attestation finalization check * Remove awkward `let` statement |
||
|
Michael Sproul
|
a236003a7b |
Update to frozen spec ❄️ (v0.8.1) (#444)
* types: first updates for v0.8 * state_processing: epoch processing v0.8.0 * state_processing: block processing v0.8.0 * tree_hash_derive: support generics in SignedRoot * types v0.8: update to use ssz_types * state_processing v0.8: use ssz_types * ssz_types: add bitwise methods and from_elem * types: fix v0.8 FIXMEs * ssz_types: add bitfield shift_up * ssz_types: iterators and DerefMut for VariableList * types,state_processing: use VariableList * ssz_types: fix BitVector Decode impl Fixed a typo in the implementation of ssz::Decode for BitVector, which caused it to be considered variable length! * types: fix test modules for v0.8 update * types: remove slow type-level arithmetic * state_processing: fix tests for v0.8 * op_pool: update for v0.8 * ssz_types: Bitfield difference length-independent Allow computing the difference of two bitfields of different lengths. * Implement compact committee support * epoch_processing: committee & active index roots * state_processing: genesis state builder v0.8 * state_processing: implement v0.8.1 * Further improve tree_hash * Strip examples, tests from cached_tree_hash * Update TreeHash, un-impl CachedTreeHash * Update bitfield TreeHash, un-impl CachedTreeHash * Update FixedLenVec TreeHash, unimpl CachedTreeHash * Update update tree_hash_derive for new TreeHash * Fix TreeHash, un-impl CachedTreeHash for ssz_types * Remove fixed_len_vec, ssz benches SSZ benches relied upon fixed_len_vec -- it is easier to just delete them and rebuild them later (when necessary) * Remove boolean_bitfield crate * Fix fake_crypto BLS compile errors * Update ef_tests for new v.8 type params * Update ef_tests submodule to v0.8.1 tag * Make fixes to support parsing ssz ef_tests * `compact_committee...` to `compact_committees...` * Derive more traits for `CompactCommittee` * Flip bitfield byte-endianness * Fix tree_hash for bitfields * Modify CLI output for ef_tests * Bump ssz crate version * Update ssz_types doc comment * Del cached tree hash tests from ssz_static tests * Tidy SSZ dependencies * Rename ssz_types crate to eth2_ssz_types * validator_client: update for v0.8 * ssz_types: update union/difference for bit order swap * beacon_node: update for v0.8, EthSpec * types: disable cached tree hash, update min spec * state_processing: fix slot bug in committee update * tests: temporarily disable fork choice harness test See #447 * committee cache: prevent out-of-bounds access In the case where we tried to access the committee of a shard that didn't have a committee in the current epoch, we were accessing elements beyond the end of the shuffling vector and panicking! This commit adds a check to make the failure safe and explicit. * fix bug in get_indexed_attestation and simplify There was a bug in our implementation of get_indexed_attestation whereby incorrect "committee indices" were used to index into the custody bitfield. The bug was only observable in the case where some bits of the custody bitfield were set to 1. The implementation has been simplified to remove the bug, and a test added. * state_proc: workaround for compact committees bug https://github.com/ethereum/eth2.0-specs/issues/1315 * v0.8: updates to make the EF tests pass * Remove redundant max operation checks. * Always supply both messages when checking attestation signatures -- allowing verification of an attestation with no signatures. * Swap the order of the fork and domain constant in `get_domain`, to match the spec. * rustfmt * ef_tests: add new epoch processing tests * Integrate v0.8 into master (compiles) * Remove unused crates, fix clippy lints * Replace v0.6.3 tags w/ v0.8.1 * Remove old comment * Ensure lmd ghost tests only run in release * Update readme |