* Address compiler warning
* Prune abandoned fork choice forks
* New approach to pruning
* Wrap some block hashes in a newtype pattern
For increased type safety.
* Add Graphviz chain dump emitter for debugging
* Fix broken test case
* Make prunes_abandoned_forks use real DiskStore
* Mark finalized blocks in the GraphViz output
* Refine debug stringification of Slot and Epoch
Before this commit: print!("{:?}", Slot(123)) == "Slot(\n123\n)".
After this commit: print!("{:?", Slot(123)) == "Slot(123)".
* Simplify build_block()
* Rewrite test case using more composable test primitives
* Working rewritten test case
* Tighten fork prunning test checks
* Add another pruning test case
* Bugfix: Finalized blocks weren't always properly detected
* Pruning: Add pruning_does_not_touch_blocks_prior_to_finalization test case
* Tighten pruning tests: check if heads are tracked properly
* Add a failing test case for a buggy scenario
* Change name of function to a more accurate one
* Fix failing test case
* Test case: Were skipped slots' states pruned?
* Style fix: Simplify dereferencing
* Tighten pruning tests: check if abandoned states are deleted
* Towards atomicity of db ops
* Correct typo
* Prune also skipped slots' states
* New logic for handling skipped states
* Make skipped slots test pass
* Post conflict resolution fixes
* Formatting fixes
* Tests passing
* Block hashes in Graphviz node labels
* Removed unused changes
* Fix bug with states having < SlotsPerHistoricalRoot roots
* Consolidate State/BlockRootsIterator for pruning
* Address review feedback
* Fix a bug in pruning tests
* Detach prune_abandoned_forks() from its object
* Move migrate.rs from store to beacon_chain
* Move forks pruning onto a background thread
* Bugfix: Heads weren't pruned when prune set contained only the head
* Rename: freeze_to_state() -> process_finalization()
* Eliminate redundant function parameter
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
* Add "pretty-ssz" tool to lcli
* Protect against OOB SSZ offset
* Add more work on decoding
* Fix benches
* Add more decode fixes
* Rename fixed_ptr
* Add, fix tests
* Add extra test
* Increase SSZ decode error granularity
* Ripples new error types across ssz crate
* Add comment to `sanitize_offset`
* Introduce max_len to SSZ list decoding
* Restrict FixedVector, check for zero-len items
* Double check for empty list
* Address Michael's comment
* Start updating types
* WIP
* Signature hacking
* Existing EF tests passing with fake_crypto
* Updates
* Delete outdated API spec
* The refactor continues
* It compiles
* WIP test fixes
* All release tests passing bar genesis state parsing
* Update and test YamlConfig
* Update to spec v0.10 compatible BLS
* Updates to BLS EF tests
* Add EF test for AggregateVerify
And delete unused hash2curve tests for uncompressed points
* Update EF tests to v0.10.1
* Use optional block root correctly in block proc
* Use genesis fork in deposit domain. All tests pass
* Cargo fmt
* Fast aggregate verify test
* Update REST API docs
* Cargo fmt
* Fix unused import
* Bump spec tags to v0.10.1
* Add `seconds_per_eth1_block` to chainspec
* Update to timestamp based eth1 voting scheme
* Return None from `get_votes_to_consider` if block cache is empty
* Handle overflows in `is_candidate_block`
* Revert to failing tests
* Fix eth1 data sets test
* Choose default vote according to spec
* Fix collect_valid_votes tests
* Fix `get_votes_to_consider` to choose all eligible blocks
* Uncomment winning_vote tests
* Add comments; remove unused code
* Reduce seconds_per_eth1_block for simulation
* Addressed review comments
* Add test for default vote case
* Fix logs
* Remove unused functions
* Meter default eth1 votes
* Fix comments
* Address review comments; remove unused dependency
* Add first attempt at attestation proc. re-write
* Add version 2 of attestation processing
* Minor fixes
* Add validator pubkey cache
* Make get_indexed_attestation take a committee
* Link signature processing into new attn verification
* First working version
* Ensure pubkey cache is updated
* Add more metrics, slight optimizations
* Clone committee cache during attestation processing
* Update shuffling cache during block processing
* Remove old commented-out code
* Fix shuffling cache insert bug
* Used indexed attestation in fork choice
* Restructure attn processing, add metrics
* Add more detailed metrics
* Tidy, fix failing tests
* Fix failing tests, tidy
* Disable/delete two outdated tests
* Add new Pubkeys struct to signature_sets
* Refactor with functional approach
* Update beacon chain
* Remove decompressed member from pubkey bytes
* Add hashmap for indices lookup
* Change `get_attesting_indices` to use Vec
* Fix failing test
* Tidy
* Add pubkey cache persistence file
* Add more comments
* Integrate persistence file into builder
* Add pubkey cache tests
* Add data_dir to beacon chain builder
* Remove Option in pubkey cache persistence file
* Ensure consistency between datadir/data_dir
* Fix failing network test
* Tidy
* Fix todos
* Improve tests
* Split up block processing metrics
* Tidy
* Refactor get_pubkey_from_state
* Remove commented-out code
* Add BeaconChain::validator_pubkey
* Use Option::filter
* Remove Box
* Comment out tests that fail due to hard-coded
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: Michael Sproul <micsproul@gmail.com>
Co-authored-by: pawan <pawandhananjay@gmail.com>
* Add PH & MS slot clock changes
* Account for genesis time
* Use checked mul
* Account for genesis slot
* Change API
* Refactor "duration to..." functions
* Unfinished progress
* Update more persistence code
* Start fixing tests
* Combine persist head and fork choice
* Persist head on reorg
* Gracefully handle op pool and eth1 cache missing
* Fix test failure
* Address Michael's comments
* Start updating types
* WIP
* Signature hacking
* Existing EF tests passing with fake_crypto
* Updates
* Delete outdated API spec
* The refactor continues
* It compiles
* WIP test fixes
* All release tests passing bar genesis state parsing
* Update and test YamlConfig
* Update to spec v0.10 compatible BLS
* Updates to BLS EF tests
* Add EF test for AggregateVerify
And delete unused hash2curve tests for uncompressed points
* Update EF tests to v0.10.1
* Use optional block root correctly in block proc
* Use genesis fork in deposit domain. All tests pass
* Cargo fmt
* Fast aggregate verify test
* Update REST API docs
* Cargo fmt
* Fix unused import
* Bump spec tags to v0.10.1
* Add `seconds_per_eth1_block` to chainspec
* Update to timestamp based eth1 voting scheme
* Return None from `get_votes_to_consider` if block cache is empty
* Handle overflows in `is_candidate_block`
* Revert to failing tests
* Fix eth1 data sets test
* Choose default vote according to spec
* Fix collect_valid_votes tests
* Fix `get_votes_to_consider` to choose all eligible blocks
* Uncomment winning_vote tests
* Add comments; remove unused code
* Reduce seconds_per_eth1_block for simulation
* Addressed review comments
* Add test for default vote case
* Fix logs
* Remove unused functions
* Meter default eth1 votes
* Fix comments
* Address review comments; remove unused dependency
* Add first attempt at attestation proc. re-write
* Add version 2 of attestation processing
* Minor fixes
* Add validator pubkey cache
* Make get_indexed_attestation take a committee
* Link signature processing into new attn verification
* First working version
* Ensure pubkey cache is updated
* Add more metrics, slight optimizations
* Clone committee cache during attestation processing
* Update shuffling cache during block processing
* Remove old commented-out code
* Fix shuffling cache insert bug
* Used indexed attestation in fork choice
* Restructure attn processing, add metrics
* Add more detailed metrics
* Tidy, fix failing tests
* Fix failing tests, tidy
* Disable/delete two outdated tests
* Tidy
* Add pubkey cache persistence file
* Add more comments
* Integrate persistence file into builder
* Add pubkey cache tests
* Add data_dir to beacon chain builder
* Remove Option in pubkey cache persistence file
* Ensure consistency between datadir/data_dir
* Fix failing network test
* Tidy
* Fix todos
* Add attestation processing tests
* Add another test
* Only run attestation tests in release
* Make attestation tests MainnetEthSpec
* Address Michael's comments
* Remove redundant check
* Fix warning
* Fix failing test
Co-authored-by: Michael Sproul <micsproul@gmail.com>
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
* Start adding interop genesis state to lcli
* Use more efficient method to generate genesis state
* Remove duplicate int_to_bytes32
* Add lcli command to change state genesis time
* Add option to allow VC to start with unsynced BN
* Set VC to do parallel key loading
* Don't default to dummy eth1 backend
* Add endpoint to dump operation pool
* Add metrics for op pool
* Remove state clone for slot notifier
* Add mem size approximation for tree hash cache
* Avoid cloning tree hash when getting head
* Avoid cloning tree hash when getting head
* Add working arena-based cached tree hash
* Add another benchmark
* Add pre-allocation for caches
* Make cache nullable
* Fix bugs in cache tree hash
* Add validator tree hash optimization
* Optimize hash_concat
* Make hash32_concat return fixed-len array
* Fix failing API tests
* Add new beacon state cache struct
* Add validator-specific cache
* Separate list and values arenas
* Add parallel validator registry hashing
* Remove MultiTreeHashCache
* Remove cached tree hash macro
* Fix failing tree hash test
* Address Michael's comments
* Add CachedTreeHash impl for ef tests
* Fix messy merge conflict
* Optimize attestation production
* Add first basic optimizations
* Fix SlotOutOfBounds error
* Resolved missed merge conflicts
* Fix another missed merge conflict
* Fix more merge conflict issues
* Add `StateSkipConfig`
* Fix test compile errors
* Add failing test
* Fix bug, make tests pass
* Add comment
* Delete unused function
* Replace deleted comment
* Add custom SSZ decode for Validator
* Move efficient decode into macro
* Don't allocate SSZ offset to heap
* Use smallvec in SszDecoder
* Fix test compile error
* Remove deprecated api_spec.yaml
* add prototype for proposer slashing
* remove clippy warnings
* Add proposer_slashing API
* Prototype for attester slashing API call
* Fix logic error in operation pool
* Finish test for attester_slashing api call
* Clean proposer_slashing test
* Cargo fmt
* Remove useless to_string after format! macro
* Cargo fmt
* Update book with new api calls
* Re-enable proposer slashing verification
* Update book with appropriate test example
* Fix proposer_slashing test
* Update comments and tests for clearer code
* Remove extraneous comments
* Fix test
* Minor fix
* Address reviewer comments
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
* Start updating types
* WIP
* Signature hacking
* Existing EF tests passing with fake_crypto
* Updates
* Delete outdated API spec
* The refactor continues
* It compiles
* WIP test fixes
* All release tests passing bar genesis state parsing
* Update and test YamlConfig
* Update to spec v0.10 compatible BLS
* Updates to BLS EF tests
* Add EF test for AggregateVerify
And delete unused hash2curve tests for uncompressed points
* Update EF tests to v0.10.1
* Use optional block root correctly in block proc
* Use genesis fork in deposit domain. All tests pass
* Cargo fmt
* Fast aggregate verify test
* Update REST API docs
* Cargo fmt
* Fix unused import
* Bump spec tags to v0.10.1
* Add `seconds_per_eth1_block` to chainspec
* Update to timestamp based eth1 voting scheme
* Return None from `get_votes_to_consider` if block cache is empty
* Handle overflows in `is_candidate_block`
* Revert to failing tests
* Fix eth1 data sets test
* Choose default vote according to spec
* Fix collect_valid_votes tests
* Fix `get_votes_to_consider` to choose all eligible blocks
* Uncomment winning_vote tests
* Add comments; remove unused code
* Reduce seconds_per_eth1_block for simulation
* Addressed review comments
* Add test for default vote case
* Fix logs
* Remove unused functions
* Meter default eth1 votes
* Fix comments
* Address review comments; remove unused dependency
* Disable/delete two outdated tests
* Bump eth1 default vote warn to error
* Delete outdated eth1 test
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
* Add LRU caches to store
* Improvements to LRU caches
* Take state by value in `Store::put_state`
* Store blocks by value, configurable cache sizes
* Use a StateBatch to efficiently store skip states
* Fix store tests
* Add CloneConfig test, remove unused metrics
* Use Mutexes instead of RwLocks for LRU caches
* Start adding interop genesis state to lcli
* Use more efficient method to generate genesis state
* Remove duplicate int_to_bytes32
* Add lcli command to change state genesis time
* Add option to allow VC to start with unsynced BN
* Set VC to do parallel key loading
* Don't default to dummy eth1 backend
* Add endpoint to dump operation pool
* Add metrics for op pool
* Remove state clone for slot notifier
* Add mem size approximation for tree hash cache
* Avoid cloning tree hash when getting head
* Avoid cloning tree hash when getting head
* Add working arena-based cached tree hash
* Add another benchmark
* Add pre-allocation for caches
* Make cache nullable
* Fix bugs in cache tree hash
* Add validator tree hash optimization
* Optimize hash_concat
* Make hash32_concat return fixed-len array
* Fix failing API tests
* Add new beacon state cache struct
* Add validator-specific cache
* Separate list and values arenas
* Add parallel validator registry hashing
* Remove MultiTreeHashCache
* Remove cached tree hash macro
* Fix failing tree hash test
* Address Michael's comments
* Add CachedTreeHash impl for ef tests
* Fix messy merge conflict
* Rename cache struct, add comments
* Rename cache struct, add comments
* Remove unnecessary mutability
* Wrap iter in result
* Tidy cached tree hash
* Address Michael comments
* Address more comments
* Use ring::Context
* Start adding interop genesis state to lcli
* Use more efficient method to generate genesis state
* Remove duplicate int_to_bytes32
* Add lcli command to change state genesis time
* Add option to allow VC to start with unsynced BN
* Set VC to do parallel key loading
* Don't default to dummy eth1 backend
* Add endpoint to dump operation pool
* Add metrics for op pool
* Remove state clone for slot notifier
* Add mem size approximation for tree hash cache
* Avoid cloning tree hash when getting head
* Fix failing API tests
* Address Michael's comments
* Add HashMap::from_par_iter
* Update op_pool to use proper rewards
* Fix missing use import for tests
* Address Michael's comments
* Revert to private ValidatorStatuses
* Rename variable for clearer code
* Fix update_cover function
* Remove expect
* Add WIP test for rewards
* Use aggregation_bits instead of earliest_attestation_validators
* Use earliest attestation in test and correct typo
* Fix op_pool test thanks to @michaelsproul 's help
* Change test name
* Sparse hot DB and block root tree
* Fix store_tests
* Ensure loads of hot states on boundaries are fast
* Milder error for unaligned finalized blocks
* Add Individual signature verification for ef tests
* Add clone state to avoid errors in ef tests
* Add state verification for VerifyIndividual in ef tests
* Add Clone derive for errors
* Update comments
* Add first draft of validator onboarding
* Update docs
* Add documentation link to main README
* Continue docs development
* Update book readme
* Update docs
* Allow vc to run without testnet subcommand
* Small change to onboarding docs
* Tidy CLI help messages
* Update docs
* Add check to val client see if beacon node is synced
* Add notifier service to validator client
* Re-order onboarding steps
* Update deposit contract address
* Update testnet dir
* Add note about public eth1 node
* Set default eth1 endpoint to sigp
* Fix broken test
* Try fix eth1 cache locking
* Be more specific about eth1 endpoint
* Increase gas limit for deposit
* Fix default deposit amount
* Update lcli to parse spec at boot, remove pycli
* Fix issues when starting with mainnet spec
* Set default spec to mainnet
* Ensure ETH1_FOLLOW_DISTANCE is in YamlConfig
* Set testnet ETH1_FOLLOW_DISTANCE to 16
* Set testnet min validator count
* Add validator count CLI flag to lcli contract deploy
* Extend genesis delay time
* Add basic block/state caching on beacon chain
* Adds checkpoint cache
* Stop storing the tree hash cache in the db
* Remove dedunant beacon state write
* Use caching get methods in fork choice
* Use caching state getting in state_by_slot
* Add state.cacheless_clone
* Attempt to improve attestation processing times
* Introduce HeadInfo struct
* Used cache tree hash for block processing
* Use cached tree hash for block production too
