* Sends attestations to the attestation service for processing
* Adds 'attnets' field to local ENR
* Adds ENR bitfield modification logic
* Link attestation service to discovery
- Updates discv5
- Links discover events to discovery
- Support for ENRBitfield
* Adds discovery config params, correct warnings
* Rust fmt fixes
* Correct tests
* Remove ping protocol
* Initial renaming of network services
* Correct rebasing relative to latest master
* Start updating types
* Adds HashMapDelay struct to utils
* Initial network restructure
* Network restructure. Adds new types for v0.2.0
* Removes build artefacts
* Shift validation to beacon chain
* Temporarily remove gossip validation
This is to be updated to match current optimisation efforts.
* Adds AggregateAndProof
* Begin rebuilding pubsub encoding/decoding
* Signature hacking
* Shift gossipsup decoding into eth2_libp2p
* Existing EF tests passing with fake_crypto
* Shifts block encoding/decoding into RPC
* Delete outdated API spec
* All release tests passing bar genesis state parsing
* Update and test YamlConfig
* Update to spec v0.10 compatible BLS
* Updates to BLS EF tests
* Add EF test for AggregateVerify
And delete unused hash2curve tests for uncompressed points
* Update EF tests to v0.10.1
* Use optional block root correctly in block proc
* Use genesis fork in deposit domain. All tests pass
* Fast aggregate verify test
* Update REST API docs
* Fix unused import
* Bump spec tags to v0.10.1
* Add `seconds_per_eth1_block` to chainspec
* Update to timestamp based eth1 voting scheme
* Return None from `get_votes_to_consider` if block cache is empty
* Handle overflows in `is_candidate_block`
* Revert to failing tests
* Fix eth1 data sets test
* Choose default vote according to spec
* Fix collect_valid_votes tests
* Fix `get_votes_to_consider` to choose all eligible blocks
* Uncomment winning_vote tests
* Add comments; remove unused code
* Reduce seconds_per_eth1_block for simulation
* Addressed review comments
* Add test for default vote case
* Fix logs
* Remove unused functions
* Meter default eth1 votes
* Fix comments
* Progress on attestation service
* Address review comments; remove unused dependency
* Initial work on removing libp2p lock
* Add LRU caches to store (rollup)
* Update attestation validation for DB changes (WIP)
* Initial version of should_forward_block
* Scaffold
* Progress on attestation validation
Also, consolidate prod+testing slot clocks so that they share much
of the same implementation and can both handle sub-slot time changes.
* Removes lock from libp2p service
* Completed network lock removal
* Finish(?) attestation processing
* Correct network termination future
* Add slot check to block check
* Correct fmt issues
* Remove Drop implementation for network service
* Add first attempt at attestation proc. re-write
* Add version 2 of attestation processing
* Minor fixes
* Add validator pubkey cache
* Make get_indexed_attestation take a committee
* Link signature processing into new attn verification
* First working version
* Ensure pubkey cache is updated
* Add more metrics, slight optimizations
* Clone committee cache during attestation processing
* Update shuffling cache during block processing
* Remove old commented-out code
* Fix shuffling cache insert bug
* Used indexed attestation in fork choice
* Restructure attn processing, add metrics
* Add more detailed metrics
* Tidy, fix failing tests
* Fix failing tests, tidy
* Address reviewers suggestions
* Disable/delete two outdated tests
* Modification of validator for subscriptions
* Add slot signing to validator client
* Further progress on validation subscription
* Adds necessary validator subscription functionality
* Add new Pubkeys struct to signature_sets
* Refactor with functional approach
* Update beacon chain
* Clean up validator <-> beacon node http types
* Add aggregator status to ValidatorDuty
* Impl Clone for manual slot clock
* Fix minor errors
* Further progress validator client subscription
* Initial subscription and aggregation handling
* Remove decompressed member from pubkey bytes
* Progress to modifying val client for attestation aggregation
* First draft of validator client upgrade for aggregate attestations
* Add hashmap for indices lookup
* Add state cache, remove store cache
* Only build the head committee cache
* Removes lock on a network channel
* Partially implement beacon node subscription http api
* Correct compilation issues
* Change `get_attesting_indices` to use Vec
* Fix failing test
* Partial implementation of timer
* Adds timer, removes exit_future, http api to op pool
* Partial multiple aggregate attestation handling
* Permits bulk messages accross gossipsub network channel
* Correct compile issues
* Improve gosispsub messaging and correct rest api helpers
* Added global gossipsub subscriptions
* Update validator subscriptions data structs
* Tidy
* Re-structure validator subscriptions
* Initial handling of subscriptions
* Re-structure network service
* Add pubkey cache persistence file
* Add more comments
* Integrate persistence file into builder
* Add pubkey cache tests
* Add HashSetDelay and introduce into attestation service
* Handles validator subscriptions
* Add data_dir to beacon chain builder
* Remove Option in pubkey cache persistence file
* Ensure consistency between datadir/data_dir
* Fix failing network test
* Peer subnet discovery gets queued for future subscriptions
* Reorganise attestation service functions
* Initial wiring of attestation service
* First draft of attestation service timing logic
* Correct minor typos
* Tidy
* Fix todos
* Improve tests
* Add PeerInfo to connected peers mapping
* Fix compile error
* Fix compile error from merge
* Split up block processing metrics
* Tidy
* Refactor get_pubkey_from_state
* Remove commented-out code
* Rename state_cache -> checkpoint_cache
* Rename Checkpoint -> Snapshot
* Tidy, add comments
* Tidy up find_head function
* Change some checkpoint -> snapshot
* Add tests
* Expose max_len
* Remove dead code
* Tidy
* Fix bug
* Add sync-speed metric
* Add first attempt at VerifiableBlock
* Start integrating into beacon chain
* Integrate VerifiableBlock
* Rename VerifableBlock -> PartialBlockVerification
* Add start of typed methods
* Add progress
* Add further progress
* Rename structs
* Add full block verification to block_processing.rs
* Further beacon chain integration
* Update checks for gossip
* Add todo
* Start adding segement verification
* Add passing chain segement test
* Initial integration with batch sync
* Minor changes
* Tidy, add more error checking
* Start adding chain_segment tests
* Finish invalid signature tests
* Include single and gossip verified blocks in tests
* Add gossip verification tests
* Start adding docs
* Finish adding comments to block_processing.rs
* Rename block_processing.rs -> block_verification
* Start removing old block processing code
* Fixes beacon_chain compilation
* Fix project-wide compile errors
* Remove old code
* Correct code to pass all tests
* Fix bug with beacon proposer index
* Fix shim for BlockProcessingError
* Only process one epoch at a time
* Fix loop in chain segment processing
* Correct tests from master merge
* Add caching for state.eth1_data_votes
* Add BeaconChain::validator_pubkey
* Revert "Add caching for state.eth1_data_votes"
This reverts commit cd73dcd6434fb8d8e6bf30c5356355598ea7b78e.
Co-authored-by: Grant Wuerker <gwuerker@gmail.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: Michael Sproul <micsproul@gmail.com>
Co-authored-by: pawan <pawandhananjay@gmail.com>
Co-authored-by: Paul Hauner <paul@paulhauner.com>
* Add noise support with fallback to secio
* Add config parameter for noise support
* Add secio/noise compatibility test
* Cleanup
* Remove config parameter for noise support
* Modify test to work between a secio swarm and a noise libp2p service
* Minor fixes
* Set random port when zero-port option is set
* Remove logic allowing lighthouse to update its own ENR
* Discovery address is set to localhost by default
* Return error if discovery-addr isn't explicit
* Create libp2p instance
* Change logger to stdlog
* test_connection initial commit
* Add gossipsub test
* Delete tests in network crate
* Add test module
* Clean tests
* Remove dependency on discovery
* Working publish between 2 nodes
TODO: Publish should be called just once
* Working 2 peer gossipsub test with additional events
* Cleanup test
* Add rpc test
* Star topology discovery WIP
* build_nodes builds and connects n nodes. Increase nodes in gossipsub test
* Add unsubscribe method and expose reference to gossipsub object for gossipsub tests
* Add gossipsub message forwarding test
* Fix gossipsub forward test
* Test improvements
* Remove discovery tests
* Simplify gossipsub forward test topology
* Add helper functions for topology building
* Clean up tests
* Update naming to new network spec
* Correct ssz encoding of protocol names
* Further additions to network upgrade
* Initial network spec update WIP
* Temp commit
* Builds one side of the streamed RPC responses
* Temporary commit
* Propagates streaming changes up into message handler
* Intermediate network update
* Partial update in upgrading to the new network spec
* Update dependencies, remove redundant deps
* Correct sync manager for block stream handling
* Re-write of RPC handler, improves efficiency and corrects bugs
* Stream termination update
* Completed refactor of rpc handler
* Remove crates
* Correct compile issues associated with test merge
* Build basic tests and testing structure for eth2-libp2p
* Enhance RPC tests and add logging
* Complete RPC testing framework and STATUS test
* Decoding bug fixes, log improvements, stream test
* Clean up RPC handler logging
* Decoder bug fix, empty block stream test
* Add BlocksByRoot RPC test
* Add Goodbye RPC test
* Syncing and stream handling bug fixes and performance improvements
* Applies discv5 bug fixes
* Adds DHT IP filtering for lighthouse - currently disabled
* Adds randomized network propagation as a CLI arg
* Add disconnect functionality
* Adds attestation handling and parent lookup
* Adds RPC error handling for the sync manager
* Allow parent's blocks to be already processed
* Update github workflow
* Adds reviewer suggestions
Updates lighthouse to the latest networking spec
- Sync re-write (#496)
- Updates to the latest eth2 networking spec (#495)
- Libp2p updates and improvements
* Renamed fork_choice::process_attestation_from_block
* Processing attestation in fork choice
* Retrieving state from store and checking signature
* Looser check on beacon state validity.
* Cleaned up get_attestation_state
* Expanded fork choice api to provide latest validator message.
* Checking if the an attestation contains a latest message
* Correct process_attestation error handling.
* Copy paste error in comment fixed.
* Tidy ancestor iterators
* Getting attestation slot via helper method
* Refactored attestation creation in test utils
* Revert "Refactored attestation creation in test utils"
This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306.
* Integration tests for free attestation processing
* Implicit conflicts resolved.
* formatting
* Do first pass on Grants code
* Add another attestation processing test
* Tidy attestation processing
* Remove old code fragment
* Add non-compiling half finished changes
* Simplify, fix bugs, add tests for chain iters
* Remove attestation processing from op pool
* Fix bug with fork choice, tidy
* Fix overly restrictive check in fork choice.
* Ensure committee cache is build during attn proc
* Ignore unknown blocks at fork choice
* Various minor fixes
* Make fork choice write lock in to read lock
* Remove unused method
* Tidy comments
* Fix attestation prod. target roots change
* Fix compile error in store iters
* Reject any attestation prior to finalization
* Begin metrics refactor
* Move beacon_chain to new metrics structure.
* Make metrics not panic if already defined
* Use global prometheus gather at rest api
* Unify common metric fns into a crate
* Add heavy metering to block processing
* Remove hypen from prometheus metric name
* Add more beacon chain metrics
* Add beacon chain persistence metric
* Prune op pool on finalization
* Add extra prom beacon chain metrics
* Prefix BeaconChain metrics with "beacon_"
* Add more store metrics
* Add basic metrics to libp2p
* Add metrics to HTTP server
* Remove old `http_server` crate
* Update metrics names to be more like standard
* Fix broken beacon chain metrics, add slot clock metrics
* Add lighthouse_metrics gather fn
* Remove http args
* Fix wrong state given to op pool prune
* Make prom metric names more consistent
* Add more metrics, tidy existing metrics
* Fix store block read metrics
* Tidy attestation metrics
* Fix minor PR comments
* Fix minor PR comments
* Remove duplicated attestation finalization check
* Remove awkward `let` statement
* Add first attempts at HTTP bootstrap
* Add beacon_block methods to rest api
* Fix serde for block.body.grafitti
* Allow travis failures on beta (see desc)
There's a non-backward compatible change in `cargo fmt`. Stable and beta
do not agree.
* Add network routes to API
* Fix rustc warnings
* Add best_slot method
* Add --bootstrap arg to beacon node
* Get bootstrapper working for ENR address
* Store intermediate states during block processing
* Allow bootstrapper to scrape libp2p address
* Update bootstrapper libp2p address finding
* Add comments
* Tidy API to be more consistent with recent decisions
* Address some review comments
* Make BeaconChainTypes Send + Sync + 'static
* Add `/network/listen_port` API endpoint
* Abandon starting the node if libp2p doesn't start
* Update bootstrapper for API changes
* Remove unnecessary trait bounds
* types: first updates for v0.8
* state_processing: epoch processing v0.8.0
* state_processing: block processing v0.8.0
* tree_hash_derive: support generics in SignedRoot
* types v0.8: update to use ssz_types
* state_processing v0.8: use ssz_types
* ssz_types: add bitwise methods and from_elem
* types: fix v0.8 FIXMEs
* ssz_types: add bitfield shift_up
* ssz_types: iterators and DerefMut for VariableList
* types,state_processing: use VariableList
* ssz_types: fix BitVector Decode impl
Fixed a typo in the implementation of ssz::Decode for BitVector, which caused it
to be considered variable length!
* types: fix test modules for v0.8 update
* types: remove slow type-level arithmetic
* state_processing: fix tests for v0.8
* op_pool: update for v0.8
* ssz_types: Bitfield difference length-independent
Allow computing the difference of two bitfields of different lengths.
* Implement compact committee support
* epoch_processing: committee & active index roots
* state_processing: genesis state builder v0.8
* state_processing: implement v0.8.1
* Further improve tree_hash
* Strip examples, tests from cached_tree_hash
* Update TreeHash, un-impl CachedTreeHash
* Update bitfield TreeHash, un-impl CachedTreeHash
* Update FixedLenVec TreeHash, unimpl CachedTreeHash
* Update update tree_hash_derive for new TreeHash
* Fix TreeHash, un-impl CachedTreeHash for ssz_types
* Remove fixed_len_vec, ssz benches
SSZ benches relied upon fixed_len_vec -- it is easier to just delete
them and rebuild them later (when necessary)
* Remove boolean_bitfield crate
* Fix fake_crypto BLS compile errors
* Update ef_tests for new v.8 type params
* Update ef_tests submodule to v0.8.1 tag
* Make fixes to support parsing ssz ef_tests
* `compact_committee...` to `compact_committees...`
* Derive more traits for `CompactCommittee`
* Flip bitfield byte-endianness
* Fix tree_hash for bitfields
* Modify CLI output for ef_tests
* Bump ssz crate version
* Update ssz_types doc comment
* Del cached tree hash tests from ssz_static tests
* Tidy SSZ dependencies
* Rename ssz_types crate to eth2_ssz_types
* validator_client: update for v0.8
* ssz_types: update union/difference for bit order swap
* beacon_node: update for v0.8, EthSpec
* types: disable cached tree hash, update min spec
* state_processing: fix slot bug in committee update
* tests: temporarily disable fork choice harness test
See #447
* committee cache: prevent out-of-bounds access
In the case where we tried to access the committee of a shard that didn't have a committee in the
current epoch, we were accessing elements beyond the end of the shuffling vector and panicking! This
commit adds a check to make the failure safe and explicit.
* fix bug in get_indexed_attestation and simplify
There was a bug in our implementation of get_indexed_attestation whereby
incorrect "committee indices" were used to index into the custody bitfield. The
bug was only observable in the case where some bits of the custody bitfield were
set to 1. The implementation has been simplified to remove the bug, and a test
added.
* state_proc: workaround for compact committees bug
https://github.com/ethereum/eth2.0-specs/issues/1315
* v0.8: updates to make the EF tests pass
* Remove redundant max operation checks.
* Always supply both messages when checking attestation signatures -- allowing
verification of an attestation with no signatures.
* Swap the order of the fork and domain constant in `get_domain`, to match
the spec.
* rustfmt
* ef_tests: add new epoch processing tests
* Integrate v0.8 into master (compiles)
* Remove unused crates, fix clippy lints
* Replace v0.6.3 tags w/ v0.8.1
* Remove old comment
* Ensure lmd ghost tests only run in release
* Update readme
* Fix syncing bugs by recursively attempting to process parents in the import queue, change BlockRootsIterator
* Swap from crossbeam channel to tokio mpsc
* Recursion fix
* Remove exess block processing
* Fix network lag, correct attestation topic
* Correct network poll logic
* Overhaul of SimpleSync and modify BlockRootsIterator to return start_slot
* Fix bug in tests relating to StateRootsIterator
* Remove old, commented-out heartbeat code.
* Tidy docs on import queue enum
* Change source logging msg in simple sync
* Rename function parameter in simple sync
* Use `BestBlockRootsIterator` in `reduced_tree`
* Update comments for `BestBlockRootsIterator`
* Fix duplicate dep in cargo.toml
