Age Manning
81b028b805
Advanced error handling for syncing ( #819 )
...
* Initial block processing thread design
* Correct compilation issues
* Increase logging and request from all given peers
* Patch peer request bug
* Adds fork choice to block processing
* Adds logging for bug isolation
* Patch syncing for chains with skip-slots
* Bump block processing error logs
* Improve logging for attestation processing
* Randomize peer selection during sync
* Resuming chains restarts from local finalized slot
* Downgrades Arc batches to Rc batches
* Add clippy fixes
* Add advanced error handling for invalid/malicious batches
* Downgrade Rc<Batch> to Option<Batch> to pass processed batches to chains
* Squash edge case rpc and syncing bugs
* Process empty batches which could end chains
* Removes last_processed_id concept to account for ending skip-slot batches
* Add logging for chain purges
* Adds retries to re-request batch logging
* Remove bug finding log
* Add reviewers suggestions
* Revert to master modifications
* Line wrapping
* Revert to master
2020-01-23 19:25:13 +11:00
Pawan Dhananjay
23a35c3767
Persist/load DHT on shutdown/startup ( #659 )
...
* Store dht enrs on shutdown
* Load enrs on startup and add tests
* Remove enr_entries from behavior
* Move all dht persisting logic to `NetworkService`
* Move `PersistedDht` from eth2-libp2p to network crate
* Add test to confirm dht persistence
* Add logging
* Remove extra call to beacon_chain persist
* Expose only mutable `add_enr` method from behaviour
* Fix tests
* Fix merge errors
2020-01-23 18:16:11 +11:00
Pawan Dhananjay
89f05e4a4f
Adds simulator for syncing ( #758 )
...
* Add CLI for beacon_chain_sim
* Rename beacon-chain-sim to simulator
* Fix simulator workflow
* Push Cargo.lock
* WIP syncing simulator
* Add cli args
* Remove eth1 stuff and deposits
* Add syncing strategy simulations
* Successful one node sync
* Clean up
* Rename to avoid confusion
* add command line args
* fix cargo fmt issues
* Add additional syncing strategies
* Run all syncing strategies one after other; add comments
* Improve cli argument parsing
* Change `end_after_checks` default to true
* Small modifications to syncing-sim
* Add `strategy` cli argument
* Documented defaults in cli help
Co-authored-by: mkinney <mike.kinney@gmail.com>
Co-authored-by: Age Manning <Age@AgeManning.com>
2020-01-23 18:07:39 +11:00
Age Manning
8c96739cab
Correct discovery address CLI functionality ( #818 )
...
* Improve handling of discovery IP address CLI config
* Remove excess debug logging
* Add reviewers suggestions
2020-01-23 17:31:08 +11:00
Age Manning
fdb6e28f94
Super/Silky smooth syncs ( #816 )
...
* Initial block processing thread design
* Correct compilation issues
* Increase logging and request from all given peers
* Patch peer request bug
* Adds fork choice to block processing
* Adds logging for bug isolation
* Patch syncing for chains with skip-slots
* Bump block processing error logs
* Improve logging for attestation processing
* Randomize peer selection during sync
* Resuming chains restarts from local finalized slot
* Downgrades Arc batches to Rc batches
* Add clippy fixes
* Downgrade Rc<Batch> to Option<Batch> to pass processed batches to chains
* Add reviewers suggestions
2020-01-23 17:30:49 +11:00
pscott
f8cff3bd2e
Optimize block production ( #820 )
...
* Remove SignatureVerif on block production; short-circuit fetching attestations when num attestations < T::MaxAttestation
* Cargo fmt
* Remove short-circuiting
2020-01-23 11:35:13 +11:00
pscott
7396cd2cab
Fix clippy warnings ( #813 )
...
* Clippy account manager
* Clippy account_manager
* Clippy beacon_node/beacon_chain
* Clippy beacon_node/client
* Clippy beacon_node/eth1
* Clippy beacon_node/eth2-libp2p
* Clippy beacon_node/genesis
* Clippy beacon_node/network
* Clippy beacon_node/rest_api
* Clippy beacon_node/src
* Clippy beacon_node/store
* Clippy eth2/lmd_ghost
* Clippy eth2/operation_pool
* Clippy eth2/state_processing
* Clippy eth2/types
* Clippy eth2/utils/bls
* Clippy eth2/utils/cahced_tree_hash
* Clippy eth2/utils/deposit_contract
* Clippy eth2/utils/eth2_interop_keypairs
* Clippy eth2/utils/eth2_testnet_config
* Clippy eth2/utils/lighthouse_metrics
* Clippy eth2/utils/ssz
* Clippy eth2/utils/ssz_types
* Clippy eth2/utils/tree_hash_derive
* Clippy lcli
* Clippy tests/beacon_chain_sim
* Clippy validator_client
* Cargo fmt
2020-01-21 18:38:56 +11:00
pscott
1abb964652
Update op_pool to use proper rewards ( #707 )
...
* Update op_pool to use proper rewards
* Fix missing use import for tests
* Address Michael's comments
* Revert to private ValidatorStatuses
* Rename variable for clearer code
* Fix update_cover function
* Remove expect
* Add WIP test for rewards
* Use aggregation_bits instead of earliest_attestation_validators
* Use earliest attestation in test and correct typo
* Fix op_pool test thanks to @michaelsproul 's help
* Change test name
2020-01-20 10:33:28 +11:00
Michael Sproul
4632e9ce52
Document the freezer DB space-time trade-off ( #808 )
2020-01-20 10:01:58 +11:00
Pawan Dhananjay
82b55ea418
Persist eth1 cache ( #760 )
...
* Add intermediate structures for bytes conversion
* Expose byte conversion methods from `Eth1Service`
* Add eth1 ssz containers
* Fix type errors
* Load eth1 cache on restart
* Fix compile errors
* Update Cargo.lock
* Add comments and minor formatting
* Add test for eth1 cache persistence
* Restrict Deposit and Block cache field visibility
* Add checks
* Fix `SszDepositCache` check
* Implement Encode/Decode directly on `BlockCache`
2020-01-20 08:02:48 +11:00
Pawan Dhananjay
3ba221e388
Persist eth1 cache ( #760 )
...
* Add intermediate structures for bytes conversion
* Expose byte conversion methods from `Eth1Service`
* Add eth1 ssz containers
* Fix type errors
* Load eth1 cache on restart
* Fix compile errors
* Update Cargo.lock
* Add comments and minor formatting
* Add test for eth1 cache persistence
* Restrict Deposit and Block cache field visibility
* Add checks
* Fix `SszDepositCache` check
* Implement Encode/Decode directly on `BlockCache`
2020-01-20 07:54:19 +11:00
Pawan Dhananjay
cb13129cd6
Persist eth1 cache ( #760 )
...
* Add intermediate structures for bytes conversion
* Expose byte conversion methods from `Eth1Service`
* Add eth1 ssz containers
* Fix type errors
* Load eth1 cache on restart
* Fix compile errors
* Update Cargo.lock
* Add comments and minor formatting
* Add test for eth1 cache persistence
* Restrict Deposit and Block cache field visibility
* Add checks
* Fix `SszDepositCache` check
* Implement Encode/Decode directly on `BlockCache`
2020-01-20 07:53:55 +11:00
Pawan Dhananjay
661ef65de8
Persist eth1 cache ( #760 )
...
* Add intermediate structures for bytes conversion
* Expose byte conversion methods from `Eth1Service`
* Add eth1 ssz containers
* Fix type errors
* Load eth1 cache on restart
* Fix compile errors
* Update Cargo.lock
* Add comments and minor formatting
* Add test for eth1 cache persistence
* Restrict Deposit and Block cache field visibility
* Add checks
* Fix `SszDepositCache` check
* Implement Encode/Decode directly on `BlockCache`
2020-01-20 07:52:58 +11:00
Tim Myers
a8da36b913
fix(dockerfile): Add ca-certificates so eth1 calls work in docker. ( #796 )
2020-01-20 07:49:59 +11:00
Pawan Dhananjay
0e3e2bbbe7
Remove outdated cli options from docs ( #802 )
2020-01-15 16:00:32 +11:00
Age Manning
13e74e5352
Correct stream timeout delay queue handling ( #803 )
...
* Correct stream timeout delay queue handling
* Correct small typo
2020-01-15 15:50:01 +11:00
Age Manning
c184a98170
Sync fixes ( #801 )
...
* Randomize peer selection for batch errors
* Downgrade attestation logging
* Handle range sync errors
* Update lock file
* Downgrade logs
* Decrease batch size for better thread handling
* Optimise peer selection in range sync
2020-01-15 14:48:09 +11:00
pscott
6285dc3aa7
Fix logic error in operation pool ( #806 )
2020-01-15 12:41:12 +11:00
Paul Hauner
03443c3e57
Bump docker rust version ( #794 )
...
* Bump docker rust version
* Improve docker ignore
2020-01-10 16:49:16 +11:00
Paul Hauner
271fdd4fb9
Bumb version tags ( #792 )
2020-01-10 16:01:37 +11:00
Paul Hauner
55a6dc9ae3
Notifier errors ( #793 )
...
* Handle notifier errors
* Log error for notifier errors
2020-01-10 16:01:22 +11:00
Paul Hauner
3afa7b0dab
Print validator pubkeys during generation ( #791 )
...
* Start docker docs
* Add progress
* Update docker.md
* Add note about geth syncing
* Fix env to use geth
* Update docs
* Remove sigp goerli node
* Change text about eth1 syncing
* Address comments from @pscott
* Add links to beaconcha.in
* Add instructions for starting an eth1 node
* Print validator pubkeys during generation
* Fix typo
* Re-organise Syncing log
* Add notes about finding the validator pubkey
* Fix double-0x prefix bug
* Fix typo
2020-01-10 15:47:07 +11:00
Paul Hauner
370c658c7c
Optimization: reduce BLS decompression ( #766 )
...
* Add RwLock-style caching for BLS pubkeys
* Tidy docker ignore
* Remove RwLocks
* Merge in master
2020-01-10 15:32:10 +11:00
Michael Sproul
5a8f2dd961
Increase default slots per restore point to 2048 ( #790 )
...
This should reduce disk usage by 32x while keeping historical state queries to
less than 10s. If historical states are required quickly, the minimum SPRP of 32
can be set on the CLI.
2020-01-10 14:42:49 +11:00
Age Manning
b3712d8e9b
Discv5 upgrade ( #788 )
...
* Update discv5 to delay queues for timeouts
* Further discv5 bug fixes
2020-01-10 14:35:07 +11:00
Paul Hauner
698af6d7ec
Update docs for testnet3 ( #773 )
...
* Start docker docs
* Add progress
* Update docker.md
* Add note about geth syncing
* Fix env to use geth
* Update docs
* Remove sigp goerli node
* Change text about eth1 syncing
* Address comments from @pscott
* Add links to beaconcha.in
* Add instructions for starting an eth1 node
* Fix typo
* Re-organise Syncing log
2020-01-10 13:56:48 +11:00
Paul Hauner
efd73230cf
Add testnet3 params ( #785 )
...
* Add testnet3 params
* Update sydney ENR
2020-01-10 11:47:00 +11:00
Age Manning
1aaeb6b505
Upgrades trace logging for Sync RPC method ( #784 )
2020-01-10 02:01:03 +11:00
Michael Sproul
95fc840e2c
Fix off-by-one error in get_latest_restore_point ( #787 )
...
* Fix off-by-one error in get_latest_restore_point
* Tighten SPRP checks for succinct hot DB change
2020-01-09 21:05:56 +11:00
Michael Sproul
d9e9c17d3b
Avoid building caches during block replay ( #783 )
...
Also, make the ExitCache safe.
2020-01-09 11:43:11 +11:00
Paul Hauner
da95a73605
Fix typo in eth1 service ( #782 )
2020-01-09 10:53:25 +11:00
Age Manning
30f51df4cf
Improved RPC timeout handling ( #770 )
...
* Handle slow app level responses for RPC
* Correct errors
* Terminates stream on error
2020-01-08 14:18:06 +11:00
Michael Sproul
f36a5a15d6
Store states efficiently in the hot database ( #746 )
...
* Sparse hot DB and block root tree
* Fix store_tests
* Ensure loads of hot states on boundaries are fast
* Milder error for unaligned finalized blocks
2020-01-08 13:58:01 +11:00
Paul Hauner
26dde26c48
Use Cow for checkpoint cache ( #775 )
2020-01-08 11:09:27 +11:00
pscott
8e1e6838d2
Add individual processing for ef tests ( #745 )
...
* Add Individual signature verification for ef tests
* Add clone state to avoid errors in ef tests
* Add state verification for VerifyIndividual in ef tests
* Add Clone derive for errors
* Update comments
2020-01-08 10:24:44 +11:00
pscott
95cc5dd22f
Rename ssz_encode_state.rs to ssz_encode_state_container.rs ( #774 )
2020-01-07 10:33:58 +11:00
Paul Hauner
f04c55075e
Add timeouts to canonical head rwlock ( #759 )
...
* Add TimeoutRwLock to BeaconChain
* Update network crate
* Update rest api
* Fix beacon chain tests
* Fix rest api tests
* Set test back to !debug_assertions
2020-01-06 17:30:37 +11:00
Paul Hauner
b0c8b2b700
Change genesis fork for testnets ( #750 )
...
* Change genesis fork for testnets
* Fix genesis fork in lcli
2020-01-06 17:30:08 +11:00
Paul Hauner
2dddbed7ce
Update notice about testnet being down ( #772 )
2020-01-06 17:29:44 +11:00
Paul Hauner
212a5a8ffa
Update deposit contract in docs ( #769 )
2020-01-06 15:44:06 +11:00
Paul Hauner
4d0ef41a2c
Fix testnet docs ( #747 )
...
* Fix testnet docs
* Minor fixes
* More fixes
* Describe http flag
* Tidy
* Revert change
* Address suggestion from Pawan
2020-01-06 14:31:43 +11:00
pscott
5427664cf4
Add log-format option to CLI ( #744 )
...
* Add log-format CLI option
* Cargo fmt
* Add log format logic for file logging. Add doc
* Review comment
* Fix compilation errors
* Remove Mutex from logger
2020-01-06 12:26:30 +11:00
Age Manning
55680ab1d3
Update libp2p and updates gossipsub ( #764 )
...
* Update gossipsub, adds anonymous source
* Remove temp protobuf fix
2020-01-03 17:56:03 +11:00
Pawan Dhananjay
7320f8497f
Remove the logic allowing lighthouse to update it's own ENR ( #682 )
...
* Set random port when zero-port option is set
* Remove logic allowing lighthouse to update its own ENR
* Discovery address is set to localhost by default
* Return error if discovery-addr isn't explicit
2020-01-03 15:37:05 +11:00
Paul Hauner
647034b637
Optimization: avoid recomputing known state roots ( #762 )
...
* Start adding optimization
* Add temp fix for protobuf issue
* Fix compile errors
* Fix protobuf import
2020-01-03 15:09:00 +11:00
pscott
39df89521f
Fix --datadir cli option ( #753 )
...
* Fix datadir bug; fix clippy warnings; update error message
* Cargo fmt
* Update error message
2020-01-03 11:00:13 +11:00
pscott
50ea669e1e
Update CONTRIBUTING.md ( #751 )
...
* Replace gitter with Discord; fix typos; Specify the need to sign commits
* Add link to contributing document in lighthouse book
2020-01-03 10:45:53 +11:00
Age Manning
01ac60d638
Documents and simplifies syncing code ( #738 )
...
* Add duplication prevention to gossipsub
* Clean up topic logs
* Add content addressed messages for gossip
* Support BlocksByRange step parameter
* Add initial docs to syncing
* Adds documentation and simplify code in sync
* Remove unnecessary comment
* Fix fmt issue
* Add batch log to sync
2019-12-23 21:23:14 +11:00
mkinney
3acb3cc640
got Docker image down to 138mb from 1.8gb ( #749 )
2019-12-20 18:30:38 +11:00
Age Manning
41d6d5fafd
Adds support for the step parameter in BlocksByRange request ( #737 )
...
* Add duplication prevention to gossipsub
* Clean up topic logs
* Add content addressed messages for gossip
* Support BlocksByRange step parameter
2019-12-20 17:12:16 +11:00
