## Issue Addressed
NA
## Proposed Changes
Allows for multiple "hardcoded" testnets.
## Additional Info
This PR is incomplete.
## TODO
- [x] Add flag to CLI, integrate with rest of Lighthouse.
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
## Issue Addressed
NA
## Proposed Changes
- Introduces the `valdiator_definitions.yml` file which serves as an explicit list of validators that should be run by the validator client.
- Removes `--strict` flag, split into `--strict-lockfiles` and `--disable-auto-discover`
- Adds a "Validator Management" page to the book.
- Adds the `common/account_utils` crate which contains some logic that was starting to duplicate across the codebase.
The new docs for this feature are the best description of it (apart from the code, I guess): 9cb87e93ce/book/src/validator-management.md
## API Changes
This change should be transparent for *most* existing users. If the `valdiator_definitions.yml` doesn't exist then it will be automatically generated using a method that will detect all the validators in their `validators_dir`.
Users will have issues if they are:
1. Using `--strict`.
1. Have keystores in their `~/.lighthouse/validators` directory that weren't being detected by the current keystore discovery method.
For users with (1), the VC will refuse to start because the `--strict` flag has been removed. They will be forced to review `--help` and choose an equivalent flag.
For users with (2), this seems fairly unlikely and since we're only in testnets there's no *real* value on the line here. I'm happy to take the risk, it would be a different case for mainnet.
## Additional Info
This PR adds functionality we will need for #1347.
## TODO
- [x] Reconsider flags
- [x] Move doc into a more reasonable chapter.
- [x] Check for compile warnings.
* Keep wallet name and password file in sync with commands
* Fix create validator example to include required param --count
The current example fails with v0.1.2 fails with "Must supply either --count or --at-most" This change proposes passing count=1, esp'ly since "Creating another validator is easy" a couple of paragraphs down.
* Fix broken links to wallet-create
Wallet links result in file not found. May be caused by the lack of `.md`?
* Include node/health in TOC
* Remove unnecessary book files
Co-authored-by: J Burnett <jburnett@users.noreply.github.com>
* Apply clippy lints to beacon node
* Remove unnecessary logging and correct formatting
* Initial bones of load-balanced range-sync
* Port bump meshsup tests
* Further structure and network handling logic added
* Basic structure, ignoring error handling
* Correct max peers delay bug
* Clean up and re-write message processor and sync manager
* Restructure directory, correct type issues
* Fix compiler issues
* Completed first testing of new sync
* Correct merge issues
* Clean up warnings
* Push attestation processed log down to dbg
* Add state enc/dec benches
* Correct math error, downgraded logs
* Add example for flamegraph
* Use `PublicKeyBytes` for `Validator`
* Ripple PublicKeyBytes change through codebase
* Add RPC error handling and improved syncing code
* Add benches, optimizations to store BeaconState
* Store BeaconState in StorageContainer too
* Optimize StorageContainer with std::mem magic
* Add libp2p stream error handling and dropping of invalid peers
* Lower logs
* Update lcli to parse spec at boot, remove pycli
* Fix issues when starting with mainnet spec
* Set default spec to mainnet
* Fix lcli --spec param
* Add discovery tweak
* Ensure ETH1_FOLLOW_DISTANCE is in YamlConfig
* Set testnet ETH1_FOLLOW_DISTANCE to 16
* Fix rest_api tests
* Set testnet min validator count
* Update with new testnet dir
* Remove some dbg, println
* Add timeout when notifier waits for libp2p lock
* Add validator count CLI flag to lcli contract deploy
* Extend genesis delay time
* Correct libp2p service locking
* Update testnet dir
* Add basic block/state caching on beacon chain
* Add decimals display to notifier sync speed
* Try merge in change to reduce fork choice calls
* Remove fork choice from process block
* Minor log fix
* Check successes > 0
* Adds checkpoint cache
* Stop storing the tree hash cache in the db
* Handles peer disconnects for sync
* Fix failing beacon chain tests
* Change eth2_testnet_config tests to Mainnet
* Add logs downgrade discovery log
* Remove dedunant beacon state write
* Fix re-org warnings
* Use caching get methods in fork choice
* Fix mistake in prev commit
* Use caching state getting in state_by_slot
* Add state.cacheless_clone
* Less fork choice (#679)
* Try merge in change to reduce fork choice calls
* Remove fork choice from process block
* Minor log fix
* Check successes > 0
* Fix failing beacon chain tests
* Fix re-org warnings
* Fix mistake in prev commit
* Attempt to improve attestation processing times
* Introduce HeadInfo struct
* Used cache tree hash for block processing
* Use cached tree hash for block production too
* Range sync refactor
- Introduces `ChainCollection`
- Correct Disconnect node handling
- Removes duplicate code
* Add more logging for DB
* Various bug fixes
* Remove unnecessary logs
* Maintain syncing state in the transition from finalied to head
* Improved disconnect handling
* Add `Speedo` struct
* Fix bugs in speedo
* Fix bug in speedo
* Fix rounding bug in speedo
* Move code around, reduce speedo observation count
* Adds forwards block interator
* Fix inf NaN
* Add first draft of validator onboarding
* Update docs
* Add documentation link to main README
* Continue docs development
* Update book readme
* Update docs
* Allow vc to run without testnet subcommand
* Small change to onboarding docs
* Tidy CLI help messages
* Update docs
* Add check to val client see if beacon node is synced
* Attempt to fix NaN bug
* Fix compile bug
* Add notifier service to validator client
* Re-order onboarding steps
* Update deposit contract address
* Update testnet dir
* Add note about public eth1 node
* Fix installation link
* Set default eth1 endpoint to sigp
* Fix broken test
* Try fix eth1 cache locking
* Be more specific about eth1 endpoint
* Increase gas limit for deposit
* Fix default deposit amount
* Fix re-org log
* Add first draft of validator onboarding
* Update docs
* Add documentation link to main README
* Continue docs development
* Update book readme
* Update docs
* Allow vc to run without testnet subcommand
* Small change to onboarding docs
* Tidy CLI help messages
* Update docs
* Add check to val client see if beacon node is synced
* Add notifier service to validator client
* Re-order onboarding steps
* Update deposit contract address
* Update testnet dir
* Add note about public eth1 node
* Set default eth1 endpoint to sigp
* Fix broken test
* Try fix eth1 cache locking
* Be more specific about eth1 endpoint
* Increase gas limit for deposit
* Fix default deposit amount
* Update to spec v0.9.0
* Update to v0.9.1
* Bump spec tags for v0.9.1
* Formatting, fix CI failures
* Resolve accidental KeyPair merge conflict
* Document new BeaconState functions
* Add `validator` changes from `validator-to-rest`
* Add initial (failing) REST api tests
* Fix signature parsing
* Add more tests
* Refactor http router
* Add working tests for publish beacon block
* Add validator duties tests
* Move account_manager under `lighthouse` binary
* Unify logfile handling in `environment` crate.
* Fix incorrect cache drops in `advance_caches`
* Update fork choice for v0.9.1
* Add `deposit_contract` crate
* Add progress on validator onboarding
* Add unfinished attesation code
* Update account manager CLI
* Write eth1 data file as hex string
* Integrate ValidatorDirectory with validator_client
* Move ValidatorDirectory into validator_client
* Clean up some FIXMEs
* Add beacon_chain_sim
* Fix a few docs/logs
* Expand `beacon_chain_sim`
* Fix spec for `beacon_chain_sim
* More testing for api
* Start work on attestation endpoint
* Reject empty attestations
* Allow attestations to genesis block
* Add working tests for `rest_api` validator endpoint
* Remove grpc from beacon_node
* Start heavy refactor of validator client
- Block production is working
* Prune old validator client files
* Start works on attestation service
* Add attestation service to validator client
* Use full pubkey for validator directories
* Add validator duties post endpoint
* Use par_iter for keypair generation
* Use bulk duties request in validator client
* Add version http endpoint tests
* Add interop keys and startup wait
* Ensure a prompt exit
* Add duties pruning
* Fix compile error in beacon node tests
* Add github workflow
* Modify rust.yaml
* Modify gitlab actions
* Add to CI file
* Add sudo to CI npm install
* Move cargo fmt to own job in tests
* Fix cargo fmt in CI
* Add rustup update before cargo fmt
* Change name of CI job
* Make other CI jobs require cargo fmt
* Add CI badge
* Remove gitlab and travis files
* Add different http timeout for debug
* Update docker file, use makefile in CI
* Use make in the dockerfile, skip the test
* Use the makefile for debug GI test
* Update book
* Tidy grpc and misc things
* Apply discv5 fixes
* Address other minor issues
* Fix warnings
* Attempt fix for addr parsing
* Tidy validator config, CLIs
* Tidy comments
* Tidy signing, reduce ForkService duplication
* Fail if skipping too many slots
* Set default recent genesis time to 0
* Add custom http timeout to validator
* Fix compile bug in node_test_rig
* Remove old bootstrap flag from val CLI
* Update docs
* Tidy val client
* Change val client log levels
* Add comments, more validity checks
* Fix compile error, add comments
* Undo changes to eth2-libp2p/src
* Reduce duplication of keypair generation
* Add more logging for validator duties
* Fix beacon_chain_sim, nitpicks
* Fix compile error, minor nits
* Address Michael's comments