lighthouse/book/src/key-managment.md

105 lines
4.9 KiB
Markdown
Raw Normal View History

Wallet-based, encrypted key management (#1138) * Update hashmap hashset to stable futures * Adds panic test to hashset delay * Port remote_beacon_node to stable futures * Fix lcli merge conflicts * Non rpc stuff compiles * Remove padding * Add error enum, zeroize more things * Fix comment * protocol.rs compiles * Port websockets, timer and notifier to stable futures (#1035) * Fix lcli * Port timer to stable futures * Fix timer * Port websocket_server to stable futures * Port notifier to stable futures * Add TODOS * Port remote_beacon_node to stable futures * Partial eth2-libp2p stable future upgrade * Finished first round of fighting RPC types * Further progress towards porting eth2-libp2p adds caching to discovery * Update behaviour * Add keystore builder * Remove keystore stuff from val client * Add more tests, comments * RPC handler to stable futures * Update RPC to master libp2p * Add more comments, test vectors * Network service additions * Progress on improving JSON validation * More JSON verification * Start moving JSON into own mod * Remove old code * Add more tests, reader/writers * Tidy * Move keystore into own file * Move more logic into keystore file * Tidy * Tidy * Fix the fallback transport construction (#1102) * Allow for odd-character hex * Correct warning * Remove hashmap delay * Compiling version of eth2-libp2p * Update all crates versions * Fix conversion function and add tests (#1113) * Add more json missing field checks * Use scrypt by default * Tidy, address comments * Test path and uuid in vectors * Fix comment * Add checks for kdf params * Enforce empty kdf message * Port validator_client to stable futures (#1114) * Add PH & MS slot clock changes * Account for genesis time * Add progress on duties refactor * Add simple is_aggregator bool to val subscription * Start work on attestation_verification.rs * Add progress on ObservedAttestations * Progress with ObservedAttestations * Fix tests * Add observed attestations to the beacon chain * Add attestation observation to processing code * Add progress on attestation verification * Add first draft of ObservedAttesters * Add more tests * Add observed attesters to beacon chain * Add observers to attestation processing * Add more attestation verification * Create ObservedAggregators map * Remove commented-out code * Add observed aggregators into chain * Add progress * Finish adding features to attestation verification * Ensure beacon chain compiles * Link attn verification into chain * Integrate new attn verification in chain * Remove old attestation processing code * Start trying to fix beacon_chain tests * Split adding into pools into two functions * Add aggregation to harness * Get test harness working again * Adjust the number of aggregators for test harness * Fix edge-case in harness * Integrate new attn processing in network * Fix compile bug in validator_client * Update validator API endpoints * Fix aggreagation in test harness * Fix enum thing * Fix attestation observation bug: * Patch failing API tests * Start adding comments to attestation verification * Remove unused attestation field * Unify "is block known" logic * Update comments * Supress fork choice errors for network processing * Add todos * Tidy * Add gossip attn tests * Disallow test harness to produce old attns * Comment out in-progress tests * Partially address pruning tests * Fix failing store test * Add aggregate tests * Add comments about which spec conditions we check * Dont re-aggregate * Split apart test harness attn production * Fix compile error in network * Make progress on commented-out test * Fix skipping attestation test * Add fork choice verification tests * Tidy attn tests, remove dead code * Remove some accidentally added code * Fix clippy lint * Rename test file * Add block tests, add cheap block proposer check * Rename block testing file * Add observed_block_producers * Tidy * Switch around block signature verification * Finish block testing * Remove gossip from signature tests * First pass of self review * Fix deviation in spec * Update test spec tags * Start moving over to hashset * Finish moving observed attesters to hashmap * Move aggregation pool over to hashmap * Make fc attn borrow again * Fix rest_api compile error * Fix missing comments * Fix monster test * Uncomment increasing slots test * Address remaining comments * Remove unsafe, use cfg test * Remove cfg test flag * Fix dodgy comment * Revert "Update hashmap hashset to stable futures" This reverts commit d432378a3cc5cd67fc29c0b15b96b886c1323554. * Revert "Adds panic test to hashset delay" This reverts commit 281502396fc5b90d9c421a309c2c056982c9525b. * Ported attestation_service * Ported duties_service * Ported fork_service * More ports * Port block_service * Minor fixes * VC compiles * Update TODOS * Borrow self where possible * Ignore aggregates that are already known. * Unify aggregator modulo logic * Fix typo in logs * Refactor validator subscription logic * Avoid reproducing selection proof * Skip HTTP call if no subscriptions * Rename DutyAndState -> DutyAndProof * Tidy logs * Print root as dbg * Fix compile errors in tests * Fix compile error in test * Re-Fix attestation and duties service * Minor fixes Co-authored-by: Paul Hauner <paul@paulhauner.com> * Expose json_keystore mod * First commits on path derivation * Progress with implementation * More progress * Passing intermediate test vectors * Tidy, add comments * Add DerivedKey structs * Move key derivation into own crate * Add zeroize structs * Return error for empty seed * Add tests * Tidy * First commits on path derivation * Progress with implementation * Move key derivation into own crate * Start defining JSON wallet * Add progress * Split out encrypt/decrypt * First commits on path derivation * Progress with implementation * More progress * Passing intermediate test vectors * Tidy, add comments * Add DerivedKey structs * Move key derivation into own crate * Add zeroize structs * Return error for empty seed * Add tests * Tidy * Add progress * Replace some password usage with slice * First commits on path derivation * Progress with implementation * More progress * Passing intermediate test vectors * Tidy, add comments * Add DerivedKey structs * Move key derivation into own crate * Add zeroize structs * Return error for empty seed * Add tests * Tidy * Add progress * Expose PlainText struct * First commits on path derivation * Progress with implementation * More progress * Passing intermediate test vectors * Tidy, add comments * Add DerivedKey structs * Move key derivation into own crate * Add zeroize structs * Return error for empty seed * Add tests * Tidy * Add builder * Expose consts, remove Password * Minor progress * Expose SALT_SIZE * First compiling version * Add test vectors * Network crate update to stable futures * Move dbg assert statement * Port account_manager to stable futures (#1121) * Port account_manager to stable futures * Run async fns in tokio environment * Port rest_api crate to stable futures (#1118) * Port rest_api lib to stable futures * Reduce tokio features * Update notifier to stable futures * Builder update * Further updates * Add mnemonic, tidy * Convert self referential async functions * Tidy * Add testing * Add first attempt at validator_dir * Present pubkey field * stable futures fixes (#1124) * Fix eth1 update functions * Fix genesis and client * Fix beacon node lib * Return appropriate runtimes from environment * Fix test rig * Refactor eth1 service update * Upgrade simulator to stable futures * Lighthouse compiles on stable futures * Add first pass of wallet manager * Progress with CLI * Remove println debugging statement * Tidy output * Tidy 600 perms * Update libp2p service, start rpc test upgrade * Add validator creation flow * Update network crate for new libp2p * Start tidying, adding comments * Update tokio::codec to futures_codec (#1128) * Further work towards RPC corrections * Correct http timeout and network service select * Add wallet mgr testing * Shift LockedWallet into own file * Add comments to fs * Start integration into VC * Use tokio runtime for libp2p * Revert "Update tokio::codec to futures_codec (#1128)" This reverts commit e57aea924acf5cbabdcea18895ac07e38a425ed7. * Upgrade RPC libp2p tests * Upgrade secio fallback test * Add lcli keypair upgrade command * Upgrade gossipsub examples * Clean up RPC protocol * Test fixes (#1133) * Correct websocket timeout and run on os thread * Fix network test * Add --secrets-dir to VC * Remove --legacy-keys from VC * Clean up PR * Correct tokio tcp move attestation service tests * Upgrade attestation service tests * Fix sim * Correct network test * Correct genesis test * Start docs * Add progress for validator generation * Tidy error messages * Test corrections * Log info when block is received * Modify logs and update attester service events * Stable futures: fixes to vc, eth1 and account manager (#1142) * Add local testnet scripts * Remove whiteblock script * Rename local testnet script * Move spawns onto handle * Fix VC panic * Initial fix to block production issue * Tidy block producer fix * Tidy further * Add local testnet clean script * Run cargo fmt * Tidy duties service * Tidy fork service * Tidy ForkService * Tidy AttestationService * Tidy notifier * Ensure await is not suppressed in eth1 * Ensure await is not suppressed in account_manager * Use .ok() instead of .unwrap_or(()) * RPC decoding test for proto * Update discv5 and eth2-libp2p deps * Run cargo fmt * Pre-build keystores for sim * Fix lcli double runtime issue (#1144) * Handle stream termination and dialing peer errors * Correct peer_info variant types * Add progress on new deposit flow * Remove unnecessary warnings * Handle subnet unsubscription removal and improve logigng * Add logs around ping * Upgrade discv5 and improve logging * Handle peer connection status for multiple connections * Improve network service logging * Add more incomplete progress * Improve logging around peer manager * Upgrade swarm poll centralise peer management * Identify clients on error * Fix `remove_peer` in sync (#1150) * remove_peer removes from all chains * Remove logs * Fix early return from loop * Improved logging, fix panic * Partially correct tests * Add deposit command * Remove old validator directory * Start adding AM tests * Stable futures: Vc sync (#1149) * Improve syncing heuristic * Add comments * Use safer method for tolerance * Fix tests * Binary testing progress * Progress with CLI tests * Use constants for flags * More account manager testing * Improve CLI tests * Move upgrade-legacy-keypairs into account man * Use rayon for VC key generation * Add comments to `validator_dir` * Add testing to validator_dir * Add fix to eth1-sim * Check errors in eth1-sim * Fix mutability issue * Ensure password file ends in .pass * Add more tests to wallet manager * Tidy deposit * Tidy account manager * Tidy account manager * Remove panic * Generate keypairs earlier in sim * Tidy eth1-sime * Try to fix eth1 sim * Address review comments * Fix typo in CLI command * Update docs * Disable eth1 sim * Remove eth1 sim completely Co-authored-by: Age Manning <Age@AgeManning.com> Co-authored-by: pawanjay176 <pawandhananjay@gmail.com>
2020-05-18 09:01:45 +00:00
# Key Management
Lighthouse uses a _hierarchical_ key management system for producing validator
keys. It is hierarchical because each validator key can be _derived_ from a
master key, making the validators keys _children_ of the master key. This
scheme means that a single 12-word mnemonic can be used to backup all of your
validator keys without providing any observable link between them (i.e., it is
privacy-retaining). Hierarchical key derivation schemes are common-place in
cryptocurrencies, they are already used by most hardware and software wallets
to secure BTC, ETH and many other coins.
## Key Concepts
We defined some terms in the context of validator key management:
- **Mnemonic**: a string of 12-words that is designed to be easy to write down
and remember. E.g., _"enemy fog enlist laundry nurse hungry discover turkey holiday resemble glad discover"_.
- Defined in BIP-39
- **Wallet**: a wallet is a JSON file which stores an
encrypted version of a mnemonic.
- Defined in EIP-2386
- **Keystore**: typically created by wallet, it contains a single encrypted BLS
keypair.
- Defined in EIP-2335.
- **Voting Keypair**: a BLS public and private keypair which is used for
signing blocks, attestations and other messages on regular intervals,
whilst staking in Phase 0.
- **Withdrawal Keypair**: a BLS public and private keypair which will be
required _after_ Phase 0 to manage ETH once a validator has exited.
## Overview
The key management system in Lighthouse involves moving down the above list of
items, starting at one easy-to-backup mnemonic and ending with multiple
keypairs. Creating a single validator looks like this:
1. Create a **wallet** and record the **mnemonic**:
- `lighthouse account wallet create --name wally --passphrase-file wally.pass`
1. Create the voting and withdrawal **keystores** for one validator:
- `lighthouse account validator create --wallet-name wally --wallet-passphrase wally.pass`
In step (1), we created a wallet in `~/.lighthouse/wallets` with the name
`mywallet`. We encrypted this using a pre-defined password in the
`mywallet.pass` file. Then, in step (2), we created a new validator in the
`~/.lighthouse/validators` directory using `mywallet` (unlocking it with
`mywallet.pass`) and storing the passwords to the validators voting key in
`~/.lighthouse/secrets`.
Thanks to the hierarchical key derivation scheme, we can delete all of the
aforementioned directories and then regenerate them as long as we remembered
the 12-word mnemonic (we don't recommend doing this, though).
Creating another validator is easy, it's just a matter of repeating step (2).
The wallet keeps track of how many validators it has generated and ensures that
a new validator is generated each time.
## Detail
### Directory Structure
There are three important directories in Lighthouse validator key management:
- `wallets/`: contains encrypted wallets which are used for hierarchical
key derivation.
- Defaults to `~/.lighthouse/wallets`
- `validators/`: contains a directory for each validator containing
encrypted keystores and other validator-specific data.
- Defaults to `~/.lighthouse/validators`
- `secrets/`: since the validator signing keys are "hot", the validator process
needs access to the passwords to decrypt the keystores in the validators
dir. These passwords are stored here.
- Defaults to `~/.lighthouse/secrets`
When the validator client boots, it searches the `validators/` for directories
containing voting keystores. When it discovers a keystore, it searches the
`secrets/` dir for a file with the same name as the 0x-prefixed hex
representation of the keystore public key. If it finds this file, it attempts
to decrypt the keystore using the contents of this file as the password. If it
fails, it logs an error and moves onto the next keystore.
The `validators/` and `secrets/` directories are kept separate to allow for
ease-of-backup; you can safely backup `validators/` without worrying about
leaking private key data.
### Withdrawal Keypairs
In Eth2 Phase 0, withdrawal keypairs do not serve any immediate purpose.
However, they become very important _after_ Phase 0: they will provide the
ultimate control of the ETH of withdrawn validators.
This presents an interesting key management scenario: withdrawal keys are very
important, but not right now. Considering this, Lighthouse has adopted a
strategy where **we do not save withdrawal keypairs to disk by default** (it is
opt-in). Instead, we assert that since the withdrawal keys can be regenerated
from a mnemonic, having them lying around on the file-system only presents risk
and complexity.
At the time or writing, we do not expose the commands to regenerate keys from
mnemonics. However, key regeneration is tested on the public Lighthouse
repository and will be exposed prior to mainnet launch.
So, in summary, withdrawal keypairs can be trivially regenerated from the
mnemonic via EIP-2333 so they are not saved to disk like the voting keypairs.