lighthouse/common/validator_dir/tests/tests.rs

274 lines
8.1 KiB
Rust
Raw Normal View History

Wallet-based, encrypted key management (#1138) * Update hashmap hashset to stable futures * Adds panic test to hashset delay * Port remote_beacon_node to stable futures * Fix lcli merge conflicts * Non rpc stuff compiles * Remove padding * Add error enum, zeroize more things * Fix comment * protocol.rs compiles * Port websockets, timer and notifier to stable futures (#1035) * Fix lcli * Port timer to stable futures * Fix timer * Port websocket_server to stable futures * Port notifier to stable futures * Add TODOS * Port remote_beacon_node to stable futures * Partial eth2-libp2p stable future upgrade * Finished first round of fighting RPC types * Further progress towards porting eth2-libp2p adds caching to discovery * Update behaviour * Add keystore builder * Remove keystore stuff from val client * Add more tests, comments * RPC handler to stable futures * Update RPC to master libp2p * Add more comments, test vectors * Network service additions * Progress on improving JSON validation * More JSON verification * Start moving JSON into own mod * Remove old code * Add more tests, reader/writers * Tidy * Move keystore into own file * Move more logic into keystore file * Tidy * Tidy * Fix the fallback transport construction (#1102) * Allow for odd-character hex * Correct warning * Remove hashmap delay * Compiling version of eth2-libp2p * Update all crates versions * Fix conversion function and add tests (#1113) * Add more json missing field checks * Use scrypt by default * Tidy, address comments * Test path and uuid in vectors * Fix comment * Add checks for kdf params * Enforce empty kdf message * Port validator_client to stable futures (#1114) * Add PH & MS slot clock changes * Account for genesis time * Add progress on duties refactor * Add simple is_aggregator bool to val subscription * Start work on attestation_verification.rs * Add progress on ObservedAttestations * Progress with ObservedAttestations * Fix tests * Add observed attestations to the beacon chain * Add attestation observation to processing code * Add progress on attestation verification * Add first draft of ObservedAttesters * Add more tests * Add observed attesters to beacon chain * Add observers to attestation processing * Add more attestation verification * Create ObservedAggregators map * Remove commented-out code * Add observed aggregators into chain * Add progress * Finish adding features to attestation verification * Ensure beacon chain compiles * Link attn verification into chain * Integrate new attn verification in chain * Remove old attestation processing code * Start trying to fix beacon_chain tests * Split adding into pools into two functions * Add aggregation to harness * Get test harness working again * Adjust the number of aggregators for test harness * Fix edge-case in harness * Integrate new attn processing in network * Fix compile bug in validator_client * Update validator API endpoints * Fix aggreagation in test harness * Fix enum thing * Fix attestation observation bug: * Patch failing API tests * Start adding comments to attestation verification * Remove unused attestation field * Unify "is block known" logic * Update comments * Supress fork choice errors for network processing * Add todos * Tidy * Add gossip attn tests * Disallow test harness to produce old attns * Comment out in-progress tests * Partially address pruning tests * Fix failing store test * Add aggregate tests * Add comments about which spec conditions we check * Dont re-aggregate * Split apart test harness attn production * Fix compile error in network * Make progress on commented-out test * Fix skipping attestation test * Add fork choice verification tests * Tidy attn tests, remove dead code * Remove some accidentally added code * Fix clippy lint * Rename test file * Add block tests, add cheap block proposer check * Rename block testing file * Add observed_block_producers * Tidy * Switch around block signature verification * Finish block testing * Remove gossip from signature tests * First pass of self review * Fix deviation in spec * Update test spec tags * Start moving over to hashset * Finish moving observed attesters to hashmap * Move aggregation pool over to hashmap * Make fc attn borrow again * Fix rest_api compile error * Fix missing comments * Fix monster test * Uncomment increasing slots test * Address remaining comments * Remove unsafe, use cfg test * Remove cfg test flag * Fix dodgy comment * Revert "Update hashmap hashset to stable futures" This reverts commit d432378a3cc5cd67fc29c0b15b96b886c1323554. * Revert "Adds panic test to hashset delay" This reverts commit 281502396fc5b90d9c421a309c2c056982c9525b. * Ported attestation_service * Ported duties_service * Ported fork_service * More ports * Port block_service * Minor fixes * VC compiles * Update TODOS * Borrow self where possible * Ignore aggregates that are already known. * Unify aggregator modulo logic * Fix typo in logs * Refactor validator subscription logic * Avoid reproducing selection proof * Skip HTTP call if no subscriptions * Rename DutyAndState -> DutyAndProof * Tidy logs * Print root as dbg * Fix compile errors in tests * Fix compile error in test * Re-Fix attestation and duties service * Minor fixes Co-authored-by: Paul Hauner <paul@paulhauner.com> * Expose json_keystore mod * First commits on path derivation * Progress with implementation * More progress * Passing intermediate test vectors * Tidy, add comments * Add DerivedKey structs * Move key derivation into own crate * Add zeroize structs * Return error for empty seed * Add tests * Tidy * First commits on path derivation * Progress with implementation * Move key derivation into own crate * Start defining JSON wallet * Add progress * Split out encrypt/decrypt * First commits on path derivation * Progress with implementation * More progress * Passing intermediate test vectors * Tidy, add comments * Add DerivedKey structs * Move key derivation into own crate * Add zeroize structs * Return error for empty seed * Add tests * Tidy * Add progress * Replace some password usage with slice * First commits on path derivation * Progress with implementation * More progress * Passing intermediate test vectors * Tidy, add comments * Add DerivedKey structs * Move key derivation into own crate * Add zeroize structs * Return error for empty seed * Add tests * Tidy * Add progress * Expose PlainText struct * First commits on path derivation * Progress with implementation * More progress * Passing intermediate test vectors * Tidy, add comments * Add DerivedKey structs * Move key derivation into own crate * Add zeroize structs * Return error for empty seed * Add tests * Tidy * Add builder * Expose consts, remove Password * Minor progress * Expose SALT_SIZE * First compiling version * Add test vectors * Network crate update to stable futures * Move dbg assert statement * Port account_manager to stable futures (#1121) * Port account_manager to stable futures * Run async fns in tokio environment * Port rest_api crate to stable futures (#1118) * Port rest_api lib to stable futures * Reduce tokio features * Update notifier to stable futures * Builder update * Further updates * Add mnemonic, tidy * Convert self referential async functions * Tidy * Add testing * Add first attempt at validator_dir * Present pubkey field * stable futures fixes (#1124) * Fix eth1 update functions * Fix genesis and client * Fix beacon node lib * Return appropriate runtimes from environment * Fix test rig * Refactor eth1 service update * Upgrade simulator to stable futures * Lighthouse compiles on stable futures * Add first pass of wallet manager * Progress with CLI * Remove println debugging statement * Tidy output * Tidy 600 perms * Update libp2p service, start rpc test upgrade * Add validator creation flow * Update network crate for new libp2p * Start tidying, adding comments * Update tokio::codec to futures_codec (#1128) * Further work towards RPC corrections * Correct http timeout and network service select * Add wallet mgr testing * Shift LockedWallet into own file * Add comments to fs * Start integration into VC * Use tokio runtime for libp2p * Revert "Update tokio::codec to futures_codec (#1128)" This reverts commit e57aea924acf5cbabdcea18895ac07e38a425ed7. * Upgrade RPC libp2p tests * Upgrade secio fallback test * Add lcli keypair upgrade command * Upgrade gossipsub examples * Clean up RPC protocol * Test fixes (#1133) * Correct websocket timeout and run on os thread * Fix network test * Add --secrets-dir to VC * Remove --legacy-keys from VC * Clean up PR * Correct tokio tcp move attestation service tests * Upgrade attestation service tests * Fix sim * Correct network test * Correct genesis test * Start docs * Add progress for validator generation * Tidy error messages * Test corrections * Log info when block is received * Modify logs and update attester service events * Stable futures: fixes to vc, eth1 and account manager (#1142) * Add local testnet scripts * Remove whiteblock script * Rename local testnet script * Move spawns onto handle * Fix VC panic * Initial fix to block production issue * Tidy block producer fix * Tidy further * Add local testnet clean script * Run cargo fmt * Tidy duties service * Tidy fork service * Tidy ForkService * Tidy AttestationService * Tidy notifier * Ensure await is not suppressed in eth1 * Ensure await is not suppressed in account_manager * Use .ok() instead of .unwrap_or(()) * RPC decoding test for proto * Update discv5 and eth2-libp2p deps * Run cargo fmt * Pre-build keystores for sim * Fix lcli double runtime issue (#1144) * Handle stream termination and dialing peer errors * Correct peer_info variant types * Add progress on new deposit flow * Remove unnecessary warnings * Handle subnet unsubscription removal and improve logigng * Add logs around ping * Upgrade discv5 and improve logging * Handle peer connection status for multiple connections * Improve network service logging * Add more incomplete progress * Improve logging around peer manager * Upgrade swarm poll centralise peer management * Identify clients on error * Fix `remove_peer` in sync (#1150) * remove_peer removes from all chains * Remove logs * Fix early return from loop * Improved logging, fix panic * Partially correct tests * Add deposit command * Remove old validator directory * Start adding AM tests * Stable futures: Vc sync (#1149) * Improve syncing heuristic * Add comments * Use safer method for tolerance * Fix tests * Binary testing progress * Progress with CLI tests * Use constants for flags * More account manager testing * Improve CLI tests * Move upgrade-legacy-keypairs into account man * Use rayon for VC key generation * Add comments to `validator_dir` * Add testing to validator_dir * Add fix to eth1-sim * Check errors in eth1-sim * Fix mutability issue * Ensure password file ends in .pass * Add more tests to wallet manager * Tidy deposit * Tidy account manager * Tidy account manager * Remove panic * Generate keypairs earlier in sim * Tidy eth1-sime * Try to fix eth1 sim * Address review comments * Fix typo in CLI command * Update docs * Disable eth1 sim * Remove eth1 sim completely Co-authored-by: Age Manning <Age@AgeManning.com> Co-authored-by: pawanjay176 <pawandhananjay@gmail.com>
2020-05-18 09:01:45 +00:00
#![cfg(not(debug_assertions))]
use eth2_keystore::{Keystore, KeystoreBuilder, PlainText};
use std::fs::{self, File};
use std::path::Path;
use tempfile::{tempdir, TempDir};
use types::{test_utils::generate_deterministic_keypair, EthSpec, Keypair, MainnetEthSpec};
use validator_dir::{
Builder, ValidatorDir, ETH1_DEPOSIT_TX_HASH_FILE, VOTING_KEYSTORE_FILE,
WITHDRAWAL_KEYSTORE_FILE,
};
/// A very weak password with which to encrypt the keystores.
pub const INSECURE_PASSWORD: &[u8] = &[30; 32];
/// Helper struct for configuring tests.
struct BuildConfig {
random_voting_keystore: bool,
random_withdrawal_keystore: bool,
deposit_amount: Option<u64>,
store_withdrawal_keystore: bool,
}
impl Default for BuildConfig {
fn default() -> Self {
Self {
random_voting_keystore: true,
random_withdrawal_keystore: true,
deposit_amount: None,
store_withdrawal_keystore: true,
}
}
}
/// Check that a keystore exists and can be decrypted with a password in password_dir
fn check_keystore<P: AsRef<Path>>(path: P, password_dir: P) -> Keypair {
let mut file = File::open(path).unwrap();
let keystore = Keystore::from_json_reader(&mut file).unwrap();
let pubkey = keystore.pubkey();
let password_path = password_dir.as_ref().join(format!("0x{}", pubkey));
let password = fs::read(password_path).unwrap();
keystore.decrypt_keypair(&password).unwrap()
}
/// Creates a keystore using `generate_deterministic_keypair`.
pub fn generate_deterministic_keystore(i: usize) -> Result<(Keystore, PlainText), String> {
let keypair = generate_deterministic_keypair(i);
let keystore = KeystoreBuilder::new(&keypair, INSECURE_PASSWORD, "".into())
.map_err(|e| format!("Unable to create keystore builder: {:?}", e))?
.build()
.map_err(|e| format!("Unable to build keystore: {:?}", e))?;
Ok((keystore, INSECURE_PASSWORD.to_vec().into()))
}
/// A testing harness for generating validator directories.
struct Harness {
validators_dir: TempDir,
password_dir: TempDir,
}
impl Harness {
/// Create a new harness using temporary directories.
pub fn new() -> Self {
Self {
validators_dir: tempdir().unwrap(),
password_dir: tempdir().unwrap(),
}
}
/// Create a `ValidatorDir` from the `config`, then assert that the `ValidatorDir` was generated
/// correctly with respect to the `config`.
pub fn create_and_test(&self, config: &BuildConfig) -> ValidatorDir {
let spec = MainnetEthSpec::default_spec();
/*
* Build the `ValidatorDir`.
*/
let builder = Builder::new(
self.validators_dir.path().into(),
self.password_dir.path().into(),
)
.store_withdrawal_keystore(config.store_withdrawal_keystore);
let builder = if config.random_voting_keystore {
builder
} else {
let (keystore, password) = generate_deterministic_keystore(0).unwrap();
builder.voting_keystore(keystore, password.as_bytes())
};
let builder = if config.random_withdrawal_keystore {
builder
} else {
let (keystore, password) = generate_deterministic_keystore(1).unwrap();
builder.withdrawal_keystore(keystore, password.as_bytes())
};
let builder = if let Some(amount) = config.deposit_amount {
builder.create_eth1_tx_data(amount, &spec)
} else {
builder
};
let mut validator = builder.build().unwrap();
/*
* Assert that the dir is consistent with the config.
*/
let withdrawal_keystore_path = validator.dir().join(WITHDRAWAL_KEYSTORE_FILE);
let password_dir = self.password_dir.path().into();
// Ensure the voting keypair exists and can be decrypted.
let voting_keypair =
check_keystore(&validator.dir().join(VOTING_KEYSTORE_FILE), &password_dir);
if !config.random_voting_keystore {
assert_eq!(voting_keypair, generate_deterministic_keypair(0))
}
// Use OR here instead of AND so we *always* check for the withdrawal keystores if random
// keystores were generated.
if config.random_withdrawal_keystore || config.store_withdrawal_keystore {
// Ensure the withdrawal keypair exists and can be decrypted.
let withdrawal_keypair = check_keystore(&withdrawal_keystore_path, &password_dir);
if !config.random_withdrawal_keystore {
assert_eq!(withdrawal_keypair, generate_deterministic_keypair(1))
}
// The withdrawal keys should be distinct from the voting keypairs.
assert_ne!(withdrawal_keypair, voting_keypair);
}
if !config.store_withdrawal_keystore && !config.random_withdrawal_keystore {
assert!(!withdrawal_keystore_path.exists())
}
if let Some(amount) = config.deposit_amount {
// Check that the deposit data can be decoded.
let data = validator.eth1_deposit_data().unwrap().unwrap();
// Ensure the amount is consistent.
assert_eq!(data.deposit_data.amount, amount);
} else {
// If there was no deposit then we should return `Ok(None)`.
assert!(validator.eth1_deposit_data().unwrap().is_none());
}
let tx_hash_path = validator.dir().join(ETH1_DEPOSIT_TX_HASH_FILE);
// The eth1 deposit file should not exist, yet.
assert!(!tx_hash_path.exists());
let tx = "junk data";
// Save a tx hash.
validator.save_eth1_deposit_tx_hash(tx).unwrap();
// Ensure the saved tx hash is correct.
assert_eq!(fs::read(tx_hash_path).unwrap(), tx.as_bytes().to_vec());
// Saving a second tx hash should fail.
validator.save_eth1_deposit_tx_hash(tx).unwrap_err();
validator
}
}
#[test]
fn concurrency() {
let harness = Harness::new();
let val_dir = harness.create_and_test(&BuildConfig::default());
let path = val_dir.dir().clone();
// Should not re-open whilst opened after build.
ValidatorDir::open(&path).unwrap_err();
drop(val_dir);
// Should re-open after drop.
let val_dir = ValidatorDir::open(&path).unwrap();
// Should not re-open when opened via ValidatorDir.
ValidatorDir::open(&path).unwrap_err();
drop(val_dir);
// Should re-open again.
ValidatorDir::open(&path).unwrap();
}
#[test]
fn deterministic_voting_keystore() {
let harness = Harness::new();
let config = BuildConfig {
random_voting_keystore: false,
..BuildConfig::default()
};
harness.create_and_test(&config);
}
#[test]
fn deterministic_withdrawal_keystore_without_saving() {
let harness = Harness::new();
let config = BuildConfig {
random_withdrawal_keystore: false,
store_withdrawal_keystore: false,
..BuildConfig::default()
};
harness.create_and_test(&config);
}
#[test]
fn deterministic_withdrawal_keystore_with_saving() {
let harness = Harness::new();
let config = BuildConfig {
random_withdrawal_keystore: false,
store_withdrawal_keystore: true,
..BuildConfig::default()
};
harness.create_and_test(&config);
}
#[test]
fn both_keystores_deterministic_without_saving() {
let harness = Harness::new();
let config = BuildConfig {
random_voting_keystore: false,
random_withdrawal_keystore: false,
store_withdrawal_keystore: false,
..BuildConfig::default()
};
harness.create_and_test(&config);
}
#[test]
fn both_keystores_deterministic_with_saving() {
let harness = Harness::new();
let config = BuildConfig {
random_voting_keystore: false,
random_withdrawal_keystore: false,
store_withdrawal_keystore: true,
..BuildConfig::default()
};
harness.create_and_test(&config);
}
#[test]
fn eth1_data() {
let harness = Harness::new();
let config = BuildConfig {
deposit_amount: Some(123456),
..BuildConfig::default()
};
harness.create_and_test(&config);
}