lighthouse/lcli/src/main.rs
Paul Hauner a0549e3842
Prepare for public testnet (#628)
* Update to spec v0.9.0

* Update to v0.9.1

* Bump spec tags for v0.9.1

* Formatting, fix CI failures

* Resolve accidental KeyPair merge conflict

* Document new BeaconState functions

* Add `validator` changes from `validator-to-rest`

* Add initial (failing) REST api tests

* Fix signature parsing

* Add more tests

* Refactor http router

* Add working tests for publish beacon block

* Add validator duties tests

* Move account_manager under `lighthouse` binary

* Unify logfile handling in `environment` crate.

* Fix incorrect cache drops in `advance_caches`

* Update fork choice for v0.9.1

* Add `deposit_contract` crate

* Add progress on validator onboarding

* Add unfinished attesation code

* Update account manager CLI

* Write eth1 data file as hex string

* Integrate ValidatorDirectory with validator_client

* Move ValidatorDirectory into validator_client

* Clean up some FIXMEs

* Add beacon_chain_sim

* Fix a few docs/logs

* Expand `beacon_chain_sim`

* Fix spec for `beacon_chain_sim

* More testing for api

* Start work on attestation endpoint

* Reject empty attestations

* Allow attestations to genesis block

* Add working tests for `rest_api` validator endpoint

* Remove grpc from beacon_node

* Start heavy refactor of validator client

- Block production is working

* Prune old validator client files

* Start works on attestation service

* Add attestation service to validator client

* Use full pubkey for validator directories

* Add validator duties post endpoint

* Use par_iter for keypair generation

* Use bulk duties request in validator client

* Add version http endpoint tests

* Add interop keys and startup wait

* Ensure a prompt exit

* Add duties pruning

* Fix compile error in beacon node tests

* Add github workflow

* Modify rust.yaml

* Modify gitlab actions

* Add to CI file

* Add sudo to CI npm install

* Move cargo fmt to own job in tests

* Fix cargo fmt in CI

* Add rustup update before cargo fmt

* Change name of CI job

* Make other CI jobs require cargo fmt

* Add CI badge

* Remove gitlab and travis files

* Add different http timeout for debug

* Update docker file, use makefile in CI

* Use make in the dockerfile, skip the test

* Use the makefile for debug GI test

* Update book

* Tidy grpc and misc things

* Apply discv5 fixes

* Address other minor issues

* Fix warnings

* Attempt fix for addr parsing

* Tidy validator config, CLIs

* Tidy comments

* Tidy signing, reduce ForkService duplication

* Fail if skipping too many slots

* Set default recent genesis time to 0

* Add custom http timeout to validator

* Fix compile bug in node_test_rig

* Remove old bootstrap flag from val CLI

* Update docs

* Tidy val client

* Change val client log levels

* Add comments, more validity checks

* Fix compile error, add comments

* Undo changes to eth2-libp2p/src

* Reduce duplication of keypair generation

* Add more logging for validator duties

* Fix beacon_chain_sim, nitpicks

* Fix compile error, minor nits

* Update to use v0.9.2 version of deposit contract

* Add efforts to automate eth1 testnet deployment

* Fix lcli testnet deployer

* Modify bn CLI to parse eth2_testnet_dir

* Progress with account_manager deposit tools

* Make account manager submit deposits

* Add password option for submitting deposits

* Allow custom deposit amount

* Add long names to lcli clap

* Add password option to lcli deploy command

* Fix minor bugs whilst testing

* Address Michael's comments

* Add refund-deposit-contract to lcli

* Use time instead of skip count for denying long skips

* Improve logging for eth1

* Fix bug with validator services exiting on error

* Drop the block cache after genesis

* Modify eth1 testnet config

* Improve eth1 logging

* Make validator wait until genesis time

* Fix bug in eth1 voting

* Add more logging to eth1 voting

* Handle errors in eth1 http module

* Set SECONDS_PER_DAY to sensible minimum

* Shorten delay before testnet start

* Ensure eth1 block is produced without any votes

* Improve eth1 logging

* Fix broken tests in eth1

* Tidy code in rest_api

* Fix failing test in deposit_contract

* Make CLI args more consistent

* Change validator/duties endpoint

* Add time-based skip slot limiting

* Add new error type missed in previous commit

* Add log when waiting for genesis

* Refactor beacon node CLI

* Remove unused dep

* Add lcli eth1-genesis command

* Fix bug in master merge

* Apply clippy lints to beacon node

* Add support for YamlConfig in Eth2TestnetDir

* Upgrade tesnet deposit contract version

* Remove unnecessary logging and correct formatting

* Add a hardcoded eth2 testnet config

* Ensure http server flag works. Overwrite configs with flags.

* Ensure boot nodes are loaded from testnet dir

* Fix account manager CLI bugs

* Fix bugs with beacon node cli

* Allow testnet dir without boot nodes

* Write genesis state as SSZ

* Remove ---/n from the start of testnet_dir files

* Set default libp2p address

* Tidy account manager CLI, add logging

* Add check to see if testnet dir exists

* Apply reviewers suggestions

* Add HeadTracker struct

* Add fork choice persistence

* Shorten slot time for simulator

* Add the /beacon/heads API endpoint

* Update hardcoded testnet

* Add tests for BeaconChain persistence + fix bugs

* Extend BeaconChain persistence testing

* Ensure chain is finalized b4 persistence tests

* Ensure boot_enr.yaml is include in binary

* Refactor beacon_chain_sim

* Move files about in beacon sim

* Update beacon_chain_sim

* Fix bug with deposit inclusion

* Increase log in genesis service, fix todo

* Tidy sim, fix broken rest_api tests

* Fix more broken tests

* Update testnet

* Fix broken rest api test

* Tidy account manager CLI

* Use tempdir for account manager

* Stop hardcoded testnet dir from creating dir

* Rename Eth2TestnetDir to Eth2TestnetConfig

* Change hardcoded -> hard_coded

* Tidy account manager

* Add log to account manager

* Tidy, ensure head tracker is loaded from disk

* Tidy beacon chain builder

* Tidy eth1_chain

* Adds log support for simulator

* Revert "Adds log support for simulator"

This reverts commit ec77c66a052350f551db145cf20f213823428dd3.

* Adds log support for simulator

* Tidy after self-review

* Change default log level

* Address Michael's delicious PR comments

* Fix off-by-one in tests
2019-12-03 15:28:57 +11:00

325 lines
13 KiB
Rust

#[macro_use]
extern crate log;
mod deploy_deposit_contract;
mod eth1_genesis;
mod parse_hex;
mod pycli;
mod refund_deposit_contract;
mod transition_blocks;
use clap::{App, Arg, SubCommand};
use environment::EnvironmentBuilder;
use log::Level;
use parse_hex::run_parse_hex;
use pycli::run_pycli;
use std::fs::File;
use std::path::PathBuf;
use std::time::{SystemTime, UNIX_EPOCH};
use transition_blocks::run_transition_blocks;
use types::{test_utils::TestingBeaconStateBuilder, EthSpec, MainnetEthSpec, MinimalEthSpec};
type LocalEthSpec = MinimalEthSpec;
fn main() {
simple_logger::init_with_level(Level::Info).expect("logger should initialize");
let matches = App::new("Lighthouse CLI Tool")
.about(
"Performs various testing-related tasks, modelled after zcli. \
by @protolambda.",
)
.subcommand(
SubCommand::with_name("genesis_yaml")
.about("Generates a genesis YAML file")
.arg(
Arg::with_name("num_validators")
.short("n")
.value_name("INTEGER")
.takes_value(true)
.required(true)
.help("Number of initial validators."),
)
.arg(
Arg::with_name("genesis_time")
.short("g")
.value_name("INTEGER")
.takes_value(true)
.required(false)
.help("Eth2 genesis time (seconds since UNIX epoch)."),
)
.arg(
Arg::with_name("spec")
.short("s")
.value_name("STRING")
.takes_value(true)
.required(true)
.possible_values(&["minimal", "mainnet"])
.default_value("minimal")
.help("Eth2 genesis time (seconds since UNIX epoch)."),
)
.arg(
Arg::with_name("output_file")
.short("f")
.value_name("PATH")
.takes_value(true)
.default_value("./genesis_state.yaml")
.help("Output file for generated state."),
),
)
.subcommand(
SubCommand::with_name("transition-blocks")
.about("Performs a state transition given a pre-state and block")
.arg(
Arg::with_name("pre-state")
.value_name("BEACON_STATE")
.takes_value(true)
.required(true)
.help("Path to a SSZ file of the pre-state."),
)
.arg(
Arg::with_name("block")
.value_name("BEACON_BLOCK")
.takes_value(true)
.required(true)
.help("Path to a SSZ file of the block to apply to pre-state."),
)
.arg(
Arg::with_name("output")
.value_name("SSZ_FILE")
.takes_value(true)
.required(true)
.default_value("./output.ssz")
.help("Path to output a SSZ file."),
),
)
.subcommand(
SubCommand::with_name("pretty-hex")
.about("Parses SSZ encoded as ASCII 0x-prefixed hex")
.arg(
Arg::with_name("type")
.value_name("TYPE")
.takes_value(true)
.required(true)
.possible_values(&["block"])
.help("The schema of the supplied SSZ."),
)
.arg(
Arg::with_name("hex_ssz")
.value_name("HEX")
.takes_value(true)
.required(true)
.help("SSZ encoded as 0x-prefixed hex"),
),
)
.subcommand(
SubCommand::with_name("deploy-deposit-contract")
.about(
"Deploy an eth1 deposit contract and create a ~/.lighthouse/testnet directory \
(unless another directory is specified).",
)
.arg(
Arg::with_name("output")
.short("o")
.long("output")
.value_name("PATH")
.takes_value(true)
.help("The output directory. Defaults to ~/.lighthouse/testnet"),
)
.arg(
Arg::with_name("min-genesis-time")
.short("t")
.long("min-genesis-time")
.value_name("UNIX_EPOCH_SECONDS")
.takes_value(true)
.default_value("0")
.help("The MIN_GENESIS_TIME constant."),
)
.arg(
Arg::with_name("eth1-endpoint")
.short("e")
.long("eth1-endpoint")
.value_name("HTTP_SERVER")
.takes_value(true)
.default_value("http://localhost:8545")
.help("The URL to the eth1 JSON-RPC http API."),
)
.arg(
Arg::with_name("confirmations")
.value_name("INTEGER")
.long("confirmations")
.takes_value(true)
.default_value("3")
.help("The number of block confirmations before declaring the contract deployed."),
)
.arg(
Arg::with_name("password")
.long("password")
.value_name("FILE")
.takes_value(true)
.help("The password file to unlock the eth1 account (see --index)"),
)
)
.subcommand(
SubCommand::with_name("refund-deposit-contract")
.about(
"Calls the steal() function on a testnet eth1 contract.",
)
.arg(
Arg::with_name("testnet-dir")
.short("d")
.long("testnet-dir")
.value_name("PATH")
.takes_value(true)
.help("The testnet dir. Defaults to ~/.lighthouse/testnet"),
)
.arg(
Arg::with_name("eth1-endpoint")
.short("e")
.long("eth1-endpoint")
.value_name("HTTP_SERVER")
.takes_value(true)
.default_value("http://localhost:8545")
.help("The URL to the eth1 JSON-RPC http API."),
)
.arg(
Arg::with_name("password")
.long("password")
.value_name("FILE")
.takes_value(true)
.help("The password file to unlock the eth1 account (see --index)"),
)
.arg(
Arg::with_name("account-index")
.short("i")
.long("account-index")
.value_name("INDEX")
.takes_value(true)
.default_value("0")
.help("The eth1 accounts[] index which will send the transaction"),
)
)
.subcommand(
SubCommand::with_name("eth1-genesis")
.about(
"Listens to the eth1 chain and finds the genesis beacon state",
)
.arg(
Arg::with_name("testnet-dir")
.short("d")
.long("testnet-dir")
.value_name("PATH")
.takes_value(true)
.help("The testnet dir. Defaults to ~/.lighthouse/testnet"),
)
.arg(
Arg::with_name("eth1-endpoint")
.short("e")
.long("eth1-endpoint")
.value_name("HTTP_SERVER")
.takes_value(true)
.default_value("http://localhost:8545")
.help("The URL to the eth1 JSON-RPC http API."),
)
)
.subcommand(
SubCommand::with_name("pycli")
.about("TODO")
.arg(
Arg::with_name("pycli-path")
.long("pycli-path")
.short("p")
.value_name("PATH")
.takes_value(true)
.default_value("../../pycli")
.help("Path to the pycli repository."),
),
)
.get_matches();
let env = EnvironmentBuilder::minimal()
.multi_threaded_tokio_runtime()
.expect("should start tokio runtime")
.async_logger("trace")
.expect("should start null logger")
.build()
.expect("should build env");
match matches.subcommand() {
("genesis_yaml", Some(matches)) => {
let num_validators = matches
.value_of("num_validators")
.expect("slog requires num_validators")
.parse::<usize>()
.expect("num_validators must be a valid integer");
let genesis_time = if let Some(string) = matches.value_of("genesis_time") {
string
.parse::<u64>()
.expect("genesis_time must be a valid integer")
} else {
warn!("No genesis time supplied via CLI, using the current time.");
SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("should obtain time since unix epoch")
.as_secs()
};
let file = matches
.value_of("output_file")
.expect("slog requires output file")
.parse::<PathBuf>()
.expect("output_file must be a valid path");
info!(
"Creating genesis state with {} validators and genesis time {}.",
num_validators, genesis_time
);
match matches.value_of("spec").expect("spec is required by slog") {
"minimal" => genesis_yaml::<MinimalEthSpec>(num_validators, genesis_time, file),
"mainnet" => genesis_yaml::<MainnetEthSpec>(num_validators, genesis_time, file),
_ => unreachable!("guarded by slog possible_values"),
};
info!("Genesis state YAML file created. Exiting successfully.");
}
("transition-blocks", Some(matches)) => run_transition_blocks(matches)
.unwrap_or_else(|e| error!("Failed to transition blocks: {}", e)),
("pretty-hex", Some(matches)) => {
run_parse_hex(matches).unwrap_or_else(|e| error!("Failed to pretty print hex: {}", e))
}
("pycli", Some(matches)) => run_pycli::<LocalEthSpec>(matches)
.unwrap_or_else(|e| error!("Failed to run pycli: {}", e)),
("deploy-deposit-contract", Some(matches)) => {
deploy_deposit_contract::run::<LocalEthSpec>(env, matches)
.unwrap_or_else(|e| error!("Failed to run deploy-deposit-contract command: {}", e))
}
("refund-deposit-contract", Some(matches)) => {
refund_deposit_contract::run::<LocalEthSpec>(env, matches)
.unwrap_or_else(|e| error!("Failed to run refund-deposit-contract command: {}", e))
}
("eth1-genesis", Some(matches)) => eth1_genesis::run::<LocalEthSpec>(env, matches)
.unwrap_or_else(|e| error!("Failed to run eth1-genesis command: {}", e)),
(other, _) => error!("Unknown subcommand {}. See --help.", other),
}
}
/// Creates a genesis state and writes it to a YAML file.
fn genesis_yaml<T: EthSpec>(validator_count: usize, genesis_time: u64, output: PathBuf) {
let spec = &T::default_spec();
let builder: TestingBeaconStateBuilder<T> =
TestingBeaconStateBuilder::from_default_keypairs_file_if_exists(validator_count, spec);
let (mut state, _keypairs) = builder.build();
state.genesis_time = genesis_time;
info!("Generated state root: {:?}", state.canonical_root());
info!("Writing genesis state to {:?}", output);
let file = File::create(output.clone())
.unwrap_or_else(|e| panic!("unable to create file: {:?}. Error: {:?}", output, e));
serde_yaml::to_writer(file, &state).expect("should be able to serialize BeaconState");
}