Prepare for public testnet (#628)

* Update to spec v0.9.0 * Update to v0.9.1 * Bump spec tags for v0.9.1 * Formatting, fix CI failures * Resolve accidental KeyPair merge conflict * Document new BeaconState functions * Add `validator` changes from `validator-to-rest` * Add initial (failing) REST api tests * Fix signature parsing * Add more tests * Refactor http router * Add working tests for publish beacon block * Add validator duties tests * Move account_manager under `lighthouse` binary * Unify logfile handling in `environment` crate. * Fix incorrect cache drops in `advance_caches` * Update fork choice for v0.9.1 * Add `deposit_contract` crate * Add progress on validator onboarding * Add unfinished attesation code * Update account manager CLI * Write eth1 data file as hex string * Integrate ValidatorDirectory with validator_client * Move ValidatorDirectory into validator_client * Clean up some FIXMEs * Add beacon_chain_sim * Fix a few docs/logs * Expand `beacon_chain_sim` * Fix spec for `beacon_chain_sim * More testing for api * Start work on attestation endpoint * Reject empty attestations * Allow attestations to genesis block * Add working tests for `rest_api` validator endpoint * Remove grpc from beacon_node * Start heavy refactor of validator client - Block production is working * Prune old validator client files * Start works on attestation service * Add attestation service to validator client * Use full pubkey for validator directories * Add validator duties post endpoint * Use par_iter for keypair generation * Use bulk duties request in validator client * Add version http endpoint tests * Add interop keys and startup wait * Ensure a prompt exit * Add duties pruning * Fix compile error in beacon node tests * Add github workflow * Modify rust.yaml * Modify gitlab actions * Add to CI file * Add sudo to CI npm install * Move cargo fmt to own job in tests * Fix cargo fmt in CI * Add rustup update before cargo fmt * Change name of CI job * Make other CI jobs require cargo fmt * Add CI badge * Remove gitlab and travis files * Add different http timeout for debug * Update docker file, use makefile in CI * Use make in the dockerfile, skip the test * Use the makefile for debug GI test * Update book * Tidy grpc and misc things * Apply discv5 fixes * Address other minor issues * Fix warnings * Attempt fix for addr parsing * Tidy validator config, CLIs * Tidy comments * Tidy signing, reduce ForkService duplication * Fail if skipping too many slots * Set default recent genesis time to 0 * Add custom http timeout to validator * Fix compile bug in node_test_rig * Remove old bootstrap flag from val CLI * Update docs * Tidy val client * Change val client log levels * Add comments, more validity checks * Fix compile error, add comments * Undo changes to eth2-libp2p/src * Reduce duplication of keypair generation * Add more logging for validator duties * Fix beacon_chain_sim, nitpicks * Fix compile error, minor nits * Update to use v0.9.2 version of deposit contract * Add efforts to automate eth1 testnet deployment * Fix lcli testnet deployer * Modify bn CLI to parse eth2_testnet_dir * Progress with account_manager deposit tools * Make account manager submit deposits * Add password option for submitting deposits * Allow custom deposit amount * Add long names to lcli clap * Add password option to lcli deploy command * Fix minor bugs whilst testing * Address Michael's comments * Add refund-deposit-contract to lcli * Use time instead of skip count for denying long skips * Improve logging for eth1 * Fix bug with validator services exiting on error * Drop the block cache after genesis * Modify eth1 testnet config * Improve eth1 logging * Make validator wait until genesis time * Fix bug in eth1 voting * Add more logging to eth1 voting * Handle errors in eth1 http module * Set SECONDS_PER_DAY to sensible minimum * Shorten delay before testnet start * Ensure eth1 block is produced without any votes * Improve eth1 logging * Fix broken tests in eth1 * Tidy code in rest_api * Fix failing test in deposit_contract * Make CLI args more consistent * Change validator/duties endpoint * Add time-based skip slot limiting * Add new error type missed in previous commit * Add log when waiting for genesis * Refactor beacon node CLI * Remove unused dep * Add lcli eth1-genesis command * Fix bug in master merge * Apply clippy lints to beacon node * Add support for YamlConfig in Eth2TestnetDir * Upgrade tesnet deposit contract version * Remove unnecessary logging and correct formatting * Add a hardcoded eth2 testnet config * Ensure http server flag works. Overwrite configs with flags. * Ensure boot nodes are loaded from testnet dir * Fix account manager CLI bugs * Fix bugs with beacon node cli * Allow testnet dir without boot nodes * Write genesis state as SSZ * Remove ---/n from the start of testnet_dir files * Set default libp2p address * Tidy account manager CLI, add logging * Add check to see if testnet dir exists * Apply reviewers suggestions * Add HeadTracker struct * Add fork choice persistence * Shorten slot time for simulator * Add the /beacon/heads API endpoint * Update hardcoded testnet * Add tests for BeaconChain persistence + fix bugs * Extend BeaconChain persistence testing * Ensure chain is finalized b4 persistence tests * Ensure boot_enr.yaml is include in binary * Refactor beacon_chain_sim * Move files about in beacon sim * Update beacon_chain_sim * Fix bug with deposit inclusion * Increase log in genesis service, fix todo * Tidy sim, fix broken rest_api tests * Fix more broken tests * Update testnet * Fix broken rest api test * Tidy account manager CLI * Use tempdir for account manager * Stop hardcoded testnet dir from creating dir * Rename Eth2TestnetDir to Eth2TestnetConfig * Change hardcoded -> hard_coded * Tidy account manager * Add log to account manager * Tidy, ensure head tracker is loaded from disk * Tidy beacon chain builder * Tidy eth1_chain * Adds log support for simulator * Revert "Adds log support for simulator" This reverts commit ec77c66a052350f551db145cf20f213823428dd3. * Adds log support for simulator * Tidy after self-review * Change default log level * Address Michael's delicious PR comments * Fix off-by-one in tests
2019-12-03 15:28:57 +11:00
parent 4b4bc6247d
commit a0549e3842
74 changed files with 3417 additions and 1294 deletions
@@ -7,17 +7,17 @@
 //! some initial issues.
 mod ganache;

-use deposit_contract::{eth1_tx_data, ABI, BYTECODE, CONTRACT_DEPLOY_GAS, DEPOSIT_GAS};
-use futures::{stream, Future, IntoFuture, Stream};
+use deposit_contract::{eth1_tx_data, testnet, ABI, BYTECODE, CONTRACT_DEPLOY_GAS, DEPOSIT_GAS};
+use futures::{future, stream, Future, IntoFuture, Stream};
 use ganache::GanacheInstance;
 use std::time::{Duration, Instant};
 use tokio::{runtime::Runtime, timer::Delay};
 use types::DepositData;
-use types::{EthSpec, Hash256, Keypair, Signature};
+use types::{test_utils::generate_deterministic_keypair, EthSpec, Hash256, Keypair, Signature};
 use web3::contract::{Contract, Options};
 use web3::transports::Http;
 use web3::types::{Address, TransactionRequest, U256};
-use web3::{Transport, Web3};
+use web3::Web3;

 pub const DEPLOYER_ACCOUNTS_INDEX: usize = 0;
 pub const DEPOSIT_ACCOUNTS_INDEX: usize = 0;
@@ -31,7 +31,7 @@ pub struct GanacheEth1Instance {
 impl GanacheEth1Instance {
    pub fn new() -> impl Future<Item = Self, Error = String> {
        GanacheInstance::new().into_future().and_then(|ganache| {
-            DepositContract::deploy(ganache.web3.clone(), 0).map(|deposit_contract| Self {
+            DepositContract::deploy(ganache.web3.clone(), 0, None).map(|deposit_contract| Self {
                ganache,
                deposit_contract,
            })
@@ -58,21 +58,52 @@ impl DepositContract {
    pub fn deploy(
        web3: Web3<Http>,
        confirmations: usize,
+        password: Option<String>,
+    ) -> impl Future<Item = Self, Error = String> {
+        Self::deploy_bytecode(web3, confirmations, BYTECODE, ABI, password)
+    }
+
+    pub fn deploy_testnet(
+        web3: Web3<Http>,
+        confirmations: usize,
+        password: Option<String>,
+    ) -> impl Future<Item = Self, Error = String> {
+        Self::deploy_bytecode(
+            web3,
+            confirmations,
+            testnet::BYTECODE,
+            testnet::ABI,
+            password,
+        )
+    }
+
+    fn deploy_bytecode(
+        web3: Web3<Http>,
+        confirmations: usize,
+        bytecode: &[u8],
+        abi: &[u8],
+        password: Option<String>,
    ) -> impl Future<Item = Self, Error = String> {
        let web3_1 = web3.clone();

-        deploy_deposit_contract(web3.clone(), confirmations)
-            .map_err(|e| {
-                format!(
-                    "Failed to deploy contract: {}. Is scripts/ganache_tests_node.sh running?.",
-                    e
-                )
-            })
-            .and_then(move |address| {
-                Contract::from_json(web3_1.eth(), address, ABI)
-                    .map_err(|e| format!("Failed to init contract: {:?}", e))
-            })
-            .map(|contract| Self { contract, web3 })
+        deploy_deposit_contract(
+            web3.clone(),
+            confirmations,
+            bytecode.to_vec(),
+            abi.to_vec(),
+            password,
+        )
+        .map_err(|e| {
+            format!(
+                "Failed to deploy contract: {}. Is scripts/ganache_tests_node.sh running?.",
+                e
+            )
+        })
+        .and_then(move |address| {
+            Contract::from_json(web3_1.eth(), address, ABI)
+                .map_err(|e| format!("Failed to init contract: {:?}", e))
+        })
+        .map(|contract| Self { contract, web3 })
    }

    /// The deposit contract's address in `0x00ab...` format.
@@ -125,6 +156,25 @@ impl DepositContract {
            .map_err(|e| format!("Deposit failed: {:?}", e))
    }

+    pub fn deposit_deterministic_async<E: EthSpec>(
+        &self,
+        keypair_index: usize,
+        amount: u64,
+    ) -> impl Future<Item = (), Error = String> {
+        let keypair = generate_deterministic_keypair(keypair_index);
+
+        let mut deposit = DepositData {
+            pubkey: keypair.pk.into(),
+            withdrawal_credentials: Hash256::zero(),
+            amount,
+            signature: Signature::empty_signature().into(),
+        };
+
+        deposit.signature = deposit.create_signature(&keypair.sk, &E::default_spec());
+
+        self.deposit_async(deposit)
+    }
+
    /// Performs a non-blocking deposit.
    pub fn deposit_async(
        &self,
@@ -208,11 +258,15 @@ fn from_gwei(gwei: u64) -> U256 {

 /// Deploys the deposit contract to the given web3 instance using the account with index
 /// `DEPLOYER_ACCOUNTS_INDEX`.
-fn deploy_deposit_contract<T: Transport>(
-    web3: Web3<T>,
+fn deploy_deposit_contract(
+    web3: Web3<Http>,
    confirmations: usize,
+    bytecode: Vec<u8>,
+    abi: Vec<u8>,
+    password_opt: Option<String>,
 ) -> impl Future<Item = Address, Error = String> {
-    let bytecode = String::from_utf8_lossy(&BYTECODE);
+    let bytecode = String::from_utf8(bytecode).expect("bytecode must be valid utf8");
+    let web3_1 = web3.clone();

    web3.eth()
        .accounts()
@@ -223,8 +277,30 @@ fn deploy_deposit_contract<T: Transport>(
                .cloned()
                .ok_or_else(|| "Insufficient accounts for deployer".to_string())
        })
+        .and_then(move |from_address| {
+            let future: Box<dyn Future<Item = Address, Error = String> + Send> =
+                if let Some(password) = password_opt {
+                    // Unlock for only a single transaction.
+                    let duration = None;
+
+                    let future = web3_1
+                        .personal()
+                        .unlock_account(from_address, &password, duration)
+                        .then(move |result| match result {
+                            Ok(true) => Ok(from_address),
+                            Ok(false) => Err("Eth1 node refused to unlock account".to_string()),
+                            Err(e) => Err(format!("Eth1 unlock request failed: {:?}", e)),
+                        });
+
+                    Box::new(future)
+                } else {
+                    Box::new(future::ok(from_address))
+                };
+
+            future
+        })
        .and_then(move |deploy_address| {
-            Contract::deploy(web3.eth(), &ABI)
+            Contract::deploy(web3.eth(), &abi)
                .map_err(|e| format!("Unable to build contract deployer: {:?}", e))?
                .confirmations(confirmations)
                .options(Options {