Prepare for public testnet (#628)

* Update to spec v0.9.0 * Update to v0.9.1 * Bump spec tags for v0.9.1 * Formatting, fix CI failures * Resolve accidental KeyPair merge conflict * Document new BeaconState functions * Add `validator` changes from `validator-to-rest` * Add initial (failing) REST api tests * Fix signature parsing * Add more tests * Refactor http router * Add working tests for publish beacon block * Add validator duties tests * Move account_manager under `lighthouse` binary * Unify logfile handling in `environment` crate. * Fix incorrect cache drops in `advance_caches` * Update fork choice for v0.9.1 * Add `deposit_contract` crate * Add progress on validator onboarding * Add unfinished attesation code * Update account manager CLI * Write eth1 data file as hex string * Integrate ValidatorDirectory with validator_client * Move ValidatorDirectory into validator_client * Clean up some FIXMEs * Add beacon_chain_sim * Fix a few docs/logs * Expand `beacon_chain_sim` * Fix spec for `beacon_chain_sim * More testing for api * Start work on attestation endpoint * Reject empty attestations * Allow attestations to genesis block * Add working tests for `rest_api` validator endpoint * Remove grpc from beacon_node * Start heavy refactor of validator client - Block production is working * Prune old validator client files * Start works on attestation service * Add attestation service to validator client * Use full pubkey for validator directories * Add validator duties post endpoint * Use par_iter for keypair generation * Use bulk duties request in validator client * Add version http endpoint tests * Add interop keys and startup wait * Ensure a prompt exit * Add duties pruning * Fix compile error in beacon node tests * Add github workflow * Modify rust.yaml * Modify gitlab actions * Add to CI file * Add sudo to CI npm install * Move cargo fmt to own job in tests * Fix cargo fmt in CI * Add rustup update before cargo fmt * Change name of CI job * Make other CI jobs require cargo fmt * Add CI badge * Remove gitlab and travis files * Add different http timeout for debug * Update docker file, use makefile in CI * Use make in the dockerfile, skip the test * Use the makefile for debug GI test * Update book * Tidy grpc and misc things * Apply discv5 fixes * Address other minor issues * Fix warnings * Attempt fix for addr parsing * Tidy validator config, CLIs * Tidy comments * Tidy signing, reduce ForkService duplication * Fail if skipping too many slots * Set default recent genesis time to 0 * Add custom http timeout to validator * Fix compile bug in node_test_rig * Remove old bootstrap flag from val CLI * Update docs * Tidy val client * Change val client log levels * Add comments, more validity checks * Fix compile error, add comments * Undo changes to eth2-libp2p/src * Reduce duplication of keypair generation * Add more logging for validator duties * Fix beacon_chain_sim, nitpicks * Fix compile error, minor nits * Update to use v0.9.2 version of deposit contract * Add efforts to automate eth1 testnet deployment * Fix lcli testnet deployer * Modify bn CLI to parse eth2_testnet_dir * Progress with account_manager deposit tools * Make account manager submit deposits * Add password option for submitting deposits * Allow custom deposit amount * Add long names to lcli clap * Add password option to lcli deploy command * Fix minor bugs whilst testing * Address Michael's comments * Add refund-deposit-contract to lcli * Use time instead of skip count for denying long skips * Improve logging for eth1 * Fix bug with validator services exiting on error * Drop the block cache after genesis * Modify eth1 testnet config * Improve eth1 logging * Make validator wait until genesis time * Fix bug in eth1 voting * Add more logging to eth1 voting * Handle errors in eth1 http module * Set SECONDS_PER_DAY to sensible minimum * Shorten delay before testnet start * Ensure eth1 block is produced without any votes * Improve eth1 logging * Fix broken tests in eth1 * Tidy code in rest_api * Fix failing test in deposit_contract * Make CLI args more consistent * Change validator/duties endpoint * Add time-based skip slot limiting * Add new error type missed in previous commit * Add log when waiting for genesis * Refactor beacon node CLI * Remove unused dep * Add lcli eth1-genesis command * Fix bug in master merge * Apply clippy lints to beacon node * Add support for YamlConfig in Eth2TestnetDir * Upgrade tesnet deposit contract version * Remove unnecessary logging and correct formatting * Add a hardcoded eth2 testnet config * Ensure http server flag works. Overwrite configs with flags. * Ensure boot nodes are loaded from testnet dir * Fix account manager CLI bugs * Fix bugs with beacon node cli * Allow testnet dir without boot nodes * Write genesis state as SSZ * Remove ---/n from the start of testnet_dir files * Set default libp2p address * Tidy account manager CLI, add logging * Add check to see if testnet dir exists * Apply reviewers suggestions * Add HeadTracker struct * Add fork choice persistence * Shorten slot time for simulator * Add the /beacon/heads API endpoint * Update hardcoded testnet * Add tests for BeaconChain persistence + fix bugs * Extend BeaconChain persistence testing * Ensure chain is finalized b4 persistence tests * Ensure boot_enr.yaml is include in binary * Refactor beacon_chain_sim * Move files about in beacon sim * Update beacon_chain_sim * Fix bug with deposit inclusion * Increase log in genesis service, fix todo * Tidy sim, fix broken rest_api tests * Fix more broken tests * Update testnet * Fix broken rest api test * Tidy account manager CLI * Use tempdir for account manager * Stop hardcoded testnet dir from creating dir * Rename Eth2TestnetDir to Eth2TestnetConfig * Change hardcoded -> hard_coded * Tidy account manager * Add log to account manager * Tidy, ensure head tracker is loaded from disk * Tidy beacon chain builder * Tidy eth1_chain * Adds log support for simulator * Revert "Adds log support for simulator" This reverts commit ec77c66a052350f551db145cf20f213823428dd3. * Adds log support for simulator * Tidy after self-review * Change default log level * Address Michael's delicious PR comments * Fix off-by-one in tests
2019-12-03 15:28:57 +11:00
parent 4b4bc6247d
commit a0549e3842
74 changed files with 3417 additions and 1294 deletions
@@ -54,6 +54,21 @@ impl BlockCache {
        self.blocks.is_empty()
    }

+    /// Returns the timestamp of the earliest block in the cache (if any).
+    pub fn earliest_block_timestamp(&self) -> Option<u64> {
+        self.blocks.first().map(|block| block.timestamp)
+    }
+
+    /// Returns the timestamp of the latest block in the cache (if any).
+    pub fn latest_block_timestamp(&self) -> Option<u64> {
+        self.blocks.last().map(|block| block.timestamp)
+    }
+
+    /// Returns the lowest block number stored.
+    pub fn lowest_block_number(&self) -> Option<u64> {
+        self.blocks.first().map(|block| block.number)
+    }
+
    /// Returns the highest block number stored.
    pub fn highest_block_number(&self) -> Option<u64> {
        self.blocks.last().map(|block| block.number)
@@ -23,7 +23,7 @@ use types::Hash256;
 pub const DEPOSIT_EVENT_TOPIC: &str =
    "0x649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c5";
 /// `keccak("get_deposit_root()")[0..4]`
-pub const DEPOSIT_ROOT_FN_SIGNATURE: &str = "0x863a311b";
+pub const DEPOSIT_ROOT_FN_SIGNATURE: &str = "0xc5f2892f";
 /// `keccak("get_deposit_count()")[0..4]`
 pub const DEPOSIT_COUNT_FN_SIGNATURE: &str = "0x621fd130";

@@ -137,19 +137,21 @@ pub fn get_deposit_count(
        block_number,
        timeout,
    )
-    .and_then(|result| result.ok_or_else(|| "No response to deposit count".to_string()))
-    .and_then(|bytes| {
-        if bytes.is_empty() {
-            Ok(None)
-        } else if bytes.len() == DEPOSIT_COUNT_RESPONSE_BYTES {
-            let mut array = [0; 8];
-            array.copy_from_slice(&bytes[32 + 32..32 + 32 + 8]);
-            Ok(Some(u64::from_le_bytes(array)))
-        } else {
-            Err(format!(
-                "Deposit count response was not {} bytes: {:?}",
-                DEPOSIT_COUNT_RESPONSE_BYTES, bytes
-            ))
+    .and_then(|result| match result {
+        None => Err(format!("Deposit root response was none")),
+        Some(bytes) => {
+            if bytes.is_empty() {
+                Ok(None)
+            } else if bytes.len() == DEPOSIT_COUNT_RESPONSE_BYTES {
+                let mut array = [0; 8];
+                array.copy_from_slice(&bytes[32 + 32..32 + 32 + 8]);
+                Ok(Some(u64::from_le_bytes(array)))
+            } else {
+                Err(format!(
+                    "Deposit count response was not {} bytes: {:?}",
+                    DEPOSIT_COUNT_RESPONSE_BYTES, bytes
+                ))
+            }
        }
    })
 }
@@ -172,17 +174,19 @@ pub fn get_deposit_root(
        block_number,
        timeout,
    )
-    .and_then(|result| result.ok_or_else(|| "No response to deposit root".to_string()))
-    .and_then(|bytes| {
-        if bytes.is_empty() {
-            Ok(None)
-        } else if bytes.len() == DEPOSIT_ROOT_BYTES {
-            Ok(Some(Hash256::from_slice(&bytes)))
-        } else {
-            Err(format!(
-                "Deposit root response was not {} bytes: {:?}",
-                DEPOSIT_ROOT_BYTES, bytes
-            ))
+    .and_then(|result| match result {
+        None => Err(format!("Deposit root response was none")),
+        Some(bytes) => {
+            if bytes.is_empty() {
+                Ok(None)
+            } else if bytes.len() == DEPOSIT_ROOT_BYTES {
+                Ok(Some(Hash256::from_slice(&bytes)))
+            } else {
+                Err(format!(
+                    "Deposit root response was not {} bytes: {:?}",
+                    DEPOSIT_ROOT_BYTES, bytes
+                ))
+            }
        }
    })
 }
@@ -369,12 +373,18 @@ pub fn send_rpc_request(

 /// Accepts an entire HTTP body (as a string) and returns the `result` field, as a serde `Value`.
 fn response_result(response: &str) -> Result<Option<Value>, String> {
-    Ok(serde_json::from_str::<Value>(&response)
-        .map_err(|e| format!("Failed to parse response: {:?}", e))?
-        .get("result")
-        .cloned()
-        .map(Some)
-        .unwrap_or_else(|| None))
+    let json = serde_json::from_str::<Value>(&response)
+        .map_err(|e| format!("Failed to parse response: {:?}", e))?;
+
+    if let Some(error) = json.get("error") {
+        Err(format!("Eth1 node returned error: {}", error))
+    } else {
+        Ok(json
+            .get("result")
+            .cloned()
+            .map(Some)
+            .unwrap_or_else(|| None))
+    }
 }

 /// Parses a `0x`-prefixed, **big-endian** hex string as a u64.
@@ -121,7 +121,7 @@ impl Default for Config {
            lowest_cached_block_number: 0,
            follow_distance: 128,
            block_cache_truncation: Some(4_096),
-            auto_update_interval_millis: 500,
+            auto_update_interval_millis: 7_000,
            blocks_per_log_query: 1_000,
            max_log_requests_per_update: None,
            max_blocks_per_update: None,
@@ -163,6 +163,26 @@ impl Service {
        &self.inner.deposit_cache
    }

+    /// Drop the block cache, replacing it with an empty one.
+    pub fn drop_block_cache(&self) {
+        *(self.inner.block_cache.write()) = BlockCache::default();
+    }
+
+    /// Returns the timestamp of the earliest block in the cache (if any).
+    pub fn earliest_block_timestamp(&self) -> Option<u64> {
+        self.inner.block_cache.read().earliest_block_timestamp()
+    }
+
+    /// Returns the timestamp of the latest block in the cache (if any).
+    pub fn latest_block_timestamp(&self) -> Option<u64> {
+        self.inner.block_cache.read().latest_block_timestamp()
+    }
+
+    /// Returns the lowest block number stored.
+    pub fn lowest_block_number(&self) -> Option<u64> {
+        self.inner.block_cache.read().lowest_block_number()
+    }
+
    /// Returns the number of currently cached blocks.
    pub fn block_cache_len(&self) -> usize {
        self.blocks().read().len()
@@ -220,6 +240,8 @@ impl Service {
    {
        let log_a = self.log.clone();
        let log_b = self.log.clone();
+        let inner_1 = self.inner.clone();
+        let inner_2 = self.inner.clone();

        let deposit_future = self
            .update_deposit_cache()
@@ -229,6 +251,7 @@ impl Service {
                    Ok(DepositCacheUpdateOutcome::Success { logs_imported }) => trace!(
                        log_a,
                        "Updated eth1 deposit cache";
+                        "cached_deposits" => inner_1.deposit_cache.read().cache.len(),
                        "logs_imported" => logs_imported,
                    ),
                    Err(e) => error!(
@@ -252,6 +275,7 @@ impl Service {
                    }) => trace!(
                        log_b,
                        "Updated eth1 block cache";
+                        "cached_blocks" => inner_2.block_cache.read().len(),
                        "blocks_imported" => blocks_imported,
                        "head_block" => head_block_number,
                    ),