Commit Graph

977 Commits

Author SHA1 Message Date
Paul Hauner
b7e43b56f9
Fix minor PR comments 2019-08-13 19:37:14 +10:00
Paul Hauner
e410e89898
Merge branch 'block-processing-times' of github.com:sigp/lighthouse into block-processing-times 2019-08-13 18:26:07 +10:00
Paul Hauner
341a83b9e8
Tidy attestation metrics 2019-08-13 16:17:11 +10:00
Paul Hauner
a3e464078a
Fix store block read metrics 2019-08-13 13:00:01 +10:00
Paul Hauner
b076b07022
Add more metrics, tidy existing metrics 2019-08-13 12:11:18 +10:00
Paul Hauner
e369e293a5
Make prom metric names more consistent 2019-08-13 09:20:39 +10:00
Paul Hauner
24b2f83713
Fix wrong state given to op pool prune 2019-08-13 07:35:52 +10:00
Age Manning
d25ec359c8
Merge attestation processing 2019-08-12 22:25:25 +10:00
Age Manning
af5372fde4
Merge latest master 2019-08-12 22:24:20 +10:00
Age Manning
5d4d2f35e1
Initial sync re-write. WIP 2019-08-12 22:07:59 +10:00
Paul Hauner
d5d60874e5
Remove http args 2019-08-12 18:20:05 +10:00
Paul Hauner
7165598b7f
Add lighthouse_metrics gather fn 2019-08-12 18:19:50 +10:00
Paul Hauner
d7c546844c
Fix broken beacon chain metrics, add slot clock metrics 2019-08-12 17:44:47 +10:00
Paul Hauner
95a320817e
Update metrics names to be more like standard 2019-08-12 15:42:23 +10:00
Paul Hauner
6a1e5f6d26
Remove old http_server crate 2019-08-12 15:19:39 +10:00
Paul Hauner
af334b2cf0
Add metrics to HTTP server 2019-08-12 14:30:46 +10:00
Paul Hauner
cac0e5c832
Add basic metrics to libp2p 2019-08-12 14:16:20 +10:00
Paul Hauner
0b4a8893a4
Add more store metrics 2019-08-12 13:49:09 +10:00
Paul Hauner
913ee4694e
Prefix BeaconChain metrics with "beacon_" 2019-08-12 13:35:16 +10:00
Paul Hauner
7140dbc45d
Add extra prom beacon chain metrics 2019-08-12 13:26:58 +10:00
Paul Hauner
6150f0ae1a
Prune op pool on finalization 2019-08-11 18:29:11 +10:00
Paul Hauner
78db947e6e
Add beacon chain persistence metric 2019-08-11 18:28:57 +10:00
Paul Hauner
42d300bdc3
Add more beacon chain metrics 2019-08-11 17:49:32 +10:00
Paul Hauner
76f42ac7ff
Remove hypen from prometheus metric name 2019-08-11 16:15:26 +10:00
Paul Hauner
441eb41b6b
Add heavy metering to block processing 2019-08-11 15:53:34 +10:00
Paul Hauner
2108895fca
Unify common metric fns into a crate 2019-08-11 15:34:10 +10:00
Paul Hauner
36ff115b04
Use global prometheus gather at rest api 2019-08-11 14:46:20 +10:00
Paul Hauner
e33d0703ef
Make metrics not panic if already defined 2019-08-11 14:43:31 +10:00
Paul Hauner
9995b390b5
Move beacon_chain to new metrics structure. 2019-08-11 14:11:13 +10:00
Paul Hauner
48733917be
Begin metrics refactor 2019-08-11 12:12:19 +10:00
Paul Hauner
4020d13064
Reject any attestation prior to finalization 2019-08-11 09:34:49 +10:00
Paul Hauner
6c9ebf4b96
Fix compile error in store iters 2019-08-11 09:15:39 +10:00
Paul Hauner
cd11eb15a5
Merge branch 'master' into attestation-processing 2019-08-11 09:08:26 +10:00
Paul Hauner
04bef689e3
Fix attestation prod. target roots change 2019-08-10 17:47:34 +10:00
Paul Hauner
963fb7bc87
Tidy comments 2019-08-10 17:36:53 +10:00
Paul Hauner
1beab66078
Remove unused method 2019-08-10 17:23:45 +10:00
Paul Hauner
989e2727d7
Changes to rest_api (#480)
* Add half-finished rest api changes

* Add basic, messy changes to rest api

* Fix expect() in ApiRequest

* Remove ApiRequest, add route for beacon state

* Tidy rest api, add get state from root

* Add api method for getting state roots by slot

* Add test for URL helper

* Simplify state_at_slot fn

* Add tests for rest api helper fns

* Add extra tests for parse root

* Fix clippy lints

* Fix compile error in rest api

* Update test to new ethereum-types
2019-08-10 17:15:15 +10:00
Age Manning
468015f9bb Initial Interop Updates (#492)
* Add interop chain spec and rename chain_id

* Add ability to connect to raw libp2p nodes

* Adds Identify protocol, clean up RPC protocol name handling

* Update to latest libp2p, gossipsub improvements

* Updates to latest interop branch.

- Shifts decoding of objects into message handler.
- Updates to latest interop gossipsub.
- Adds interop spec constant.

* Configuration updates allow for verbosity CLI flag and spec constants

* Update submodules to master

* Correct minimal chainspec modifications

* Duplication of validator polls are no longer fatal

* Apply PR suggestions
2019-08-10 11:44:17 +10:00
Paul Hauner
64a6e1475c
Various minor fixes 2019-08-10 11:31:31 +10:00
Age Manning
66419d00ea
Remove redundant slog dependencies 2019-08-09 13:27:31 +10:00
Age Manning
ec73dfe90b
Starting of req/resp overhaul 2019-08-09 13:27:31 +10:00
Age Manning
5a74239ebc
Add decoding/encoding for extended gossip topics. Correct logging CLI 2019-08-09 13:27:31 +10:00
Age Manning
80f15f5d70
Correct gossipsub message encoding. Add extended topics 2019-08-09 13:27:31 +10:00
Age Manning
d83fa67068
Subscribe to all required gossipsub topics 2019-08-09 13:27:31 +10:00
Age Manning
ce50616032
Improve logging 2019-08-09 13:27:30 +10:00
Age Manning
3210489a36
Apply PR suggestions 2019-08-09 13:23:47 +10:00
Paul Hauner
f4121d9deb
Ignore unknown blocks at fork choice 2019-08-09 12:34:56 +10:00
Paul Hauner
67fe21c1c0
Ensure committee cache is build during attn proc 2019-08-09 12:32:32 +10:00
Paul Hauner
d191812d4b
Fix overly restrictive check in fork choice. 2019-08-09 12:23:10 +10:00
Paul Hauner
76bb671084
Fix bug with fork choice, tidy 2019-08-09 11:54:35 +10:00
Age Manning
826abea8e0
Merge latest 'master' 2019-08-09 10:46:49 +10:00
Paul Hauner
284166c7f8
Merge branch 'master' into attestation-processing 2019-08-08 17:11:19 +10:00
Paul Hauner
b1591c3c12
Remove attestation processing from op pool 2019-08-08 16:49:27 +10:00
Paul Hauner
7c134a7504
Simplify, fix bugs, add tests for chain iters 2019-08-08 16:47:24 +10:00
Paul Hauner
3a1f56a42e
Tidy ancestor iterators (#490)
* Tidy ancestor iterators

* Improve comments, remove code fragement
2019-08-08 12:28:10 +10:00
Paul Hauner
8cfa36fedd
Publish ssz_types (and deps) to crates.io (#468)
* Rename `hashing` crate to `eth2_hashing`

* Add license, desc to eth2_hashing Cargo.toml

* Remove merkle root from eth2 hashing

* Remove old benches folder (zombied from old branch)

* Add docs to eth2_hashing

* Prepare tree_hash for publishing on crates.io

* Update deps to use crates.io instead of paths

* Update all crates to pull ssz from crates.io

* Remove cached_tree_hash, add patches to manifest

* Fix compile error in benches

* Remove unused code

* Fix fake_crypto compile error
2019-08-08 11:39:47 +10:00
Paul Hauner
65ce94b2ef
Remove old code fragment 2019-08-07 16:54:35 +10:00
Paul Hauner
378fe05c89
Tidy attestation processing 2019-08-07 16:42:07 +10:00
Paul Hauner
fe2402b361
Add another attestation processing test 2019-08-07 16:02:30 +10:00
Paul Hauner
436c87abcd
Merge branch 'iter-fixes' into attestation-processing 2019-08-07 15:45:42 +10:00
Age Manning
907a4e5a4b
Configuration updates allow for verbosity CLI flag and spec constants 2019-08-07 14:54:08 +10:00
Paul Hauner
2c3fc318ba
Do first pass on Grants code 2019-08-07 13:20:15 +10:00
Grant Wuerker
ce73705498
formatting 2019-08-06 19:17:15 +02:00
Grant Wuerker
c431bd993e
Implicit conflicts resolved. 2019-08-06 14:56:13 +02:00
Grant Wuerker
d11839c392
Merge branch 'master' into process-free-attestation 2019-08-06 12:28:30 +02:00
Grant Wuerker
f4b169ce80
Integration tests for free attestation processing 2019-08-06 10:03:24 +02:00
Grant Wuerker
dcac8d56bd
Revert "Refactored attestation creation in test utils"
This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306.
2019-08-06 10:03:22 +02:00
Grant Wuerker
78f3911522
Refactored attestation creation in test utils 2019-08-06 10:01:48 +02:00
Grant Wuerker
edd99fafb6
Getting attestation slot via helper method 2019-08-06 10:01:06 +02:00
Age Manning
107bbdcccd
Updates to latest interop branch.
- Shifts decoding of objects into message handler.
- Updates to latest interop gossipsub.
- Adds interop spec constant.
2019-08-06 17:54:38 +10:00
Age Manning
0613bc16fc
Update to latest libp2p, gossipsub improvements 2019-08-06 15:46:19 +10:00
Age Manning
04ce9ec95e
Adds Identify protocol, clean up RPC protocol name handling 2019-08-06 15:46:16 +10:00
Age Manning
15c4062761
Add ability to connect to raw libp2p nodes 2019-08-06 15:35:54 +10:00
Age Manning
40c0b70b22
Add interop chain spec and rename chain_id 2019-08-06 15:35:05 +10:00
Pawan Dhananjay
88e89f9ab2 Update ethereum types (#489)
* Update ethereum-types to version 0.6

* Fix tests

* Run rustfmt
2019-08-06 14:41:42 +10:00
Paul Hauner
b096e3a643
Tidy ancestor iterators 2019-08-06 14:29:45 +10:00
Paul Hauner
845f336a59
Interop chain start strategies (#479)
* Implement more flexible beacon chain genesis

* Fix compile issues from rebase on master

* Rename CLI flag

* Adds initial documentation for TOML files

* Update docs readme

* Add first version of cli_util

* Dont write cache fields in serde

* Tidy cli_util

* Add code to load genesis YAML file

* Move serde_utils out of tests in `types`

* Update logging text

* Fix serde YAML for Fork

* Make yaml hex decoding more strict

* Update deterministic key generate for interop

* Set deposit count on testing genesis state

* Make some fixes for deposit count

* Remove code fragements

* Large restructure of docs

* Tidy docs

* Fix readme link

* Add interop docs

* Tidy README
2019-08-06 13:29:27 +10:00
Paul Hauner
e21d3fed05
Revert "Tidy ancestor iterators"
This reverts commit 5079c25bb2.

Accidental push to master.. my bad!
2019-08-05 16:27:55 +10:00
Paul Hauner
5079c25bb2
Tidy ancestor iterators 2019-08-05 16:25:21 +10:00
Michael Sproul
89cb01cc93 Delete stray RocksDB store implementation (#477) 2019-08-01 07:59:23 +10:00
Luke Anderson
0052ea711e First RESTful HTTP API (#399)
* Added generated code for REST API.
 - Created a new crate rest_api, which will adapt the openapi generated code to Lighthouse
 - Committed automatically generated code from openapi-generator-cli (via docker). Should hopfully not have to modify this at all, and do all changes in the rest_api crate.

* Removed openapi generated code, because it was the rust client, not the rust server.

* Added the correct rust-server code, automatically generated from openapi.

* Added generated code for REST API.
 - Created a new crate rest_api, which will adapt the openapi generated code to Lighthouse
 - Committed automatically generated code from openapi-generator-cli (via docker). Should hopfully not have to modify this at all, and do all changes in the rest_api crate.

* Removed openapi generated code, because it was the rust client, not the rust server.

* Added the correct rust-server code, automatically generated from openapi.

* Included REST API in configuratuion.
 - Started adding the rest_api into the beacon node's dependencies.
 - Set up configuration file for rest_api and integrated into main client config
 - Added CLI flags for REST API.

* Futher work on REST API.
 - Adding the dependencies to rest_api crate
 - Created a skeleton BeaconNodeService, which will handle /node requests.
 - Started the rest_api server definition, with the high level request handling logic.

* Added generated code for REST API.
 - Created a new crate rest_api, which will adapt the openapi generated code to Lighthouse
 - Committed automatically generated code from openapi-generator-cli (via docker). Should hopfully not have to modify this at all, and do all changes in the rest_api crate.

* Removed openapi generated code, because it was the rust client, not the rust server.

* Added the correct rust-server code, automatically generated from openapi.

* Included REST API in configuratuion.
 - Started adding the rest_api into the beacon node's dependencies.
 - Set up configuration file for rest_api and integrated into main client config
 - Added CLI flags for REST API.

* Futher work on REST API.
 - Adding the dependencies to rest_api crate
 - Created a skeleton BeaconNodeService, which will handle /node requests.
 - Started the rest_api server definition, with the high level request handling logic.

* WIP: Restructured REST API to use hyper_router and separate services.

* WIP: Fixing rust for REST API

* WIP: Fixed up many bugs in trying to get router to compile.

* WIP: Got the beacon_node to compile with the REST changes

* Basic API works!
 - Changed CLI flags from rest-api* to api*
 - Fixed port cli flag
 - Tested, works over HTTP

* WIP: Moved things around so that we can get state inside the handlers.

* WIP: Significant API updates.
 - Started writing a macro for getting the handler functions.
 - Added the BeaconChain into the type map, gives stateful access to the beacon state.
 - Created new generic error types (haven't figured out yet), to reduce code duplication.
 - Moved common stuff into lib.rs

* WIP: Factored macros, defined API result and error.
 - did more logging when creating HTTP responses
 - Tried moving stuff into macros, but can't get macros in macros to compile.
 - Pulled out a lot of placeholder code.

* Fixed macros so that things compile.

* Cleaned up code.
 - Removed unused imports
 - Removed comments
 - Addressed all compiler warnings.
 - Ran cargo fmt.

* Removed auto-generated OpenAPI code.

* Addressed Paul's suggestions.
 - Fixed spelling mistake
 - Moved the simple macros into functions, since it doesn't make sense for them to be macros.
 - Removed redundant code & inclusions.

* Removed redundant validate_request function.

* Included graceful shutdown in Hyper server.

* Fixing the dropped exit_signal, which prevented the API from starting.

* Wrapped the exit signal, to get an API shutdown log line.
2019-07-31 18:29:41 +10:00
blacktemplar
309b10c4a8 add logging functionality for aligning key value pairs after message (#461)
* add logging functionality for aligning key value pairs after the main messages

* move to own crate, change default message width to 40

* use FullFormat in validator_client (CompactFormat is not compatible with aligning)

* move logging to eth2/utils/logging
2019-07-31 09:06:53 +10:00
Age Manning
c695aa3c92
Merge pull request #471 from b-m-f/split-network-loops
Split network and swarm discovery loop, break each when not ready
2019-07-30 23:37:43 +10:00
b-m-f
eb6ba50544 Split network and swarm discovery loop, break each when not ready 2019-07-30 11:33:37 +01:00
Michael Sproul
a236003a7b Update to frozen spec ❄️ (v0.8.1) (#444)
* types: first updates for v0.8

* state_processing: epoch processing v0.8.0

* state_processing: block processing v0.8.0

* tree_hash_derive: support generics in SignedRoot

* types v0.8: update to use ssz_types

* state_processing v0.8: use ssz_types

* ssz_types: add bitwise methods and from_elem

* types: fix v0.8 FIXMEs

* ssz_types: add bitfield shift_up

* ssz_types: iterators and DerefMut for VariableList

* types,state_processing: use VariableList

* ssz_types: fix BitVector Decode impl

Fixed a typo in the implementation of ssz::Decode for BitVector, which caused it
to be considered variable length!

* types: fix test modules for v0.8 update

* types: remove slow type-level arithmetic

* state_processing: fix tests for v0.8

* op_pool: update for v0.8

* ssz_types: Bitfield difference length-independent

Allow computing the difference of two bitfields of different lengths.

* Implement compact committee support

* epoch_processing: committee & active index roots

* state_processing: genesis state builder v0.8

* state_processing: implement v0.8.1

* Further improve tree_hash

* Strip examples, tests from cached_tree_hash

* Update TreeHash, un-impl CachedTreeHash

* Update bitfield TreeHash, un-impl CachedTreeHash

* Update FixedLenVec TreeHash, unimpl CachedTreeHash

* Update update tree_hash_derive for new TreeHash

* Fix TreeHash, un-impl CachedTreeHash for ssz_types

* Remove fixed_len_vec, ssz benches

SSZ benches relied upon fixed_len_vec -- it is easier to just delete
them and rebuild them later (when necessary)

* Remove boolean_bitfield crate

* Fix fake_crypto BLS compile errors

* Update ef_tests for new v.8 type params

* Update ef_tests submodule to v0.8.1 tag

* Make fixes to support parsing ssz ef_tests

* `compact_committee...` to `compact_committees...`

* Derive more traits for `CompactCommittee`

* Flip bitfield byte-endianness

* Fix tree_hash for bitfields

* Modify CLI output for ef_tests

* Bump ssz crate version

* Update ssz_types doc comment

* Del cached tree hash tests from ssz_static tests

* Tidy SSZ dependencies

* Rename ssz_types crate to eth2_ssz_types

* validator_client: update for v0.8

* ssz_types: update union/difference for bit order swap

* beacon_node: update for v0.8, EthSpec

* types: disable cached tree hash, update min spec

* state_processing: fix slot bug in committee update

* tests: temporarily disable fork choice harness test

See #447

* committee cache: prevent out-of-bounds access

In the case where we tried to access the committee of a shard that didn't have a committee in the
current epoch, we were accessing elements beyond the end of the shuffling vector and panicking! This
commit adds a check to make the failure safe and explicit.

* fix bug in get_indexed_attestation and simplify

There was a bug in our implementation of get_indexed_attestation whereby
incorrect "committee indices" were used to index into the custody bitfield. The
bug was only observable in the case where some bits of the custody bitfield were
set to 1. The implementation has been simplified to remove the bug, and a test
added.

* state_proc: workaround for compact committees bug

https://github.com/ethereum/eth2.0-specs/issues/1315

* v0.8: updates to make the EF tests pass

* Remove redundant max operation checks.
* Always supply both messages when checking attestation signatures -- allowing
  verification of an attestation with no signatures.
* Swap the order of the fork and domain constant in `get_domain`, to match
  the spec.

* rustfmt

* ef_tests: add new epoch processing tests

* Integrate v0.8 into master (compiles)

* Remove unused crates, fix clippy lints

* Replace v0.6.3 tags w/ v0.8.1

* Remove old comment

* Ensure lmd ghost tests only run in release

* Update readme
2019-07-30 12:44:51 +10:00
Paul Hauner
177df12149
Testnet stability (#451)
* Change reduced tree for adding weightless node

* Add more comments for reduced tree fork choice

* Small refactor on reduced tree for readability

* Move test_harness forking logic into itself

* Add new `AncestorIter` trait to store

* Add unfinished tests to fork choice

* Make `beacon_state.genesis_block_root` public

* Add failing lmd_ghost fork choice tests

* Extend fork_choice tests, create failing test

* Implement Debug for generic ReducedTree

* Add lazy_static to fork choice tests

* Add verify_integrity fn to reduced tree

* Fix bugs in reduced tree

* Ensure all reduced tree tests verify integrity

* Slightly alter reduce tree test params

* Add (failing) reduced tree test

* Fix bug in fork choice

Iter ancestors was not working well with skip slots

* Put maximum depth for common ancestor search

Ensures that we don't search back past the finalized root.

* Add basic finalization tests for reduced tree

* Change fork choice to use beacon_block_root

Previously it was using target_root, which was wrong

* Change reduced tree for adding weightless node

* Add more comments for reduced tree fork choice

* Small refactor on reduced tree for readability

* Move test_harness forking logic into itself

* Add new `AncestorIter` trait to store

* Add unfinished tests to fork choice

* Make `beacon_state.genesis_block_root` public

* Add failing lmd_ghost fork choice tests

* Extend fork_choice tests, create failing test

* Implement Debug for generic ReducedTree

* Add lazy_static to fork choice tests

* Add verify_integrity fn to reduced tree

* Fix bugs in reduced tree

* Ensure all reduced tree tests verify integrity

* Slightly alter reduce tree test params

* Add (failing) reduced tree test

* Fix bug in fork choice

Iter ancestors was not working well with skip slots

* Put maximum depth for common ancestor search

Ensures that we don't search back past the finalized root.

* Add basic finalization tests for reduced tree

* Add network dir CLI flag

* Simplify "NewSlot" log message

* Rename network-dir CLI flag

* Change fork choice to use beacon_block_root

Previously it was using target_root, which was wrong

* Update db dir size for metrics

* Change slog to use `FullFormat` logging

* Update some comments and log formatting

* Add prom gauge for best block root

* Only add known target blocks to fork choice

* Add finalized and justified root prom metrics

* Add CLI flag for setting log level

* Add logger to beacon chain

* Add debug-level CLI flag to validator

* Allow block processing if fork choice fails

* Create warn log when there's low libp2p peer count

* Minor change to logging

* Make ancestor iter return option

* Disable fork choice test when !debug_assertions

* Fix type, removed code fragment

* Tidy some borrow-checker evading

* Lower reduced tree random test iterations
2019-07-29 13:45:45 +10:00
Paul Hauner
7458022fcf
Fork choice bug fixes (#449)
* Change reduced tree for adding weightless node

* Add more comments for reduced tree fork choice

* Small refactor on reduced tree for readability

* Move test_harness forking logic into itself

* Add new `AncestorIter` trait to store

* Add unfinished tests to fork choice

* Make `beacon_state.genesis_block_root` public

* Add failing lmd_ghost fork choice tests

* Extend fork_choice tests, create failing test

* Implement Debug for generic ReducedTree

* Add lazy_static to fork choice tests

* Add verify_integrity fn to reduced tree

* Fix bugs in reduced tree

* Ensure all reduced tree tests verify integrity

* Slightly alter reduce tree test params

* Add (failing) reduced tree test

* Fix bug in fork choice

Iter ancestors was not working well with skip slots

* Put maximum depth for common ancestor search

Ensures that we don't search back past the finalized root.

* Add basic finalization tests for reduced tree

* Change fork choice to use beacon_block_root

Previously it was using target_root, which was wrong

* Make ancestor iter return option

* Disable fork choice test when !debug_assertions

* Fix type, removed code fragment

* Tidy some borrow-checker evading

* Lower reduced tree random test iterations
2019-07-29 12:08:52 +10:00
Pawan Dhananjay
db094022b9 Remove unused dependencies (#456) 2019-07-29 09:55:57 +10:00
John Adler
fec7168512
Fix lots of typos. 2019-07-26 15:26:06 -04:00
Grant Wuerker
5e3fe2467a
Merge branch 'master' into process-free-attestation 2019-07-25 13:35:39 +02:00
Grant Wuerker
b49d592eee
Copy paste error in comment fixed. 2019-07-24 18:06:18 +02:00
Grant Wuerker
51645aa9af
Correct process_attestation error handling. 2019-07-24 18:03:48 +02:00
Age Manning
b5af73d056
Merge pull request #441 from sigp/rpc-update
RPC Rewrite
2019-07-24 20:08:16 +10:00
Age Manning
ae96325c81
Allows Libp2p service to be read outside network thread 2019-07-24 17:45:31 +10:00
Age Manning
7d38cba252
Remove encoding error and redundant code 2019-07-24 15:17:44 +10:00
Grant Wuerker
b2471eca49
Checking if the an attestation contains a latest message 2019-07-23 20:50:18 +02:00
Age Manning
89ff7fb6b8
Complete initial testing of new RPC 2019-07-23 22:45:42 +10:00
Age Manning
b350a78fec
Update RPC. WIP 2019-07-22 14:13:18 +10:00
Grant Wuerker
bef7ca6bfb
Cleaned up get_attestation_state 2019-07-20 12:49:41 +02:00
Grant Wuerker
7cdfa3cc27
Looser check on beacon state validity. 2019-07-19 14:52:01 +02:00
b-m-f
e9671084af
Saves ENR to disk on every address update
Signed-off-by: b-m-f <max@ehlers.berlin>
2019-07-18 16:48:04 +01:00
Grant Wuerker
b90edaf7f6
Merge branch 'master' into process-free-attestation 2019-07-18 12:33:22 +02:00
Age Manning
b2a1b20e24
Merge latest master 2019-07-16 23:26:31 +10:00
Age Manning
414d41cb57
Shift changes into message handler and simple sync for rpc-rewrite 2019-07-16 22:32:37 +10:00
Kirk Baird
0513559252 Fix syncing bugs by recursively attempting to process parents in the … (#429)
* Fix syncing bugs by recursively attempting to process parents in the import queue, change BlockRootsIterator

* Swap from crossbeam channel to tokio mpsc

* Recursion fix

* Remove exess block processing

* Fix network lag, correct attestation topic

* Correct network poll logic

* Overhaul of SimpleSync and modify BlockRootsIterator to return start_slot

* Fix bug in tests relating to StateRootsIterator

* Remove old, commented-out heartbeat code.

* Tidy docs on import queue enum

* Change source logging msg in simple sync

* Rename function parameter in simple sync

* Use `BestBlockRootsIterator` in `reduced_tree`

* Update comments for `BestBlockRootsIterator`

* Fix duplicate dep in cargo.toml
2019-07-16 17:28:15 +10:00
Age Manning
704263e35f
Clean up Protocol types 2019-07-15 18:41:05 +10:00
Age Manning
15c99b5f37
RPC Update. WIP 2019-07-15 17:07:23 +10:00
Age Manning
15cdd2afb9
Initial codec module 2019-07-13 18:57:18 +10:00
Age Manning
d84780a339
Initial base codec 2019-07-13 18:35:33 +10:00
Grant Wuerker
40b166edcd
Retrieving state from store and checking signature 2019-07-11 16:32:01 +02:00
Age Manning
0292679f27
Improved error handling. Switching to codecs for easier encoding support 2019-07-11 15:11:31 +10:00
Kirk Baird
08069704c1
Add cli flag for logging to JSON file 2019-07-10 10:27:44 +10:00
Grant Wuerker
adf1d9c533
Processing attestation in fork choice 2019-07-09 12:36:59 +02:00
Grant Wuerker
e23726c0a1
Renamed fork_choice::process_attestation_from_block 2019-07-09 12:36:26 +02:00
Age Manning
4a84b2f7cc
Improved RPC handling. WIP 2019-07-09 15:44:23 +10:00
Age Manning
bb0e28b8e3
Improved rpc protocols handler. WIP 2019-07-06 23:43:44 +10:00
Age Manning
f1127e4e0d
Initial addition of an RPC Protocol Handler 2019-07-06 21:32:32 +10:00
Age Manning
cda61c1577
Adds basic inbound/outbound upgrades for eth2 rpc 2019-07-05 18:59:53 +10:00
Age Manning
746935fa41
Initial sub-protocol implementation 2019-07-04 15:26:02 +10:00
Age Manning
1aeec12b78
Improve error handling of default directory 2019-07-04 14:14:27 +10:00
Age Manning
dd410535cb
Remove Phase 1 TODO 2019-07-04 14:14:27 +10:00
Age Manning
bffe6c327f
Removes left-over debugging statements 2019-07-04 14:14:27 +10:00
Age Manning
177daf2609
Typo fixes 2019-07-04 14:14:27 +10:00
Age Manning
af28d5e20c
Add persistent network identification 2019-07-04 14:14:27 +10:00
Age Manning
7dc5e2f959
Update to latest libp2p 2019-07-04 14:14:27 +10:00
Age Manning
0952a36a21
Initial integration of discovery v5 2019-07-04 14:14:27 +10:00
Age Manning
44c9058477
Integrate discv5 into lighthouse 2019-07-04 14:14:27 +10:00
Age Manning
6ee2b4df34
Complete merging of network addition branch 2019-07-04 14:14:26 +10:00
Age Manning
c7e17c8641
Updates for latest master 2019-07-04 14:14:26 +10:00
Age Manning
43135484ca
Update to lastest libp2p 2019-07-04 14:13:43 +10:00
Age Manning
b33ce5dd10
Initial core grouping of libp2p behaviours 2019-07-04 14:13:43 +10:00
Age Manning
cb7d5eba1c
Discovery and gossip bug fixes 2019-07-04 14:12:06 +10:00
Age Manning
f80c34b74f
Builds on discovery. Adds identify to discovery 2019-07-04 14:11:17 +10:00
Age Manning
d2f80e3b2a
Adds env logger to output libp2p logs 2019-07-04 14:11:17 +10:00
Age Manning
e36fa3152d
Adds verbosity cli flag 2019-07-04 14:11:17 +10:00
Age Manning
a38f4c4cd1
Adds Kademlia for peer discovery 2019-07-04 14:08:48 +10:00
Age Manning
be6ebb5ffa
Add custom inactivity timeout to gossipsub 2019-07-04 14:08:48 +10:00
Age Manning
2d710f19fc
Update to latest libp2p 2019-07-04 14:08:48 +10:00
Age Manning
7920f8098f
Complete moving network logc into beacon node 2019-07-04 14:07:26 +10:00
Age Manning
64abd0bc5b
Removes network parameters from chain spec 2019-07-04 14:07:26 +10:00
Age Manning
a31d6bcb22
RPC methods get pubsub topics from chain spec 2019-07-04 14:07:26 +10:00
Age Manning
4e24c8e651
Add topics to chain id 2019-07-04 14:07:26 +10:00
Age Manning
c7bd02caaf
Propogate valid attestations accross the network 2019-07-04 14:07:26 +10:00
Paul Hauner
027f0a539d
Prepare ssz for publishing on crates.io 2019-07-03 16:06:20 +10:00
Paul Hauner
d395feb027
Merge pull request #418 from sigp/fix-syncing
Improve syncing, fix bugs
2019-07-03 13:48:44 +10:00
Age Manning
76371659e9
Improve error handling of default directory 2019-07-02 17:32:14 +10:00
Age Manning
6c18b417c3
Remove Phase 1 TODO 2019-07-02 11:15:35 +10:00
Age Manning
1f62221235
Removes left-over debugging statements 2019-07-01 17:23:14 +10:00
Age Manning
f8e20d8027
Typo fixes 2019-07-01 16:40:35 +10:00
Age Manning
5521c53d36
Add persistent network identification 2019-07-01 16:38:42 +10:00
Michael Sproul
73c4171b52
op_pool: finish persistence support 2019-07-01 10:55:55 +10:00
Michael Sproul
7fe458af45
op_pool: re-jig deposit handling (needs more work) 2019-07-01 10:55:55 +10:00
Michael Sproul
604fe2d97f
op_pool: partial persistence support 2019-07-01 10:55:55 +10:00
Paul Hauner
2a7122beaf
Partially refactor simple_sync, makes improvement 2019-06-27 18:05:03 +10:00
Age Manning
955574f469
Update to latest libp2p 2019-06-25 18:57:11 +10:00
Age Manning
a64a6c7d3a
Initial integration of discovery v5 2019-06-25 18:02:50 +10:00
Age Manning
81f0b6c238
Integrate discv5 into lighthouse 2019-06-25 18:02:50 +10:00
Age Manning
84ea5adffe
Complete merging of network addition branch 2019-06-25 18:02:50 +10:00
Age Manning
cf459b60a9
Updates for latest master 2019-06-25 18:02:50 +10:00
Age Manning
fc8dc6dfa7
Update to lastest libp2p 2019-06-25 18:02:50 +10:00
Age Manning
f7c2e4c5af
Initial core grouping of libp2p behaviours 2019-06-25 18:02:50 +10:00
Age Manning
75959cc9a2
Discovery and gossip bug fixes 2019-06-25 18:02:50 +10:00
Age Manning
25f37ad967
Builds on discovery. Adds identify to discovery 2019-06-25 18:02:50 +10:00
Age Manning
89282c1e09
Adds env logger to output libp2p logs 2019-06-25 18:02:49 +10:00
Age Manning
ad1438db53
Adds verbosity cli flag 2019-06-25 18:02:49 +10:00
Age Manning
8d5d228270
Adds Kademlia for peer discovery 2019-06-25 18:02:49 +10:00
Age Manning
dea6ac7923
Add custom inactivity timeout to gossipsub 2019-06-25 18:02:49 +10:00
Age Manning
7ad9805f02
Update to latest libp2p 2019-06-25 18:02:49 +10:00
Age Manning
c1d609902a
Complete moving network logc into beacon node 2019-06-25 18:02:49 +10:00
Age Manning
dd3a4f0b43
Removes network parameters from chain spec 2019-06-25 18:02:49 +10:00
Age Manning
f54bd79f84
RPC methods get pubsub topics from chain spec 2019-06-25 18:02:48 +10:00
Age Manning
e9181e120c
Add topics to chain id 2019-06-25 18:02:48 +10:00
Age Manning
95ce8ca6e6
Propogate valid attestations accross the network 2019-06-25 18:02:48 +10:00
Paul Hauner
906580be15
Attempt to catch edge case in syncing 2019-06-25 16:05:26 +10:00
Paul Hauner
8afe8b3569
Implement fixes from PR review 2019-06-24 15:31:36 +10:00
Paul Hauner
3a196f3fdc
Tidy, fix clippy lints 2019-06-24 09:34:56 +10:00
Paul Hauner
8e13237b7f
Only perform beacon chain tests on release 2019-06-24 09:34:56 +10:00
Paul Hauner
075c989375
Remove old fork_choice crate 2019-06-24 09:34:56 +10:00
Paul Hauner
77fba0b98e
Fix bugs in fork choice, add more tests 2019-06-24 09:34:56 +10:00
Paul Hauner
f8fb011d6c
Finish fork test for beacon chain 2019-06-24 09:34:55 +10:00
Paul Hauner
8ed03e391d
Allow harness to produce fork blocks 2019-06-24 09:34:55 +10:00
Paul Hauner
e904e0a5a8
Fix bug in operations for block production 2019-06-24 09:34:55 +10:00
Paul Hauner
fbb40485c1
Add additional chain tests 2019-06-24 09:34:55 +10:00
Paul Hauner
299b4cb207
Document beacon chain harness 2019-06-24 09:34:55 +10:00
Paul Hauner
723283bd01
Add attestation strategy to chain harness 2019-06-24 09:34:55 +10:00
Paul Hauner
46c0e17682
Add arbitrary attestation for beacon chain harness 2019-06-24 09:34:55 +10:00
Paul Hauner
7a4c3e26ac
Fix bug in reduced tree fork choice 2019-06-24 09:34:54 +10:00
Paul Hauner
0b2ad4d0a1
Mark reduced_tree fork choice as incomplete 2019-06-24 09:34:54 +10:00
Paul Hauner
d0037f49d8
Add progress on debugging fork choice 2019-06-24 09:34:54 +10:00
Paul Hauner
85b23f9f1b
Add incomplete progress on fork choice 2019-06-24 09:34:54 +10:00
Paul Hauner
41a36da765
Update BeaconChain tests 2019-06-24 09:34:54 +10:00
Paul Hauner
55818e285a
Refactor block prod. to produce for forks 2019-06-24 09:34:53 +10:00
Paul Hauner
55196dff64
Remove iter mod from beacon chain
Now the iter mod in store is the only implementation
2019-06-24 09:34:53 +10:00
Paul Hauner
952e08ba38
Add state roots iter to store 2019-06-24 09:34:53 +10:00
Paul Hauner
c6e24572c7
Improve chain harness tests 2019-06-24 09:34:53 +10:00
Paul Hauner
9c2bbb6c05
Add stubbed-out block processing to fork choice 2019-06-24 09:34:52 +10:00
Paul Hauner
f6c86d0f7f
Add attestations to beacon chain harness 2019-06-24 09:34:52 +10:00
Paul Hauner
1638a7aa62
Alias zero hash to genesis in find head 2019-06-24 09:34:52 +10:00
Paul Hauner
8fb6ffffe2
Add new beacon chain test harness 2019-06-24 09:34:52 +10:00
Paul Hauner
1128de535d
Ignore attestations to zero hash 2019-06-24 09:34:52 +10:00
Paul Hauner
f4621a9f1a
Improve reduced tree fork choice 2019-06-24 09:34:52 +10:00
Paul Hauner
7756a658a7
Update fork choice find head fn 2019-06-24 09:34:51 +10:00
Paul Hauner
2ee71aa808
Add new fork choice struct to beacon chain 2019-06-24 09:34:51 +10:00
Paul Hauner
4a3d54761a
Add progress on reduced tree fork choice 2019-06-24 09:34:51 +10:00
Paul Hauner
25e93f60fd
Tidy comments 2019-06-17 10:01:23 -04:00
Paul Hauner
cf3d8b0688
Run cargo fmt --all 2019-06-13 12:21:09 -04:00
Paul Hauner
4c0724fba6
Refactor block processing results, some sync logic 2019-06-12 23:54:16 -04:00
Paul Hauner
059699736b
Return errors instead of panic in libp2p 2019-06-10 21:37:41 -04:00
Paul Hauner
e550c0218f
Add various fixes to clippy lints
Thou shalt appease clippy
2019-06-10 11:01:25 -04:00
Paul Hauner
a9284bec18
Fix type in default confi filename 2019-06-09 06:41:51 -04:00
Paul Hauner
4a871dbcc7
Change default config filenames 2019-06-09 06:35:36 -04:00
Paul Hauner
a662c3a940
Fix various compile errors and warnings 2019-06-09 06:26:34 -04:00
Paul Hauner
ab12787610
Update account manager config parsing 2019-06-09 04:34:56 -04:00
Paul Hauner
3487b16ce5
Add eth2_config crate, integrate into val client 2019-06-08 20:21:50 -04:00
Paul Hauner
eb23b003b4
Ensure beacon node generates the right eth2 spec 2019-06-08 17:53:22 -04:00
Paul Hauner
d8fc5f31d8
Add Eth2Config to runtime 2019-06-08 13:17:03 -04:00
Paul Hauner
fd6766c268
Tidy beacon node runtime code 2019-06-08 09:46:04 -04:00
Paul Hauner
749f2fcb5f
Unify EthSpecs in Mainnet and Minimal 2019-06-08 08:49:04 -04:00
Paul Hauner
caddeba81b
Remove genesis_epoch from ChainSpec 2019-06-08 08:17:42 -04:00
Paul Hauner
e74d49fc8a
Remove dupe info between ChainSpec and EthSpec 2019-06-08 07:57:25 -04:00
Paul Hauner
f69d9093a3
Add concept of beacon node configuration TOML 2019-06-07 21:00:34 -04:00
Paul Hauner
e73a31c37f
Refactor ClientConfig, add serde to it 2019-06-07 19:44:27 -04:00
Paul Hauner
39ec96ad82
Run rustfmt 2019-06-07 02:55:43 -04:00
Paul Hauner
719dd72de6
Fix recently introduced sync bugs 2019-06-07 02:55:16 -04:00
Paul Hauner
4cecf05198
Fix beacon chain block iters 2019-06-07 02:48:26 -04:00
Paul Hauner
591c8ae219
Reject re-processing the genesis block 2019-06-07 02:48:08 -04:00
Paul Hauner
c975828195
Merge branch 'rpc-fix' into docker-env 2019-06-06 09:51:48 -04:00
Paul Hauner
1707aa5bf3
Fix bug in RPC wire encoding
I broke this in a previous commit.
2019-06-06 09:47:21 -04:00
Paul Hauner
af96dd08c8
Simplify simple_sync code 2019-06-06 00:32:09 -04:00
Paul Hauner
f52d66a136
Fix bug in rev block iter 2019-06-06 00:31:18 -04:00
Paul Hauner
ed9f655877
Update BeaconChain iters 2019-06-06 00:28:36 -04:00
Paul Hauner
df366f99dc
Add block iter to beacon chain 2019-06-05 13:33:32 +10:00
Paul Hauner
8ab1d28c9f
Fix attestation processing metrics 2019-06-04 17:10:25 +10:00
Paul Hauner
f4b4709999
Publish attestations from RPC to P2P 2019-06-04 16:33:35 +10:00
Paul Hauner
f95711c15a
Add present_epoch metric 2019-06-04 15:04:05 +10:00
Paul Hauner
67fdb4a7fb
Store beacon state committee cache in DB 2019-06-04 13:13:58 +10:00
Paul Hauner
f530f5a848
Ensure committees are built for block processing 2019-06-04 13:13:38 +10:00
Paul Hauner
82202a7765
Update components to suit v0.6.1 API 2019-06-04 09:37:40 +10:00
Paul Hauner
39cb63e01e
Merge branch 'v0.6.1' into docker-env 2019-06-04 09:25:00 +10:00
Michael Sproul
6bde64bd6a
spec v0.6: update beacon_node, validator_client 2019-06-03 16:13:51 +10:00
Paul Hauner
e310bdd904
Merge branch 'master' into v0.6.1 2019-06-02 13:49:24 +10:00
Paul Hauner
997095fc43
Add attestations per block metric 2019-06-01 15:29:13 +10:00
Paul Hauner
244ffbc604
Store genesis root in beacon chain, fix fork bug 2019-06-01 15:02:19 +10:00
Paul Hauner
29c5f297a6
Add database size metric 2019-06-01 14:43:08 +10:00
Paul Hauner
c8ba44b0b5
Create db-level alias for genesis block 2019-06-01 12:56:35 +10:00
Paul Hauner
8831db1e0f
Fix recently introduced errors for gRPC block prod 2019-06-01 12:36:10 +10:00
Paul Hauner
7058f62b50
Add extra validator balances metric 2019-06-01 08:26:25 +10:00
Paul Hauner
b33b5c28b4
Add additional metrics for BeaconChain 2019-05-31 18:13:35 +10:00
Paul Hauner
5b425c9bf3
Switch runtime fork choice to optimized 2019-05-31 17:53:52 +10:00
Paul Hauner
08bf5817c9
Update fork choice to take just. head 2019-05-31 17:51:32 +10:00
Paul Hauner
2f9f8bf772
Tidy BeaconChain; del spec, unify head updates 2019-05-31 17:32:20 +10:00
Paul Hauner
8acffcc0db
Make some BeaconChain functions private 2019-05-30 18:38:41 +10:00
Paul Hauner
5a5eebca06
Remove BeaconChain wrapper trait from rpc 2019-05-30 18:35:27 +10:00
Paul Hauner
0590504261
Remove BeaconChain wrapper trait from network 2019-05-30 18:27:19 +10:00
Paul Hauner
f44170cab1
Add process metrics to http-server 2019-05-30 12:36:27 +10:00
Paul Hauner
4851d8be03
Fix some metrics labels 2019-05-29 18:09:51 +10:00
Paul Hauner
42b6e0c8a9
Run fork-choice after every block import 2019-05-29 17:55:38 +10:00
Paul Hauner
0b719e1523
Break store fns into smaller pieces 2019-05-29 17:53:13 +10:00
Paul Hauner
9f1039a350
Add metrics to BeaconChain, tidy fork choice 2019-05-29 17:52:43 +10:00
Paul Hauner
64fbc6bf3c
Add additional metrics to BeaconChain 2019-05-29 15:45:09 +10:00
Paul Hauner
a153f24bfa
Fix duplicate metric label 2019-05-29 14:21:34 +10:00
Paul Hauner
f89cb65360
Add best_slot metric 2019-05-29 13:55:17 +10:00
Paul Hauner
6d27c43666
Fix bug with metrics scraping 2019-05-28 18:05:52 +10:00
Paul Hauner
345f7d5f18
Add global metrics registry, pass to BeaconState 2019-05-28 17:30:09 +10:00
Paul Hauner
e756a0aaa4
Add extra metrics for prom 2019-05-28 16:11:16 +10:00
Paul Hauner
3e73a008eb
Add constant for testnet validator count 2019-05-28 16:10:52 +10:00
Paul Hauner
cb11656e12
Use LighthouseTestnet params, not FewValidators 2019-05-28 14:33:13 +10:00
Paul Hauner
3f27fd4edf
Parse http CLI args for HTTP server 2019-05-28 13:50:51 +10:00
Paul Hauner
2a04da8bf7
Switch HTTP listen port to 5052
This is a quick-fix to allow gRPC and HTTP to co-exist. In the future I
think we should swap this back to 5051.
2019-05-28 12:45:48 +10:00
Paul Hauner
21ecaddac1
Fix various clippy lints 2019-05-28 10:56:05 +10:00
Paul Hauner
6e5e1721f7
Fix lints introduced in Rust 1.35 2019-05-28 10:46:01 +10:00
Paul Hauner
b28fa3d20b
Save the BeaconChain to store on state update 2019-05-27 17:58:15 +10:00
Paul Hauner
faa682a9b5
Implement saving BeaconChain on client drop 2019-05-27 17:58:15 +10:00
Paul Hauner
9ed8a4d380
Implement basic BeaconChain persistence. 2019-05-27 17:58:15 +10:00
Paul Hauner
76602a65fc
Add new fns to ForkChoice and SlotClock 2019-05-27 17:58:15 +10:00
Paul Hauner
9922ed2239
Merge pull request #391 from sigp/http
Add iron HTTP server
2019-05-27 17:36:58 +10:00
Paul Hauner
ed4d7aa44a
Replace http_server unwrap with 500 error 2019-05-27 17:09:16 +10:00
Paul Hauner
3a65f84b12
Improve comments for http_server 2019-05-27 12:56:09 +10:00
Paul Hauner
255590ef3b
Add node/fork endpoint to HTTP API, tidy 2019-05-27 11:34:22 +10:00
Paul Hauner
705edf0e45
Remove commented-out code 2019-05-27 09:01:50 +10:00
Paul Hauner
855222fa28
Rename prom HTTP endpoint 2019-05-26 15:59:52 +10:00
Paul Hauner
ee8d13573f
Create BeaconChainTypes, thread through runtime 2019-05-26 15:59:29 +10:00
Paul Hauner
45e3a1759c
Add slot to prometheus endpoint 2019-05-25 17:57:07 +10:00
Paul Hauner
85211ebccd
Add basic prometheus endpoint 2019-05-25 17:25:21 +10:00
Paul Hauner
596ff5178b
Add http server shutdown, tidy 2019-05-25 16:17:48 +10:00
Paul Hauner
8dd07dd7d2
Add http server to beacon node w/ hello world 2019-05-25 14:31:13 +10:00
Matthew Slipper
95c218355e Support multiple bootnodes 2019-05-23 18:27:35 -07:00
Paul Hauner
b41f91db1d
Rename disk db dir 2019-05-21 19:32:07 +10:00
Paul Hauner
c840b76cac
Tidy store crate, add comments 2019-05-21 18:49:24 +10:00
Paul Hauner
3bcf5ba706
Rename db crate to store 2019-05-21 18:20:23 +10:00
Paul Hauner
b62f4477e1
More project-wide fixes for new DB 2019-05-21 17:45:35 +10:00
Paul Hauner
058829b64d
Update beacon_chain to latest DB 2019-05-21 17:27:06 +10:00
Paul Hauner
78368cc2cd
Make LevelDB key type concrete (not generic) 2019-05-21 16:49:56 +10:00
Paul Hauner
54f28df5b1
Improve testing for Store impls 2019-05-21 16:37:15 +10:00
Paul Hauner
b3a94de086
Remove unnecessary RwLock from LevelDB 2019-05-21 16:36:06 +10:00
Paul Hauner
7d067926dd
Replace RocksDB with LevelDB 2019-05-21 16:29:34 +10:00
Paul Hauner
f1584dada4
Update BeaconChain struct to use new store 2019-05-21 13:36:14 +10:00
Paul Hauner
f8c425d6b4
Fix and add tests for db get block at slot 2019-05-21 12:58:11 +10:00
Paul Hauner
182135b832
Remove old DB crates, start fixing fork_choice 2019-05-20 18:01:51 +10:00
Paul Hauner
658b425cce
Merge branch 'master' into disk-db 2019-05-20 16:56:04 +10:00