Commit Graph

1122 Commits

Author SHA1 Message Date
Paul Hauner
b4806d27eb
Fix comments from Michael 2019-09-26 10:46:56 +10:00
Paul Hauner
4ba6e916d5
Merge branch 'interop' of github.com:sigp/lighthouse into interop 2019-09-23 22:32:09 +10:00
Paul Hauner
4fccec158a
Fix wasted iterations when getting previous state 2019-09-23 22:22:19 +10:00
Paul Hauner
e7a580393c
Remove unnecessary drop() 2019-09-23 22:21:34 +10:00
Paul Hauner
b316086be0
Add write_ssz_files feature to beacon_chain crate 2019-09-23 22:20:47 +10:00
Paul Hauner
8ceb2e3d95
Refactor slot clock to remove underflow
Previously I had used `Instant` to refer to the genesis time.
2019-09-21 11:21:47 +10:00
Luke Anderson
43c110f228
Merge branch 'api-alignment' into interop 2019-09-16 09:36:47 +10:00
Paul Hauner
1b497e2e24
Gracefully shutdown the websocket server 2019-09-15 09:32:27 -04:00
Paul Hauner
9c5eded1ab
Add websocket config to CLI 2019-09-14 22:57:46 -04:00
Paul Hauner
da26341011
Add beacon chain event for finalization 2019-09-14 22:26:09 -04:00
Paul Hauner
07990e0e92
Adds beacon chain events, websocket event handler 2019-09-14 15:41:35 -04:00
Paul Hauner
110e627d7b
Add basic, not-useful websocket server 2019-09-14 10:34:03 -04:00
Paul Hauner
2676c8a62d
Remove some dust code from beacon chain 2019-09-13 18:33:17 -04:00
Luke Anderson
23ce271b5f
Return HTTP 202 to indicate processing error.
- A processing error of a validator's block or attestation should not prevent publishing. Now a 202 error is returned, to indicate that it has not been processed, but has still been published.
 - Added a publish_attestation function to the API, handling POST requests for /beacon/validator/attestation.
2019-09-13 21:22:32 +10:00
Luke Anderson
00a5f003c4
Merge branch 'api-alignment' into validator-post-fns 2019-09-13 20:53:38 +10:00
Luke Anderson
ec65934061
Merge branch 'interop' into api-alignment 2019-09-13 20:53:20 +10:00
Luke Anderson
d3ce182ddc
Renamed 'InvalidQueryParams' to 'BadRequest', since it is a more general error that is returned in a number of cases. 2019-09-13 20:52:12 +10:00
Luke Anderson
f48311900e
Restructured response builder to give YAML or JSON when SSZ is not available, not just JSON. 2019-09-13 20:42:56 +10:00
Luke Anderson
1dd86baf1a
Cleaning up the rest of the API functions.
- Removed all unused imports
 - Fixed random compiler errors
 - Removed all of the 'sucess_response' helpers.
 - Enabled all of the API endpoints again, wrapping in 'into_boxfut'
 - Tidied up /metrics endpoint
 - Added a 'body_text' part to ResponseBuilder, mainly for the Prometheus /metrics endpoint
 - Cleaned up the unnecessary helpers::* imports, to be more explicit.
2019-09-13 19:38:40 +10:00
Luke Anderson
006350c0cd
Fixed small bug with get_fork function 2019-09-13 19:14:09 +10:00
Luke Anderson
91f5f17566
Removing 'success_respons' in favour of 'ResponseBuilder' 2019-09-13 19:10:11 +10:00
Luke Anderson
09e11dd081
Merge branch 'api-alignment' into validator-post-fns 2019-09-13 19:06:32 +10:00
Luke Anderson
0b2f3bbf00
Updated another function for head_state (missing from previous commit). 2019-09-13 18:59:01 +10:00
Luke Anderson
b0e3ce7885
Addressed Paul's comments regarding head_state. 2019-09-13 18:58:08 +10:00
Paul Hauner
7c211f3791
Add explicit fails for 32bit architecture 2019-09-12 10:40:29 -04:00
Luke Anderson
cd8f40b4b7
Getting regular endpoint functions to return futures.
- Wrapped endpoint functions in new into_boxfut function
 - Undid changes to Network API service, now returning ApiResult again.
 - Cleaning up of functions, and removal of success_response functions in updated endpoints.
 - A bunch of other clean-ups.
2019-09-12 15:20:31 +10:00
Luke Anderson
82f4303787
Merge branch 'api-alignment' into validator-post-fns 2019-09-12 15:12:54 +10:00
Luke Anderson
ea9d2285c3
Merge branch 'interop' into validator-post-fns. 2019-09-12 12:50:04 +10:00
Luke Anderson
c254ac8c2e
Separated acquisition of BeaconChain and BeaconState. 2019-09-12 01:44:45 +10:00
Luke Anderson
d9b7e684c9
Merge branch 'interop' into api-alignment 2019-09-12 01:32:29 +10:00
Luke Anderson
0bd8187ff6
Revert most of commit ebd97730d5.
- Undoing making of the beacon chain read stuff return futures, instead dealing with it elsewhere.
2019-09-11 18:21:51 +10:00
Luke Anderson
2739ee83f9
Further restructuring futures API.
- Adding try_future! macros where necessary
 - Returning ApiResult and mapping it to future instead
 - Upgrading POST publish block to return a future directly
2019-09-11 18:02:00 +10:00
Paul Hauner
5d91d59481
Fix deadlock on becaon chain head 2019-09-10 22:42:07 -04:00
Paul Hauner
3fe61f5044
Add additional logs to validator client 2019-09-10 17:40:21 -04:00
Paul Hauner
b33f9c2bc9
Merge branch 'interop' of github.com:sigp/lighthouse into interop 2019-09-10 12:18:37 -04:00
Paul Hauner
8c5a8034b6
Add whiteblock script, CLI options to support it 2019-09-10 12:13:54 -04:00
Age Manning
04b43a51f9
Updates gossipsub to LRUcache 2019-09-11 02:07:39 +10:00
Luke Anderson
ebd97730d5
Converted the Beacon API service to Futures
- Added SSZ encode for HeadResponse
 - Converted all of the /beacon/ endpoints to return BoxFut instead of ApiResult
 - Wrapped all of the '?'s in a new macro try_future!()
 - Copied the try macro to try_future, so that a boxed future can easily be returned.
 - Replaced all of the response serializations to use the new success_response
2019-09-11 01:43:49 +10:00
Luke Anderson
b8667217f0
Made async functions work!
- Cleaned up imports
 - Moved ApiError and such to it's own error.rs
 - Obsoleted 'success_response' in favour of new async regular and json only flavours
 - Made ApiError work async and be derived from hyper errors
 - Added a check to ensure an error is thrown if a non-json encoding is requested on a json-only function
 - Made all the individual service functions return futures (only node and network for now)
2019-09-11 00:40:22 +10:00
Luke Anderson
b0090df543
Getting the restructured ApiService to work. 2019-09-10 19:30:36 +10:00
Michael Sproul
4dcad27381
Fix ApiService woes (hopefully) 2019-09-10 17:27:28 +10:00
Luke Anderson
576712cefe
WIP: Trying to get futures to work... 2019-09-10 17:16:41 +10:00
Luke Anderson
965d6f1df9
WIP: More restructuring to have ApiService be a future. 2019-09-10 15:35:54 +10:00
Luke Anderson
405a59e8b9
WIP: Trying to restructure ApiService to be async. 2019-09-10 10:56:50 +10:00
Luke Anderson
ca9094e79a
WIP: Made block publishing validator function, which sends to a network channel. Untested. 2019-09-10 10:54:37 +10:00
Paul Hauner
5de80f2799
Add extra logging when new head found 2019-09-09 17:12:47 -04:00
Paul Hauner
c931446fb1
Merge branch 'interop' of github.com:sigp/lighthouse into interop 2019-09-09 16:44:12 -04:00
Paul Hauner
66fd1586ca
Add more logging around block/attn production 2019-09-09 16:43:08 -04:00
Age Manning
60f37789a6
Allowing lighthouse to suit Alex's non-spec shinanigans 2019-09-10 03:57:44 +10:00
Luke Anderson
0136eb33b0
WIP: Added POST functionality for pusblish_beacon_block. Currently doesn't compile, struggling with the borrow checker. 2019-09-09 12:54:14 +10:00
Luke Anderson
99c673045c
Moved chain/cache building into separate function, and made sure that all REST API endpoints are using this function to get the state. 2019-09-09 12:10:41 +10:00
Paul Hauner
7b7a44e2f2
Add const to control writing of ssz files 2019-09-08 21:57:48 -04:00
Luke Anderson
a2267dc4d3
Merge branch 'interop' into api-alignment 2019-09-09 11:52:41 +10:00
Paul Hauner
09b0db2535
Change which pre state is logged 2019-09-08 21:11:16 -04:00
Paul Hauner
e7b324966d
Log all states and blocks processed 2019-09-08 20:55:15 -04:00
Paul Hauner
37cd98f3ce
Add additional logging for gossip messages 2019-09-08 18:51:24 -04:00
Paul Hauner
92c16bb911
Add extra logs to gossip object processing 2019-09-08 14:20:48 -04:00
Luke Anderson
99414155d1
Merging interop into api-alignment; fixing conflicts. 2019-09-09 03:02:41 +10:00
Paul Hauner
834c36d602
Merge branch 'interop' of github.com:sigp/lighthouse into interop 2019-09-08 12:25:35 -04:00
Paul Hauner
6a870d468c
Add ssz_fixed_len method to ssz::Encode 2019-09-08 12:23:37 -04:00
Age Manning
e8619399f2
Patch to correct for single byte RPC responses 2019-09-08 07:10:36 +10:00
Age Manning
69442a2ab3
Correct warnings 2019-09-08 01:57:56 +10:00
Age Manning
1eafa63c2a
Merge branch 'sync-threading' into interop 2019-09-07 09:32:09 +10:00
Age Manning
04b47a357b
Correct bugs in new sync threading 2019-09-07 09:31:05 +10:00
Age Manning
812e1fbe26
Implements a new thread dedicated for syncing 2019-09-07 00:28:54 +10:00
Luke Anderson
e6a0d038e9
Added YAML types for list of validators and added some logging to duties function. 2019-09-06 14:10:49 +10:00
Paul Hauner
8b69a48fc5
Allow validator client to start before genesis 2019-09-06 10:03:45 +10:00
Age Manning
ee25766cae
Correct recent beacon block request bug 2019-09-05 22:18:17 +10:00
Age Manning
a3877b6135
Updates syncing stability, fixes large RPC message codec, corrects beacon chain referencing 2019-09-05 08:07:57 +10:00
Age Manning
8256621230
Merge message validation 2019-09-05 03:06:57 +10:00
Age Manning
2a1d6587a7
Merge latest interop 2019-09-05 03:03:45 +10:00
Age Manning
e7ab89a783
Adds gossipsub object validation and verification 2019-09-05 02:06:39 +10:00
Luke Anderson
32ca8e951d
Updated content-type acceptance and returning, mainly for /spec/eth2_config 2019-09-05 00:36:06 +10:00
Luke Anderson
bf2f459773
Extended API
- Added a /beacon/validators function, to list all validators active in a particular epoch
 - Moved 'get_genesis_state' function, to align with router.
 - Added content-type for error responses
 - Tried adding a cache update call to fix issue getting validator duties (this is WIP)
2019-09-04 23:03:05 +10:00
Luke Anderson
eeba69cd0f
Moved beacon chain from request functionality into its own function. 2019-09-04 22:03:55 +10:00
Luke Anderson
01b652ab0e
Fixed merge conflict, updated calls.
- Fixed function re-names
 - Updated the function which gets the beacon_chain from the request, and makes sure caches are updated.
 - Updated this api-alignment branch with the new code from interop branch.
2019-09-04 16:06:14 +10:00
Paul Hauner
3b40b691ab
Download ENR during bootstrap 2019-09-04 15:58:51 +10:00
Luke Anderson
28a2ce2bdc
Fix formatting with rustfmt. 2019-09-04 14:19:48 +10:00
Paul Hauner
572df4f37e
Make bootstrapper block til connection established 2019-09-04 13:56:30 +10:00
Luke Anderson
0c1ceab527
Addressed Paul's suggestions.
- Updated some comments.
 - Replaced match statements with map functions.
2019-09-04 13:43:45 +10:00
Paul Hauner
009a7eb9c7
Fix bug with invalid state root 2019-09-04 12:04:15 +10:00
Luke Anderson
b432c8c58c
Replaced unnecessary match statements with map_err and ok_or 2019-09-04 11:18:29 +10:00
Luke Anderson
dcd074877b
Removed block publish feature, since it's incomplete currently. 2019-09-04 10:57:09 +10:00
Luke Anderson
39a03ce586
Merge branch 'master' into api-alignment 2019-09-04 10:54:23 +10:00
Paul Hauner
7edc5f37b9
Move BeaconChainHarness to interop spec 2019-09-04 10:25:30 +10:00
Paul Hauner
2706025a34
Move data dir cleaning in node runtime start 2019-09-04 09:07:33 +10:00
Paul Hauner
d80d9dba4c
Add flag for web3 server 2019-09-03 16:40:53 +10:00
Luke Anderson
c13f27e245
Updating REST API.
- Made /beacon/state return the current 'head' state when no parameters are provided.
 - Added some of the YAML api spec stuff to the /beacon/state endpoint in the rest_api spec.
2019-09-03 16:30:04 +10:00
Paul Hauner
29584ca087
Add docs to Eth1Chain 2019-09-03 15:56:11 +10:00
Paul Hauner
31557704eb
Add Eth1Chain member to BeaconChain 2019-09-03 15:52:25 +10:00
Paul Hauner
ab2b8accd4
Add first pass at Eth1Chain trait 2019-09-03 15:22:00 +10:00
Paul Hauner
1b4679e5bc
Improve block processing outcomes enum 2019-09-03 14:18:45 +10:00
Paul Hauner
969b6d7575
Tidy BeaconStateCow 2019-09-03 09:50:15 +10:00
Age Manning
13b5df56b3
Account manager, bootnodes, RPC display and sync fixes 2019-09-03 07:50:44 +10:00
Age Manning
cd7b6da88e
Updates syncing, corrects CLI variables 2019-09-03 00:34:41 +10:00
Paul Hauner
a0e019b4d7
Fix interop eth1 blockhash 2019-09-02 18:19:30 +10:00
Paul Hauner
70f4052b2e
Allow starting testnet from JSON state 2019-09-02 15:58:53 +10:00
Paul Hauner
a8fce42ea9
Merge branch 'master' into interop 2019-09-02 15:15:43 +10:00
Paul Hauner
bfbe776712
Add special genesis state API endpoint 2019-09-02 15:07:48 +10:00
Paul Hauner
11a1505784
Allow starting from SSZ genesis state 2019-09-02 15:07:32 +10:00
Paul Hauner
6c50758bdf
Add ResponseBuilder to rest_api 2019-09-02 14:29:36 +10:00
Paul Hauner
d4bf1390c9
Add interop eth1 data stub 2019-09-02 11:39:28 +10:00
Paul Hauner
82dc84ebbf
Add slot-time CLI argument 2019-09-02 10:22:29 +10:00
Paul Hauner
4aa12dc408
Set a default BN bootstrap address 2019-09-02 09:59:52 +10:00
Age Manning
c1614b110b
Merge interop branch 2019-09-02 05:45:00 +10:00
Age Manning
74baeb4d08
WIP - Upgrade Sync algorithm 2019-09-02 05:38:11 +10:00
Paul Hauner
a78b030f58
Fix rpc port-bump bug 2019-09-01 22:22:09 +10:00
Paul Hauner
960082fe4e
Set all listen addresses to 0.0.0.0 for testnets 2019-09-01 22:21:46 +10:00
Paul Hauner
82b4a1b3eb
Fix multiple data_dirs in config 2019-09-01 22:21:29 +10:00
Paul Hauner
6db1a19169
Remove stray dbg! 2019-09-01 22:20:31 +10:00
Paul Hauner
4a69d01a37
Add first changes to validator CLI 2019-09-01 19:33:43 +10:00
Paul Hauner
fa6ba51eb7
Make gRPC and HTTP on by default 2019-09-01 19:33:21 +10:00
Paul Hauner
543e9457b7
Move bootstrapper into own crate 2019-09-01 19:32:57 +10:00
Luke Anderson
632c13a9ec
Fixing some API bits
- Adding the validator routes into the main function.
 - Fixing the setting of the aggregation bits, and handling errors correctly.
 - Rust format fixes, and addressing compiler warnings.
2019-09-01 15:41:03 +10:00
Luke Anderson
8ea1167563
Factored out getting beacon_chain from request into it's own function. 2019-09-01 15:09:01 +10:00
Luke Anderson
5ee1bb20b7
WIP: Furthered attestation production for validator. 2019-08-31 23:56:35 +10:00
Paul Hauner
5f0509be50
Improve and extend CLI interface 2019-08-31 12:34:27 +10:00
Paul Hauner
6ba093d14f
Add warning when disconnecting peer 2019-08-30 17:06:59 +10:00
Paul Hauner
9ffb6d0fe1
Fix fake_crypto test fails 2019-08-30 15:49:06 +10:00
Paul Hauner
6234adc0d6
Add interop-spec genesis procedure 2019-08-30 15:33:34 +10:00
Paul Hauner
3365106340
Fix bug with block production 2019-08-30 11:04:15 +10:00
Paul Hauner
31bbb0f573
Modify RPC duties endpoint 2019-08-30 01:51:18 +10:00
Paul Hauner
8060cd8f5c
Change RPC slot behaviour 2019-08-30 01:14:57 +10:00
Paul Hauner
ea562595ed
Fix bugs with gRPC API 2019-08-30 00:46:25 +10:00
Paul Hauner
ae114889c1
Fix bugs from removing speculative state 2019-08-30 00:24:46 +10:00
Age Manning
192380cb58
Ethereum 2.0 Network Specification Upgrade (#510)
Updates lighthouse to the latest networking spec

-  Sync re-write (#496)
-  Updates to the latest eth2 networking spec (#495)
-  Libp2p updates and improvements
2019-08-29 13:23:28 +02:00
Paul Hauner
682081ef07
Add first pass at removing speculative state 2019-08-29 19:14:52 +10:00
Paul Hauner
29c8477242
Merge branch 'master' into interop 2019-08-29 16:00:22 +10:00
Paul Hauner
75ac21604f
Add long minutes CLI flag 2019-08-29 15:03:52 +10:00
Paul Hauner
314780e634
Allow for customizable recent genesis window 2019-08-29 14:59:32 +10:00
Luke Anderson
eaec5e7b69
Start implementation of 'get attstation' validator function.
- Created new /beacon/validator/attestation endpoint
 - Updated some small issues with the API spec.
2019-08-29 14:58:49 +10:00
Paul Hauner
8cfbe8bbfb
Change seconds_per_slot to milliseconds_per_slot 2019-08-29 14:32:21 +10:00
Paul Hauner
7d03806107
Upgrade codebase to new SlotClock API 2019-08-29 14:26:30 +10:00
Luke Anderson
b9276da9db
Flesh spec. & update display bugs.
- Add correct string formatting when incorrect parameters provided.
 - Fill /beacon/block and /beacon/block_root endpoints
 - Add 500 error responses to endpoints as appropriate
2019-08-29 13:36:51 +10:00
Paul Hauner
4557d2c84b
Merge branch 'interop' into millsecond-slots 2019-08-29 13:27:06 +10:00
Paul Hauner
bcd53a8b10
Migrate codebase across to new SlotClock API 2019-08-29 13:25:55 +10:00
Luke Anderson
ca07d72453
Removed methods for 'chain_id', since this is no longer applicable to ETH2. 2019-08-29 12:42:45 +10:00
Paul Hauner
bcffe42712
Bulk signature verification (#507)
* Add basic block processing benches

* Start reviving state processing benches

* Fix old block builders

* Add optimization for faster pubkey add

* Tidy benches, add another

* Add extra block processing bench

* Start working on faster BLS scheme

* Add partially complete sig verify optimization

* Add .gitignore to state processing

* Add progress on faster signature verification

* Fix SignatureSet for fake_crypto

* Tidy attester slashings sig set

* Tidy bulk signature verifier

* Refactor signature sets to be cleaner

* Start threading SignatureStrategy through code

* Add (empty) test dir

* Move BenchingBlockBuilder

* Add initial block signature verification tests

* Add tests for bulk signature verification

* Start threading SignatureStrategy in block proc.

* Refactor per_block_processing errors

* Use sig set tuples instead of lists of two

* Remove dead code

* Thread VerifySignatures through per_block_processing

* Add bulk signature verification

* Introduce parallel bulk signature verification

* Expand state processing benches

* Fix additional compile errors

* Fix issue where par iter chunks is 0

* Update milagro_bls dep

* Remove debugs, code fragment in beacon chain

* Tidy, add comments to block sig verifier

* Fix various PR comments

* Add block_root option to per_block_processing

* Fix comment in block signature verifier

* Fix comments from PR review

* Remove old comment

* Fix comment
2019-08-29 11:34:25 +10:00
Luke Anderson
faef347d18
Fleshed out some API endpoints.
- Added the /beacon/validator/block endpoint for GET (untested)
 - Added the /beacon/fork endpoint for GET
 - Cleaned up a bunch of unused imports & variables
 - Removed '/network/block_discovery' endpoint
2019-08-28 23:33:34 +10:00
Luke Anderson
77e2f576af
Further aligning the API & implementation.
- Completed implementation of /beacon/head
 - renamed 'latest_finalized_checkpoint' to 'current_finalized_checkpoint' for consistency
 - Reorganised list of endpoints in both spec & router so that they match
 - Fixed the content-type modifications for /metrics
 - Added a new 'RFC' tag to the spec, to tag things that we have not implemented and aren't sure if it's useful.
 - Moved 'deposit_contract' under /spec
2019-08-28 21:25:38 +10:00
Luke Anderson
16ec330a79
Started aligning API spec with implementation.
- Adding some missing fields to structs
 - Rearranged the endpoints in the rest_api router, and renamed, using an 'implementation_pending' function
 - Added 'content-type' headers, to distinguish difference with /node/metrics
 - Updated OpenAPI spec to v0.2.0
    - Split /node/fork into /node/chain_id and /beacon/fork
    - Moved /metrics to /node/metrics
    - Added example to /node/metrics, since it's text/plain
    - Moved /node/network to just /network
    - Added lots of stubs for endpoints which exist in the router
    - Reordered large parts of the OpenAPI spec
    - Moved /chain/beacon/... to just /beacon/...
2019-08-28 02:05:19 +10:00
Luke Anderson
328f11d564
Validator API (#504)
* Implemented more REST API endpoints.
 - Added many of the endpoints, which return 501 - Not Implemented
 - Created helper function to return a not implemented error
 - Created a parse_pubkey function to get a PublicKey from a hex string
 - Created a HTTP handler for the validator endpoints
 - Started implementing validator/duties endpoint.

* Fleshed out get validator duties.
 - Re-implemented the get validator duties function for the REST API
 - Added an 'as_hex_string' function to FakePublicKey, beacuse it was missing.

* Fixed small caching/state bug.

* Extended to array of API inputs.
 - Created  function for getting arrays from GET parameters.
 - Extended get validator duties function to support array of validator duties.

* Tidy API to be more consistent with recent decisions

* Addressing Paul's comments.
 - Cleaning up function to get list of proposers.
 - Removing unnecessary serde annotations
 - Clarifying error messages
 - Only accept pubkeys if they are '0x' prefixed.

* Fixed formatting with rustfmt.
2019-08-28 00:40:35 +10:00
Paul Hauner
3207eb1d68
Merge branch 'interop' of github.com:sigp/lighthouse into interop 2019-08-27 11:20:35 +10:00
Paul Hauner
4e0ac0169b
Merge branch 'fix-fork-choice' into interop 2019-08-27 11:20:16 +10:00
Paul Hauner
ed6c39e25a
Add log for fork choice integrity in beacon chain 2019-08-27 11:19:50 +10:00
Paul Hauner
7f6b700b98
Remove old git merge relic 2019-08-27 00:05:25 +10:00
Paul Hauner
6875ae8af5
Pull Eth2Config during bootstrap 2019-08-27 00:04:15 +10:00
Paul Hauner
81c898586e
Merge branch 'eth2-network-spec' into interop 2019-08-26 17:44:05 +10:00
Paul Hauner
901393b664
Clean datadir after config files have been loaded 2019-08-26 16:02:05 +10:00
Paul Hauner
39be2ed1d2
Improve CLI error messages 2019-08-26 15:57:40 +10:00
Paul Hauner
bab1f2b064
Rename CLI flag 2019-08-26 15:51:11 +10:00
Paul Hauner
b58aa1d148
Add custom config options to testnet sub-cmd 2019-08-26 15:47:03 +10:00
Paul Hauner
cf435d9653
Refactor beacon chain start code 2019-08-26 14:45:49 +10:00
Paul Hauner
140c677a38
Add much more progress to new CLI setup 2019-08-25 12:14:04 +10:00
Age Manning
1bea1755c4
Remove redundant code 2019-08-25 10:13:17 +10:00
Paul Hauner
7fd7aa2cdb
Tidy ConfigBuilder 2019-08-25 10:09:51 +10:00
Age Manning
9cdcc7d198
Update to latest libp2p 2019-08-25 10:02:54 +10:00
Paul Hauner
66d7838707
Remove GenesisConfig, add BeaconChainStartMethod 2019-08-25 09:43:03 +10:00
Age Manning
72cf50d904
Update to latest master 2019-08-25 09:06:26 +10:00
Age Manning
7ee080db60
Updated syncing algorithm 2019-08-25 08:25:54 +10:00
Age Manning
0d56df474a
Main batch sync debugging 2019-08-25 00:27:47 +10:00
Age Manning
b078385362
Improved syncing compilation issues 2019-08-24 01:09:29 +10:00
Paul Hauner
cdf3ade63f
Add further CLI progress 2019-08-23 18:23:58 +10:00
Paul Hauner
453c8e2255
Re-arrange CLI to suit new "testnet" pattern 2019-08-23 16:39:32 +10:00
Paul Hauner
0c3fdcd57c
Bootstrap (#501)
* Renamed fork_choice::process_attestation_from_block

* Processing attestation in fork choice

* Retrieving state from store and checking signature

* Looser check on beacon state validity.

* Cleaned up get_attestation_state

* Expanded fork choice api to provide latest validator message.

* Checking if the an attestation contains a latest message

* Correct process_attestation error handling.

* Copy paste error in comment fixed.

* Tidy ancestor iterators

* Getting attestation slot via helper method

* Refactored attestation creation in test utils

* Revert "Refactored attestation creation in test utils"

This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306.

* Integration tests for free attestation processing

* Implicit conflicts resolved.

* formatting

* Do first pass on Grants code

* Add another attestation processing test

* Tidy attestation processing

* Remove old code fragment

* Add non-compiling half finished changes

* Simplify, fix bugs, add tests for chain iters

* Remove attestation processing from op pool

* Fix bug with fork choice, tidy

* Fix overly restrictive check in fork choice.

* Ensure committee cache is build during attn proc

* Ignore unknown blocks at fork choice

* Various minor fixes

* Make fork choice write lock in to read lock

* Remove unused method

* Tidy comments

* Fix attestation prod. target roots change

* Fix compile error in store iters

* Reject any attestation prior to finalization

* Begin metrics refactor

* Move beacon_chain to new metrics structure.

* Make metrics not panic if already defined

* Use global prometheus gather at rest api

* Unify common metric fns into a crate

* Add heavy metering to block processing

* Remove hypen from prometheus metric name

* Add more beacon chain metrics

* Add beacon chain persistence metric

* Prune op pool on finalization

* Add extra prom beacon chain metrics

* Prefix BeaconChain metrics with "beacon_"

* Add more store metrics

* Add basic metrics to libp2p

* Add metrics to HTTP server

* Remove old `http_server` crate

* Update metrics names to be more like standard

* Fix broken beacon chain metrics, add slot clock metrics

* Add lighthouse_metrics gather fn

* Remove http args

* Fix wrong state given to op pool prune

* Make prom metric names more consistent

* Add more metrics, tidy existing metrics

* Fix store block read metrics

* Tidy attestation metrics

* Fix minor PR comments

* Fix minor PR comments

* Remove duplicated attestation finalization check

* Remove awkward `let` statement

* Add first attempts at HTTP bootstrap

* Add beacon_block methods to rest api

* Fix serde for block.body.grafitti

* Allow travis failures on beta (see desc)

There's a non-backward compatible change in `cargo fmt`. Stable and beta
do not agree.

* Add network routes to API

* Fix rustc warnings

* Add best_slot method

* Add --bootstrap arg to beacon node

* Get bootstrapper working for ENR address

* Store intermediate states during block processing

* Allow bootstrapper to scrape libp2p address

* Update bootstrapper libp2p address finding

* Add comments

* Tidy API to be more consistent with recent decisions

* Address some review comments

* Make BeaconChainTypes Send + Sync + 'static

* Add `/network/listen_port` API endpoint

* Abandon starting the node if libp2p doesn't start

* Update bootstrapper for API changes

* Remove unnecessary trait bounds
2019-08-23 15:53:53 +10:00
Paul Hauner
bb63d300f2
Merge branch 'bootstrap' into interop 2019-08-23 13:03:59 +10:00
Paul Hauner
a8de94ca13
Remove unnecessary trait bounds 2019-08-23 13:02:17 +10:00
Paul Hauner
a358bbc1b1
Update bootstrapper for API changes 2019-08-23 12:45:31 +10:00
Paul Hauner
7d11d78299
Abandon starting the node if libp2p doesn't start 2019-08-23 12:43:34 +10:00
Paul Hauner
94d987cb6a
Add /network/listen_port API endpoint 2019-08-23 12:13:03 +10:00
Paul Hauner
112e323c1e
Merge branch 'bootstrap' into interop 2019-08-22 17:50:05 +10:00
Paul Hauner
11dc72a442
Start implementing BeaconChainBuilder 2019-08-22 17:48:13 +10:00
Paul Hauner
853344af8a
Make BeaconChainTypes Send + Sync + 'static 2019-08-22 16:34:21 +10:00
Paul Hauner
5a34f86e77
Address some review comments 2019-08-22 16:14:51 +10:00
Paul Hauner
b912e26b79
Tidy API to be more consistent with recent decisions 2019-08-22 14:37:47 +10:00
Paul Hauner
a8daf46d5f
Add comments 2019-08-21 14:48:49 +10:00
Paul Hauner
a6da933886
Merge branch 'master' into bootstrap 2019-08-21 14:16:19 +10:00
Paul Hauner
c4ced3e0d2
Fix block processing blowup, upgrade metrics (#500)
* Renamed fork_choice::process_attestation_from_block

* Processing attestation in fork choice

* Retrieving state from store and checking signature

* Looser check on beacon state validity.

* Cleaned up get_attestation_state

* Expanded fork choice api to provide latest validator message.

* Checking if the an attestation contains a latest message

* Correct process_attestation error handling.

* Copy paste error in comment fixed.

* Tidy ancestor iterators

* Getting attestation slot via helper method

* Refactored attestation creation in test utils

* Revert "Refactored attestation creation in test utils"

This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306.

* Integration tests for free attestation processing

* Implicit conflicts resolved.

* formatting

* Do first pass on Grants code

* Add another attestation processing test

* Tidy attestation processing

* Remove old code fragment

* Add non-compiling half finished changes

* Simplify, fix bugs, add tests for chain iters

* Remove attestation processing from op pool

* Fix bug with fork choice, tidy

* Fix overly restrictive check in fork choice.

* Ensure committee cache is build during attn proc

* Ignore unknown blocks at fork choice

* Various minor fixes

* Make fork choice write lock in to read lock

* Remove unused method

* Tidy comments

* Fix attestation prod. target roots change

* Fix compile error in store iters

* Reject any attestation prior to finalization

* Begin metrics refactor

* Move beacon_chain to new metrics structure.

* Make metrics not panic if already defined

* Use global prometheus gather at rest api

* Unify common metric fns into a crate

* Add heavy metering to block processing

* Remove hypen from prometheus metric name

* Add more beacon chain metrics

* Add beacon chain persistence metric

* Prune op pool on finalization

* Add extra prom beacon chain metrics

* Prefix BeaconChain metrics with "beacon_"

* Add more store metrics

* Add basic metrics to libp2p

* Add metrics to HTTP server

* Remove old `http_server` crate

* Update metrics names to be more like standard

* Fix broken beacon chain metrics, add slot clock metrics

* Add lighthouse_metrics gather fn

* Remove http args

* Fix wrong state given to op pool prune

* Make prom metric names more consistent

* Add more metrics, tidy existing metrics

* Fix store block read metrics

* Tidy attestation metrics

* Fix minor PR comments

* Allow travis failures on beta (see desc)

There's a non-backward compatible change in `cargo fmt`. Stable and beta
do not agree.

* Tidy `lighthouse_metrics` docs

* Fix typo
2019-08-19 21:02:34 +10:00
Age Manning
c259d6c006
First draft sync re-write. WIP 2019-08-18 03:36:13 +10:00
Paul Hauner
7cd963e6bb
Update bootstrapper libp2p address finding 2019-08-15 18:48:39 +10:00
Paul Hauner
ce37f95861
Allow bootstrapper to scrape libp2p address 2019-08-15 16:41:02 +10:00
Paul Hauner
4678524659
Store intermediate states during block processing 2019-08-15 14:52:00 +10:00
Paul Hauner
b244826749
Get bootstrapper working for ENR address 2019-08-15 13:58:04 +10:00
Paul Hauner
fda208b103
Add --bootstrap arg to beacon node 2019-08-15 12:48:34 +10:00
Paul Hauner
c97b3b20cb
Add best_slot method 2019-08-14 20:58:51 +10:00
Paul Hauner
bb166a2599
Fix rustc warnings 2019-08-14 18:58:01 +10:00
Paul Hauner
c93d2baa91
Add network routes to API 2019-08-14 18:24:42 +10:00
Paul Hauner
f2dedfac50
Merge branch 'block-processing-times' into bootstrap 2019-08-14 12:32:16 +10:00
Paul Hauner
072eb82fe4
Merge branch 'master' into block-processing-times 2019-08-14 12:25:28 +10:00
Paul Hauner
2bf0d5c071
Add beacon_block methods to rest api 2019-08-14 11:22:43 +10:00
Paul Hauner
cd26a19a70
Attestation processing (#497)
* Renamed fork_choice::process_attestation_from_block

* Processing attestation in fork choice

* Retrieving state from store and checking signature

* Looser check on beacon state validity.

* Cleaned up get_attestation_state

* Expanded fork choice api to provide latest validator message.

* Checking if the an attestation contains a latest message

* Correct process_attestation error handling.

* Copy paste error in comment fixed.

* Tidy ancestor iterators

* Getting attestation slot via helper method

* Refactored attestation creation in test utils

* Revert "Refactored attestation creation in test utils"

This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306.

* Integration tests for free attestation processing

* Implicit conflicts resolved.

* formatting

* Do first pass on Grants code

* Add another attestation processing test

* Tidy attestation processing

* Remove old code fragment

* Add non-compiling half finished changes

* Simplify, fix bugs, add tests for chain iters

* Remove attestation processing from op pool

* Fix bug with fork choice, tidy

* Fix overly restrictive check in fork choice.

* Ensure committee cache is build during attn proc

* Ignore unknown blocks at fork choice

* Various minor fixes

* Make fork choice write lock in to read lock

* Remove unused method

* Tidy comments

* Fix attestation prod. target roots change

* Fix compile error in store iters

* Reject any attestation prior to finalization

* Fix minor PR comments

* Remove duplicated attestation finalization check

* Remove awkward `let` statement
2019-08-14 10:55:24 +10:00
Paul Hauner
4f98a3985f
Add first attempts at HTTP bootstrap 2019-08-14 10:36:55 +10:00
Paul Hauner
82e8aafb01
Remove awkward let statement 2019-08-13 19:59:29 +10:00
Paul Hauner
8fb9e1f648
Remove duplicated attestation finalization check 2019-08-13 19:48:03 +10:00
Paul Hauner
6cd0af766e
Fix minor PR comments 2019-08-13 19:38:33 +10:00
Paul Hauner
b7e43b56f9
Fix minor PR comments 2019-08-13 19:37:14 +10:00
Paul Hauner
e410e89898
Merge branch 'block-processing-times' of github.com:sigp/lighthouse into block-processing-times 2019-08-13 18:26:07 +10:00
Paul Hauner
341a83b9e8
Tidy attestation metrics 2019-08-13 16:17:11 +10:00
Paul Hauner
a3e464078a
Fix store block read metrics 2019-08-13 13:00:01 +10:00
Paul Hauner
b076b07022
Add more metrics, tidy existing metrics 2019-08-13 12:11:18 +10:00
Paul Hauner
e369e293a5
Make prom metric names more consistent 2019-08-13 09:20:39 +10:00
Paul Hauner
24b2f83713
Fix wrong state given to op pool prune 2019-08-13 07:35:52 +10:00
Age Manning
d25ec359c8
Merge attestation processing 2019-08-12 22:25:25 +10:00
Age Manning
af5372fde4
Merge latest master 2019-08-12 22:24:20 +10:00
Age Manning
5d4d2f35e1
Initial sync re-write. WIP 2019-08-12 22:07:59 +10:00
Paul Hauner
d5d60874e5
Remove http args 2019-08-12 18:20:05 +10:00
Paul Hauner
7165598b7f
Add lighthouse_metrics gather fn 2019-08-12 18:19:50 +10:00
Paul Hauner
d7c546844c
Fix broken beacon chain metrics, add slot clock metrics 2019-08-12 17:44:47 +10:00
Paul Hauner
95a320817e
Update metrics names to be more like standard 2019-08-12 15:42:23 +10:00
Paul Hauner
6a1e5f6d26
Remove old http_server crate 2019-08-12 15:19:39 +10:00
Paul Hauner
af334b2cf0
Add metrics to HTTP server 2019-08-12 14:30:46 +10:00
Paul Hauner
cac0e5c832
Add basic metrics to libp2p 2019-08-12 14:16:20 +10:00
Paul Hauner
0b4a8893a4
Add more store metrics 2019-08-12 13:49:09 +10:00
Paul Hauner
913ee4694e
Prefix BeaconChain metrics with "beacon_" 2019-08-12 13:35:16 +10:00
Paul Hauner
7140dbc45d
Add extra prom beacon chain metrics 2019-08-12 13:26:58 +10:00
Paul Hauner
6150f0ae1a
Prune op pool on finalization 2019-08-11 18:29:11 +10:00
Paul Hauner
78db947e6e
Add beacon chain persistence metric 2019-08-11 18:28:57 +10:00
Paul Hauner
42d300bdc3
Add more beacon chain metrics 2019-08-11 17:49:32 +10:00
Paul Hauner
76f42ac7ff
Remove hypen from prometheus metric name 2019-08-11 16:15:26 +10:00
Paul Hauner
441eb41b6b
Add heavy metering to block processing 2019-08-11 15:53:34 +10:00
Paul Hauner
2108895fca
Unify common metric fns into a crate 2019-08-11 15:34:10 +10:00
Paul Hauner
36ff115b04
Use global prometheus gather at rest api 2019-08-11 14:46:20 +10:00
Paul Hauner
e33d0703ef
Make metrics not panic if already defined 2019-08-11 14:43:31 +10:00
Paul Hauner
9995b390b5
Move beacon_chain to new metrics structure. 2019-08-11 14:11:13 +10:00
Paul Hauner
48733917be
Begin metrics refactor 2019-08-11 12:12:19 +10:00
Paul Hauner
4020d13064
Reject any attestation prior to finalization 2019-08-11 09:34:49 +10:00
Paul Hauner
6c9ebf4b96
Fix compile error in store iters 2019-08-11 09:15:39 +10:00
Paul Hauner
cd11eb15a5
Merge branch 'master' into attestation-processing 2019-08-11 09:08:26 +10:00
Paul Hauner
04bef689e3
Fix attestation prod. target roots change 2019-08-10 17:47:34 +10:00
Paul Hauner
963fb7bc87
Tidy comments 2019-08-10 17:36:53 +10:00
Paul Hauner
1beab66078
Remove unused method 2019-08-10 17:23:45 +10:00
Paul Hauner
989e2727d7
Changes to rest_api (#480)
* Add half-finished rest api changes

* Add basic, messy changes to rest api

* Fix expect() in ApiRequest

* Remove ApiRequest, add route for beacon state

* Tidy rest api, add get state from root

* Add api method for getting state roots by slot

* Add test for URL helper

* Simplify state_at_slot fn

* Add tests for rest api helper fns

* Add extra tests for parse root

* Fix clippy lints

* Fix compile error in rest api

* Update test to new ethereum-types
2019-08-10 17:15:15 +10:00
Age Manning
468015f9bb Initial Interop Updates (#492)
* Add interop chain spec and rename chain_id

* Add ability to connect to raw libp2p nodes

* Adds Identify protocol, clean up RPC protocol name handling

* Update to latest libp2p, gossipsub improvements

* Updates to latest interop branch.

- Shifts decoding of objects into message handler.
- Updates to latest interop gossipsub.
- Adds interop spec constant.

* Configuration updates allow for verbosity CLI flag and spec constants

* Update submodules to master

* Correct minimal chainspec modifications

* Duplication of validator polls are no longer fatal

* Apply PR suggestions
2019-08-10 11:44:17 +10:00
Paul Hauner
64a6e1475c
Various minor fixes 2019-08-10 11:31:31 +10:00
Age Manning
66419d00ea
Remove redundant slog dependencies 2019-08-09 13:27:31 +10:00
Age Manning
ec73dfe90b
Starting of req/resp overhaul 2019-08-09 13:27:31 +10:00
Age Manning
5a74239ebc
Add decoding/encoding for extended gossip topics. Correct logging CLI 2019-08-09 13:27:31 +10:00
Age Manning
80f15f5d70
Correct gossipsub message encoding. Add extended topics 2019-08-09 13:27:31 +10:00
Age Manning
d83fa67068
Subscribe to all required gossipsub topics 2019-08-09 13:27:31 +10:00
Age Manning
ce50616032
Improve logging 2019-08-09 13:27:30 +10:00
Age Manning
3210489a36
Apply PR suggestions 2019-08-09 13:23:47 +10:00
Paul Hauner
f4121d9deb
Ignore unknown blocks at fork choice 2019-08-09 12:34:56 +10:00
Paul Hauner
67fe21c1c0
Ensure committee cache is build during attn proc 2019-08-09 12:32:32 +10:00
Paul Hauner
d191812d4b
Fix overly restrictive check in fork choice. 2019-08-09 12:23:10 +10:00
Paul Hauner
76bb671084
Fix bug with fork choice, tidy 2019-08-09 11:54:35 +10:00
Age Manning
826abea8e0
Merge latest 'master' 2019-08-09 10:46:49 +10:00
Paul Hauner
284166c7f8
Merge branch 'master' into attestation-processing 2019-08-08 17:11:19 +10:00
Paul Hauner
b1591c3c12
Remove attestation processing from op pool 2019-08-08 16:49:27 +10:00
Paul Hauner
7c134a7504
Simplify, fix bugs, add tests for chain iters 2019-08-08 16:47:24 +10:00
Paul Hauner
3a1f56a42e
Tidy ancestor iterators (#490)
* Tidy ancestor iterators

* Improve comments, remove code fragement
2019-08-08 12:28:10 +10:00