Commit Graph

3619 Commits

Author SHA1 Message Date
Paul Hauner
a413b43fed Add eth1 deposit confirmations (#1370)
## Issue Addressed

NA

## Proposed Changes

Allow `lighthouse account validator deposit` to await for confirmations after deposit submissions.

## Additional Info

NA
2020-07-27 00:08:12 +00:00
Michael Sproul
5f013548c0 Update to EF tests v0.12.2 (#1392)
Update the EF test vectors to v0.12.2 so that they include the new finality tests. Also, correct a typo that caused the epoch processing final update tests not to run on the minimal spec.
2020-07-26 23:19:49 +00:00
Paul Hauner
0b5be9b2c0
Add info about peer scoring to block/attestation errors (#1393)
* Add comments to `BlockError`

* Add `AttnError` comments

* Clean up
2020-07-26 13:16:49 +10:00
Paul Hauner
e5d9d6179f Add info about valid deposit count to logs (#1391)
## Issue Addressed

NA

## Proposed Changes

Adds a `valid_deposits` field to the logs whilst waiting for genesis:

```
Jul 25 11:02:25.631 INFO Waiting for more deposits               valid_deposits: 3085, total_deposits: 3188, min_genesis_active_validators: 16384, service: beacon
```

In this example we can see there are `3188` deposits, but only `3085` have valid signatures.

## Additional Info

NA
2020-07-25 04:44:10 +00:00
Paul Hauner
b73c497be2 Support multiple BLS implementations (#1335)
## Issue Addressed

NA

## Proposed Changes

- Refactor the `bls` crate to support multiple BLS "backends" (e.g., milagro, blst, etc).
- Removes some duplicate, unused code in `common/rest_types/src/validator.rs`.
- Removes the old "upgrade legacy keypairs" functionality (these were unencrypted keys that haven't been supported for a few testnets, no one should be using them anymore).

## Additional Info

Most of the files changed are just inconsequential changes to function names.

## TODO

- [x] Optimization levels
- [x] Infinity point: https://github.com/supranational/blst/issues/11
- [x] Ensure milagro *and* blst are tested via CI
- [x] What to do with unsafe code?
- [x] Test infinity point in signature sets
2020-07-25 02:03:18 +00:00
Paul Hauner
21bcc8848d Add caching for state.eth1_data_votes (#919)
## Issue Addressed

NA

## Proposed Changes

Adds additional tree hash caching for `state.eth1_data_votes`.

Presently, each time we tree hash the `BeaconState`, we recompute the `state.eth1_data_votes` tree in it's entirety. This is because we only previous had support for caching fixed-length lists.

This PR adds the `Eth1DataVotesTreeHashCache` which provides caching for the `state.eth1_data_votes` list. The cache is aware of `SLOTS_PER_ETH1_VOTING_PERIOD` and will reset itself whenever that boundary is crossed.

This cache adds a new (but somewhat fundamental) restriction to tree hash caching:

*For some state `s`, `s.tree_hash_cache` is only valid for `s` or descendants of `s` that have been reached via state transitions that are faithful to the specification (invalid blocks are permitted, as long as they are faithfully processed).*
2020-07-24 02:19:47 +00:00
blacktemplar
23a8f31f83 Fix clippy warnings (#1385)
## Issue Addressed

NA

## Proposed Changes

Fixes most clippy warnings and ignores the rest of them, see issue #1388.
2020-07-23 14:18:00 +00:00
divma
ba10c80633 Refactor inbound substream logic with async (#1325)
## Issue Addressed
#1112 

The logic is slightly different but still valid wrt to error handling.
- Inbound state is either Busy with a future that return the subtream (and info about the processing)
- The state machine works as follows:
  - `Idle` with pending responses => `Busy`
  - `Busy` => finished ? if so and there are new pending responses then `Busy`, if not then `Idle`
               => not finished remains `Busy`
- Add an `InboundInfo` for readability
- Other stuff:
  - Close inbound substreams when all expected responses are sent
  - Remove the error variants from `RPCCodedResponse` and use the codes instead
  - Fix various spelling mistakes because I got sloppy last time

Sorry for the delay

Co-authored-by: Age Manning <Age@AgeManning.com>
2020-07-23 12:30:43 +00:00
blacktemplar
3c4daec9af
replace max_peers cli argument by target_peers and use excess peers above target_peers capped by a new constant PEER_EXCESS_FACTOR (relative to target_peers) (#1383) 2020-07-23 13:55:36 +10:00
Pawan Dhananjay
3a888d6ef3 Fix early return from DepositLog parsing (#1382)
## Issue Addressed

N/A

## Proposed Changes

When parsing deposit logs, we were returning an error in case `PublicKeyBytes` or `SignatureBytes` didn't convert to valid bls `PublicKey` or `Signature` types. This would stall our import of deposit logs. 
Fixes this by returning `signature_is_valid` as `false` in `DepositLog` if the bytes are invalid `PublicKey/Signature` types.

Tested this fix on the Onyx deposit contract where the bug was observed and it works correctly as expected.
2020-07-22 10:24:37 +00:00
Akihito Nakano
41f7547645 Remove unused event handler function (#1377)
## Issue Addressed

`websocket_event_handler` has been unused since #1107.
2020-07-22 10:24:35 +00:00
Akihito Nakano
ea0e936ac4 Small improvement: encapsulate a public field (#1362)
## Issue Addressed

This PR makes the `Eth1Chain::use_dummy_backend` field private. I believe this could be good to ensure the consistency  of a Eth1Chain instance. 💡
2020-07-22 09:34:57 +00:00
Paul Hauner
e26da35cbf Introduce validator definition file for VC (#1357)
## Issue Addressed

NA

## Proposed Changes

- Introduces the `valdiator_definitions.yml` file which serves as an explicit list of validators that should be run by the validator client.
  - Removes `--strict` flag, split into `--strict-lockfiles` and `--disable-auto-discover`  
  - Adds a "Validator Management" page to the book.
- Adds the `common/account_utils` crate which contains some logic that was starting to duplicate across the codebase.

The new docs for this feature are the best description of it (apart from the code, I guess): 9cb87e93ce/book/src/validator-management.md

## API Changes

This change should be transparent for *most* existing users. If the `valdiator_definitions.yml` doesn't exist then it will be automatically generated using a method that will detect all the validators in their `validators_dir`.

Users will have issues if they are:

1. Using `--strict`.
1. Have keystores in their `~/.lighthouse/validators` directory that weren't being detected by the current keystore discovery method.

For users with (1), the VC will refuse to start because the `--strict` flag has been removed. They will be forced to review `--help` and choose an equivalent flag.

For users with (2), this seems fairly unlikely and since we're only in testnets there's no *real* value on the line here. I'm happy to take the risk, it would be a different case for mainnet.

## Additional Info

This PR adds functionality we will need for #1347.

## TODO

- [x] Reconsider flags
- [x] Move doc into a more reasonable chapter.
- [x] Check for compile warnings.
2020-07-22 09:34:55 +00:00
Michael Sproul
393782f632 Prefer viable nightly Rust builds for CI (#1379)
## Proposed Changes

CI is failing on PRs because of a regression in nightly Rust. This change forces `rustup` to install a version of the nightly compiler that can at least build `rustfmt`, which should reduce the frequency of this happening.

## Additional Info

Example failing run: https://github.com/sigp/lighthouse/runs/896694659

Alternatively, we could allow CI to succeed even when `cargo-udeps` fails, but I think we should avoid doing that unless we really have to.
2020-07-22 07:34:30 +00:00
blacktemplar
f61a7113ac
Do not send regular status updates during syncing (#1375) 2020-07-22 15:39:56 +10:00
Michael Sproul
2870172e0d Deny warnings on CI (#1372)
## Issue Addressed

Prevent CI from succeeding when there are warnings. Code can still be built and tested with warnings locally, but CI may fail during a Rust update (which is fine IMO).

## Proposed Changes

* Deny warnings for all stable compiler jobs on CI (excludes `cargo udeps`, which runs under nightly)
* Fix the warnings currently on `master` related to unnecessary `mem::replace`
2020-07-21 05:51:33 +00:00
Age Manning
0620f54f2f
Update discv5 and remove discovery retries (#1373) 2020-07-21 14:19:55 +10:00
Pawan Dhananjay
1c90c816b7
Close outbound stream after sending (#1369) 2020-07-21 10:57:44 +10:00
Age Manning
e940dcea47
Updates libp2p and noise hack (#1367) 2020-07-21 10:54:47 +10:00
Pawan Dhananjay
b885d79ac3
Fix attestation propagation (#1360)
* Add `should_process` for conditional processing of Attestations

* Remove ATTESTATIONS_IGNORED metric
2020-07-20 12:55:32 +10:00
Adam Szkoda
fc5e6cbbb0 Add ability to configure CORS header (#1345)
## Issue Addressed

https://github.com/sigp/lighthouse/issues/1177

## Proposed Changes

Add a command line option (`--http-allow-origin`) and a config item for configuring the `Access-Control-Allow-Origin` response header.  This should unblock making XMLHttpRequests.
2020-07-16 07:23:14 +00:00
Age Manning
4a01f44206 Downgrade libp2p and gosispsub (#1358)
Downgrades libp2p and the gossipsub updates. 

This looks to resolve the CPU usage issue we have been seeing. 

The root cause is likely inside the latest gossipsub updates, which will be addressed in a later PR
2020-07-15 05:04:09 +00:00
Akihito Nakano
4b213032b2 Tiny improvement: lifetime parameters (#1354)
## Issue Addressed

This PR improves lifetime parameters that can be elided. :octocat:
2020-07-14 22:26:39 +00:00
Age Manning
c80860c17e Correct environment compiling (#1350)
Adds the macro feature to tokio to allow the environment crate to compile independently
2020-07-14 21:31:59 +00:00
pscott
e164371083 Set Graffiti via CLI (#1320)
## Issue Addressed

Closes #1319 

## Proposed Changes

This issue:
1. Allows users to edit their Graffiti via the cli option `--graffiti`. If the graffiti is too long, lighthouse will not start and throw an error message. Otherwise, it will set the Graffiti to be the one provided by the user, right-padded with 0s.
2. Create a new `Graffiti` type and unify the code around it. With this type, everything is enforced at compile-time, and the code can be (I think...) panic-free! :)

## Additional info

Currently, only `&str` are supported, as this is the returned type by `.arg("graffiti")`.
Since this is user-input, I tried being as careful as I could. This is also why I created the `Graffiti` type, to make sure I could check as much as possible at compile time.
2020-07-14 08:05:02 +00:00
Paul Hauner
00c89c51c8 Return 32-bit architecture error earlier (#1356)
## Issue Addressed

NA

## Proposed Changes

Exits *earlier* when system is 32-bit. Previously we were trying (and failing) to parse the genesis SSZ first. This made for a bad user experience.

## Additional Info

NA
2020-07-14 06:23:33 +00:00
Akihito Nakano
3c7f2d651a Tweak for appearance (#1351)
## Issue Addressed

This PR tweaks the appearance of description for `boot_node` subcommand. 👀 

![image](https://user-images.githubusercontent.com/1885716/87242856-97782680-c46b-11ea-9210-a161c630f942.png)
2020-07-13 03:48:20 +00:00
Age Manning
e6a8635b38
Discovery update (#1349)
* Improve logging

* Discovery update
2020-07-11 12:35:59 +10:00
Akihito Nakano
9ae218bfac
Remove outdated comments (#1336) 2020-07-11 10:33:11 +10:00
Pawan Dhananjay
57e0b6a615
Fix condition for resubscription (#1348) 2020-07-11 10:32:19 +10:00
Adam Szkoda
e8d5d37bc1 Tune CI (#1344)
`pr/*` branches now trigger CI before creating a PR to avoid submitting PR with failing tests.

Details in the commit messages.
2020-07-09 09:23:28 +00:00
Pawan Dhananjay
275148a152
Fix query expiry computation (#1346) 2020-07-09 11:43:48 +10:00
Adam Szkoda
559b7c8faa Fix bors timeouts (#1341)
... by moving "license/cla" from `status` to `pr_status` in `bors.toml`.
2020-07-07 13:44:35 +00:00
Age Manning
025b262e01
Peer scoring updates (#1342)
* Prevent banned peers from interacting with the behaviour

* Update smallvec

* Add logs

* Correct log update
2020-07-07 20:08:05 +10:00
Age Manning
ac2ce2ba6b
Correct book links (#1338)
* Correct book links

* Add count to validator creation

* Correct more links
2020-07-07 17:28:59 +10:00
Age Manning
f500b24242
Update smallvec (#1339) 2020-07-07 16:57:27 +10:00
Adam Szkoda
51fbaefe41 Add bors configuration (#1340)
Melded together stuff from https://github.com/sigp/lighthouse/pull/1326 and https://github.com/sigp/lighthouse/pull/1337.

Note that I've reduced the timeout to 2 hours to save some time in case the build gets stuck.

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2020-07-07 06:31:20 +00:00
Michael Sproul
20a48df80a
Fix race condition in VC block proposal service (#1282)
Closes #918
Closes #923
2020-07-07 14:03:21 +10:00
Age Manning
5bc8fea2e0
Activate peer scoring (#1284)
* Initial score structure

* Peer manager update

* Updates to dialing

* Correct tests

* Correct typos and remove unused function

* Integrate scoring into the network crate

* Clean warnings

* Formatting

* Shift core functionality into the behaviour

* Temp commit

* Shift disconnections into the behaviour

* Temp commit

* Update libp2p and gossipsub

* Remove gossipsub lru cache

* Correct merge conflicts

* Modify handler and correct tests

* Update enr network globals on socket update

* Apply clippy lints

* Add new prysm fingerprint

* More clippy fixes
2020-07-07 10:13:16 +10:00
Age Manning
5977c00edb
Update libp2p and gossipsub (#1324)
* Update libp2p and gossipsub

* Remove gossipsub lru cache
2020-07-06 20:34:40 +10:00
Pawan Dhananjay
f631155304
Add latest commit info to git version (#1316)
* Add latest commit info to git version

* Testing docker build

* Use fallback; modify format

* Revert "Testing docker build"

This reverts commit 197140d6973e22a6b891ebc0f1798d60c87b182a.

* Modify fallback to have crate version
2020-07-06 19:08:37 +10:00
Paul Hauner
bae4835308
Remove validator client docs (#1334) 2020-07-06 19:02:10 +10:00
Paul Hauner
e429c3eefe
Remove old block processing shim (#1327)
* Remove old block processing shim

* Run rustfmt

* Fix log formatting

* Swap peer ids over to display
2020-07-06 16:28:00 +10:00
Paul Hauner
2856f5122d
Add checksum verification for deposit contract (#1330) 2020-07-06 11:56:44 +10:00
Paul Hauner
25cd91ce26
Update deps (#1322)
* Run cargo update

* Upgrade prometheus

* Update hex

* Upgrade parking-lot

* Upgrade num-bigint

* Upgrade sha2

* Update dockerfile Rust version

* Run cargo update
2020-07-06 11:55:56 +10:00
Adam Szkoda
c7f47af9fb
Harden the freezing procedure against failures (#1323)
* Enable logging in tests

* Migrate states to the freezer atomically
2020-07-03 09:47:31 +10:00
realbigsean
9dab928572
swap out rust-crypto for RustCrypto libraries for key management (#1270)
* swap out rust-crypto for RustCrypto libraries for key management

* change scrypt params unrwap to expect

* delete eth2 key derivation path file, fix error handling in eth2 keystore, pr updates
2020-07-01 18:01:03 +10:00
Adam Szkoda
536728b975
Write new blocks and states to the database atomically (#1285)
* Mostly atomic put_state()
* Reduce number of vec allocations
* Make crucial db operations atomic
* Save restore points
* Remove StateBatch
* Merge two HotColdDB impls
* Further reduce allocations
* Review feedback
* Silence clippy warning
2020-07-01 12:45:57 +10:00
Paul Hauner
ac89bb190a
Fix invalid attestation verification condition (#1321)
* Fix bug with attestation target

* Change comment wording
2020-07-01 12:45:34 +10:00
Pawan Dhananjay
314c077870
Fix altona deposit contract address (#1317) 2020-06-30 08:44:55 +10:00