lighthouse/validator_client
Paul Hauner 78d82d9193
Validator client refactor (#618)
* Update to spec v0.9.0

* Update to v0.9.1

* Bump spec tags for v0.9.1

* Formatting, fix CI failures

* Resolve accidental KeyPair merge conflict

* Document new BeaconState functions

* Add `validator` changes from `validator-to-rest`

* Add initial (failing) REST api tests

* Fix signature parsing

* Add more tests

* Refactor http router

* Add working tests for publish beacon block

* Add validator duties tests

* Move account_manager under `lighthouse` binary

* Unify logfile handling in `environment` crate.

* Fix incorrect cache drops in `advance_caches`

* Update fork choice for v0.9.1

* Add `deposit_contract` crate

* Add progress on validator onboarding

* Add unfinished attesation code

* Update account manager CLI

* Write eth1 data file as hex string

* Integrate ValidatorDirectory with validator_client

* Move ValidatorDirectory into validator_client

* Clean up some FIXMEs

* Add beacon_chain_sim

* Fix a few docs/logs

* Expand `beacon_chain_sim`

* Fix spec for `beacon_chain_sim

* More testing for api

* Start work on attestation endpoint

* Reject empty attestations

* Allow attestations to genesis block

* Add working tests for `rest_api` validator endpoint

* Remove grpc from beacon_node

* Start heavy refactor of validator client

- Block production is working

* Prune old validator client files

* Start works on attestation service

* Add attestation service to validator client

* Use full pubkey for validator directories

* Add validator duties post endpoint

* Use par_iter for keypair generation

* Use bulk duties request in validator client

* Add version http endpoint tests

* Add interop keys and startup wait

* Ensure a prompt exit

* Add duties pruning

* Fix compile error in beacon node tests

* Add github workflow

* Modify rust.yaml

* Modify gitlab actions

* Add to CI file

* Add sudo to CI npm install

* Move cargo fmt to own job in tests

* Fix cargo fmt in CI

* Add rustup update before cargo fmt

* Change name of CI job

* Make other CI jobs require cargo fmt

* Add CI badge

* Remove gitlab and travis files

* Add different http timeout for debug

* Update docker file, use makefile in CI

* Use make in the dockerfile, skip the test

* Use the makefile for debug GI test

* Update book

* Tidy grpc and misc things

* Apply discv5 fixes

* Address other minor issues

* Fix warnings

* Attempt fix for addr parsing

* Tidy validator config, CLIs

* Tidy comments

* Tidy signing, reduce ForkService duplication

* Fail if skipping too many slots

* Set default recent genesis time to 0

* Add custom http timeout to validator

* Fix compile bug in node_test_rig

* Remove old bootstrap flag from val CLI

* Update docs

* Tidy val client

* Change val client log levels

* Add comments, more validity checks

* Fix compile error, add comments

* Undo changes to eth2-libp2p/src

* Reduce duplication of keypair generation

* Add more logging for validator duties

* Fix beacon_chain_sim, nitpicks

* Fix compile error, minor nits

* Address Michael's comments
2019-11-25 15:48:24 +11:00
..
src Validator client refactor (#618) 2019-11-25 15:48:24 +11:00
Cargo.toml Validator client refactor (#618) 2019-11-25 15:48:24 +11:00
README.md Update to spec v0.9.1 (#597) 2019-11-21 11:47:30 +11:00

Lighthouse Validator Client

The Validator Client (VC) is a stand-alone binary which connects to a Beacon Node (BN) and fulfils the roles of a validator.

Roles

The VC is responsible for the following tasks:

  • Requesting validator duties (a.k.a. shuffling) from the BN.
  • Prompting the BN to produce a new block, when a validator's block production duties require.
  • Completing all the fields on a new block (e.g., RANDAO reveal, signature) and publishing the block to a BN.
  • Prompting the BN to produce a new attestation as per a validator's duties.
  • Ensuring that no slashable messages are signed by a validator private key.
  • Keeping track of the system clock and how it relates to slots/epochs.

The VC is capable of managing multiple validators in the same process tree.

Implementation

This section describes the present implementation of this VC binary.

Services

Each validator is represented by two services, one which tracks the validator duties and another which performs block production duties.

A separate thread is maintained for each service, for each validator. As such, a single validator utilises three (3) threads (one for the base VC and two for each service) and two validators utilise five (5) threads.

DutiesManagerService

Polls a BN and requests validator responsibilities, as well as a validator index. The outcome of a successful poll is a EpochDuties struct:

EpochDuties {
	validator_index: u64,
	block_production_slot: u64,
}

This is stored in the EpochDutiesMap, a HashMap mapping epoch -> EpochDuties.

BlockProducerService

Polls the system clock and determines if a block needs to be produced. Reads from the EpochDutiesMap maintained by the DutiesManagerService.

If block production is required, performs all the necessary duties to request, complete and return a block from the BN.

Configuration

Validator configurations are stored in a separate data directory from the main Beacon Node binary. The validator data directory defaults to: $HOME/.lighthouse-validator, however an alternative can be specified on the command line with --datadir.

The configuration directory structure looks like:

~/.lighthouse-validator
    ├── 3cf4210d58ec
    │   └── private.key
    ├── 9b5d8b5be4e7
    │   └── private.key
    └── cf6e07188f48
        └── private.key

Where the hex value of the directory is a portion of the validator public key.

Validator keys must be generated using the separate account_manager binary, which will place the keys into this directory structure in a format compatible with the validator client. Be sure to check the readme for account_manager.

The chain specification (slot length, BLS domain, etc.) defaults to foundation parameters, however is temporary and an upgrade will allow these parameters to be read from a file (or initialized on first-boot).

BN Communication

The VC communicates with the BN via a gRPC/protobuf connection.