lighthouse/validator_client
Paul Hauner ad5bd6412a
Add attestation gossip pre-verification (#983)
* Add PH & MS slot clock changes

* Account for genesis time

* Add progress on duties refactor

* Add simple is_aggregator bool to val subscription

* Start work on attestation_verification.rs

* Add progress on ObservedAttestations

* Progress with ObservedAttestations

* Fix tests

* Add observed attestations to the beacon chain

* Add attestation observation to processing code

* Add progress on attestation verification

* Add first draft of ObservedAttesters

* Add more tests

* Add observed attesters to beacon chain

* Add observers to attestation processing

* Add more attestation verification

* Create ObservedAggregators map

* Remove commented-out code

* Add observed aggregators into chain

* Add progress

* Finish adding features to attestation verification

* Ensure beacon chain compiles

* Link attn verification into chain

* Integrate new attn verification in chain

* Remove old attestation processing code

* Start trying to fix beacon_chain tests

* Split adding into pools into two functions

* Add aggregation to harness

* Get test harness working again

* Adjust the number of aggregators for test harness

* Fix edge-case in harness

* Integrate new attn processing in network

* Fix compile bug in validator_client

* Update validator API endpoints

* Fix aggreagation in test harness

* Fix enum thing

* Fix attestation observation bug:

* Patch failing API tests

* Start adding comments to attestation verification

* Remove unused attestation field

* Unify "is block known" logic

* Update comments

* Supress fork choice errors for network processing

* Add todos

* Tidy

* Add gossip attn tests

* Disallow test harness to produce old attns

* Comment out in-progress tests

* Partially address pruning tests

* Fix failing store test

* Add aggregate tests

* Add comments about which spec conditions we check

* Dont re-aggregate

* Split apart test harness attn production

* Fix compile error in network

* Make progress on commented-out test

* Fix skipping attestation test

* Add fork choice verification tests

* Tidy attn tests, remove dead code

* Remove some accidentally added code

* Fix clippy lint

* Rename test file

* Add block tests, add cheap block proposer check

* Rename block testing file

* Add observed_block_producers

* Tidy

* Switch around block signature verification

* Finish block testing

* Remove gossip from signature tests

* First pass of self review

* Fix deviation in spec

* Update test spec tags

* Start moving over to hashset

* Finish moving observed attesters to hashmap

* Move aggregation pool over to hashmap

* Make fc attn borrow again

* Fix rest_api compile error

* Fix missing comments

* Fix monster test

* Uncomment increasing slots test

* Address remaining comments

* Remove unsafe, use cfg test

* Remove cfg test flag

* Fix dodgy comment

* Ignore aggregates that are already known.

* Unify aggregator modulo logic

* Fix typo in logs

* Refactor validator subscription logic

* Avoid reproducing selection proof

* Skip HTTP call if no subscriptions

* Rename DutyAndState -> DutyAndProof

* Tidy logs

* Print root as dbg

* Fix compile errors in tests

* Fix compile error in test
2020-05-06 21:42:56 +10:00
..
src Add attestation gossip pre-verification (#983) 2020-05-06 21:42:56 +10:00
Cargo.toml Update testnet tooling (#1001) 2020-04-19 02:20:43 +00:00
README.md Update to spec v0.9.1 (#597) 2019-11-21 11:47:30 +11:00

Lighthouse Validator Client

The Validator Client (VC) is a stand-alone binary which connects to a Beacon Node (BN) and fulfils the roles of a validator.

Roles

The VC is responsible for the following tasks:

  • Requesting validator duties (a.k.a. shuffling) from the BN.
  • Prompting the BN to produce a new block, when a validator's block production duties require.
  • Completing all the fields on a new block (e.g., RANDAO reveal, signature) and publishing the block to a BN.
  • Prompting the BN to produce a new attestation as per a validator's duties.
  • Ensuring that no slashable messages are signed by a validator private key.
  • Keeping track of the system clock and how it relates to slots/epochs.

The VC is capable of managing multiple validators in the same process tree.

Implementation

This section describes the present implementation of this VC binary.

Services

Each validator is represented by two services, one which tracks the validator duties and another which performs block production duties.

A separate thread is maintained for each service, for each validator. As such, a single validator utilises three (3) threads (one for the base VC and two for each service) and two validators utilise five (5) threads.

DutiesManagerService

Polls a BN and requests validator responsibilities, as well as a validator index. The outcome of a successful poll is a EpochDuties struct:

EpochDuties {
	validator_index: u64,
	block_production_slot: u64,
}

This is stored in the EpochDutiesMap, a HashMap mapping epoch -> EpochDuties.

BlockProducerService

Polls the system clock and determines if a block needs to be produced. Reads from the EpochDutiesMap maintained by the DutiesManagerService.

If block production is required, performs all the necessary duties to request, complete and return a block from the BN.

Configuration

Validator configurations are stored in a separate data directory from the main Beacon Node binary. The validator data directory defaults to: $HOME/.lighthouse-validator, however an alternative can be specified on the command line with --datadir.

The configuration directory structure looks like:

~/.lighthouse-validator
    ├── 3cf4210d58ec
    │   └── private.key
    ├── 9b5d8b5be4e7
    │   └── private.key
    └── cf6e07188f48
        └── private.key

Where the hex value of the directory is a portion of the validator public key.

Validator keys must be generated using the separate account_manager binary, which will place the keys into this directory structure in a format compatible with the validator client. Be sure to check the readme for account_manager.

The chain specification (slot length, BLS domain, etc.) defaults to foundation parameters, however is temporary and an upgrade will allow these parameters to be read from a file (or initialized on first-boot).

BN Communication

The VC communicates with the BN via a gRPC/protobuf connection.