lighthouse/consensus/types/src
Paul Hauner c5c7476518 Web3Signer support for VC (#2522)
[EIP-3030]: https://eips.ethereum.org/EIPS/eip-3030
[Web3Signer]: https://consensys.github.io/web3signer/web3signer-eth2.html

## Issue Addressed

Resolves #2498

## Proposed Changes

Allows the VC to call out to a [Web3Signer] remote signer to obtain signatures.


## Additional Info

### Making Signing Functions `async`

To allow remote signing, I needed to make all the signing functions `async`. This caused a bit of noise where I had to convert iterators into `for` loops.

In `duties_service.rs` there was a particularly tricky case where we couldn't hold a write-lock across an `await`, so I had to first take a read-lock, then grab a write-lock.

### Move Signing from Core Executor

Whilst implementing this feature, I noticed that we signing was happening on the core tokio executor. I suspect this was causing the executor to temporarily lock and occasionally trigger some HTTP timeouts (and potentially SQL pool timeouts, but I can't verify this). Since moving all signing into blocking tokio tasks, I noticed a distinct drop in the "atttestations_http_get" metric on a Prater node:

![http_get_times](https://user-images.githubusercontent.com/6660660/132143737-82fd3836-2e7e-445b-a143-cb347783baad.png)

I think this graph indicates that freeing the core executor allows the VC to operate more smoothly.

### Refactor TaskExecutor

I noticed that the `TaskExecutor::spawn_blocking_handle` function would fail to spawn tasks if it were unable to obtain handles to some metrics (this can happen if the same metric is defined twice). It seemed that a more sensible approach would be to keep spawning tasks, but without metrics. To that end, I refactored the function so that it would still function without metrics. There are no other changes made.

## TODO

- [x] Restructure to support multiple signing methods.
- [x] Add calls to remote signer from VC.
- [x] Documentation
- [x] Test all endpoints
- [x] Test HTTPS certificate
- [x] Allow adding remote signer validators via the API
- [x] Add Altair support via [21.8.1-rc1](https://github.com/ConsenSys/web3signer/releases/tag/21.8.1-rc1)
- [x] Create issue to start using latest version of web3signer. (See #2570)

## Notes

- ~~Web3Signer doesn't yet support the Altair fork for Prater. See https://github.com/ConsenSys/web3signer/issues/423.~~
- ~~There is not yet a release of Web3Signer which supports Altair blocks. See https://github.com/ConsenSys/web3signer/issues/391.~~
2021-09-16 03:26:33 +00:00
..
beacon_state Optimize process_attestation with active balance cache (#2560) 2021-09-03 07:50:43 +00:00
test_utils [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
aggregate_and_proof.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
attestation_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
attestation_duty.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
attestation.rs Web3Signer support for VC (#2522) 2021-09-16 03:26:33 +00:00
attester_slashing.rs Update for clippy 1.50 (#2193) 2021-02-15 00:09:12 +00:00
beacon_block_body.rs Metrics for sync aggregate fullness (#2439) 2021-07-13 02:22:55 +00:00
beacon_block_header.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
beacon_block.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
beacon_committee.rs Directory Restructure (#1163) 2020-05-18 21:24:23 +10:00
beacon_state.rs Optimize process_attestation with active balance cache (#2560) 2021-09-03 07:50:43 +00:00
chain_spec.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
checkpoint.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
config_and_preset.rs Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
consts.rs Update to spec v1.1.0-beta.1 (#2460) 2021-07-27 05:43:35 +00:00
contribution_and_proof.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
deposit_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
deposit_message.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
deposit.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
enr_fork_id.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
eth1_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
eth_spec.rs Altair networking (#2300) 2021-08-04 01:44:57 +00:00
fork_context.rs Altair networking (#2300) 2021-08-04 01:44:57 +00:00
fork_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
fork_name.rs Case insensitive match for ForkName (#2587) 2021-09-11 06:07:20 +00:00
fork.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
free_attestation.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
graffiti.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
historical_batch.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
indexed_attestation.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
lib.rs Altair validator client and HTTP API (#2404) 2021-08-06 00:47:31 +00:00
participation_flags.rs Tree hash caching and optimisations for Altair (#2459) 2021-07-23 00:23:53 +00:00
participation_list.rs Rust 1.54.0 lints (#2483) 2021-07-30 01:11:47 +00:00
pending_attestation.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
preset.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
proposer_slashing.rs Implement slasher (#1567) 2020-11-23 03:43:22 +00:00
relative_epoch.rs Remove saturating arith from state_processing (#1644) 2020-09-25 05:18:21 +00:00
selection_proof.rs Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
shuffling_id.rs Optimize validator duties (#2243) 2021-03-17 05:09:57 +00:00
signed_aggregate_and_proof.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
signed_beacon_block_header.rs Update for clippy 1.50 (#2193) 2021-02-15 00:09:12 +00:00
signed_beacon_block.rs Revert bad blocks on missed fork (#2529) 2021-08-30 06:41:31 +00:00
signed_contribution_and_proof.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
signed_voluntary_exit.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
signing_data.rs Consensus updates for v0.12 (#1228) 2020-06-03 14:56:54 +10:00
slot_data.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
slot_epoch_macros.rs Updated to comply with new clippy formatting rules (#2336) 2021-05-10 00:53:09 +00:00
slot_epoch.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sqlite.rs Directory Restructure (#1163) 2020-05-18 21:24:23 +10:00
subnet_id.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_aggregate.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
sync_aggregator_selection_data.rs Web3Signer support for VC (#2522) 2021-09-16 03:26:33 +00:00
sync_committee_contribution.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_committee_message.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_committee_subscription.rs Add quoted u64/u64_vec to SyncCommitteeSubscription (#2589) 2021-09-11 23:56:16 +00:00
sync_committee.rs Validator monitor support for sync committees (#2476) 2021-08-31 23:31:36 +00:00
sync_duty.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_selection_proof.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
sync_subnet_id.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
tree_hash_impls.rs Rust 1.54.0 lints (#2483) 2021-07-30 01:11:47 +00:00
validator_subscription.rs Implement standard eth2.0 API (#1569) 2020-10-01 11:12:36 +10:00
validator.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
voluntary_exit.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00