lighthouse/consensus
Paul Hauner 2cd3e3a768 Avoid duplicate committee cache loads (#3574)
## Issue Addressed

NA

## Proposed Changes

I have observed scenarios on Goerli where Lighthouse was receiving attestations which reference the same, un-cached shuffling on multiple threads at the same time. Lighthouse was then loading the same state from database and determining the shuffling on multiple threads at the same time. This is unnecessary load on the disk and RAM.

This PR modifies the shuffling cache so that each entry can be either:

- A committee
- A promise for a committee (i.e., a `crossbeam_channel::Receiver`)

Now, in the scenario where we have thread A and thread B simultaneously requesting the same un-cached shuffling, we will have the following:

1. Thread A will take the write-lock on the shuffling cache, find that there's no cached committee and then create a "promise" (a `crossbeam_channel::Sender`) for a committee before dropping the write-lock.
1. Thread B will then be allowed to take the write-lock for the shuffling cache and find the promise created by thread A. It will block the current thread waiting for thread A to fulfill that promise.
1. Thread A will load the state from disk, obtain the shuffling, send it down the channel, insert the entry into the cache and then continue to verify the attestation.
1. Thread B will then receive the shuffling from the receiver, be un-blocked and then continue to verify the attestation.

In the case where thread A fails to generate the shuffling and drops the sender, the next time that specific shuffling is requested we will detect that the channel is disconnected and return a `None` entry for that shuffling. This will cause the shuffling to be re-calculated.

## Additional Info

NA
2022-09-16 08:54:03 +00:00
..
cached_tree_hash Update and consolidate dependencies (#3136) 2022-04-04 00:26:16 +00:00
fork_choice Strict count unrealized (#3522) 2022-09-05 04:50:47 +00:00
int_to_bytes Update to Rust 1.59 and 2021 edition (#3038) 2022-02-25 00:10:17 +00:00
merkle_proof Update and consolidate dependencies (#3136) 2022-04-04 00:26:16 +00:00
proto_array Strict count unrealized (#3522) 2022-09-05 04:50:47 +00:00
safe_arith Inline safe_arith methods (#3229) 2022-05-31 06:09:12 +00:00
serde_utils Add a u256_hex_be module to encode/decode U256 types (#3321) 2022-07-15 07:31:21 +00:00
ssz Remove equivocating validators from fork choice (#3371) 2022-07-28 09:43:41 +00:00
ssz_derive Update to Rust 1.59 and 2021 edition (#3038) 2022-02-25 00:10:17 +00:00
ssz_types Fix lints for Rust 1.63 (#3459) 2022-08-12 00:56:39 +00:00
state_processing Fix attestation performance API InvalidValidatorIndex error (#3503) 2022-09-05 04:50:45 +00:00
swap_or_not_shuffle Update and consolidate dependencies (#3136) 2022-04-04 00:26:16 +00:00
tree_hash Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
tree_hash_derive Update to Rust 1.59 and 2021 edition (#3038) 2022-02-25 00:10:17 +00:00
types Avoid duplicate committee cache loads (#3574) 2022-09-16 08:54:03 +00:00