lighthouse/beacon_node
Paul Hauner 2cd3e3a768 Avoid duplicate committee cache loads (#3574)
## Issue Addressed

NA

## Proposed Changes

I have observed scenarios on Goerli where Lighthouse was receiving attestations which reference the same, un-cached shuffling on multiple threads at the same time. Lighthouse was then loading the same state from database and determining the shuffling on multiple threads at the same time. This is unnecessary load on the disk and RAM.

This PR modifies the shuffling cache so that each entry can be either:

- A committee
- A promise for a committee (i.e., a `crossbeam_channel::Receiver`)

Now, in the scenario where we have thread A and thread B simultaneously requesting the same un-cached shuffling, we will have the following:

1. Thread A will take the write-lock on the shuffling cache, find that there's no cached committee and then create a "promise" (a `crossbeam_channel::Sender`) for a committee before dropping the write-lock.
1. Thread B will then be allowed to take the write-lock for the shuffling cache and find the promise created by thread A. It will block the current thread waiting for thread A to fulfill that promise.
1. Thread A will load the state from disk, obtain the shuffling, send it down the channel, insert the entry into the cache and then continue to verify the attestation.
1. Thread B will then receive the shuffling from the receiver, be un-blocked and then continue to verify the attestation.

In the case where thread A fails to generate the shuffling and drops the sender, the next time that specific shuffling is requested we will detect that the channel is disconnected and return a `None` entry for that shuffling. This will cause the shuffling to be re-calculated.

## Additional Info

NA
2022-09-16 08:54:03 +00:00
..
beacon_chain Avoid duplicate committee cache loads (#3574) 2022-09-16 08:54:03 +00:00
builder_client Builder Specs v0.2.0 (#3134) 2022-07-30 00:22:37 +00:00
client Separate committee subscriptions queue (#3508) 2022-08-30 05:47:31 +00:00
eth1 Tidy eth1/deposit contract logging (#3397) 2022-08-01 07:20:43 +00:00
execution_layer Pin mev rs deps (#3557) 2022-09-08 23:46:03 +00:00
genesis Unify execution layer endpoints (#3214) 2022-06-29 09:07:09 +00:00
http_api Builder profit threshold flag (#3534) 2022-09-05 04:50:49 +00:00
http_metrics Support IPv6 in BN and VC HTTP APIs (#3104) 2022-03-24 00:04:49 +00:00
lighthouse_network Subscribe to subnets only when needed (#3419) 2022-09-05 00:22:48 +00:00
network Avoid duplicate committee cache loads (#3574) 2022-09-16 08:54:03 +00:00
operation_pool Refactor op pool for speed and correctness (#3312) 2022-08-29 09:10:26 +00:00
src Configurable monitoring endpoint frequency (#3530) 2022-09-05 08:29:00 +00:00
store Refactor op pool for speed and correctness (#3312) 2022-08-29 09:10:26 +00:00
tests Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
timer Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
Cargo.toml v3.1.0 (#3525) 2022-08-31 22:21:55 +00:00