lighthouse/beacon_node/beacon_chain/src
Paul Hauner 2cd3e3a768 Avoid duplicate committee cache loads (#3574)
## Issue Addressed

NA

## Proposed Changes

I have observed scenarios on Goerli where Lighthouse was receiving attestations which reference the same, un-cached shuffling on multiple threads at the same time. Lighthouse was then loading the same state from database and determining the shuffling on multiple threads at the same time. This is unnecessary load on the disk and RAM.

This PR modifies the shuffling cache so that each entry can be either:

- A committee
- A promise for a committee (i.e., a `crossbeam_channel::Receiver`)

Now, in the scenario where we have thread A and thread B simultaneously requesting the same un-cached shuffling, we will have the following:

1. Thread A will take the write-lock on the shuffling cache, find that there's no cached committee and then create a "promise" (a `crossbeam_channel::Sender`) for a committee before dropping the write-lock.
1. Thread B will then be allowed to take the write-lock for the shuffling cache and find the promise created by thread A. It will block the current thread waiting for thread A to fulfill that promise.
1. Thread A will load the state from disk, obtain the shuffling, send it down the channel, insert the entry into the cache and then continue to verify the attestation.
1. Thread B will then receive the shuffling from the receiver, be un-blocked and then continue to verify the attestation.

In the case where thread A fails to generate the shuffling and drops the sender, the next time that specific shuffling is requested we will detect that the channel is disconnected and return a `None` entry for that shuffling. This will cause the shuffling to be re-calculated.

## Additional Info

NA
2022-09-16 08:54:03 +00:00
..
attestation_verification Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
schema_change Strict count unrealized (#3522) 2022-09-05 04:50:47 +00:00
attestation_verification.rs Refactor op pool for speed and correctness (#3312) 2022-08-29 09:10:26 +00:00
attester_cache.rs Add early attester cache (#2872) 2022-01-11 01:35:55 +00:00
beacon_chain.rs Avoid duplicate committee cache loads (#3574) 2022-09-16 08:54:03 +00:00
beacon_fork_choice_store.rs Remove equivocating validators from fork choice (#3371) 2022-07-28 09:43:41 +00:00
beacon_proposer_cache.rs Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
beacon_snapshot.rs Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
block_reward.rs Refactor op pool for speed and correctness (#3312) 2022-08-29 09:10:26 +00:00
block_times_cache.rs Add BlockTimesCache to allow additional block delay metrics (#2546) 2021-09-30 04:31:41 +00:00
block_verification.rs Refactor op pool for speed and correctness (#3312) 2022-08-29 09:10:26 +00:00
builder.rs Strict count unrealized (#3522) 2022-09-05 04:50:47 +00:00
canonical_head.rs Add metric for re-org distance (#3566) 2022-09-13 17:19:27 +00:00
chain_config.rs Strict count unrealized (#3522) 2022-09-05 04:50:47 +00:00
early_attester_cache.rs Fix some typos (#3376) 2022-07-27 00:51:06 +00:00
errors.rs Avoid duplicate committee cache loads (#3574) 2022-09-16 08:54:03 +00:00
eth1_chain.rs Removed PowBlock struct that never got used (#2813) 2021-12-02 14:29:20 +11:00
events.rs Implement API for block rewards (#2628) 2022-01-27 01:06:02 +00:00
execution_payload.rs Add more logging for invalid payloads (#3515) 2022-08-29 14:34:42 +00:00
fork_choice_signal.rs Run fork choice before block proposal (#3168) 2022-05-20 05:02:11 +00:00
fork_revert.rs Strict count unrealized (#3522) 2022-09-05 04:50:47 +00:00
head_tracker.rs Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
historical_blocks.rs Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
lib.rs Strict count unrealized (#3522) 2022-09-05 04:50:47 +00:00
merge_readiness.rs Increase merge-readiness lookhead (#3463) 2022-08-15 01:30:59 +00:00
metrics.rs Avoid duplicate committee cache loads (#3574) 2022-09-16 08:54:03 +00:00
migrate.rs Avoid parallel fork choice runs during sync (#3217) 2022-05-25 03:27:30 +00:00
naive_aggregation_pool.rs Update to Rust 1.59 and 2021 edition (#3038) 2022-02-25 00:10:17 +00:00
observed_aggregates.rs v2.2.0 (#3139) 2022-04-05 02:53:09 +00:00
observed_attesters.rs Ensure doppelganger detects attestations in blocks (#2495) 2021-08-09 02:43:03 +00:00
observed_block_producers.rs Doppelganger detection (#2230) 2021-07-31 03:50:52 +00:00
observed_operations.rs Refactor op pool for speed and correctness (#3312) 2022-08-29 09:10:26 +00:00
otb_verification_service.rs Initial Commit of Retrospective OTB Verification (#3372) 2022-07-30 00:22:38 +00:00
persisted_beacon_chain.rs Fix head tracker concurrency bugs (#1771) 2020-10-19 05:58:39 +00:00
persisted_fork_choice.rs Remove equivocating validators from fork choice (#3371) 2022-07-28 09:43:41 +00:00
pre_finalization_cache.rs Separate execution payloads in the DB (#3157) 2022-05-12 00:42:17 +00:00
proposer_prep_service.rs Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
schema_change.rs Refactor op pool for speed and correctness (#3312) 2022-08-29 09:10:26 +00:00
shuffling_cache.rs Avoid duplicate committee cache loads (#3574) 2022-09-16 08:54:03 +00:00
snapshot_cache.rs Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
state_advance_timer.rs Avoid duplicate committee cache loads (#3574) 2022-09-16 08:54:03 +00:00
sync_committee_verification.rs Update to Rust 1.59 and 2021 edition (#3038) 2022-02-25 00:10:17 +00:00
test_utils.rs Refactor op pool for speed and correctness (#3312) 2022-08-29 09:10:26 +00:00
timeout_rw_lock.rs Add flag to disable lock timeouts (#2714) 2021-10-19 00:30:40 +00:00
validator_monitor.rs Fix validator_monitor_prev_epoch_ metrics (#2911) 2022-06-20 04:06:30 +00:00
validator_pubkey_cache.rs Add support for beaconAPI in lcli functions (#3252) 2022-08-09 06:05:13 +00:00