lighthouse/beacon_node/beacon_chain/src
Michael Sproul d2e3d4c6f1 Add flag to disable lock timeouts (#2714)
## Issue Addressed

Mitigates #1096

## Proposed Changes

Add a flag to the beacon node called `--disable-lock-timeouts` which allows opting out of lock timeouts.

The lock timeouts serve a dual purpose:

1. They prevent any single operation from hogging the lock for too long. When a timeout occurs it logs a nasty error which indicates that there's suboptimal lock use occurring, which we can then act on.
2. They allow deadlock detection. We're fairly sure there are no deadlocks left in Lighthouse anymore but the timeout locks offer a safeguard against that.

However, timeouts on locks are not without downsides:

They allow for the possibility of livelock, particularly on slower hardware. If lock timeouts keep failing spuriously the node can be prevented from making any progress, even if it would be able to make progress slowly without the timeout. One particularly concerning scenario which could occur would be if a DoS attack succeeded in slowing block signature verification times across the network, and all Lighthouse nodes got livelocked because they timed out repeatedly. This could also occur on just a subset of nodes (e.g. dual core VPSs or Raspberri Pis).

By making the behaviour runtime configurable this PR allows us to choose the behaviour we want depending on circumstance. I suspect that long term we could make the timeout-free approach the default (#2381 moves in this direction) and just enable the timeouts on our testnet nodes for debugging purposes. This PR conservatively leaves the default as-is so we can gain some more experience before switching the default.
2021-10-19 00:30:40 +00:00
..
attestation_verification Batch BLS verification for attestations (#2399) 2021-09-22 08:49:41 +00:00
attestation_verification.rs Batch BLS verification for attestations (#2399) 2021-09-22 08:49:41 +00:00
attester_cache.rs Add AttesterCache for attestation production (#2478) 2021-07-29 04:38:26 +00:00
beacon_chain.rs Add Altair tests to op pool (#2723) 2021-10-16 05:07:23 +00:00
beacon_fork_choice_store.rs Tree hash caching and optimisations for Altair (#2459) 2021-07-23 00:23:53 +00:00
beacon_proposer_cache.rs Optimize validator duties (#2243) 2021-03-17 05:09:57 +00:00
beacon_snapshot.rs Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
block_times_cache.rs Add BlockTimesCache to allow additional block delay metrics (#2546) 2021-09-30 04:31:41 +00:00
block_verification.rs Implement checkpoint sync (#2244) 2021-09-22 00:37:28 +00:00
builder.rs Fix I/O atomicity issues with checkpoint sync (#2671) 2021-10-05 03:53:17 +00:00
chain_config.rs Add flag to disable lock timeouts (#2714) 2021-10-19 00:30:40 +00:00
errors.rs Implement checkpoint sync (#2244) 2021-09-22 00:37:28 +00:00
eth1_chain.rs Rust 1.54.0 lints (#2483) 2021-07-30 01:11:47 +00:00
events.rs Add BlockTimesCache to allow additional block delay metrics (#2546) 2021-09-30 04:31:41 +00:00
fork_revert.rs Revert bad blocks on missed fork (#2529) 2021-08-30 06:41:31 +00:00
head_tracker.rs Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
historical_blocks.rs Reduce lock contention in backfill sync (#2716) 2021-10-15 03:28:03 +00:00
lib.rs Add flag to disable lock timeouts (#2714) 2021-10-19 00:30:40 +00:00
metrics.rs Reduce lock contention in backfill sync (#2716) 2021-10-15 03:28:03 +00:00
migrate.rs Implement checkpoint sync (#2244) 2021-09-22 00:37:28 +00:00
naive_aggregation_pool.rs Rust 1.54.0 lints (#2483) 2021-07-30 01:11:47 +00:00
observed_aggregates.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
observed_attesters.rs Ensure doppelganger detects attestations in blocks (#2495) 2021-08-09 02:43:03 +00:00
observed_block_producers.rs Doppelganger detection (#2230) 2021-07-31 03:50:52 +00:00
observed_operations.rs Clippy 1.49.0 updates and dht persistence test fix (#2156) 2021-01-19 00:34:28 +00:00
persisted_beacon_chain.rs Fix head tracker concurrency bugs (#1771) 2020-10-19 05:58:39 +00:00
persisted_fork_choice.rs v0.12 fork choice update (#1229) 2020-06-17 11:10:22 +10:00
schema_change.rs Implement checkpoint sync (#2244) 2021-09-22 00:37:28 +00:00
shuffling_cache.rs Advance state to next slot after importing block (#2174) 2021-02-15 07:17:52 +00:00
snapshot_cache.rs Add BeaconChainHarness::builder (#2707) 2021-10-14 02:58:10 +00:00
state_advance_timer.rs Add AttesterCache for attestation production (#2478) 2021-07-29 04:38:26 +00:00
sync_committee_verification.rs Validator monitor support for sync committees (#2476) 2021-08-31 23:31:36 +00:00
test_utils.rs Add Altair tests to op pool (#2723) 2021-10-16 05:07:23 +00:00
timeout_rw_lock.rs Add flag to disable lock timeouts (#2714) 2021-10-19 00:30:40 +00:00
validator_monitor.rs Reduce validator monitor logging verbosity (#2606) 2021-10-12 05:06:48 +00:00
validator_pubkey_cache.rs Add BeaconChainHarness::builder (#2707) 2021-10-14 02:58:10 +00:00