lighthouse/beacon_node
Paul Hauner c25825a539 Move the BeaconProcessor into a new crate (#4435)
*Replaces #4434. It is identical, but this PR has a smaller diff due to a curated commit history.*

## Issue Addressed

NA

## Proposed Changes

This PR moves the scheduling logic for the `BeaconProcessor` into a new crate in `beacon_node/beacon_processor`. Previously it existed in the `beacon_node/network` crate.

This addresses a circular-dependency problem where it's not possible to use the `BeaconProcessor` from the `beacon_chain` crate. The `network` crate depends on the `beacon_chain` crate (`network -> beacon_chain`), but importing the `BeaconProcessor` into the `beacon_chain` crate would create a circular dependancy of `beacon_chain -> network`.

The `BeaconProcessor` was designed to provide queuing and prioritized scheduling for messages from the network. It has proven to be quite valuable and I believe we'd make Lighthouse more stable and effective by using it elsewhere. In particular, I think we should use the `BeaconProcessor` for:

1. HTTP API requests.
1. Scheduled tasks in the `BeaconChain` (e.g., state advance).

Using the `BeaconProcessor` for these tasks would help prevent the BN from becoming overwhelmed and would also help it to prioritize operations (e.g., choosing to process blocks from gossip before responding to low-priority HTTP API requests).

## Additional Info

This PR is intended to have zero impact on runtime behaviour. It aims to simply separate the *scheduling* code (i.e., the `BeaconProcessor`) from the *business logic* in the `network` crate (i.e., the `Worker` impls). Future PRs (see #4462) can build upon these works to actually use the `BeaconProcessor` for more operations.

I've gone to some effort to use `git mv` to make the diff look more like "file was moved and modified" rather than "file was deleted and a new one added". This should reduce review burden and help maintain commit attribution.
2023-07-10 07:45:54 +00:00
..
beacon_chain Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
beacon_processor Move the BeaconProcessor into a new crate (#4435) 2023-07-10 07:45:54 +00:00
builder_client Tidy formatting of Reqwest errors (#4336) 2023-06-27 01:06:50 +00:00
client Move the BeaconProcessor into a new crate (#4435) 2023-07-10 07:45:54 +00:00
eth1 Replace ganache-cli with anvil (#3555) 2023-05-15 07:22:02 +00:00
execution_layer Tidy formatting of Reqwest errors (#4336) 2023-06-27 01:06:50 +00:00
genesis Replace ganache-cli with anvil (#3555) 2023-05-15 07:22:02 +00:00
http_api Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
http_metrics Appease clippy in Rust 1.70 (#4365) 2023-06-02 03:17:40 +00:00
lighthouse_network Aggregate subsets (#3493) 2023-06-27 01:06:49 +00:00
network Move the BeaconProcessor into a new crate (#4435) 2023-07-10 07:45:54 +00:00
operation_pool Split common crates out into their own repos (#3890) 2023-04-28 01:15:40 +00:00
src Bump default checkpoint sync timeout to 3 minutes (#4466) 2023-07-08 13:16:06 +00:00
store Cache target attester balances for unrealized FFG progression calculation (#4362) 2023-06-30 01:13:06 +00:00
tests Appease Clippy 1.68 and refactor http_api (#4068) 2023-03-13 01:40:03 +00:00
timer Use async code when interacting with EL (#3244) 2022-07-03 05:36:50 +00:00
Cargo.toml Move the BeaconProcessor into a new crate (#4435) 2023-07-10 07:45:54 +00:00