lighthouse/common/slot_clock/src/lib.rs
realbigsean a62e52f319
Single blob lookups (#4152)
* some blob reprocessing work

* remove ForceBlockLookup

* reorder enum match arms in sync manager

* a lot more reprocessing work

* impl logic for triggerng blob lookups along with block lookups

* deal with rpc blobs in groups per block in the da checker. don't cache missing blob ids in the da checker.

* make single block lookup generic

* more work

* add delayed processing logic and combine some requests

* start fixing some compile errors

* fix compilation in main block lookup mod

* much work

* get things compiling

* parent blob lookups

* fix compile

* revert red/stevie changes

* fix up sync manager delay message logic

* add peer usefulness enum

* should remove lookup refactor

* consolidate retry error handling

* improve peer scoring during certain failures in parent lookups

* improve retry code

* drop parent lookup if either req has a peer disconnect during download

* refactor single block processed method

* processing peer refactor

* smol bugfix

* fix some todos

* fix lints

* fix lints

* fix compile in lookup tests

* fix lints

* fix lints

* fix existing block lookup tests

* renamings

* fix after merge

* cargo fmt

* compilation fix in beacon chain tests

* fix

* refactor lookup tests to work with multiple forks and response types

* make tests into macros

* wrap availability check error

* fix compile after merge

* add random blobs

* start fixing up lookup verify error handling

* some bug fixes and the start of deneb only tests

* make tests work for all forks

* track information about peer source

* error refactoring

* improve peer scoring

* fix test compilation

* make sure blobs are sent for processing after stream termination, delete copied tests

* add some tests and fix a bug

* smol bugfixes and moar tests

* add tests and fix some things

* compile after merge

* lots of refactoring

* retry on invalid block/blob

* merge unknown parent messages before current slot lookup

* get tests compiling

* penalize blob peer on invalid blobs

* Check disk on in-memory cache miss

* Update beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs

* Update beacon_node/network/src/sync/network_context.rs

Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>

* fix bug in matching blocks and blobs in range sync

* pr feedback

* fix conflicts

* upgrade logs from warn to crit when we receive incorrect response in range

* synced_and_connected_within_tolerance -> should_search_for_block

* remove todo

* Fix Broken Overflow Tests

* fix merge conflicts

* checkpoint sync without alignment

* add import

* query for checkpoint state by slot rather than state root (teku doesn't serve by state root)

* get state first and query by most recent block root

* simplify delay logic

* rename unknown parent sync message variants

* rename parameter, block_slot -> slot

* add some docs to the lookup module

* use interval instead of sleep

* drop request if blocks and blobs requests both return `None` for `Id`

* clean up `find_single_lookup` logic

* add lookup source enum

* clean up `find_single_lookup` logic

* add docs to find_single_lookup_request

* move LookupSource our of param where unnecessary

* remove unnecessary todo

* query for block by `state.latest_block_header.slot`

* fix lint

* fix test

* fix test

* fix observed  blob sidecars test

* PR updates

* use optional params instead of a closure

* create lookup and trigger request in separate method calls

* remove `LookupSource`

* make sure duplicate lookups are not dropped

---------

Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Mark Mackey <mark@sigmaprime.io>
Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>
2023-06-15 12:59:10 -04:00

150 lines
5.9 KiB
Rust

#[macro_use]
extern crate lazy_static;
mod manual_slot_clock;
mod metrics;
mod system_time_slot_clock;
use std::time::Duration;
pub use crate::manual_slot_clock::ManualSlotClock as TestingSlotClock;
pub use crate::manual_slot_clock::ManualSlotClock;
pub use crate::system_time_slot_clock::SystemTimeSlotClock;
pub use metrics::scrape_for_metrics;
use types::consts::merge::INTERVALS_PER_SLOT;
pub use types::Slot;
/// A clock that reports the current slot.
///
/// The clock is not required to be monotonically increasing and may go backwards.
pub trait SlotClock: Send + Sync + Sized + Clone {
/// Creates a new slot clock where the first slot is `genesis_slot`, genesis occurred
/// `genesis_duration` after the `UNIX_EPOCH` and each slot is `slot_duration` apart.
fn new(genesis_slot: Slot, genesis_duration: Duration, slot_duration: Duration) -> Self;
/// Returns the slot at this present time.
fn now(&self) -> Option<Slot>;
/// Returns the slot at this present time if genesis has happened. Otherwise, returns the
/// genesis slot. Returns `None` if there is an error reading the clock.
fn now_or_genesis(&self) -> Option<Slot> {
if self.is_prior_to_genesis()? {
Some(self.genesis_slot())
} else {
self.now()
}
}
/// Indicates if the current time is prior to genesis time.
///
/// Returns `None` if the system clock cannot be read.
fn is_prior_to_genesis(&self) -> Option<bool>;
/// Returns the present time as a duration since the UNIX epoch.
///
/// Returns `None` if the present time is before the UNIX epoch (unlikely).
fn now_duration(&self) -> Option<Duration>;
/// Returns the slot of the given duration since the UNIX epoch.
fn slot_of(&self, now: Duration) -> Option<Slot>;
/// Returns the duration between slots
fn slot_duration(&self) -> Duration;
/// Returns the duration from now until `slot`.
fn duration_to_slot(&self, slot: Slot) -> Option<Duration>;
/// Returns the duration until the next slot.
fn duration_to_next_slot(&self) -> Option<Duration>;
/// Returns the duration until the first slot of the next epoch.
fn duration_to_next_epoch(&self, slots_per_epoch: u64) -> Option<Duration>;
/// Returns the start time of the slot, as a duration since `UNIX_EPOCH`.
fn start_of(&self, slot: Slot) -> Option<Duration>;
/// Returns the first slot to be returned at the genesis time.
fn genesis_slot(&self) -> Slot;
/// Returns the `Duration` from `UNIX_EPOCH` to the genesis time.
fn genesis_duration(&self) -> Duration;
/// Returns the slot if the internal clock were advanced by `duration`.
fn now_with_future_tolerance(&self, tolerance: Duration) -> Option<Slot> {
self.slot_of(self.now_duration()?.checked_add(tolerance)?)
}
/// Returns the slot if the internal clock were reversed by `duration`.
fn now_with_past_tolerance(&self, tolerance: Duration) -> Option<Slot> {
self.slot_of(self.now_duration()?.checked_sub(tolerance)?)
.or_else(|| Some(self.genesis_slot()))
}
/// Returns the delay between the start of the slot and when unaggregated attestations should be
/// produced.
fn unagg_attestation_production_delay(&self) -> Duration {
self.slot_duration() / INTERVALS_PER_SLOT as u32
}
/// Returns the delay between the start of the slot and when sync committee messages should be
/// produced.
fn sync_committee_message_production_delay(&self) -> Duration {
self.slot_duration() / INTERVALS_PER_SLOT as u32
}
/// Returns the delay between the start of the slot and when aggregated attestations should be
/// produced.
fn agg_attestation_production_delay(&self) -> Duration {
self.slot_duration() * 2 / INTERVALS_PER_SLOT as u32
}
/// Returns the delay between the start of the slot and when partially aggregated `SyncCommitteeContribution` should be
/// produced.
fn sync_committee_contribution_production_delay(&self) -> Duration {
self.slot_duration() * 2 / INTERVALS_PER_SLOT as u32
}
/// Returns the `Duration` since the start of the current `Slot` at seconds precision. Useful in determining whether to apply proposer boosts.
fn seconds_from_current_slot_start(&self) -> Option<Duration> {
self.now_duration()
.and_then(|now| now.checked_sub(self.genesis_duration()))
.map(|duration_into_slot| {
Duration::from_secs(duration_into_slot.as_secs() % self.slot_duration().as_secs())
})
}
/// Returns the `Duration` since the start of the current `Slot` at milliseconds precision.
fn millis_from_current_slot_start(&self) -> Option<Duration> {
self.now_duration()
.and_then(|now| now.checked_sub(self.genesis_duration()))
.map(|duration_into_slot| {
Duration::from_millis(
(duration_into_slot.as_millis() % self.slot_duration().as_millis()) as u64,
)
})
}
/// Produces a *new* slot clock with the same configuration of `self`, except that clock is
/// "frozen" at the `freeze_at` time.
///
/// This is useful for observing the slot clock at arbitrary fixed points in time.
fn freeze_at(&self, freeze_at: Duration) -> ManualSlotClock {
let slot_clock = ManualSlotClock::new(
self.genesis_slot(),
self.genesis_duration(),
self.slot_duration(),
);
slot_clock.set_current_time(freeze_at);
slot_clock
}
/// Returns the delay between the start of the slot and when a request for block components
/// missed over gossip in the current slot should be made via RPC.
///
/// Currently set equal to 1/2 of the `unagg_attestation_production_delay`, but this may be
/// changed in the future.
fn single_lookup_delay(&self) -> Duration {
self.unagg_attestation_production_delay() / 2
}
}