Commit Graph

2884 Commits

Author SHA1 Message Date
Pawan Dhananjay
b55b58b3c6
Fix indices filter in blobs_sidecar http endpoint (#5118)
* add get blobs unit test

* Use a multi_key_query for blob_sidecar indices

* Fix test

* Remove env_logger

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2024-01-24 09:35:24 +11:00
realbigsean
1cebf41452
Backfill blob storage fix (#5119)
* store blobs in the correct db in backfill

* add database migration

* add migration file

* remove log info suggesting deneb isn't schedule

* add batching in blob migration
2024-01-24 09:35:02 +11:00
Age Manning
5851027bfd
Correct discovery logic (#5111) 2024-01-23 16:58:24 +11:00
Age Manning
b7c9b27872
Gossipsub fanout correction (#5110)
* Correct fanout in gossipsub

* Upgrade discv5 to pin new libp2p version

* Update cargo.lock
2024-01-23 16:38:00 +11:00
Michael Sproul
a403138ed0
Reduce size of futures in HTTP API to prevent stack overflows (#5104)
* Box::pin a few big futures

* Arc the blocks early in publication

* Fix more tests
2024-01-23 15:32:07 +11:00
ethDreamer
02d1f36090
Small Readability Improvement in Networking Code (#5098) 2024-01-23 12:08:57 +08:00
ethDreamer
9a630e4dbb
Stop Penalizing Peers in Parent SingleBlobLookup (#5096)
* Stop Penalizing Peers in Parent SingleBlobLookup

* Add test for parent lookup bug (#13)

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2024-01-22 05:35:06 +01:00
Lion - dapplion
585124fb2f
Hold HeadTracker lock until persisting to disk (#5084)
* Fix head tracker drop order on un-ordered shutdown

* lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2024-01-22 15:14:11 +11:00
Michael Sproul
185646acb2
Fix PublishBlockRequest SSZ decoding (#5078)
* Fix PublishBlockRequest SSZ decoding

* Send header for block v1 ssz client

* Delete unused function
2024-01-19 12:56:30 +11:00
Mac L
47b28c4935
Remove blobs_db when purge-db (#5081) 2024-01-18 15:59:08 -05:00
Mac L
a0b407c15d
Add Deneb readiness logging (#5074) 2024-01-18 15:21:38 -05:00
Akihito Nakano
e10e4b7811
Fix zero port error (#5021)
* Fix zero port error and add tests

* Tweak indentation

* assign 0 to quic_port if tcp_port == 0

* Remove unnecessary deps
2024-01-16 12:19:49 +11:00
vuittont60
0613eb7a21
docs: fix typos (#5059)
* docs: fix typos

* docs: fix typo

* docs: update by 'make cli'
2024-01-16 12:18:54 +11:00
Eitan Seri-Levi
72bcf47dd0
add content-type octet stream helper fn (#5062) 2024-01-15 16:23:39 +11:00
Paul Hauner
2e8e160679
v4.6.0-rc.0 (#5042) 2024-01-11 10:46:29 +11:00
João Oliveira
38df87c3c5
Switch libp2p sigp gossipsub fork (#4999)
* switch libp2p source to sigp fork

* Shift the connection closing inside RPC behaviour

* Tag specific commits

* Add slow peer scoring

* Fix test

* Use default yamux config

* Pin discv5 to our libp2p fork and cargo update

* Upgrade libp2p to enable yamux gains

* Add a comment specifying the branch being used

* cleanup build output from within container
(prevents CI warnings related to fs permissions)

* Remove revision tags add branches for testing, will revert back once we're happy

* Update to latest rust-libp2p version

* Pin forks

* Update cargo.lock

* Re-pin to panic-free rust

---------

Co-authored-by: Age Manning <Age@AgeManning.com>
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: antondlr <anton@delaruelle.net>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2024-01-10 16:26:52 +11:00
Paul Hauner
be79f74c6d
Fix IncorrectAttestationSource error in attn. simulator (#5048)
* Don't produce attestations when syncing

* Handle `IncorrectAttestationSource`
2024-01-10 10:47:00 +11:00
Michael Sproul
7e948eec9d
Fix block v3 reward encodings (#5049)
* Fix block v3 reward encodings

* Use crates.io version
2024-01-10 10:44:07 +11:00
zhiqiangxu
4a65d28b3b
remove needless Into (#5026) 2024-01-10 08:38:16 +11:00
Paul Hauner
12d3d237cd
Disallow genesis sync outside blob pruning window (#5038)
* Disallow Syncing From Genesis By Default

* Fix CLI Tests

* Perform checks in the `ClientBuilder`

* Tidy, fix tests

* Return an error based on the Deneb fork

* Fix typos

* Fix failing test

* Add missing CLI flag

* Fix CLI flags

* Add suggestion from Sean

* Fix conflict with blob sidecars epochs

---------

Co-authored-by: Mark Mackey <mark@sigmaprime.io>
2024-01-09 14:17:01 -05:00
realbigsean
b47e3f252e
Runtime rpc request sizes (#4841)
* add runtime variable list type

* add configs to ChainSpec

* git rid of max request blocks type

* fix tests and lints

* remove todos

* git rid of old const usage

* fix decode impl

* add new config to `Config` api struct

* add docs fix compilt

* move methods for per-fork-spec to chainspec

* get values off chain spec

* fix compile

* remove min by root size

* add tests for runtime var list

---------

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2024-01-09 10:23:47 +11:00
realbigsean
f70c32ec70
create unified slashing cache (#5033)
* create unified slashing cache

* add observed slashable file

* fix broadcast validation tests

* revert block seen cache changes

* clean up slashable cache test

* check header signatures for RPC blobs

* don't throw error on RPC signature invalie
2024-01-08 10:30:57 -05:00
Eitan Seri-Levi
9c1505d082
Block v3 builder boost factor (#5035)
* builder boost factor

* default boost factor

* revert

* deprecate always_prefer_builder_payload, builder-profit-threshold, ignore_builder_override_suggestion_threshold and builder_comparison_factor flags

* revert

* set deprecated flags to no op, revert should_override_builder

* fix test, calc boosted relay value correctly, dont calculate if none

* Add deprecation warnings and restore CLI docs
2024-01-08 11:10:32 +11:00
Michael Sproul
af11e78ae1
Clean up blockv3 metadata and client (#5015)
* Improve block production v3 client

* Delete wayward line

* Overhaul JSON endpoint as well

* Rename timeout param

* Update tests

* I broke everything

* Ah this is an insane fix

* Remove unnecessary optionals

* Doc fix
2023-12-22 09:39:17 -05:00
realbigsean
c55608be10
suppress error on duplicate blobs (#4995) 2023-12-18 12:15:12 -05:00
Jimmy Chen
dfc3b3714a
Fix incorrect blob queue metrics (#5014)
* Fix blob queue metrics.

* Update blob metric description
2023-12-18 12:04:57 -05:00
Jimmy Chen
b0c374c1ca
Update dependencies to get rid of the yanked deps (#4994)
* Initial attempt to upgrade hashbrown to latest to get rid of the crate warnings.

* Replace `.expect()` usage with use of `const`.

* Update ahash 0.7 as well

* Remove unsafe code.

* Update `lru` to 0.12 and fix release test errors.

* Set non-blocking socket

* Bump testcontainers to 0.15.

* Fix lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2023-12-15 18:31:59 +11:00
Michael Sproul
f1113540d8
Fix off-by-one bug in missed block detection (#5012)
* Regression test

* Fix the bug
2023-12-15 14:23:54 +11:00
Jimmy Chen
366f0d7ac2
Add flag to disable warning logs for duplicate gossip messages (#5009)
* Add flag to disable warning logs for duplicate gossip messages.

* Update Lighthouse book.
2023-12-15 09:26:51 +11:00
Pawan Dhananjay
ae4a296089
Convert a FullPayload to a BlindedPayload correctly (#5005) 2023-12-14 08:44:14 -05:00
Joel Rousseau
189430a45c
Add attestation simulator (#4880)
* basic scaffold

* remove unnecessary ?

* check if committee cache is init

* typed ValidatorMonitor with ethspecs + store attestations within

* nits

* process unaggregated attestation

* typo

* extract in func

* add tests

* better naming

* better naming 2

* less verbose

* use same naming as validator monitor

* use attestation_simulator

* add metrics

* remove cache

* refacto flag_indices process

* add lag

* remove copying state

* clean and lint

* extract metrics

* nits

* compare prom metrics in tests

* implement lag

* nits

* nits

* add attestation simulator service

* fmt

* return beacon_chain as arc

* nit: debug

* sed s/unaggregated/unagg.//

* fmt

* fmt

* nit: remove unused comments

* increase max unaggregated attestation hashmap to 64

* nit: sed s/clone/copied//

* improve perf: remove unecessary hashmap copy

* fix flag indices comp

* start service in client builder

* remove //

* cargo fmt

* lint

* cloned keys

* fmt

* use Slot value instead of pointer

* Update beacon_node/beacon_chain/src/attestation_simulator.rs

Co-authored-by: Paul Hauner <paul@paulhauner.com>

---------

Co-authored-by: Paul Hauner <paul@paulhauner.com>
2023-12-14 11:44:56 +11:00
Pawan Dhananjay
a3a370302a
Use the block header to compute the canonical_root (#5003) 2023-12-14 09:24:36 +11:00
Gua00va
a3fb27c99b
add forK_choice_read_lock as parameter (#4978) 2023-12-13 16:06:00 +11:00
Age Manning
69f1b7afec
Disable flood publishing (#4383)
* Disable flood publish

* Change default configuration
2023-12-12 09:45:54 +11:00
ethDreamer
78ffa378b4
Batch Verify RPC Blobs (#4934) 2023-12-08 16:48:03 -05:00
realbigsean
46184e5ce4
Remove delayed lookups (#4992)
* initial rip out

* fix unused imports

* delete tests and fix lint

* fix peers scoring for blobs
2023-12-08 15:42:55 -05:00
Michael Sproul
b882519d2f
Implement POST validators/validator_balances APIs (#4872)
* Add POST for fetching validators from state

* Implement POST for balances

* Tests
2023-12-08 12:09:36 +11:00
Divma
6c0c41c7ac
upgrade libp2p to v0.53.* (#4935)
* update libp2p and address compiler errors

* remove bandwidth logging from transport

* use libp2p registry

* make clippy happy

* use rust 1.73

* correct rpc keep alive

* remove comments and obsolte code

* remove libp2p prefix

* make clippy happy

* use quic under facade

* remove fast msg id

* bubble up close statements

* fix wrong comment
2023-12-07 20:39:59 +11:00
Jimmy Chen
67e0569d9b
Fix corrupted DB on networks where the first slot is skipped (Holesky) (#4985)
* Fix zero block roots on skip slots.

* Remove temporary comment, println code and unused imports.

* Remove `println!` in test.
2023-12-07 15:12:06 +11:00
ethDreamer
52117f43ba
Small Improvements (#4980)
* initial changes

* use arc<blobsidecar> in vector

* Utilize new pattern for KzgVerifiedBlob

* fmt

* Update beacon_node/beacon_chain/src/blob_verification.rs

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* forgot to save..

* lint

* fmt.. again

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2023-12-06 15:51:40 -05:00
Pawan Dhananjay
31044402ee
Sidecar inclusion proof (#4900)
* Refactor BlobSidecar to new type

* Fix some compile errors

* Gossip verification compiles

* Fix http api types take 1

* Fix another round of compile errors

* Beacon node crate compiles

* EF tests compile

* Remove all blob signing from VC

* fmt

* Tests compile

* Fix some tests

* Fix more http tests

* get compiling

* Fix gossip conditions and tests

* Add basic proof generation and verification

* remove unnecessary ssz decode

* add back build_sidecar

* remove default at fork for blobs

* fix beacon chain tests

* get relase tests compiling

* fix lints

* fix existing spec tests

* add new ef tests

* fix gossip duplicate rule

* lints

* add back sidecar signature check in gossip

* add finalized descendant check to blob sidecar gossip

* fix error conversion

* fix release tests

* sidecar inclusion self review cleanup

* Add proof verification and computation metrics

* Remove accidentally committed file

* Unify some block and blob errors; add slashing conditions for sidecars

* Address review comment

* Clean up re-org tests (#4957)

* Address more review comments

* Add Comments & Eliminate Unnecessary Clones

* update names

* Update beacon_node/beacon_chain/src/metrics.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Update beacon_node/network/src/network_beacon_processor/tests.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* pr feedback

* fix test compile

* Sidecar Inclusion proof small refactor and updates (#4967)

* Update some comments, variables and small cosmetic fixes.

* Couple blobs and proofs into a tuple in `PayloadAndBlobs` for simplicity and safety.

* Update function comment.

* Update testing/ef_tests/src/cases/merkle_proof_validity.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Rename the block and blob wrapper types used in the beacon API interfaces.

* make sure gossip invalid blobs are passed to the slasher (#4970)

* Add blob headers to slasher before adding to DA checker

* Replace Vec with HashSet in BlockQueue

* fmt

* Rename gindex -> index

* Simplify gossip condition

---------

Co-authored-by: realbigsean <seananderson33@gmail.com>
Co-authored-by: realbigsean <sean@sigmaprime.io>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: Mark Mackey <mark@sigmaprime.io>
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2023-12-05 11:19:59 -05:00
Michael Sproul
c8b2324880
Enable progressive balances fast mode by default (#4971)
* Enable progressive balances fast mode by default

* Fix default in chain_config
2023-12-04 14:42:49 +11:00
Gua00va
44aaf13ff0
Standard Liveness Endpoint (#4853)
* Changes to use required Endpoint

* Format

* fixed doppleganger service

* minor fix

* efficiency changes

* fixed tests

* remove commented line

---------

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2023-11-30 17:41:22 +11:00
Michael Sproul
547ed1de63
Clone state ahead of block production (#4925)
* Clone state ahead of block production

* Add pruning and fix logging

* Don't hold 2 states in mem
2023-11-30 13:49:35 +11:00
ethDreamer
43d98153d6
Refactor & Fix Bugs in Payload Selection Logic (#4950)
* Refactor & Fix Bugs in Payload Selection Logic

* Fix lint

* Update beacon_node/execution_layer/src/lib.rs

* Finish renaming function
2023-11-29 15:20:12 +11:00
GeemoCandama
8a599ec7dc
API for LightClientBootstrap, LightClientFinalityUpdate, LightClientOptimisticUpdate and light client events (#3954)
* rebase and add comment

* conditional test

* test

* optimistic chould be working now

* finality should be working now

* try again

* try again

* clippy fix

* add lc bootstrap beacon api

* add lc optimistic/finality update to events

* fmt

* That error isn't occuring on my computer but I think this should fix it

* Add missing test file

* Update light client types to comply with Altair light client spec.

* Fix test compilation

* Support deserializing light client structures for the Bellatrix fork

* Move `get_light_client_bootstrap` logic to `BeaconChain`. `LightClientBootstrap` API to return `ForkVersionedResponse`.

* Misc fixes.
- log cleanup
- move http_api config mutation to `config::get_config` for consistency
- fix light client API responses

* Add light client bootstrap API test and fix existing ones.

* Fix test for `light-client-server` http api config.

* Appease clippy

* Efficiency improvement when retrieving beacon state.

---------

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2023-11-28 17:14:29 +11:00
Eitan Seri-Levi
98f159cc18 fmt 2023-11-20 21:37:58 -08:00
Eitan Seri-Levi
24a0a7ffd0 remove cache check 2023-11-20 21:37:04 -08:00
Eitan Seri-Levi
228180bb35 add some more block v3 tests 2023-11-20 20:46:13 -08:00
Jimmy Chen
6b63d18420
Fix Rust beta compiler warnings (rustc 1.75.0-beta.1 (782883f60 2023-11-12)) (#4932) 2023-11-18 03:55:11 +11:00