Commit Graph

5397 Commits

Author SHA1 Message Date
Emilia Hane
2f565d25b2
Prune blobs in bg after canonical head update 2023-02-08 11:44:35 +01:00
Emilia Hane
8752deeced
Store orphan block roots 2023-02-08 11:44:35 +01:00
Emilia Hane
c7f53a9062
Delete blobs that conflict with finalization 2023-02-08 11:44:34 +01:00
Emilia Hane
94aa2cef67
Log info loaded from disk 2023-02-08 11:44:34 +01:00
Emilia Hane
a2b8c6ee69
Save fetching state for blobs pruning 2023-02-08 11:44:33 +01:00
Emilia Hane
6f5ca02ac9
Improve syntax
Co-authored-by: Michael Sproul <micsproul@gmail.com>
2023-02-08 11:44:33 +01:00
Emilia Hane
667cca5cf2
Fix try_prune_blobs to use state root 2023-02-08 11:44:33 +01:00
Emilia Hane
d67468d737
Prune blobs on migration in addition to start-up 2023-02-08 11:44:32 +01:00
Emilia Hane
82ffec378a
Fix typo
Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2023-02-08 11:44:32 +01:00
Emilia Hane
ce2db355de
Fix rebase conflict 2023-02-08 11:44:32 +01:00
Emilia Hane
a211e6afee
Fix rebase conflict 2023-02-08 11:44:31 +01:00
Emilia Hane
28e1e635c3
Fix rebase conflict 2023-02-08 11:44:31 +01:00
Emilia Hane
d3b94d8617
fixup! Prune blobs before data availability breakpoint 2023-02-08 11:44:31 +01:00
Emilia Hane
934f3ab587
Remove inaccurate guess for db index 2023-02-08 11:44:31 +01:00
Emilia Hane
d21c66ddf4
fixup! Plug in pruning of blobs into app 2023-02-08 11:44:31 +01:00
Emilia Hane
b88d888145
fixup! Plug in pruning of blobs into app 2023-02-08 11:44:30 +01:00
Emilia Hane
2a41f25d68
fixup! Prune blobs before data availability breakpoint 2023-02-08 11:44:30 +01:00
Emilia Hane
fe0c911402
Plug in pruning of blobs into app 2023-02-08 11:44:30 +01:00
Emilia Hane
7bf88c2336
Prune blobs before data availability breakpoint 2023-02-08 11:44:30 +01:00
Emilia Hane
e2a6da4274
Boiler plate code for blobs pruning 2023-02-08 11:44:20 +01:00
realbigsean
8661477675
use hex decode instead of parse 2023-02-07 21:51:19 -05:00
Michael Sproul
1cae98856c Update dependencies (#3946)
## Issue Addressed

Resolves the cargo-audit failure caused by https://rustsec.org/advisories/RUSTSEC-2023-0010.

I also removed the ignore for `RUSTSEC-2020-0159` as we are no longer using a vulnerable version of `chrono`. We still need the other ignore for `time 0.1` because we depend on it via `sloggers -> chrono -> time 0.1`.
2023-02-08 02:18:54 +00:00
Divma
ceb986549d Self rate limiting dev flag (#3928)
## Issue Addressed
Adds self rate limiting options, mainly with the idea to comply with peer's rate limits in small testnets

## Proposed Changes
Add a hidden flag `self-limiter` this can take no value, or customs values to configure quotas per protocol

## Additional Info
### How to use
`--self-limiter` will turn on the self rate limiter applying the same params we apply to inbound requests (requests from other peers)
`--self-limiter "beacon_blocks_by_range:64/1"` will turn on the self rate limiter for ALL protocols, but change the quota for bbrange to 64 requested blocks per 1 second.
`--self-limiter "beacon_blocks_by_range:64/1;ping:1/10"` same as previous one, changing the quota for ping as well.

### Caveats
- The rate limiter is either on or off for all protocols. I added the custom values to be able to change the quotas per protocol so that some protocols can be given extremely loose or tight quotas. I think this should satisfy every need even if we can't technically turn off rate limits per protocol.
- This reuses the rate limiter struct for the inbound requests so there is this ugly part of the code in which we need to deal with the inbound only protocols (light client stuff) if this becomes too ugly as we add lc protocols, we might want to split the rate limiters. I've checked this and looks doable with const generics to avoid so much code duplication

### Knowing if this is on
```
Feb 06 21:12:05.493 DEBG Using self rate limiting params         config: OutboundRateLimiterConfig { ping: 2/10s, metadata: 1/15s, status: 5/15s, goodbye: 1/10s, blocks_by_range: 1024/10s, blocks_by_root: 128/10s }, service: libp2p_rpc, service: libp2p
```
2023-02-08 02:18:53 +00:00
Nazar Hussain
7934485aef Update the docker build to include features based images (#3875)
## Proposed Changes

There are some features that are enabled/disabled with the `FEATURES` env variable. This PR would introduce a pattern to introduce docker images based on those features. This can be useful later on to have specific images for some experimental features in the future.

## Additional Info

We at Lodesart need to have `minimal` spec support for some cross-client network testing. To make it efficient on the CI, we tend to use minimal preset.
2023-02-08 02:18:51 +00:00
Diva M
493784366f
self rate limiting 2023-02-07 13:00:35 -05:00
realbigsean
a42d07592c
fix compilation issues after merge 2023-02-07 12:33:29 -05:00
realbigsean
26a296246d
Merge branch 'capella' of https://github.com/sigp/lighthouse into eip4844
# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs
#	beacon_node/beacon_chain/src/block_verification.rs
#	beacon_node/beacon_chain/src/test_utils.rs
#	beacon_node/execution_layer/src/engine_api.rs
#	beacon_node/execution_layer/src/engine_api/http.rs
#	beacon_node/execution_layer/src/lib.rs
#	beacon_node/execution_layer/src/test_utils/handle_rpc.rs
#	beacon_node/http_api/src/lib.rs
#	beacon_node/http_api/tests/fork_tests.rs
#	beacon_node/network/src/beacon_processor/mod.rs
#	beacon_node/network/src/beacon_processor/work_reprocessing_queue.rs
#	beacon_node/network/src/beacon_processor/worker/sync_methods.rs
#	beacon_node/operation_pool/src/bls_to_execution_changes.rs
#	beacon_node/operation_pool/src/lib.rs
#	beacon_node/operation_pool/src/persistence.rs
#	consensus/serde_utils/src/u256_hex_be_opt.rs
#	testing/antithesis/Dockerfile.libvoidstar
2023-02-07 12:12:56 -05:00
realbigsean
21e5b7fecd
Merge pull request #3941 from realbigsean/blob-decoding
Blob decoding
2023-02-07 08:38:20 -05:00
realbigsean
3533ed418e
pr feedback and bugfixes 2023-02-07 08:36:35 -05:00
naviechan
9547ac069c Implement block_rewards API (per-validator reward) (#3907)
## Issue Addressed

[#3661](https://github.com/sigp/lighthouse/issues/3661)

## Proposed Changes

`/eth/v1/beacon/rewards/blocks/{block_id}`

```
{
  "execution_optimistic": false,
  "finalized": false,
  "data": {
    "proposer_index": "123",
    "total": "123",
    "attestations": "123",
    "sync_aggregate": "123",
    "proposer_slashings": "123",
    "attester_slashings": "123"
  }
}
```

The issue contains the implementation of three per-validator reward APIs:
* `sync_committee_rewards`
* [`attestation_rewards`](https://github.com/sigp/lighthouse/pull/3822)
* `block_rewards`

This PR only implements the `block_rewards`.

The endpoints can be viewed in the Ethereum Beacon nodes API browser: [https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Rewards](https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Rewards)

## Additional Info

The implementation of [consensus client reward APIs](https://github.com/eth-protocol-fellows/cohort-three/blob/master/projects/project-ideas.md#consensus-client-reward-apis) is part of the [EPF](https://github.com/eth-protocol-fellows/cohort-three).

Co-authored-by: kevinbogner <kevbogner@gmail.com>
Co-authored-by: navie <naviechan@gmail.com>
2023-02-07 08:33:23 +00:00
Paul Hauner
e062a7cf76
Broadcast address changes at Capella (#3919)
* Add first efforts at broadcast

* Tidy

* Move broadcast code to client

* Progress with broadcast impl

* Rename to address change

* Fix compile errors

* Use `while` loop

* Tidy

* Flip broadcast condition

* Switch to forgetting individual indices

* Always broadcast when the node starts

* Refactor into two functions

* Add testing

* Add another test

* Tidy, add more testing

* Tidy

* Add test, rename enum

* Rename enum again

* Tidy

* Break loop early

* Add V15 schema migration

* Bump schema version

* Progress with migration

* Update beacon_node/client/src/address_change_broadcast.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix typo in function name

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>
2023-02-07 17:13:49 +11:00
Michael Sproul
2073518f0f
Remove unused u256_hex_be_opt (#3942) 2023-02-07 11:23:36 +11:00
kevinbogner
4d07e40501 Implement attestation_rewards API (per-validator reward) (#3822)
## Issue Addressed

#3661 

## Proposed Changes
`/eth/v1/beacon/rewards/attestations/{epoch}`

```json
{
  "execution_optimistic": false,
  "finalized": false,
  "data": [
    {
      "ideal_rewards": [
        {
          "effective_balance": "1000000000",
          "head": "2500",
          "target": "5000",
          "source": "5000"
        }
      ],
      "total_rewards": [
        {
          "validator_index": "0",
          "head": "2000",
          "target": "2000",
          "source": "4000",
          "inclusion_delay": "2000"
        }
      ]
    }
  ]
}
```

The issue contains the implementation of three per-validator reward APIs:
- [`sync_committee_rewards`](https://github.com/sigp/lighthouse/pull/3790)
- `attestation_rewards`
- `block_rewards`.

This PR *only* implements the `attestation_rewards`.

The endpoints can be viewed in the Ethereum Beacon nodes API browser: https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Rewards

## Additional Info
The implementation of [consensus client reward APIs](https://github.com/eth-protocol-fellows/cohort-three/blob/master/projects/project-ideas.md#consensus-client-reward-apis) is part of the [EPF](https://github.com/eth-protocol-fellows/cohort-three).

---
- [x] `get_state`
- [x] Calculate *ideal rewards* with some logic from  `get_flag_index_deltas`
- [x] Calculate *actual rewards*  with some logic from `get_flag_index_deltas`
- [x] Code cleanup
- [x] Testing
2023-02-07 00:00:19 +00:00
Michael Sproul
c56706efae Unpin fixed-hash (#3917)
## Proposed Changes
Remove the `[patch]` for `fixed-hash`.

We pinned it years ago in #2710 to fix `arbitrary` support. Nowadays the 0.7 version of `fixed-hash` is only used by the `web3` crate and doesn't need `arbitrary`.

~~Blocked on #3916 but could be merged in the same Bors batch.~~
2023-02-06 04:18:03 +00:00
ethDreamer
5b398b1990
Don't Reject all Builder Bids After Capella (#3940)
* Fix bug in Builder API Post-Capella

* Fix Clippy Complaints
2023-02-06 13:09:13 +11:00
sean
e5896d9b71 re-order methods 2023-02-05 18:14:24 -05:00
sean
1315098c15 variable list from -> new 2023-02-05 17:56:03 -05:00
sean
38db8d7952 add back in 4844 tx consistencycheck during payload reconstruction 2023-02-05 17:31:20 -05:00
sean
f22aac1603 improve error handling 2023-02-05 17:26:36 -05:00
sean
90e25dc6cf from to new 2023-02-05 16:55:55 -05:00
sean
94a369b9df blob tx decoding 2023-02-05 16:39:15 -05:00
realbigsean
d9e83e6cec blob decoding 2023-02-03 17:06:33 -05:00
ethDreamer
90b6ae62e6
Use Local Payload if More Profitable than Builder (#3934)
* Use Local Payload if More Profitable than Builder

* Rename clone -> clone_from_ref

* Minimize Clones of GetPayloadResponse

* Cleanup & Fix Tests

* Added Tests for Payload Choice by Profit

* Fix Outdated Comments
2023-02-01 19:37:46 -06:00
Michael Sproul
4d0955cd0b
Merge pull request #3933 from ethDreamer/capella
Merge `unstable` into `capella`
2023-02-01 09:25:49 +11:00
Mark Mackey
d842215a44
Merge branch 'upstream/unstable' into capella 2023-01-31 12:16:26 -06:00
ethDreamer
7b7595347d
exchangeCapabilities & Capella Readiness Logging (#3918)
* Undo Passing Spec to Engine API

* Utilize engine_exchangeCapabilities

* Add Logging to Indicate Capella Readiness

* Add exchangeCapabilities to mock_execution_layer

* Send Nested Array for engine_exchangeCapabilities

* Use Mutex Instead of RwLock for EngineCapabilities

* Improve Locking to Avoid Deadlock

* Prettier logic for get_engine_capabilities

* Improve Comments

* Update beacon_node/beacon_chain/src/capella_readiness.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/beacon_chain/src/capella_readiness.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/beacon_chain/src/capella_readiness.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/beacon_chain/src/capella_readiness.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/beacon_chain/src/capella_readiness.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/client/src/notifier.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/execution_layer/src/engine_api/http.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Addressed Michael's Comments

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>
2023-01-31 18:26:23 +01:00
realbigsean
6d2dff66a0
Merge pull request #3926 from divagant-martian/stream-timeout-blob-bug
send stream terminators
2023-01-27 19:05:04 +01:00
realbigsean
b7e20fb87a
Update beacon_node/lighthouse_network/src/rpc/protocol.rs 2023-01-27 19:03:43 +01:00
Diva M
9976d3bbbc
send stream terminators 2023-01-27 18:11:26 +01:00
realbigsean
c0bdc1d120
Merge pull request #3920 from realbigsean/fix-and-loosen-execution-block-decoding
Fix and loosen execution block decoding
2023-01-27 18:10:47 +01:00