Commit Graph

5747 Commits

Author SHA1 Message Date
Jimmy Chen
81c9af5aaf
Use patched versions of common libraries 2023-05-30 22:46:22 +10:00
Jimmy Chen
70c4ae35ab
Merge branch 'unstable' into deneb-free-blobs
# Conflicts:
#	.github/workflows/docker.yml
#	.github/workflows/local-testnet.yml
#	.github/workflows/test-suite.yml
#	Cargo.lock
#	Cargo.toml
#	beacon_node/beacon_chain/src/beacon_chain.rs
#	beacon_node/beacon_chain/src/builder.rs
#	beacon_node/beacon_chain/src/test_utils.rs
#	beacon_node/execution_layer/src/engine_api/json_structures.rs
#	beacon_node/network/src/beacon_processor/mod.rs
#	beacon_node/network/src/beacon_processor/worker/gossip_methods.rs
#	beacon_node/network/src/sync/backfill_sync/mod.rs
#	beacon_node/store/src/config.rs
#	beacon_node/store/src/hot_cold_store.rs
#	common/eth2_network_config/Cargo.toml
#	consensus/ssz/src/decode/impls.rs
#	consensus/ssz_derive/src/lib.rs
#	consensus/ssz_derive/tests/tests.rs
#	consensus/ssz_types/src/serde_utils/mod.rs
#	consensus/tree_hash/src/impls.rs
#	consensus/tree_hash/src/lib.rs
#	consensus/types/Cargo.toml
#	consensus/types/src/beacon_state.rs
#	consensus/types/src/chain_spec.rs
#	consensus/types/src/eth_spec.rs
#	consensus/types/src/fork_name.rs
#	lcli/Cargo.toml
#	lcli/src/main.rs
#	lcli/src/new_testnet.rs
#	scripts/local_testnet/el_bootnode.sh
#	scripts/local_testnet/genesis.json
#	scripts/local_testnet/geth.sh
#	scripts/local_testnet/setup.sh
#	scripts/local_testnet/start_local_testnet.sh
#	scripts/local_testnet/vars.env
#	scripts/tests/doppelganger_protection.sh
#	scripts/tests/genesis.json
#	scripts/tests/vars.env
#	testing/ef_tests/Cargo.toml
#	validator_client/src/block_service.rs
2023-05-30 22:44:05 +10:00
Jimmy Chen
e6deaad91e Remove unused crate publishing Github action and script (#4347)
## Issue Addressed

`publish-crate` action and its script no longer used after https://github.com/sigp/lighthouse/pull/3890.
2023-05-30 06:15:58 +00:00
chonghe
f536932935 Add SSH tunneling in Lighthouse UI Siren (#4328)
## Issue Addressed

-

## Proposed Changes

Add information to use SSH to connect to Siren beyond the local computer

## Additional Info

Please provide any additional information. For example, future considerations
or information useful for reviewers.


Co-authored-by: chonghe <44791194+chong-he@users.noreply.github.com>
Co-authored-by: Michael Sproul <micsproul@gmail.com>
2023-05-30 06:15:57 +00:00
Age Manning
fdea8f2b27 Shift subnet backbone structure (attnets revamp) (#4304)
This PR address the following spec change: https://github.com/ethereum/consensus-specs/pull/3312

Instead of subscribing to a long-lived subnet for every attached validator to a beacon node, all beacon nodes will subscribe to `SUBNETS_PER_NODE` long-lived subnets. This is currently set to 2 for mainnet. 

This PR does not include any scoring or advanced discovery mechanisms. A future PR will improve discovery and we can implement scoring after the next hard fork when we expect all client teams and all implementations to respect this spec change.

This will be a significant change in the subnet network structure for consensus clients and we will likely have to monitor and tweak our peer management logic.
2023-05-30 06:15:56 +00:00
Daniel Ramirez Chiquillo
d150ccbee5 Add libpq-dev and docker to the to the list of additional requirements for developers in the Book (#4282)
## Issue Addressed

Realized this was missing while discussing #4280 

## Proposed Changes

Add an Item to the list of additional requirements for developers.
2023-05-30 06:15:54 +00:00
Michael Sproul
baad729fa7 Fix Rust 1.71.0 warnings (#4348)
## Issue Addressed

The Rust 1.70 release is imminent, so CI is using 1.71 for the Beta compiler, which is failing with a warning.
2023-05-30 01:38:51 +00:00
Pawan Dhananjay
a7da331f6a Fix geth scripts (#4342)
## Issue Addressed

N/A

## Proposed Changes

Geth's latest release breaks our CI with the following message
```
Fatal: Failed to register the Ethereum service: ethash is only supported as a historical component of already merged networks
Shutting down
```
Latest geth version has removed support for PoW networks. Hence, we need to add an extra `terminalTotalDifficultyPassed ` parameter in the genesis config to start from a merged network.
2023-05-30 01:38:50 +00:00
Eitan Seri-Levi
2a7e54d8bd swap unnecessary write lock to read lock in block_verification (#4340)
## Issue Addressed

[#4334](https://github.com/sigp/lighthouse/issues/4334)

## Proposed Changes

swap unnecessary write lock to read lock

## Additional Info

N/A


Co-authored-by: Michael Sproul <micsproul@gmail.com>
2023-05-30 01:38:49 +00:00
Jimmy Chen
88abaaae05 Add db inspect --output values option to support dumping raw db values (#4324)
## Issue Addressed

Add a new `--output values` option to `db inspect` for dumping raw database values to SSZ files.

This could be useful for inspecting the database when we're unable to start the beacon node.

Example usage:

```
# Output the `ForkChoice` column to an SSZ file
lighthouse db inspect --column frk --output values
```

By default, it stores the output files in the current directory, and can be overriden with `--ouput-dir`.

List of columns can be found here:
c547a11b0d/beacon_node/store/src/lib.rs (L169-L216)
2023-05-30 01:38:48 +00:00
Jimmy Chen
10318fa34b Update blog link in README (#4322)
## Issue Addressed

Update blog link in README.md

Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
2023-05-30 01:38:47 +00:00
Philippe Schommers
02ef7ae016 chore: Bellatrix occurred for Gnosis (#4301)
## Issue Addressed

None

## Proposed Changes

Tiny change to the documentation: Bellatrix happened for Gnosis so also needs a merge-ready config.

## Additional Info

None
2023-05-30 01:38:46 +00:00
Eitan Seri-Levi
744b1950e5 Keep payload cache idempotent (#4256)
## Issue Addressed

[#4239](https://github.com/sigp/lighthouse/issues/4239)

## Proposed Changes

keep the payload cache entry intact after fetching it

## Additional Info
2023-05-30 01:38:45 +00:00
Paul Hauner
c547a11b0d v4.2.0 (#4309)
## Issue Addressed

NA

## Proposed Changes

Bump versions

## Additional Info

NA
2023-05-23 00:17:10 +00:00
Age Manning
aa1ed787e9 Logging via the HTTP API (#4074)
This PR adds the ability to read the Lighthouse logs from the HTTP API for both the BN and the VC. 

This is done in such a way to as minimize any kind of performance hit by adding this feature.

The current design creates a tokio broadcast channel and mixes is into a form of slog drain that combines with our main global logger drain, only if the http api is enabled. 

The drain gets the logs, checks the log level and drops them if they are below INFO. If they are INFO or higher, it sends them via a broadcast channel only if there are users subscribed to the HTTP API channel. If not, it drops the logs. 

If there are more than one subscriber, the channel clones the log records and converts them to json in their independent HTTP API tasks. 

Co-authored-by: Michael Sproul <micsproul@gmail.com>
2023-05-22 05:57:08 +00:00
Michael Sproul
c27f2bf9c6 Avoid excessive logging of BN online status (#4315)
## Issue Addressed

https://github.com/sigp/lighthouse/pull/4309#issuecomment-1556052261

## Proposed Changes

Log the `Connected to beacon node` message only if the node was previously offline. This avoids a regression in logging after #4295, whereby the `Connected to beacon node` message would be logged every slot.

The new reduced logging is _slightly different_ from what we had prior to my changes in #4295. The main difference is that we used to log the `Connected` message whenever a node was online and subject to a health check (for being unhealthy in some other way). I think the new behaviour is reasonable, as the `Connected` message isn't particularly helpful if the BN is unhealthy, and the specific reason for unhealthiness will be logged by the warnings for `is_compatible`/`is_synced`.
2023-05-22 02:36:43 +00:00
Lion - dapplion
981c72e863 Update gnosis capella preset (#4302)
## Issue Addressed

N/A

## Proposed Changes

Gnosis preset values have been updated from previous placeholder values. This changes are required for chiado since it inherits the preset from gnosis mainnet.

- preset values update PR ref: https://github.com/gnosischain/configs/pull/11

## Additional Info

N/A
2023-05-19 10:07:12 +00:00
int88
283c327f85 add more sepolia bootnodes (#4297)
## Issue Addressed

#4288

## Proposed Changes

add more sepolia bootnodes

## Additional Info

Before add this fix:
```bash
May 16 08:13:59.161 INFO ENR Initialised                         tcp4: None, udp4: None, ip4: None, id: 0xf5b4..a912, seq: 1, enr: enr:-K24QADMSAVTKfbfTeiHkxDNMoW8OzVTsROE_FkYbo1Bny9kIEj74Q8Rqkz5c1q_PIWG9_3GvtSMfsjPI6h5S6z_wLsBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpBH63KzkAAAcv__________gmlkgnY0iXNlY3AyNTZrMaECD_KuAkUG3mbySM7ifutb6k6iRuG_q6-aUUHevTw7mayIc3luY25ldHMAg3RjcIIjKA, service: libp2p
May 16 08:13:59.162 DEBG Adding node to routing table            tcp: None, udp: Some(9001), ip: Some(178.128.150.254), peer_id: 16Uiu2HAm7xtvD82P5sCTV3N8acdrNpYKFsoY5Da3PShxaJ4zkquH, node_id: 0x2335..612e, service: libp2p
May 16 08:13:59.162 DEBG Discovery service started               service: libp2p
May 16 08:13:59.162 DEBG Starting a peer discovery request       target_peers: 16, service: libp2p
```

After add this fix:
```bash
May 16 08:52:04.263 INFO ENR Initialised                         udp6: None, tcp6: None, tcp4: Some(9000), udp4: None, ip4: None, id: 0x5065..83f0, seq: 1, enr: enr:-K24QIBmjZ2bYWd30jnYRoBGUvdGV7O0zd90FejaQWNQGIPbCZzY1VfOrdfvVv-CQdwZlCa3izxURu9BbCvZ2o_anJQBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpBH63KzkAAAcv__________gmlkgnY0iXNlY3AyNTZrMaECo-FsxPuF0qVXaGKkNvaWCMTtpscPwFzJyfSb9F7LNtiIc3luY25ldHMAg3RjcIIjKA, service: libp2p
May 16 08:52:04.263 DEBG Adding node to routing table            tcp: None, udp: Some(9001), ip: Some(178.128.150.254), peer_id: 16Uiu2HAm7xtvD82P5sCTV3N8acdrNpYKFsoY5Da3PShxaJ4zkquH, node_id: 0x2335..612e, service: libp2p
May 16 08:52:04.264 DEBG Adding node to routing table            tcp: Some(9000), udp: Some(9000), ip: Some(165.22.196.173), peer_id: 16Uiu2HAmCC5FvoVuvrTgaQyUzf8kf86D3XMe8PshNCLyiQAfzQGi, node_id: 0xfcac..7b09, service: libp2p
May 16 08:52:04.264 DEBG Adding node to routing table            tcp: Some(9000), udp: Some(9000), ip: Some(18.216.38.49), peer_id: 16Uiu2HAkxxexUkjjh9rzPTkoqvT2jvdegLpC8GXp1s6euFeiHpmW, node_id: 0x1977..3ea6, service: libp2p
May 16 08:52:04.264 DEBG Discovery service started               service: libp2p
May 16 08:52:04.264 DEBG Starting a peer discovery request       target_peers: 16, service: libp2p
```

And the nodes add to the route table is the same as the results decoded from `boot_enr.yaml` which means the fix is working!
2023-05-19 07:40:35 +00:00
Paul Hauner
01ae37ac37 Add more metrics for tracking sync messages (#4308)
## Issue Addressed

NA

## Proposed Changes

Adds metrics to track validators that are submitting equivocating (but not slashable) sync messages. This follows on from some research we've been doing in a separate fork of LH.

## Additional Info

@jimmygchen and @michaelsproul have already run their eyes over this so it should be easy to get into v4.2.0, IMO.
2023-05-19 05:13:07 +00:00
Jimmy Chen
75aea7054c Enshrine head state shuffling in the shuffling_cache (#4296)
## Issue Addressed

#4281 

## Proposed Changes

- Change `ShufflingCache` implementation from using `LruCache` to a custom cache that removes entry with lowest epoch instead of oldest insertion time.
- Protect the "enshrined" head shufflings when inserting new committee cache entries. The shuffling ids matching the head's previous, current, and future epochs will never be ejected from the cache during `Self::insert_cache_item`.

## Additional Info

There is a bonus point on shuffling preferences in the issue description that hasn't been implemented yet, as I haven't figured out a good way to do this:

> However I'm not convinced since there are some complexities around tie-breaking when two entries have the same epoch. Perhaps preferring entries in the canonical chain is best? 

We should be able to check if a block is on the canonical chain by:

```rust
canonical_head
        .fork_choice_read_lock()
        .contains_block(root)
```

However we need to interleave the shuffling and fork choice locks, which may cause deadlocks if we're not careful (mentioned by @paulhauner). Alternatively, we could use the `state.block_roots` field of the `chain.canonical_head.snapshot.beacon_state`, which avoids deadlock but requires more work.

I'd like to get some feedback on review & testing before I dig deeper into the preferences stuff, as having the canonical head preference may already be quite useful in preventing the issue raised.


Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
2023-05-19 05:13:05 +00:00
Michael Sproul
3052db29fe Implement el_offline and use it in the VC (#4295)
## Issue Addressed

Closes https://github.com/sigp/lighthouse/issues/4291, part of #3613.

## Proposed Changes

- Implement the `el_offline` field on `/eth/v1/node/syncing`. We set `el_offline=true` if:
  - The EL's internal status is `Offline` or `AuthFailed`, _or_
  - The most recent call to `newPayload` resulted in an error (more on this in a moment).

- Use the `el_offline` field in the VC to mark nodes with offline ELs as _unsynced_. These nodes will still be used, but only after synced nodes.
- Overhaul the usage of `RequireSynced` so that `::No` is used almost everywhere. The `--allow-unsynced` flag was broken and had the opposite effect to intended, so it has been deprecated.
- Add tests for the EL being offline on the upcheck call, and being offline due to the newPayload check.


## Why track `newPayload` errors?

Tracking the EL's online/offline status is too coarse-grained to be useful in practice, because:

- If the EL is timing out to some calls, it's unlikely to timeout on the `upcheck` call, which is _just_ `eth_syncing`. Every failed call is followed by an upcheck [here](693886b941/beacon_node/execution_layer/src/engines.rs (L372-L380)), which would have the effect of masking the failure and keeping the status _online_.
- The `newPayload` call is the most likely to time out. It's the call in which ELs tend to do most of their work (often 1-2 seconds), with `forkchoiceUpdated` usually returning much faster (<50ms).
- If `newPayload` is failing consistently (e.g. timing out) then this is a good indication that either the node's EL is in trouble, or the network as a whole is. In the first case validator clients _should_ prefer other BNs if they have one available. In the second case, all of their BNs will likely report `el_offline` and they'll just have to proceed with trying to use them.

## Additional Changes

- Add utility method `ForkName::latest` which is quite convenient for test writing, but probably other things too.
- Delete some stale comments from when we used to support multiple execution nodes.
2023-05-17 05:51:56 +00:00
ethDreamer
aaa118ff0e Fix PERSIST_ETH1_CACHE / PERSIST_OP_POOL Metrics (#4278)
Do these metrics ever get read? As far as I'm aware, they're only ever updated when lighthouse is shutting down?
2023-05-17 05:51:55 +00:00
Pawan Dhananjay
91a7f51ab0 Post merge local testnets (#3807)
## Issue Addressed

N/A

## Proposed Changes

Modifies the local testnet scripts to start a network with genesis validators embedded into the genesis state. This allows us to start a local testnet without the need for deploying a deposit contract or depositing validators pre-genesis.

This also enables us to start a local test network at any fork we want without going through fork transitions. Also adds scripts to start multiple geth clients and peer them with each other and peer the geth clients with beacon nodes to start a post merge local testnet.

## Additional info

Adds a new lcli command `mnemonics-validators` that generates validator directories derived from a given mnemonic. 
Adds a new `derived-genesis-state` option to the `lcli new-testnet` command to generate a genesis state populated with validators derived from a mnemonic.
2023-05-17 05:51:54 +00:00
realbigsean
9b55d74c1c
fix db startup (#4298) 2023-05-16 09:43:26 -04:00
Jack McPherson
b29bb2e037 Remove redundant gossipsub tests (#4294)
## Issue Addressed

#2335 

## Proposed Changes

 - Remove the `lighthouse-network::tests::gossipsub_tests` module
 - Remove dead code from the `lighthouse-network::tests::common` helper module (`build_full_mesh`)

## Additional Info

After discussion with both @divagant-martian and @AgeManning, these tests seem to have two main issues in that they are:

 - Redundant, in that they don't test anything meaningful (due to our handling of duplicate messages)
 - Out-of-place, in that it doesn't really test Lighthouse-specific functionality (rather libp2p functionality)

As such, this PR supersedes #4286.
2023-05-16 01:10:47 +00:00
realbigsean
c4b2f1c8ac
fix count usage in blobs by range (#4289) 2023-05-15 12:37:51 -04:00
Paul Hauner
7c0b2755c2 Don't requeue already-known RPC blocks (#4214)
## Issue Addressed

NA

## Proposed Changes

Adds an additional check to a feature introduced in #4179 to prevent us from re-queuing already-known blocks that could be rejected immediately.

## Additional Info

Ideally this would have been included in v4.1.0, however we came across it too late to release it safely. We decided that the safest path forward is to release *without* this check and then patch it in the next version. The lack of this check should only result in a very minor performance impact (the impact is totally negligible in my assessment).
2023-05-15 07:22:04 +00:00
Paul Hauner
714ed53839 Add a flag for storing invalid blocks (#4194)
## Issue Addressed

NA

## Proposed Changes

Adds a flag to store invalid blocks on disk for teh debugz. Only *some* invalid blocks are stored, those which:

- Were received via gossip (rather than RPC, for instance)
    - This keeps things simple to start with and should capture most blocks.
- Passed gossip verification
    - This reduces the ability for random people to fill up our disk. A proposer signature is required to write something to disk.

## Additional Info

It's possible that we'll store blocks that aren't necessarily invalid, but we had an internal error during verification. Those blocks seem like they might be useful sometimes.
2023-05-15 07:22:03 +00:00
Pawan Dhananjay
8a3eb4df9c Replace ganache-cli with anvil (#3555)
## Issue Addressed

N/A

## Proposed Changes

Replace ganache-cli with anvil https://github.com/foundry-rs/foundry/blob/master/anvil/README.md
We can lose all js dependencies in CI as a consequence.

## Additional info
Also changes the ethers-rs version used in the execution layer (for the transaction reconstruction) to a newer one. This was necessary to get use the ethers utils for anvil. The fixed execution engine integration tests should catch any potential issues with the payload reconstruction after #3592 


Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2023-05-15 07:22:02 +00:00
Mac L
3c029d48bf DB migration for fork choice cleanup (#4265)
## Issue Addressed

#4233

## Proposed Changes

Remove the `best_justified_checkpoint` from the `PersistedForkChoiceStore` type as it is now unused.
Additionally, remove the `Option`'s wrapping the `justified_checkpoint` and `finalized_checkpoint` fields on `ProtoNode` which were only present to facilitate a previous migration.

Include the necessary code to facilitate the migration to a new DB schema.
2023-05-15 02:10:42 +00:00
Jimmy Chen
40abaefffb Attestation verification uses head state fork (#4263)
## Issue Addressed

Addresses #4238 

## Proposed Changes

- [x] Add tests for the scenarios
- [x] Use the fork of the attestation slot for signature verification.
2023-05-15 02:10:41 +00:00
Jimmy Chen
cf239fed61 Reduce bandwidth over the VC<>BN API using dependant roots (#4170)
## Issue Addressed

#4157 

## Proposed Changes

See description in #4157.

In diagram form:

![reduce-attestation-bandwidth](https://user-images.githubusercontent.com/742762/230277084-f97301c1-0c5d-4fb3-92f9-91f99e4dc7d4.png)


Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
2023-05-15 02:10:40 +00:00
ethDreamer
46db30416d
Implement Overflow LRU Cache for Pending Blobs (#4203)
* All Necessary Objects Implement Encode/Decode

* Major Components for LRUOverflowCache Implemented

* Finish Database Code

* Add Maintenance Methods

* Added Maintenance Service

* Persist Blobs on Shutdown / Reload on Startup

* Address Clippy Complaints

* Add (emum_behaviour = "tag") to ssz_derive

* Convert Encode/Decode Implementations to "tag"

* Started Adding Tests

* Added a ton of tests

* 1 character fix

* Feature Guard Minimal Spec Tests

* Update beacon_node/beacon_chain/src/data_availability_checker.rs

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* Address Sean's Comments

* Add iter_raw_keys method

* Remove TODOs

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2023-05-12 10:08:24 -04:00
chonghe
b7b4549545 Update links in Lighthouse Book (#4279)
## Issue Addressed

Update some broken links in Lighthouse Book

## Proposed Changes

The updated links correctly link to the section of the book

## Additional Info

Please provide any additional information. For example, future considerations
or information useful for reviewers.
2023-05-10 00:33:11 +00:00
Jimmy Chen
8d9c748025 Fix attestation withdrawals root mismatch (#4249)
## Issue Addressed

Addresses #4234 

## Proposed Changes

- Skip withdrawals processing in an inconsistent state replay. 
- Repurpose `StateRootStrategy`: rename to `StateProcessingStrategy` and always skip withdrawals if using `StateProcessingStrategy::Inconsistent`
- Add a test to reproduce the scenario


Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
2023-05-09 10:48:15 +00:00
Nikita Kryuchkov
c7c51062ab Fix log on initializing external block builder (#4267)
## Issue Addressed

#4266 

## Proposed Changes

- Log `Using external block builder` instead of `Connected to external block builder` on its initialization to resolve the confusion (there's no actual connection there)

## Additional Info

The log is mentioned in builders docs, so it's changed there too.
2023-05-09 07:15:06 +00:00
ethDreamer
a22e4bf636
Implement KZG EF Tests (#4274) 2023-05-08 15:58:23 -04:00
Jack McPherson
d64be0dd78 Add branching instructions to the contribution guide (#4272)
## Issue Addressed

N/A

## Proposed Changes

 - Add an explicit base branch name (i.e., `unstable`) to the [contributing guide](https://github.com/sigp/lighthouse/blob/stable/CONTRIBUTING.md).

## Additional Info

N/A
2023-05-08 12:20:19 +00:00
Jack McPherson
6235e452e1 Do not attempt to resubscribe to core topics (#4271)
This commit adds a check to the networking service when handling core gossipsub topic subscription requests. If the BN is already subscribed to the core topics, we won't attempt to resubscribe.

## Issue Addressed

#4258 

## Proposed Changes

 - In the networking service, check if we're already subscribed to all of the core gossipsub topics and, if so, do nothing

## Additional Info

N/A
2023-05-08 07:15:26 +00:00
Age Manning
35ca086269 Backfill blocks only to the WSP by default (#4082)
## Limit Backfill Sync

This PR transitions Lighthouse from syncing all the way back to genesis to only syncing back to the weak subjectivity point (~ 5 months) when syncing via a checkpoint sync.

There are a number of important points to note with this PR:

- Firstly and most importantly, this PR fundamentally shifts the default security guarantees of checkpoint syncing in Lighthouse. Prior to this PR, Lighthouse could verify the checkpoint of any given chain by ensuring the chain eventually terminates at the corresponding genesis. This guarantee can still be employed via the new CLI flag --genesis-backfill which will prompt lighthouse to the old behaviour of downloading all blocks back to genesis. The new behaviour only checks the proposer signatures for the last 5 months of blocks but cannot guarantee the chain matches the genesis chain.
- I have not modified any of the peer scoring or RPC responses. Clients syncing from gensis, will downscore new Lighthouse peers that do not possess blocks prior to the WSP. This is by design, as Lighthouse nodes of this form, need a mechanism to sort through peers in order to find useful peers in order to complete their genesis sync. We therefore do not discriminate between empty/error responses for blocks prior or post the local WSP. If we request a block that a peer does not posses, then fundamentally that peer is less useful to us than other peers.
- This will make a radical shift in that the majority of nodes will no longer store the full history of the chain. In the future we could add a pruning mechanism to remove old blocks from the db also.


Co-authored-by: Paul Hauner <paul@paulhauner.com>
2023-05-05 03:49:23 +00:00
int88
60d13c3fea improve api-bn.md (#4244)
## Issue Addressed

NA

## Proposed Changes

improve doc `api-bn.md` to make user convenient to copy and run the command. 

## Additional Info

NA
2023-05-05 00:51:58 +00:00
int88
6d8d212da8 use state cache to optimise historical state lookup (#4228)
## Issue Addressed

#3873

## Proposed Changes

add a cache to optimise historical state lookup.

## Additional Info

N/A


Co-authored-by: Michael Sproul <micsproul@gmail.com>
2023-05-05 00:51:57 +00:00
chonghe
45835f6a6b Update Lighthouse book Sec 3-6 and FAQ (#4221)
## Issue Addressed

Update Lighthouse book to include latest information especially after Capella upgrade

## Proposed Changes

Notable changes:
- Combine Sec 4.1 & 6.1 into Sec 4, because Sec 6.1 is importing validator key which is a required step when want to run a validator
- Combine Sec 5.1 & 5.2 with Sec 5, and move Sec 5 to under Sec 9
- Added partial withdrawals in Sec 6



## Additional Info

Please provide any additional information. For example, future considerations
or information useful for reviewers.


Co-authored-by: chonghe <tanck2005@gmail.com>
2023-05-05 00:51:56 +00:00
Eitan Seri-Levi
b1416c8a43 simplify calculate_committee_fraction (#4213)
## Issue Addressed

[#4211](https://github.com/sigp/lighthouse/issues/4211)

## Proposed Changes

This PR conforms the helper function `calculate_committee_fraction` to the [v1.3.0 spec](https://github.com/ethereum/consensus-specs/blob/v1.3.0/specs/phase0/fork-choice.md#get_weight)

## Additional Info

the old definition of `calculate_committee_fraction` is almost identical, but the new definition is simpler.
2023-05-03 09:02:58 +00:00
Akihito Nakano
edbb47dd03 Update igd to v0.12.1 (#4257)
## Issue Addressed

https://github.com/sigp/lighthouse/issues/4171

## Proposed Changes

Through [this PR](https://github.com/sbstp/rust-igd/pull/56) in rust-igd, `igd` v0.12.1 no longer panics if there is an issue while searching for a gateway. So updating igd makes lighthouse emit a helpful log instead of panicking.

## Additional Info

No CHANGELOG exists in rust-igd. 👀 Here is the commit history between v0.11.1 and v0.12.1. No breaking changes.

https://github.com/sbstp/rust-igd/compare/v0.11.1...v0.12.1
2023-05-03 04:12:14 +00:00
Jimmy Chen
2aef2db66f Un-deprecate test utils functions such as extend_chain (#4255)
## Issue Addressed

This PR un-deprecates some commonly used test util functions, e.g. `extend_chain`. Most of these were deprecated in 2020 but some of us still found them quite convenient and they're still being used a lot. If there's no issue with using them, I think we should remove the "Deprecated" comment to avoid confusion.
2023-05-03 04:12:12 +00:00
Michael Sproul
bb651a503d Enable build profiles for Docker source builds (#4237)
## Proposed Changes

- Allow Docker images to be built with different profiles via e.g. `--build-arg PROFILE=maxperf`.
- Include the build profile in `lighthouse --version`.

## Additional Info

This only affects Docker images built from source. Our published Docker images use `cross`-compiled binaries that get copied into place.
2023-05-03 04:12:11 +00:00
Age Manning
616bee6757 Maintain trusted peers (#4159)
## Issue Addressed
#4150 

## Proposed Changes

Maintain trusted peers in the pruning logic. ~~In principle the changes here are not necessary as a trusted peer has a max score (100) and all other peers can have at most 0 (because we don't implement positive scores). This means that we should never prune trusted peers unless we have more trusted peers than the target peer count.~~

This change shifts this logic to explicitly never prune trusted peers which I expect is the intuitive behaviour. 

~~I suspect the issue in #4150 arises when a trusted peer disconnects from us for one reason or another and then we remove that peer from our peerdb as it becomes stale. When it re-connects at some large time later, it is no longer a trusted peer.~~

Currently we do disconnect trusted peers, and this PR corrects this to maintain trusted peers in the pruning logic.

As suggested in #4150 we maintain trusted peers in the db and thus we remember them even if they disconnect from us.
2023-05-03 04:12:10 +00:00
realbigsean
9db6b39dc3
fix check on max request size (#4250) 2023-05-02 19:14:02 -04:00
Michael Sproul
826e748629
Prepare CI for merge queues (#4252)
* Prepare CI for merge queues

* Fix syntax SNAFUs
2023-05-02 01:59:51 +00:00