limit simultaneous outgoing connections attempts to a reasonable top as an extra layer of protection
also shift the keep alive logic of the rpc handler to avoid needing to update it by hand. I think In rare cases this could make shutting down a connection a bit faster.
## Issue Addressed
#1419
## Proposed Changes
Creates a `--graffiti` cli flag in the validator client. If the flag is set, it overrides graffiti in the beacon node.
## Additional Info
## Issue Addressed
#1483
## Proposed Changes
Upgrades the log to a critical if a listener fails. We are able to listen on many interfaces so a single instance is not critical. We should however gracefully shutdown the client if we have no listeners, although the client can still function solely on outgoing connections.
For now a critical is raised and I leave #1494 for more sophisticated handling of this.
This also updates discv5 to handle errors of binding to a UDP socket such that lighthouse is now able to handle them.
## Issue Addressed
Some nodes not following head, high CPU usage and HTTP API delays
## Proposed Changes
Patches gossipsub. Gossipsub was using an `lru_time_cache` to check for duplicates. This contained an `O(N)` lookup for every gossipsub message to update the time cache. This was causing high cpu usage and blocking network threads.
This PR introduces a custom cache without `O(N)` inserts.
This also adds built in safety mechanisms to prevent gossipsub from excessively retrying connections upon failure. A maximum limit is set after which we disconnect from the node from too many failed substream connections.
## Issue Addressed
Peers that connected after the peer limit may remain connected in some circumstances.
This ensures peers not in the peer manager's list get disconnected. Further logging is also added to track this behaviour.
## Issue Addressed
NA
## Proposed Changes
- When producing a block, go and ensure every attestation in the naive aggregation pool is included in the operation pool. This should help us increase the number of useful attestations in a block.
- Lift the `RwLock`s inside `NaiveAggregationPool` up into a single high-level lock. There were race conditions in the existing setup and it was hard to reason about.
## Additional Info
NA
Install cmake on macOS
## Issue Addressed
Installation error on macOS
## Proposed Changes
Add instructions for installing `cmake` on macOS via homebrew.
## Issue Addressed
#1028
A bit late, but I think if `BlockError` had a kind (the current `BlockError` minus everything on the variants that comes directly from the block) and the original block, more clones could be removed
## Issue Addressed
- Resolves#1461
## Proposed Changes
Copy the `.git` directory across when building docker so we can get commit information.
Unfortunately this means duplicating you `.git` directory which might be quite large (mine is >100mb). Notably this directory isn't contained in the final image, just the intermediate builder image.
## Additional Info
NA
## Issue Addressed
NA
## Proposed Changes
- Moves the git-based versioning we were doing into the `lighthouse_version` crate in `common`.
- Removes the `beacon_node/version` crate, replacing it with `lighthouse_version`.
- Bumps the version to `v0.2.0`.
## Additional Info
There are now two types of version string:
1. `const VERSION: &str = Lighthouse/v0.2.0-1419501f2+`
1. `version_with_platform() = Lighthouse/v0.2.0-1419501f2+/x86_64-linux`
(1) is handy cause it's a `const` and shorter. (2) has platform info so it's more useful. Note that the plus-sign (`+`) indicates the the git commit is dirty (it used to be `(modified)` but I had to shorten it to fit into graffiti).
These version strings are now included on:
- `lighthouse --version`
- `lcli --version`
- `curl localhost:5052/node/version`
- p2p messages when we communicate our version
You can update the version by changing this constant (version is not related to a `Cargo.toml`):
b9ad7102d5/common/lighthouse_version/src/lib.rs (L4-L15)
## Issue Addressed
Banned peer's scores would decay rapidly once the banning timeout expired.
## Proposed Changes
Update the decay logic for banned peers to start once the banned peer timeout expires.
## Issue Addressed
Sync was breaking occasionally. The root cause appears to be identify crashing as events we being sent to the protocol after nodes were banned. Have not been able to reproduce sync issues since this update.
## Proposed Changes
Only send messages to sub-behaviour protocols if the peer manager thinks the peer is connected. All other messages are dropped.
## Issue Addressed
NA
## Proposed Changes
- Include the Medalla `genesis.ssz` file.
- Adds the boot nodes from here: 27e7e7cc87/medalla/bootnodes.txt
## Additional Info
NA
## Issue Addressed
The most recent gossipsub update had an issue where some privacy settings lead to not sending a sequence number with the message. Although Lighthouse treats these as valid (based on current configuration) other clients may not.
This corrects gossipsub to send sequence numbers where expected and based on the configuration settings.
## Proposed Changes
In the continuing war against unportable binaries I figured we should have an option to enable building the Lighthouse binary itself with Milagro. This PR adds a `milagro` feature that can be used with `cargo install --path lighthouse --features milagro --force --locked`. The BLS library in-use will also show up under `lighthouse --version` like this:
```
Lighthouse 0.1.2-7d8acc20a(modified)
BLS Library: milagro
```
Future work: add other cool stuff like the compiler version and CPU target to `--version`.
## Proposed Changes
Having been bitten by submitting deposits with invalid signatures (for the wrong testnet), I'm proposing we check the signatures of deposits when submitting them via `lighthouse account validator deposit`.
The impact on performance is likely to be minimal because waiting for an Eth1 transaction confirmation takes a lot longer than verifying a single signature.
It would be great to have the same protection in the deposit web UI in the Lighthouse Book, but I imagine that will be obsoleted by the validator web UI work.
## Issue Addressed
#1056
## Proposed Changes
- Add a rate limiter to the RPC behaviour. This also means the rate limiting occurs just before the door to the application level, so the number of connections a peer opens does not affect this (this would happen in the future if put on the handler)
- The algorithm used is the leaky bucket as a meter / token bucket implemented the GCRA way
- Each protocol has its own limit. Due to the way the algorithm works, the "small" protocols have a hard limit, while bbrange and bbroot allow [burstiness](https://www.wikiwand.com/en/Burstiness). This is so that a peer can't request hundreds of individual requests expecting only one block in a short period of time, it also allows a peer to send two half size requests instead of one with max if they want to without getting limited, and.. it also allows a peer to request a batch of the maximum size and then send _appropriately spaced_ requests of really small sizes. From what I've seen in sync this is plausible when reaching the target slot.
## Additional Info
Needs to be heavily tested
## Issue Addressed
Closes#1395
## Proposed Changes
* Add a feature to `lighthouse` and `lcli` called `portable` which enables the `portable` feature on our fork of BLST. This feature turns off the `-march=native` C compiler flag that produces binaries highly targeted to the host CPU's instruction set.
* Tweak the `Makefile` so that when the `PORTABLE` environment variable is set to `true`, it compiles with this feature.
* Temporarily enable `PORTABLE=true` in the Docker build so that the image on Docker Hub is portable. Eventually I think we should enable `PORTABLE=true` _only on Docker Hub_, so that users building locally can take advantage of the tasty compiler magic. This seems to be possible by setting a Docker Hub environment variable: https://docs.docker.com/docker-hub/builds/#environment-variables-for-builds
## Additional Info
Tested by compiling on a very new CPU (Intel Core i7-8550U) and copying the binary to a very old CPU (Intel Core i3 530). Before the portability fix, this produced the SIGILL crash described in #1395, and after the fix, it worked smoothly.
I'm in the process of testing the Docker build and running some benches to confirm that the performance penalty isn't too severe.
## Issue Addressed
NA
## Proposed Changes
Allows for multiple "hardcoded" testnets.
## Additional Info
This PR is incomplete.
## TODO
- [x] Add flag to CLI, integrate with rest of Lighthouse.
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
## Issue Addressed
Recurring sync loop and invalid batch downloading
## Proposed Changes
Shifts the batches to include the first slot of each epoch. This ensures the finalized is always downloaded once a chain has completed syncing.
Also add in logic to prevent re-dialing disconnected peers. Non-performant peers get disconnected during sync, this prevents re-connection to these during sync.
## Additional Info
N/A
## Proposed Changes
Fixes some sharp edges on the new `lighthouse account validator list` command, and the account manager CLI.
* Validator names/keys are always printed in the same order due to the use of a sorted `BTreeMap`
* The `validator list` subcommand now respects the `--validator-dir` flag, instead of always looking in `~/.lighthouse/validators`
* The `--help` now shows a description for the `wallet` subcommand instead of just `TODO`
## Issue Addressed
N/A
## Proposed Changes
This provides a number of corrections and improvements to gossipsub. Specifically
- Enables options for greater privacy around the message author
- Provides greater flexibility on message validation
- Prevents unvalidated messages from being gossiped
- Shifts the duplicate cache to a time-based cache inside gossipsub
- Updates the message-id to handle bytes
- Bug fixes related to mesh maintenance and topic subscription. This should improve our attestation inclusion rate.
## Issue Addressed
#1388 partially (eth2_libp2p & network)
## Proposed Changes
TLDR at the end
- *Complex types* are 3 on the handlers/Behaviours but the types are `Poll<ComplexType>` where `ComplexType` comes from the traits of libp2p. Those, I don't thing are worth an alias. A couple more were from using tokio combinators and were removed writing things the async way and using [`BoxFuture`](https://docs.rs/futures/0.3.5/futures/future/type.BoxFuture.html)
- The *cognitive complexity*.. I tried to address those before (they come from the poll functions too) and tbh they are cognitively simpler to understand the way they are now. Moving separate parts to functions doesn't add much since that code is not repeated and they all do early returns. If moved those returns would now need to be wrapped in an Option, probably, and checked to be returned again. I would leave them like that but that's just preference.
- *Too many arguments*: They are not easily put together in a wrapping struct since the parameters don't relate semantically (Ex: fn new with a log, a reference to the chain, a peer, etc) but some may differ.
- *Needless returns* were indeed needless
## Additional Info
TLDR: removed needless return, used BoxFuture and async, left the rest untouched since those lgtm
