b06559ae97
## Issue Addressed The non-finality period on Pyrmont between epochs [`9114`](https://pyrmont.beaconcha.in/epoch/9114) and [`9182`](https://pyrmont.beaconcha.in/epoch/9182) was contributed to by all the `lighthouse_team` validators going down. The nodes saw excessive CPU and RAM usage, resulting in the system to kill the `lighthouse bn` process. The `Restart=on-failure` directive for `systemd` caused the process to bounce in ~10-30m intervals. Diagnosis with `heaptrack` showed that the `BeaconChain::produce_unaggregated_attestation` function was calling `store::beacon_state::get_full_state` and sometimes resulting in a tree hash cache allocation. These allocations were approximately the size of the hosts physical memory and still allocated when `lighthouse bn` was killed by the OS. There was no CPU analysis (e.g., `perf`), but the `BeaconChain::produce_unaggregated_attestation` is very CPU-heavy so it is reasonable to assume it is the cause of the excessive CPU usage, too. ## Proposed Changes `BeaconChain::produce_unaggregated_attestation` has two paths: 1. Fast path: attesting to the head slot or later. 2. Slow path: attesting to a slot earlier than the head block. Path (2) is the only path that calls `store::beacon_state::get_full_state`, therefore it is the path causing this excessive CPU/RAM usage. This PR removes the current functionality of path (2) and replaces it with a static error (`BeaconChainError::AttestingPriorToHead`). This change reduces the generality of `BeaconChain::produce_unaggregated_attestation` (and therefore [`/eth/v1/validator/attestation_data`](https://ethereum.github.io/eth2.0-APIs/#/Validator/produceAttestationData)), but I argue that this functionality is an edge-case and arguably a violation of the [Honest Validator spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/validator.md). It's possible that a validator goes back to a prior slot to "catch up" and submit some missed attestations. This change would prevent such behaviour, returning an error. My concerns with this catch-up behaviour is that it is: - Not specified as "honest validator" attesting behaviour. - Is behaviour that is risky for slashing (although, all validator clients *should* have slashing protection and will eventually fail if they do not). - It disguises clock-sync issues between a BN and VC. ## Additional Info It's likely feasible to implement path (2) if we implement some sort of caching mechanism. This would be a multi-week task and this PR gets the issue patched in the short term. I haven't created an issue to add path (2), instead I think we should implement it if we get user-demand. |
||
---|---|---|
.github | ||
account_manager | ||
beacon_node | ||
book | ||
boot_node | ||
common | ||
consensus | ||
crypto | ||
lcli | ||
lighthouse | ||
remote_signer | ||
scripts | ||
slasher | ||
testing | ||
validator_client | ||
.dockerignore | ||
.editorconfig | ||
.gitignore | ||
.gitmodules | ||
bors.toml | ||
Cargo.lock | ||
Cargo.toml | ||
CONTRIBUTING.md | ||
Cross.toml | ||
Dockerfile | ||
Dockerfile.cross | ||
LICENSE | ||
Makefile | ||
README.md |
Lighthouse: Ethereum 2.0
An open-source Ethereum 2.0 client, written in Rust and maintained by Sigma Prime.
Overview
Lighthouse is:
- Ready for use on Eth2 mainnet.
- Fully open-source, licensed under Apache 2.0.
- Security-focused. Fuzzing techniques have been continuously applied and several external security reviews have been performed.
- Built in Rust, a modern language providing unique safety guarantees and excellent performance (comparable to C++).
- Funded by various organisations, including Sigma Prime, the Ethereum Foundation, ConsenSys, the Decentralization Foundation and private individuals.
- Actively involved in the specification and security analysis of the Ethereum 2.0 specification.
Eth2 Deposit Contract
The Lighthouse team acknowledges
0x00000000219ab540356cBB839Cbe05303d7705Fa
as the canonical Eth2 deposit contract address.
Documentation
The Lighthouse Book contains information for users and developers.
The Lighthouse team maintains a blog at lighthouse.sigmaprime.io which contains periodical progress updates, roadmap insights and interesting findings.
Branches
Lighthouse maintains two permanent branches:
stable
: Always points to the latest stable release.- This is ideal for most users.
unstable
: Used for development, contains the latest PRs.- Developers should base thier PRs on this branch.
Contributing
Lighthouse welcomes contributors.
If you are looking to contribute, please head to the Contributing section of the Lighthouse book.
Contact
The best place for discussion is the Lighthouse Discord server. Alternatively, you may use the sigp/lighthouse gitter.
Sign up to the Lighthouse Development Updates mailing list for email notifications about releases, network status and other important information.
Encrypt sensitive messages using our PGP key.
Donations
Lighthouse is an open-source project and a public good. Funding public goods is hard and we're grateful for the donations we receive from the community via:
- Gitcoin Grants.
- Ethereum address:
0x25c4a76E7d118705e7Ea2e9b7d8C59930d8aCD3b
(donation.sigmaprime.eth).