Ethereum consensus client in Rust
bcdd960ab1
## Proposed Changes Reduce post-merge disk usage by not storing finalized execution payloads in Lighthouse's database. ⚠️ **This is achieved in a backwards-incompatible way for networks that have already merged** ⚠️. Kiln users and shadow fork enjoyers will be unable to downgrade after running the code from this PR. The upgrade migration may take several minutes to run, and can't be aborted after it begins. The main changes are: - New column in the database called `ExecPayload`, keyed by beacon block root. - The `BeaconBlock` column now stores blinded blocks only. - Lots of places that previously used full blocks now use blinded blocks, e.g. analytics APIs, block replay in the DB, etc. - On finalization: - `prune_abanonded_forks` deletes non-canonical payloads whilst deleting non-canonical blocks. - `migrate_db` deletes finalized canonical payloads whilst deleting finalized states. - Conversions between blinded and full blocks are implemented in a compositional way, duplicating some work from Sean's PR #3134. - The execution layer has a new `get_payload_by_block_hash` method that reconstructs a payload using the EE's `eth_getBlockByHash` call. - I've tested manually that it works on Kiln, using Geth and Nethermind. - This isn't necessarily the most efficient method, and new engine APIs are being discussed to improve this: https://github.com/ethereum/execution-apis/pull/146. - We're depending on the `ethers` master branch, due to lots of recent changes. We're also using a workaround for https://github.com/gakonst/ethers-rs/issues/1134. - Payload reconstruction is used in the HTTP API via `BeaconChain::get_block`, which is now `async`. Due to the `async` fn, the `blocking_json` wrapper has been removed. - Payload reconstruction is used in network RPC to serve blocks-by-{root,range} responses. Here the `async` adjustment is messier, although I think I've managed to come up with a reasonable compromise: the handlers take the `SendOnDrop` by value so that they can drop it on _task completion_ (after the `fn` returns). Still, this is introducing disk reads onto core executor threads, which may have a negative performance impact (thoughts appreciated). ## Additional Info - [x] For performance it would be great to remove the cloning of full blocks when converting them to blinded blocks to write to disk. I'm going to experiment with a `put_block` API that takes the block by value, breaks it into a blinded block and a payload, stores the blinded block, and then re-assembles the full block for the caller. - [x] We should measure the latency of blocks-by-root and blocks-by-range responses. - [x] We should add integration tests that stress the payload reconstruction (basic tests done, issue for more extensive tests: https://github.com/sigp/lighthouse/issues/3159) - [x] We should (manually) test the schema v9 migration from several prior versions, particularly as blocks have changed on disk and some migrations rely on being able to load blocks. Co-authored-by: Paul Hauner <paul@paulhauner.com> |
||
|---|---|---|
| .github | ||
| account_manager | ||
| beacon_node | ||
| book | ||
| boot_node | ||
| common | ||
| consensus | ||
| crypto | ||
| database_manager | ||
| lcli | ||
| lighthouse | ||
| scripts | ||
| slasher | ||
| testing | ||
| validator_client | ||
| .dockerignore | ||
| .editorconfig | ||
| .gitignore | ||
| .gitmodules | ||
| bors.toml | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CONTRIBUTING.md | ||
| Cross.toml | ||
| Dockerfile | ||
| Dockerfile.cross | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| SECURITY.md | ||
Lighthouse: Ethereum consensus client
An open-source Ethereum consensus client, written in Rust and maintained by Sigma Prime.
Overview
Lighthouse is:
- Ready for use on Ethereum consensus mainnet.
- Fully open-source, licensed under Apache 2.0.
- Security-focused. Fuzzing techniques have been continuously applied and several external security reviews have been performed.
- Built in Rust, a modern language providing unique safety guarantees and excellent performance (comparable to C++).
- Funded by various organisations, including Sigma Prime, the Ethereum Foundation, ConsenSys, the Decentralization Foundation and private individuals.
- Actively involved in the specification and security analysis of the Ethereum proof-of-stake consensus specification.
Staking Deposit Contract
The Lighthouse team acknowledges
0x00000000219ab540356cBB839Cbe05303d7705Fa
as the canonical staking deposit contract address.
Documentation
The Lighthouse Book contains information for users and developers.
The Lighthouse team maintains a blog at lighthouse-blog.sigmaprime.io which contains periodical progress updates, roadmap insights and interesting findings.
Branches
Lighthouse maintains two permanent branches:
stable: Always points to the latest stable release.- This is ideal for most users.
unstable: Used for development, contains the latest PRs.- Developers should base their PRs on this branch.
Contributing
Lighthouse welcomes contributors.
If you are looking to contribute, please head to the Contributing section of the Lighthouse book.
Contact
The best place for discussion is the Lighthouse Discord server.
Sign up to the Lighthouse Development Updates mailing list for email notifications about releases, network status and other important information.
Encrypt sensitive messages using our PGP key.
Donations
Lighthouse is an open-source project and a public good. Funding public goods is hard and we're grateful for the donations we receive from the community via:
- Gitcoin Grants.
- Ethereum address:
0x25c4a76E7d118705e7Ea2e9b7d8C59930d8aCD3b(donation.sigmaprime.eth).