Ethereum consensus client in Rust
b8013b7b2c
## Issue Addressed In principle.. closes #1551 but in general are improvements for performance, maintainability and readability. The logic for the optimistic sync in actually simple ## Proposed Changes There are miscellaneous things here: - Remove unnecessary `BatchProcessResult::Partial` to simplify the batch validation logic - Make batches a state machine. This is done to ensure batch state transitions respect our logic (this was previously done by moving batches between `Vec`s) and to ease the cognitive load of the `SyncingChain` struct - Move most batch-related logic to the batch - Remove `PendingBatches` in favor of a map of peers to their batches. This is to avoid duplicating peers inside the chain (peer_pool and pending_batches) - Add `must_use` decoration to the `ProcessingResult` so that chains that request to be removed are handled accordingly. This also means that chains are now removed in more places than before to account for unhandled cases - Store batches in a sorted map (`BTreeMap`) access is not O(1) but since the number of _active_ batches is bounded this should be fast, and saves performing hashing ops. Batches are indexed by the epoch they start. Sorted, to easily handle chain advancements (range logic) - Produce the chain Id from the identifying fields: target root and target slot. This, to guarantee there can't be duplicated chains and be able to consistently search chains by either Id or checkpoint - Fix chain_id not being present in all chain loggers - Handle mega-edge case where the processor's work queue is full and the batch can't be sent. In this case the chain would lose the blocks, remain in a "syncing" state and waiting for a result that won't arrive, effectively stalling sync. - When a batch imports blocks or the chain starts syncing with a local finalized epoch greater that the chain's start epoch, the chain is advanced instead of reset. This is to avoid losing download progress and validate batches faster. This also means that the old `start_epoch` now means "current first unvalidated batch", so it represents more accurately the progress of the chain. - Batch status peers from the same chain to reduce Arc access. - Handle a couple of cases where the retry counters for a batch were not updated/checked are now handled via the batch state machine. Basically now if we forget to do it, we will know. - Do not send back the blocks from the processor to the batch. Instead register the attempt before sending the blocks (does not count as failed) - When re-requesting a batch, try to avoid not only the last failed peer, but all previous failed peers. - Optimize requesting batches ahead in the buffer by shuffling idle peers just once (this is just addressing a couple of old TODOs in the code) - In chain_collection, store chains by their id in a map - Include a mapping from request_ids to (chain, batch) that requested the batch to avoid the double O(n) search on block responses - Other stuff: - impl `slog::KV` for batches - impl `slog::KV` for syncing chains - PSA: when logging, we can use `%thing` if `thing` implements `Display`. Same for `?` and `Debug` ### Optimistic syncing: Try first the batch that contains the current head, if the batch imports any block, advance the chain. If not, if this optimistic batch is inside the current processing window leave it there for future use, if not drop it. The tolerance for this block is the same for downloading, but just once for processing Co-authored-by: Age Manning <Age@AgeManning.com> |
||
|---|---|---|
| .github | ||
| account_manager | ||
| beacon_node | ||
| book | ||
| boot_node | ||
| common | ||
| consensus | ||
| crypto | ||
| hooks | ||
| lcli | ||
| lighthouse | ||
| scripts | ||
| testing | ||
| validator_client | ||
| .dockerignore | ||
| .editorconfig | ||
| .gitignore | ||
| .gitmodules | ||
| bors.toml | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CONTRIBUTING.md | ||
| Cross.toml | ||
| Dockerfile | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
Lighthouse: Ethereum 2.0
An open-source Ethereum 2.0 client, written in Rust and maintained by Sigma Prime.
Overview
Lighthouse is:
- Fully open-source, licensed under Apache 2.0.
- Security-focused. Fuzzing has begun and security reviews are underway.
- Built in Rust, a modern language providing unique safety guarantees and excellent performance (comparable to C++).
- Funded by various organisations, including Sigma Prime, the Ethereum Foundation, ConsenSys and private individuals.
- Actively involved in the specification and security analysis of the emerging Ethereum 2.0 specification.
Like all Ethereum 2.0 clients, Lighthouse is a work-in-progress.
Development Status
Current development overview:
- Specification
v0.12.1implemented, optimized and passing test vectors. - Rust-native libp2p with Gossipsub and Discv5.
- RESTful JSON API via HTTP server.
- Events via WebSocket.
- Metrics via Prometheus.
Roadmap
April 2019: Inital single-client testnets.September 2019: Inter-operability with other Ethereum 2.0 clients.Q1 2020:lighthouse-0.1.0release: All major phase 0 features implemented.Q2 2020: Public, multi-client testnet with user-facing functionality.Q2 2020: Third-party security review.- Q3 2020: Additional third-party security reviews.
- Q3 2020: Long-lived, multi-client Beacon Chain testnet
- Q4 2020: Production Beacon Chain (tentative).
Documentation
The Lighthouse Book contains information for testnet users and developers.
If you'd like some background on Sigma Prime, please see the Lighthouse Update #00 blog post or sigmaprime.io.
Contributing
Lighthouse welcomes contributors.
If you are looking to contribute, please head to the Contributing section of the Lighthouse book.
Contact
The best place for discussion is the Lighthouse Discord server. Alternatively, you may use the sigp/lighthouse gitter.
Encrypt sensitive messages using our PGP key.
Donations
Lighthouse is an open-source project and a public good. Funding public goods is hard and we're grateful for the donations we receive from the community via:
- Gitcoin Grants.
- Ethereum address:
0x25c4a76E7d118705e7Ea2e9b7d8C59930d8aCD3b(donation.sigmaprime.eth).