cerc-io/lighthouse
16
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity
6625aa4afe
lighthouse/slasher/Cargo.toml

35 lines
813 B
TOML
Raw Normal View History

Implement slasher (#1567) This is an implementation of a slasher that lives inside the BN and can be enabled via `lighthouse bn --slasher`. Features included in this PR: - [x] Detection of attester slashing conditions (double votes, surrounds existing, surrounded by existing) - [x] Integration into Lighthouse's attestation verification flow - [x] Detection of proposer slashing conditions - [x] Extraction of attestations from blocks as they are verified - [x] Compression of chunks - [x] Configurable history length - [x] Pruning of old attestations and blocks - [x] More tests Future work: * Focus on a slice of history separate from the most recent N epochs (e.g. epochs `current - K` to `current - M`) * Run out-of-process * Ingest attestations from the chain without a resync Design notes are here https://hackmd.io/@sproul/HJSEklmPL
2020-11-23 03:43:22 +00:00
[package]
name = "slasher"
version = "0.1.0"
authors = ["Michael Sproul <michael@sigmaprime.io>"]
edition = "2018"
[dependencies]
bincode = "1.3.1"
byteorder = "1.3.4"
Implement SSZ union type (#2579) ## Issue Addressed NA ## Proposed Changes Implements the "union" type from the SSZ spec for `ssz`, `ssz_derive`, `tree_hash` and `tree_hash_derive` so it may be derived for `enums`: https://github.com/ethereum/consensus-specs/blob/v1.1.0-beta.3/ssz/simple-serialize.md#union The union type is required for the merge, since the `Transaction` type is defined as a single-variant union `Union[OpaqueTransaction]`. ### Crate Updates This PR will (hopefully) cause CI to publish new versions for the following crates: - `eth2_ssz_derive`: `0.2.1` -> `0.3.0` - `eth2_ssz`: `0.3.0` -> `0.4.0` - `eth2_ssz_types`: `0.2.0` -> `0.2.1` - `tree_hash`: `0.3.0` -> `0.4.0` - `tree_hash_derive`: `0.3.0` -> `0.4.0` These these crates depend on each other, I've had to add a workspace-level `[patch]` for these crates. A follow-up PR will need to remove this patch, ones the new versions are published. ### Union Behaviors We already had SSZ `Encode` and `TreeHash` derive for enums, however it just did a "transparent" pass-through of the inner value. Since the "union" decoding from the spec is in conflict with the transparent method, I've required that all `enum` have exactly one of the following enum-level attributes: #### SSZ - `#[ssz(enum_behaviour = "union")]` - matches the spec used for the merge - `#[ssz(enum_behaviour = "transparent")]` - maintains existing functionality - not supported for `Decode` (never was) #### TreeHash - `#[tree_hash(enum_behaviour = "union")]` - matches the spec used for the merge - `#[tree_hash(enum_behaviour = "transparent")]` - maintains existing functionality This means that we can maintain the existing transparent behaviour, but all existing users will get a compile-time error until they explicitly opt-in to being transparent. ### Legacy Option Encoding Before this PR, we already had a union-esque encoding for `Option<T>`. However, this was with the *old* SSZ spec where the union selector was 4 bytes. During merge specification, the spec was changed to use 1 byte for the selector. Whilst the 4-byte `Option` encoding was never used in the spec, we used it in our database. Writing a migrate script for all occurrences of `Option` in the database would be painful, especially since it's used in the `CommitteeCache`. To avoid the migrate script, I added a serde-esque `#[ssz(with = "module")]` field-level attribute to `ssz_derive` so that we can opt into the 4-byte encoding on a field-by-field basis. The `ssz::legacy::four_byte_impl!` macro allows a one-liner to define the module required for the `#[ssz(with = "module")]` for some `Option<T> where T: Encode + Decode`. Notably, **I have removed `Encode` and `Decode` impls for `Option`**. I've done this to force a break on downstream users. Like I mentioned, `Option` isn't used in the spec so I don't think it'll be *that* annoying. I think it's nicer than quietly having two different union implementations or quietly breaking the existing `Option` impl. ### Crate Publish Ordering I've modified the order in which CI publishes crates to ensure that we don't publish a crate without ensuring we already published a crate that it depends upon. ## TODO - [ ] Queue a follow-up `[patch]`-removing PR.
2021-09-25 05:58:36 +00:00
eth2_ssz = "0.4.0"
eth2_ssz_derive = "0.3.0"
Implement slasher (#1567) This is an implementation of a slasher that lives inside the BN and can be enabled via `lighthouse bn --slasher`. Features included in this PR: - [x] Detection of attester slashing conditions (double votes, surrounds existing, surrounded by existing) - [x] Integration into Lighthouse's attestation verification flow - [x] Detection of proposer slashing conditions - [x] Extraction of attestations from blocks as they are verified - [x] Compression of chunks - [x] Configurable history length - [x] Pruning of old attestations and blocks - [x] More tests Future work: * Focus on a slice of history separate from the most recent N epochs (e.g. epochs `current - K` to `current - M`) * Run out-of-process * Ingest attestations from the chain without a resync Design notes are here https://hackmd.io/@sproul/HJSEklmPL
2020-11-23 03:43:22 +00:00
flate2 = { version = "1.0.14", features = ["zlib"], default-features = false }
lazy_static = "1.4.0"
lighthouse_metrics = { path = "../common/lighthouse_metrics" }
Enable Compatibility with Windows (#2333) ## Issue Addressed Windows incompatibility. ## Proposed Changes On windows, lighthouse needs to default to STDIN as tty doesn't exist. Also Windows uses ACLs for file permissions. So to mirror chmod 600, we will remove every entry in a file's ACL and add only a single SID that is an alias for the file owner. Beyond that, there were several changes made to different unit tests because windows has slightly different error messages as well as frustrating nuances around killing a process :/ ## Additional Info Tested on my Windows VM and it appears to work, also compiled & tested on Linux with these changes. Permissions look correct on both platforms now. Just waiting for my validator to activate on Prater so I can test running full validator client on windows. Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> Co-authored-by: Michael Sproul <micsproul@gmail.com>
2021-05-19 23:05:16 +00:00
filesystem = { path = "../common/filesystem" }
Implement slasher (#1567) This is an implementation of a slasher that lives inside the BN and can be enabled via `lighthouse bn --slasher`. Features included in this PR: - [x] Detection of attester slashing conditions (double votes, surrounds existing, surrounded by existing) - [x] Integration into Lighthouse's attestation verification flow - [x] Detection of proposer slashing conditions - [x] Extraction of attestations from blocks as they are verified - [x] Compression of chunks - [x] Configurable history length - [x] Pruning of old attestations and blocks - [x] More tests Future work: * Focus on a slice of history separate from the most recent N epochs (e.g. epochs `current - K` to `current - M`) * Run out-of-process * Ingest attestations from the chain without a resync Design notes are here https://hackmd.io/@sproul/HJSEklmPL
2020-11-23 03:43:22 +00:00
lmdb = "0.8"
lmdb-sys = "0.8"
parking_lot = "0.11.0"
Upgrade dependencies (#2513) ## Proposed Changes * Consolidate Tokio versions: everything now uses the latest v1.10.0, no more `tokio-compat`. * Many semver-compatible changes via `cargo update`. Notably this upgrades from the yanked v0.8.0 version of crossbeam-deque which is present in v1.5.0-rc.0 * Many semver incompatible upgrades via `cargo upgrades` and `cargo upgrade --workspace pkg_name`. Notable ommissions: - Prometheus, to be handled separately: https://github.com/sigp/lighthouse/issues/2485 - `rand`, `rand_xorshift`: the libsecp256k1 package requires 0.7.x, so we'll stick with that for now - `ethereum-types` is pinned at 0.11.0 because that's what `web3` is using and it seems nice to have just a single version ## Additional Info We still have two versions of `libp2p-core` due to `discv5` depending on the v0.29.0 release rather than `master`. AFAIK it should be OK to release in this state (cc @AgeManning )
2021-08-17 01:00:24 +00:00
rand = "0.7.3"
Implement slasher (#1567) This is an implementation of a slasher that lives inside the BN and can be enabled via `lighthouse bn --slasher`. Features included in this PR: - [x] Detection of attester slashing conditions (double votes, surrounds existing, surrounded by existing) - [x] Integration into Lighthouse's attestation verification flow - [x] Detection of proposer slashing conditions - [x] Extraction of attestations from blocks as they are verified - [x] Compression of chunks - [x] Configurable history length - [x] Pruning of old attestations and blocks - [x] More tests Future work: * Focus on a slice of history separate from the most recent N epochs (e.g. epochs `current - K` to `current - M`) * Run out-of-process * Ingest attestations from the chain without a resync Design notes are here https://hackmd.io/@sproul/HJSEklmPL
2020-11-23 03:43:22 +00:00
safe_arith = { path = "../consensus/safe_arith" }
serde = "1.0"
serde_derive = "1.0"
slog = "2.5.2"
Update sloggers to v2.0.2 (#2588) fixes #2584
2021-09-14 06:48:26 +00:00
sloggers = "2.0.2"
Implement SSZ union type (#2579) ## Issue Addressed NA ## Proposed Changes Implements the "union" type from the SSZ spec for `ssz`, `ssz_derive`, `tree_hash` and `tree_hash_derive` so it may be derived for `enums`: https://github.com/ethereum/consensus-specs/blob/v1.1.0-beta.3/ssz/simple-serialize.md#union The union type is required for the merge, since the `Transaction` type is defined as a single-variant union `Union[OpaqueTransaction]`. ### Crate Updates This PR will (hopefully) cause CI to publish new versions for the following crates: - `eth2_ssz_derive`: `0.2.1` -> `0.3.0` - `eth2_ssz`: `0.3.0` -> `0.4.0` - `eth2_ssz_types`: `0.2.0` -> `0.2.1` - `tree_hash`: `0.3.0` -> `0.4.0` - `tree_hash_derive`: `0.3.0` -> `0.4.0` These these crates depend on each other, I've had to add a workspace-level `[patch]` for these crates. A follow-up PR will need to remove this patch, ones the new versions are published. ### Union Behaviors We already had SSZ `Encode` and `TreeHash` derive for enums, however it just did a "transparent" pass-through of the inner value. Since the "union" decoding from the spec is in conflict with the transparent method, I've required that all `enum` have exactly one of the following enum-level attributes: #### SSZ - `#[ssz(enum_behaviour = "union")]` - matches the spec used for the merge - `#[ssz(enum_behaviour = "transparent")]` - maintains existing functionality - not supported for `Decode` (never was) #### TreeHash - `#[tree_hash(enum_behaviour = "union")]` - matches the spec used for the merge - `#[tree_hash(enum_behaviour = "transparent")]` - maintains existing functionality This means that we can maintain the existing transparent behaviour, but all existing users will get a compile-time error until they explicitly opt-in to being transparent. ### Legacy Option Encoding Before this PR, we already had a union-esque encoding for `Option<T>`. However, this was with the *old* SSZ spec where the union selector was 4 bytes. During merge specification, the spec was changed to use 1 byte for the selector. Whilst the 4-byte `Option` encoding was never used in the spec, we used it in our database. Writing a migrate script for all occurrences of `Option` in the database would be painful, especially since it's used in the `CommitteeCache`. To avoid the migrate script, I added a serde-esque `#[ssz(with = "module")]` field-level attribute to `ssz_derive` so that we can opt into the 4-byte encoding on a field-by-field basis. The `ssz::legacy::four_byte_impl!` macro allows a one-liner to define the module required for the `#[ssz(with = "module")]` for some `Option<T> where T: Encode + Decode`. Notably, **I have removed `Encode` and `Decode` impls for `Option`**. I've done this to force a break on downstream users. Like I mentioned, `Option` isn't used in the spec so I don't think it'll be *that* annoying. I think it's nicer than quietly having two different union implementations or quietly breaking the existing `Option` impl. ### Crate Publish Ordering I've modified the order in which CI publishes crates to ensure that we don't publish a crate without ensuring we already published a crate that it depends upon. ## TODO - [ ] Queue a follow-up `[patch]`-removing PR.
2021-09-25 05:58:36 +00:00
tree_hash = "0.4.0"
tree_hash_derive = "0.4.0"
Implement slasher (#1567) This is an implementation of a slasher that lives inside the BN and can be enabled via `lighthouse bn --slasher`. Features included in this PR: - [x] Detection of attester slashing conditions (double votes, surrounds existing, surrounded by existing) - [x] Integration into Lighthouse's attestation verification flow - [x] Detection of proposer slashing conditions - [x] Extraction of attestations from blocks as they are verified - [x] Compression of chunks - [x] Configurable history length - [x] Pruning of old attestations and blocks - [x] More tests Future work: * Focus on a slice of history separate from the most recent N epochs (e.g. epochs `current - K` to `current - M`) * Run out-of-process * Ingest attestations from the chain without a resync Design notes are here https://hackmd.io/@sproul/HJSEklmPL
2020-11-23 03:43:22 +00:00
types = { path = "../consensus/types" }
[dev-dependencies]
maplit = "1.0.2"
rayon = "1.3.0"
replace tempdir by tempfile (#2143) ## Issue Addressed Fixes #2141 Remove [tempdir](https://docs.rs/tempdir/0.3.7/tempdir/) in favor of [tempfile](https://docs.rs/tempfile/3.1.0/tempfile/). ## Proposed Changes `tempfile` has a slightly different api that makes creating temp folders with a name prefix a chore (`tempdir::TempDir::new("toto")` => `tempfile::Builder::new().prefix("toto").tempdir()`). So I removed temp folder name prefix where I deemed it not useful. Otherwise, the functionality is the same.
2021-01-06 06:36:11 +00:00
tempfile = "3.1.0"
Add test_logger as feature to logging (#2586) ## Issue Addressed Fix #2585 ## Proposed Changes Provide a canonical version of test_logger that can be used throughout lighthouse. ## Additional Info This allows tests to conditionally emit logging data by adding test_logger as the default logger. And then when executing `cargo test --features logging/test_logger` log output will be visible: wink@3900x:~/lighthouse/common/logging/tests/test-feature-test_logger (Add-test_logger-as-feature-to-logging) $ cargo test --features logging/test_logger Finished test [unoptimized + debuginfo] target(s) in 0.02s Running unittests (target/debug/deps/test_logger-e20115db6a5e3714) running 1 test Sep 10 12:53:45.212 INFO hi, module: test_logger:8 test tests::test_fn_with_logging ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s Doc-tests test-logger running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s Or, in normal scenarios where logging isn't needed, executing `cargo test` the log output will not be visible: wink@3900x:~/lighthouse/common/logging/tests/test-feature-test_logger (Add-test_logger-as-feature-to-logging) $ cargo test Finished test [unoptimized + debuginfo] target(s) in 0.02s Running unittests (target/debug/deps/test_logger-02e02f8d41e8cf8a) running 1 test test tests::test_fn_with_logging ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s Doc-tests test-logger running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
2021-10-06 00:46:07 +00:00
logging = { path = "../common/logging" }
Implement slasher (#1567) This is an implementation of a slasher that lives inside the BN and can be enabled via `lighthouse bn --slasher`. Features included in this PR: - [x] Detection of attester slashing conditions (double votes, surrounds existing, surrounded by existing) - [x] Integration into Lighthouse's attestation verification flow - [x] Detection of proposer slashing conditions - [x] Extraction of attestations from blocks as they are verified - [x] Compression of chunks - [x] Configurable history length - [x] Pruning of old attestations and blocks - [x] More tests Future work: * Focus on a slice of history separate from the most recent N epochs (e.g. epochs `current - K` to `current - M`) * Run out-of-process * Ingest attestations from the chain without a resync Design notes are here https://hackmd.io/@sproul/HJSEklmPL
2020-11-23 03:43:22 +00:00
Reference in New Issue Copy Permalink