lighthouse/testing/state_transition_vectors/Cargo.toml

[package]
name = "state_transition_vectors"
version = "0.1.0"
authors = ["Paul Hauner <paul@paulhauner.com>"]
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
state_processing = { path = "../../consensus/state_processing" }
types = { path = "../../consensus/types" }
eth2_ssz = "0.4.1"
beacon_chain = { path = "../../beacon_node/beacon_chain" }
lazy_static = "1.4.0"
tokio = { version = "1.14.0", features = ["rt-multi-thread"] }
Fix parallelism bug in exit processing (#1110) * Fix parallelism bug in exit processing Also: * Remove parallelism for all other operations except deposit merkle proofs * Improve exit tests * Fix broken attestation test Closes #1090 * Allow for generating block/pre/post states from some unit tests (#1123) * Add post-state checks, comments * Add state_transition_vectors crate * Integrate new testing crate with CI * Add readme * Add additional valid tests * Remove ExitTests (they were moved to new crate) * Small test fixes * Delete incorrect saturating_sub in slash_validator And clean-up the balance increase/decrease functions to look more like the spec. Co-authored-by: Paul Hauner <paul@paulhauner.com> 2020-05-08 23:37:21 +00:00			`[package]`
			`name = "state_transition_vectors"`
			`version = "0.1.0"`
			`authors = ["Paul Hauner <paul@paulhauner.com>"]`
Update to Rust 1.59 and 2021 edition (#3038) ## Proposed Changes Lots of lint updates related to `flat_map`, `unwrap_or_else` and string patterns. I did a little more creative refactoring in the op pool, but otherwise followed Clippy's suggestions. ## Additional Info We need this PR to unblock CI. 2022-02-25 00:10:17 +00:00			`edition = "2021"`
Fix parallelism bug in exit processing (#1110) * Fix parallelism bug in exit processing Also: * Remove parallelism for all other operations except deposit merkle proofs * Improve exit tests * Fix broken attestation test Closes #1090 * Allow for generating block/pre/post states from some unit tests (#1123) * Add post-state checks, comments * Add state_transition_vectors crate * Integrate new testing crate with CI * Add readme * Add additional valid tests * Remove ExitTests (they were moved to new crate) * Small test fixes * Delete incorrect saturating_sub in slash_validator And clean-up the balance increase/decrease functions to look more like the spec. Co-authored-by: Paul Hauner <paul@paulhauner.com> 2020-05-08 23:37:21 +00:00
			`# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html`

			`[dependencies]`
Directory Restructure (#1163) * Move tests -> testing * Directory restructure * Update Cargo.toml during restructure * Update Makefile during restructure * Fix arbitrary path 2020-05-18 11:24:23 +00:00			`state_processing = { path = "../../consensus/state_processing" }`
			`types = { path = "../../consensus/types" }`
Bump crate versions (#2829) 2021-11-29 03:57:54 +00:00			`eth2_ssz = "0.4.1"`
Altair consensus changes and refactors (#2279) ## Proposed Changes Implement the consensus changes necessary for the upcoming Altair hard fork. ## Additional Info This is quite a heavy refactor, with pivotal types like the `BeaconState` and `BeaconBlock` changing from structs to enums. This ripples through the whole codebase with field accesses changing to methods, e.g. `state.slot` => `state.slot()`. Co-authored-by: realbigsean <seananderson33@gmail.com> 2021-07-09 06:15:32 +00:00			`beacon_chain = { path = "../../beacon_node/beacon_chain" }`
			`lazy_static = "1.4.0"`
Use async code when interacting with EL (#3244) ## Overview This rather extensive PR achieves two primary goals: 1. Uses the finalized/justified checkpoints of fork choice (FC), rather than that of the head state. 2. Refactors fork choice, block production and block processing to `async` functions. Additionally, it achieves: - Concurrent forkchoice updates to the EL and cache pruning after a new head is selected. - Concurrent "block packing" (attestations, etc) and execution payload retrieval during block production. - Concurrent per-block-processing and execution payload verification during block processing. - The `Arc`-ification of `SignedBeaconBlock` during block processing (it's never mutated, so why not?): - I had to do this to deal with sending blocks into spawned tasks. - Previously we were cloning the beacon block at least 2 times during each block processing, these clones are either removed or turned into cheaper `Arc` clones. - We were also `Box`-ing and un-`Box`-ing beacon blocks as they moved throughout the networking crate. This is not a big deal, but it's nice to avoid shifting things between the stack and heap. - Avoids cloning all the blocks in every chain segment during sync. - It also has the potential to clean up our code where we need to pass an owned block around so we can send it back in the case of an error (I didn't do much of this, my PR is already big enough :sweat_smile:) - The `BeaconChain::HeadSafetyStatus` struct was removed. It was an old relic from prior merge specs. For motivation for this change, see https://github.com/sigp/lighthouse/pull/3244#issuecomment-1160963273 ## Changes to `canonical_head` and `fork_choice` Previously, the `BeaconChain` had two separate fields: ``` canonical_head: RwLock<Snapshot>, fork_choice: RwLock<BeaconForkChoice> ``` Now, we have grouped these values under a single struct: ``` canonical_head: CanonicalHead { cached_head: RwLock<Arc<Snapshot>>, fork_choice: RwLock<BeaconForkChoice> } ``` Apart from ergonomics, the only actual change here is wrapping the canonical head snapshot in an `Arc`. This means that we no longer need to hold the `cached_head` (`canonical_head`, in old terms) lock when we want to pull some values from it. This was done to avoid deadlock risks by preventing functions from acquiring (and holding) the `cached_head` and `fork_choice` locks simultaneously. ## Breaking Changes ### The `state` (root) field in the `finalized_checkpoint` SSE event Consider the scenario where epoch `n` is just finalized, but `start_slot(n)` is skipped. There are two state roots we might in the `finalized_checkpoint` SSE event: 1. The state root of the finalized block, which is `get_block(finalized_checkpoint.root).state_root`. 4. The state root at slot of `start_slot(n)`, which would be the state from (1), but "skipped forward" through any skip slots. Previously, Lighthouse would choose (2). However, we can see that when [Teku generates that event](https://github.com/ConsenSys/teku/blob/de2b2801c89ef5abf983d6bf37867c37fc47121f/data/beaconrestapi/src/main/java/tech/pegasys/teku/beaconrestapi/handlers/v1/events/EventSubscriptionManager.java#L171-L182) it uses [`getStateRootFromBlockRoot`](https://github.com/ConsenSys/teku/blob/de2b2801c89ef5abf983d6bf37867c37fc47121f/data/provider/src/main/java/tech/pegasys/teku/api/ChainDataProvider.java#L336-L341) which uses (1). I have switched Lighthouse from (2) to (1). I think it's a somewhat arbitrary choice between the two, where (1) is easier to compute and is consistent with Teku. ## Notes for Reviewers I've renamed `BeaconChain::fork_choice` to `BeaconChain::recompute_head`. Doing this helped ensure I broke all previous uses of fork choice and I also find it more descriptive. It describes an action and can't be confused with trying to get a reference to the `ForkChoice` struct. I've changed the ordering of SSE events when a block is received. It used to be `[block, finalized, head]` and now it's `[block, head, finalized]`. It was easier this way and I don't think we were making any promises about SSE event ordering so it's not "breaking". I've made it so fork choice will run when it's first constructed. I did this because I wanted to have a cached version of the last call to `get_head`. Ensuring `get_head` has been run at least once means that the cached values doesn't need to wrapped in an `Option`. This was fairly simple, it just involved passing a `slot` to the constructor so it knows when it's being run. When loading a fork choice from the store and a slot clock isn't handy I've just used the `slot` that was saved in the `fork_choice_store`. That seems like it would be a faithful representation of the slot when we saved it. I added the `genesis_time: u64` to the `BeaconChain`. It's small, constant and nice to have around. Since we're using FC for the fin/just checkpoints, we no longer get the `0x00..00` roots at genesis. You can see I had to remove a work-around in `ef-tests` here: b56be3bc2. I can't find any reason why this would be an issue, if anything I think it'll be better since the genesis-alias has caught us out a few times (0x00..00 isn't actually a real root). Edit: I did find a case where the `network` expected the 0x00..00 alias and patched it here: 3f26ac3e2. You'll notice a lot of changes in tests. Generally, tests should be functionally equivalent. Here are the things creating the most diff-noise in tests: - Changing tests to be `tokio::async` tests. - Adding `.await` to fork choice, block processing and block production functions. - Refactor of the `canonical_head` "API" provided by the `BeaconChain`. E.g., `chain.canonical_head.cached_head()` instead of `chain.canonical_head.read()`. - Wrapping `SignedBeaconBlock` in an `Arc`. - In the `beacon_chain/tests/block_verification`, we can't use the `lazy_static` `CHAIN_SEGMENT` variable anymore since it's generated with an async function. We just generate it in each test, not so efficient but hopefully insignificant. I had to disable `rayon` concurrent tests in the `fork_choice` tests. This is because the use of `rayon` and `block_on` was causing a panic. Co-authored-by: Mac L <mjladson@pm.me> 2022-07-03 05:36:50 +00:00			`tokio = { version = "1.14.0", features = ["rt-multi-thread"] }`