lighthouse/consensus/types/src
Paul Hauner fe52322088 Implement SSZ union type (#2579)
## Issue Addressed

NA

## Proposed Changes

Implements the "union" type from the SSZ spec for `ssz`, `ssz_derive`, `tree_hash` and `tree_hash_derive` so it may be derived for `enums`:

https://github.com/ethereum/consensus-specs/blob/v1.1.0-beta.3/ssz/simple-serialize.md#union

The union type is required for the merge, since the `Transaction` type is defined as a single-variant union `Union[OpaqueTransaction]`.

### Crate Updates

This PR will (hopefully) cause CI to publish new versions for the following crates:

- `eth2_ssz_derive`: `0.2.1` -> `0.3.0`
- `eth2_ssz`: `0.3.0` -> `0.4.0`
- `eth2_ssz_types`: `0.2.0` -> `0.2.1`
- `tree_hash`: `0.3.0` -> `0.4.0`
- `tree_hash_derive`: `0.3.0` -> `0.4.0`

These these crates depend on each other, I've had to add a workspace-level `[patch]` for these crates. A follow-up PR will need to remove this patch, ones the new versions are published.

### Union Behaviors

We already had SSZ `Encode` and `TreeHash` derive for enums, however it just did a "transparent" pass-through of the inner value. Since the "union" decoding from the spec is in conflict with the transparent method, I've required that all `enum` have exactly one of the following enum-level attributes:

#### SSZ

-  `#[ssz(enum_behaviour = "union")]`
    - matches the spec used for the merge
-  `#[ssz(enum_behaviour = "transparent")]`
    - maintains existing functionality
    - not supported for `Decode` (never was)
    
#### TreeHash

-  `#[tree_hash(enum_behaviour = "union")]`
    - matches the spec used for the merge
-  `#[tree_hash(enum_behaviour = "transparent")]`
    - maintains existing functionality

This means that we can maintain the existing transparent behaviour, but all existing users will get a compile-time error until they explicitly opt-in to being transparent.

### Legacy Option Encoding

Before this PR, we already had a union-esque encoding for `Option<T>`. However, this was with the *old* SSZ spec where the union selector was 4 bytes. During merge specification, the spec was changed to use 1 byte for the selector.

Whilst the 4-byte `Option` encoding was never used in the spec, we used it in our database. Writing a migrate script for all occurrences of `Option` in the database would be painful, especially since it's used in the `CommitteeCache`. To avoid the migrate script, I added a serde-esque `#[ssz(with = "module")]` field-level attribute to `ssz_derive` so that we can opt into the 4-byte encoding on a field-by-field basis.

The `ssz::legacy::four_byte_impl!` macro allows a one-liner to define the module required for the `#[ssz(with = "module")]` for some `Option<T> where T: Encode + Decode`.

Notably, **I have removed `Encode` and `Decode` impls for `Option`**. I've done this to force a break on downstream users. Like I mentioned, `Option` isn't used in the spec so I don't think it'll be *that* annoying. I think it's nicer than quietly having two different union implementations or quietly breaking the existing `Option` impl.

### Crate Publish Ordering

I've modified the order in which CI publishes crates to ensure that we don't publish a crate without ensuring we already published a crate that it depends upon.

## TODO

- [ ] Queue a follow-up `[patch]`-removing PR.
2021-09-25 05:58:36 +00:00
..
beacon_state Implement SSZ union type (#2579) 2021-09-25 05:58:36 +00:00
test_utils [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
aggregate_and_proof.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
attestation_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
attestation_duty.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
attestation.rs Web3Signer support for VC (#2522) 2021-09-16 03:26:33 +00:00
attester_slashing.rs Update for clippy 1.50 (#2193) 2021-02-15 00:09:12 +00:00
beacon_block_body.rs Metrics for sync aggregate fullness (#2439) 2021-07-13 02:22:55 +00:00
beacon_block_header.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
beacon_block.rs Implement SSZ union type (#2579) 2021-09-25 05:58:36 +00:00
beacon_committee.rs Directory Restructure (#1163) 2020-05-18 21:24:23 +10:00
beacon_state.rs Implement SSZ union type (#2579) 2021-09-25 05:58:36 +00:00
chain_spec.rs Update spec tests to v1.1.0-beta.4 (#2548) 2021-09-25 05:58:35 +00:00
checkpoint.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
config_and_preset.rs Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
consts.rs Update to spec v1.1.0-beta.1 (#2460) 2021-07-27 05:43:35 +00:00
contribution_and_proof.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
deposit_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
deposit_message.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
deposit.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
enr_fork_id.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
eth1_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
eth_spec.rs Altair networking (#2300) 2021-08-04 01:44:57 +00:00
fork_context.rs Subscribe to altair gossip topics 2 slots before fork (#2532) 2021-09-17 01:11:16 +00:00
fork_data.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
fork_name.rs Case insensitive match for ForkName (#2587) 2021-09-11 06:07:20 +00:00
fork.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
free_attestation.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
graffiti.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
historical_batch.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
indexed_attestation.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
lib.rs Altair validator client and HTTP API (#2404) 2021-08-06 00:47:31 +00:00
participation_flags.rs Tree hash caching and optimisations for Altair (#2459) 2021-07-23 00:23:53 +00:00
participation_list.rs Rust 1.54.0 lints (#2483) 2021-07-30 01:11:47 +00:00
pending_attestation.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
preset.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
proposer_slashing.rs Implement slasher (#1567) 2020-11-23 03:43:22 +00:00
relative_epoch.rs Remove saturating arith from state_processing (#1644) 2020-09-25 05:18:21 +00:00
selection_proof.rs Altair consensus changes and refactors (#2279) 2021-07-09 06:15:32 +00:00
shuffling_id.rs Optimize validator duties (#2243) 2021-03-17 05:09:57 +00:00
signed_aggregate_and_proof.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
signed_beacon_block_header.rs Update for clippy 1.50 (#2193) 2021-02-15 00:09:12 +00:00
signed_beacon_block.rs Implement SSZ union type (#2579) 2021-09-25 05:58:36 +00:00
signed_contribution_and_proof.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
signed_voluntary_exit.rs Bump all spec tags to v0.12.1 (#1275) 2020-06-19 11:18:27 +10:00
signing_data.rs Consensus updates for v0.12 (#1228) 2020-06-03 14:56:54 +10:00
slot_data.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
slot_epoch_macros.rs Updated to comply with new clippy formatting rules (#2336) 2021-05-10 00:53:09 +00:00
slot_epoch.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sqlite.rs Directory Restructure (#1163) 2020-05-18 21:24:23 +10:00
subnet_id.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_aggregate.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
sync_aggregator_selection_data.rs Web3Signer support for VC (#2522) 2021-09-16 03:26:33 +00:00
sync_committee_contribution.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_committee_message.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_committee_subscription.rs Add quoted u64/u64_vec to SyncCommitteeSubscription (#2589) 2021-09-11 23:56:16 +00:00
sync_committee.rs Validator monitor support for sync committees (#2476) 2021-08-31 23:31:36 +00:00
sync_duty.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
sync_selection_proof.rs [Altair] Sync committee pools (#2321) 2021-07-15 00:52:02 +00:00
sync_subnet_id.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
tree_hash_impls.rs Rust 1.54.0 lints (#2483) 2021-07-30 01:11:47 +00:00
validator_subscription.rs Implement standard eth2.0 API (#1569) 2020-10-01 11:12:36 +10:00
validator.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00
voluntary_exit.rs Updates to make crates publishable (#2472) 2021-09-03 01:10:25 +00:00