2022-05-23 03:52:32 +00:00
|
|
|
# Database Migrations
|
|
|
|
|
|
|
|
|
|
Lighthouse uses a versioned database schema to allow its database design to evolve over time.
|
|
|
|
|
|
|
|
|
|
Since beacon chain genesis in December 2020 there have been several database upgrades that have
|
|
|
|
|
been applied automatically and in a _backwards compatible_ way.
|
|
|
|
|
|
|
|
|
|
However, backwards compatibility does not imply the ability to _downgrade_ to a prior version of
|
|
|
|
|
Lighthouse after upgrading. To facilitate smooth downgrades, Lighthouse v2.3.0 and above includes a
|
|
|
|
|
command for applying database downgrades.
|
|
|
|
|
|
|
|
|
|
**Everything on this page applies to the Lighthouse _beacon node_, not to the
|
|
|
|
|
validator client or the slasher**.
|
|
|
|
|
|
2022-08-02 00:58:23 +00:00
|
|
|
## List of schema versions
|
|
|
|
|
|
|
|
|
|
| Lighthouse version | Release date | Schema version | Downgrade available? |
|
|
|
|
|
|--------------------|--------------|----------------|----------------------|
|
|
|
|
|
| v2.0.0 | Oct 2021 | v5 | no |
|
|
|
|
|
| v2.1.0 | Jan 2022 | v8 | no |
|
|
|
|
|
| v2.2.0 | Apr 2022 | v8 | no |
|
2022-12-03 20:05:25 +00:00
|
|
|
| v2.3.0 | May 2022 | v9 | yes from <= v3.3.0 |
|
|
|
|
|
| v2.4.0 | Jul 2022 | v9 | yes from <= v3.3.0 |
|
2022-08-02 00:58:23 +00:00
|
|
|
| v2.5.0 | Aug 2022 | v11 | yes |
|
2022-08-23 03:50:58 +00:00
|
|
|
| v3.0.0 | Aug 2022 | v11 | yes |
|
Refactor op pool for speed and correctness (#3312)
## Proposed Changes
This PR has two aims: to speed up attestation packing in the op pool, and to fix bugs in the verification of attester slashings, proposer slashings and voluntary exits. The changes are bundled into a single database schema upgrade (v12).
Attestation packing is sped up by removing several inefficiencies:
- No more recalculation of `attesting_indices` during packing.
- No (unnecessary) examination of the `ParticipationFlags`: a bitfield suffices. See `RewardCache`.
- No re-checking of attestation validity during packing: the `AttestationMap` provides attestations which are "correct by construction" (I have checked this using Hydra).
- No SSZ re-serialization for the clunky `AttestationId` type (it can be removed in a future release).
So far the speed-up seems to be roughly 2-10x, from 500ms down to 50-100ms.
Verification of attester slashings, proposer slashings and voluntary exits is fixed by:
- Tracking the `ForkVersion`s that were used to verify each message inside the `SigVerifiedOp`. This allows us to quickly re-verify that they match the head state's opinion of what the `ForkVersion` should be at the epoch(s) relevant to the message.
- Storing the `SigVerifiedOp` on disk rather than the raw operation. This allows us to continue track the fork versions after a reboot.
This is mostly contained in this commit 52bb1840ae5c4356a8fc3a51e5df23ed65ed2c7f.
## Additional Info
The schema upgrade uses the justified state to re-verify attestations and compute `attesting_indices` for them. It will drop any attestations that fail to verify, by the logic that attestations are most valuable in the few slots after they're observed, and are probably stale and useless by the time a node restarts. Exits and proposer slashings and similarly re-verified to obtain `SigVerifiedOp`s.
This PR contains a runtime killswitch `--paranoid-block-proposal` which opts out of all the optimisations in favour of closely verifying every included message. Although I'm quite sure that the optimisations are correct this flag could be useful in the event of an unforeseen emergency.
Finally, you might notice that the `RewardCache` appears quite useless in its current form because it is only updated on the hot-path immediately before proposal. My hope is that in future we can shift calls to `RewardCache::update` into the background, e.g. while performing the state advance. It is also forward-looking to `tree-states` compatibility, where iterating and indexing `state.{previous,current}_epoch_participation` is expensive and needs to be minimised.
2022-08-29 09:10:26 +00:00
|
|
|
| v3.1.0 | Sep 2022 | v12 | yes |
|
2022-11-07 06:48:32 +00:00
|
|
|
| v3.2.0 | Oct 2022 | v12 | yes |
|
2022-12-03 20:05:25 +00:00
|
|
|
| v3.3.0 | Nov 2022 | v13 | yes |
|
2023-02-20 06:50:42 +00:00
|
|
|
| v3.4.0 | Jan 2023 | v13 | yes |
|
|
|
|
|
| v3.5.0 | Feb 2023 | v15 | yes before Capella |
|
2023-03-31 05:00:50 +00:00
|
|
|
| v4.0.1 | Mar 2023 | v16 | yes before Capella |
|
2023-06-02 03:17:36 +00:00
|
|
|
| v4.2.0 | May 2023 | v17 | yes |
|
2022-08-02 00:58:23 +00:00
|
|
|
|
|
|
|
|
> **Note**: All point releases (e.g. v2.3.1) are schema-compatible with the prior minor release
|
|
|
|
|
> (e.g. v2.3.0).
|
|
|
|
|
|
2023-02-20 06:50:42 +00:00
|
|
|
> **Note**: Support for old schemas is gradually removed from newer versions of Lighthouse. We
|
|
|
|
|
usually do this after a major version has been out for a while and everyone has upgraded. In this
|
|
|
|
|
case the above table will continue to record the deprecated schema changes for reference.
|
|
|
|
|
|
2022-05-23 03:52:32 +00:00
|
|
|
## How to apply a database downgrade
|
|
|
|
|
|
|
|
|
|
To apply a downgrade you need to use the `lighthouse db migrate` command with the correct parameters.
|
|
|
|
|
|
|
|
|
|
1. Make sure you have a copy of the latest version of Lighthouse. This will be the version that
|
|
|
|
|
knows about the latest schema change, and has the ability to revert it.
|
2022-08-02 00:58:23 +00:00
|
|
|
2. Work out the schema version you would like to downgrade to by checking the table above, or the
|
|
|
|
|
Lighthouse release notes. E.g. if you want to downgrade from v2.3.0, which upgraded the version
|
|
|
|
|
from v8 to v9, then you'll want to _downgrade_ to v8 in order to run v2.2.x or earlier.
|
2022-05-23 03:52:32 +00:00
|
|
|
3. **Ensure that downgrading is feasible**. Not all schema upgrades can be reverted, and some of
|
|
|
|
|
them are time-sensitive. The release notes will state whether a downgrade is available and
|
|
|
|
|
whether any caveats apply to it.
|
|
|
|
|
4. Work out the parameters for [Running `lighthouse db` correctly][run-correctly], including your
|
|
|
|
|
Lighthouse user, your datadir and your network flag.
|
|
|
|
|
5. After stopping the beacon node, run the migrate command with the `--to` parameter set to the
|
|
|
|
|
schema version you would like to downgrade to.
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
sudo -u "$LH_USER" lighthouse db migrate --to "$VERSION" --datadir "$LH_DATADIR" --network "$NET"
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
For example if you want to downgrade to Lighthouse v2.1 or v2.2 from v2.3 and you followed Somer
|
|
|
|
|
Esat's guide, you would run:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
sudo -u lighthousebeacon lighthouse db migrate --to 8 --datadir /var/lib/lighthouse --network mainnet
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Where `lighthouse` is Lighthouse v2.3.0+. After the downgrade succeeds you can then replace your
|
|
|
|
|
global `lighthouse` binary with the older version and start your node again.
|
|
|
|
|
|
|
|
|
|
## How to apply a database upgrade
|
|
|
|
|
|
|
|
|
|
Database _upgrades_ happen automatically upon installing a new version of Lighthouse. We will
|
|
|
|
|
highlight in the release notes when a database upgrade is included, and make note of the schema
|
|
|
|
|
versions involved (e.g. v2.3.0 includes an upgrade from v8 to v9).
|
|
|
|
|
|
|
|
|
|
They can also be applied using the `--to` parameter to `lighthouse db migrate`. See the section
|
|
|
|
|
on downgrades above.
|
|
|
|
|
|
|
|
|
|
## How to check the schema version
|
|
|
|
|
|
|
|
|
|
To check the schema version of a running Lighthouse instance you can use the HTTP API:
|
|
|
|
|
|
|
|
|
|
```bash
|
2023-06-02 03:17:36 +00:00
|
|
|
curl "http://localhost:5052/lighthouse/database/info" | jq
|
2022-05-23 03:52:32 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
2023-06-02 03:17:36 +00:00
|
|
|
"schema_version": 16,
|
2022-05-23 03:52:32 +00:00
|
|
|
"config": {
|
|
|
|
|
"slots_per_restore_point": 8192,
|
2023-06-02 03:17:36 +00:00
|
|
|
"slots_per_restore_point_set_explicitly": false,
|
2022-05-23 03:52:32 +00:00
|
|
|
"block_cache_size": 5,
|
2023-05-05 00:51:57 +00:00
|
|
|
"historic_state_cache_size": 1,
|
2022-05-23 03:52:32 +00:00
|
|
|
"compact_on_init": false,
|
2023-06-02 03:17:36 +00:00
|
|
|
"compact_on_prune": true,
|
|
|
|
|
"prune_payloads": true
|
|
|
|
|
},
|
|
|
|
|
"split": {
|
|
|
|
|
"slot": "5485952",
|
|
|
|
|
"state_root": "0xcfe5d41e6ab5a9dab0de00d89d97ae55ecaeed3b08e4acda836e69b2bef698b4"
|
|
|
|
|
},
|
|
|
|
|
"anchor": {
|
|
|
|
|
"anchor_slot": "5414688",
|
|
|
|
|
"oldest_block_slot": "0",
|
|
|
|
|
"oldest_block_parent": "0x0000000000000000000000000000000000000000000000000000000000000000",
|
|
|
|
|
"state_upper_limit": "5414912",
|
|
|
|
|
"state_lower_limit": "8192"
|
2022-05-23 03:52:32 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
2023-06-02 03:17:36 +00:00
|
|
|
The `schema_version` key indicates that this database is using schema version 16.
|
2022-05-23 03:52:32 +00:00
|
|
|
|
|
|
|
|
Alternatively, you can check the schema version with the `lighthouse db` command.
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
sudo -u lighthousebeacon lighthouse db version --datadir /var/lib/lighthouse --network mainnet
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
See the section on [Running `lighthouse db` correctly][run-correctly] for details.
|
|
|
|
|
|
|
|
|
|
## How to run `lighthouse db` correctly
|
|
|
|
|
|
|
|
|
|
Several conditions need to be met in order to run `lighthouse db`:
|
|
|
|
|
|
|
|
|
|
1. The beacon node must be **stopped** (not running). If you are using systemd a command like
|
|
|
|
|
`sudo systemctl stop lighthousebeacon` will accomplish this.
|
|
|
|
|
2. The command must run as the user that owns the beacon node database. If you are using systemd then
|
|
|
|
|
your beacon node might run as a user called `lighthousebeacon`.
|
|
|
|
|
3. The `--datadir` flag must be set to the location of the Lighthouse data directory.
|
2023-06-02 03:17:36 +00:00
|
|
|
4. The `--network` flag must be set to the correct network, e.g. `mainnet`, `goerli` or `sepolia`.
|
2022-05-23 03:52:32 +00:00
|
|
|
|
|
|
|
|
The general form for a `lighthouse db` command is:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
sudo -u "$LH_USER" lighthouse db version --datadir "$LH_DATADIR" --network "$NET"
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
If you followed Somer Esat's guide for mainnet:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
sudo systemctl stop lighthousebeacon
|
|
|
|
|
```
|
|
|
|
|
```
|
|
|
|
|
sudo -u lighthousebeacon lighthouse db version --datadir /var/lib/lighthouse --network mainnet
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
If you followed the CoinCashew guide for mainnet:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
sudo systemctl stop beacon-chain
|
|
|
|
|
```
|
|
|
|
|
```
|
|
|
|
|
lighthouse db version --network mainnet
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
[run-correctly]: #how-to-run-lighthouse-db-correctly
|
