docs(store/v2): Add missing documents (#22118)
This commit is contained in:
cool-developer
+9
-4
@@ -1,7 +1,7 @@
|
||||
# Store
|
||||
|
||||
The `store` package contains the implementation of store/v2, which is the SDK's
|
||||
abstraction around managing historical and committed state. See [ADR-065](../docs/architecture/adr-065-store-v2.md)
|
||||
abstraction around managing historical and committed state. See [ADR-065](../../docs/architecture/adr-065-store-v2.md)
|
||||
and [Store v2 Design](https://docs.google.com/document/d/1l6uXIjTPHOOWM5N4sUUmUfCZvePoa5SNfIEtmgvgQSU/edit#heading=h.nz8dqy6wa4g1) for a high-level overview of the design and rationale.
|
||||
|
||||
## Usage
|
||||
@@ -42,21 +42,26 @@ sequenceDiagram
|
||||
end
|
||||
```
|
||||
|
||||
`Prune store keys` does not remove the data from the SC and SS instantly. It only
|
||||
`PruneStoreKeys` does not remove the data from the SC and SS instantly. It only
|
||||
marks the store keys as pruned. The actual data removal is done by the pruning
|
||||
process of the underlying SS and SC.
|
||||
|
||||
## Migration
|
||||
|
||||
<!-- TODO -->
|
||||
The migration from store/v1 to store/v2 is supported by the `MigrationManager` in
|
||||
the `migration` package. See [Migration Manager](./migration/README.md) for more details.
|
||||
|
||||
## Pruning
|
||||
|
||||
The `root.Store` is NOT responsible for pruning. Rather, pruning is the responsibility
|
||||
of the underlying SS and SC layers. This means pruning can be implementation specific,
|
||||
such as being synchronous or asynchronous.
|
||||
such as being synchronous or asynchronous. See [Pruning Manager](./pruning/README.md) for more details.
|
||||
|
||||
|
||||
## State Sync
|
||||
|
||||
The `root.Store` is NOT responsible for state sync. See [Snapshots Manager](./snapshots/README.md)
|
||||
for more details.
|
||||
|
||||
## Test Coverage
|
||||
|
||||
|
||||
@@ -28,7 +28,11 @@ See this [section](https://docs.google.com/document/d/1l6uXIjTPHOOWM5N4sUUmUfCZv
|
||||
|
||||
## Pruning
|
||||
|
||||
<!-- TODO -->
|
||||
Pruning is the process of efficiently managing and removing outdated data from the
|
||||
State Commitment (SC). To facilitate this, the SC backend must implement the `Pruner`
|
||||
interface, allowing the `PruningManager` to execute data pruning operations according
|
||||
to the specified `PruningOption`. Optionally, the SC backend can implement the
|
||||
`PausablePruner` interface to pause pruning during a commit.
|
||||
|
||||
## State Sync
|
||||
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
# Migration Manager
|
||||
|
||||
The `migration` package contains the `migration.Manager`, which is responsible
|
||||
for migrating data from `store/v1` to `store/v2`. To ensure a smooth transition,
|
||||
the process is designed to **lazily** migrate data in the background without blocking
|
||||
`root.Store` operations.
|
||||
|
||||
## Overview
|
||||
|
||||
The migration process involves several steps:
|
||||
|
||||
1. **Create a snapshot** of the current state while `Commit` operations continue to
|
||||
function with `store/v1`.
|
||||
2. **Restore the snapshot** into the new StateStorage (SS) and StateCommitment (SC).
|
||||
3. **Sync recent state changes** from `store/v1` to the new SS and SC.
|
||||
4. After syncing, the `Commit` operation will be switched to the new `store/v2`.
|
||||
|
||||
Taking a snapshot is a lightweight operation. The snapshot is not stored on disk but
|
||||
consumed by the `Restore` process, which replays state changes to the new SS and SC.
|
||||
|
||||
> **Note:** After migration, `store/v2` does **not** support historical queries.
|
||||
If historical data access is required, a full state migration to `store/v2` is necessary.
|
||||
|
||||
## Usage
|
||||
|
||||
You can create a new `migration.Manager` by calling the following function:
|
||||
|
||||
```go
|
||||
func NewManager(
|
||||
db corestore.KVStoreWithBatch,
|
||||
sm *snapshots.Manager,
|
||||
ss *storage.StorageStore,
|
||||
sc *commitment.CommitStore,
|
||||
logger log.Logger
|
||||
) *Manager
|
||||
```
|
||||
|
||||
* `sc` (Commitment Store) can be `nil`. In that case, the Manager will migrate only
|
||||
the state storage.
|
||||
* The migration process is lazy, meaning data is migrated in the background while
|
||||
`root.Store` remains fully operational.
|
||||
|
||||
To initiate the migration process, call the `Start` method:
|
||||
|
||||
```go
|
||||
func (m *Manager) Start(ctx context.Context) error
|
||||
```
|
||||
|
||||
> **Note:** It should be called by the RootStore, running in the background.
|
||||
|
||||
## Migration Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
|
||||
participant A as RootStore
|
||||
participant B as MigrationManager
|
||||
participant C as SnapshotsManager
|
||||
participant D as StateCommitment
|
||||
participant E as StateStorage
|
||||
|
||||
A->>B: Start
|
||||
loop Old Data Migration
|
||||
B->>C: Create Snapshot
|
||||
C->>B: Stream Snapshot
|
||||
B->>D: State Sync (Restore)
|
||||
B->>E: Write Changeset (Restore)
|
||||
end
|
||||
|
||||
loop New Commit Data Sync
|
||||
A->>B: Commit(Changeset)
|
||||
B->>B: Store Changeset
|
||||
B->>D: Commit Changeset
|
||||
B->>E: Write Changeset
|
||||
end
|
||||
|
||||
B->>A: Switch to new store/v2
|
||||
```
|
||||
|
||||
## Key Considerations
|
||||
|
||||
### Laziness and Background Operation
|
||||
|
||||
The migration is performed lazily, meaning it occurs in the background without
|
||||
interrupting the current operations on root.Store. This allows the chain to continue
|
||||
running while data is gradually migrated to `store/v2`. State synchronization ensures
|
||||
that any new state changes during the migration are also applied to `store/v2`.
|
||||
|
||||
However, note that there may be a performance impact depending on the size of the data
|
||||
being migrated, and it’s essential to monitor the migration process in production
|
||||
environments.
|
||||
|
||||
### Handling Failures and Rollbacks
|
||||
|
||||
It is important to consider how the migration manager handles errors or system failures
|
||||
during the migration process:
|
||||
|
||||
* If the migration fails, there is no impact on the existing `store/v1` operations,
|
||||
but need to restart the migration process from the scratch.
|
||||
* In the event of a critical failure after migration, a rollback may not be possible,
|
||||
and it is needed to keep the `store/v1` backup for a certain period.
|
||||
|
||||
### Impact on Historical Queries
|
||||
|
||||
After the migration, the new `store/v2` does not support historical queries.
|
||||
This limitation should be clearly understood before starting the migration process,
|
||||
especially if the node relies on historical data for any operations.
|
||||
|
||||
If historical queries are required, users must fully migrate all historical data to `store/v2`.
|
||||
Alternatively, keeping store/v1 accessible for historical queries could be an option.
|
||||
@@ -70,13 +70,10 @@ Iterate/backend_rocksdb_versiondb_opts-10 778ms ± 0%
|
||||
|
||||
## Pruning
|
||||
|
||||
Pruning is an implementation and responsibility of the underlying SS backend.
|
||||
Specifically, the `StorageStore` accepts `store.PruningOption` which defines the
|
||||
pruning configuration. During `ApplyChangeset`, the `StorageStore` will check if
|
||||
pruning should occur based on the current height being committed. If so, it will
|
||||
delegate a `Prune` call on the underlying SS backend, which can be defined specific
|
||||
to the implementation, e.g. asynchronous or synchronous.
|
||||
|
||||
Pruning is the process of efficiently managing and removing outdated or redundant
|
||||
data from the State Storage (SS). To facilitate this, the SS backend must implement
|
||||
the `Pruner` interface, allowing the `PruningManager` to execute data pruning operations
|
||||
according to the specified `PruningOption`.
|
||||
|
||||
## State Sync
|
||||
|
||||
|
||||
Reference in New Issue
Block a user