236 lines
11 KiB
Markdown
236 lines
11 KiB
Markdown
# Store
|
|
|
|
The store package defines the interfaces, types and abstractions for Cosmos SDK
|
|
modules to read and write to Merkleized state within a Cosmos SDK application.
|
|
The store package provides many primitives for developers to use in order to
|
|
work with both state storage and state commitment. Below we describe the various
|
|
abstractions.
|
|
|
|
## Types
|
|
|
|
### `Store`
|
|
|
|
The bulk of the store interfaces are defined [here](https://github.com/cosmos/cosmos-sdk/blob/main/store/store.go),
|
|
where the base primitive interface, for which other interfaces build off of, is
|
|
the `Store` type. The `Store` interface defines the ability to tell the type of
|
|
the implementing store and the ability to cache wrap via the `CacheWrapper` interface.
|
|
|
|
### `CacheWrapper` & `CacheWrap`
|
|
|
|
One of the most important features a store has the ability to perform is the
|
|
ability to cache wrap. Cache wrapping is essentially the underlying store wrapping
|
|
itself within another store type that performs caching for both reads and writes
|
|
with the ability to flush writes via `Write()`.
|
|
|
|
### `KVStore` & `CacheKVStore`
|
|
|
|
One of the most important interfaces that both developers and modules interface
|
|
with, which also provides the basis of most state storage and commitment operations,
|
|
is the `KVStore`. The `KVStore` interface provides basic CRUD abilities and
|
|
prefix-based iteration, including reverse iteration.
|
|
|
|
Typically, each module has its own dedicated `KVStore` instance, which it can
|
|
get access to via the `sdk.Context` and the use of a pointer-based named key --
|
|
`KVStoreKey`. The `KVStoreKey` provides pseudo-OCAP. How a exactly a `KVStoreKey`
|
|
maps to a `KVStore` will be illustrated below through the `CommitMultiStore`.
|
|
|
|
Note, a `KVStore` cannot directly commit state. Instead, a `KVStore` can be wrapped
|
|
by a `CacheKVStore` which extends a `KVStore` and provides the ability for the
|
|
caller to execute `Write()` which commits state to the underlying state storage.
|
|
Note, this doesn't actually flush writes to disk as writes are held in memory
|
|
until `Commit()` is called on the `CommitMultiStore`.
|
|
|
|
### `CommitMultiStore`
|
|
|
|
The `CommitMultiStore` interface exposes the top-level interface that is used
|
|
to manage state commitment and storage by an SDK application and abstracts the
|
|
concept of multiple `KVStore`s which are used by multiple modules. Specifically,
|
|
it supports the following high-level primitives:
|
|
|
|
* Allows for a caller to retrieve a `KVStore` by providing a `KVStoreKey`.
|
|
* Exposes pruning mechanisms to remove state pinned against a specific height/version
|
|
in the past.
|
|
* Allows for loading state storage at a particular height/version in the past to
|
|
provide current head and historical queries.
|
|
* Provides the ability to rollback state to a previous height/version.
|
|
* Provides the ability to load state storage at a particular height/version
|
|
while also performing store upgrades, which are used during live hard-fork
|
|
application state migrations.
|
|
* Provides the ability to commit all current accumulated state to disk and performs
|
|
Merkle commitment.
|
|
|
|
## Implementation Details
|
|
|
|
While there are many interfaces that the `store` package provides, there is
|
|
typically a core implementation for each main interface that modules and
|
|
developers interact with that are defined in the Cosmos SDK.
|
|
|
|
### `iavl.Store`
|
|
|
|
The `iavl.Store` provides the core implementation for state storage and commitment
|
|
by implementing the following interfaces:
|
|
|
|
* `KVStore`
|
|
* `CommitStore`
|
|
* `CommitKVStore`
|
|
* `Queryable`
|
|
* `StoreWithInitialVersion`
|
|
|
|
It allows for all CRUD operations to be performed along with allowing current
|
|
and historical state queries, prefix iteration, and state commitment along with
|
|
Merkle proof operations. The `iavl.Store` also provides the ability to remove
|
|
historical state from the state commitment layer.
|
|
|
|
An overview of the IAVL implementation can be found [here](https://github.com/cosmos/iavl/blob/master/docs/overview.md).
|
|
It is important to note that the IAVL store provides both state commitment and
|
|
logical storage operations, which comes with drawbacks as there are various
|
|
performance impacts, some of which are very drastic, when it comes to the
|
|
operations mentioned above.
|
|
|
|
When dealing with state management in modules and clients, the Cosmos SDK provides
|
|
various layers of abstractions or "store wrapping", where the `iavl.Store` is the
|
|
bottom most layer. When requesting a store to perform reads or writes in a module,
|
|
the typical abstraction layer in order is defined as follows:
|
|
|
|
```text
|
|
iavl.Store <- cachekv.Store <- gaskv.Store <- cachemulti.Store <- rootmulti.Store
|
|
```
|
|
|
|
### Concurrent use of IAVL store
|
|
|
|
The tree under `iavl.Store` is not safe for concurrent use. It is the
|
|
responsibility of the caller to ensure that concurrent access to the store is
|
|
not performed.
|
|
|
|
The main issue with concurrent use is when data is written at the same time as
|
|
it's being iterated over. Doing so will cause a irrecoverable fatal error because
|
|
of concurrent reads and writes to an internal map.
|
|
|
|
Although it's not recommended, you can iterate through values while writing to
|
|
it by disabling "FastNode" **without guarantees that the values being written will
|
|
be returned during the iteration** (if you need this, you might want to reconsider
|
|
the design of your application). This is done by setting `iavl-disable-fastnode`
|
|
to `true` in the config TOML file.
|
|
|
|
### `cachekv.Store`
|
|
|
|
The `cachekv.Store` store wraps an underlying `KVStore`, typically a `iavl.Store`
|
|
and contains an in-memory cache for storing pending writes to underlying `KVStore`.
|
|
`Set` and `Delete` calls are executed on the in-memory cache, whereas `Has` calls
|
|
are proxied to the underlying `KVStore`.
|
|
|
|
One of the most important calls to a `cachekv.Store` is `Write()`, which ensures
|
|
that key-value pairs are written to the underlying `KVStore` in a deterministic
|
|
and ordered manner by sorting the keys first. The store keeps track of "dirty"
|
|
keys and uses these to determine what keys to sort. In addition, it also keeps
|
|
track of deleted keys and ensures these are also removed from the underlying
|
|
`KVStore`.
|
|
|
|
The `cachekv.Store` also provides the ability to perform iteration and reverse
|
|
iteration. Iteration is performed through the `cacheMergeIterator` type and uses
|
|
both the dirty cache and underlying `KVStore` to iterate over key-value pairs.
|
|
|
|
Note, all calls to CRUD and iteration operations on a `cachekv.Store` are thread-safe.
|
|
|
|
### `gaskv.Store`
|
|
|
|
The `gaskv.Store` store provides a simple implementation of a `KVStore`.
|
|
Specifically, it just wraps an existing `KVStore`, such as a cache-wrapped
|
|
`iavl.Store`, and incurs configurable gas costs for CRUD operations via
|
|
`ConsumeGas()` calls defined on the `GasMeter` which exists in a `sdk.Context`
|
|
and then proxies the underlying CRUD call to the underlying store. Note, the
|
|
`GasMeter` is reset on each block.
|
|
|
|
### `cachemulti.Store` & `rootmulti.Store`
|
|
|
|
The `rootmulti.Store` acts as an abstraction around a series of stores. Namely,
|
|
it implements the `CommitMultiStore` an `Queryable` interfaces. Through the
|
|
`rootmulti.Store`, an SDK module can request access to a `KVStore` to perform
|
|
state CRUD operations and queries by holding access to a unique `KVStoreKey`.
|
|
|
|
The `rootmulti.Store` ensures these queries and state operations are performed
|
|
through cached-wrapped instances of `cachekv.Store` which is described above. The
|
|
`rootmulti.Store` implementation is also responsible for committing all accumulated
|
|
state from each `KVStore` to disk and returning an application state Merkle root.
|
|
|
|
Queries can be performed to return state data along with associated state
|
|
commitment proofs for both previous heights/versions and the current state root.
|
|
Queries are routed based on store name, i.e. a module, along with other parameters
|
|
which are defined in `abci.QueryRequest`.
|
|
|
|
The `rootmulti.Store` also provides primitives for pruning data at a given
|
|
height/version from state storage. When a height is committed, the `rootmulti.Store`
|
|
will determine if other previous heights should be considered for removal based
|
|
on the operator's pruning settings defined by `PruningOptions`, which defines
|
|
how many recent versions to keep on disk and the interval at which to remove
|
|
"staged" pruned heights from disk. During each interval, the staged heights are
|
|
removed from each `KVStore`. Note, it is up to the underlying `KVStore`
|
|
implementation to determine how pruning is actually performed. The `PruningOptions`
|
|
are defined as follows:
|
|
|
|
```go
|
|
type PruningOptions struct {
|
|
// KeepRecent defines how many recent heights to keep on disk.
|
|
KeepRecent uint64
|
|
|
|
// Interval defines when the pruned heights are removed from disk.
|
|
Interval uint64
|
|
|
|
// Strategy defines the kind of pruning strategy. See below for more information on each.
|
|
Strategy PruningStrategy
|
|
}
|
|
```
|
|
|
|
The Cosmos SDK defines a preset number of pruning "strategies": `default`, `everything`
|
|
`nothing`, and `custom`.
|
|
|
|
It is important to note that the `rootmulti.Store` considers each `KVStore` as a
|
|
separate logical store. In other words, they do not share a Merkle tree or
|
|
comparable data structure. This means that when state is committed via
|
|
`rootmulti.Store`, each store is committed in sequence and thus is not atomic.
|
|
|
|
In terms of store construction and wiring, each Cosmos SDK application contains
|
|
a `BaseApp` instance which internally has a reference to a `CommitMultiStore`
|
|
that is implemented by a `rootmulti.Store`. The application then registers one or
|
|
more `KVStoreKey` that pertain to a unique module and thus a `KVStore`. Through
|
|
the use of an `sdk.Context` and a `KVStoreKey`, each module can get direct access
|
|
to it's respective `KVStore` instance.
|
|
|
|
Example:
|
|
|
|
```go
|
|
func NewApp(...) Application {
|
|
// ...
|
|
|
|
bApp := baseapp.NewBaseApp(appName, logger, db, txConfig.TxDecoder(), baseAppOptions...)
|
|
bApp.SetCommitMultiStoreTracer(traceStore)
|
|
bApp.SetVersion(version.Version)
|
|
bApp.SetInterfaceRegistry(interfaceRegistry)
|
|
|
|
// ...
|
|
|
|
keys := sdk.NewKVStoreKeys(...)
|
|
transientKeys := sdk.NewTransientStoreKeys(...)
|
|
memKeys := sdk.NewMemoryStoreKeys(...)
|
|
|
|
// ...
|
|
|
|
// initialize stores
|
|
app.MountKVStores(keys)
|
|
app.MountTransientStores(transientKeys)
|
|
app.MountMemoryStores(memKeys)
|
|
|
|
// ...
|
|
}
|
|
```
|
|
|
|
The `rootmulti.Store` itself can be cache-wrapped which returns an instance of a
|
|
`cachemulti.Store`. For each block, `BaseApp` ensures that the proper abstractions
|
|
are created on the `CommitMultiStore`, i.e. ensuring that the `rootmulti.Store`
|
|
is cached-wrapped and uses the resulting `cachemulti.Store` to be set on the
|
|
`sdk.Context` which is then used for block and transaction execution. As a result,
|
|
all state mutations due to block and transaction execution are actually held
|
|
ephemerally until `Commit()` is called by the ABCI client. This concept is further
|
|
expanded upon when the AnteHandler is executed per transaction to ensure state
|
|
is not committed for transactions that failed CheckTx.
|