docs: cache-wrapping and virtual store (#8102)

* docs: cache-wrapping and virtual store

* Finish the docs update

* Update docs/building-modules/msg-services.md

Co-authored-by: Amaury <amaury.martiny@protonmail.com>

* Update docs/building-modules/msg-services.md

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>

Co-authored-by: Amaury <amaury.martiny@protonmail.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
This commit is contained in:
Robert Zaremba
2021-01-05 15:57:33 +00:00
committed by GitHub
co-authored by Amaury Aleksandr Bezobchuk
parent 46b697ca23
commit e481f13ff3
18 changed files with 96 additions and 96 deletions
+1 -1
View File
@@ -67,7 +67,7 @@ Here is the typical structure of a `handler` function:
Let us break it down:
- The [`Msg`](./messages-and-queries.md#messages) is the actual object being processed.
- The [`Context`](../core/context.md) contains all the necessary information needed to process the `msg`, as well as a cache-wrapped copy of the latest state. If the `msg` is succesfully processed, the modified version of the temporary state contained in the `ctx` will be written to the main state.
- The [`Context`](../core/context.md) contains all the necessary information needed to process the `msg`, as well as a branch of the latest state. If the `msg` is successfully processed, the branched version of the state contained in the `ctx` will be written to the main state (branch).
- The [`*Result`] returned to `BaseApp` contains (among other things) information on the execution of the `handler` and [events](../core/events.md).
Module `handler`s are typically implemented in a `./handler.go` file inside the module's folder. The [module manager](./module-manager.md) is used to add the module's `handler`s to the
+1 -1
View File
@@ -21,7 +21,7 @@ Let us break it down:
- The `path` is an array of `string`s that contains the type of the query, and that can also contain `query` arguments. See [`queries`](./messages-and-queries.md#queries) for more information.
- The `req` itself is primarily used to retrieve arguments if they are too large to fit in the `path`. This is done using the `Data` field of `req`.
- The [`Context`](../core/context.md) contains all the necessary information needed to process the `query`, as well as a cache-wrapped copy of the latest state. It is primarily used by the [`keeper`](./keeper.md) to access the state.
- The [`Context`](../core/context.md) contains all the necessary information needed to process the `query`, as well as a branch of the latest state. It is primarily used by the [`keeper`](./keeper.md) to access the state.
- The result `res` returned to `BaseApp`, marshalled using the application's [`codec`](../core/encoding.md).
## Implementation of a module query service
+15 -14
View File
@@ -126,15 +126,16 @@ is the canonical state of the application and the volatile states, `checkState`
are used to handle state transitions in-between the main state made during [`Commit`](#commit).
Internally, there is only a single `CommitMultiStore` which we refer to as the main or root state.
From this root state, we derive two volatile state through a mechanism called cache-wrapping. The
types can be illustrated as follows:
From this root state, we derive two volatile state through a mechanism called _store branching_ (performed by `CacheWrap` function).
The types can be illustrated as follows:
![Types](./baseapp_state_types.png)
### InitChain State Updates
During `InitChain`, the two volatile states, `checkState` and `deliverState` are set by cache-wrapping
the root `CommitMultiStore`. Any subsequent reads and writes happen on cached versions of the `CommitMultiStore`.
During `InitChain`, the two volatile states, `checkState` and `deliverState` are set by branching
the root `CommitMultiStore`. Any subsequent reads and writes happen on branched versions of the `CommitMultiStore`.
To avoid unnecessary roundtrip to the main state, all reads to the branched store are cached.
![InitChain](./baseapp_state-initchain.png)
@@ -142,8 +143,8 @@ the root `CommitMultiStore`. Any subsequent reads and writes happen on cached ve
During `CheckTx`, the `checkState`, which is based off of the last committed state from the root
store, is used for any reads and writes. Here we only execute the `AnteHandler` and verify a service router
exists for every message in the transaction. Note, when we execute the `AnteHandler`, we cache-wrap
the already cache-wrapped `checkState`. This has the side effect that if the `AnteHandler` fails,
exists for every message in the transaction. Note, when we execute the `AnteHandler`, we branch
the already branched `checkState`. This has the side effect that if the `AnteHandler` fails,
the state transitions won't be reflected in the `checkState` -- i.e. `checkState` is only updated on
success.
@@ -152,7 +153,7 @@ success.
### BeginBlock State Updates
During `BeginBlock`, the `deliverState` is set for use in subsequent `DeliverTx` ABCI messages. The
`deliverState` is based off of the last committed state from the root store and is cache-wrapped.
`deliverState` is based off of the last committed state from the root store and is branched.
Note, the `deliverState` is set to `nil` on [`Commit`](#commit).
![BeginBlock](./baseapp_state-begin_block.png)
@@ -161,7 +162,7 @@ Note, the `deliverState` is set to `nil` on [`Commit`](#commit).
The state flow for `DeliverTx` is nearly identical to `CheckTx` except state transitions occur on
the `deliverState` and messages in a transaction are executed. Similarly to `CheckTx`, state transitions
occur on a doubly cache-wrapped state -- `deliverState`. Successful message execution results in
occur on a doubly branched state -- `deliverState`. Successful message execution results in
writes being committed to `deliverState`. Note, if message execution fails, state transitions from
the AnteHandler are persisted.
@@ -283,7 +284,7 @@ Before the first transaction of a given block is processed, a [volatile state](#
`DeliverTx` performs the **exact same steps as `CheckTx`**, with a little caveat at step 3 and the addition of a fifth step:
1. The `AnteHandler` does **not** check that the transaction's `gas-prices` is sufficient. That is because the `min-gas-prices` value `gas-prices` is checked against is local to the node, and therefore what is enough for one full-node might not be for another. This means that the proposer can potentially include transactions for free, although they are not incentivised to do so, as they earn a bonus on the total fee of the block they propose.
2. For each `Msg` in the transaction, route to the appropriate module's [`Msg` service](../building-modules/msg-services.md). Additional _stateful_ checks are performed, and the cache-wrapped multistore held in `deliverState`'s `context` is updated by the module's `keeper`. If the `Msg` service returns successfully, the cache-wrapped multistore held in `context` is written to `deliverState` `CacheMultiStore`.
2. For each `Msg` in the transaction, route to the appropriate module's [`Msg` service](../building-modules/msg-services.md). Additional _stateful_ checks are performed, and the branched multistore held in `deliverState`'s `context` is updated by the module's `keeper`. If the `Msg` service returns successfully, the branched multistore held in `context` is written to `deliverState` `CacheMultiStore`.
During step 5., each read/write to the store increases the value of `GasConsumed`. You can find the default cost of each operation:
@@ -308,17 +309,17 @@ At any point, if `GasConsumed > GasWanted`, the function returns with `Code != 0
`RunTx` is called from `CheckTx`/`DeliverTx` to handle the transaction, with `runTxModeCheck` or `runTxModeDeliver` as parameter to differentiate between the two modes of execution. Note that when `RunTx` receives a transaction, it has already been decoded.
The first thing `RunTx` does upon being called is to retrieve the `context`'s `CacheMultiStore` by calling the `getContextForTx()` function with the appropriate mode (either `runTxModeCheck` or `runTxModeDeliver`). This `CacheMultiStore` is a cached version of the main store instantiated during `BeginBlock` for `DeliverTx` and during the `Commit` of the previous block for `CheckTx`. After that, two `defer func()` are called for [`gas`](../basics/gas-fees.md) management. They are executed when `runTx` returns and make sure `gas` is actually consumed, and will throw errors, if any.
The first thing `RunTx` does upon being called is to retrieve the `context`'s `CacheMultiStore` by calling the `getContextForTx()` function with the appropriate mode (either `runTxModeCheck` or `runTxModeDeliver`). This `CacheMultiStore` is a branch of the main store, with cache functionality (for query requests), instantiated during `BeginBlock` for `DeliverTx` and during the `Commit` of the previous block for `CheckTx`. After that, two `defer func()` are called for [`gas`](../basics/gas-fees.md) management. They are executed when `runTx` returns and make sure `gas` is actually consumed, and will throw errors, if any.
After that, `RunTx()` calls `ValidateBasic()` on each `Msg`in the `Tx`, which runs preliminary _stateless_ validity checks. If any `Msg` fails to pass `ValidateBasic()`, `RunTx()` returns with an error.
Then, the [`anteHandler`](#antehandler) of the application is run (if it exists). In preparation of this step, both the `checkState`/`deliverState`'s `context` and `context`'s `CacheMultiStore` are cached-wrapped using the `cacheTxContext()` function.
Then, the [`anteHandler`](#antehandler) of the application is run (if it exists). In preparation of this step, both the `checkState`/`deliverState`'s `context` and `context`'s `CacheMultiStore` are branched using the `cacheTxContext()` function.
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc3/baseapp/baseapp.go#L623-L630
This allows `RunTx` not to commit the changes made to the state during the execution of `anteHandler` if it ends up failing. It also prevents the module implementing the `anteHandler` from writing to state, which is an important part of the [object-capabilities](./ocap.md) of the Cosmos SDK.
Finally, the [`RunMsgs()`](#runmsgs) function is called to process the `Msg`s in the `Tx`. In preparation of this step, just like with the `anteHandler`, both the `checkState`/`deliverState`'s `context` and `context`'s `CacheMultiStore` are cached-wrapped using the `cacheTxContext()` function.
Finally, the [`RunMsgs()`](#runmsgs) function is called to process the `Msg`s in the `Tx`. In preparation of this step, just like with the `anteHandler`, both the `checkState`/`deliverState`'s `context` and `context`'s `CacheMultiStore` are branched using the `cacheTxContext()` function.
### AnteHandler
@@ -373,7 +374,7 @@ The [`EndBlock` ABCI message](#https://tendermint.com/docs/app-dev/abci-spec.htm
The [`Commit` ABCI message](https://tendermint.com/docs/app-dev/abci-spec.html#commit) is sent from the underlying Tendermint engine after the full-node has received _precommits_ from 2/3+ of validators (weighted by voting power). On the `BaseApp` end, the `Commit(res abci.ResponseCommit)` function is implemented to commit all the valid state transitions that occured during `BeginBlock`, `DeliverTx` and `EndBlock` and to reset state for the next block.
To commit state-transitions, the `Commit` function calls the `Write()` function on `deliverState.ms`, where `deliverState.ms` is a cached multistore of the main store `app.cms`. Then, the `Commit` function sets `checkState` to the latest header (obtbained from `deliverState.ctx.BlockHeader`) and `deliverState` to `nil`.
To commit state-transitions, the `Commit` function calls the `Write()` function on `deliverState.ms`, where `deliverState.ms` is a branched multistore of the main store `app.cms`. Then, the `Commit` function sets `checkState` to the latest header (obtbained from `deliverState.ctx.BlockHeader`) and `deliverState` to `nil`.
Finally, `Commit` returns the hash of the commitment of `app.cms` back to the underlying consensus engine. This hash is used as a reference in the header of the next block.
@@ -390,7 +391,7 @@ Each Tendermint `query` comes with a `path`, which is a `string` which denotes w
- Application-related queries like querying the application's version, which are served via the `handleQueryApp` method.
- Direct queries to the multistore, which are served by the `handlerQueryStore` method. These direct queryeis are different from custom queries which go through `app.queryRouter`, and are mainly used by third-party service provider like block explorers.
- P2P queries, which are served via the `handleQueryP2P` method. These queries return either `app.addrPeerFilter` or `app.ipPeerFilter` that contain the list of peers filtered by address or IP respectively. These lists are first initialized via `options` in `BaseApp`'s [constructor](#constructor).
- Custom queries, which encompass legacy queries (before the introduction of gRPC queries), are served via the `handleQueryCustom` method. The `handleQueryCustom` cache-wraps the multistore before using the `queryRoute` obtained from `app.queryRouter` to map the query to the appropriate module's [legacy `querier`](../building-modules/query-services.md#legacy-queriers).
- Custom queries, which encompass legacy queries (before the introduction of gRPC queries), are served via the `handleQueryCustom` method. The `handleQueryCustom` branches the multistore before using the `queryRoute` obtained from `app.queryRouter` to map the query to the appropriate module's [legacy `querier`](../building-modules/query-services.md#legacy-queriers).
## Next {hide}
+13 -11
View File
@@ -4,7 +4,7 @@ order: 3
# Context
The `context` is a data structure intended to be passed from function to function that carries information about the current state of the application. It holds a cached copy of the entire state as well as useful objects and information like `gasMeter`, `block height`, `consensus parameters` and more. {synopsis}
The `context` is a data structure intended to be passed from function to function that carries information about the current state of the application. It provides an access to a branched storage (a safe branch of the entire state) as well as useful objects and information like `gasMeter`, `block height`, `consensus parameters` and more. {synopsis}
## Pre-requisites Readings
@@ -54,21 +54,23 @@ childCtx = parentCtx.WithBlockHeader(header)
The [Golang Context Package](https://golang.org/pkg/context) documentation instructs developers to
explicitly pass a context `ctx` as the first argument of a process.
## Cache Wrapping
## Store branching
The `Context` contains a `MultiStore`, which allows for cache-wrapping functionality: a `CacheMultiStore`
where each `KVStore` is is wrapped with an ephemeral cache. Processes are free to write changes to
the `CacheMultiStore`, then write the changes back to the original state or disregard them if something
The `Context` contains a `MultiStore`, which allows for branchinig and caching functionality using `CacheMultiStore`
(queries in `CacheMultiStore` are cached to avoid future round trips).
Each `KVStore` is branched in a safe and isolated ephemeral storage. Processes are free to write changes to
the `CacheMultiStore`. If a state-transition sequence is performed without issue, the store branch can
be committed to the underlying store at the end of the sequence or disregard them if something
goes wrong. The pattern of usage for a Context is as follows:
1. A process receives a Context `ctx` from its parent process, which provides information needed to
perform the process.
2. The `ctx.ms` is **cache wrapped**, i.e. a cached copy of the [multistore](./store.md#multistore) is made so that the process can make changes to the state as it executes, without changing the original`ctx.ms`. This is useful to protect the underlying multistore in case the changes need to be reverted at some point in the execution.
2. The `ctx.ms` is a **branched store**, i.e. a branch of the [multistore](./store.md#multistore) is made so that the process can make changes to the state as it executes, without changing the original`ctx.ms`. This is useful to protect the underlying multistore in case the changes need to be reverted at some point in the execution.
3. The process may read and write from `ctx` as it is executing. It may call a subprocess and pass
`ctx` to it as needed.
4. When a subprocess returns, it checks if the result is a success or failure. If a failure, nothing
needs to be done - the cache wrapped `ctx` is simply discarded. If successful, the changes made to
the cache-wrapped `MultiStore` can be committed to the original `ctx.ms` via `Write()`.
needs to be done - the branch `ctx` is simply discarded. If successful, the changes made to
the `CacheMultiStore` can be committed to the original `ctx.ms` via `Write()`.
For example, here is a snippet from the [`runTx`](./baseapp.md#runtx-and-runmsgs) function in
[`baseapp`](./baseapp.md):
@@ -90,12 +92,12 @@ if result.IsOK() {
Here is the process:
1. Prior to calling `runMsgs` on the message(s) in the transaction, it uses `app.cacheTxContext()`
to cache-wrap the context and multistore.
2. The cache-wrapped context, `runMsgCtx`, is used in `runMsgs` to return a result.
to branch and cache the context and multistore.
2. `runMsgCtx` - the context with branched store, is used in `runMsgs` to return a result.
3. If the process is running in [`checkTxMode`](./baseapp.md#checktx), there is no need to write the
changes - the result is returned immediately.
4. If the process is running in [`deliverTxMode`](./baseapp.md#delivertx) and the result indicates
a successful run over all the messages, the cached multistore is written back to the original.
a successful run over all the messages, the branched multistore is written back to the original.
## Next {hide}
+10 -10
View File
@@ -56,15 +56,15 @@ The Cosmos SDK comes with a large set of stores to persist the state of applicat
### Store Interface
At its very core, a Cosmos SDK `store` is an object that holds a `CacheWrapper` and implements a `GetStoreType()` method:
At its very core, a Cosmos SDK `store` is an object that holds a `CacheWrapper` and has a `GetStoreType()` method:
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc3/store/types/store.go#L15-L18
The `GetStoreType` is a simple method that returns the type of store, whereas a `CacheWrapper` is a simple interface that specifies cache-wrapping and `Write` methods:
The `GetStoreType` is a simple method that returns the type of store, whereas a `CacheWrapper` is a simple interface that implements store read caching and write branching through `Write` method:
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc3/store/types/store.go#L240-L264
Cache-wrapping is used ubiquitously in the Cosmos SDK and required to be implemented on every store type. A cache-wrapper creates a light snapshot of a store that can be passed around and updated without affecting the main underlying store. This is used to trigger temporary state-transitions that may be reverted later should an error occur. If a state-transition sequence is performed without issue, the cached store can be committed to the underlying store at the end of the sequence.
Branching and cache is used ubiquitously in the Cosmos SDK and required to be implemented on every store type. A storage branch creates an isolated, ephemeral branch of a store that can be passed around and updated without affecting the main underlying store. This is used to trigger temporary state-transitions that may be reverted later should an error occur. Read more about it in [context](./context.md#Store-branching)
### Commit Store
@@ -88,7 +88,7 @@ Each Cosmos SDK application holds a multistore at its root to persist its state.
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc3/store/types/store.go#L104-L133
If tracing is enabled, then cache-wrapping the multistore will wrap all the underlying `KVStore` in [`TraceKv.Store`](#tracekv-store) before caching them.
If tracing is enabled, then branching the multistore will firstly wrap all the underlying `KVStore` in [`TraceKv.Store`](#tracekv-store).
### CommitMultiStore
@@ -104,11 +104,11 @@ The `rootMulti.Store` is a base-layer multistore built around a `db` on top of w
### CacheMultiStore
Whenever the `rootMulti.Store` needs to be cached-wrapped, a [`cachemulti.Store`](https://github.com/cosmos/cosmos-sdk/blob/master/store/cachemulti/store.go) is used.
Whenever the `rootMulti.Store` needs to be branched, a [`cachemulti.Store`](https://github.com/cosmos/cosmos-sdk/blob/master/store/cachemulti/store.go) is used.
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc3/store/cachemulti/store.go#L17-L28
`cachemulti.Store` cache wraps all substores in its constructor and hold them in `Store.stores`. `Store.GetKVStore()` returns the store from `Store.stores`, and `Store.Write()` recursively calls `CacheWrap.Write()` on all the substores.
`cachemulti.Store` branches all substores (creates a virtual store for each substore) in its constructor and hold them in `Store.stores`. Moreover cachese all read queries. `Store.GetKVStore()` returns the store from `Store.stores`, and `Store.Write()` recursively calls `CacheWrap.Write()` on all the substores.
## Base-layer KVStores
@@ -172,19 +172,19 @@ Transient stores are typically accessed via the [`context`](./context.md) via th
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc3/store/cachekv/store.go#L27-L34
This is the type used whenever an IAVL Store needs to be cache-wrapped (typically when setting value that might be reverted later).
This is the type used whenever an IAVL Store needs to be branched to create an isolated store (typically when we need to mutate a state that might be reverted later).
#### `Get`
`Store.Get()` checks `Store.cache` first in order to find if there is any cached value associated with the key. If the value exists, the function returns it. If not, the function calls `Store.parent.Get()`, sets the key-value pair to the `Store.cache`, and returns it.
`Store.Get()` firstly checks if `Store.cache` has an associated value with the key. If the value exists, the function returns it. If not, the function calls `Store.parent.Get()`, caches the result in `Store.cache`, and returns it.
#### `Set`
`Store.Set()` sets the key-value pair to the `Store.cache`. `cValue` has the field dirty bool which indicates whether the cached value is different from the underlying value. When `Store.Set()` cache new pair, the `cValue.dirty` is set `true` so when `Store.Write()` is called it can be written to the underlying store.
`Store.Set()` sets the key-value pair to the `Store.cache`. `cValue` has the field dirty bool which indicates whether the cached value is different from the underlying value. When `Store.Set()` cachees a new pair, the `cValue.dirty` is set `true` so when `Store.Write()` is called it can be written to the underlying store.
#### `Iterator`
`Store.Iterator()` have to traverse on both caches items and the original items. In `Store.iterator()`, two iterators are generated for each of them, and merged. `memIterator` is essentially a slice of the `KVPairs`, used for cached items. `mergeIterator` is a combination of two iterators, where traverse happens ordered on both iterators.
`Store.Iterator()` have to traverse on both cached items and the original items. In `Store.iterator()`, two iterators are generated for each of them, and merged. `memIterator` is essentially a slice of the `KVPairs`, used for cached items. `mergeIterator` is a combination of two iterators, where traverse happens ordered on both iterators.
### `GasKv` Store