laconicd/docs/architecture/adr-001-state.md

ADR 001: State

Changelog

• 2021-06-14: updates after implementation
• 2021-05-15: first draft

Status

PROPOSED, Implemented

Abstract

The current ADR proposes a state machine breaking change to the EVM module state operations (Keeper, StateDB and StateTransition) with the goal of reducing code maintenance, increase performance, and document all the transaction and state cycles and flows.

Context

This ADR addresses the issues of 3 different components of the EVM state: the StateDB interface, the live stateObject accounts, and the StateTransition functionality. These issues are outlined below in the section for each corresponding component:

StateDB

In order to execute state transitions, the EVM receives a reference to a database interface to perform CRUD operations on accounts, balances, code and state storage, among other state queries. This database interface is defined by go-ethereum's vm.StateDB, which is currently implemented using the CommitStateDB concrete type.

The CommitStateDB performs state updates by having a direct access to the sdk.Context, the evm's sdk.StoreKey and external Keepers for account and balances. Currently, the context field needs to be set on every block or state transition using WithContext(ctx) in order to pass the updated block and transaction data to the CommitStateDB.

However, traditionally in Cosmos SDK-based chains, the Keeper type has been the de-facto abstraction that manages access the key-value store (KVStore) owned by the module through the store key. Keepers usually hold a reference to external module Keepers to perform functionality outside of the scope of their module.

In the existing architecture of the EVM module, both CommitStateDB and Keeper have access to state.

State Objects

The CommitStateDB also holds references of stateObjects, defined as "live ethereum consensus accounts (i.e any balance, account nonces or storage) which will get modified while processing a state transition".

Upon a state transition, these objects will be modified and marked as 'dirty' (a.k.a stateless update) on the CommitStateDB. Then, at every EndBlock, the state of these modified objects will be 'finalized' and committed to the store, resetting all the dirty list of objects.

The core issue arises when a chain that uses the EVM module can have also have their account and balances updated through operations from other modules. This means that an EVM state object can be modified through an EVM transaction (evm.MsgEthereumTx) and other transactions like bank.MsgSend or ibctransfer.MsgTransfer. This can lead to unexpected behaviors like state overwrites, due to the current behavior that caches the dirty state on the EVM instead of committing any changes directly.

State Transition

A general EVM state transition is performed by calling the ethereum vm.EVM Create or Call functions, depending on whether the transaction creates a contract or performs a transfer or call to a given contract.

In the case of the x/evm module, it currently uses a modified version of Geth's TransitionDB, that wraps these two vm.EVM methods. The reason for using this modified function, is due to several reasons:

1. The use of sdk.Msg (MsgEthereumTx) instead of the ethereum core.Message type for the vm.EVM functions, preventing the direct use of the core.ApplyMessage.
2. The use of custom gas accounting through the transaction GasMeter available on the sdk.Context to consume the same amount of gas as on Ethereum.
3. Simulate logic via ABCI CheckTx, that prevents the state from being finalized.

Decision

StateDB

The CommitStateDB type will be removed in favor turning the module's Keeper into a StateDB concrete implementation.

// Keeper now fully implements the StateDB interface
var _ vm.StateDB = (*Keeper)(nil)

// Keeper defines the EVM module state keeper for CRUD operations. 
// It also implements the go-ethereum vm.StateDB interface. Instead of using
// a trie and database for querying and persistence, the Keeper uses KVStores
// and external Keepers to facilitate state transitions for accounts and balance
// accounting.
type Keeper struct {
  // store key and encoding codec
  // external module keepers (account, bank, etc) and params subspace
  // cache fields and sdk.Context (reset every block)
  // other CommitStateDB fields (journal, accessList, etc)
}

This means that a Keeper pointer will now directly be passed to the vm.EVM for accessing the state and performing state transitions.

The ABCI BeginBlock and EndBlock are have now been refactored to only keep track of internal fields (hashes, block bloom, etc).

func (k *Keeper) BeginBlock(ctx sdk.Context, req abci.RequestBeginBlock) {
  // ...

  // update context
  k.WithContext(ctx)
  //...
}

func (k Keeper) EndBlock(ctx sdk.Context, req abci.RequestEndBlock) []abci.ValidatorUpdate {
  // NOTE: UpdateAccounts, Commit and Reset execution steps have been removed in favor of directly
  // updating the state.

  // Gas costs are handled within msg handler so costs should be ignored
  infCtx := ctx.WithGasMeter(sdk.NewInfiniteGasMeter())
  k.WithContext(ctx)

  // get the block bloom bytes from the transient store and set it to the persistent storage
  bloomBig, found := k.GetBlockBloomTransient()
  if !found {
    bloomBig = big.NewInt(0)
  }

  bloom := ethtypes.BytesToBloom(bloomBig.Bytes())
  k.SetBlockBloom(infCtx, req.Height, bloom)
  k.WithContext(ctx)

  return []abci.ValidatorUpdate{}
}

The new StateDB (Keeper) will adopt the use of the TransientStore that discards the existing values of the store when the block is commited.

The fields that have been modified to use the TransientStore are:

• Block bloom filter (cleared at the end of every block)
• Tx index (updated on every transaction)
• Gas amount refunded (updated on every transaction)
• Suicided account (cleared at the end of every block)
• AccessList address and slot (cleared at the end of every block)

State Objects

The stateObject type will be completely removed in favor of updating the store directly through the use of the auth AccountKeeper and the bank Keeper. For the storage State and Code, the evm module Keeper will store these values directly on the KVStore using the EVM module store key and corresponding prefix keys.

State Transition

The state transition logic will be refactored to use the ApplyTransaction function from the core package of go-ethereum as reference. This method calls creates a go-ethereum StateTransition instance and, as it name implies, applies a Ethereum message to execute it and update the state. This ApplyMessage call will be wrapped in the Keeper's ApplyTransaction function, which will generate the required arguments for this call (EVM, core.Message, chain config, and gas pool), thus performing the same gas accounting as before.

func (k *Keeper) ApplyTransaction(tx *ethtypes.Transaction) (*types.MsgEthereumTxResponse, error) {
 // ...
  cfg, found := k.GetChainConfig(infCtx)
  if !found {
    // return error
  }

  ethCfg := cfg.EthereumConfig(chainID)

  signer := MakeSigner(ethCfg, height)

  msg, err := tx.AsMessage(signer)
  if err != nil {
   // return error
  }

  evm := k.NewEVM(msg, ethCfg)

  k.IncreaseTxIndexTransient()

  // create an ethereum StateTransition instance and run TransitionDb
  res, err := k.ApplyMessage(evm, msg, ethCfg)
  if err != nil {
    // return error
  }

  // ...

  return res, nil
}

ApplyMessage computes the new state by applying the given message against the existing state. If the message fails, the VM execution error with the reason will be returned to the client and the transaction won't be committed to the store.

func (k *Keeper) ApplyMessage(evm *vm.EVM, msg core.Message, cfg *params.ChainConfig) (*types.MsgEthereumTxResponse, error) {
  var (
    ret   []byte // return bytes from evm execution
    vmErr error  // vm errors do not effect consensus and are therefore not assigned to err
  )

  sender := vm.AccountRef(msg.From())
  contractCreation := msg.To() == nil

  // transaction gas meter (tracks limit and usage)
  gasConsumed := k.ctx.GasMeter().GasConsumed()
  leftoverGas := k.ctx.GasMeter().Limit() - k.ctx.GasMeter().GasConsumedToLimit()

  // NOTE: Since CRUD operations on the SDK store consume gas we need to set up an infinite gas meter so that we only consume
  // the gas used by the Ethereum message execution.
  // Not setting the infinite gas meter here would mean that we are incurring in additional gas costs
  k.WithContext(k.ctx.WithGasMeter(sdk.NewInfiniteGasMeter()))

  // NOTE: gas limit is the GasLimit defined in the message minus the Intrinsic Gas that has already been
  // consumed on the AnteHandler.

  // ensure gas is consistent during CheckTx
  if k.ctx.IsCheckTx() {
    // check gas consumption correctness
  }

  if contractCreation {
    ret, _, leftoverGas, vmErr = evm.Create(sender, msg.Data(), leftoverGas, msg.Value())
  } else {
    ret, leftoverGas, vmErr = evm.Call(sender, *msg.To(), msg.Data(), leftoverGas, msg.Value())
  }

  // refund gas prior to handling the vm error in order to set the updated gas meter
  if err := k.RefundGas(msg, leftoverGas); err != nil {
    // return error
  }

  if vmErr != nil {
    if errors.Is(vmErr, vm.ErrExecutionReverted) {
      // return error with revert reason
    }

    // return execution error
  }

  return &types.MsgEthereumTxResponse{
    Ret:      ret,
    Reverted: false,
  }, nil
}

The EVM is created as follows:

func (k *Keeper) NewEVM(msg core.Message, config *params.ChainConfig) *vm.EVM {
  blockCtx := vm.BlockContext{
    CanTransfer: core.CanTransfer,
    Transfer:    core.Transfer,
    GetHash:     k.GetHashFn(),
    Coinbase:    common.Address{}, // there's no beneficiary since we're not mining
    GasLimit:    blockGasMeter.Limit(),
    BlockNumber: blockHeight,
    Time:        blockTime,
    Difficulty:  0, // unused. Only required in PoW context
  }

  txCtx := core.NewEVMTxContext(msg)
  vmConfig := k.VMConfig()

  return vm.NewEVM(blockCtx, txCtx, k, config, vmConfig)
}

Consequences

Backwards Compatibility

The proposed ADR is a breaking state machine change and will not have any backwards compatibility since no chain that uses this code is in a production ready-state (at the moment of writing).

Positive

• Improve maintenance by simplifying the state transition logic
• Defines a single option for accessing the store through the Keeper, thus removing the CommitStateDB type.
• State operations and tests are now all located in the evm/keeper/ package
• Removes the concept of stateObject by committing to the store directly
• Delete operations on EndBlock for updating and committing dirty state objects.
• Split the state transition functionality to modularize components that can be beneficial for further customization (eg: using an alternative EVM)

Negative

• Increases the dependency of external packages (eg: go-ethereum)

Neutral

• Some of the fields from the CommitStateDB will have to be added to the Keeper
• Some state changes will have to be kept in store (eg: suicide state)

Further Discussions

Test Cases [optional]

References