feat: Adding READMEs throughout the codebase (#248)
* abci readme * done wit example for abci.go * SEA * base lane done * proposals * block readme * fixes * proposals fix
This commit is contained in:
David Terpay
+103
@@ -0,0 +1,103 @@
|
||||
# Block SDK Mempool & Lanes
|
||||
|
||||
## Overview
|
||||
|
||||
> This document describes how the Block SDK mempool and lanes operate at a high level. To learn more about how to construct lanes, please visit the [build my own lane readme](../lanes/build-your-own/README.md) and/or the [base lane documentation](./base/README.md). To read about how proposals are construct, visit the [abci readme](../abci/README.md).
|
||||
|
||||
Mempools are traditionally used to temporarily store transactions before they are added to a block. The Block SDK mempool is no different. However, instead of treating each transaction the same, the Block SDK allows for developers to create `Lanes` that permit transactions to be ordered differently based on the properties of the transaction itself.
|
||||
|
||||
What was once a single monolithic data structure, is now a collection of sub-mempools that can be configured to order transactions in a way that makes sense for the application.
|
||||
|
||||
## Lanes
|
||||
|
||||
Lanes are utilized to allow developers to create custom transaction order, validation, and execution logic. Each lane is responsible for maintaining its own mempool - ordering transactions as it desires only for the transactions it wants to accept. For example, a lane may only accept transactions that are staking related, such as the free lane. The free lane may then order the transactions based on the user's on-chain stake.
|
||||
|
||||
When proposals are constructed, the transactions from a given lane are selected based on highest to lowest priority, validated according to the lane's verfication logic, and included in the proposal.
|
||||
|
||||
Each lane must implement the `Lane` interface, although it is highly recommended that developers extend the [base lane](./base/README.md) to create new lanes.
|
||||
|
||||
```go
|
||||
// LaneMempool defines the interface a lane's mempool should implement. The basic API
|
||||
// is the same as the sdk.Mempool, but it also includes a Compare function that is used
|
||||
// to determine the relative priority of two transactions belonging in the same lane.
|
||||
type LaneMempool interface {
|
||||
sdkmempool.Mempool
|
||||
|
||||
// Compare determines the relative priority of two transactions belonging in the same lane. Compare
|
||||
// will return -1 if this transaction has a lower priority than the other transaction, 0 if they have
|
||||
// the same priority, and 1 if this transaction has a higher priority than the other transaction.
|
||||
Compare(ctx sdk.Context, this, other sdk.Tx) (int, error)
|
||||
|
||||
// Contains returns true if the transaction is contained in the mempool.
|
||||
Contains(tx sdk.Tx) bool
|
||||
|
||||
// Priority returns the priority of a transaction that belongs to this lane.
|
||||
Priority(ctx sdk.Context, tx sdk.Tx) any
|
||||
}
|
||||
|
||||
// Lane defines an interface used for matching transactions to lanes, storing transactions,
|
||||
// and constructing partial blocks.
|
||||
type Lane interface {
|
||||
LaneMempool
|
||||
|
||||
// PrepareLane builds a portion of the block. It inputs the current context, proposal, and a
|
||||
// function to call the next lane in the chain. This handler should update the context as needed
|
||||
// and add transactions to the proposal. Note, the lane should only add transactions up to the
|
||||
// max block space for the lane.
|
||||
PrepareLane(
|
||||
ctx sdk.Context,
|
||||
proposal proposals.Proposal,
|
||||
next PrepareLanesHandler,
|
||||
) (proposals.Proposal, error)
|
||||
|
||||
// ProcessLane verifies this lane's portion of a proposed block. It inputs the current context,
|
||||
// proposal, transactions that belong to this lane, and a function to call the next lane in the
|
||||
// chain. This handler should update the context as needed and add transactions to the proposal.
|
||||
// The entire process lane chain should end up constructing the same proposal as the prepare lane
|
||||
// chain.
|
||||
ProcessLane(
|
||||
ctx sdk.Context,
|
||||
proposal proposals.Proposal,
|
||||
txs []sdk.Tx,
|
||||
next ProcessLanesHandler,
|
||||
) (proposals.Proposal, error)
|
||||
|
||||
// GetMaxBlockSpace returns the max block space for the lane as a relative percentage.
|
||||
GetMaxBlockSpace() math.LegacyDec
|
||||
|
||||
// SetMaxBlockSpace sets the max block space for the lane as a relative percentage.
|
||||
SetMaxBlockSpace(math.LegacyDec)
|
||||
|
||||
// Name returns the name of the lane.
|
||||
Name() string
|
||||
|
||||
// SetAnteHandler sets the lane's antehandler.
|
||||
SetAnteHandler(antehander sdk.AnteHandler)
|
||||
|
||||
// Match determines if a transaction belongs to this lane.
|
||||
Match(ctx sdk.Context, tx sdk.Tx) bool
|
||||
|
||||
// GetTxInfo returns various information about the transaction that
|
||||
// belongs to the lane including its priority, signer's, sequence number,
|
||||
// size and more.
|
||||
GetTxInfo(ctx sdk.Context, tx sdk.Tx) (utils.TxWithInfo, error)
|
||||
}
|
||||
```
|
||||
|
||||
## Lane Priorities
|
||||
|
||||
Each lane has a priority that is used to determine the order in which lanes are processed. The higher the priority, the earlier the lane is processed. For example, if we have three lanes - MEV, free, and default - proposals will be constructed in the following order:
|
||||
|
||||
1. MEV
|
||||
2. Free
|
||||
3. Default
|
||||
|
||||
Proposals will then be verified in the same order. Please see the [readme above](../abci/README.md) for more information on how proposals are constructed using lanes.
|
||||
|
||||
The ordering of lane's priorities is determined based on the order passed into the constructor of the Block SDK mempool i.e. `LanedMempool`.
|
||||
|
||||
## Block SDK mempool
|
||||
|
||||
The `LanedMempool` is a wrapper on top of the collection of lanes. It is solely responsible for adding transactions to the appropriate lanes. Transactions are always inserted / removed to the first lane that accepts / matches the transactions. **Transactions should only match to one lane.**. **In the case where a transaction can match to multiple lanes, the transaction will be inserted into the lane that has the highest priority.**
|
||||
|
||||
To read more about the underlying implementation of the Block SDK mempool, please see the implementation [here](./mempool.go).
|
||||
@@ -0,0 +1,145 @@
|
||||
# Base Lane
|
||||
|
||||
## Overview
|
||||
|
||||
> The base lane is purposefully built to be a simple lane that can be extended (inherited) by other lanes. It provides the basic functionality that is required by all lanes with the option to override any of the methods.
|
||||
|
||||
The base lane implements the lane interface and provides a few critical methods that allow application developers to create a lane that has custom transaction ordering and execution logic. The most important things you need to know in order to build a custom lane are:
|
||||
|
||||
* `MatchHandler`: This method is responsible for determining if a given transaction should be accepted by the lane.
|
||||
* `PrepareLaneHandler`: This method is responsible for reaping transactions from the mempool, validating them, re-ordering (if necessary), and returning them to be included in a block proposal.
|
||||
* `ProcessLaneHandler`: This method is responsible verifying the matched transactions that were included in a block proposal.
|
||||
* `LaneMempool`: This allows developers to have the freedom to implement their own mempools with custom transaction ordering logic.
|
||||
* `LaneConfig`: This allows developers to customize how the lane behaves in terms of max block space, max transaction count, and more.
|
||||
|
||||
## MatchHandler
|
||||
|
||||
MatchHandler is utilized to determine if a transaction should be included in the lane. This function can be a stateless or stateful check on the transaction. The function signature is as follows:
|
||||
|
||||
```go
|
||||
MatchHandler func(ctx sdk.Context, tx sdk.Tx) bool
|
||||
```
|
||||
|
||||
To create a custom lane with a custom `MatchHandler`, you must implement this function and pass it into the constructor for the base lane. For example, the [free lane](../../lanes/free/lane.go) inherits all the base lane functionality but overrides the `MatchHandler` to only accept staking related transactions.
|
||||
|
||||
```go
|
||||
// DefaultMatchHandler returns the default match handler for the free lane. The
|
||||
// default implementation matches transactions that are staking related. In particular,
|
||||
// any transaction that is a MsgDelegate, MsgBeginRedelegate, or MsgCancelUnbondingDelegation.
|
||||
func DefaultMatchHandler() base.MatchHandler {
|
||||
return func(ctx sdk.Context, tx sdk.Tx) bool {
|
||||
for _, msg := range tx.GetMsgs() {
|
||||
switch msg.(type) {
|
||||
case *types.MsgDelegate:
|
||||
return true
|
||||
case *types.MsgBeginRedelegate:
|
||||
return true
|
||||
case *types.MsgCancelUnbondingDelegation:
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The default `MatchHandler` is implemented in the [base lane](./handlers.go) and matches all transactions.
|
||||
|
||||
## PrepareLaneHandler
|
||||
|
||||
The `PrepareLaneHandler` is responsible for reaping transactions from the mempool, validating them, re-ordering (if necessary), and returning them to be included in a block proposal. If any of the transactions were invalid, it should return them alongside the transactions it wants to include in the proposal. The invalid transactions will subsequently be removed from the lane's mempool. The function signature is as follows:
|
||||
|
||||
```go
|
||||
PrepareLaneHandler func(
|
||||
ctx sdk.Context,
|
||||
proposal proposals.Proposal,
|
||||
limit proposals.LaneLimits,
|
||||
) (txsToInclude []sdk.Tx, txsToRemove []sdk.Tx, err error)
|
||||
```
|
||||
|
||||
To create a custom lane with a custom `PrepareLaneHandler`, you must implement this function and set it on the lane after it has been created. Please visit the [MEV lane's](../../lanes/mev/abci.go) `PrepareLaneHandler` for an example of how to implement this function.
|
||||
|
||||
The default `PrepareLaneHandler` is implemented in the [base lane](./handlers.go). It reaps transactions from the mempool, validates them, ensures that the lane's block space limit is not exceeded, and returns the transactions to be included in the block and the ones that need to be removed.
|
||||
|
||||
## ProcessLaneHandler
|
||||
|
||||
The `ProcessLaneHandler` is responsible for verifying the transactions that belong to a given lane that were included in a block proposal and returning those that did not to the next lane. The function signature is as follows:
|
||||
|
||||
```go
|
||||
ProcessLaneHandler func(ctx sdk.Context, partialProposal []sdk.Tx) (
|
||||
txsFromLane []sdk.Tx,
|
||||
remainingTxs []sdk.Tx,
|
||||
err error,
|
||||
)
|
||||
```
|
||||
|
||||
Note that block proposals built using the Block SDK contain contiguous sections of transactions in the block that belong to a given lane, to read more about how proposals are constructed relative to other lanes, please visit the [abci section](../../abci/README.md). As such, a given lane will recieve some transactions in (partialProposal) that belong to it and some that do not. The transactions that belong to it must be contiguous from the start, and the transactions that do not belong to it must be contiguous from the end. The lane must return the transactions that belong to it and the transactions that do not belong to it. The transactions that do not belong to it will be passed to the next lane in the proposal. The default `ProcessLaneHandler` is implemented in the [base lane](./handlers.go). It verifies the transactions that belong to the lane and returns them alongside the transactions that do not belong to the lane.
|
||||
|
||||
Please visit the [MEV lane's](../../lanes/mev/abci.go) `ProcessLaneHandler` for an example of how to implement a custom handler.
|
||||
|
||||
## LaneMempool
|
||||
|
||||
The lane mempool is the data structure that is responsible for storing transactions that belong to a given lane, before they are included in a block proposal. The lane mempool input's a `TxPriority` object that allows developers to customize how they want to order transactions within their mempool. Additionally, it also accepts a signer extrator adapter that allows for custom signature schemes to be used (although the default covers Cosmos SDK transactions). To read more about the signer extractor adapter, please visit the [signer extractor section](../../adapters/signer_extraction_adapter/README.md).
|
||||
|
||||
### TxPriority
|
||||
|
||||
The `TxPriority` object is responsible for ordering transactions within the mempool. The definition of the `TxPriority` object is as follows:
|
||||
|
||||
```go
|
||||
// TxPriority defines a type that is used to retrieve and compare transaction
|
||||
// priorities. Priorities must be comparable.
|
||||
TxPriority[C comparable] struct {
|
||||
// GetTxPriority returns the priority of the transaction. A priority must be
|
||||
// comparable via Compare.
|
||||
GetTxPriority func(ctx context.Context, tx sdk.Tx) C
|
||||
|
||||
// CompareTxPriority compares two transaction priorities. The result should be
|
||||
// 0 if a == b, -1 if a < b, and +1 if a > b.
|
||||
Compare func(a, b C) int
|
||||
|
||||
// MinValue defines the minimum priority value, e.g. MinInt64. This value is
|
||||
// used when instantiating a new iterator and comparing weights.
|
||||
MinValue C
|
||||
}
|
||||
```
|
||||
|
||||
The default implementation can be found in the [base lane](./mempool.go). It orders transactions by their gas price in descending order. The `TxPriority` object is passed into the lane mempool constructor. Please visit the [MEV lane's](../../lanes/mev/mempool.go) `TxPriority` for an example of how to implement a custom `TxPriority`.
|
||||
|
||||
## LaneConfig
|
||||
|
||||
The lane config is the object that is responsible for configuring the lane. It allows developers to customize how the lane behaves in terms of max block space, max transaction count, and more. The definition of the `LaneConfig` object is as follows:
|
||||
|
||||
```go
|
||||
// LaneConfig defines the basic configurations needed for a lane.
|
||||
type LaneConfig struct {
|
||||
Logger log.Logger
|
||||
TxEncoder sdk.TxEncoder
|
||||
TxDecoder sdk.TxDecoder
|
||||
AnteHandler sdk.AnteHandler
|
||||
|
||||
// SignerExtractor defines the interface used for extracting the expected signers of a transaction
|
||||
// from the transaction.
|
||||
SignerExtractor signer_extraction.Adapter
|
||||
|
||||
// MaxBlockSpace defines the relative percentage of block space that can be
|
||||
// used by this lane. NOTE: If this is set to zero, then there is no limit
|
||||
// on the number of transactions that can be included in the block for this
|
||||
// lane (up to maxTxBytes as provided by the request). This is useful for the default lane.
|
||||
MaxBlockSpace math.LegacyDec
|
||||
|
||||
// MaxTxs sets the maximum number of transactions allowed in the mempool with
|
||||
// the semantics:
|
||||
// - if MaxTx == 0, there is no cap on the number of transactions in the mempool
|
||||
// - if MaxTx > 0, the mempool will cap the number of transactions it stores,
|
||||
// and will prioritize transactions by their priority and sender-nonce
|
||||
// (sequence number) when evicting transactions.
|
||||
// - if MaxTx < 0, `Insert` is a no-op.
|
||||
MaxTxs int
|
||||
}
|
||||
```
|
||||
|
||||
Each lane must define its own custom `LaneConfig` in order to be properly instantiated. Please visit [`app.go`](../../tests/app/app.go) for an example of how to implement a custom `LaneConfig`.
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# Proposals
|
||||
|
||||
## Overview
|
||||
|
||||
> The proposal type - `proposals.Proposal` - is utilized to represent a block proposal. It contains information about the total gas utilization, block size, number of transactions, raw transactions, and much more. It is recommended that you read the [proposal construction and verification](../../abci/README.md) section before continuing.
|
||||
|
||||
## Proposal
|
||||
|
||||
After a given lane executes its `PrepareLaneHandler` or `ProcessLaneHandler`, it will return a set of transactions that need to be added to the current proposal that is being constructed. To update the proposal, `Update` is called with the lane that needs to add transactions to the proposal as well as the transactions that need to be added.
|
||||
|
||||
Proposals are updated _iff_:
|
||||
|
||||
1. The total gas utilization of the partial proposal (i.e. the transactions it wants to add) are under the limits allocated for the lane and are less than the maximum gas utilization of the proposal.
|
||||
2. The total size in bytes of the partial proposal is under the limits allocated for the lane and is less than the maximum size of the proposal.
|
||||
3. The transactions have not already been added to the proposal.
|
||||
4. The lane has not already attempted to add transactions to the proposal.
|
||||
|
||||
If any of these conditions fail, the proposal will not be updated and the transactions will not be added to the proposal. The lane will be marked as having attempted to add transactions to the proposal.
|
||||
|
||||
The proposal is responsible for determining the `LaneLimits` for a given lane. The `LaneLimits` are the maximum gas utilization and size in bytes that a given lane can utilize in a block proposal. This is a function of the max gas utilization and size defined by the application, the current gas utilization and size of the proposal, and the `MaxBlockSpace` allocated to the lane as defined by its `LaneConfig`. To read more about how `LaneConfigs` are defined, please visit the [lane config section](../base/README.md#laneconfig) or see an example implementation in [`app.go`](../../tests/app/app.go).
|
||||
|
||||
Reference in New Issue
Block a user