diff --git a/README.md b/README.md index ca77dc2..17d1904 100644 --- a/README.md +++ b/README.md @@ -9,80 +9,57 @@ [![License: Apache-2.0](https://img.shields.io/github/license/skip-mev/pob.svg?style=flat-square)](https://github.com/skip-mev/pob/blob/main/LICENSE) [![Lines Of Code](https://img.shields.io/tokei/lines/github/skip-mev/pob?style=flat-square)](https://github.com/skip-mev/pob) -## πŸ“– Overview +### πŸ€”Β What is the Block SDK? -The Block SDK is a set of Cosmos SDK and ABCI++ primitives that allow chains to fully customize blocks to specific use cases. It turns your chain's blocks into a **transaction highway** consisting of individual lanes with their own special functionality. - -## πŸ€” How does it work - -### πŸ” Transaction Lifecycle - -The best way to understand how lanes work is to first understand the lifecycle -of a transaction. A transaction begins its lifecycle when it is first signed and -broadcasted to a chain. After it is broadcasted to a validator, it will be checked -in `CheckTx` by the base application. If the transaction is valid, it will be -inserted into the applications mempool. - -The transaction then waits in the mempool until a new block needs to be proposed. -When a new block needs to be proposed, the application will call `PrepareProposal` -(which is a new ABCI++ addition) to request a new block from the current -proposer. The proposer will look at what transactions currently waiting to -be included in a block by looking at their mempool. The proposer will then -iteratively select transactions until the block is full. The proposer will then -send the block to other validators in the network. - -When a validator receives a proposed block, the validator will first want to -verify the contents of the block before signing off on it. The validator will -call `ProcessProposal` to verify the contents of the block. If the block is -valid, the validator will sign off on the block and broadcast their vote to the -network. If the block is invalid, the validator will reject the block. Once a -block is accepted by the network, it is committed and the transactions that -were included in the block are removed from the validator's mempool (as they no -longer need to be considered). - -### πŸ›£οΈ Lane Lifecycle - -After a transaction is verified in CheckTx, it will attempt to be inserted -into the `LanedMempool`. A LanedMempool is composed of several distinct `Lanes` -that have the ability to store their own transactions. The LanedMempool will -insert the transaction into all lanes that will accept it. The criteria for -whether a lane will accept a transaction is defined by the lane's -`MatchHandler`. The default implementation of a MatchHandler will accept all transactions. +> **🌐 The Block SDK is a toolkit for building customized blocks** +> The Block SDK is a set of Cosmos SDK and ABCI++ primitives that allow chains to fully customize blocks to specific use cases. It turns your chain's blocks into a **`highway`** consisting of individual **`lanes`** with their own special functionality. -When a new block is proposed, the `PrepareProposalHandler` will iteratively call -`PrepareLane` on each lane (in the order in which they are defined in the -LanedMempool). The PrepareLane method is anaolgous to PrepareProposal. Calling -PrepareLane on a lane will trigger the lane to reap transactions from its mempool -and add them to the proposal (given they are valid respecting the verification rules -of the lane). +Skip has built out a number of plug-and-play `lanes` on the SDK that your protocol can use, including in-protocol MEV recapture and Oracles! Additionally, the Block SDK can be extended to add **your own custom `lanes`** to configure your blocks to exactly fit your application needs. -When proposals need to be verified in `ProcessProposal`, the `ProcessProposalHandler` -defined in `abci/abci.go` will call `ProcessLane` on each lane in the same order -as they were called in the PrepareProposalHandler. Each subsequent call to -ProcessLane will filter out transactions that belong to previous lanes. A given -lane's ProcessLane will only verify transactions that belong to that lane. +### ❌ Problems: Blocks are not Customizable -> **Scenario** -> -> Let's say we have a LanedMempool composed of two lanes: LaneA and LaneB. -> LaneA is defined first in the LanedMempool and LaneB is defined second. -> LaneA contains transactions Tx1 and Tx2 and LaneB contains transactions -> Tx3 and Tx4. +Most Cosmos chains today utilize standard `CometBFT` block construction - which is too limited. + +- The standard `CometBFT` block building is susceptible to MEV-related issues, such as front-running and sandwich attacks, since proposers have monopolistic rights on ordering and no verification of good behavior. MEV that is created cannot be redistributed to the protocol. +- The standard `CometBFT` block building uses a one-size-fits-all approach, which can result in inefficient transaction processing for specific applications or use cases and sub-optimal fee markets. +- Transactions tailored for specific applications may need custom prioritization, ordering or validation rules that the mempool is otherwise unaware of because transactions within a block are currently in-differentiable when a blockchain might want them to be. + +### βœ… Solution: The Block SDK + +You can think of the Block SDK as a **transaction `highway` system**, where each +`lane` on the highway serves a specific purpose and has its own set of rules and +traffic flow. + +In the Block SDK, each `lane` has its own set of rules and transaction flow management systems. + +* A `lane` is what we might traditionally consider to be a standard mempool + where transaction **_validation_**, **_ordering_** and **_prioritization_** for + contained transactions are shared. +* `lanes` implement a **standard interface** that allows each individual `lane` to + propose and validate a portion of a block. +* `lanes` are ordered with each other, configurable by developers. All `lanes` + together define the desired block structure of a chain. + +### ✨ Block SDK Use Cases + +A block with separate `lanes` can be used for: + +1. **MEV mitigation**: a top of block lane could be designed to create an in-protocol top-of-block auction (as we are doing with POB) to recapture MEV in a transparent and governable way. +2. **Free/reduced fee txs**: transactions with certain properties (e.g. from trusted accounts or performing encouraged actions) could leverage a free lane to reward behavior. +3. **Dedicated oracle space** Oracles could be included before other kinds of transactions to ensure that price updates occur first, and are not able to be sandwiched or manipulated. +4. **Orderflow auctions**: an OFA lane could be constructed such that order flow providers can have their submitted transactions bundled with specific backrunners, to guarantee MEV rewards are attributed back to users. Imagine MEV-share but in protocol. +5. **Enhanced and customizable privacy**: privacy-enhancing features could be introduced, such as threshold encrypted lanes, to protect user data and maintain privacy for specific use cases. +6. **Fee market improvements**: one or many fee markets - such as EIP-1559 - could be easily adopted for different lanes (potentially custom for certain dApps). Each smart contract/exchange could have its own fee market or auction for transaction ordering. +7. **Congestion management**: segmentation of transactions to lanes can help mitigate network congestion by capping usage of certain applications and tailoring fee markets. -When a new block needs to be proposed, the PrepareProposalHandler will call -PrepareLane on LaneA first and LaneB second. When PrepareLane is called -on LaneA, LaneA will reap transactions from its mempool and add them to the -proposal. Same applies for LaneB. Say LaneA reaps transactions Tx1 and Tx2 -and LaneB reaps transactions Tx3 and Tx4. This gives us a proposal composed -of the following: +### πŸ“š Block SDK Documentation -* Tx1, Tx2, Tx3, Tx4 +#### Lane App Store -When the ProcessProposalHandler is called, it will call ProcessLane on LaneA -with the proposal composed of Tx1, Tx2, Tx3, and Tx4. LaneA will then -verify Tx1 and Tx2 and return the remaining transactions - Tx3 and Tx4. -The ProcessProposalHandler will then call ProcessLane on LaneB with the -remaining transactions - Tx3 and Tx4. LaneB will then verify Tx3 and Tx4 -and return no remaining transactions. +To read more about Skip's pre-built `lanes` and how to use them, check out the [Lane App Store](). + +#### Lane Development + +To read more about how to build your own custom `lanes`, check out the [Build Your Own Lane](). diff --git a/block/base/README.md b/block/base/README.md deleted file mode 100644 index 51d10ae..0000000 --- a/block/base/README.md +++ /dev/null @@ -1,452 +0,0 @@ -# 🎨 Base Lane - -> πŸ—οΈ Build your own lane in less than 10 minutes using the Base Lane - -## πŸ’‘ Overview - -The Base Lane is a generic implementation of a lane. It comes out of the -box with default implementations for all the required interfaces. It is meant to -be used as a starting point for building your own lane. - -## πŸ€” How to use it - -> **Default Implementations** -> -> There are default implementations for all of the below which can be found in -> the `block/base` package. It is highly recommended that developers overview -> the default implementations before building their own lane. - -There are **three** critical components to building a custom lane using the lane -constructor: - -1. `LaneConfig` - The lane configuration which determines the basic properties -of the lane including the maximum block space that the lane can fill up. -2. `LaneMempool` - The lane mempool which is responsible for storing -transactions that have been verified and are waiting to be included in proposals. -3. `MatchHandler` - This is responsible for determining whether a transaction should -belong to this lane. -4. [**OPTIONAL**] `PrepareLaneHandler` - Allows developers to define their own -handler to customize the how transactions are verified and ordered before they -are included into a proposal. -5. [**OPTIONAL**] `CheckOrderHandler` - Allows developers to define their own -handler that will run any custom checks on whether transactions included in -block proposals are in the correct order (respecting the ordering rules of the -lane and the ordering rules of the other lanes). -6. [**OPTIONAL**] `ProcessLaneHandler` - Allows developers to define their own -handler for processing transactions that are included in block proposals. - - -### 1. πŸ“ Lane Config - -The lane config (`LaneConfig`) is a simple configuration -object that defines the desired amount of block space the lane should -utilize when building a proposal, an antehandler that is used to verify -transactions as they are added/verified to/in a proposal, and more. By default, -we recommend that user's pass in all of the base apps configurations (txDecoder, -logger, etc.). A sample `LaneConfig` might look like the following: - -```golang -config := block.LaneConfig{ - Logger: app.Logger(), - TxDecoder: app.TxDecoder(), - TxEncoder: app.TxEncoder(), - AnteHandler: app.AnteHandler(), - MaxTxs: 0, - MaxBlockSpace: math.LegacyZeroDec(), - IgnoreList: []block.Lane{}, -} -``` - -The three most important parameters to set are the `AnteHandler`, `MaxTxs`, and -`MaxBlockSpace`. - -#### **AnteHandler** - -With the default implementation, the `AnteHandler` is responsible for verifying -transactions as they are being considered for a new proposal or are being processed -in a proposed block. We recommend user's utilize the same antehandler chain that -is used in the base app. If developers want a certain `AnteDecorator` to be -ignored if it qualifies for a given lane, they can do so by using the `NewIgnoreDecorator` -defined in `block/utils/ante.go`. - -For example, a free lane might want to ignore the `DeductFeeDecorator` so that it's -transactions are not charged any fees. Where ever the `AnteHandler` is defined, -we could add the following to ignore the `DeductFeeDecorator`: - -```golang -anteDecorators := []sdk.AnteDecorator{ - ante.NewSetUpContextDecorator(), - ..., - utils.NewIgnoreDecorator( - ante.NewDeductFeeDecorator( - options.BaseOptions.AccountKeeper, - options.BaseOptions.BankKeeper, - options.BaseOptions.FeegrantKeeper, - options.BaseOptions.TxFeeChecker, - ), - options.FreeLane, - ), - ..., -} -``` - -Anytime a transaction that qualifies for the free lane is being processed, the -`DeductFeeDecorator` will be ignored and no fees will be deducted! - - -#### **MaxTxs** - -This sets the maximum number of transactions allowed in the mempool with -the semantics: - -* if `MaxTxs` == 0, there is no cap on the number of transactions in the mempool -* if `MaxTxs` > 0, the mempool will cap the number of transactions it stores, - and will prioritize transactions by their priority and sender-nonce - (sequence number) when evicting transactions. -* if `MaxTxs` < 0, `Insert` is a no-op. - -#### **MaxBlockSpace** - -MaxBlockSpace is the maximum amount of block space that the lane will attempt to -fill when building a proposal. This parameter may be useful lanes that should be -limited (such as a free or onboarding lane) in space usage. Setting this to 0 -will allow the lane to fill the block with as many transactions as possible. - -If a block proposal request has a `MaxTxBytes` of 1000 and the lane has a -`MaxBlockSpace` of 0.5, the lane will attempt to fill the block with 500 bytes. - -#### **[OPTIONAL] IgnoreList** - -`IgnoreList` defines the list of lanes to ignore when processing transactions. -For example, say there are two lanes: default and free. The free lane is -processed after the default lane. In this case, the free lane should be added -to the ignore list of the default lane. Otherwise, the transactions that belong -to the free lane will be processed by the default lane (which accepts all -transactions by default). - - -### 2. πŸ—„οΈ LaneMempool - -This is the data structure that is responsible for storing transactions -as they are being verified and are waiting to be included in proposals. `block/base/mempool.go` -provides an out-of-the-box implementation that should be used as a starting -point for building out the mempool and should cover most use cases. To -utilize the mempool, you must implement a `TxPriority[C]` struct that does the -following: - -* Implements a `GetTxPriority` method that returns the priority (as defined -by the type `[C]`) of a given transaction. -* Implements a `Compare` method that returns the relative priority of two -transactions. If the first transaction has a higher priority, the method -should return -1, if the second transaction has a higher priority the method -should return 1, otherwise the method should return 0. -* Implements a `MinValue` method that returns the minimum priority value -that a transaction can have. - -The default implementation can be found in `block/base/mempool.go`. What -if we wanted to prioritize transactions by the amount they have staked on a chain? -Well we could do something like the following: - -```golang -// CustomTxPriority returns a TxPriority that prioritizes transactions by the -// amount they have staked on chain. This means that transactions with a higher -// amount staked will be prioritized over transactions with a lower amount staked. -func (p *CustomTxPriority) CustomTxPriority() TxPriority[string] { - return TxPriority[string]{ - GetTxPriority: func(ctx context.Context, tx sdk.Tx) string { - // Get the signer of the transaction. - signer := p.getTransactionSigner(tx) - - // Get the total amount staked by the signer on chain. - // This is abstracted away in the example, but you can - // implement this using the staking keeper. - totalStake, err := p.getTotalStake(ctx, signer) - if err != nil { - return "" - } - - return totalStake.String() - }, - Compare: func(a, b string) int { - aCoins, _ := sdk.ParseCoinsNormalized(a) - bCoins, _ := sdk.ParseCoinsNormalized(b) - - switch { - case aCoins == nil && bCoins == nil: - return 0 - - case aCoins == nil: - return -1 - - case bCoins == nil: - return 1 - - default: - switch { - case aCoins.IsAllGT(bCoins): - return 1 - - case aCoins.IsAllLT(bCoins): - return -1 - - default: - return 0 - } - } - }, - MinValue: "", - } -} -``` - -#### Using a Custom TxPriority - -To utilize this new priority configuration in a lane, all you have to then do -is pass in the `TxPriority[C]` to the `NewLaneMempool` function. - -```golang -// Create the lane config -laneCfg := NewLaneConfig( - ... - MaxTxs: 100, - ... -) - -// Pseudocode for creating the custom tx priority -priorityCfg := NewPriorityConfig( - stakingKeeper, - accountKeeper, - ... -) - - -// define your mempool that orders transactions by on-chain stake -mempool := constructor.NewMempool[string]( - priorityCfg.CustomTxPriority(), - laneCfg.TxEncoder, - laneCfg.MaxTxs, -) - -// Initialize your lane with the mempool -lane := constructor.NewBaseLane( - laneCfg, - LaneName, - mempool, - constructor.DefaultMatchHandler(), -) -``` - -### 3. 🀝 MatchHandler - -`MatchHandler` is utilized to determine if a transaction should be included in -the lane. This function can be a stateless or stateful check on the transaction. -The default implementation can be found in `block/base/handlers.go`. - -The match handler can be as custom as desired. Following the example above, if -we wanted to make a lane that only accepts transactions if they have a large -amount staked, we could do the following: - -```golang -// CustomMatchHandler returns a custom implementation of the MatchHandler. It -// matches transactions that have a large amount staked. These transactions -// will then be charged no fees at execution time. -// -// NOTE: This is a stateful check on the transaction. The details of how to -// implement this are abstracted away in the example, but you can implement -// this using the staking keeper. -func (h *Handler) CustomMatchHandler() block.MatchHandler { - return func(ctx sdk.Context, tx sdk.Tx) bool { - if !h.IsStakingTx(tx) { - return false - } - - signer, err := getTxSigner(tx) - if err != nil { - return false - } - - stakedAmount, err := h.GetStakedAmount(signer) - if err != nil { - return false - } - - // The transaction can only be considered for inclusion if the amount - // staked is greater than some predetermined threshold. - return stakeAmount.GT(h.Threshold) - } -} -``` - -#### Using a Custom MatchHandler - -If we wanted to create the lane using the custom match handler along with the -custom mempool, we could do the following: - -```golang -// Pseudocode for creating the custom match handler -handler := NewHandler( - stakingKeeper, - accountKeeper, - ... -) - -// define your mempool that orders transactions by on chain stake -mempool := constructor.NewMempool[string]( - priorityCfg.CustomTxPriority(), - cfg.TxEncoder, - cfg.MaxTxs, -) - -// Initialize your lane with the mempool -lane := constructor.NewBaseLane( - cfg, - LaneName, - mempool, - handler.CustomMatchHandler(), -) -``` - -### Summary on Steps 1-3 - -The following is a summary of the steps above: - -1. Create a custom `LaneConfig` struct that defines the configuration of the lane. -2. Create a custom `TxPriority[C]` struct to have a custom mempool that orders -transactions via a custom priority mechanism. -3. Create a custom `MatchHandler` that implements the `block.MatchHandler` to -have a custom lane that only accepts transactions that match a custom criteria. - -### [OPTIONAL] Steps 4-6 - -The remaining steps walk through the process of creating custom block -building/verification logic. The default implementation found in `block/base/handlers.go` -should fit most use cases. Please reference that file for more details on -the default implementation and whether it fits your use case. - -Implementing custom block building/verification logic is a bit more involved -than the previous steps and is a all or nothing approach. This means that if -you implement any of the handlers, you must implement all of them in most cases. -If you do not implement all of them, the lane may have unintended behavior. - -### 4. πŸ› οΈ PrepareLaneHandler - -The `PrepareLaneHandler` is an optional field you can set on the lane constructor. -This handler is responsible for the transaction selection logic when a new proposal -is requested. - -The handler should return the following for a given lane: - -1. The transactions to be included in the block proposal. -2. The transactions to be removed from the lane's mempool. -3. An error if the lane is unable to prepare a block proposal. - -```golang -// PrepareLaneHandler is responsible for preparing transactions to be included -// in the block from a given lane. Given a lane, this function should return -// the transactions to include in the block, the transactions that must be -// removed from the lane, and an error if one occurred. -PrepareLaneHandler func(ctx sdk.Context,proposal BlockProposal,maxTxBytes int64) - (txsToInclude [][]byte, txsToRemove []sdk.Tx, err error) -``` - -The default implementation is simple. It will continue to select transactions -from its mempool under the following criteria: - -1. The transactions is not already included in the block proposal. -2. The transaction is valid and passes the AnteHandler check. -3. The transaction is not too large to be included in the block. - -If a more involved selection process is required, you can implement your own -`PrepareLaneHandler` and and set it after creating the lane constructor. - -```golang -// Pseudocode for creating the custom prepare lane handler -// This assumes that the CustomLane inherits from the constructor -// lane. -customLane := constructor.NewCustomLane( - cfg, - LaneName, - mempool, - handler.CustomMatchHandler(), -) - -// Set the custom PrepareLaneHandler on the lane -customLane.SetPrepareLaneHandler(customlane.PrepareLaneHandler()) -``` - -### 5. βœ… CheckOrderHandler - -The `CheckOrderHandler` is an optional field you can set on the lane constructor. -This handler is responsible for verifying the ordering of the transactions in the -block proposal that belong to the lane. - -```golang -// CheckOrderHandler is responsible for checking the order of transactions that -// belong to a given lane. This handler should be used to verify that the -// ordering of transactions passed into the function respect the ordering logic -// of the lane (if any transactions from the lane are included). This function -// should also ensure that transactions that belong to this lane are contiguous -// and do not have any transactions from other lanes in between them. -CheckOrderHandler func(ctx sdk.Context, txs []sdk.Tx) error -``` - -The default implementation is simple and utilizes the same `TxPriority` struct -that the mempool uses to determine if transactions are in order. The criteria -for determining if transactions are in order is as follows: - -1. The transactions are in order according to the `TxPriority` struct. i.e. any -two transactions (that match to the lane) `tx1` and `tx2` where `tx1` has a -higher priority than `tx2` should be ordered before `tx2`. -2. The transactions are contiguous. i.e. there are no transactions from other -lanes in between the transactions that belong to this lane. i.e. if `tx1` and -`tx2` belong to the lane, there should be no transactions from other lanes in -between `tx1` and `tx2`. - -If a more involved ordering process is required, you can implement your own -`CheckOrderHandler` and and set it after creating the lane constructor. - -```golang -// Pseudocode for creating the custom check order handler -// This assumes that the CustomLane inherits from the constructor -// lane. -customLane := constructor.NewCustomLane( - cfg, - LaneName, - mempool, - handler.CustomMatchHandler(), -) - -// Set the custom CheckOrderHandler on the lane -customLane.SetCheckOrderHandler(customlane.CheckOrderHandler()) -``` - - -### 6. πŸ†— ProcessLaneHandler - -The `ProcessLaneHandler` is an optional field you can set on the lane constructor. -This handler is responsible for verifying the transactions in the block proposal -that belong to the lane. This handler is executed after the `CheckOrderHandler` -so the transactions passed into this function SHOULD already be in order -respecting the ordering rules of the lane and respecting the ordering rules of -mempool relative to the lanes it has. This means that if the first transaction -does not belong to the lane, the remaining transactions should not belong to the -lane either. - - -```golang -// ProcessLaneHandler is responsible for processing transactions that are -// included in a block and belong to a given lane. ProcessLaneHandler is -// executed after CheckOrderHandler so the transactions passed into this -// function SHOULD already be in order respecting the ordering rules of the -// lane and respecting the ordering rules of mempool relative to the lanes it has. -ProcessLaneHandler func(ctx sdk.Context, txs []sdk.Tx) ([]sdk.Tx, error) -``` - -Given the invarients above, the default implementation is simple. It will -continue to verify transactions in the block proposal under the following -criteria: - -1. If a transaction matches to this lane, verify it and continue. If it is not -valid, return an error. -2. If a transaction does not match to this lane, return the remaining transactions -to the next lane to process.