diff --git a/README.md b/README.md index c769bd1..ca77dc2 100644 --- a/README.md +++ b/README.md @@ -85,4 +85,4 @@ with the proposal composed of Tx1, Tx2, Tx3, and Tx4. LaneA will then verify Tx1 and Tx2 and return the remaining transactions - Tx3 and Tx4. The ProcessProposalHandler will then call ProcessLane on LaneB with the remaining transactions - Tx3 and Tx4. LaneB will then verify Tx3 and Tx4 -and return no remaining transactions. \ No newline at end of file +and return no remaining transactions. diff --git a/block/README.md b/block/README.md deleted file mode 100644 index 9c6082d..0000000 --- a/block/README.md +++ /dev/null @@ -1,202 +0,0 @@ -# Block SDK 🧱 - -> 🤓 Learn and read all about how proposals are constructed and verified using -> the Block SDK - -## 📖 Overview - -The Block SDK is a framework for building smarter blocks. The Block SDK is built -harnessing the power of ABCI++ which is a new ABCI implementation that allows -for more complex and expressive applications to be built on top of the Cosmos SDK. -The process of building and verifiying proposals can be broken down into two -distinct parts: - -1. Preparing a proposal during `PrepareProposal`. -2. Processing a proposal during `ProcessProposal`. - -The Block SDK provides a framework for building and verifying proposals by -segmenting a single block into multiple lanes. Each lane can be responsible for -proposing and verifying specific types of transaction. The Block SDK provides -a default implementation of a lane that can be used to build and verify proposals -similar to how they are built and verified in the Cosmos SDK today while also -providing a framework for building more complex lanes that can be used to build -and verify much more complex proposals. - -## 🤔 How does it work - -### 🔁 Transaction Lifecycle - -The best way to understand how lanes work is to first understand the lifecycle -of a transaction. A transaction begins its lifecycle when it is first signed and -broadcasted to a chain. After it is broadcasted to a validator, it will be checked -in `CheckTx` by the base application. If the transaction is valid, it will be -inserted into the applications mempool. - -The transaction then waits in the mempool until a new block needs to be proposed. -When a new block needs to be proposed, the application will call `PrepareProposal` -(which is a new ABCI++ addition) to request a new block from the current -proposer. The proposer will look at what transactions currently waiting to -be included in a block by looking at their mempool. The proposer will then -iteratively select transactions until the block is full. The proposer will then -send the block to other validators in the network. - -When a validator receives a proposed block, the validator will first want to -verify the contents of the block before signing off on it. The validator will -call `ProcessProposal` to verify the contents of the block. If the block is -valid, the validator will sign off on the block and broadcast their vote to the -network. If the block is invalid, the validator will reject the block. Once a -block is accepted by the network, it is committed and the transactions that -were included in the block are removed from the validator's mempool (as they no -longer need to be considered). - -### 🛣️ Lane Lifecycle - -After a transaction is verified in `CheckTx`, it will attempt to be inserted -into the `LanedMempool`. A `LanedMempool` is composed of several distinct `Lanes` -that have the ability to store their own transactions. The `LanedMempool` will -insert the transaction into all lanes that will accept it. The criteria for -whether a lane will accept a transaction is defined by the lane's -`MatchHandler`. The default implementation of a `MatchHandler` will accept all transactions. - - -When a new block is proposed, the `PrepareProposalHandler` will iteratively call -`PrepareLane` on each lane (in the order in which they are defined in the -`LanedMempool`). The `PrepareLane` method is anaolgous to `PrepareProposal`. Calling -`PrepareLane` on a lane will trigger the lane to reap transactions from its mempool -and add them to the proposal (given they are valid respecting the verification rules -of the lane). - -When proposals need to be verified in `ProcessProposal`, the `ProcessProposalHandler` -defined in `abci/abci.go` will call `ProcessLane` on each lane in the same order -as they were called in the `PrepareProposalHandler`. Each subsequent call to -`ProcessLane` will filter out transactions that belong to previous lanes. A given -lane's `ProcessLane` will only verify transactions that belong to that lane. - -> **Scenario** -> -> Let's say we have a `LanedMempool` composed of two lanes: `LaneA` and `LaneB`. -> `LaneA` is defined first in the `LanedMempool` and `LaneB` is defined second. -> `LaneA` contains transactions `Tx1` and `Tx2` and `LaneB` contains transactions -> `Tx3` and `Tx4`. - - -When a new block needs to be proposed, the `PrepareProposalHandler` will call -`PrepareLane` on `LaneA` first and `LaneB` second. When `PrepareLane` is called -on `LaneA`, `LaneA` will reap transactions from its mempool and add them to the -proposal. Same applies for `LaneB`. Say `LaneA` reaps transactions `Tx1` and `Tx2` -and `LaneB` reaps transactions `Tx3` and `Tx4`. This gives us a proposal composed -of the following: - -* `Tx1`, `Tx2`, `Tx3`, `Tx4` - -When the `ProcessProposalHandler` is called, it will call `ProcessLane` on `LaneA` -with the proposal composed of `Tx1`, `Tx2`, `Tx3`, and `Tx4`. `LaneA` will then -verify `Tx1` and `Tx2` and return the remaining transactions - `Tx3` and `Tx4`. -The `ProcessProposalHandler` will then call `ProcessLane` on `LaneB` with the -remaining transactions - `Tx3` and `Tx4`. `LaneB` will then verify `Tx3` and `Tx4` -and return no remaining transactions. - -## 🏗️ Setup - -> **Note** -> -> For a more in depth example of how to use the Block SDK, check out our -> example application in `block-sdk/tests/app/app.go`. - -### 📦 Dependencies - -The Block SDK is built on top of the Cosmos SDK. The Block SDK is currently -compatible with Cosmos SDK versions greater than or equal to `v0.47.0`. - -### 📥 Installation - -To install the Block SDK, run the following command: - -```bash -go get github.com/skip-mev/block-sdk/abci -``` - -### 📚 Usage - -First determine the set of lanes that you want to use in your application. The available -lanes can be found in our **Lane App Store** in `block-sdk/lanes`. In your base -application, you will need to create a `LanedMempool` composed of the lanes that -you want to use. You will also need to create a `PrepareProposalHandler` and a -`ProcessProposalHandler` that will be responsible for preparing and processing -proposals respectively. - -```golang -// 1. Create the lanes. -// -// NOTE: The lanes are ordered by priority. The first lane is the highest priority -// lane and the last lane is the lowest priority lane. Top of block lane allows -// transactions to bid for inclusion at the top of the next block. -// -// For more information on how to utilize the LaneConfig please -// visit the README in block-sdk/block/base. -// -// MEV lane hosts an action at the top of the block. -mevConfig := constructor.LaneConfig{ - Logger: app.Logger(), - TxEncoder: app.txConfig.TxEncoder(), - TxDecoder: app.txConfig.TxDecoder(), - MaxBlockSpace: math.LegacyZeroDec(), - MaxTxs: 0, -} -mevLane := mev.NewMEVLane( - mevConfig, - mev.NewDefaultAuctionFactory(app.txConfig.TxDecoder()), -) - -// Free lane allows transactions to be included in the next block for free. -freeConfig := constructor.LaneConfig{ - Logger: app.Logger(), - TxEncoder: app.txConfig.TxEncoder(), - TxDecoder: app.txConfig.TxDecoder(), - MaxBlockSpace: math.LegacyZeroDec(), - MaxTxs: 0, -} -freeLane := free.NewFreeLane( - freeConfig, - constructor.DefaultTxPriority(), - free.DefaultMatchHandler(), -) - -// Default lane accepts all other transactions. -defaultConfig := constructor.LaneConfig{ - Logger: app.Logger(), - TxEncoder: app.txConfig.TxEncoder(), - TxDecoder: app.txConfig.TxDecoder(), - MaxBlockSpace: math.LegacyZeroDec(), - MaxTxs: 0, -} -defaultLane := base.NewStandardLane(defaultConfig) - -// Set the lanes into the mempool. -lanes := []block.Lane{ - mevLane, - freeLane, - defaultLane, -} -mempool := block.NewLanedMempool(app.Logger(), true, lanes...) -app.App.SetMempool(mempool) - -... - -anteHandler := NewAnteHandler(options) - -// Set the lane ante handlers on the lanes. -for _, lane := range lanes { - lane.SetAnteHandler(anteHandler) -} -app.App.SetAnteHandler(anteHandler) - -// Set the abci handlers on base app -proposalHandler := abci.NewProposalHandler( - app.Logger(), - app.TxConfig().TxDecoder(), - lanes, -) -app.App.SetPrepareProposal(proposalHandler.PrepareProposalHandler()) -app.App.SetProcessProposal(proposalHandler.ProcessProposalHandler()) -``` diff --git a/lanes/free/README.md b/lanes/free/README.md index 165a579..42816a5 100644 --- a/lanes/free/README.md +++ b/lanes/free/README.md @@ -28,8 +28,7 @@ compatible with Cosmos SDK versions greater than or equal to `v0.47.0`. To install the Block SDK, run the following command: ```bash -$ go get github.com/skip-mev/block-sdk/abci -$ go get github.com/skip-mev/block-sdk/lanes/free +$ go install github.com/skip-mev/block-sdk ``` ### 📚 Usage diff --git a/lanes/mev/README.md b/lanes/mev/README.md index 4c7c701..b8d3b0f 100644 --- a/lanes/mev/README.md +++ b/lanes/mev/README.md @@ -17,144 +17,279 @@ type, called a MsgAuctionBid, that allows the submitter to execute multiple transactions at the top of the block atomically (atomically = directly next to each other). -## 🏗️ Setup +## Install -> **Note** -> -> For a more in depth example of how to use the Block SDK, check out our -> example application in `block-sdk/tests/app/app.go`. - -### 📦 Dependencies - -The Block SDK is built on top of the Cosmos SDK. The Block SDK is currently -compatible with Cosmos SDK versions greater than or equal to `v0.47.0`. - -### 📥 Installation - -To install the Block SDK, run the following command: - -```bash -$ go get github.com/skip-mev/block-sdk/abci -$ go get github.com/skip-mev/block-sdk/lanes/mev +```shell +$ go install github.com/skip-mev/block-sdk ``` -### 📚 Usage +## Setup -1. First determine the set of lanes that you want to use in your application. The -available lanes can be found in our **Lane App Store** in `block-sdk/lanes`. In -your base application, you will need to create a `LanedMempool` composed of the -lanes that you want to use. -2. Next, order the lanes by priority. The first lane is the highest priority lane -and the last lane is the lowest priority lane. **It is recommended that the first -lane is the MEV lane as the top of block is the most valuable block space.** -3. You will also need to create a `PrepareProposalHandler` and a -`ProcessProposalHandler` that will be responsible for preparing and processing -proposals respectively. Configure the order of the lanes in the -`PrepareProposalHandler` and `ProcessProposalHandler` to match the order of the -lanes in the `LanedMempool`. +> This set up guide will walk you through the process of setting up a POB +> application. In particular, we will configure an application with the +> following features: +> +>* MEV lane (auction lane). This will create an MEV lane where users can bid to +> have their transactions executed at the top of the block. +>* Free lane. This will create a free lane where users can submit transactions +> that will be executed for free (no fees). +>* Default lane. This will create a default lane where users can submit +> transactions that will be executed with the default app logic. +>* Builder module that pairs with the auction lane to process auction +> transactions and distribute revenue to the auction house. -```golang -import ( - "github.com/skip-mev/block-sdk/abci" - "github.com/skip-mev/block-sdk/lanes/mev" -) +1. Import the necessary dependencies into your application. This includes the + Block SDK proposal handlers + mempool, keeper, builder types, and builder + module. This tutorial will go into more detail into each of the dependencies. -... -``` - -```golang -func NewApp() { + ```go + import ( ... - // 1. Create the lanes. - // - // NOTE: The lanes are ordered by priority. The first lane is the highest priority - // lane and the last lane is the lowest priority lane. Top of block lane allows - // transactions to bid for inclusion at the top of the next block. - // - // For more information on how to utilize the LaneConfig please - // visit the README in block-sdk/block/base. - // - // MEV lane hosts an action at the top of the block. - mevConfig := constructor.LaneConfig{ - Logger: app.Logger(), - TxEncoder: app.txConfig.TxEncoder(), - TxDecoder: app.txConfig.TxDecoder(), - MaxBlockSpace: math.LegacyZeroDec(), - MaxTxs: 0, - } - mevLane := mev.NewMEVLane( - mevConfig, - mev.NewDefaultAuctionFactory(app.txConfig.TxDecoder()), - ) + "github.com/skip-mev/pob/block-sdk" + "github.com/skip-mev/pob/block-sdk/abci" + "github.com/skip-mev/pob/block-sdk/lanes/auction" + "github.com/skip-mev/pob/block-sdk/lanes/base" + "github.com/skip-mev/pob/block-sdk/lanes/free" + buildermodule "github.com/skip-mev/block-sdk/x/builder" + builderkeeper "github.com/skip-mev/block-sdk/x/builder/keeper" + ... + ) + ``` - // Free lane allows transactions to be included in the next block for free. - freeConfig := constructor.LaneConfig{ - Logger: app.Logger(), - TxEncoder: app.txConfig.TxEncoder(), - TxDecoder: app.txConfig.TxDecoder(), - MaxBlockSpace: math.LegacyZeroDec(), - MaxTxs: 0, - } - freeLane := free.NewFreeLane( - freeConfig, - constructor.DefaultTxPriority(), - free.DefaultMatchHandler(), - ) +2. Add your module to the the `AppModuleBasic` manager. This manager is in + charge of setting up basic, non-dependent module elements such as codec + registration and genesis verification. This will register the special + `MsgAuctionBid` message. When users want to bid for top of block execution, + they will submit a transaction - which we call an auction transaction - that + includes a single `MsgAuctionBid`. We prevent any other messages from being + included in auction transaction to prevent malicious behavior - such as front + running or sandwiching. - // Default lane accepts all other transactions. - defaultConfig := constructor.LaneConfig{ - Logger: app.Logger(), - TxEncoder: app.txConfig.TxEncoder(), - TxDecoder: app.txConfig.TxDecoder(), - MaxBlockSpace: math.LegacyZeroDec(), - MaxTxs: 0, - } - defaultLane := base.NewStandardLane(defaultConfig) + ```go + var ( + ModuleBasics = module.NewBasicManager( + ... + buildermodule.AppModuleBasic{}, + ) + ... + ) + ``` - // 2. Set up the relateive priority of lanes - lanes := []block.Lane{ - mevLane, - freeLane, - defaultLane, - } - mempool := block.NewLanedMempool(app.Logger(), true, lanes...) - app.App.SetMempool(mempool) +3. The builder `Keeper` is POB's gateway to processing special `MsgAuctionBid` + messages that allow users to participate in the top of block auction, distribute + revenue to the auction house, and ensure the validity of auction transactions. - ... + a. First add the keeper to the app's struct definition. We also want to add + MEV lane's custom checkTx handler to the app's struct definition. This will + allow us to override the default checkTx handler to process bid transactions + before they are inserted into the mempool. NOTE: The custom handler is + required as otherwise the auction can be held hostage by a malicious + users. - // 3. Set up the ante handler. - anteDecorators := []sdk.AnteDecorator{ - ante.NewSetUpContextDecorator(), + ```go + type App struct { ... - utils.NewIgnoreDecorator( - ante.NewDeductFeeDecorator( - options.BaseOptions.AccountKeeper, - options.BaseOptions.BankKeeper, - options.BaseOptions.FeegrantKeeper, - options.BaseOptions.TxFeeChecker, - ), - options.FreeLane, - ), + // BuilderKeeper is the keeper that handles processing auction transactions + BuilderKeeper builderkeeper.Keeper + + // Custom checkTx handler + checkTxHandler mev.CheckTx + } + ``` + + b. Add the builder module to the list of module account permissions. This will + instantiate the builder module account on genesis. + + ```go + maccPerms = map[string][]string{ + builder.ModuleName: nil, ... - } + } + ``` - anteHandler := sdk.ChainAnteDecorators(anteDecorators...) + c. Instantiate the blockbuster mempool with the application's desired lanes. - // Set the lane ante handlers on the lanes. - for _, lane := range lanes { - lane.SetAnteHandler(anteHandler) - } - app.App.SetAnteHandler(anteHandler) + ```go + // 1. Create the lanes. + // + // NOTE: The lanes are ordered by priority. The first lane is the + // highest priority lane and the last lane is the lowest priority lane. + // Top of block lane allows transactions to bid for inclusion at the + // top of the next block. + // + // For more information on how to utilize the LaneConfig please + // visit the README in block-sdk/block/base. + // + // MEV lane hosts an auction at the top of the block. + mevConfig := constructor.LaneConfig{ + Logger: app.Logger(), + TxEncoder: app.txConfig.TxEncoder(), + TxDecoder: app.txConfig.TxDecoder(), + MaxBlockSpace: math.LegacyZeroDec(), + MaxTxs: 0, + } + mevLane := mev.NewMEVLane( + mevConfig, + mev.NewDefaultAuctionFactory(app.txConfig.TxDecoder()), + ) - // 4. Set the abci handlers on base app + // Free lane allows transactions to be included in the next block for free. + freeConfig := constructor.LaneConfig{ + Logger: app.Logger(), + TxEncoder: app.txConfig.TxEncoder(), + TxDecoder: app.txConfig.TxDecoder(), + MaxBlockSpace: math.LegacyZeroDec(), + MaxTxs: 0, + } + freeLane := free.NewFreeLane( + freeConfig, + constructor.DefaultTxPriority(), + free.DefaultMatchHandler(), + ) + + // Standard lane accepts all other transactions. + defaultConfig := constructor.LaneConfig{ + Logger: app.Logger(), + TxEncoder: app.txConfig.TxEncoder(), + TxDecoder: app.txConfig.TxDecoder(), + MaxBlockSpace: math.LegacyZeroDec(), + MaxTxs: 0, + } + defaultLane := base.NewStandardLane(defaultConfig) + + // 2. Set up the relateive priority of lanes + lanes := []block.Lane{ + mevLane, + freeLane, + defaultLane, + } + mempool := block.NewLanedMempool(app.Logger(), true, lanes...) + app.App.SetMempool(mempool) + ``` + + d. Instantiate the antehandler chain for the application with awareness of the + blockbuster mempool. This will allow the application to verify the validity + of a transaction respecting the desired logic of a given lane. In this walkthrough, + we want the `FeeDecorator` to be ignored for all transactions that should + belong to the free lane. Additionally, we want to add the `x/builder` + module's `AuctionDecorator` to the ante-handler chain. The `AuctionDecorator` + is an AnteHandler decorator that enforces various chain configurable MEV rules. + + ```go + import ( + ... + "github.com/skip-mev/pob/blockbuster" + "github.com/skip-mev/pob/blockbuster/utils" + builderante "github.com/skip-mev/pob/x/builder/ante" + ... + ) + + anteDecorators := []sdk.AnteDecorator{ + ante.NewSetUpContextDecorator(), + ... + utils.NewIgnoreDecorator( + ante.NewDeductFeeDecorator( + options.BaseOptions.AccountKeeper, + options.BaseOptions.BankKeeper, + options.BaseOptions.FeegrantKeeper, + options.BaseOptions.TxFeeChecker, + ), + options.FreeLane, + ), + ... + builderante.NewBuilderDecorator( + options.BuilderKeeper, + options.TxEncoder, + options.TOBLane, + options.Mempool, + ), + } + + anteHandler := sdk.ChainAnteDecorators(anteDecorators...) + app.SetAnteHandler(anteHandler) + + // Set the antehandlers on the lanes. + for _, lane := range lanes { + lane.SetAnteHandler(anteHandler) + } + app.App.SetAnteHandler(anteHandler) + ``` + + e. Instantiate the builder keeper, store keys, and module manager. Note, be + sure to do this after all the required keeper dependencies have been instantiated. + + ```go + keys := storetypes.NewKVStoreKeys( + buildertypes.StoreKey, + ... + ) + + ... + app.BuilderKeeper := builderkeeper.NewKeeper( + appCodec, + keys[buildertypes.StoreKey], + app.AccountKeeper, + app.BankKeeper, + app.DistrKeeper, + app.StakingKeeper, + authtypes.NewModuleAddress(govv1.ModuleName).String(), + ) + + + app.ModuleManager = module.NewManager( + builder.NewAppModule(appCodec, app.BuilderKeeper), + ... + ) + ``` + + e. Configure the proposal/checkTx handlers on base app. + + ```go + + // Create the proposal handler that will be used to build and validate blocks. proposalHandler := abci.NewProposalHandler( - app.Logger(), - app.TxConfig().TxDecoder(), - lanes, + app.Logger(), + app.txConfig.TxDecoder(), + mempool, ) app.App.SetPrepareProposal(proposalHandler.PrepareProposalHandler()) app.App.SetProcessProposal(proposalHandler.ProcessProposalHandler()) + + // Set the custom CheckTx handler on BaseApp. + checkTxHandler := abci.NewCheckTxHandler( + app.App, + app.txConfig.TxDecoder(), + tobLane, + anteHandler, + app.ChainID(), + ) + app.SetCheckTx(checkTxHandler.CheckTx()) ... -} -``` + + + func (app *TestApp) CheckTx(req cometabci.RequestCheckTx) + cometabci.ResponseCheckTx { + return app.checkTxHandler(req) + } + + // SetCheckTx sets the checkTxHandler for the app. + func (app *TestApp) SetCheckTx(handler abci.CheckTx) { + app.checkTxHandler = handler + } + ``` + + f. Finally, update the app's `InitGenesis` order and ante-handler chain. + + ```go + genesisModuleOrder := []string{ + buildertypes.ModuleName, + ..., + } + ``` + +## Params + +Note, before building or upgrading the application, make sure to initialize the +escrow address for POB in the parameters of the module. The default parameters +initialize the escrow address to be the module account address. diff --git a/lanes/standard/README.md b/lanes/standard/README.md index f466958..d194863 100644 --- a/lanes/standard/README.md +++ b/lanes/standard/README.md @@ -35,8 +35,7 @@ compatible with Cosmos SDK versions greater than or equal to `v0.47.0`. To install the Block SDK, run the following command: ```bash -$ go get github.com/skip-mev/block-sdk/abci -$ go get github.com/skip-mev/block-sdk/lanes/standard +$ go install github.com/skip-mev/block-sdk ``` ### 📚 Usage