80 lines
4.4 KiB
Markdown
80 lines
4.4 KiB
Markdown
|
The main goal of creating a transformer is to fetch specific log events from Ethereum, convert/decode them into usable data and then persist them to VulcanizeDB. For Maker there are two main types of log events that we're tracking: custom events that are defined in the contract solidity code, and LogNote events which utilize the [DSNote library](https://github.com/dapphub/ds-note). The transformer process for each of these different log types is the same, except for the converting process, as denoted below.
|
||
|
|
|
||
|
|
## Creating a Transformer for custom events (i.e. FlopKick)
|
||
|
|
To illustrate how to create a custom log event transformer we'll use the Kick event defined in [flop.sol](https://github.com/makerdao/dss/blob/master/src/flop.sol) as an example.
|
||
|
|
|
||
|
|
1. Get an example FlopKick log event either from mainnet (if the
|
||
|
|
contract has already been deployed), from the Kovan testnet, or by
|
||
|
|
deploying the contract to a local chain and emitting the event manually.
|
||
|
|
We will use the example log event to test drive converting the log to a
|
||
|
|
FlopKick database model.
|
||
|
|
|
||
|
|
1. Fetch the appropriate logs from the chain.
|
||
|
|
- Most transformers use `shared.LogFetcher`
|
||
|
|
- Price Feeds
|
||
|
|
|
||
|
|
1. Convert the raw log into a database model.
|
||
|
|
- For Custom Events, such as FlopKick
|
||
|
|
1. Create a converter to convert the raw log into a Go structure.
|
||
|
|
- We've been using [go-ethereum's abigen tool](https://github.com/ethereum/go-ethereum/tree/master/cmd/abigen) to get the contract's ABI, and a Go struct that represents the event log. We will unpack the raw logs into this struct.
|
||
|
|
- To use abigen: `abigen --sol flip.sol --pkg flip --out {/path/to/output_file}`
|
||
|
|
- sol: this is the path to the solidity contract
|
||
|
|
- pkg: a package name for the generated Go code
|
||
|
|
- out: the file path for the generated Go code (optional)
|
||
|
|
- the output for `flop.sol` will include the FlopperAbi and the FlopperKick struct:
|
||
|
|
```go
|
||
|
|
type FlopperKick struct {
|
||
|
|
Id *big.Int
|
||
|
|
Lot *big.Int
|
||
|
|
Bid *big.Int
|
||
|
|
Gal common.Address
|
||
|
|
End *big.Int
|
||
|
|
Raw types.Log
|
||
|
|
}
|
||
|
|
```
|
||
|
|
- Using go-ethereum's `contract.UnpackLog` method we can unpack the raw log into the FlopperKick struct (which we're referring to as the `entity`).
|
||
|
|
- See the `ToEntity` method in `pkg/transformers/flop_kick/converter`.
|
||
|
|
- The unpack method will not add the `Raw` or `TransactionIndex` values to the entity struct - both of these values are accessible from the entity.
|
||
|
|
1. Then convert the entity into a database model. See the `ToModel` method in `pkg/transformers/flop_kick/converter`.
|
||
|
|
|
||
|
|
- For LogNote Events, such as tend.
|
||
|
|
- Since LogNote events are a generic structure, they depend on the
|
||
|
|
method signature of the method that is calling them. For example,
|
||
|
|
when the `tend` method is called on the
|
||
|
|
[flip.sol contract](https://github.com/makerdao/dss/blob/master/src/flip.sol#L117), the method signature looks like this: `tend(uint id, uint lot, uint bid)`
|
||
|
|
- the LogNote event will take the first
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
1. Persist the log record to VulcanizeDB.
|
||
|
|
- Each event log has it's own table in the database, as well as it's
|
||
|
|
own column in the `checked_headers` table.
|
||
|
|
- The `checked_headers` table alllows us to keep track of which
|
||
|
|
headers have been queried for a given log type.
|
||
|
|
- To create a new migration file: `migrate create -ext sql -dir ./db/migrations/ create_flop_kick`
|
||
|
|
- See `db/migrations/1536942529_create_flop_kick.up.sql`.
|
||
|
|
- The specific log event tables are all created in the `maker`
|
||
|
|
schema.
|
||
|
|
- There is a one-many association between `headers` and the log
|
||
|
|
event tables. This is so that if a header is removed due to a
|
||
|
|
reorg, the associated log event records are also removed.
|
||
|
|
- We have been following a repository pattern to interact with the
|
||
|
|
database for each table, see the `Create` method in `pkg/transformers/flop_kick/repository.go`.
|
||
|
|
1. Get all MissingHeaders. //TODO//
|
||
|
|
- The repository is also responsible for querying for all header
|
||
|
|
records that have not yet been.
|
||
|
|
1. MarkHeaderChecked//TODO//
|
||
|
|
|
||
|
|
1. Wire each component up in the transformer.
|
||
|
|
|
||
|
|
- Each transformer's `Execute` method iterates through all of the
|
||
|
|
MissingHeaders
|
||
|
|
- Currently each transformer's `Execute` method is responsible for
|
||
|
|
fetching the missing headers, meaning the headers that haven't yet
|
||
|
|
been checked for a given log type
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
#### For LogNote events
|