cerc-io/ipld-eth-server
18
0
Fork 1
Code Issues 10 Pull Requests 1 Actions Packages Projects Releases 37 Wiki Activity

fixes for review comments

Browse Source
This commit is contained in:
Ian Norden 2019-03-13 11:14:35 -05:00
parent 37e581c7ec
commit 37fc038605
31 changed files with 634 additions and 1017 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
README.md
View File

@ -156,9 +156,17 @@ and a transformer is an interface for filtering through that raw Ethereum data t
### contractWatcher ### contractWatcher
The `contractWatcher` command is a built-in generic contract watcher. It can watch any and all events for a given contract provided the contract's ABI is available. The `contractWatcher` command is a built-in generic contract watcher. It can watch any and all events for a given contract provided the contract's ABI is available.
It also provides some state variable coverage by automating polling of public methods, with some restrictions. It also provides some state variable coverage by automating polling of public methods, with some restrictions:
1. The method must have 2 or less arguments
2. The method's arguments must all be of type address or bytes32 (hash)
3. The method must return a single value
This command requires a pre-synced (full or light) vulcanizeDB (see above sections) and currently requires the contract ABI be available on etherscan or provided by the user. This command operates in two modes- `light` and `full`- which require a light or full-synced vulcanizeDB, respectively.
This command requires the contract ABI be available on Etherscan if it is not provided in the config file by the user.
If method polling is turned on we require an archival node at the ETH ipc endpoint in our config, whether or not we are operating in `light` or `full` mode.
Otherwise, when operating in `light` mode, we only need to connect to a full node to fetch event logs.
This command takes a config of the form: This command takes a config of the form:
@ -240,10 +248,26 @@ The 'method' and 'event' identifiers are tacked onto the end of the table names
Example: Example:
Running `./vulcanizedb contractWatcher --config <path to config> --starting-block-number=5197514 --contract-address=0x8dd5fbce2f6a956c3022ba3663759011dd51e73e --events=Transfer --events=Mint --methods=balanceOf` Running `./vulcanizedb contractWatcher --config=./environments/example.toml --mode=light`
watches Transfer and Mint events of the TrueUSD contract and polls its balanceOf method using the addresses we find emitted from those events
It produces and populates a schema with three tables: Runs our contract watcher in light mode, configured to watch the contracts specified in the config file. Note that
by default we operate in `light` mode but the flag is included here to demonstrate its use.
The example config we link to in this example watches two contracts, the ENS Registry (0x314159265dD8dbb310642f98f50C066173C1259b) and TrueUSD (0x8dd5fbCe2F6a956C3022bA3663759011Dd51e73E).
Because the ENS Registry is configured with only an ABI and a starting block, we will watch all events for this contract and poll none of its methods. Note that the ENS Registry is an example
of a contract which does not have its ABI available over Etherscan and must have it included in the config file.
The TrueUSD contract is configured with two events (`Transfer` and `Mint`) and a single method (`balanceOf`), as such it will watch these two events and use any addresses it collects emitted from them
to poll the `balanceOf` method with those addresses at every block. Note that we do not provide an ABI for TrueUSD as its ABI can be fetched from Etherscan.
For the ENS contract, it produces and populates a schema with four tables"
`light_0x314159265dd8dbb310642f98f50c066173c1259b.newowner_event`
`light_0x314159265dd8dbb310642f98f50c066173c1259b.newresolver_event`
`light_0x314159265dd8dbb310642f98f50c066173c1259b.newttl_event`
`light_0x314159265dd8dbb310642f98f50c066173c1259b.transfer_event`
For the TrusUSD contract, it produces and populates a schema with three tables:
`light_0x8dd5fbce2f6a956c3022ba3663759011dd51e73e.transfer_event` `light_0x8dd5fbce2f6a956c3022ba3663759011dd51e73e.transfer_event`
`light_0x8dd5fbce2f6a956c3022ba3663759011dd51e73e.mint_event` `light_0x8dd5fbce2f6a956c3022ba3663759011dd51e73e.mint_event`
@ -276,7 +300,9 @@ Table "light_0x8dd5fbce2f6a956c3022ba3663759011dd51e73e.balanceof_method"
| who_ | character varying(66) | | not null | | extended | | | | who_ | character varying(66) | | not null | | extended | | |
| returned | numeric | | not null | | main | | | | returned | numeric | | not null | | main | | |
The addition of '_' after table names is to prevent collisions with reserved Postgres words The addition of '_' after table names is to prevent collisions with reserved Postgres words.
Also notice that the contract address used for the schema name has been down-cased.
### composeAndExecute ### composeAndExecute
The `composeAndExecute` command is used to compose and execute over an arbitrary set of custom transformers. The `composeAndExecute` command is used to compose and execute over an arbitrary set of custom transformers.
@ -359,9 +385,9 @@ The config provides information for composing a set of transformers:
that fetches state and storage diffs from an ETH node (instead of, for example, from IPFS) that fetches state and storage diffs from an ETH node (instead of, for example, from IPFS)
- `eth_event` indicates the transformer works with the [event watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/staging/libraries/shared/watcher/event_watcher.go) - `eth_event` indicates the transformer works with the [event watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/staging/libraries/shared/watcher/event_watcher.go)
that fetches event logs from an ETH node that fetches event logs from an ETH node
- `eth_generic` indicates the transformer works with the [generic watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/omni_update/libraries/shared/watcher/generic_watcher.go) - `eth_contract` indicates the transformer works with the [contract watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/omni_update/libraries/shared/watcher/generic_watcher.go)
that is made to work with [omni pkg](https://github.com/vulcanize/maker-vulcanizedb/tree/staging/pkg/omni) that is made to work with [contract_watcher pkg](https://github.com/vulcanize/maker-vulcanizedb/tree/staging/pkg/omni)
based transformers which work with either a light or full sync vDB to watch events and poll public methods based transformers which work with either a light or full sync vDB to watch events and poll public methods ([example](https://github.com/vulcanize/ens_transformers/blob/working/transformers/domain_records/transformer.go))
- `migrations` is the relative path from `repository` to the db migrations directory for the transformer - `migrations` is the relative path from `repository` to the db migrations directory for the transformer
- `rank` determines the order that migrations are ran, with lower ranked migrations running first - `rank` determines the order that migrations are ran, with lower ranked migrations running first
- this is to help isolate any potential conflicts between transformer migrations - this is to help isolate any potential conflicts between transformer migrations
@ -393,13 +419,13 @@ type exporter string
var Exporter exporter var Exporter exporter
func (e exporter) Export() []interface1.EventTransformerInitializer, []interface1.StorageTransformerInitializer, []interface1.GenericTransformerInitializer { func (e exporter) Export() []interface1.EventTransformerInitializer, []interface1.StorageTransformerInitializer, []interface1.ContractTransformerInitializer {
return []interface1.TransformerInitializer{ return []interface1.TransformerInitializer{
transformer1.TransformerInitializer, transformer1.TransformerInitializer,
transformer3.TransformerInitializer, transformer3.TransformerInitializer,
}, []interface1.StorageTransformerInitializer{ }, []interface1.StorageTransformerInitializer{
transformer4.StorageTransformerInitializer, transformer4.StorageTransformerInitializer,
}, []interface1.GenericTransformerInitializer{ }, []interface1.ContractTransformerInitializer{
transformer2.TransformerInitializer, transformer2.TransformerInitializer,
} }
} }
@ -409,12 +435,12 @@ func (e exporter) Export() []interface1.EventTransformerInitializer, []interface
To plug in an external transformer we need to: To plug in an external transformer we need to:
* Create a [package](https://github.com/vulcanize/ens_transformers/blob/working/transformers/registry/new_owner/initializer/initializer.go) * Create a [package](https://github.com/vulcanize/ens_transformers/blob/working/transformers/registry/new_owner/initializer/initializer.go)
that exports a variable `TransformerInitializer`, `StorageTransformerInitializer`, or `GenericTransformerInitializer` that are of type [TransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/event_transformer.go#L33) that exports a variable `TransformerInitializer`, `StorageTransformerInitializer`, or `ContractTransformerInitializer` that are of type [TransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/event_transformer.go#L33)
or [StorageTransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/storage_transformer.go#L31), or [StorageTransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/storage_transformer.go#L31),
or [GenericTransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/omni_update/libraries/shared/transformer/generic_transformer.go#L31), respectively or [ContractTransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/omni_update/libraries/shared/transformer/contract_transformer.go#L31), respectively
* Design the transformers to work in the context of their [event](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/event_watcher.go#L83), * Design the transformers to work in the context of their [event](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/event_watcher.go#L83),
[storage](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/storage_watcher.go#L53), [storage](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/storage_watcher.go#L53),
or [generic](https://github.com/vulcanize/maker-vulcanizedb/blob/omni_update/libraries/shared/watcher/generic_watcher.go#L68) watcher execution modes or [contract](https://github.com/vulcanize/maker-vulcanizedb/blob/omni_update/libraries/shared/watcher/contract_watcher.go#L68) watcher execution modes
* Create db migrations to run against vulcanizeDB so that we can store the transformer output * Create db migrations to run against vulcanizeDB so that we can store the transformer output
* Do not `goose fix` the transformer migrations * Do not `goose fix` the transformer migrations
* Specify migration locations for each transformer in the config with the `exporter.transformer.migrations` fields * Specify migration locations for each transformer in the config with the `exporter.transformer.migrations` fields

7
cmd/compose.go
View File

@ -62,7 +62,7 @@ var composeCmd = &cobra.Command{
rank = "0" rank = "0"
[exporter.transformer2] [exporter.transformer2]
path = "path/to/transformer2" path = "path/to/transformer2"
type = "eth_generic" type = "eth_contract"
repository = "github.com/account/repo" repository = "github.com/account/repo"
migrations = "db/migrations" migrations = "db/migrations"
rank = "0" rank = "0"
@ -91,8 +91,9 @@ from it and loaded into and executed over by the appropriate watcher.
The type of watcher that the transformer works with is specified using the The type of watcher that the transformer works with is specified using the
type variable for each transformer in the config. Currently there are watchers type variable for each transformer in the config. Currently there are watchers
of event data from an eth node (eth_event) and storage data from an eth node of event data from an eth node (eth_event) and storage data from an eth node
(eth_storage), and a more generic interface for accepting omni pkg based transformers (eth_storage), and a more generic interface for accepting contract_watcher pkg
which can perform both event watching and public method polling. based transformers which can perform both event watching and public method
polling (eth_contract).
Transformers of different types can be ran together in the same command using a Transformers of different types can be ran together in the same command using a
single config file or in separate command instances using different config files single config file or in separate command instances using different config files

19
cmd/composeAndExecute.go
View File

@ -62,7 +62,7 @@ var composeAndExecuteCmd = &cobra.Command{
rank = "0" rank = "0"
[exporter.transformer2] [exporter.transformer2]
path = "path/to/transformer2" path = "path/to/transformer2"
type = "eth_generic" type = "eth_contract"
repository = "github.com/account/repo" repository = "github.com/account/repo"
migrations = "db/migrations" migrations = "db/migrations"
rank = "2" rank = "2"
@ -91,8 +91,9 @@ from it and loaded into and executed over by the appropriate watcher.
The type of watcher that the transformer works with is specified using the The type of watcher that the transformer works with is specified using the
type variable for each transformer in the config. Currently there are watchers type variable for each transformer in the config. Currently there are watchers
of event data from an eth node (eth_event) and storage data from an eth node of event data from an eth node (eth_event) and storage data from an eth node
(eth_storage), and a more generic interface for accepting omni pkg based transformers (eth_storage), and a more generic interface for accepting contract_watcher pkg
which can perform both event watching and public method polling. based transformers which can perform both event watching and public method
polling (eth_contract).
Transformers of different types can be ran together in the same command using a Transformers of different types can be ran together in the same command using a
single config file or in separate command instances using different config files single config file or in separate command instances using different config files
@ -150,8 +151,8 @@ func composeAndExecute() {
os.Exit(1) os.Exit(1)
} }
// Use the Exporters export method to load the EventTransformerInitializer and StorageTransformerInitializer sets // Use the Exporters export method to load the EventTransformerInitializer, StorageTransformerInitializer, and ContractTransformerInitializer sets
ethEventInitializers, ethStorageInitializers, genericInitializers := exporter.Export() ethEventInitializers, ethStorageInitializers, ethContractInitializers := exporter.Export()
// Setup bc and db objects // Setup bc and db objects
blockChain := getBlockChain() blockChain := getBlockChain()
@ -175,11 +176,11 @@ func composeAndExecute() {
go watchEthStorage(&sw, &wg) go watchEthStorage(&sw, &wg)
} }
if len(genericInitializers) > 0 { if len(ethContractInitializers) > 0 {
gw := watcher.NewGenericWatcher(&db, blockChain) gw := watcher.NewContractWatcher(&db, blockChain)
gw.AddTransformers(genericInitializers) gw.AddTransformers(ethContractInitializers)
wg.Add(1) wg.Add(1)
go genericWatching(&gw, &wg) go contractWatching(&gw, &wg)
} }
wg.Wait() wg.Wait()
} }

2
cmd/contractWatcher.go
View File

@ -94,7 +94,7 @@ func contractWatcher() {
blockChain := getBlockChain() blockChain := getBlockChain()
db := utils.LoadPostgres(databaseConfig, blockChain.Node()) db := utils.LoadPostgres(databaseConfig, blockChain.Node())
var t st.GenericTransformer var t st.ContractTransformer
con := config.ContractConfig{} con := config.ContractConfig{}
con.PrepConfig() con.PrepConfig()
switch mode { switch mode {

20
cmd/execute.go
View File

@ -99,8 +99,8 @@ func execute() {
os.Exit(1) os.Exit(1)
} }
// Use the Exporters export method to load the EventTransformerInitializer and StorageTransformerInitializer sets // Use the Exporters export method to load the EventTransformerInitializer, StorageTransformerInitializer, and ContractTransformerInitializer sets
ethEventInitializers, ethStorageInitializers, genericInitializers := exporter.Export() ethEventInitializers, ethStorageInitializers, ethContractInitializers := exporter.Export()
// Setup bc and db objects // Setup bc and db objects
blockChain := getBlockChain() blockChain := getBlockChain()
@ -124,11 +124,11 @@ func execute() {
go watchEthStorage(&sw, &wg) go watchEthStorage(&sw, &wg)
} }
if len(genericInitializers) > 0 { if len(ethContractInitializers) > 0 {
gw := watcher.NewGenericWatcher(&db, blockChain) gw := watcher.NewContractWatcher(&db, blockChain)
gw.AddTransformers(genericInitializers) gw.AddTransformers(ethContractInitializers)
wg.Add(1) wg.Add(1)
go genericWatching(&gw, &wg) go contractWatching(&gw, &wg)
} }
wg.Wait() wg.Wait()
} }
@ -139,7 +139,7 @@ func init() {
} }
type Exporter interface { type Exporter interface {
Export() ([]transformer.EventTransformerInitializer, []transformer.StorageTransformerInitializer, []transformer.GenericTransformerInitializer) Export() ([]transformer.EventTransformerInitializer, []transformer.StorageTransformerInitializer, []transformer.ContractTransformerInitializer)
} }
func watchEthEvents(w *watcher.EventWatcher, wg *syn.WaitGroup) { func watchEthEvents(w *watcher.EventWatcher, wg *syn.WaitGroup) {
@ -176,10 +176,10 @@ func watchEthStorage(w *watcher.StorageWatcher, wg *syn.WaitGroup) {
} }
} }
func genericWatching(w *watcher.GenericWatcher, wg *syn.WaitGroup) { func contractWatching(w *watcher.ContractWatcher, wg *syn.WaitGroup) {
defer wg.Done() defer wg.Done()
// Execute over the GenericTransformerInitializer set using the generic watcher // Execute over the ContractTransformerInitializer set using the contract watcher
log.Info("executing generic transformers") log.Info("executing contract_watcher transformers")
ticker := time.NewTicker(pollingInterval) ticker := time.NewTicker(pollingInterval)
defer ticker.Stop() defer ticker.Stop()
for range ticker.C { for range ticker.C {

10
environments/example.toml
View File

File diff suppressed because one or more lines are too long

47
integration_test/contract_watcher_full_transformer_test.go
View File

@ -42,7 +42,7 @@ var _ = Describe("contractWatcher full transformer", func() {
It("Initializes transformer's contract objects", func() { It("Initializes transformer's contract objects", func() {
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock1) blockRepository.CreateOrUpdateBlock(mocks.TransferBlock1)
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock2) blockRepository.CreateOrUpdateBlock(mocks.TransferBlock2)
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) t := transformer.NewTransformer(test_helpers.TusdConfig, blockChain, db)
err = t.Init() err = t.Init()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
@ -55,6 +55,28 @@ var _ = Describe("contractWatcher full transformer", func() {
Expect(c.Name).To(Equal("TrueUSD")) Expect(c.Name).To(Equal("TrueUSD"))
Expect(c.Address).To(Equal(tusdAddr)) Expect(c.Address).To(Equal(tusdAddr))
}) })
It("Fails to initialize if first and most recent blocks cannot be fetched from vDB", func() {
t := transformer.NewTransformer(test_helpers.TusdConfig, blockChain, db)
err = t.Init()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
})
It("Does nothing if watched events are unset", func() {
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock1)
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock2)
var testConf config.ContractConfig
testConf = test_helpers.TusdConfig
testConf.Events = nil
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no filters created"))
_, ok := t.Contracts[tusdAddr]
Expect(ok).To(Equal(false))
})
}) })
Describe("Execute", func() { Describe("Execute", func() {
@ -64,7 +86,7 @@ var _ = Describe("contractWatcher full transformer", func() {
}) })
It("Transforms watched contract data into custom repositories", func() { It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) t := transformer.NewTransformer(test_helpers.TusdConfig, blockChain, db)
err = t.Init() err = t.Init()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
@ -84,7 +106,7 @@ var _ = Describe("contractWatcher full transformer", func() {
It("Keeps track of contract-related addresses while transforming event data if they need to be used for later method polling", func() { It("Keeps track of contract-related addresses while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.TusdConfig testConf = test_helpers.TusdConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
tusdAddr: {"balanceOf"}, tusdAddr: {"balanceOf"},
} }
@ -121,7 +143,7 @@ var _ = Describe("contractWatcher full transformer", func() {
It("Polls given methods using generated token holder address", func() { It("Polls given methods using generated token holder address", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.TusdConfig testConf = test_helpers.TusdConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
tusdAddr: {"balanceOf"}, tusdAddr: {"balanceOf"},
} }
@ -146,13 +168,15 @@ var _ = Describe("contractWatcher full transformer", func() {
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.balanceof_method WHERE who_ = '0xfE9e8709d3215310075d67E3ed32A380CCf451C8' AND block = '6194634'", tusdAddr)).StructScan(&res) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.balanceof_method WHERE who_ = '0xfE9e8709d3215310075d67E3ed32A380CCf451C8' AND block = '6194634'", tusdAddr)).StructScan(&res)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
}) })
It("Fails if initialization has not been done", func() { It("Fails if initialization has not been done", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) t := transformer.NewTransformer(test_helpers.TusdConfig, blockChain, db)
err = t.Execute() err = t.Execute()
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("transformer has no initialized contracts to work with"))
}) })
}) })
@ -163,7 +187,7 @@ var _ = Describe("contractWatcher full transformer", func() {
}) })
It("Transforms watched contract data into custom repositories", func() { It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.ENSConfig, blockChain, db) t := transformer.NewTransformer(test_helpers.ENSConfig, blockChain, db)
err = t.Init() err = t.Init()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
@ -184,7 +208,7 @@ var _ = Describe("contractWatcher full transformer", func() {
It("Keeps track of contract-related hashes while transforming event data if they need to be used for later method polling", func() { It("Keeps track of contract-related hashes while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSConfig testConf = test_helpers.ENSConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
ensAddr: {"owner"}, ensAddr: {"owner"},
} }
@ -214,7 +238,7 @@ var _ = Describe("contractWatcher full transformer", func() {
It("Polls given methods using generated token holder address", func() { It("Polls given methods using generated token holder address", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSConfig testConf = test_helpers.ENSConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
ensAddr: {"owner"}, ensAddr: {"owner"},
} }
@ -238,11 +262,12 @@ var _ = Describe("contractWatcher full transformer", func() {
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.owner_method WHERE node_ = '0x9THIS110dcc444fIS242510c09bbAbe21aFAKEcacNODE82f7b843HASH61ba391' AND block = '6194636'", ensAddr)).StructScan(&res) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.owner_method WHERE node_ = '0x9THIS110dcc444fIS242510c09bbAbe21aFAKEcacNODE82f7b843HASH61ba391' AND block = '6194636'", ensAddr)).StructScan(&res)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
}) })
It("It does not perist events if they do not pass the emitted arg filter", func() { It("It does not perist events if they do not pass the emitted arg filter", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSConfig testConf = test_helpers.ENSConfig
testConf.EventArgs = map[string][]string{ testConf.EventArgs = map[string][]string{
ensAddr: {"fake_filter_value"}, ensAddr: {"fake_filter_value"},
} }
@ -257,11 +282,12 @@ var _ = Describe("contractWatcher full transformer", func() {
log := test_helpers.LightNewOwnerLog{} log := test_helpers.LightNewOwnerLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.newowner_event", ensAddr)).StructScan(&log) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.newowner_event", ensAddr)).StructScan(&log)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("does not exist"))
}) })
It("If a method arg filter is applied, only those arguments are used in polling", func() { It("If a method arg filter is applied, only those arguments are used in polling", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSConfig testConf = test_helpers.ENSConfig
testConf.MethodArgs = map[string][]string{ testConf.MethodArgs = map[string][]string{
ensAddr: {"0x0000000000000000000000000000000000000000000000000000c02aaa39b223"}, ensAddr: {"0x0000000000000000000000000000000000000000000000000000c02aaa39b223"},
} }
@ -283,6 +309,7 @@ var _ = Describe("contractWatcher full transformer", func() {
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.owner_method WHERE node_ = '0x9dd48110dcc444fdc242510c09bbbbe21a5975cac061d82f7b843bce061ba391' AND block = '6194636'", ensAddr)).StructScan(&res) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.owner_method WHERE node_ = '0x9dd48110dcc444fdc242510c09bbbbe21a5975cac061d82f7b843bce061ba391' AND block = '6194636'", ensAddr)).StructScan(&res)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
}) })
}) })
}) })

56
integration_test/contract_watcher_light_transformer_test.go
View File

@ -2,13 +2,13 @@ package integration
import ( import (
"fmt" "fmt"
"github.com/vulcanize/vulcanizedb/pkg/config"
"strings" "strings"
"github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common"
. "github.com/onsi/ginkgo" . "github.com/onsi/ginkgo"
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
"github.com/vulcanize/vulcanizedb/pkg/config"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/light/transformer" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/light/transformer"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/constants" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/constants"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/helpers/test_helpers" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/helpers/test_helpers"
@ -40,7 +40,7 @@ var _ = Describe("contractWatcher light transformer", func() {
It("Initializes transformer's contract objects", func() { It("Initializes transformer's contract objects", func() {
headerRepository.CreateOrUpdateHeader(mocks.MockHeader1) headerRepository.CreateOrUpdateHeader(mocks.MockHeader1)
headerRepository.CreateOrUpdateHeader(mocks.MockHeader3) headerRepository.CreateOrUpdateHeader(mocks.MockHeader3)
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) t := transformer.NewTransformer(test_helpers.TusdConfig, blockChain, db)
err = t.Init() err = t.Init()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
@ -53,6 +53,27 @@ var _ = Describe("contractWatcher light transformer", func() {
Expect(c.Name).To(Equal("TrueUSD")) Expect(c.Name).To(Equal("TrueUSD"))
Expect(c.Address).To(Equal(tusdAddr)) Expect(c.Address).To(Equal(tusdAddr))
}) })
It("Fails to initialize if first and block cannot be fetched from vDB headers table", func() {
t := transformer.NewTransformer(test_helpers.TusdConfig, blockChain, db)
err = t.Init()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
})
It("Does nothing if nothing if no addresses are configured", func() {
headerRepository.CreateOrUpdateHeader(mocks.MockHeader1)
headerRepository.CreateOrUpdateHeader(mocks.MockHeader3)
var testConf config.ContractConfig
testConf = test_helpers.TusdConfig
testConf.Addresses = nil
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
_, ok := t.Contracts[tusdAddr]
Expect(ok).To(Equal(false))
})
}) })
Describe("Execute- against TrueUSD contract", func() { Describe("Execute- against TrueUSD contract", func() {
@ -70,7 +91,7 @@ var _ = Describe("contractWatcher light transformer", func() {
}) })
It("Transforms watched contract data into custom repositories", func() { It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) t := transformer.NewTransformer(test_helpers.TusdConfig, blockChain, db)
err = t.Init() err = t.Init()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
err = t.Execute() err = t.Execute()
@ -88,7 +109,7 @@ var _ = Describe("contractWatcher light transformer", func() {
It("Keeps track of contract-related addresses while transforming event data if they need to be used for later method polling", func() { It("Keeps track of contract-related addresses while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.TusdConfig testConf = test_helpers.TusdConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
tusdAddr: {"balanceOf"}, tusdAddr: {"balanceOf"},
} }
@ -133,7 +154,7 @@ var _ = Describe("contractWatcher light transformer", func() {
It("Polls given methods using generated token holder address", func() { It("Polls given methods using generated token holder address", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.TusdConfig testConf = test_helpers.TusdConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
tusdAddr: {"balanceOf"}, tusdAddr: {"balanceOf"},
} }
@ -151,12 +172,14 @@ var _ = Describe("contractWatcher light transformer", func() {
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x09BbBBE21a5975cAc061D82f7b843b1234567890' AND block = '6791669'", tusdAddr)).StructScan(&res) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x09BbBBE21a5975cAc061D82f7b843b1234567890' AND block = '6791669'", tusdAddr)).StructScan(&res)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
}) })
It("Fails if initialization has not been done", func() { It("Fails if initialization has not been done", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) t := transformer.NewTransformer(test_helpers.TusdConfig, blockChain, db)
err = t.Execute() err = t.Execute()
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("transformer has no initialized contracts"))
}) })
}) })
@ -175,7 +198,7 @@ var _ = Describe("contractWatcher light transformer", func() {
}) })
It("Transforms watched contract data into custom repositories", func() { It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.ENSConfig, blockChain, db) t := transformer.NewTransformer(test_helpers.ENSConfig, blockChain, db)
err = t.Init() err = t.Init()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
err = t.Execute() err = t.Execute()
@ -193,7 +216,7 @@ var _ = Describe("contractWatcher light transformer", func() {
It("Keeps track of contract-related hashes while transforming event data if they need to be used for later method polling", func() { It("Keeps track of contract-related hashes while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSConfig testConf = test_helpers.ENSConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
ensAddr: {"owner"}, ensAddr: {"owner"},
} }
@ -222,7 +245,7 @@ var _ = Describe("contractWatcher light transformer", func() {
It("Polls given method using list of collected hashes", func() { It("Polls given method using list of collected hashes", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSConfig testConf = test_helpers.ENSConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
ensAddr: {"owner"}, ensAddr: {"owner"},
} }
@ -245,11 +268,12 @@ var _ = Describe("contractWatcher light transformer", func() {
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x9THIS110dcc444fIS242510c09bbAbe21aFAKEcacNODE82f7b843HASH61ba391' AND block = '6885696'", ensAddr)).StructScan(&res) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x9THIS110dcc444fIS242510c09bbAbe21aFAKEcacNODE82f7b843HASH61ba391' AND block = '6885696'", ensAddr)).StructScan(&res)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
}) })
It("It does not persist events if they do not pass the emitted arg filter", func() { It("It does not persist events if they do not pass the emitted arg filter", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSConfig testConf = test_helpers.ENSConfig
testConf.EventArgs = map[string][]string{ testConf.EventArgs = map[string][]string{
ensAddr: {"fake_filter_value"}, ensAddr: {"fake_filter_value"},
} }
@ -262,11 +286,12 @@ var _ = Describe("contractWatcher light transformer", func() {
log := test_helpers.LightNewOwnerLog{} log := test_helpers.LightNewOwnerLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.newowner_event", ensAddr)).StructScan(&log) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.newowner_event", ensAddr)).StructScan(&log)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("does not exist"))
}) })
It("If a method arg filter is applied, only those arguments are used in polling", func() { It("If a method arg filter is applied, only those arguments are used in polling", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSConfig testConf = test_helpers.ENSConfig
testConf.MethodArgs = map[string][]string{ testConf.MethodArgs = map[string][]string{
ensAddr: {"0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae"}, ensAddr: {"0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae"},
} }
@ -287,6 +312,7 @@ var _ = Describe("contractWatcher light transformer", func() {
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x95832c7a47ff8a7840e28b78ce695797aaf402b1c186bad9eca28842625b5047' AND block = '6885696'", ensAddr)).StructScan(&res) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x95832c7a47ff8a7840e28b78ce695797aaf402b1c186bad9eca28842625b5047' AND block = '6885696'", ensAddr)).StructScan(&res)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
}) })
}) })
@ -301,7 +327,7 @@ var _ = Describe("contractWatcher light transformer", func() {
}) })
It("Transforms watched contract data into custom repositories", func() { It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.ENSandTusdConfig, blockChain, db) t := transformer.NewTransformer(test_helpers.ENSandTusdConfig, blockChain, db)
err = t.Init() err = t.Init()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
err = t.Execute() err = t.Execute()
@ -326,7 +352,7 @@ var _ = Describe("contractWatcher light transformer", func() {
It("Keeps track of contract-related hashes and addresses while transforming event data if they need to be used for later method polling", func() { It("Keeps track of contract-related hashes and addresses while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSandTusdConfig testConf = test_helpers.ENSandTusdConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
ensAddr: {"owner"}, ensAddr: {"owner"},
tusdAddr: {"balanceOf"}, tusdAddr: {"balanceOf"},
@ -367,7 +393,7 @@ var _ = Describe("contractWatcher light transformer", func() {
It("Polls given methods for each contract, using list of collected values", func() { It("Polls given methods for each contract, using list of collected values", func() {
var testConf config.ContractConfig var testConf config.ContractConfig
testConf = mocks.ENSandTusdConfig testConf = test_helpers.ENSandTusdConfig
testConf.Methods = map[string][]string{ testConf.Methods = map[string][]string{
ensAddr: {"owner"}, ensAddr: {"owner"},
tusdAddr: {"balanceOf"}, tusdAddr: {"balanceOf"},
@ -391,6 +417,7 @@ var _ = Describe("contractWatcher light transformer", func() {
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x95832c7a47ff8a7840e28b78ceMADEUPaaf4HASHc186badTHItransformers.8IS625bFAKE' AND block = '6885696'", ensAddr)).StructScan(&owner) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x95832c7a47ff8a7840e28b78ceMADEUPaaf4HASHc186badTHItransformers.8IS625bFAKE' AND block = '6885696'", ensAddr)).StructScan(&owner)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
bal := test_helpers.BalanceOf{} bal := test_helpers.BalanceOf{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x8cA465764873E71CEa525F5EB6AE973d650c22C2' AND block = '6885701'", tusdAddr)).StructScan(&bal) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x8cA465764873E71CEa525F5EB6AE973d650c22C2' AND block = '6885701'", tusdAddr)).StructScan(&bal)
@ -400,6 +427,7 @@ var _ = Describe("contractWatcher light transformer", func() {
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x09BbBBE21a5975cAc061D82f7b843b1234567890' AND block = '6885701'", tusdAddr)).StructScan(&bal) err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x09BbBBE21a5975cAc061D82f7b843b1234567890' AND block = '6885701'", tusdAddr)).StructScan(&bal)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no rows in result set"))
}) })
}) })
}) })

4
libraries/shared/transformer/generic_transformer.go → libraries/shared/transformer/contract_transformer.go
View File

@ -22,10 +22,10 @@ import (
"github.com/vulcanize/vulcanizedb/pkg/datastore/postgres" "github.com/vulcanize/vulcanizedb/pkg/datastore/postgres"
) )
type GenericTransformer interface { type ContractTransformer interface {
Init() error Init() error
Execute() error Execute() error
GetConfig() config.ContractConfig GetConfig() config.ContractConfig
} }
type GenericTransformerInitializer func(db *postgres.DB, bc core.BlockChain) GenericTransformer type ContractTransformerInitializer func(db *postgres.DB, bc core.BlockChain) ContractTransformer

24
libraries/shared/watcher/generic_watcher.go → libraries/shared/watcher/contract_watcher.go
View File

@ -14,10 +14,10 @@
// You should have received a copy of the GNU Affero General Public License // You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>. // along with this program. If not, see <http://www.gnu.org/licenses/>.
// Dynamic watcher is built with a more generic interface // Contract watcher is built with a more generic interface
// that allows offloading more of the operatinal logic to // that allows offloading more of the operational logic to
// the transformers, allowing them to act more dynamically // the transformers, allowing them to act more dynamically
// Built to work primarily with the omni pkging // Built to work primarily with the contract_watcher packaging
package watcher package watcher
import ( import (
@ -30,26 +30,26 @@ import (
"github.com/vulcanize/vulcanizedb/pkg/datastore/postgres" "github.com/vulcanize/vulcanizedb/pkg/datastore/postgres"
) )
type GenericWatcher struct { type ContractWatcher struct {
Transformers []transformer.GenericTransformer Transformers []transformer.ContractTransformer
DB *postgres.DB DB *postgres.DB
BlockChain core.BlockChain BlockChain core.BlockChain
} }
func NewGenericWatcher(db *postgres.DB, bc core.BlockChain) GenericWatcher { func NewContractWatcher(db *postgres.DB, bc core.BlockChain) ContractWatcher {
return GenericWatcher{ return ContractWatcher{
DB: db, DB: db,
BlockChain: bc, BlockChain: bc,
} }
} }
func (watcher *GenericWatcher) AddTransformers(inits interface{}) error { func (watcher *ContractWatcher) AddTransformers(inits interface{}) error {
initializers, ok := inits.([]transformer.GenericTransformerInitializer) initializers, ok := inits.([]transformer.ContractTransformerInitializer)
if !ok { if !ok {
return fmt.Errorf("initializers of type %T, not %T", inits, []transformer.GenericTransformerInitializer{}) return fmt.Errorf("initializers of type %T, not %T", inits, []transformer.ContractTransformerInitializer{})
} }
watcher.Transformers = make([]transformer.GenericTransformer, 0, len(initializers)) watcher.Transformers = make([]transformer.ContractTransformer, 0, len(initializers))
for _, initializer := range initializers { for _, initializer := range initializers {
t := initializer(watcher.DB, watcher.BlockChain) t := initializer(watcher.DB, watcher.BlockChain)
watcher.Transformers = append(watcher.Transformers, t) watcher.Transformers = append(watcher.Transformers, t)
@ -65,7 +65,7 @@ func (watcher *GenericWatcher) AddTransformers(inits interface{}) error {
return nil return nil
} }
func (watcher *GenericWatcher) Execute(interface{}) error { func (watcher *ContractWatcher) Execute(interface{}) error {
for _, transformer := range watcher.Transformers { for _, transformer := range watcher.Transformers {
err := transformer.Execute() err := transformer.Execute()
if err != nil { if err != nil {

155
pkg/config/contract.go
View File

@ -71,7 +71,7 @@ func (oc *ContractConfig) PrepConfig() {
oc.Addresses = make(map[string]bool, len(addrs)) oc.Addresses = make(map[string]bool, len(addrs))
oc.Abis = make(map[string]string, len(addrs)) oc.Abis = make(map[string]string, len(addrs))
oc.Methods = make(map[string][]string, len(addrs)) oc.Methods = make(map[string][]string, len(addrs))
oc.EventArgs = make(map[string][]string, len(addrs)) oc.Events = make(map[string][]string, len(addrs))
oc.MethodArgs = make(map[string][]string, len(addrs)) oc.MethodArgs = make(map[string][]string, len(addrs))
oc.EventArgs = make(map[string][]string, len(addrs)) oc.EventArgs = make(map[string][]string, len(addrs))
oc.StartingBlocks = make(map[string]int64, len(addrs)) oc.StartingBlocks = make(map[string]int64, len(addrs))
@ -86,92 +86,131 @@ func (oc *ContractConfig) PrepConfig() {
transformer := viper.GetStringMap("contract." + addr) transformer := viper.GetStringMap("contract." + addr)
// Get and check abi // Get and check abi
abi, abiOK := transformer["abi"] var abi string
if !abiOK || abi == nil { _, abiOK := transformer["abi"]
log.Fatal(addr, "transformer config is missing `abi` value")
}
abiRef, abiOK := abi.(string)
if !abiOK { if !abiOK {
log.Fatal(addr, "transformer `events` not of type []string") log.Warnf("contract %s not configured with an ABI, will attempt to fetch it from Etherscan\r\n", addr)
} else {
abiInterface := transformer["abi"]
abi, abiOK = abiInterface.(string)
if !abiOK {
log.Fatal(addr, "transformer `abi` not of type []string")
} }
oc.Abis[strings.ToLower(addr)] = abiRef }
oc.Abis[strings.ToLower(addr)] = abi
// Get and check events // Get and check events
events, eventsOK := transformer["events"] events := make([]string, 0)
if !eventsOK || events == nil { _, eventsOK := transformer["events"]
log.Fatal(addr, "transformer config is missing `events` value")
}
eventsRef, eventsOK := events.([]string)
if !eventsOK { if !eventsOK {
log.Fatal(addr, "transformer `events` not of type []string") log.Warnf("contract %s not configured with a list of events to watch, will watch all events\r\n", addr)
events = []string{}
} else {
eventsInterface := transformer["events"]
eventsI, eventsOK := eventsInterface.([]interface{})
if !eventsOK {
log.Fatal(addr, "transformer `events` not of type []string\r\n")
} }
if eventsRef == nil { for _, strI := range eventsI {
eventsRef = []string{} str, strOK := strI.(string)
if !strOK {
log.Fatal(addr, "transformer `events` not of type []string\r\n")
} }
oc.Events[strings.ToLower(addr)] = eventsRef events = append(events, str)
}
}
oc.Events[strings.ToLower(addr)] = events
// Get and check methods // Get and check methods
methods, methodsOK := transformer["methods"] methods := make([]string, 0)
if !methodsOK || methods == nil { _, methodsOK := transformer["methods"]
log.Fatal(addr, "transformer config is missing `methods` value")
}
methodsRef, methodsOK := methods.([]string)
if !methodsOK { if !methodsOK {
log.Fatal(addr, "transformer `methods` not of type []string") log.Warnf("contract %s not configured with a list of methods to poll, will not poll any methods\r\n", addr)
methods = []string{}
} else {
methodsInterface := transformer["methods"]
methodsI, methodsOK := methodsInterface.([]interface{})
if !methodsOK {
log.Fatal(addr, "transformer `methods` not of type []string\r\n")
} }
if methodsRef == nil { for _, strI := range methodsI {
methodsRef = []string{} str, strOK := strI.(string)
if !strOK {
log.Fatal(addr, "transformer `methods` not of type []string\r\n")
} }
oc.Methods[strings.ToLower(addr)] = methodsRef methods = append(methods, str)
}
}
oc.Methods[strings.ToLower(addr)] = methods
// Get and check eventArgs // Get and check eventArgs
eventArgs, eventArgsOK := transformer["eventArgs"] eventArgs := make([]string, 0)
if !eventArgsOK || eventArgs == nil { _, eventArgsOK := transformer["eventArgs"]
log.Fatal(addr, "transformer config is missing `eventArgs` value")
}
eventArgsRef, eventArgsOK := eventArgs.([]string)
if !eventArgsOK { if !eventArgsOK {
log.Fatal(addr, "transformer `eventArgs` not of type []string") log.Warnf("contract %s not configured with a list of event arguments to filter for, will not filter events for specific emitted values\r\n", addr)
eventArgs = []string{}
} else {
eventArgsInterface := transformer["eventArgs"]
eventArgsI, eventArgsOK := eventArgsInterface.([]interface{})
if !eventArgsOK {
log.Fatal(addr, "transformer `eventArgs` not of type []string\r\n")
} }
if eventArgsRef == nil { for _, strI := range eventArgsI {
eventArgsRef = []string{} str, strOK := strI.(string)
if !strOK {
log.Fatal(addr, "transformer `eventArgs` not of type []string\r\n")
} }
oc.EventArgs[strings.ToLower(addr)] = eventArgsRef eventArgs = append(eventArgs, str)
}
}
oc.EventArgs[strings.ToLower(addr)] = eventArgs
// Get and check methodArgs // Get and check methodArgs
methodArgs, methodArgsOK := transformer["methodArgs"] methodArgs := make([]string, 0)
if !methodArgsOK || methodArgs == nil { _, methodArgsOK := transformer["methodArgs"]
log.Fatal(addr, "transformer config is missing `methodArgs` value")
}
methodArgsRef, methodArgsOK := methodArgs.([]string)
if !methodArgsOK { if !methodArgsOK {
log.Fatal(addr, "transformer `methodArgs` not of type []string") log.Warnf("contract %s not configured with a list of method argument values to poll with, will poll methods with all available arguments\r\n", addr)
methodArgs = []string{}
} else {
methodArgsInterface := transformer["methodArgs"]
methodArgsI, methodArgsOK := methodArgsInterface.([]interface{})
if !methodArgsOK {
log.Fatal(addr, "transformer `methodArgs` not of type []string\r\n")
} }
if methodArgsRef == nil { for _, strI := range methodArgsI {
methodArgsRef = []string{} str, strOK := strI.(string)
if !strOK {
log.Fatal(addr, "transformer `methodArgs` not of type []string\r\n")
} }
oc.MethodArgs[strings.ToLower(addr)] = methodArgsRef methodArgs = append(methodArgs, str)
}
}
oc.MethodArgs[strings.ToLower(addr)] = methodArgs
// Get and check startingBlock // Get and check startingBlock
start, startOK := transformer["startingBlock"] startInterface, startOK := transformer["startingblock"]
if !startOK || start == nil {
log.Fatal(addr, "transformer config is missing `startingBlock` value")
}
startRef, startOK := start.(int64)
if !startOK { if !startOK {
log.Fatal(addr, "transformer `startingBlock` not of type int") log.Fatal(addr, "transformer config is missing `startingBlock` value\r\n")
} }
oc.StartingBlocks[strings.ToLower(addr)] = startRef start, startOK := startInterface.(int64)
if !startOK {
log.Fatal(addr, "transformer `startingBlock` not of type int\r\n")
}
oc.StartingBlocks[strings.ToLower(addr)] = start
// Get pipping // Get pipping
pipe, pipeOK := transformer["pipping"] var piping bool
if !pipeOK || pipe == nil { _, pipeOK := transformer["piping"]
log.Fatal(addr, "transformer config is missing `pipping` value")
}
pipeRef, pipeOK := pipe.(bool)
if !pipeOK { if !pipeOK {
log.Fatal(addr, "transformer `piping` not of type bool") log.Warnf("contract %s does not have its `piping` set, by default piping is turned off\r\n", addr)
piping = false
} else {
pipingInterface := transformer["piping"]
piping, pipeOK = pipingInterface.(bool)
if !pipeOK {
log.Fatal(addr, "transformer `piping` not of type bool\r\n")
} }
oc.Piping[strings.ToLower(addr)] = pipeRef }
oc.Piping[strings.ToLower(addr)] = piping
} }
} }

28
pkg/config/plugin.go
View File

@ -41,13 +41,13 @@ type Transformer struct {
RepositoryPath string RepositoryPath string
} }
func (c *Plugin) GetPluginPaths() (string, string, error) { func (pluginConfig *Plugin) GetPluginPaths() (string, string, error) {
path, err := helpers.CleanPath(c.FilePath) path, err := helpers.CleanPath(pluginConfig.FilePath)
if err != nil { if err != nil {
return "", "", err return "", "", err
} }
name := strings.Split(c.FileName, ".")[0] name := strings.Split(pluginConfig.FileName, ".")[0]
goFile := filepath.Join(path, name+".go") goFile := filepath.Join(path, name+".go")
soFile := filepath.Join(path, name+".so") soFile := filepath.Join(path, name+".so")
@ -55,13 +55,13 @@ func (c *Plugin) GetPluginPaths() (string, string, error) {
} }
// Removes duplicate migration paths and returns them in ranked order // Removes duplicate migration paths and returns them in ranked order
func (c *Plugin) GetMigrationsPaths() ([]string, error) { func (pluginConfig *Plugin) GetMigrationsPaths() ([]string, error) {
paths := make(map[uint64]string) paths := make(map[uint64]string)
highestRank := -1 highestRank := -1
for name, transformer := range c.Transformers { for name, transformer := range pluginConfig.Transformers {
repo := transformer.RepositoryPath repo := transformer.RepositoryPath
mig := transformer.MigrationPath mig := transformer.MigrationPath
path := filepath.Join("$GOPATH/src", c.Home, "vendor", repo, mig) path := filepath.Join("$GOPATH/src", pluginConfig.Home, "vendor", repo, mig)
cleanPath, err := helpers.CleanPath(path) cleanPath, err := helpers.CleanPath(path)
if err != nil { if err != nil {
return nil, err return nil, err
@ -96,9 +96,9 @@ func (c *Plugin) GetMigrationsPaths() ([]string, error) {
} }
// Removes duplicate repo paths before returning them // Removes duplicate repo paths before returning them
func (c *Plugin) GetRepoPaths() map[string]bool { func (pluginConfig *Plugin) GetRepoPaths() map[string]bool {
paths := make(map[string]bool) paths := make(map[string]bool)
for _, transformer := range c.Transformers { for _, transformer := range pluginConfig.Transformers {
paths[transformer.RepositoryPath] = true paths[transformer.RepositoryPath] = true
} }
@ -111,29 +111,29 @@ const (
UnknownTransformerType TransformerType = iota UnknownTransformerType TransformerType = iota
EthEvent EthEvent
EthStorage EthStorage
EthGeneric EthContract
) )
func (pt TransformerType) String() string { func (transformerType TransformerType) String() string {
names := [...]string{ names := [...]string{
"Unknown", "Unknown",
"eth_event", "eth_event",
"eth_storage", "eth_storage",
"eth_generic", "eth_contract",
} }
if pt > EthGeneric || pt < EthEvent { if transformerType > EthContract || transformerType < EthEvent {
return "Unknown" return "Unknown"
} }
return names[pt] return names[transformerType]
} }
func GetTransformerType(str string) TransformerType { func GetTransformerType(str string) TransformerType {
types := [...]TransformerType{ types := [...]TransformerType{
EthEvent, EthEvent,
EthStorage, EthStorage,
EthGeneric, EthContract,
} }
for _, ty := range types { for _, ty := range types {

14
pkg/contract_watcher/full/converter/converter.go
View File

@ -34,27 +34,21 @@ import (
// Converter is used to convert watched event logs to // Converter is used to convert watched event logs to
// custom logs containing event input name => value maps // custom logs containing event input name => value maps
type Converter interface { type ConverterInterface interface {
Convert(watchedEvent core.WatchedEvent, event types.Event) (*types.Log, error) Convert(watchedEvent core.WatchedEvent, event types.Event) (*types.Log, error)
Update(info *contract.Contract) Update(info *contract.Contract)
} }
type converter struct { type Converter struct {
ContractInfo *contract.Contract ContractInfo *contract.Contract
} }
func NewConverter(info *contract.Contract) *converter { func (c *Converter) Update(info *contract.Contract) {
return &converter{
ContractInfo: info,
}
}
func (c *converter) Update(info *contract.Contract) {
c.ContractInfo = info c.ContractInfo = info
} }
// Convert the given watched event log into a types.Log for the given event // Convert the given watched event log into a types.Log for the given event
func (c *converter) Convert(watchedEvent core.WatchedEvent, event types.Event) (*types.Log, error) { func (c *Converter) Convert(watchedEvent core.WatchedEvent, event types.Event) (*types.Log, error) {
contract := bind.NewBoundContract(common.HexToAddress(c.ContractInfo.Address), c.ContractInfo.ParsedAbi, nil, nil, nil) contract := bind.NewBoundContract(common.HexToAddress(c.ContractInfo.Address), c.ContractInfo.ParsedAbi, nil, nil, nil)
values := make(map[string]interface{}) values := make(map[string]interface{})
log := helpers.ConvertToLog(watchedEvent) log := helpers.ConvertToLog(watchedEvent)

12
pkg/contract_watcher/full/converter/converter_test.go
View File

@ -39,7 +39,8 @@ var _ = Describe("Converter", func() {
Describe("Update", func() { Describe("Update", func() {
It("Updates contract con held by the converter", func() { It("Updates contract con held by the converter", func() {
c := converter.NewConverter(con) c := converter.Converter{}
c.Update(con)
Expect(c.ContractInfo).To(Equal(con)) Expect(c.ContractInfo).To(Equal(con))
con := test_helpers.SetupTusdContract([]string{}, []string{}) con := test_helpers.SetupTusdContract([]string{}, []string{})
@ -58,7 +59,8 @@ var _ = Describe("Converter", func() {
err = con.GenerateFilters() err = con.GenerateFilters()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
c := converter.NewConverter(con) c := converter.Converter{}
c.Update(con)
log, err := c.Convert(mocks.MockTranferEvent, event) log, err := c.Convert(mocks.MockTranferEvent, event)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
@ -77,7 +79,8 @@ var _ = Describe("Converter", func() {
event, ok := con.Events["Transfer"] event, ok := con.Events["Transfer"]
Expect(ok).To(Equal(true)) Expect(ok).To(Equal(true))
c := converter.NewConverter(con) c := converter.Converter{}
c.Update(con)
_, err := c.Convert(mocks.MockTranferEvent, event) _, err := c.Convert(mocks.MockTranferEvent, event)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
@ -101,7 +104,8 @@ var _ = Describe("Converter", func() {
It("Fails with an empty contract", func() { It("Fails with an empty contract", func() {
event := con.Events["Transfer"] event := con.Events["Transfer"]
c := converter.NewConverter(&contract.Contract{}) c := converter.Converter{}
c.Update(&contract.Contract{})
_, err = c.Convert(mocks.MockTranferEvent, event) _, err = c.Convert(mocks.MockTranferEvent, event)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
}) })

20
pkg/contract_watcher/full/transformer/transformer.go
View File

@ -34,7 +34,7 @@ import (
) )
// Requires a fully synced vDB and a running eth node (or infura) // Requires a fully synced vDB and a running eth node (or infura)
type transformer struct { type Transformer struct {
// Database interfaces // Database interfaces
datastore.FilterRepository // Log filters repo; accepts filters generated by Contract.GenerateFilters() datastore.FilterRepository // Log filters repo; accepts filters generated by Contract.GenerateFilters()
datastore.WatchedEventRepository // Watched event log views, created by the log filters datastore.WatchedEventRepository // Watched event log views, created by the log filters
@ -45,7 +45,7 @@ type transformer struct {
retriever.BlockRetriever // Retrieves first block for contract and current block height retriever.BlockRetriever // Retrieves first block for contract and current block height
// Processing interfaces // Processing interfaces
converter.Converter // Converts watched event logs into custom log converter.ConverterInterface // Converts watched event logs into custom log
poller.Poller // Polls methods using contract's token holder addresses and persists them using method datastore poller.Poller // Polls methods using contract's token holder addresses and persists them using method datastore
// Store contract configuration information // Store contract configuration information
@ -56,12 +56,12 @@ type transformer struct {
} }
// Transformer takes in config for blockchain, database, and network id // Transformer takes in config for blockchain, database, and network id
func NewTransformer(con config.ContractConfig, BC core.BlockChain, DB *postgres.DB) *transformer { func NewTransformer(con config.ContractConfig, BC core.BlockChain, DB *postgres.DB) *Transformer {
return &transformer{ return &Transformer{
Poller: poller.NewPoller(BC, DB, types.FullSync), Poller: poller.NewPoller(BC, DB, types.FullSync),
Parser: parser.NewParser(con.Network), Parser: parser.NewParser(con.Network),
BlockRetriever: retriever.NewBlockRetriever(DB), BlockRetriever: retriever.NewBlockRetriever(DB),
Converter: converter.NewConverter(&contract.Contract{}), ConverterInterface: &converter.Converter{},
Contracts: map[string]*contract.Contract{}, Contracts: map[string]*contract.Contract{},
WatchedEventRepository: repositories.WatchedEventRepository{DB: DB}, WatchedEventRepository: repositories.WatchedEventRepository{DB: DB},
FilterRepository: repositories.FilterRepository{DB: DB}, FilterRepository: repositories.FilterRepository{DB: DB},
@ -74,7 +74,7 @@ func NewTransformer(con config.ContractConfig, BC core.BlockChain, DB *postgres.
// Loops over all of the addr => filter sets // Loops over all of the addr => filter sets
// Uses parser to pull event info from abi // Uses parser to pull event info from abi
// Use this info to generate event filters // Use this info to generate event filters
func (tr *transformer) Init() error { func (tr *Transformer) Init() error {
for contractAddr := range tr.Config.Addresses { for contractAddr := range tr.Config.Addresses {
// Configure Abi // Configure Abi
if tr.Config.Abis[contractAddr] == "" { if tr.Config.Abis[contractAddr] == "" {
@ -108,7 +108,7 @@ func (tr *transformer) Init() error {
// Get contract name if it has one // Get contract name if it has one
var name = new(string) var name = new(string)
tr.FetchContractData(tr.Abi(), contractAddr, "name", nil, &name, lastBlock) tr.FetchContractData(tr.Abi(), contractAddr, "name", nil, name, lastBlock)
// Remove any potential accidental duplicate inputs in arg filter values // Remove any potential accidental duplicate inputs in arg filter values
eventArgs := map[string]bool{} eventArgs := map[string]bool{}
@ -162,7 +162,7 @@ func (tr *transformer) Init() error {
// Uses converter to convert logs into custom log type // Uses converter to convert logs into custom log type
// Persists converted logs into custuom postgres tables // Persists converted logs into custuom postgres tables
// Calls selected methods, using token holder address generated during event log conversion // Calls selected methods, using token holder address generated during event log conversion
func (tr *transformer) Execute() error { func (tr *Transformer) Execute() error {
if len(tr.Contracts) == 0 { if len(tr.Contracts) == 0 {
return errors.New("error: transformer has no initialized contracts to work with") return errors.New("error: transformer has no initialized contracts to work with")
} }
@ -181,7 +181,7 @@ func (tr *transformer) Execute() error {
// Iterate over watched event logs // Iterate over watched event logs
for _, we := range watchedEvents { for _, we := range watchedEvents {
// Convert them to our custom log type // Convert them to our custom log type
cstm, err := tr.Converter.Convert(*we, con.Events[eventSig]) cstm, err := tr.ConverterInterface.Convert(*we, con.Events[eventSig])
if err != nil { if err != nil {
return err return err
} }
@ -210,6 +210,6 @@ func (tr *transformer) Execute() error {
return nil return nil
} }
func (tr *transformer) GetConfig() config.ContractConfig { func (tr *Transformer) GetConfig() config.ContractConfig {
return tr.Config return tr.Config
} }

333
pkg/contract_watcher/full/transformer/transformer_test.go
View File

@ -17,307 +17,82 @@
package transformer_test package transformer_test
import ( import (
"fmt" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/helpers/test_helpers/mocks"
"github.com/vulcanize/vulcanizedb/pkg/config"
"math/rand" "math/rand"
"strings"
"time" "time"
"github.com/ethereum/go-ethereum/common"
. "github.com/onsi/ginkgo" . "github.com/onsi/ginkgo"
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/full/retriever"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/full/transformer" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/full/transformer"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/constants" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/contract"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/helpers/test_helpers" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/parser"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/helpers/test_helpers/mocks" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/poller"
"github.com/vulcanize/vulcanizedb/pkg/core" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/types"
"github.com/vulcanize/vulcanizedb/pkg/datastore/postgres" "github.com/vulcanize/vulcanizedb/pkg/fakes"
"github.com/vulcanize/vulcanizedb/pkg/datastore/postgres/repositories"
) )
var _ = Describe("Transformer", func() { var _ = Describe("Transformer", func() {
var db *postgres.DB var fakeAddress = "0x1234567890abcdef"
var err error
var blockChain core.BlockChain
var blockRepository repositories.BlockRepository
var ensAddr = strings.ToLower(constants.EnsContractAddress)
var tusdAddr = strings.ToLower(constants.TusdContractAddress)
rand.Seed(time.Now().UnixNano()) rand.Seed(time.Now().UnixNano())
BeforeEach(func() {
db, blockChain = test_helpers.SetupDBandBC()
blockRepository = *repositories.NewBlockRepository(db)
})
AfterEach(func() {
test_helpers.TearDown(db)
})
Describe("Init", func() { Describe("Init", func() {
It("Initializes transformer's contract objects", func() { It("Initializes transformer's contract objects", func() {
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock1) blockRetriever := &fakes.MockFullBlockRetriever{}
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock2) firstBlock := int64(1)
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) mostRecentBlock := int64(2)
err = t.Init() blockRetriever.FirstBlock = firstBlock
blockRetriever.MostRecentBlock = mostRecentBlock
parsr := &fakes.MockParser{}
fakeAbi := "fake_abi"
eventName := "Transfer"
event := types.Event{}
parsr.AbiToReturn = fakeAbi
parsr.EventName = eventName
parsr.Event = event
pollr := &fakes.MockPoller{}
fakeContractName := "fake_contract_name"
pollr.ContractName = fakeContractName
t := getTransformer(blockRetriever, parsr, pollr)
err := t.Init()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
c, ok := t.Contracts[tusdAddr] c, ok := t.Contracts[fakeAddress]
Expect(ok).To(Equal(true)) Expect(ok).To(Equal(true))
Expect(c.StartingBlock).To(Equal(int64(6194633))) Expect(c.StartingBlock).To(Equal(firstBlock))
Expect(c.LastBlock).To(Equal(int64(6194634))) Expect(c.LastBlock).To(Equal(mostRecentBlock))
Expect(c.Abi).To(Equal(constants.TusdAbiString)) Expect(c.Abi).To(Equal(fakeAbi))
Expect(c.Name).To(Equal("TrueUSD")) Expect(c.Name).To(Equal(fakeContractName))
Expect(c.Address).To(Equal(tusdAddr)) Expect(c.Address).To(Equal(fakeAddress))
}) })
It("Fails to initialize if first and most recent blocks cannot be fetched from vDB", func() { It("Fails to initialize if first and most recent blocks cannot be fetched from vDB", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) blockRetriever := &fakes.MockFullBlockRetriever{}
err = t.Init() blockRetriever.FirstBlockErr = fakes.FakeError
Expect(err).To(HaveOccurred()) t := getTransformer(blockRetriever, &fakes.MockParser{}, &fakes.MockPoller{})
})
err := t.Init()
It("Does nothing if watched events are unset", func() {
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock1)
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock2)
var testConf config.ContractConfig
testConf = mocks.TusdConfig
testConf.Events = nil
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).To(HaveOccurred())
_, ok := t.Contracts[tusdAddr]
Expect(ok).To(Equal(false))
})
})
Describe("Execute", func() {
BeforeEach(func() {
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock1)
blockRepository.CreateOrUpdateBlock(mocks.TransferBlock2)
})
It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
log := test_helpers.TransferLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.transfer_event WHERE block = 6194634", tusdAddr)).StructScan(&log)
// We don't know vulcID, so compare individual fields instead of complete structures
Expect(log.Tx).To(Equal("0x135391a0962a63944e5908e6fedfff90fb4be3e3290a21017861099bad654eee"))
Expect(log.Block).To(Equal(int64(6194634)))
Expect(log.From).To(Equal("0x000000000000000000000000000000000000Af21"))
Expect(log.To).To(Equal("0x09BbBBE21a5975cAc061D82f7b843bCE061BA391"))
Expect(log.Value).To(Equal("1097077688018008265106216665536940668749033598146"))
})
It("Keeps track of contract-related addresses while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig
testConf = mocks.TusdConfig
testConf.Methods = map[string][]string{
tusdAddr: {"balanceOf"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
c, ok := t.Contracts[tusdAddr]
Expect(ok).To(Equal(true))
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
b, ok := c.EmittedAddrs[common.HexToAddress("0x000000000000000000000000000000000000Af21")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
b, ok = c.EmittedAddrs[common.HexToAddress("0x09BbBBE21a5975cAc061D82f7b843bCE061BA391")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
_, ok = c.EmittedAddrs[common.HexToAddress("0x09BbBBE21a5975cAc061D82f7b843b1234567890")]
Expect(ok).To(Equal(false))
_, ok = c.EmittedAddrs[common.HexToAddress("0x")]
Expect(ok).To(Equal(false))
_, ok = c.EmittedAddrs[""]
Expect(ok).To(Equal(false))
_, ok = c.EmittedAddrs[common.HexToAddress("0x09THISE21a5IS5cFAKE1D82fAND43bCE06MADEUP")]
Expect(ok).To(Equal(false))
})
It("Polls given methods using generated token holder address", func() {
var testConf config.ContractConfig
testConf = mocks.TusdConfig
testConf.Methods = map[string][]string{
tusdAddr: {"balanceOf"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
res := test_helpers.BalanceOf{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.balanceof_method WHERE who_ = '0x000000000000000000000000000000000000Af21' AND block = '6194634'", tusdAddr)).StructScan(&res)
Expect(err).ToNot(HaveOccurred())
Expect(res.Balance).To(Equal("0"))
Expect(res.TokenName).To(Equal("TrueUSD"))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.balanceof_method WHERE who_ = '0x09BbBBE21a5975cAc061D82f7b843bCE061BA391' AND block = '6194634'", tusdAddr)).StructScan(&res)
Expect(err).ToNot(HaveOccurred())
Expect(res.Balance).To(Equal("0"))
Expect(res.TokenName).To(Equal("TrueUSD"))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.balanceof_method WHERE who_ = '0xfE9e8709d3215310075d67E3ed32A380CCf451C8' AND block = '6194634'", tusdAddr)).StructScan(&res)
Expect(err).To(HaveOccurred())
})
It("Fails if initialization has not been done", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db)
err = t.Execute()
Expect(err).To(HaveOccurred())
})
})
Describe("Execute- against ENS registry contract", func() {
BeforeEach(func() {
blockRepository.CreateOrUpdateBlock(mocks.NewOwnerBlock1)
blockRepository.CreateOrUpdateBlock(mocks.NewOwnerBlock2)
})
It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.ENSConfig, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
log := test_helpers.NewOwnerLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.newowner_event", ensAddr)).StructScan(&log)
// We don't know vulcID, so compare individual fields instead of complete structures
Expect(log.Tx).To(Equal("0x135391a0962a63944e5908e6fedfff90fb4be3e3290a21017861099bad654bbb"))
Expect(log.Block).To(Equal(int64(6194635)))
Expect(log.Node).To(Equal("0x0000000000000000000000000000000000000000000000000000c02aaa39b223"))
Expect(log.Label).To(Equal("0x9dd48110dcc444fdc242510c09bbbbe21a5975cac061d82f7b843bce061ba391"))
Expect(log.Owner).To(Equal("0x000000000000000000000000000000000000Af21"))
})
It("Keeps track of contract-related hashes while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig
testConf = mocks.ENSConfig
testConf.Methods = map[string][]string{
ensAddr: {"owner"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
c, ok := t.Contracts[ensAddr]
Expect(ok).To(Equal(true))
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
Expect(len(c.EmittedHashes)).To(Equal(3))
b, ok := c.EmittedHashes[common.HexToHash("0x0000000000000000000000000000000000000000000000000000c02aaa39b223")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
b, ok = c.EmittedHashes[common.HexToHash("0x9dd48110dcc444fdc242510c09bbbbe21a5975cac061d82f7b843bce061ba391")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
// Doesn't keep track of address since it wouldn't be used in calling the 'owner' method
_, ok = c.EmittedAddrs[common.HexToAddress("0x000000000000000000000000000000000000Af21")]
Expect(ok).To(Equal(false))
})
It("Polls given methods using generated token holder address", func() {
var testConf config.ContractConfig
testConf = mocks.ENSConfig
testConf.Methods = map[string][]string{
ensAddr: {"owner"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
res := test_helpers.Owner{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.owner_method WHERE node_ = '0x0000000000000000000000000000000000000000000000000000c02aaa39b223' AND block = '6194636'", ensAddr)).StructScan(&res)
Expect(err).ToNot(HaveOccurred())
Expect(res.Address).To(Equal("0x0000000000000000000000000000000000000000"))
Expect(res.TokenName).To(Equal(""))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.owner_method WHERE node_ = '0x9dd48110dcc444fdc242510c09bbbbe21a5975cac061d82f7b843bce061ba391' AND block = '6194636'", ensAddr)).StructScan(&res)
Expect(err).ToNot(HaveOccurred())
Expect(res.Address).To(Equal("0x0000000000000000000000000000000000000000"))
Expect(res.TokenName).To(Equal(""))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.owner_method WHERE node_ = '0x9THIS110dcc444fIS242510c09bbAbe21aFAKEcacNODE82f7b843HASH61ba391' AND block = '6194636'", ensAddr)).StructScan(&res)
Expect(err).To(HaveOccurred())
})
It("It does not perist events if they do not pass the emitted arg filter", func() {
var testConf config.ContractConfig
testConf = mocks.ENSConfig
testConf.EventArgs = map[string][]string{
ensAddr: {"fake_filter_value"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
log := test_helpers.LightNewOwnerLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.newowner_event", ensAddr)).StructScan(&log)
Expect(err).To(HaveOccurred())
})
It("If a method arg filter is applied, only those arguments are used in polling", func() {
var testConf config.ContractConfig
testConf = mocks.ENSConfig
testConf.Methods = map[string][]string{
ensAddr: {"owner"},
}
testConf.MethodArgs = map[string][]string{
ensAddr: {"0x0000000000000000000000000000000000000000000000000000c02aaa39b223"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
res := test_helpers.Owner{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.owner_method WHERE node_ = '0x0000000000000000000000000000000000000000000000000000c02aaa39b223' AND block = '6194636'", ensAddr)).StructScan(&res)
Expect(err).ToNot(HaveOccurred())
Expect(res.Address).To(Equal("0x0000000000000000000000000000000000000000"))
Expect(res.TokenName).To(Equal(""))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM full_%s.owner_method WHERE node_ = '0x9dd48110dcc444fdc242510c09bbbbe21a5975cac061d82f7b843bce061ba391' AND block = '6194636'", ensAddr)).StructScan(&res)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err).To(MatchError(fakes.FakeError))
}) })
}) })
}) })
func getTransformer(blockRetriever retriever.BlockRetriever, parsr parser.Parser, pollr poller.Poller) transformer.Transformer {
return transformer.Transformer{
FilterRepository: &fakes.MockFilterRepository{},
Parser: parsr,
BlockRetriever: blockRetriever,
Poller: pollr,
Contracts: map[string]*contract.Contract{},
Config: mocks.MockConfig,
}
}

16
pkg/contract_watcher/light/converter/converter.go
View File

@ -32,28 +32,22 @@ import (
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/types" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/types"
) )
type Converter interface { type ConverterInterface interface {
Convert(logs []gethTypes.Log, event types.Event, headerID int64) ([]types.Log, error) Convert(logs []gethTypes.Log, event types.Event, headerID int64) ([]types.Log, error)
ConvertBatch(logs []gethTypes.Log, events map[string]types.Event, headerID int64) (map[string][]types.Log, error) ConvertBatch(logs []gethTypes.Log, events map[string]types.Event, headerID int64) (map[string][]types.Log, error)
Update(info *contract.Contract) Update(info *contract.Contract)
} }
type converter struct { type Converter struct {
ContractInfo *contract.Contract ContractInfo *contract.Contract
} }
func NewConverter(info *contract.Contract) *converter { func (c *Converter) Update(info *contract.Contract) {
return &converter{
ContractInfo: info,
}
}
func (c *converter) Update(info *contract.Contract) {
c.ContractInfo = info c.ContractInfo = info
} }
// Convert the given watched event log into a types.Log for the given event // Convert the given watched event log into a types.Log for the given event
func (c *converter) Convert(logs []gethTypes.Log, event types.Event, headerID int64) ([]types.Log, error) { func (c *Converter) Convert(logs []gethTypes.Log, event types.Event, headerID int64) ([]types.Log, error) {
contract := bind.NewBoundContract(common.HexToAddress(c.ContractInfo.Address), c.ContractInfo.ParsedAbi, nil, nil, nil) contract := bind.NewBoundContract(common.HexToAddress(c.ContractInfo.Address), c.ContractInfo.ParsedAbi, nil, nil, nil)
returnLogs := make([]types.Log, 0, len(logs)) returnLogs := make([]types.Log, 0, len(logs))
for _, log := range logs { for _, log := range logs {
@ -132,7 +126,7 @@ func (c *converter) Convert(logs []gethTypes.Log, event types.Event, headerID in
} }
// Convert the given watched event logs into types.Logs; returns a map of event names to a slice of their converted logs // Convert the given watched event logs into types.Logs; returns a map of event names to a slice of their converted logs
func (c *converter) ConvertBatch(logs []gethTypes.Log, events map[string]types.Event, headerID int64) (map[string][]types.Log, error) { func (c *Converter) ConvertBatch(logs []gethTypes.Log, events map[string]types.Event, headerID int64) (map[string][]types.Log, error) {
contract := bind.NewBoundContract(common.HexToAddress(c.ContractInfo.Address), c.ContractInfo.ParsedAbi, nil, nil, nil) contract := bind.NewBoundContract(common.HexToAddress(c.ContractInfo.Address), c.ContractInfo.ParsedAbi, nil, nil, nil)
eventsToLogs := make(map[string][]types.Log) eventsToLogs := make(map[string][]types.Log)
for _, event := range events { for _, event := range events {

15
pkg/contract_watcher/light/converter/converter_test.go
View File

@ -38,7 +38,8 @@ var _ = Describe("Converter", func() {
Describe("Update", func() { Describe("Update", func() {
It("Updates contract info held by the converter", func() { It("Updates contract info held by the converter", func() {
con = test_helpers.SetupTusdContract(tusdWantedEvents, []string{}) con = test_helpers.SetupTusdContract(tusdWantedEvents, []string{})
c := converter.NewConverter(con) c := converter.Converter{}
c.Update(con)
Expect(c.ContractInfo).To(Equal(con)) Expect(c.ContractInfo).To(Equal(con))
info := test_helpers.SetupTusdContract([]string{}, []string{}) info := test_helpers.SetupTusdContract([]string{}, []string{})
@ -56,7 +57,8 @@ var _ = Describe("Converter", func() {
event, ok := con.Events["Transfer"] event, ok := con.Events["Transfer"]
Expect(ok).To(Equal(true)) Expect(ok).To(Equal(true))
c := converter.NewConverter(con) c := converter.Converter{}
c.Update(con)
logs, err := c.Convert([]types.Log{mocks.MockTransferLog1, mocks.MockTransferLog2}, event, 232) logs, err := c.Convert([]types.Log{mocks.MockTransferLog1, mocks.MockTransferLog2}, event, 232)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(logs)).To(Equal(2)) Expect(len(logs)).To(Equal(2))
@ -80,7 +82,8 @@ var _ = Describe("Converter", func() {
event, ok := con.Events["Transfer"] event, ok := con.Events["Transfer"]
Expect(ok).To(Equal(true)) Expect(ok).To(Equal(true))
c := converter.NewConverter(con) c := converter.Converter{}
c.Update(con)
_, err := c.Convert([]types.Log{mocks.MockTransferLog1, mocks.MockTransferLog2}, event, 232) _, err := c.Convert([]types.Log{mocks.MockTransferLog1, mocks.MockTransferLog2}, event, 232)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
@ -110,7 +113,8 @@ var _ = Describe("Converter", func() {
event, ok := con.Events["NewOwner"] event, ok := con.Events["NewOwner"]
Expect(ok).To(Equal(true)) Expect(ok).To(Equal(true))
c := converter.NewConverter(con) c := converter.Converter{}
c.Update(con)
_, err := c.Convert([]types.Log{mocks.MockNewOwnerLog1, mocks.MockNewOwnerLog2}, event, 232) _, err := c.Convert([]types.Log{mocks.MockNewOwnerLog1, mocks.MockNewOwnerLog2}, event, 232)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(con.EmittedHashes)).To(Equal(3)) Expect(len(con.EmittedHashes)).To(Equal(3))
@ -143,7 +147,8 @@ var _ = Describe("Converter", func() {
It("Fails with an empty contract", func() { It("Fails with an empty contract", func() {
event := con.Events["Transfer"] event := con.Events["Transfer"]
c := converter.NewConverter(&contract.Contract{}) c := converter.Converter{}
c.Update(&contract.Contract{})
_, err = c.Convert([]types.Log{mocks.MockTransferLog1}, event, 232) _, err = c.Convert([]types.Log{mocks.MockTransferLog1}, event, 232)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
}) })

108
pkg/contract_watcher/light/repository/header_repository_test.go
View File

@ -33,7 +33,7 @@ import (
var _ = Describe("Repository", func() { var _ = Describe("Repository", func() {
var db *postgres.DB var db *postgres.DB
var bc core.BlockChain var bc core.BlockChain
var omniHeaderRepo repository.HeaderRepository // omni/light header repository var contractHeaderRepo repository.HeaderRepository // contract_watcher light header repository
var coreHeaderRepo repositories.HeaderRepository // pkg/datastore header repository var coreHeaderRepo repositories.HeaderRepository // pkg/datastore header repository
var eventIDs = []string{ var eventIDs = []string{
"eventName_contractAddr", "eventName_contractAddr",
@ -48,7 +48,7 @@ var _ = Describe("Repository", func() {
BeforeEach(func() { BeforeEach(func() {
db, bc = test_helpers.SetupDBandBC() db, bc = test_helpers.SetupDBandBC()
omniHeaderRepo = repository.NewHeaderRepository(db) contractHeaderRepo = repository.NewHeaderRepository(db)
coreHeaderRepo = repositories.NewHeaderRepository(db) coreHeaderRepo = repositories.NewHeaderRepository(db)
}) })
@ -62,7 +62,7 @@ var _ = Describe("Repository", func() {
_, err := db.Exec(query) _, err := db.Exec(query)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
err = omniHeaderRepo.AddCheckColumn(eventIDs[0]) err = contractHeaderRepo.AddCheckColumn(eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
_, err = db.Exec(query) _, err = db.Exec(query)
@ -70,13 +70,13 @@ var _ = Describe("Repository", func() {
}) })
It("Caches column it creates so that it does not need to repeatedly query the database to check for it's existence", func() { It("Caches column it creates so that it does not need to repeatedly query the database to check for it's existence", func() {
_, ok := omniHeaderRepo.CheckCache(eventIDs[0]) _, ok := contractHeaderRepo.CheckCache(eventIDs[0])
Expect(ok).To(Equal(false)) Expect(ok).To(Equal(false))
err := omniHeaderRepo.AddCheckColumn(eventIDs[0]) err := contractHeaderRepo.AddCheckColumn(eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
v, ok := omniHeaderRepo.CheckCache(eventIDs[0]) v, ok := contractHeaderRepo.CheckCache(eventIDs[0])
Expect(ok).To(Equal(true)) Expect(ok).To(Equal(true))
Expect(v).To(Equal(true)) Expect(v).To(Equal(true))
}) })
@ -89,7 +89,7 @@ var _ = Describe("Repository", func() {
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
} }
err := omniHeaderRepo.AddCheckColumns(eventIDs) err := contractHeaderRepo.AddCheckColumns(eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
for _, id := range eventIDs { for _, id := range eventIDs {
@ -100,15 +100,15 @@ var _ = Describe("Repository", func() {
It("Caches columns it creates so that it does not need to repeatedly query the database to check for it's existence", func() { It("Caches columns it creates so that it does not need to repeatedly query the database to check for it's existence", func() {
for _, id := range eventIDs { for _, id := range eventIDs {
_, ok := omniHeaderRepo.CheckCache(id) _, ok := contractHeaderRepo.CheckCache(id)
Expect(ok).To(Equal(false)) Expect(ok).To(Equal(false))
} }
err := omniHeaderRepo.AddCheckColumns(eventIDs) err := contractHeaderRepo.AddCheckColumns(eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
for _, id := range eventIDs { for _, id := range eventIDs {
v, ok := omniHeaderRepo.CheckCache(id) v, ok := contractHeaderRepo.CheckCache(id)
Expect(ok).To(Equal(true)) Expect(ok).To(Equal(true))
Expect(v).To(Equal(true)) Expect(v).To(Equal(true))
} }
@ -118,20 +118,20 @@ var _ = Describe("Repository", func() {
Describe("MissingHeaders", func() { Describe("MissingHeaders", func() {
It("Returns all unchecked headers for the given eventID", func() { It("Returns all unchecked headers for the given eventID", func() {
addHeaders(coreHeaderRepo) addHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumn(eventIDs[0]) err := contractHeaderRepo.AddCheckColumn(eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err := omniHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0]) missingHeaders, err := contractHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
}) })
It("Returns unchecked headers in ascending order", func() { It("Returns unchecked headers in ascending order", func() {
addHeaders(coreHeaderRepo) addHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumn(eventIDs[0]) err := contractHeaderRepo.AddCheckColumn(eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err := omniHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0]) missingHeaders, err := contractHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
@ -145,10 +145,10 @@ var _ = Describe("Repository", func() {
It("Returns only contiguous chunks of headers", func() { It("Returns only contiguous chunks of headers", func() {
addDiscontinuousHeaders(coreHeaderRepo) addDiscontinuousHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumns(eventIDs) err := contractHeaderRepo.AddCheckColumns(eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err := omniHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0]) missingHeaders, err := contractHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(2)) Expect(len(missingHeaders)).To(Equal(2))
Expect(missingHeaders[0].BlockNumber).To(Equal(int64(6194632))) Expect(missingHeaders[0].BlockNumber).To(Equal(int64(6194632)))
@ -157,10 +157,10 @@ var _ = Describe("Repository", func() {
It("Fails if eventID does not yet exist in check_headers table", func() { It("Fails if eventID does not yet exist in check_headers table", func() {
addHeaders(coreHeaderRepo) addHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumn(eventIDs[0]) err := contractHeaderRepo.AddCheckColumn(eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
_, err = omniHeaderRepo.MissingHeaders(6194632, 6194635, "notEventId") _, err = contractHeaderRepo.MissingHeaders(6194632, 6194635, "notEventId")
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
}) })
}) })
@ -168,36 +168,36 @@ var _ = Describe("Repository", func() {
Describe("MissingHeadersForAll", func() { // HERE Describe("MissingHeadersForAll", func() { // HERE
It("Returns all headers that have not been checked for all of the ids provided", func() { It("Returns all headers that have not been checked for all of the ids provided", func() {
addHeaders(coreHeaderRepo) addHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumns(eventIDs) err := contractHeaderRepo.AddCheckColumns(eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err := omniHeaderRepo.MissingHeadersForAll(6194632, 6194635, eventIDs) missingHeaders, err := contractHeaderRepo.MissingHeadersForAll(6194632, 6194635, eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
err = omniHeaderRepo.MarkHeaderChecked(missingHeaders[0].Id, eventIDs[0]) err = contractHeaderRepo.MarkHeaderChecked(missingHeaders[0].Id, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err = omniHeaderRepo.MissingHeadersForAll(6194632, 6194635, eventIDs) missingHeaders, err = contractHeaderRepo.MissingHeadersForAll(6194632, 6194635, eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
err = omniHeaderRepo.MarkHeaderChecked(missingHeaders[0].Id, eventIDs[1]) err = contractHeaderRepo.MarkHeaderChecked(missingHeaders[0].Id, eventIDs[1])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
err = omniHeaderRepo.MarkHeaderChecked(missingHeaders[0].Id, eventIDs[2]) err = contractHeaderRepo.MarkHeaderChecked(missingHeaders[0].Id, eventIDs[2])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err = omniHeaderRepo.MissingHeadersForAll(6194633, 6194635, eventIDs) missingHeaders, err = contractHeaderRepo.MissingHeadersForAll(6194633, 6194635, eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(2)) Expect(len(missingHeaders)).To(Equal(2))
}) })
It("Returns only contiguous chunks of headers", func() { It("Returns only contiguous chunks of headers", func() {
addDiscontinuousHeaders(coreHeaderRepo) addDiscontinuousHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumns(eventIDs) err := contractHeaderRepo.AddCheckColumns(eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err := omniHeaderRepo.MissingHeadersForAll(6194632, 6194635, eventIDs) missingHeaders, err := contractHeaderRepo.MissingHeadersForAll(6194632, 6194635, eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(2)) Expect(len(missingHeaders)).To(Equal(2))
Expect(missingHeaders[0].BlockNumber).To(Equal(int64(6194632))) Expect(missingHeaders[0].BlockNumber).To(Equal(int64(6194632)))
@ -206,10 +206,10 @@ var _ = Describe("Repository", func() {
It("Returns at most 100 headers", func() { It("Returns at most 100 headers", func() {
add102Headers(coreHeaderRepo, bc) add102Headers(coreHeaderRepo, bc)
err := omniHeaderRepo.AddCheckColumns(eventIDs) err := contractHeaderRepo.AddCheckColumns(eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err := omniHeaderRepo.MissingHeadersForAll(6194632, 6194733, eventIDs) missingHeaders, err := contractHeaderRepo.MissingHeadersForAll(6194632, 6194733, eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(100)) Expect(len(missingHeaders)).To(Equal(100))
Expect(missingHeaders[0].BlockNumber).To(Equal(int64(6194632))) Expect(missingHeaders[0].BlockNumber).To(Equal(int64(6194632)))
@ -218,11 +218,11 @@ var _ = Describe("Repository", func() {
It("Fails if one of the eventIDs does not yet exist in check_headers table", func() { It("Fails if one of the eventIDs does not yet exist in check_headers table", func() {
addHeaders(coreHeaderRepo) addHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumns(eventIDs) err := contractHeaderRepo.AddCheckColumns(eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
badEventIDs := append(eventIDs, "notEventId") badEventIDs := append(eventIDs, "notEventId")
_, err = omniHeaderRepo.MissingHeadersForAll(6194632, 6194635, badEventIDs) _, err = contractHeaderRepo.MissingHeadersForAll(6194632, 6194635, badEventIDs)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
}) })
}) })
@ -230,36 +230,36 @@ var _ = Describe("Repository", func() {
Describe("MarkHeaderChecked", func() { Describe("MarkHeaderChecked", func() {
It("Marks the header checked for the given eventID", func() { It("Marks the header checked for the given eventID", func() {
addHeaders(coreHeaderRepo) addHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumn(eventIDs[0]) err := contractHeaderRepo.AddCheckColumn(eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err := omniHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0]) missingHeaders, err := contractHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
headerID := missingHeaders[0].Id headerID := missingHeaders[0].Id
err = omniHeaderRepo.MarkHeaderChecked(headerID, eventIDs[0]) err = contractHeaderRepo.MarkHeaderChecked(headerID, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err = omniHeaderRepo.MissingHeaders(6194633, 6194635, eventIDs[0]) missingHeaders, err = contractHeaderRepo.MissingHeaders(6194633, 6194635, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(2)) Expect(len(missingHeaders)).To(Equal(2))
}) })
It("Fails if eventID does not yet exist in check_headers table", func() { It("Fails if eventID does not yet exist in check_headers table", func() {
addHeaders(coreHeaderRepo) addHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumn(eventIDs[0]) err := contractHeaderRepo.AddCheckColumn(eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err := omniHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0]) missingHeaders, err := contractHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
headerID := missingHeaders[0].Id headerID := missingHeaders[0].Id
err = omniHeaderRepo.MarkHeaderChecked(headerID, "notEventId") err = contractHeaderRepo.MarkHeaderChecked(headerID, "notEventId")
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
missingHeaders, err = omniHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0]) missingHeaders, err = contractHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
}) })
@ -268,18 +268,18 @@ var _ = Describe("Repository", func() {
Describe("MarkHeaderCheckedForAll", func() { Describe("MarkHeaderCheckedForAll", func() {
It("Marks the header checked for all provided column ids", func() { It("Marks the header checked for all provided column ids", func() {
addHeaders(coreHeaderRepo) addHeaders(coreHeaderRepo)
err := omniHeaderRepo.AddCheckColumns(eventIDs) err := contractHeaderRepo.AddCheckColumns(eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err := omniHeaderRepo.MissingHeadersForAll(6194632, 6194635, eventIDs) missingHeaders, err := contractHeaderRepo.MissingHeadersForAll(6194632, 6194635, eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
headerID := missingHeaders[0].Id headerID := missingHeaders[0].Id
err = omniHeaderRepo.MarkHeaderCheckedForAll(headerID, eventIDs) err = contractHeaderRepo.MarkHeaderCheckedForAll(headerID, eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err = omniHeaderRepo.MissingHeaders(6194633, 6194635, eventIDs[0]) missingHeaders, err = contractHeaderRepo.MissingHeaders(6194633, 6194635, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(2)) Expect(len(missingHeaders)).To(Equal(2))
}) })
@ -296,17 +296,17 @@ var _ = Describe("Repository", func() {
var missingHeaders []core.Header var missingHeaders []core.Header
for _, id := range methodIDs { for _, id := range methodIDs {
err := omniHeaderRepo.AddCheckColumn(id) err := contractHeaderRepo.AddCheckColumn(id)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
missingHeaders, err = omniHeaderRepo.MissingHeaders(6194632, 6194635, id) missingHeaders, err = contractHeaderRepo.MissingHeaders(6194632, 6194635, id)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
} }
err := omniHeaderRepo.MarkHeadersCheckedForAll(missingHeaders, methodIDs) err := contractHeaderRepo.MarkHeadersCheckedForAll(missingHeaders, methodIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
for _, id := range methodIDs { for _, id := range methodIDs {
missingHeaders, err = omniHeaderRepo.MissingHeaders(6194632, 6194635, id) missingHeaders, err = contractHeaderRepo.MissingHeaders(6194632, 6194635, id)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(0)) Expect(len(missingHeaders)).To(Equal(0))
} }
@ -317,28 +317,28 @@ var _ = Describe("Repository", func() {
It("Returns headers that have been checked for all the provided events but have not been checked for all the provided methods", func() { It("Returns headers that have been checked for all the provided events but have not been checked for all the provided methods", func() {
addHeaders(coreHeaderRepo) addHeaders(coreHeaderRepo)
for i, id := range eventIDs { for i, id := range eventIDs {
err := omniHeaderRepo.AddCheckColumn(id) err := contractHeaderRepo.AddCheckColumn(id)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
err = omniHeaderRepo.AddCheckColumn(methodIDs[i]) err = contractHeaderRepo.AddCheckColumn(methodIDs[i])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
} }
missingHeaders, err := omniHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0]) missingHeaders, err := contractHeaderRepo.MissingHeaders(6194632, 6194635, eventIDs[0])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(missingHeaders)).To(Equal(3)) Expect(len(missingHeaders)).To(Equal(3))
headerID := missingHeaders[0].Id headerID := missingHeaders[0].Id
headerID2 := missingHeaders[1].Id headerID2 := missingHeaders[1].Id
for i, id := range eventIDs { for i, id := range eventIDs {
err = omniHeaderRepo.MarkHeaderChecked(headerID, id) err = contractHeaderRepo.MarkHeaderChecked(headerID, id)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
err = omniHeaderRepo.MarkHeaderChecked(headerID2, id) err = contractHeaderRepo.MarkHeaderChecked(headerID2, id)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
err = omniHeaderRepo.MarkHeaderChecked(headerID, methodIDs[i]) err = contractHeaderRepo.MarkHeaderChecked(headerID, methodIDs[i])
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
} }
intersectionHeaders, err := omniHeaderRepo.MissingMethodsCheckedEventsIntersection(6194632, 6194635, methodIDs, eventIDs) intersectionHeaders, err := contractHeaderRepo.MissingMethodsCheckedEventsIntersection(6194632, 6194635, methodIDs, eventIDs)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(len(intersectionHeaders)).To(Equal(1)) Expect(len(intersectionHeaders)).To(Equal(1))
Expect(intersectionHeaders[0].Id).To(Equal(headerID2)) Expect(intersectionHeaders[0].Id).To(Equal(headerID2))

2
pkg/contract_watcher/light/retriever/retriever_suite_test.go
View File

@ -27,7 +27,7 @@ import (
func TestRetriever(t *testing.T) { func TestRetriever(t *testing.T) {
RegisterFailHandler(Fail) RegisterFailHandler(Fail)
RunSpecs(t, "Light BLock Number Retriever Suite Test") RunSpecs(t, "Light Block Number Retriever Suite Test")
} }
var _ = BeforeSuite(func() { var _ = BeforeSuite(func() {

28
pkg/contract_watcher/light/transformer/transformer.go
View File

@ -38,7 +38,7 @@ import (
) )
// Requires a light synced vDB (headers) and a running eth node (or infura) // Requires a light synced vDB (headers) and a running eth node (or infura)
type transformer struct { type Transformer struct {
// Database interfaces // Database interfaces
srep.EventRepository // Holds transformed watched event log data srep.EventRepository // Holds transformed watched event log data
repository.HeaderRepository // Interface for interaction with header repositories repository.HeaderRepository // Interface for interaction with header repositories
@ -49,7 +49,7 @@ type transformer struct {
// Processing interfaces // Processing interfaces
fetcher.Fetcher // Fetches event logs, using header hashes fetcher.Fetcher // Fetches event logs, using header hashes
converter.Converter // Converts watched event logs into custom log converter.ConverterInterface // Converts watched event logs into custom log
poller.Poller // Polls methods using arguments collected from events and persists them using a method datastore poller.Poller // Polls methods using arguments collected from events and persists them using a method datastore
// Store contract configuration information // Store contract configuration information
@ -74,15 +74,15 @@ type transformer struct {
// 4. Execute // 4. Execute
// Transformer takes in config for blockchain, database, and network id // Transformer takes in config for blockchain, database, and network id
func NewTransformer(con config.ContractConfig, bc core.BlockChain, db *postgres.DB) *transformer { func NewTransformer(con config.ContractConfig, bc core.BlockChain, db *postgres.DB) *Transformer {
return &transformer{ return &Transformer{
Poller: poller.NewPoller(bc, db, types.LightSync), Poller: poller.NewPoller(bc, db, types.LightSync),
Fetcher: fetcher.NewFetcher(bc), Fetcher: fetcher.NewFetcher(bc),
Parser: parser.NewParser(con.Network), Parser: parser.NewParser(con.Network),
HeaderRepository: repository.NewHeaderRepository(db), HeaderRepository: repository.NewHeaderRepository(db),
BlockRetriever: retriever.NewBlockRetriever(db), BlockRetriever: retriever.NewBlockRetriever(db),
Converter: converter.NewConverter(&contract.Contract{}), ConverterInterface: &converter.Converter{},
Contracts: map[string]*contract.Contract{}, Contracts: map[string]*contract.Contract{},
EventRepository: srep.NewEventRepository(db, types.LightSync), EventRepository: srep.NewEventRepository(db, types.LightSync),
Config: con, Config: con,
@ -93,7 +93,7 @@ func NewTransformer(con config.ContractConfig, bc core.BlockChain, db *postgres.
// Loops over all of the addr => filter sets // Loops over all of the addr => filter sets
// Uses parser to pull event info from abi // Uses parser to pull event info from abi
// Use this info to generate event filters // Use this info to generate event filters
func (tr *transformer) Init() error { func (tr *Transformer) Init() error {
// Initialize internally configured transformer settings // Initialize internally configured transformer settings
tr.contractAddresses = make([]string, 0) // Holds all contract addresses, for batch fetching of logs tr.contractAddresses = make([]string, 0) // Holds all contract addresses, for batch fetching of logs
tr.sortedEventIds = make(map[string][]string) // Map to sort event column ids by contract, for post fetch processing and persisting of logs tr.sortedEventIds = make(map[string][]string) // Map to sort event column ids by contract, for post fetch processing and persisting of logs
@ -124,10 +124,6 @@ func (tr *transformer) Init() error {
if err != nil { if err != nil {
return err return err
} }
lastBlock, err := tr.BlockRetriever.RetrieveMostRecentBlock()
if err != nil {
return err
}
// Set to specified range if it falls within the bounds // Set to specified range if it falls within the bounds
if firstBlock < tr.Config.StartingBlocks[contractAddr] { if firstBlock < tr.Config.StartingBlocks[contractAddr] {
@ -136,7 +132,7 @@ func (tr *transformer) Init() error {
// Get contract name if it has one // Get contract name if it has one
var name = new(string) var name = new(string)
tr.FetchContractData(tr.Abi(), contractAddr, "name", nil, &name, lastBlock) tr.Poller.FetchContractData(tr.Abi(), contractAddr, "name", nil, name, -1)
// Remove any potential accidental duplicate inputs // Remove any potential accidental duplicate inputs
eventArgs := map[string]bool{} eventArgs := map[string]bool{}
@ -201,7 +197,7 @@ func (tr *transformer) Init() error {
return nil return nil
} }
func (tr *transformer) Execute() error { func (tr *Transformer) Execute() error {
if len(tr.Contracts) == 0 { if len(tr.Contracts) == 0 {
return errors.New("error: transformer has no initialized contracts") return errors.New("error: transformer has no initialized contracts")
} }
@ -253,10 +249,10 @@ func (tr *transformer) Execute() error {
} }
// Configure converter with this contract // Configure converter with this contract
con := tr.Contracts[conAddr] con := tr.Contracts[conAddr]
tr.Converter.Update(con) tr.ConverterInterface.Update(con)
// Convert logs into batches of log mappings (eventName => []types.Logs // Convert logs into batches of log mappings (eventName => []types.Logs
convertedLogs, err := tr.Converter.ConvertBatch(logs, con.Events, header.Id) convertedLogs, err := tr.ConverterInterface.ConvertBatch(logs, con.Events, header.Id)
if err != nil { if err != nil {
return err return err
} }
@ -293,7 +289,7 @@ func (tr *transformer) Execute() error {
} }
// Used to poll contract methods at a given header // Used to poll contract methods at a given header
func (tr *transformer) methodPolling(header core.Header, sortedMethodIds map[string][]string) error { func (tr *Transformer) methodPolling(header core.Header, sortedMethodIds map[string][]string) error {
for _, con := range tr.Contracts { for _, con := range tr.Contracts {
// Skip method polling processes if no methods are specified // Skip method polling processes if no methods are specified
// Also don't try to poll methods below this contract's specified starting block // Also don't try to poll methods below this contract's specified starting block
@ -317,6 +313,6 @@ func (tr *transformer) methodPolling(header core.Header, sortedMethodIds map[str
return nil return nil
} }
func (tr *transformer) GetConfig() config.ContractConfig { func (tr *Transformer) GetConfig() config.ContractConfig {
return tr.Config return tr.Config
} }

448
pkg/contract_watcher/light/transformer/transformer_test.go
View File

@ -17,425 +17,69 @@
package transformer_test package transformer_test
import ( import (
"fmt"
"github.com/vulcanize/vulcanizedb/pkg/config"
"strings"
"github.com/ethereum/go-ethereum/common"
. "github.com/onsi/ginkgo" . "github.com/onsi/ginkgo"
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/light/retriever"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/light/transformer" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/light/transformer"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/constants" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/contract"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/helpers/test_helpers"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/helpers/test_helpers/mocks" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/helpers/test_helpers/mocks"
"github.com/vulcanize/vulcanizedb/pkg/core" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/parser"
"github.com/vulcanize/vulcanizedb/pkg/datastore/postgres" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/poller"
"github.com/vulcanize/vulcanizedb/pkg/datastore/postgres/repositories" "github.com/vulcanize/vulcanizedb/pkg/fakes"
) )
var _ = Describe("Transformer", func() { var _ = Describe("Transformer", func() {
var db *postgres.DB var fakeAddress = "0x1234567890abcdef"
var err error
var blockChain core.BlockChain
var headerRepository repositories.HeaderRepository
var headerID int64
var ensAddr = strings.ToLower(constants.EnsContractAddress)
var tusdAddr = strings.ToLower(constants.TusdContractAddress)
BeforeEach(func() {
db, blockChain = test_helpers.SetupDBandBC()
headerRepository = repositories.NewHeaderRepository(db)
})
AfterEach(func() {
test_helpers.TearDown(db)
})
Describe("Init", func() { Describe("Init", func() {
It("Initializes transformer's contract objects", func() { It("Initializes transformer's contract objects", func() {
headerRepository.CreateOrUpdateHeader(mocks.MockHeader1) blockRetriever := &fakes.MockLightBlockRetriever{}
headerRepository.CreateOrUpdateHeader(mocks.MockHeader3) firstBlock := int64(1)
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) blockRetriever.FirstBlock = firstBlock
err = t.Init()
parsr := &fakes.MockParser{}
fakeAbi := "fake_abi"
parsr.AbiToReturn = fakeAbi
pollr := &fakes.MockPoller{}
fakeContractName := "fake_contract_name"
pollr.ContractName = fakeContractName
t := getFakeTransformer(blockRetriever, parsr, pollr)
err := t.Init()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
c, ok := t.Contracts[tusdAddr] c, ok := t.Contracts[fakeAddress]
Expect(ok).To(Equal(true)) Expect(ok).To(Equal(true))
Expect(c.StartingBlock).To(Equal(int64(6194632))) Expect(c.StartingBlock).To(Equal(firstBlock))
Expect(c.LastBlock).To(Equal(int64(-1))) Expect(c.LastBlock).To(Equal(int64(-1)))
Expect(c.Abi).To(Equal(constants.TusdAbiString)) Expect(c.Abi).To(Equal(fakeAbi))
Expect(c.Name).To(Equal("TrueUSD")) Expect(c.Name).To(Equal(fakeContractName))
Expect(c.Address).To(Equal(tusdAddr)) Expect(c.Address).To(Equal(fakeAddress))
}) })
It("Fails to initialize if first and most recent block numbers cannot be fetched from vDB headers table", func() { It("Fails to initialize if first block cannot be fetched from vDB headers table", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db) blockRetriever := &fakes.MockLightBlockRetriever{}
err = t.Init() blockRetriever.FirstBlockErr = fakes.FakeError
Expect(err).To(HaveOccurred()) t := getFakeTransformer(blockRetriever, &fakes.MockParser{}, &fakes.MockPoller{})
})
err := t.Init()
It("Does nothing if nothing if no addresses are configured", func() {
headerRepository.CreateOrUpdateHeader(mocks.MockHeader1)
headerRepository.CreateOrUpdateHeader(mocks.MockHeader3)
var testConf config.ContractConfig
testConf = mocks.TusdConfig
testConf.Addresses = nil
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
_, ok := t.Contracts[tusdAddr]
Expect(ok).To(Equal(false))
})
})
Describe("Execute- against TrueUSD contract", func() {
BeforeEach(func() {
header1, err := blockChain.GetHeaderByNumber(6791668)
Expect(err).ToNot(HaveOccurred())
header2, err := blockChain.GetHeaderByNumber(6791669)
Expect(err).ToNot(HaveOccurred())
header3, err := blockChain.GetHeaderByNumber(6791670)
Expect(err).ToNot(HaveOccurred())
headerRepository.CreateOrUpdateHeader(header1)
headerID, err = headerRepository.CreateOrUpdateHeader(header2)
Expect(err).ToNot(HaveOccurred())
headerRepository.CreateOrUpdateHeader(header3)
})
It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
log := test_helpers.LightTransferLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.transfer_event", tusdAddr)).StructScan(&log)
Expect(err).ToNot(HaveOccurred())
// We don't know vulcID, so compare individual fields instead of complete structures
Expect(log.HeaderID).To(Equal(headerID))
Expect(log.From).To(Equal("0x1062a747393198f70F71ec65A582423Dba7E5Ab3"))
Expect(log.To).To(Equal("0x2930096dB16b4A44Ecd4084EA4bd26F7EeF1AEf0"))
Expect(log.Value).To(Equal("9998940000000000000000"))
})
It("Keeps track of contract-related addresses while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig
testConf = mocks.TusdConfig
testConf.Methods = map[string][]string{
tusdAddr: {"balanceOf"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
c, ok := t.Contracts[tusdAddr]
Expect(ok).To(Equal(true))
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
Expect(len(c.EmittedAddrs)).To(Equal(4))
Expect(len(c.EmittedHashes)).To(Equal(0))
b, ok := c.EmittedAddrs[common.HexToAddress("0x1062a747393198f70F71ec65A582423Dba7E5Ab3")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
b, ok = c.EmittedAddrs[common.HexToAddress("0x2930096dB16b4A44Ecd4084EA4bd26F7EeF1AEf0")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
b, ok = c.EmittedAddrs[common.HexToAddress("0x571A326f5B15E16917dC17761c340c1ec5d06f6d")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
b, ok = c.EmittedAddrs[common.HexToAddress("0xFBb1b73C4f0BDa4f67dcA266ce6Ef42f520fBB98")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
_, ok = c.EmittedAddrs[common.HexToAddress("0x09BbBBE21a5975cAc061D82f7b843b1234567890")]
Expect(ok).To(Equal(false))
_, ok = c.EmittedAddrs[common.HexToAddress("0x")]
Expect(ok).To(Equal(false))
_, ok = c.EmittedAddrs[""]
Expect(ok).To(Equal(false))
_, ok = c.EmittedAddrs[common.HexToAddress("0x09THISE21a5IS5cFAKE1D82fAND43bCE06MADEUP")]
Expect(ok).To(Equal(false))
})
It("Polls given methods using generated token holder address", func() {
var testConf config.ContractConfig
testConf = mocks.TusdConfig
testConf.Methods = map[string][]string{
tusdAddr: {"balanceOf"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
res := test_helpers.BalanceOf{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x1062a747393198f70F71ec65A582423Dba7E5Ab3' AND block = '6791669'", tusdAddr)).StructScan(&res)
Expect(err).ToNot(HaveOccurred())
Expect(res.Balance).To(Equal("55849938025000000000000"))
Expect(res.TokenName).To(Equal("TrueUSD"))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x09BbBBE21a5975cAc061D82f7b843b1234567890' AND block = '6791669'", tusdAddr)).StructScan(&res)
Expect(err).To(HaveOccurred())
})
It("Fails if initialization has not been done", func() {
t := transformer.NewTransformer(mocks.TusdConfig, blockChain, db)
err = t.Execute()
Expect(err).To(HaveOccurred())
})
})
Describe("Execute- against ENS registry contract", func() {
BeforeEach(func() {
header1, err := blockChain.GetHeaderByNumber(6885695)
Expect(err).ToNot(HaveOccurred())
header2, err := blockChain.GetHeaderByNumber(6885696)
Expect(err).ToNot(HaveOccurred())
header3, err := blockChain.GetHeaderByNumber(6885697)
Expect(err).ToNot(HaveOccurred())
headerRepository.CreateOrUpdateHeader(header1)
headerID, err = headerRepository.CreateOrUpdateHeader(header2)
Expect(err).ToNot(HaveOccurred())
headerRepository.CreateOrUpdateHeader(header3)
})
It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.ENSConfig, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
log := test_helpers.LightNewOwnerLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.newowner_event", ensAddr)).StructScan(&log)
Expect(err).ToNot(HaveOccurred())
// We don't know vulcID, so compare individual fields instead of complete structures
Expect(log.HeaderID).To(Equal(headerID))
Expect(log.Node).To(Equal("0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae"))
Expect(log.Label).To(Equal("0x95832c7a47ff8a7840e28b78ce695797aaf402b1c186bad9eca28842625b5047"))
Expect(log.Owner).To(Equal("0x6090A6e47849629b7245Dfa1Ca21D94cd15878Ef"))
})
It("Keeps track of contract-related hashes while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig
testConf = mocks.ENSConfig
testConf.Methods = map[string][]string{
ensAddr: {"owner"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
c, ok := t.Contracts[ensAddr]
Expect(ok).To(Equal(true))
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
Expect(len(c.EmittedHashes)).To(Equal(2))
Expect(len(c.EmittedAddrs)).To(Equal(0))
b, ok := c.EmittedHashes[common.HexToHash("0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
b, ok = c.EmittedHashes[common.HexToHash("0x95832c7a47ff8a7840e28b78ce695797aaf402b1c186bad9eca28842625b5047")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
// Doesn't keep track of address since it wouldn't be used in calling the 'owner' method
_, ok = c.EmittedAddrs[common.HexToAddress("0x6090A6e47849629b7245Dfa1Ca21D94cd15878Ef")]
Expect(ok).To(Equal(false))
})
It("Polls given method using list of collected hashes", func() {
var testConf config.ContractConfig
testConf = mocks.ENSConfig
testConf.Methods = map[string][]string{
ensAddr: {"owner"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
res := test_helpers.Owner{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae' AND block = '6885696'", ensAddr)).StructScan(&res)
Expect(err).ToNot(HaveOccurred())
Expect(res.Address).To(Equal("0x6090A6e47849629b7245Dfa1Ca21D94cd15878Ef"))
Expect(res.TokenName).To(Equal(""))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x95832c7a47ff8a7840e28b78ce695797aaf402b1c186bad9eca28842625b5047' AND block = '6885696'", ensAddr)).StructScan(&res)
Expect(err).ToNot(HaveOccurred())
Expect(res.Address).To(Equal("0x0000000000000000000000000000000000000000"))
Expect(res.TokenName).To(Equal(""))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x9THIS110dcc444fIS242510c09bbAbe21aFAKEcacNODE82f7b843HASH61ba391' AND block = '6885696'", ensAddr)).StructScan(&res)
Expect(err).To(HaveOccurred())
})
It("It does not persist events if they do not pass the emitted arg filter", func() {
var testConf config.ContractConfig
testConf = mocks.ENSConfig
testConf.EventArgs = map[string][]string{
ensAddr: {"fake_filter_value"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
log := test_helpers.LightNewOwnerLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.newowner_event", ensAddr)).StructScan(&log)
Expect(err).To(HaveOccurred())
})
It("If a method arg filter is applied, only those arguments are used in polling", func() {
var testConf config.ContractConfig
testConf = mocks.ENSConfig
testConf.Methods = map[string][]string{
ensAddr: {"owner"},
}
testConf.MethodArgs = map[string][]string{
ensAddr: {"0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
res := test_helpers.Owner{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae' AND block = '6885696'", ensAddr)).StructScan(&res)
Expect(err).ToNot(HaveOccurred())
Expect(res.Address).To(Equal("0x6090A6e47849629b7245Dfa1Ca21D94cd15878Ef"))
Expect(res.TokenName).To(Equal(""))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x95832c7a47ff8a7840e28b78ce695797aaf402b1c186bad9eca28842625b5047' AND block = '6885696'", ensAddr)).StructScan(&res)
Expect(err).To(HaveOccurred())
})
})
Describe("Execute- against both ENS and TrueUSD", func() {
BeforeEach(func() {
for i := 6885692; i < 6885702; i++ {
header, err := blockChain.GetHeaderByNumber(int64(i))
Expect(err).ToNot(HaveOccurred())
_, err = headerRepository.CreateOrUpdateHeader(header)
Expect(err).ToNot(HaveOccurred())
}
})
It("Transforms watched contract data into custom repositories", func() {
t := transformer.NewTransformer(mocks.ENSandTusdConfig, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
newOwnerLog := test_helpers.LightNewOwnerLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.newowner_event", ensAddr)).StructScan(&newOwnerLog)
Expect(err).ToNot(HaveOccurred())
// We don't know vulcID, so compare individual fields instead of complete structures
Expect(newOwnerLog.Node).To(Equal("0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae"))
Expect(newOwnerLog.Label).To(Equal("0x95832c7a47ff8a7840e28b78ce695797aaf402b1c186bad9eca28842625b5047"))
Expect(newOwnerLog.Owner).To(Equal("0x6090A6e47849629b7245Dfa1Ca21D94cd15878Ef"))
transferLog := test_helpers.LightTransferLog{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.transfer_event", tusdAddr)).StructScan(&transferLog)
Expect(err).ToNot(HaveOccurred())
// We don't know vulcID, so compare individual fields instead of complete structures
Expect(transferLog.From).To(Equal("0x8cA465764873E71CEa525F5EB6AE973d650c22C2"))
Expect(transferLog.To).To(Equal("0xc338482360651E5D30BEd77b7c85358cbBFB2E0e"))
Expect(transferLog.Value).To(Equal("2800000000000000000000"))
})
It("Keeps track of contract-related hashes and addresses while transforming event data if they need to be used for later method polling", func() {
var testConf config.ContractConfig
testConf = mocks.ENSandTusdConfig
testConf.Methods = map[string][]string{
ensAddr: {"owner"},
tusdAddr: {"balanceOf"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
ens, ok := t.Contracts[ensAddr]
Expect(ok).To(Equal(true))
tusd, ok := t.Contracts[tusdAddr]
Expect(ok).To(Equal(true))
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
Expect(len(ens.EmittedHashes)).To(Equal(2))
Expect(len(ens.EmittedAddrs)).To(Equal(0))
Expect(len(tusd.EmittedAddrs)).To(Equal(2))
Expect(len(tusd.EmittedHashes)).To(Equal(0))
b, ok := ens.EmittedHashes[common.HexToHash("0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
b, ok = ens.EmittedHashes[common.HexToHash("0x95832c7a47ff8a7840e28b78ce695797aaf402b1c186bad9eca28842625b5047")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
b, ok = tusd.EmittedAddrs[common.HexToAddress("0x8cA465764873E71CEa525F5EB6AE973d650c22C2")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
b, ok = tusd.EmittedAddrs[common.HexToAddress("0xc338482360651E5D30BEd77b7c85358cbBFB2E0e")]
Expect(ok).To(Equal(true))
Expect(b).To(Equal(true))
_, ok = tusd.EmittedAddrs[common.HexToAddress("0x6090A6e47849629b7245Dfa1Ca21D94cd15878Ef")]
Expect(ok).To(Equal(false))
})
It("Polls given methods for each contract, using list of collected values", func() {
var testConf config.ContractConfig
testConf = mocks.ENSandTusdConfig
testConf.Methods = map[string][]string{
ensAddr: {"owner"},
tusdAddr: {"balanceOf"},
}
t := transformer.NewTransformer(testConf, blockChain, db)
err = t.Init()
Expect(err).ToNot(HaveOccurred())
err = t.Execute()
Expect(err).ToNot(HaveOccurred())
owner := test_helpers.Owner{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x93cdeb708b7545dc668eb9280176169d1c33cfd8ed6f04690a0bcc88a93fc4ae' AND block = '6885696'", ensAddr)).StructScan(&owner)
Expect(err).ToNot(HaveOccurred())
Expect(owner.Address).To(Equal("0x6090A6e47849629b7245Dfa1Ca21D94cd15878Ef"))
Expect(owner.TokenName).To(Equal(""))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x95832c7a47ff8a7840e28b78ce695797aaf402b1c186bad9eca28842625b5047' AND block = '6885696'", ensAddr)).StructScan(&owner)
Expect(err).ToNot(HaveOccurred())
Expect(owner.Address).To(Equal("0x0000000000000000000000000000000000000000"))
Expect(owner.TokenName).To(Equal(""))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.owner_method WHERE node_ = '0x95832c7a47ff8a7840e28b78ceMADEUPaaf4HASHc186badTHItransformers.8IS625bFAKE' AND block = '6885696'", ensAddr)).StructScan(&owner)
Expect(err).To(HaveOccurred())
bal := test_helpers.BalanceOf{}
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x8cA465764873E71CEa525F5EB6AE973d650c22C2' AND block = '6885701'", tusdAddr)).StructScan(&bal)
Expect(err).ToNot(HaveOccurred())
Expect(bal.Balance).To(Equal("1954436000000000000000"))
Expect(bal.TokenName).To(Equal("TrueUSD"))
err = db.QueryRowx(fmt.Sprintf("SELECT * FROM light_%s.balanceof_method WHERE who_ = '0x09BbBBE21a5975cAc061D82f7b843b1234567890' AND block = '6885701'", tusdAddr)).StructScan(&bal)
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(err).To(MatchError(fakes.FakeError))
}) })
}) })
}) })
func getFakeTransformer(blockRetriever retriever.BlockRetriever, parsr parser.Parser, pollr poller.Poller) transformer.Transformer {
return transformer.Transformer{
Parser: parsr,
BlockRetriever: blockRetriever,
Poller: pollr,
HeaderRepository: &fakes.MockLightHeaderRepository{},
Contracts: map[string]*contract.Contract{},
Config: mocks.MockConfig,
}
}

2
pkg/contract_watcher/shared/contract/contract.go
View File

@ -67,7 +67,7 @@ func (c Contract) Init() *Contract {
return &c return &c
} }
// Use contract info to generate event filters - full sync omni watcher only // Use contract info to generate event filters - full sync contract watcher only
func (c *Contract) GenerateFilters() error { func (c *Contract) GenerateFilters() error {
c.Filters = map[string]filters.LogFilter{} c.Filters = map[string]filters.LogFilter{}

58
pkg/contract_watcher/shared/helpers/test_helpers/mocks/entities.go
View File

@ -18,13 +18,13 @@ package mocks
import ( import (
"encoding/json" "encoding/json"
"github.com/vulcanize/vulcanizedb/pkg/config"
"strings" "strings"
"github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil" "github.com/ethereum/go-ethereum/common/hexutil"
"github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/core/types"
"github.com/vulcanize/vulcanizedb/pkg/config"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/constants" "github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/constants"
"github.com/vulcanize/vulcanizedb/pkg/core" "github.com/vulcanize/vulcanizedb/pkg/core"
"github.com/vulcanize/vulcanizedb/pkg/filters" "github.com/vulcanize/vulcanizedb/pkg/filters"
@ -262,74 +262,46 @@ var MockNewOwnerLog2 = types.Log{
var ens = strings.ToLower(constants.EnsContractAddress) var ens = strings.ToLower(constants.EnsContractAddress)
var tusd = strings.ToLower(constants.TusdContractAddress) var tusd = strings.ToLower(constants.TusdContractAddress)
var TusdConfig = config.ContractConfig{ var MockConfig = config.ContractConfig{
Network: "", Network: "",
Addresses: map[string]bool{ Addresses: map[string]bool{
tusd: true, "0x1234567890abcdef": true,
}, },
Abis: map[string]string{ Abis: map[string]string{
tusd: "", "0x1234567890abcdef": "fake_abi",
}, },
Events: map[string][]string{ Events: map[string][]string{
tusd: []string{"Transfer"}, "0x1234567890abcdef": []string{"Transfer"},
}, },
Methods: map[string][]string{ Methods: map[string][]string{
tusd: nil, "0x1234567890abcdef": nil,
}, },
MethodArgs: map[string][]string{ MethodArgs: map[string][]string{
tusd: nil, "0x1234567890abcdef": nil,
}, },
EventArgs: map[string][]string{ EventArgs: map[string][]string{
tusd: nil, "0x1234567890abcdef": nil,
}, },
} }
var ENSConfig = config.ContractConfig{ var MockEmptyConfig = config.ContractConfig{
Network: "", Network: "",
Addresses: map[string]bool{ Addresses: map[string]bool{
ens: true, "0x1234567890abcdef": true,
}, },
Abis: map[string]string{ Abis: map[string]string{
ens: "", "0x1234567890abcdef": "fake_abi",
}, },
Events: map[string][]string{ Events: map[string][]string{
ens: []string{"NewOwner"}, "0x1234567890abcdef": nil,
}, },
Methods: map[string][]string{ Methods: map[string][]string{
ens: nil, "0x1234567890abcdef": nil,
}, },
MethodArgs: map[string][]string{ MethodArgs: map[string][]string{
ens: nil, "0x1234567890abcdef": nil,
}, },
EventArgs: map[string][]string{ EventArgs: map[string][]string{
ens: nil, "0x1234567890abcdef": nil,
},
}
var ENSandTusdConfig = config.ContractConfig{
Network: "",
Addresses: map[string]bool{
ens: true,
tusd: true,
},
Abis: map[string]string{
ens: "",
tusd: "",
},
Events: map[string][]string{
ens: []string{"NewOwner"},
tusd: []string{"Transfer"},
},
Methods: map[string][]string{
ens: nil,
tusd: nil,
},
MethodArgs: map[string][]string{
ens: nil,
tusd: nil,
},
EventArgs: map[string][]string{
ens: nil,
tusd: nil,
}, },
} }

109
pkg/contract_watcher/shared/helpers/test_helpers/test_data.go Normal file
View File

@ -0,0 +1,109 @@
// VulcanizeDB
// Copyright © 2019 Vulcanize
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package test_helpers
import (
"strings"
"github.com/vulcanize/vulcanizedb/pkg/config"
"github.com/vulcanize/vulcanizedb/pkg/contract_watcher/shared/constants"
)
var ens = strings.ToLower(constants.EnsContractAddress)
var tusd = strings.ToLower(constants.TusdContractAddress)
var TusdConfig = config.ContractConfig{
Network: "",
Addresses: map[string]bool{
tusd: true,
},
Abis: map[string]string{
tusd: "",
},
Events: map[string][]string{
tusd: []string{"Transfer"},
},
Methods: map[string][]string{
tusd: nil,
},
MethodArgs: map[string][]string{
tusd: nil,
},
EventArgs: map[string][]string{
tusd: nil,
},
StartingBlocks: map[string]int64{
tusd: 5197514,
},
}
var ENSConfig = config.ContractConfig{
Network: "",
Addresses: map[string]bool{
ens: true,
},
Abis: map[string]string{
ens: "",
},
Events: map[string][]string{
ens: []string{"NewOwner"},
},
Methods: map[string][]string{
ens: nil,
},
MethodArgs: map[string][]string{
ens: nil,
},
EventArgs: map[string][]string{
ens: nil,
},
StartingBlocks: map[string]int64{
ens: 3327417,
},
}
var ENSandTusdConfig = config.ContractConfig{
Network: "",
Addresses: map[string]bool{
ens: true,
tusd: true,
},
Abis: map[string]string{
ens: "",
tusd: "",
},
Events: map[string][]string{
ens: []string{"NewOwner"},
tusd: []string{"Transfer"},
},
Methods: map[string][]string{
ens: nil,
tusd: nil,
},
MethodArgs: map[string][]string{
ens: nil,
tusd: nil,
},
EventArgs: map[string][]string{
ens: nil,
tusd: nil,
},
StartingBlocks: map[string]int64{
ens: 3327417,
tusd: 5197514,
},
}

6
pkg/contract_watcher/shared/repository/event_repository_test.go
View File

@ -134,7 +134,8 @@ var _ = Describe("Repository", func() {
Describe("PersistLogs", func() { Describe("PersistLogs", func() {
BeforeEach(func() { BeforeEach(func() {
c := fc.NewConverter(con) c := fc.Converter{}
c.Update(con)
log, err = c.Convert(mockEvent, event) log, err = c.Convert(mockEvent, event)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
}) })
@ -276,7 +277,8 @@ var _ = Describe("Repository", func() {
headerRepository := repositories.NewHeaderRepository(db) headerRepository := repositories.NewHeaderRepository(db)
headerID, err = headerRepository.CreateOrUpdateHeader(mocks.MockHeader1) headerID, err = headerRepository.CreateOrUpdateHeader(mocks.MockHeader1)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
c := lc.NewConverter(con) c := lc.Converter{}
c.Update(con)
logs, err = c.Convert([]geth.Log{mockLog1, mockLog2}, event, headerID) logs, err = c.Convert([]geth.Log{mockLog1, mockLog2}, event, headerID)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
}) })

3
pkg/contract_watcher/shared/retriever/address_retriever_test.go
View File

@ -64,7 +64,8 @@ var _ = Describe("Address Retriever Test", func() {
err = info.GenerateFilters() err = info.GenerateFilters()
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
c := converter.NewConverter(info) c := converter.Converter{}
c.Update(info)
log, err = c.Convert(mockEvent, event) log, err = c.Convert(mockEvent, event)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())

2
pkg/contract_watcher/shared/types/event.go
View File

@ -38,7 +38,7 @@ type Field struct {
// Struct to hold instance of an event log data // Struct to hold instance of an event log data
type Log struct { type Log struct {
Id int64 // VulcanizeIdLog for full sync and header ID for light sync omni watcher Id int64 // VulcanizeIdLog for full sync and header ID for light sync contract watcher
Values map[string]string // Map of event input names to their values Values map[string]string // Map of event input names to their values
// Used for full sync only // Used for full sync only

4
pkg/contract_watcher/shared/types/mode.go
View File

@ -47,7 +47,7 @@ func (mode Mode) MarshalText() ([]byte, error) {
case FullSync: case FullSync:
return []byte("full"), nil return []byte("full"), nil
default: default:
return nil, fmt.Errorf("omni watcher: unknown mode %d, want LightSync or FullSync", mode) return nil, fmt.Errorf("contract watcher: unknown mode %d, want LightSync or FullSync", mode)
} }
} }
@ -58,7 +58,7 @@ func (mode *Mode) UnmarshalText(text []byte) error {
case "full": case "full":
*mode = FullSync *mode = FullSync
default: default:
return fmt.Errorf(`omni watcher: unknown mode %q, want "light" or "full"`, text) return fmt.Errorf(`contract watcher: unknown mode %q, want "light" or "full"`, text)
} }
return nil return nil
} }

5
pkg/fakes/mock_parser.go
View File

@ -15,8 +15,9 @@ func (*MockParser) Parse(contractAddr string) error {
return nil return nil
} }
func (*MockParser) ParseAbiStr(abiStr string) error { func (m *MockParser) ParseAbiStr(abiStr string) error {
panic("implement me") m.AbiToReturn = abiStr
return nil
} }
func (parser *MockParser) Abi() string { func (parser *MockParser) Abi() string {

8
pkg/plugin/writer/writer.go
View File

@ -74,7 +74,7 @@ func (w *writer) WritePlugin() error {
f.Func().Params(Id("e").Id("exporter")).Id("Export").Params().Parens(List( f.Func().Params(Id("e").Id("exporter")).Id("Export").Params().Parens(List(
Index().Qual("github.com/vulcanize/vulcanizedb/libraries/shared/transformer", "EventTransformerInitializer"), Index().Qual("github.com/vulcanize/vulcanizedb/libraries/shared/transformer", "EventTransformerInitializer"),
Index().Qual("github.com/vulcanize/vulcanizedb/libraries/shared/transformer", "StorageTransformerInitializer"), Index().Qual("github.com/vulcanize/vulcanizedb/libraries/shared/transformer", "StorageTransformerInitializer"),
Index().Qual("github.com/vulcanize/vulcanizedb/libraries/shared/transformer", "GenericTransformerInitializer"), Index().Qual("github.com/vulcanize/vulcanizedb/libraries/shared/transformer", "ContractTransformerInitializer"),
)).Block(Return( )).Block(Return(
Index().Qual( Index().Qual(
"github.com/vulcanize/vulcanizedb/libraries/shared/transformer", "github.com/vulcanize/vulcanizedb/libraries/shared/transformer",
@ -84,7 +84,7 @@ func (w *writer) WritePlugin() error {
"StorageTransformerInitializer").Values(code[config.EthStorage]...), "StorageTransformerInitializer").Values(code[config.EthStorage]...),
Index().Qual( Index().Qual(
"github.com/vulcanize/vulcanizedb/libraries/shared/transformer", "github.com/vulcanize/vulcanizedb/libraries/shared/transformer",
"GenericTransformerInitializer").Values(code[config.EthGeneric]...))) // Exports the collected event and storage transformer initializers "ContractTransformerInitializer").Values(code[config.EthContract]...))) // Exports the collected event and storage transformer initializers
// Write code to destination file // Write code to destination file
err = f.Save(goFile) err = f.Save(goFile)
@ -104,8 +104,8 @@ func (w *writer) collectTransformers() (map[config.TransformerType][]Code, error
code[config.EthEvent] = append(code[config.EthEvent], Qual(path, "EventTransformerInitializer")) code[config.EthEvent] = append(code[config.EthEvent], Qual(path, "EventTransformerInitializer"))
case config.EthStorage: case config.EthStorage:
code[config.EthStorage] = append(code[config.EthStorage], Qual(path, "StorageTransformerInitializer")) code[config.EthStorage] = append(code[config.EthStorage], Qual(path, "StorageTransformerInitializer"))
case config.EthGeneric: case config.EthContract:
code[config.EthGeneric] = append(code[config.EthGeneric], Qual(path, "GenericTransformerInitializer")) code[config.EthContract] = append(code[config.EthContract], Qual(path, "ContractTransformerInitializer"))
default: default:
return nil, errors.New(fmt.Sprintf("invalid transformer type %s", transformer.Type)) return nil, errors.New(fmt.Sprintf("invalid transformer type %s", transformer.Type))
} }