cerc-io/ipld-eth-server
18
0
Fork 1
Code Issues 10 Pull Requests 1 Actions Packages Projects Releases 37 Wiki Activity

move maker plugin generator test to mcd_transformers

Browse Source
This commit is contained in:
Ian Norden 2019-02-18 05:23:36 -06:00
parent f2072561a7
commit 17f4d3dfa5
10 changed files with 60 additions and 439 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
README.md
View File

@ -259,6 +259,8 @@ The information provided in the .toml config is used to generate the plugin:
ipcPath = "http://kovan0.vulcanize.io:8545" ipcPath = "http://kovan0.vulcanize.io:8545"
[exporter] [exporter]
home = "github.com/vulcanize/vulcanizedb"
clone = false
name = "eventTransformerExporter" name = "eventTransformerExporter"
save = false save = false
transformerNames = [ transformerNames = [
@ -288,20 +290,23 @@ The information provided in the .toml config is used to generate the plugin:
repository = "github.com/account2/repo2" repository = "github.com/account2/repo2"
migrations = "to/db/migrations" migrations = "to/db/migrations"
``` ```
- `home` is the name of the package you are building the plugin for, in most cases this is github.com/vulcanize/vulcanizedb
- `clone` this signifies whether or not to retrieve transformer packages by cloning them; by default we attempt to work with transformer packages located in
our `$GOPATH` but setting this to `true` overrides that. This needs to be set to `true` for the configs used in tests in order for them to work with Travis.
- `name` is the name used for the plugin files (.so and .go) - `name` is the name used for the plugin files (.so and .go)
- `save` indicates whether or not the user wants to save the .go file instead of removing it after .so compilation (useful for debugging) - `save` indicates whether or not the user wants to save the .go file instead of removing it after .so compilation. Sometimes useful for debugging/trouble-shooting purposes.
- `transformerNames` is the list of the names of the transformers we are composing together, so we know how to access their submaps in the exporter map - `transformerNames` is the list of the names of the transformers we are composing together, so we know how to access their submaps in the exporter map
- `exporter.<transformerName>`s are the sub-mappings containing config info for the transformers - `exporter.<transformerName>`s are the sub-mappings containing config info for the transformers
- `repository` is the path for the repository which contains the transformer and its `TransformerInitializer` - `repository` is the path for the repository which contains the transformer and its `TransformerInitializer`
- `path` is the relative path from `repository` to the transformer's `TransformerInitializer` - `path` is the relative path from `repository` to the transformer's `TransformerInitializer` directory (initializer package)
- `type` is the type of the transformer; indicating which type of watcher it works with (for now, there are only two options: "eth_event" and "eth_storage") - `type` is the type of the transformer; indicating which type of watcher it works with (for now, there are only two options: `eth_event` and `eth_storage`)
- "eth_storage" indicates the transformer works with the [storage watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/storage_watcher.go) - `eth_storage` indicates the transformer works with the [storage watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/storage_watcher.go)
that fetches state and storage diffs from an ETH node (instead of, for example, from IPFS) that fetches state and storage diffs from an ETH node (instead of, for example, from IPFS)
- "eth_event" indicates the transformer works with the [event watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/event_watcher.go) - `eth_event` indicates the transformer works with the [event watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/event_watcher.go)
that fetches event logs from an ETH node that fetches event logs from an ETH node
- `migrations` is the relative path from `repository` to the db migrations for the transformer - `migrations` is the relative path from `repository` to the db migrations directory for the transformer
Note: If any of the imported transformer need additional Note: If any of the imported transformers need additional
config variables do not forget to include those as well config variables do not forget to include those as well
This information is used to write and build a go plugin with a transformer This information is used to write and build a go plugin with a transformer
@ -345,7 +350,7 @@ To plug in an external transformer we need to:
* create a [package](https://github.com/vulcanize/mcd_transformers/blob/staging/transformers/bite/initializer/initializer.go) * create a [package](https://github.com/vulcanize/mcd_transformers/blob/staging/transformers/bite/initializer/initializer.go)
that exports a variable `TransformerInitializer` or `StorageTransformerInitializer` that are of type [TransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/event_transformer.go#L33) that exports a variable `TransformerInitializer` or `StorageTransformerInitializer` that are of type [TransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/event_transformer.go#L33)
and [StorageTransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/storage_transformer.go#L31), respectively or [StorageTransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/storage_transformer.go#L31), respectively
* design the transformers to work in the context of the [event](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/event_watcher.go#L83) * design the transformers to work in the context of their [event](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/event_watcher.go#L83)
or [storage](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/storage_watcher.go#L53) watchers or [storage](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/storage_watcher.go#L53) watchers
* create db migrations to run against vulcanizeDB so that we can store the transformed data * create db migrations to run against vulcanizeDB so that we can store the transformed data

2
cmd/composeAndExecute.go
View File

@ -263,5 +263,7 @@ func prepConfig() {
FilePath: "$GOPATH/src/github.com/vulcanize/vulcanizedb/plugins", FilePath: "$GOPATH/src/github.com/vulcanize/vulcanizedb/plugins",
FileName: viper.GetString("exporter.name"), FileName: viper.GetString("exporter.name"),
Save: viper.GetBool("exporter.save"), Save: viper.GetBool("exporter.save"),
Home: viper.GetString("exporter.home"),
Clone: viper.GetBool("exporter.clone"),
} }
} }

12
environments/public.toml.example
View File

@ -6,15 +6,3 @@ port = 5432
[client] [client]
ipcPath = <local node's IPC filepath> ipcPath = <local node's IPC filepath>
levelDbPath = <local node's LevelDB chaindata filepath> levelDbPath = <local node's LevelDB chaindata filepath>
[contract]
cat = "0x2f34f22a00ee4b7a8f8bbc4eaee1658774c624e0"
drip = "0x891c04639a5edcae088e546fa125b5d7fb6a2b9d"
eth_flip = "0x32D496Ad866D110060866B7125981C73642cc509"
mcd_flap = "0x8868BAd8e74FcA4505676D1B5B21EcC23328d132"
mcd_flop = "0x6191C9b0086c2eBF92300cC507009b53996FbFFa"
pep = "0xB1997239Cfc3d15578A3a09730f7f84A90BB4975"
pip = "0x9FfFE440258B79c5d6604001674A4722FfC0f7Bc"
pit = "0xe7cf3198787c9a4daac73371a38f29aaeeced87e"
rep = "0xf88bBDc1E2718F8857F30A180076ec38d53cf296"
vat = "0xcd726790550afcd77e9a7a47e86a3f9010af126b"

4
pkg/config/plugin.go
View File

@ -28,6 +28,8 @@ type Plugin struct {
FilePath string FilePath string
FileName string FileName string
Save bool Save bool
Home string
Clone bool
} }
type Transformer struct { type Transformer struct {
@ -56,7 +58,7 @@ func (c *Plugin) GetMigrationsPaths() (map[string]bool, error) {
for _, transformer := range c.Transformers { for _, transformer := range c.Transformers {
repo := transformer.RepositoryPath repo := transformer.RepositoryPath
mig := transformer.MigrationPath mig := transformer.MigrationPath
path := filepath.Join("$GOPATH/src/github.com/vulcanize/vulcanizedb/vendor", repo, mig) path := filepath.Join("$GOPATH/src", c.Home, "vendor", repo, mig)
cleanPath, err := helpers.CleanPath(path) cleanPath, err := helpers.CleanPath(path)
if err != nil { if err != nil {
return nil, err return nil, err

34
pkg/plugin/builder/builder.go
View File

@ -78,7 +78,7 @@ func (b *builder) BuildPlugin() error {
// This is to work around a conflict between plugins and vendoring (https://github.com/golang/go/issues/20481) // This is to work around a conflict between plugins and vendoring (https://github.com/golang/go/issues/20481)
func (b *builder) setupBuildEnv() error { func (b *builder) setupBuildEnv() error {
// TODO: Less hacky way of handling plugin build deps // TODO: Less hacky way of handling plugin build deps
vendorPath, err := helpers.CleanPath("$GOPATH/src/github.com/vulcanize/vulcanizedb/vendor") vendorPath, err := helpers.CleanPath(filepath.Join("$GOPATH/src", b.GenConfig.Home, "vendor"))
if err != nil { if err != nil {
return err return err
} }
@ -87,25 +87,37 @@ func (b *builder) setupBuildEnv() error {
// Import transformer dependencies so that we can build our plugin // Import transformer dependencies so that we can build our plugin
for importPath := range repoPaths { for importPath := range repoPaths {
// Use dependency paths in config to form git ssh string dstPath := filepath.Join(vendorPath, importPath)
// TODO: Change this to https once we are no longer working private transformer repos // When testing on Travis we need to clone the libs
// Right now since vulcanize/mcd_transformers is a private repo we if b.GenConfig.Clone {
// are using ssh and uploading ssh key to travis for testing // And if we want to be able to work with a private repo we need to use ssh instead of https
// This is slower and more involved than using https urls // and upload a permissioned ssh key to travis before deploying tests there
index := strings.Index(importPath, "/") index := strings.Index(importPath, "/")
gitPath := importPath[:index] + ":" + importPath[index+1:] gitPath := importPath[:index] + ":" + importPath[index+1:]
importURL := "git@" + gitPath + ".git" importURL := "git@" + gitPath + ".git"
depPath := filepath.Join(vendorPath, importPath) err = exec.Command("git", "clone", importURL, dstPath).Run()
err = exec.Command("git", "clone", importURL, depPath).Run()
if err != nil { if err != nil {
return errors.New(fmt.Sprintf("unable to clone transformer dependency from %s: %s", importPath, err.Error())) return errors.New(fmt.Sprintf("unable to clone transformer dependency from %s to %s: %s", importPath, dstPath, err.Error()))
} }
err := os.RemoveAll(filepath.Join(depPath, "vendor/")) } else { // If not on Travis we can work with libs at $GOPATH
srcDir, err := helpers.CleanPath(filepath.Join("$GOPATH/src", importPath))
if err != nil {
return err
}
sp := strings.Split(dstPath, "/")
spj := strings.Join(sp[:len(sp)-1], "/")
err = exec.Command("rsync", "-a", srcDir, spj).Run()
if err != nil {
return errors.New(fmt.Sprintf("unable to copy transformer dependency from %s to %s: %s", srcDir, dstPath, err.Error()))
}
}
// Have to clear out the copied over vendor lib or plugin won't build (see issue above)
err := os.RemoveAll(filepath.Join(dstPath, "vendor"))
if err != nil { if err != nil {
return err return err
} }
// Keep track of this vendor directory to clear later // Keep track of this vendor directory to clear later
b.tmpVenDirs = append(b.tmpVenDirs, depPath) b.tmpVenDirs = append(b.tmpVenDirs, dstPath)
} }
return nil return nil

35
pkg/plugin/generator_suite_test.go
View File

@ -1,35 +0,0 @@
// VulcanizeDB
// Copyright © 2018 Vulcanize
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package plugin_test
import (
"io/ioutil"
"log"
"testing"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
)
func TestRepository(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Plugin Suite Test")
}
var _ = BeforeSuite(func() {
log.SetOutput(ioutil.Discard)
})

356
pkg/plugin/generator_test.go
View File

@ -1,356 +0,0 @@
// VulcanizeDB
// Copyright © 2018 Vulcanize
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package plugin_test
import (
"plugin"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
"github.com/spf13/viper"
"github.com/vulcanize/vulcanizedb/libraries/shared/constants"
"github.com/vulcanize/vulcanizedb/libraries/shared/transformer"
"github.com/vulcanize/vulcanizedb/libraries/shared/watcher"
"github.com/vulcanize/vulcanizedb/pkg/config"
"github.com/vulcanize/vulcanizedb/pkg/core"
"github.com/vulcanize/vulcanizedb/pkg/datastore/postgres"
"github.com/vulcanize/vulcanizedb/pkg/datastore/postgres/repositories"
"github.com/vulcanize/vulcanizedb/pkg/fs"
p2 "github.com/vulcanize/vulcanizedb/pkg/plugin"
"github.com/vulcanize/vulcanizedb/pkg/plugin/helpers"
"github.com/vulcanize/vulcanizedb/pkg/plugin/test_helpers"
)
var eventConfig = config.Plugin{
Transformers: map[string]config.Transformer{
"bite": {
Path: "transformers/bite/initializer",
Type: config.EthEvent,
MigrationPath: "db/migrations",
RepositoryPath: "github.com/vulcanize/mcd_transformers",
},
"deal": {
Path: "transformers/deal/initializer",
Type: config.EthEvent,
MigrationPath: "db/migrations",
RepositoryPath: "github.com/vulcanize/mcd_transformers",
},
},
FileName: "testEventTransformerSet",
FilePath: "$GOPATH/src/github.com/vulcanize/vulcanizedb/pkg/plugin/test_helpers/test",
Save: false,
}
var storageConfig = config.Plugin{
Transformers: map[string]config.Transformer{
"pit": {
Path: "transformers/storage_diffs/maker/pit/initializer",
Type: config.EthStorage,
MigrationPath: "db/migrations",
RepositoryPath: "github.com/vulcanize/mcd_transformers",
},
"vat": {
Path: "transformers/storage_diffs/maker/vat/initializer",
Type: config.EthStorage,
MigrationPath: "db/migrations",
RepositoryPath: "github.com/vulcanize/mcd_transformers",
},
},
FileName: "testStorageTransformerSet",
FilePath: "$GOPATH/src/github.com/vulcanize/vulcanizedb/pkg/plugin/test_helpers/test",
Save: false,
}
var combinedConfig = config.Plugin{
Transformers: map[string]config.Transformer{
"bite": {
Path: "transformers/bite/initializer",
Type: config.EthEvent,
MigrationPath: "db/migrations",
RepositoryPath: "github.com/vulcanize/mcd_transformers",
},
"deal": {
Path: "transformers/deal/initializer",
Type: config.EthEvent,
MigrationPath: "db/migrations",
RepositoryPath: "github.com/vulcanize/mcd_transformers",
},
"pit": {
Path: "transformers/storage_diffs/maker/pit/initializer",
Type: config.EthStorage,
MigrationPath: "db/migrations",
RepositoryPath: "github.com/vulcanize/mcd_transformers",
},
"vat": {
Path: "transformers/storage_diffs/maker/vat/initializer",
Type: config.EthStorage,
MigrationPath: "db/migrations",
RepositoryPath: "github.com/vulcanize/mcd_transformers",
},
},
FileName: "testComboTransformerSet",
FilePath: "$GOPATH/src/github.com/vulcanize/vulcanizedb/pkg/plugin/test_helpers/test",
Save: false,
}
var dbConfig = config.Database{
Hostname: "localhost",
Port: 5432,
Name: "vulcanize_private",
}
type Exporter interface {
Export() ([]transformer.TransformerInitializer, []transformer.StorageTransformerInitializer)
}
var _ = Describe("Generator test", func() {
var g p2.Generator
var goPath, soPath string
var err error
var bc core.BlockChain
var db *postgres.DB
var hr repositories.HeaderRepository
var headerID int64
viper.SetConfigName("compose")
viper.AddConfigPath("$GOPATH/src/github.com/vulcanize/vulcanizedb/environments/")
AfterSuite(func() {
test_helpers.DropTestSchema(db)
})
Describe("Event Transformers only", func() {
BeforeEach(func() {
goPath, soPath, err = eventConfig.GetPluginPaths()
Expect(err).ToNot(HaveOccurred())
g, err = p2.NewGenerator(eventConfig, dbConfig)
Expect(err).ToNot(HaveOccurred())
err = g.GenerateExporterPlugin()
Expect(err).ToNot(HaveOccurred())
})
AfterEach(func() {
err := helpers.ClearFiles(goPath, soPath)
Expect(err).ToNot(HaveOccurred())
})
Describe("GenerateTransformerPlugin", func() {
It("It bundles the specified TransformerInitializers into a Exporter object and creates .so", func() {
plug, err := plugin.Open(soPath)
Expect(err).ToNot(HaveOccurred())
symExporter, err := plug.Lookup("Exporter")
Expect(err).ToNot(HaveOccurred())
exporter, ok := symExporter.(Exporter)
Expect(ok).To(Equal(true))
initializers, store := exporter.Export()
Expect(len(initializers)).To(Equal(2))
Expect(len(store)).To(Equal(0))
})
It("Loads our generated Exporter and uses it to import an arbitrary set of TransformerInitializers that we can execute over", func() {
db, bc = test_helpers.SetupDBandBC()
defer test_helpers.TearDown(db)
hr = repositories.NewHeaderRepository(db)
header1, err := bc.GetHeaderByNumber(9377319)
Expect(err).ToNot(HaveOccurred())
headerID, err = hr.CreateOrUpdateHeader(header1)
Expect(err).ToNot(HaveOccurred())
plug, err := plugin.Open(soPath)
Expect(err).ToNot(HaveOccurred())
symExporter, err := plug.Lookup("Exporter")
Expect(err).ToNot(HaveOccurred())
exporter, ok := symExporter.(Exporter)
Expect(ok).To(Equal(true))
initializers, _ := exporter.Export()
w := watcher.NewEventWatcher(db, bc)
w.AddTransformers(initializers)
err = w.Execute(constants.HeaderMissing)
Expect(err).ToNot(HaveOccurred())
type model struct {
Ilk string
Urn string
Ink string
Art string
IArt string
Tab string
NFlip string
LogIndex uint `db:"log_idx"`
TransactionIndex uint `db:"tx_idx"`
Raw []byte `db:"raw_log"`
Id int64 `db:"id"`
HeaderId int64 `db:"header_id"`
}
returned := model{}
err = db.Get(&returned, `SELECT * FROM maker.bite WHERE header_id = $1`, headerID)
Expect(err).ToNot(HaveOccurred())
Expect(returned.Ilk).To(Equal("4554480000000000000000000000000000000000000000000000000000000000"))
Expect(returned.Urn).To(Equal("0000000000000000000000000000d8b4147eda80fec7122ae16da2479cbd7ffb"))
Expect(returned.Ink).To(Equal("80000000000000000000"))
Expect(returned.Art).To(Equal("11000000000000000000000"))
Expect(returned.IArt).To(Equal("12496609999999999999992"))
Expect(returned.Tab).To(Equal("11000000000000000000000"))
Expect(returned.NFlip).To(Equal("7"))
Expect(returned.TransactionIndex).To(Equal(uint(1)))
Expect(returned.LogIndex).To(Equal(uint(4)))
})
})
})
Describe("Storage Transformers only", func() {
BeforeEach(func() {
goPath, soPath, err = storageConfig.GetPluginPaths()
Expect(err).ToNot(HaveOccurred())
g, err = p2.NewGenerator(storageConfig, dbConfig)
Expect(err).ToNot(HaveOccurred())
err = g.GenerateExporterPlugin()
Expect(err).ToNot(HaveOccurred())
})
AfterEach(func() {
err := helpers.ClearFiles(goPath, soPath)
Expect(err).ToNot(HaveOccurred())
})
Describe("GenerateTransformerPlugin", func() {
It("It bundles the specified StorageTransformerInitializers into a Exporter object and creates .so", func() {
plug, err := plugin.Open(soPath)
Expect(err).ToNot(HaveOccurred())
symExporter, err := plug.Lookup("Exporter")
Expect(err).ToNot(HaveOccurred())
exporter, ok := symExporter.(Exporter)
Expect(ok).To(Equal(true))
event, initializers := exporter.Export()
Expect(len(initializers)).To(Equal(2))
Expect(len(event)).To(Equal(0))
})
It("Loads our generated Exporter and uses it to import an arbitrary set of StorageTransformerInitializers that we can execute over", func() {
db, _ = test_helpers.SetupDBandBC()
defer test_helpers.TearDown(db)
plug, err := plugin.Open(soPath)
Expect(err).ToNot(HaveOccurred())
symExporter, err := plug.Lookup("Exporter")
Expect(err).ToNot(HaveOccurred())
exporter, ok := symExporter.(Exporter)
Expect(ok).To(Equal(true))
_, initializers := exporter.Export()
tailer := fs.FileTailer{Path: viper.GetString("filesystem.storageDiffsPath")}
w := watcher.NewStorageWatcher(tailer, db)
w.AddTransformers(initializers)
// This blocks right now, need to make test file to read from
//err = w.Execute()
//Expect(err).ToNot(HaveOccurred())
})
})
})
Describe("Event and Storage Transformers in same instance", func() {
BeforeEach(func() {
goPath, soPath, err = combinedConfig.GetPluginPaths()
Expect(err).ToNot(HaveOccurred())
g, err = p2.NewGenerator(combinedConfig, dbConfig)
Expect(err).ToNot(HaveOccurred())
err = g.GenerateExporterPlugin()
Expect(err).ToNot(HaveOccurred())
})
AfterEach(func() {
err := helpers.ClearFiles(goPath, soPath)
Expect(err).ToNot(HaveOccurred())
})
Describe("GenerateTransformerPlugin", func() {
It("It bundles the specified TransformerInitializers and StorageTransformerInitializers into a Exporter object and creates .so", func() {
plug, err := plugin.Open(soPath)
Expect(err).ToNot(HaveOccurred())
symExporter, err := plug.Lookup("Exporter")
Expect(err).ToNot(HaveOccurred())
exporter, ok := symExporter.(Exporter)
Expect(ok).To(Equal(true))
eventInitializers, storageInitializers := exporter.Export()
Expect(len(eventInitializers)).To(Equal(2))
Expect(len(storageInitializers)).To(Equal(2))
})
It("Loads our generated Exporter and uses it to import an arbitrary set of TransformerInitializers and StorageTransformerInitializers that we can execute over", func() {
db, bc = test_helpers.SetupDBandBC()
defer test_helpers.TearDown(db)
hr = repositories.NewHeaderRepository(db)
header1, err := bc.GetHeaderByNumber(9377319)
Expect(err).ToNot(HaveOccurred())
headerID, err = hr.CreateOrUpdateHeader(header1)
Expect(err).ToNot(HaveOccurred())
plug, err := plugin.Open(soPath)
Expect(err).ToNot(HaveOccurred())
symExporter, err := plug.Lookup("Exporter")
Expect(err).ToNot(HaveOccurred())
exporter, ok := symExporter.(Exporter)
Expect(ok).To(Equal(true))
eventInitializers, storageInitializers := exporter.Export()
ew := watcher.NewEventWatcher(db, bc)
ew.AddTransformers(eventInitializers)
err = ew.Execute(constants.HeaderMissing)
Expect(err).ToNot(HaveOccurred())
type model struct {
Ilk string
Urn string
Ink string
Art string
IArt string
Tab string
NFlip string
LogIndex uint `db:"log_idx"`
TransactionIndex uint `db:"tx_idx"`
Raw []byte `db:"raw_log"`
Id int64 `db:"id"`
HeaderId int64 `db:"header_id"`
}
returned := model{}
err = db.Get(&returned, `SELECT * FROM maker.bite WHERE header_id = $1`, headerID)
Expect(err).ToNot(HaveOccurred())
Expect(returned.Ilk).To(Equal("4554480000000000000000000000000000000000000000000000000000000000"))
Expect(returned.Urn).To(Equal("0000000000000000000000000000d8b4147eda80fec7122ae16da2479cbd7ffb"))
Expect(returned.Ink).To(Equal("80000000000000000000"))
Expect(returned.Art).To(Equal("11000000000000000000000"))
Expect(returned.IArt).To(Equal("12496609999999999999992"))
Expect(returned.Tab).To(Equal("11000000000000000000000"))
Expect(returned.NFlip).To(Equal("7"))
Expect(returned.TransactionIndex).To(Equal(uint(1)))
Expect(returned.LogIndex).To(Equal(uint(4)))
tailer := fs.FileTailer{Path: viper.GetString("filesystem.storageDiffsPath")}
sw := watcher.NewStorageWatcher(tailer, db)
sw.AddTransformers(storageInitializers)
// This blocks right now, need to make test file to read from
//err = w.Execute()
//Expect(err).ToNot(HaveOccurred())
})
})
})
})

6
pkg/plugin/helpers/helpers.go
View File

@ -62,16 +62,18 @@ func CopyFile(src, dst string) error {
if err != nil { if err != nil {
return err return err
} }
defer in.Close()
out, err := os.OpenFile(dst, syscall.O_CREAT|syscall.O_EXCL|os.O_WRONLY, os.FileMode(0666)) // Doesn't overwrite files out, err := os.OpenFile(dst, syscall.O_CREAT|syscall.O_EXCL|os.O_WRONLY, os.FileMode(0666)) // Doesn't overwrite files
if err != nil { if err != nil {
in.Close()
return err return err
} }
defer out.Close()
_, err = io.Copy(out, in) _, err = io.Copy(out, in)
if err != nil { if err != nil {
in.Close()
out.Close()
return err return err
} }
in.Close()
return out.Close() return out.Close()
} }

2
pkg/plugin/manager/manager.go
View File

@ -88,7 +88,7 @@ func (m *manager) RunMigrations() error {
// Setup a temporary directory to hold transformer db migrations // Setup a temporary directory to hold transformer db migrations
func (m *manager) setupMigrationEnv() error { func (m *manager) setupMigrationEnv() error {
var err error var err error
m.tmpMigDir, err = helpers.CleanPath("$GOPATH/src/github.com/vulcanize/vulcanizedb/db/plugin_migrations") m.tmpMigDir, err = helpers.CleanPath(filepath.Join("$GOPATH/src", m.GenConfig.Home+".plugin_migrations"))
if err != nil { if err != nil {
return err return err
} }

5
pkg/plugin/test_helpers/database.go
View File

@ -17,6 +17,7 @@
package test_helpers package test_helpers
import ( import (
"fmt"
"github.com/ethereum/go-ethereum/ethclient" "github.com/ethereum/go-ethereum/ethclient"
"github.com/ethereum/go-ethereum/rpc" "github.com/ethereum/go-ethereum/rpc"
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
@ -77,7 +78,7 @@ func TearDown(db *postgres.DB) {
Expect(err).NotTo(HaveOccurred()) Expect(err).NotTo(HaveOccurred())
} }
func DropTestSchema(db *postgres.DB) { func DropTestSchema(db *postgres.DB, schema string) {
_, err := db.Exec(`DROP SCHEMA IF EXISTS maker CASCADE`) _, err := db.Exec(fmt.Sprintf(`DROP SCHEMA IF EXISTS %s CASCADE`, schema))
Expect(err).NotTo(HaveOccurred()) Expect(err).NotTo(HaveOccurred())
} }