composeAndExecute command that loads and executes over arbitrary transformer set exported from a go plugin generated according to config file; test for plugin generation, loading, and execution; work on plugins README

This commit is contained in:
Ian Norden
2019-02-24 15:23:35 -06:00
parent fa4abe4e5c
commit 6c2d895023
116 changed files with 10554 additions and 454 deletions
+58
View File
@@ -0,0 +1,58 @@
## Plugins
This directory is for Exporter plugins (.go and .so files) generated by, output from, and linked to from the composeAndExecute command
These plugins are generated using information provided in a .toml config file
The config file requires, at a minimum, the below fields:
```toml
[database]
name = "vulcanize_public"
hostname = "localhost"
user = "vulcanize"
password = "vulcanize"
port = 5432
[client]
ipcPath = "http://kovan0.vulcanize.io:8545"
[exporter]
filePath = "$GOPATH/src/github.com/vulcanize/vulcanizedb/plugins/"
fileName = "exporter"
[exporter.transformers]
transformer1 = "github.com/path/to/transformer1"
transformer2 = "github.com/path/to/transformer2"
transformer3 = "github.com/path/to/transformer3"
```
In the above, the exporter.transformers are mappings of import aliases to their import paths
If the individual transformers require additional configuration variables be sure to include them in the .toml file
The general structure of a plugin .go file, and what we would see with the above config is below
Note that `shared_transformer` is a reserved alias needed for the generic TransformerInitializer type:
```go
package main
import (
shared_transformer "github.com/vulcanize/vulcanizedb/libraries/shared/transformer"
transformer1 "github.com/path/to/transformer1"
transformer2 "github.com/path/to/transformer2"
transformer3 "github.com/path/to/transformer3"
)
type exporter string
var Exporter exporter
func (e exporter) Export() []shared_transformer.TransformerInitializer {
return []shared_transformer.TransformerInitializer{
transformer1.TransformerInitializer,
transformer2.TransformerInitializer,
transformer3.TransformerInitializer,
}
}
```
As such, to plug in an external transformer all we need to do is create a [package](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/pkg/autogen/test_helpers/bite/initializer.go) that exports a variable `TransformerInitializer` that is of type [TransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/transformer.go#L19)
As long as the imported transformers abide by the required interfaces, we can execute over any arbitrary set of them