ipld-eth-server/plugins/README.md

## Plugins

This directory is for Exporter plugins (.go and .so files) generated by, output from, and linked to from the composeAndExecute command  
These plugins are generated using information provided in a .toml config file 

The config file requires, at a minimum, the below fields:

```toml 
[database]
   name = "vulcanize_public"
   hostname = "localhost"
   user = "vulcanize"
   password = "vulcanize"
   port = 5432

[client]
   ipcPath = "http://kovan0.vulcanize.io:8545"

[exporter]
   name = "exporter"
   [exporter.transformers]
           transformer1 = "path/to/transformer1"
           transformer2 = "path/to/transformer2"
           transformer3 = "path/to/transformer3"
           transformer4 = "path/to/transformer4"
[exporter.types]
           transformer1 = "eth_event"
           transformer2 = "eth_event"
           transformer3 = "eth_event"
           transformer4 = "eth_storage"
   [exporter.repositories]
           transformers = "github.com/account/repo"
           transformer4 = "github.com/account2/repo2"
   [exporter.migrations]
           transformers = "db/migrations"
           transformer4 = "to/db/migrations"
```
- `exporter.transformers` are mappings of import aliases to paths to `TransformerInitializer`s  
    -  Import aliases can be arbitrarily named but note that `interface1` is a reserved alias needed for the generic TransformerInitializer type  
- `exporter.repositores` are the paths to the repositories which contain the transformers   
- `exporter.migrations` are the relative paths to the db migrations found within the `exporter.repositores`  
    - Migrations need to be located in the repos in `exporter.repositores`  
    - Keys should match the keys for the corresponding repo  
    
Note: If any of the imported transformer need additional
config variables do not forget to include those as well  

This information is used to write and build a go plugin with a transformer
set composed from the transformer imports specified in the config file   
This plugin is loaded and the set of transformer initializers is exported
from it and loaded into and executed over by the appropriate watcher   

The type of watcher that the transformer works with is specified using the 
exporter.types config variable as shown above   
Currently there are watchers for event data from an eth node (eth_event) 
and storage data from an eth node (eth_storage)   
In the future there will be watchers for ipfs (ipfs_event and ipfs_storage)   

Transformers of different types can be ran together in the same command using a 
single config file or in separate command instances using different config files   

Specify config location when executing the command:  
`./vulcanizedb composeAndExecute --config=./environments/config_name.toml`  

The general structure of a plugin .go file, and what we would see with the above config is shown below   

```go
package main

import (
	interface1 "github.com/vulcanize/vulcanizedb/libraries/shared/transformer"
	transformer1 "github.com/path/to/transformer1"
	transformer2 "github.com/path/to/transformer2"
	transformer3 "github.com/path/to/transformer3"
	transformer4 "github.com/different/path/to/transformer1"
)

type exporter string

var Exporter exporter

func (e exporter) Export() []interface1.TransformerInitializer, []interface1.StorageTransformerInitializer {
	return []interface1.TransformerInitializer{
		transformer1.TransformerInitializer,
		transformer2.TransformerInitializer,
		transformer3.TransformerInitializer,
	},     []interface1.StorageTransformerInitializer{
		transformer4.StorageTransformerInitializer,
    }
}
```

To plug in an external transformer we need to:
* create a [package](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/pkg/autogen/test_helpers/bite/initializer.go) 
that exports a variable `TransformerInitializer` or `StorageTransformerInitializer` that are of type [TransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/event_transformer.go#L33)  
and [StorageTransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/storage_transformer.go#L31), respectively   
* design the transformers to work in the context of the [event](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/event_watcher.go#L83)
or [storage](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/storage_watcher.go#L53) watchers
* create db migrations to run against vulcanizeDB so that we can store the transformed data   
    * store the db migrations required for a transformer in the same repository as the transformer(s) that require them
    * specify their relative paths in that repo in the config, as discussed above   
    * NOTE: due to a bug with plugin migrations, currently need to leave the `exporter.migrations` blank and manually run migrations before running composeAndExecute