cerc-io/ipld-eth-server
18
0
Fork 1
Code Issues 10 Pull Requests 1 Actions Packages Projects Releases 37 Wiki Activity

Remove links to Maker

Browse Source
This commit is contained in:
Rob Mulholand 2019-04-01 15:50:43 -05:00
parent 8b5b5b8f17
commit 39471e1d04
2 changed files with 20 additions and 23 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
README.md
View File

@ -1,8 +1,6 @@
# Vulcanize DB # Vulcanize DB
[![Join the chat at https://gitter.im/vulcanizeio/VulcanizeDB](https://badges.gitter.im/vulcanizeio/VulcanizeDB.svg)](https://gitter.im/vulcanizeio/VulcanizeDB?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![Build Status](https://travis-ci.org/vulcanize/vulcanizedb.svg?branch=master)](https://travis-ci.org/vulcanize/vulcanizedb)
[![Build Status](https://travis-ci.com/vulcanize/maker-vulcanizedb.svg?token=MKcE2K7CRvKtdxSSnbap&branch=staging)](https://travis-ci.com/vulcanize/maker-vulcanizedb)
## About ## About
@ -41,7 +39,6 @@ In order to install packages with `dep`, ensure you are in the project directory
After `dep` finishes, dependencies should be installed within your `GOPATH` at the versions specified in `Gopkg.toml`. After `dep` finishes, dependencies should be installed within your `GOPATH` at the versions specified in `Gopkg.toml`.
Because we are working with a modified version of the go-ethereum accounts/abi package, after running `dep ensure` you will need to run `git checkout vendor/github/ethereum/go-ethereum/accounts/abi` to checkout the modified dependency. Because we are working with a modified version of the go-ethereum accounts/abi package, after running `dep ensure` you will need to run `git checkout vendor/github/ethereum/go-ethereum/accounts/abi` to checkout the modified dependency.
This is explained in greater detail [here](https://github.com/vulcanize/maker-vulcanizedb/issues/41).
Lastly, ensure that `GOPATH` is defined in your shell. If necessary, `GOPATH` can be set in `~/.bashrc` or `~/.bash_profile`, depending upon your system. It can be additionally helpful to add `$GOPATH/bin` to your shell's `$PATH`. Lastly, ensure that `GOPATH` is defined in your shell. If necessary, `GOPATH` can be set in `~/.bashrc` or `~/.bash_profile`, depending upon your system. It can be additionally helpful to add `$GOPATH/bin` to your shell's `$PATH`.
@ -318,17 +315,17 @@ Also notice that the contract address used for the schema name has been down-cas
## composeAndExecute ## composeAndExecute
The `composeAndExecute` command is used to compose and execute over an arbitrary set of custom transformers. The `composeAndExecute` command is used to compose and execute over an arbitrary set of custom transformers.
This is accomplished by generating a Go pluggin which allows our `vulcanizedb` binary to link to external transformers, so This is accomplished by generating a Go pluggin which allows our `vulcanizedb` binary to link to external transformers, so
long as they abide by our standard [interfaces](https://github.com/vulcanize/maker-vulcanizedb/tree/compose_and_execute/libraries/shared/transformer). long as they abide by our standard [interfaces](../staging/libraries/shared/transformer).
This command requires Go 1.11+ and [Go plugins](https://golang.org/pkg/plugin/) only work on Unix based systems. This command requires Go 1.11+ and [Go plugins](https://golang.org/pkg/plugin/) only work on Unix based systems.
### Writing custom transformers ### Writing custom transformers
Storage Transformers Storage Transformers
* [Guide](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/factories/storage/README.md) * [Guide](../staging/libraries/shared/factories/storage/README.md)
* [Example](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/factories/storage/EXAMPLE.md) * [Example](../staging/libraries/shared/factories/storage/EXAMPLE.md)
Event Transformers Event Transformers
* [Guide](https://github.com/vulcanize/maker-vulcanizedb/blob/event_docs/libraries/shared/factories/README.md) * [Guide](../staging/libraries/shared/factories/event/README.md)
* [Example](https://github.com/vulcanize/ens_transformers/tree/working) * [Example](https://github.com/vulcanize/ens_transformers/tree/working)
### composeAndExecute configuration ### composeAndExecute configuration
@ -392,12 +389,12 @@ The config provides information for composing a set of transformers:
- `path` is the relative path from `repository` to the transformer's `TransformerInitializer` directory (initializer package). - `path` is the relative path from `repository` to the transformer's `TransformerInitializer` directory (initializer package).
- Transformer repositories need to be cloned into the user's $GOPATH (`go get`) - Transformer repositories need to be cloned into the user's $GOPATH (`go get`)
- `type` is the type of the transformer; indicating which type of watcher it works with (for now, there are only two options: `eth_event` and `eth_storage`) - `type` is the type of the transformer; indicating which type of watcher it works with (for now, there are only two options: `eth_event` and `eth_storage`)
- `eth_storage` indicates the transformer works with the [storage watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/staging/libraries/shared/watcher/storage_watcher.go) - `eth_storage` indicates the transformer works with the [storage watcher](../staging/libraries/shared/watcher/storage_watcher.go)
that fetches state and storage diffs from an ETH node (instead of, for example, from IPFS) that fetches state and storage diffs from an ETH node (instead of, for example, from IPFS)
- `eth_event` indicates the transformer works with the [event watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/staging/libraries/shared/watcher/event_watcher.go) - `eth_event` indicates the transformer works with the [event watcher](../staging/libraries/shared/watcher/event_watcher.go)
that fetches event logs from an ETH node that fetches event logs from an ETH node
- `eth_contract` indicates the transformer works with the [contract watcher](https://github.com/vulcanize/maker-vulcanizedb/blob/omni_update/libraries/shared/watcher/generic_watcher.go) - `eth_contract` indicates the transformer works with the [contract watcher](../staging/libraries/shared/watcher/contract_watcher.go)
that is made to work with [contract_watcher pkg](https://github.com/vulcanize/maker-vulcanizedb/tree/staging/pkg/omni) that is made to work with [contract_watcher pkg](../staging/pkg/contract_watcher)
based transformers which work with either a light or full sync vDB to watch events and poll public methods ([example](https://github.com/vulcanize/ens_transformers/blob/working/transformers/domain_records/transformer.go)) based transformers which work with either a light or full sync vDB to watch events and poll public methods ([example](https://github.com/vulcanize/ens_transformers/blob/working/transformers/domain_records/transformer.go))
- `migrations` is the relative path from `repository` to the db migrations directory for the transformer - `migrations` is the relative path from `repository` to the db migrations directory for the transformer
- `rank` determines the order that migrations are ran, with lower ranked migrations running first - `rank` determines the order that migrations are ran, with lower ranked migrations running first
@ -446,12 +443,12 @@ func (e exporter) Export() []interface1.EventTransformerInitializer, []interface
To plug in an external transformer we need to: To plug in an external transformer we need to:
* Create a [package](https://github.com/vulcanize/ens_transformers/blob/working/transformers/registry/new_owner/initializer/initializer.go) * Create a [package](https://github.com/vulcanize/ens_transformers/blob/working/transformers/registry/new_owner/initializer/initializer.go)
that exports a variable `TransformerInitializer`, `StorageTransformerInitializer`, or `ContractTransformerInitializer` that are of type [TransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/event_transformer.go#L33) that exports a variable `TransformerInitializer`, `StorageTransformerInitializer`, or `ContractTransformerInitializer` that are of type [TransformerInitializer](../staging/libraries/shared/transformer/event_transformer.go#L33)
or [StorageTransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/transformer/storage_transformer.go#L31), or [StorageTransformerInitializer](../staging/libraries/shared/transformer/storage_transformer.go#L31),
or [ContractTransformerInitializer](https://github.com/vulcanize/maker-vulcanizedb/blob/omni_update/libraries/shared/transformer/contract_transformer.go#L31), respectively or [ContractTransformerInitializer](../staging/libraries/shared/transformer/contract_transformer.go#L31), respectively
* Design the transformers to work in the context of their [event](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/event_watcher.go#L83), * Design the transformers to work in the context of their [event](../staging/libraries/shared/watcher/event_watcher.go#L83),
[storage](https://github.com/vulcanize/maker-vulcanizedb/blob/compose_and_execute/libraries/shared/watcher/storage_watcher.go#L53), [storage](../staging/libraries/shared/watcher/storage_watcher.go#L53),
or [contract](https://github.com/vulcanize/maker-vulcanizedb/blob/omni_update/libraries/shared/watcher/contract_watcher.go#L68) watcher execution modes or [contract](../staging/libraries/shared/watcher/contract_watcher.go#L68) watcher execution modes
* Create db migrations to run against vulcanizeDB so that we can store the transformer output * Create db migrations to run against vulcanizeDB so that we can store the transformer output
* Do not `goose fix` the transformer migrations * Do not `goose fix` the transformer migrations
* Specify migration locations for each transformer in the config with the `exporter.transformer.migrations` fields * Specify migration locations for each transformer in the config with the `exporter.transformer.migrations` fields

10
libraries/shared/factories/event/README.md
View File

@ -17,13 +17,13 @@ Looking forward, we will be building fetchers that enable sourcing data from IPF
VulcanizeDB has shared code built out for building and plugging in event transformers VulcanizeDB has shared code built out for building and plugging in event transformers
### [Event Watcher (light sync)](https://github.com/vulcanize/maker-vulcanizedb/blob/staging/libraries/shared/watcher/event_watcher.go) ### [Event Watcher (light sync)](../staging/libraries/shared/watcher/event_watcher.go)
The event watcher is responsible for continuously fetching and delegating chunks of logs and their associated header to the appropriate transformers. The event watcher is responsible for continuously fetching and delegating chunks of logs and their associated header to the appropriate transformers.
Using the `compose` or `composeAndExecute` command, event watchers can be loaded with plugin event transformers and execute over them. Using the `compose` or `composeAndExecute` command, event watchers can be loaded with plugin event transformers and execute over them.
### [Event Transformer](https://github.com/vulcanize/maker-vulcanizedb/blob/staging/libraries/shared/transformer/event_transformer.go) ### [Event Transformer](../staging/libraries/shared/transformer/event_transformer.go)
The event transformer is responsible for converting event logs into more useful data objects and storing them in Postgres. The event transformer is responsible for converting event logs into more useful data objects and storing them in Postgres.
The event transformer is composed of converter and repository interfaces and a config struct: The event transformer is composed of converter and repository interfaces and a config struct:
@ -371,10 +371,10 @@ ALTER TABLE public.checked_headers
-- +goose Down -- +goose Down
DROP TABLE maker.bite; DROP TABLE example_schema.example_event;
ALTER TABLE public.checked_headers ALTER TABLE public.checked_headers
DROP COLUMN bite_checked; DROP COLUMN example_event_checked;
``` ```
Notice that we have also added a column to the `checked_headers` table for this event so that we can keep track Notice that we have also added a column to the `checked_headers` table for this event so that we can keep track
@ -385,4 +385,4 @@ of which headers we have already filtered through for this event.
To create a transformer for a contract event we need to create entities for unpacking the raw log, models to represent To create a transformer for a contract event we need to create entities for unpacking the raw log, models to represent
the final data structure, a converter to mediate this unpacking and conversion between entities to models, a repository to write the final data structure, a converter to mediate this unpacking and conversion between entities to models, a repository to write
these models to Postgres, db migrations to accommodate these models in Postgres, and a EventTransformerInitializer to export the these models to Postgres, db migrations to accommodate these models in Postgres, and a EventTransformerInitializer to export the
configured transformer and load it as a plugin to the `compose` or `composeAndExecute` commands as described in the [main readme](https://github.com/vulcanize/maker-vulcanizedb/blob/staging/README.md#composeandexecute-configuration). configured transformer and load it as a plugin to the `compose` or `composeAndExecute` commands as described in the main readme.