review fixes
This commit is contained in:
Ian Norden
parent
42890cdaa7
commit
8e3cf55b7f
65
README.md
65
README.md
@ -7,15 +7,14 @@
|
||||
|
||||
|
||||
## Table of Contents
|
||||
1. [Background](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#background)
|
||||
1. [Dependencies](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#dependencies)
|
||||
1. [Install](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#install)
|
||||
1. [Usage](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#usage)
|
||||
1. [Tests](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#tests)
|
||||
1. [Deploying](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#deploying)
|
||||
1. [API](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#API)
|
||||
1. [Contributing](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#contributing)
|
||||
1. [License](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#license)
|
||||
1. [Background](../staging/README.md#background)
|
||||
1. [Dependencies](../staging/README.md#dependencies)
|
||||
1. [Install](../staging/README.md#install)
|
||||
1. [Usage](../staging/README.md#usage)
|
||||
1. [Tests](../staging/README.md#tests)
|
||||
1. [API](../staging/README.md#API)
|
||||
1. [Contributing](../staging/README.md#contributing)
|
||||
1. [License](../staging/README.md#license)
|
||||
|
||||
|
||||
## Background
|
||||
@ -32,28 +31,28 @@ complicate data accessibility and usability for dApp developers.
|
||||
|
||||
|
||||
## Install
|
||||
1. [Fetching the project](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#fetching-the-project)
|
||||
1. [Installing project dependencies](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#installing-project-dependencies)
|
||||
1. [Configuring shell environment](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#configuring-shell-environment)
|
||||
1. [Setting up the database](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#setting-up-the-database)
|
||||
1. [Configuring a synced Ethereum node](https://github.com/vulcanize/maker-vulcanizedb/tree/readme#configuring-a-synced-ethereum-node)
|
||||
1. [Building the project](../staging/README.md#building-the-project)
|
||||
1. [Setting up the database](../staging/README.md#setting-up-the-database)
|
||||
1. [Configuring a synced Ethereum node](../staging/README.md#configuring-a-synced-ethereum-node)
|
||||
|
||||
### Fetching the project
|
||||
### Building the project
|
||||
Download the codebase to your local `GOPATH` via:
|
||||
|
||||
`go get github.com/vulcanize/vulcanizedb`
|
||||
|
||||
### Installing project dependencies
|
||||
Once fetched, dependencies can be installed via `go get` or (the preferred method) at specific versions via `golang/dep`, the prototype golang pakcage manager. Installation instructions are [here](https://golang.github.io/dep/docs/installation.html).
|
||||
Move to the project directory and use [golang/dep](https://github.com/golang/dep) to install the dependencies:
|
||||
|
||||
In order to install packages with `dep`, ensure you are in the project directory now within your `GOPATH` (default location is `~/go/src/github.com/vulcanize/vulcanizedb/`) and run:
|
||||
`cd $GOPATH/src/github.com/vulcanize/vulcanizedb`
|
||||
|
||||
`dep ensure`
|
||||
|
||||
After `dep` finishes, dependencies should be installed within your `GOPATH` at the versions specified in `Gopkg.toml`.
|
||||
Once the dependencies have been successfully installed, build the executable with:
|
||||
|
||||
### Configuring shell environment
|
||||
Lastly, ensure that `GOPATH` is defined in your shell. If necessary, `GOPATH` can be set in `~/.bashrc` or `~/.bash_profile`, depending upon your system. It can be additionally helpful to add `$GOPATH/bin` to your shell's `$PATH`.
|
||||
`make build`
|
||||
|
||||
If you are running into issues at this stage, ensure that `GOPATH` is defined in your shell.
|
||||
If necessary, `GOPATH` can be set in `~/.bashrc` or `~/.bash_profile`, depending upon your system.
|
||||
It can be additionally helpful to add `$GOPATH/bin` to your shell's `$PATH`.
|
||||
|
||||
### Setting up the database
|
||||
1. Install Postgres
|
||||
@ -95,9 +94,6 @@ In some cases (such as recent Ubuntu systems), it may be necessary to overcome f
|
||||
- Linux: `<full home path>/ethereum/geth/chaindata`
|
||||
- `levelDbPath` is irrelevant (and `coldImport` is currently unavailable) if only running parity.
|
||||
|
||||
- See `environments/infura.toml` to configure commands to run against infura, if a local node is unavailable.
|
||||
- Copy `environments/local.toml.example` to `environments/local.toml` to configure commands to run against a local node such as [Ganache](https://truffleframework.com/ganache) or [ganache-cli](https://github.com/trufflesuite/ganache-clihttps://github.com/trufflesuite/ganache-cli).
|
||||
|
||||
|
||||
## Usage
|
||||
Usage is broken up into two processes:
|
||||
@ -105,41 +101,34 @@ Usage is broken up into two processes:
|
||||
### Data syncing
|
||||
To provide data for transformations, raw Ethereum data must first be synced into vDB.
|
||||
This is accomplished through the use of the `lightSync`, `sync`, or `coldImport` commands.
|
||||
These commands are described in detail [here](https://github.com/vulcanize/maker-vulcanizedb/blob/readme/documentation/sync.md).
|
||||
These commands are described in detail [here](../staging/documentation/sync.md).
|
||||
|
||||
### Data transformation
|
||||
Contract watchers use the raw data that has been synced into Postgres to filter out and apply transformations to specific data of interest.
|
||||
|
||||
There is a built-in `contractWatcher` command which provides generic transformation of most contract data. This command is described in detail [here](https://github.com/vulcanize/maker-vulcanizedb/blob/readme/documentation/contractWatcher.md).
|
||||
There is a built-in `contractWatcher` command which provides generic transformation of most contract data. This command is described in detail [here](../staging/documentation/contractWatcher.md).
|
||||
|
||||
In many cases a custom transformer or set of transformers will need to be written to provide complete or more comprehensive coverage or to optimize other aspects of the output for a specific end-use.
|
||||
In this case we have provided the `compose`, `execute`, and `composeAndExecute` commands for running custom transformers from external repositories. This is described in detail [here](https://github.com/vulcanize/maker-vulcanizedb/blob/readme/documentation/composeAndExecute.md).
|
||||
In this case we have provided the `compose`, `execute`, and `composeAndExecute` commands for running custom transformers from external repositories. This is described in detail [here](../staging/documentation/composeAndExecute.md).
|
||||
|
||||
|
||||
## Tests
|
||||
- Replace the empty `ipcPath` in the `environments/infura.toml` with a path to a full archival node's eth_jsonrpc endpoint (e.g. local geth node ipc path or infura url)
|
||||
- Replace the empty `ipcPath` in the `environments/infura.toml` with a path to a full node's eth_jsonrpc endpoint (e.g. local geth node ipc path or infura url)
|
||||
- Note: integration tests require configuration with an archival node
|
||||
- `createdb vulcanize_private` will create the test db
|
||||
- `make migrate NAME=vulcanize_private` will run the db migrations
|
||||
- `make test` will run the unit tests and skip the integration tests
|
||||
- `make integrationtest` will run just the integration tests
|
||||
- Note: only the integration tests require configuration with an archival node
|
||||
|
||||
|
||||
## Deploying
|
||||
1. you will need to make sure you have ssh agent running and your ssh key added to it. instructions [here](https://developer.github.com/v3/guides/using-ssh-agent-forwarding/#your-key-must-be-available-to-ssh-agent)
|
||||
1. `go get -u github.com/pressly/sup/cmd/sup`
|
||||
1. `sup staging deploy`
|
||||
|
||||
|
||||
## API
|
||||
[Postgraphile](https://www.graphile.org/postgraphile/) is used to expose GraphQL endpoints for our database schemas, this is described in detail [here](../staging/postgraphile/README.md).
|
||||
|
||||
|
||||
## Contributing
|
||||
Contributions are welcome! For more on this, please see [here](https://github.com/vulcanize/maker-vulcanizedb/blob/readme/documentation/contributing.md).
|
||||
Contributions are welcome! For more on this, please see [here](../staging/documentation/contributing.md).
|
||||
|
||||
Small note: If editing the Readme, please conform to the [standard-readme specification](https://github.com/RichardLitt/standard-readme).
|
||||
|
||||
|
||||
## License
|
||||
[AGPL-3.0](https://github.com/vulcanize/maker-vulcanizedb/blob/staging/LICENSE) © Vulcanize Inc
|
||||
[AGPL-3.0](../staging/LICENSE) © Vulcanize Inc
|
||||
@ -12,7 +12,13 @@ Storage Transformers
|
||||
|
||||
Event Transformers
|
||||
* [Guide](../../staging/libraries/shared/factories/event/README.md)
|
||||
* [Example](https://github.com/vulcanize/ens_transformers)
|
||||
* [Example 1](https://github.com/vulcanize/ens_transformers/tree/master/transformers/registar)
|
||||
* [Example 2](https://github.com/vulcanize/ens_transformers/tree/master/transformers/registry)
|
||||
* [Example 3](https://github.com/vulcanize/ens_transformers/tree/master/transformers/resolver)
|
||||
|
||||
Contract Transformers
|
||||
* [Example 1](https://github.com/vulcanize/account_transformers)
|
||||
* [Example 2](https://github.com/vulcanize/ens_transformers/tree/master/transformers/domain_records)
|
||||
|
||||
## Preparing transformers to work as a plugin for composeAndExecute
|
||||
To plug in an external transformer we need to:
|
||||
@ -24,7 +30,7 @@ or [ContractTransformerInitializer](../../staging/libraries/shared/transformer/c
|
||||
[storage](../../staging/libraries/shared/watcher/storage_watcher.go#L53),
|
||||
or [contract](../../staging/libraries/shared/watcher/contract_watcher.go#L68) watcher execution modes
|
||||
3. Create db migrations to run against vulcanizeDB so that we can store the transformer output
|
||||
* Do not `goose fix` the transformer migrations
|
||||
* Do not `goose fix` the transformer migrations, this is to ensure they are always ran after the core vulcanizedb migrations which are kept in their fixed form
|
||||
* Specify migration locations for each transformer in the config with the `exporter.transformer.migrations` fields
|
||||
* If the base vDB migrations occupy this path as well, they need to be in their `goose fix`ed form
|
||||
as they are [here](../../staging/db/migrations)
|
||||
|
||||
@ -1,22 +0,0 @@
|
||||
# Start full environment in docker by single command
|
||||
|
||||
## Geth Rinkeby
|
||||
|
||||
make command | description
|
||||
------------------- | ----------------
|
||||
rinkeby_env_up | start geth, postgres and rolling migrations, after migrations done starting vulcanizedb container
|
||||
rinkeby_env_deploy | build and run vulcanizedb container in rinkeby environment
|
||||
rinkeby_env_migrate | build and run rinkeby env migrations
|
||||
rinkeby_env_down | stop and remove all rinkeby env containers
|
||||
|
||||
Success run of the VulcanizeDB container require full geth state sync,
|
||||
attach to geth console and check sync state:
|
||||
|
||||
```bash
|
||||
$ docker exec -it rinkeby_vulcanizedb_geth geth --rinkeby attach
|
||||
...
|
||||
> eth.syncing
|
||||
false
|
||||
```
|
||||
|
||||
If you have full rinkeby chaindata you can move it to `rinkeby_vulcanizedb_geth_data` docker volume to skip long wait of sync.
|
||||
Loading…
Reference in New Issue
Block a user