Server backend for indexed ETH IPLD objects
Go to file
2018-09-13 09:06:26 -05:00
cmd add datadog monitoring for vdb 2018-09-04 15:41:24 -05:00
db Deal transformer (#19) 2018-09-13 09:06:26 -05:00
dockerfiles/rinkeby dockerfiles/rinkeby: added named volume for pg container 2018-06-25 23:34:49 +03:00
environments add datadog monitoring for vdb 2018-09-04 15:41:24 -05:00
examples Backfill Frob log events 2018-08-14 10:47:43 -05:00
integration_test Add Pep price feed 2018-08-16 11:22:16 -05:00
libraries/shared Backfill Frob log events 2018-08-14 10:47:43 -05:00
pkg Deal transformer (#19) 2018-09-13 09:06:26 -05:00
scripts Get transactions (#45) 2018-03-27 16:06:12 -05:00
test_config Add raw log to price feeds 2018-09-12 11:35:32 -05:00
utils Handle events 2018-03-05 10:01:50 -06:00
vendor Add Vat init transformer 2018-09-11 16:30:29 -05:00
.gitignore Handle events 2018-03-05 10:01:50 -06:00
.private_blockchain_password Add integration test 2017-10-24 15:36:50 -05:00
.travis.yml Refactoring 2018-08-07 15:17:29 -05:00
Gopkg.lock Add Vat init transformer 2018-09-11 16:30:29 -05:00
Gopkg.toml Allow Parity as ingest node (#36) 2018-03-07 15:29:21 -06:00
LICENSE Add LICENSE 2017-11-09 12:58:17 -06:00
main.go Add Vat init transformer 2018-09-11 16:30:29 -05:00
Makefile Rename dev_env to rinkeby_env, added make commands description to readme 2018-06-21 22:21:34 +03:00
README.md add datadog monitoring for vdb 2018-09-04 15:41:24 -05:00

Vulcanize DB

Join the chat at https://gitter.im/vulcanizeio/VulcanizeDB

Build Status

About

Vulcanize DB is a set of tools that make it easier for developers to write application-specific indexes and caches for dapps built on Ethereum.

Dependencies

Installation

go get github.com/vulcanize/vulcanizedb go get gopkg.in/DataDog/dd-trace-go.v1/ddtrace

Setting up the Database

  1. Install Postgres

  2. Create a superuser for yourself and make sure psql --list works without prompting for a password.

  3. createdb vulcanize_public

  4. cd $GOPATH/src/github.com/vulcanize/vulcanizedb

  5. Run the migrations: make migrate HOST_NAME=localhost NAME=vulcanize_public PORT=5432

    • See below for configuring additional environments

Create a migration file (up and down)

  1. ./script/create_migrate create_bite_table

Configuration

  • To use a local Ethereum node, copy environments/public.toml.example to environments/public.toml and update the ipcPath and levelDbPath.

    • ipcPath should match the local node's IPC filepath:

      • when using geth:

        • The IPC file is called geth.ipc.
        • The geth IPC file path is printed to the console when you start geth.
        • The default location is:
          • Mac: $HOME/Library/Ethereum
          • Linux: $HOME/.ethereum
      • when using parity:

        • The IPC file is called jsonrpc.ipc.
        • The default location is:
          • Mac: $HOME/Library/Application\ Support/io.parity.ethereum/
          • Linux: $HOME/.local/share/io.parity.ethereum/
    • levelDbPath should match Geth's chaindata directory path.

      • The geth LevelDB chaindata path is printed to the console when you start geth.
      • The default location is:
        • Mac: $HOME/Library/Ethereum/geth/chaindata
        • Linux: $HOME/.ethereum/geth/chaindata
      • levelDbPath is irrelevant (and coldImport is currently unavailable) if only running parity.
  • See environments/infura.toml to configure commands to run against infura, if a local node is unavailable.

  • Copy environments/local.toml.example to environments/local.toml to configure commands to run against a local node such as Ganache or ganache-cli.

Start syncing with postgres

Syncs VulcanizeDB with the configured Ethereum node, populating blocks, transactions, receipts, and logs. This command is useful when you want to maintain a broad cache of what's happening on the blockchain.

  1. Start Ethereum node (if fast syncing your Ethereum node, wait for initial sync to finish)
  2. In a separate terminal start VulcanizeDB:
    • ./vulcanizedb sync --config <config.toml> --starting-block-number <block-number>

Alternatively, sync from Geth's underlying LevelDB

Sync VulcanizeDB from the LevelDB underlying a Geth node.

  1. Assure node is not running, and that it has synced to the desired block height.
  2. Start vulcanize_db
    • ./vulcanizedb coldImport --config <config.toml> --starting-block-number <block-number> --ending-block-number <block-number>
  3. Optional flags:
    • --starting-block-number/-s: block number to start syncing from
    • --ending-block-number/-e: block number to sync to
    • --all/-a: sync all missing blocks

Alternatively, sync in "light" mode

Syncs VulcanizeDB with the configured Ethereum node, populating only block headers. This command is useful when you want a minimal baseline from which to track targeted data on the blockchain (e.g. individual smart contract storage values).

  1. Start Ethereum node
  2. In a separate terminal start VulcanizeDB:
    • ./vulcanizedb lightSync --config <config.toml> --starting-block-number <block-number>

Backfill Maker event logs from light sync

Backfills Maker event logs from the configured Ethereum node based on the populated block headers. This includes logs related to auctions, multi-collateral dai, and price feeds. This command requires that a light sync (see command above) has previously been run.

Since auction/mcd contracts have not yet been deployed, this command will need to be run a local blockchain at the moment. As such, a new environment file will need to be added. See environments/local.toml.example.

  1. Start Ethereum node
  2. In a separate terminal run the backfill command:
  • ./vulcanizedb backfillMakerLogs --config <config.toml>

Start full environment in docker by single command

Geth Rinkeby

make command description
rinkeby_env_up start geth, postgres and rolling migrations, after migrations done starting vulcanizedb container
rinkeby_env_deploy build and run vulcanizedb container in rinkeby environment
rinkeby_env_migrate build and run rinkeby env migrations
rinkeby_env_down stop and remove all rinkeby env containers

Success run of the VulcanizeDB container require full geth state sync, attach to geth console and check sync state:

$ docker exec -it rinkeby_vulcanizedb_geth geth --rinkeby attach
...
> eth.syncing
false

If you have full rinkeby chaindata you can move it to rinkeby_vulcanizedb_geth_data docker volume to skip long wait of sync.

Running the Tests

  • make test
  • Note: requires Ganache chain setup and seeded with flip-kick.js and frob.js (in that order)