Server backend for indexed ETH IPLD objects

Go to file

Rob Mulholand d5c2ab33fc Sync only missing blocks on cold import = Add eth_node_fingerprint to block that can be imitated by both hot and cold imports - Only sync missing blocks (blocks that are missing or don't share a fingerprint) on cold import - Set block is_final status after import		2018-05-07 16:38:04 -05:00
cmd	Sync only missing blocks on cold import	2018-05-07 16:38:04 -05:00
db	Sync only missing blocks on cold import	2018-05-07 16:38:04 -05:00
environments	Add cold import script	2018-05-04 10:54:00 -05:00
integration_test	Update block reading test to resolve periodic fails (#53 )	2018-05-04 10:41:56 -05:00
libraries/shared	Rename node table and node column	2018-03-21 13:24:13 -05:00
pkg	Sync only missing blocks on cold import	2018-05-07 16:38:04 -05:00
scripts	Get transactions (#45 )	2018-03-27 16:06:12 -05:00
test_config	Remove geth --dev dep	2018-03-21 13:57:18 -05:00
utils	Handle events	2018-03-05 10:01:50 -06:00
vendor	Add cold import script	2018-05-04 10:54:00 -05:00
.gitignore	Handle events	2018-03-05 10:01:50 -06:00
.private_blockchain_password	Add integration test	2017-10-24 15:36:50 -05:00
.travis.yml	Get transactions (#45 )	2018-03-27 16:06:12 -05:00
Gopkg.lock	Add cold import script	2018-05-04 10:54:00 -05:00
Gopkg.toml	Allow Parity as ingest node (#36 )	2018-03-07 15:29:21 -06:00
LICENSE	Add LICENSE	2017-11-09 12:58:17 -06:00
main.go	Merge old private repo into vulcanize	2018-01-25 18:08:26 -06:00
Makefile	Get transactions (#45 )	2018-03-27 16:06:12 -05:00
README.md	Sync only missing blocks on cold import	2018-05-07 16:38:04 -05:00

README.md

Vulcanize DB

Dependencies

Go 1.9+
Postgres 10
Ethereum Node
- Go Ethereum (1.8+)
- Parity 1.8.11+

Installation

go get github.com/vulcanize/vulcanizedb

Setting up the Database

Install Postgres
Create a superuser for yourself and make sure psql --list works without prompting for a password.
createdb vulcanize_public
cd $GOPATH/src/github.com/vulcanize/vulcanizedb
Run the migrations: make migrate HOST_NAME=localhost NAME=vulcanize_public PORT=5432
- See below for configuring additional environments

Configuration

To use a local Ethereum node, copy environments/public.toml.example to environments/public.toml and update the ipcPath and levelDbPath.
- ipcPath should match the local node's IPC filepath:
  - when using geth:
    - The IPC file is called geth.ipc.
    - The geth IPC file path is printed to the console when you start geth.
    - The default location is:
      - Mac: $HOME/Library/Ethereum
      - Linux: $HOME/.ethereum
  - when using parity:
    - The IPC file is called jsonrpc.ipc.
    - The default location is:
      - Mac: $HOME/Library/Application\ Support/io.parity.ethereum/
      - Linux: $HOME/.local/share/io.parity.ethereum/
- levelDbPath should match Geth's chaindata directory path.
  - The geth LevelDB chaindata path is printed to the console when you start geth.
  - The default location is:
    - Mac: $HOME/Library/Ethereum/geth/chaindata
    - Linux: $HOME/.ethereum/geth/chaindata
  - levelDbPath is irrelevant (and coldImport is currently unavailable) if only running parity.
See environments/infura.toml to configure commands to run against infura, if a local node is unavailable

Start syncing with postgres

Syncs VulcanizeDB with the configured Ethereum node.

Start node (if fast syncing wait for initial sync to finish)
In a separate terminal start vulcanize_db
- ./vulcanizedb sync --config <config.toml> --starting-block-number <block-number>

Alternatively, sync from Geth's underlying LevelDB

Sync VulcanizeDB from the LevelDB underlying a Geth node.

Assure node is not running, and that it has synced to the desired block height.
Start vulcanize_db
- ./vulcanizedb coldImport --config <config.toml> --starting-block-number <block-number> --ending-block-number <block-number>
Optional flags:
- --starting-block-number/-s: block number to start syncing from
- --ending-block-number/-e: block number to sync to
- --all/-a: sync all missing blocks