Server backend for indexed ETH IPLD objects

Go to file

Matt K c0f737868b Merge pull request #66 from 8thlight/postgres Refactor postgres to be initialized with config		2017-11-09 12:47:34 -06:00
cmd	Refactor postgres to be initialized with config	2017-11-09 12:42:24 -06:00
db	Nest migrations under db directory	2017-11-06 13:12:57 -06:00
Gododir	Refactor config to return an error instead of aborting	2017-11-09 12:41:02 -06:00
integration_test	Refactor config to return an error instead of aborting	2017-11-09 12:41:02 -06:00
pkg	Refactor postgres to be initialized with config	2017-11-09 12:42:24 -06:00
scripts	Add the concept of multiple environments	2017-11-01 12:15:55 -05:00
.gitignore	Nest packages under pkg	2017-11-06 13:06:03 -06:00
.private_blockchain_password	Add integration test	2017-10-24 15:36:50 -05:00
.travis.yml	Fix Travis	2017-11-06 13:22:02 -06:00
Gopkg.lock	Update to use TOML config	2017-11-03 11:53:31 -05:00
Gopkg.toml	Add BlockchainDBObserver	2017-10-25 15:57:05 -05:00
README.md	Add ability to populate missing blocks	2017-11-08 14:52:38 -06:00

README.md

Vulcanize DB

Development Setup

Dependencies

Go 1.9+
https://github.com/golang/dep
- go get -u github.com/golang/dep/cmd/dep
https://github.com/go-godo/godo
- go get -u gopkg.in/godo.v2/cmd/godo
Postgres 10

Cloning the Repository

git config --global url."git@github.com:".insteadOf "https://github.com/"
- By default, go get does not work for private GitHub repos. This will fix that.
go get github.com/8thlight/vulcanizedb
go get github.com/ethereum/go-ethereum
- This will take a while and gives poor indication of progress.
go install github.com/ethereum/go-ethereum/cmd/geth
cd $GOPATH/src/github.com/8thlight/vulcanizedb
dep ensure

Setting up the Databases

Install Postgres
Create a superuser for yourself and make sure psql --list works without prompting for a password.
go get -u -d github.com/mattes/migrate/cli github.com/lib/pq
go build -tags 'postgres' -o /usr/local/bin/migrate github.com/mattes/migrate/cli
createdb vulcanize_private
cd $GOPATH/src/github.com/8thlight/vulcanizedb
godo migrate -- --environment=<some-environment>
- See below for configuring additional environments

Adding a new migration: ./scripts/create_migration <migration-name>

Creating/Using a Private Blockchain

Syncing the public blockchain takes many hours for the initial sync and will download 20+ GB of data. Here are some instructions for creating a private blockchain that does not depend on having a network connection.

Run ./scripts/setup to create a private blockchain with a new account.
- This will result in a warning.
Run ./scripts/start_private_blockchain.
Run godo run -- --environment=private to start listener.

Connecting to the Public Blockchain

./scripts/start_blockchain

IPC File Paths

The default location for Ethereum is:

$HOME/Library/Ethereum for Mac
$HOME/.ethereum for Ubuntu
$GOPATH/src/gihub.com/8thlight/vulcanizedb/test_data_dir/geth.ipc for private blockchain.

Note the location of the ipc file is outputted when you connect to a blockchain. It is needed to for configuration

Running Listener

Start a blockchain.
In a separate terminal start listener (ipcDir location)
- godo run -- --environment=<some-environment>

Retrieving Historical Data

Start a blockchain.
In a separate terminal start listener (ipcDir location)
- godo populateBlocks -- --environment=<some-environment> --starting-number=<starting-block-number>

Configuring Additional Environments

You can create configuration files for additional environments.

Among other things, it will require the IPC file path
See pkg/config/environments/private.toml for an example
You will need to do this if you want to run a node connecting to the public blockchain

Running the Tests

Integration Test

In order to run the integration tests, you will need to run them against a real blockchain.

Run ./scripts/start_private_blockchain as a separate process.
go test ./... to run all tests.

Unit Tests

go test ./pkg/...