ipld-eth-server/README.md

# Vulcanize DB

[![Build Status](https://travis-ci.com/8thlight/vulcanizedb.svg?token=3psFYN2533rYjhRbvjte&branch=master)](https://travis-ci.com/8thlight/vulcanizedb)

### Dependencies

 - Go 1.9+
 - Postgres 10 
 - Go Ethereum 
    - https://ethereum.github.io/go-ethereum/downloads/ 
 
### Installation 
    go get github.com/vulcanize/vulcanizedb
    
### Setting up the Databases

1. Install Postgres
2. Create a superuser for yourself and make sure `psql --list` works without prompting for a password.
3. `createdb vulcanize_private`
4. `cd $GOPATH/src/github.com/vulcanize/vulcanizedb`
5. Import the schema: `psql vulcanize_private < db/schema.sql`

   or run the migrations: `make migrate HOST_NAME=localhost NAME=vulcanize_public PORT=5432`
    * See below for configuring additional environments
    
Adding a new migration: `./scripts/create_migration <migration-name>`

## Start syncing with postgres
1. Start geth node (**if fast syncing wait for geth to finsh initial sync**)
2. In a separate terminal start vulcanize_db
    - `vulcanizedb sync --config <config.toml> --starting-block-number <block-numbe>`
    
   * see `environments` for example config 

## Watch specific events
1. Start geth 
2. In a separate terminal start vulcanize_db
    - `vulcanizedb sync --config <config.toml> --starting-block-number <block-numbe>`
3. Create event filter 
    - `vulcanizedb addFilter --config <config.toml> --filter-filepath <filter.json>`
4. The filters are tracked in the `log_filters` table and the filtered events 
will show up in the `watched_log_events` view
   * see `./filters` for example filter 
     
## Development Setup

### Creating/Using a test node

Syncing the against the public network takes many hours for the initial sync and will download 20+ GB of data.
Here are some instructions for creating a private test node that does not depend on having a network connection.

1. Run `./scripts/setup` to create a private blockchain with a new account.
    * This will result in a warning.
2. Run `./scripts/start_private_blockchain`.

### IPC File Paths

The default location for Ethereum is:
 - `$HOME/Library/Ethereum` for Mac
 - `$HOME/.ethereum` for Ubuntu
 - `$GOPATH/src/gihub.com/vulcanize/vulcanizedb/test_data_dir/geth.ipc` for private blockchain.

**Note the location of the ipc file is printed to the console when you start geth. It is needed to for configuration**

## Running the Tests

### Unit Tests

1. `go test ./pkg/...`

### Integration Test

In order to run the integration tests, you will need to run them against a real blockchain. At the moment the integration tests require [Geth v1.7.2](https://ethereum.github.io/go-ethereum/downloads/) as they depend on the `--dev` mode, which changed in v1.7.3 

1. Run `make startprivate` in a separate terminal
2. Setup a test database and import the schema: 

   `createdb vulcanize_private`
   
    `psql vulcanize_private < db/schema.sql`
3. `go test ./...` to run all tests.
Add Travis indicator to readme 2017-10-19 14:45:37 +00:00			`# Vulcanize DB`
WIP 2018-01-25 23:19:15 +00:00
Update deps, schema and ci 2018-01-26 00:10:54 +00:00			`[![Build Status](https://travis-ci.com/8thlight/vulcanizedb.svg?token=3psFYN2533rYjhRbvjte&branch=master)](https://travis-ci.com/8thlight/vulcanizedb)`
Add README 2017-10-19 14:38:47 +00:00
Updates to readme 2017-10-25 14:01:13 +00:00			`### Dependencies`

			`- Go 1.9+`
Block categorization (#110) * Add block categorization (is_final=) * Add godo task for vulcanizeDB (Example of how everything could work together) * Add unique constraint on block_number and node * Add index on block_id for transactions_table * Add node_id index on blocks table * Sort transactions returned from FindBlock by tx_hash * lowercase tx_to, tx_from like etherscan 2017-12-20 20:06:22 +00:00			`- Postgres 10`
			`- Go Ethereum`
			`- https://ethereum.github.io/go-ethereum/downloads/`
Remove godo (#136) 2018-01-25 21:46:55 +00:00
			`### Installation`
Merge old private repo into vulcanize 2018-01-26 00:08:26 +00:00			`go get github.com/vulcanize/vulcanizedb`
Block categorization (#110) * Add block categorization (is_final=) * Add godo task for vulcanizeDB (Example of how everything could work together) * Add unique constraint on block_number and node * Add index on block_id for transactions_table * Add node_id index on blocks table * Sort transactions returned from FindBlock by tx_hash * lowercase tx_to, tx_from like etherscan 2017-12-20 20:06:22 +00:00
Add the concept of multiple environments * Right now, the two environments are Public and Private * This was to reflect the two blockchains we run against 2017-11-01 15:17:01 +00:00			`### Setting up the Databases`
Add instructions for creating the database and the initial table 2017-10-25 15:05:29 +00:00
			`1. Install Postgres`
			2. Create a superuser for yourself and make sure `psql --list` works without prompting for a password.
Remove godo (#136) 2018-01-25 21:46:55 +00:00			3. `createdb vulcanize_private`
Merge old private repo into vulcanize 2018-01-26 00:08:26 +00:00			4. `cd $GOPATH/src/github.com/vulcanize/vulcanizedb`
Update README, add State sync test (#1) 2018-01-26 19:38:14 +00:00			5. Import the schema: `psql vulcanize_private < db/schema.sql`

			or run the migrations: `make migrate HOST_NAME=localhost NAME=vulcanize_public PORT=5432`
Update README to account for environment based changes 2017-11-03 21:52:19 +00:00			`* See below for configuring additional environments`
Remove godo (#136) 2018-01-25 21:46:55 +00:00
Add scripts to help with migrations 2017-10-27 19:18:12 +00:00			Adding a new migration: `./scripts/create_migration <migration-name>`
Add instructions for creating the database and the initial table 2017-10-25 15:05:29 +00:00
Remove godo (#136) 2018-01-25 21:46:55 +00:00			`## Start syncing with postgres`
Update README, add State sync test (#1) 2018-01-26 19:38:14 +00:00			`1. Start geth node (if fast syncing wait for geth to finsh initial sync)`
Block categorization (#110) * Add block categorization (is_final=) * Add godo task for vulcanizeDB (Example of how everything could work together) * Add unique constraint on block_number and node * Add index on block_id for transactions_table * Add node_id index on blocks table * Sort transactions returned from FindBlock by tx_hash * lowercase tx_to, tx_from like etherscan 2017-12-20 20:06:22 +00:00			`2. In a separate terminal start vulcanize_db`
Remove godo (#136) 2018-01-25 21:46:55 +00:00			- `vulcanizedb sync --config <config.toml> --starting-block-number <block-numbe>`

			* see `environments` for example config
Block categorization (#110) * Add block categorization (is_final=) * Add godo task for vulcanizeDB (Example of how everything could work together) * Add unique constraint on block_number and node * Add index on block_id for transactions_table * Add node_id index on blocks table * Sort transactions returned from FindBlock by tx_hash * lowercase tx_to, tx_from like etherscan 2017-12-20 20:06:22 +00:00
Update README, add State sync test (#1) 2018-01-26 19:38:14 +00:00			`## Watch specific events`
			`1. Start geth`
Add Filters (#133) * Add LogFilter struct * Add log_filters table * Add view for events watching * Add cmd line "add_filter" to mimic eventual endpoint * Allow multiple filters in config 2018-01-23 18:43:35 +00:00			`2. In a separate terminal start vulcanize_db`
Remove godo (#136) 2018-01-25 21:46:55 +00:00			- `vulcanizedb sync --config <config.toml> --starting-block-number <block-numbe>`
Add Filters (#133) * Add LogFilter struct * Add log_filters table * Add view for events watching * Add cmd line "add_filter" to mimic eventual endpoint * Allow multiple filters in config 2018-01-23 18:43:35 +00:00			`3. Create event filter`
Remove godo (#136) 2018-01-25 21:46:55 +00:00			- `vulcanizedb addFilter --config <config.toml> --filter-filepath <filter.json>`
Update README, add State sync test (#1) 2018-01-26 19:38:14 +00:00			4. The filters are tracked in the `log_filters` table and the filtered events
			will show up in the `watched_log_events` view
			* see `./filters` for example filter
Add Filters (#133) * Add LogFilter struct * Add log_filters table * Add view for events watching * Add cmd line "add_filter" to mimic eventual endpoint * Allow multiple filters in config 2018-01-23 18:43:35 +00:00
Remove godo (#136) 2018-01-25 21:46:55 +00:00			`## Development Setup`
Update README 2017-10-25 18:27:42 +00:00
Remove godo (#136) 2018-01-25 21:46:55 +00:00			`### Creating/Using a test node`
Updates to readme 2017-10-25 14:01:13 +00:00
Remove godo (#136) 2018-01-25 21:46:55 +00:00			`Syncing the against the public network takes many hours for the initial sync and will download 20+ GB of data.`
			`Here are some instructions for creating a private test node that does not depend on having a network connection.`
Updates to readme 2017-10-25 14:01:13 +00:00
			1. Run `./scripts/setup` to create a private blockchain with a new account.
			`* This will result in a warning.`
			2. Run `./scripts/start_private_blockchain`.

			`### IPC File Paths`

Update README 2017-10-25 18:27:42 +00:00			`The default location for Ethereum is:`
Updates to readme 2017-10-25 14:01:13 +00:00			- `$HOME/Library/Ethereum` for Mac
			- `$HOME/.ethereum` for Ubuntu
replace '8thlight' with 'vulcanize' 2018-01-06 20:31:53 +00:00			- `$GOPATH/src/gihub.com/vulcanize/vulcanizedb/test_data_dir/geth.ipc` for private blockchain.
Add README 2017-10-19 14:38:47 +00:00
Remove godo (#136) 2018-01-25 21:46:55 +00:00			`Note the location of the ipc file is printed to the console when you start geth. It is needed to for configuration`
Update README 2017-10-25 18:27:42 +00:00
Add Travis indicator to readme 2017-10-19 14:45:37 +00:00			`## Running the Tests`
Add README 2017-10-19 14:38:47 +00:00
Block categorization (#110) * Add block categorization (is_final=) * Add godo task for vulcanizeDB (Example of how everything could work together) * Add unique constraint on block_number and node * Add index on block_id for transactions_table * Add node_id index on blocks table * Sort transactions returned from FindBlock by tx_hash * lowercase tx_to, tx_from like etherscan 2017-12-20 20:06:22 +00:00			`### Unit Tests`
Add integration test * Update Travis to run integration tests 2017-10-24 14:24:07 +00:00
Block categorization (#110) * Add block categorization (is_final=) * Add godo task for vulcanizeDB (Example of how everything could work together) * Add unique constraint on block_number and node * Add index on block_id for transactions_table * Add node_id index on blocks table * Sort transactions returned from FindBlock by tx_hash * lowercase tx_to, tx_from like etherscan 2017-12-20 20:06:22 +00:00			1. `go test ./pkg/...`
Add integration test * Update Travis to run integration tests 2017-10-24 14:24:07 +00:00
Block categorization (#110) * Add block categorization (is_final=) * Add godo task for vulcanizeDB (Example of how everything could work together) * Add unique constraint on block_number and node * Add index on block_id for transactions_table * Add node_id index on blocks table * Sort transactions returned from FindBlock by tx_hash * lowercase tx_to, tx_from like etherscan 2017-12-20 20:06:22 +00:00			`### Integration Test`
Add integration test * Update Travis to run integration tests 2017-10-24 14:24:07 +00:00
Block categorization (#110) * Add block categorization (is_final=) * Add godo task for vulcanizeDB (Example of how everything could work together) * Add unique constraint on block_number and node * Add index on block_id for transactions_table * Add node_id index on blocks table * Sort transactions returned from FindBlock by tx_hash * lowercase tx_to, tx_from like etherscan 2017-12-20 20:06:22 +00:00			In order to run the integration tests, you will need to run them against a real blockchain. At the moment the integration tests require [Geth v1.7.2](https://ethereum.github.io/go-ethereum/downloads/) as they depend on the `--dev` mode, which changed in v1.7.3
Add integration test * Update Travis to run integration tests 2017-10-24 14:24:07 +00:00
Update README, add State sync test (#1) 2018-01-26 19:38:14 +00:00			1. Run `make startprivate` in a separate terminal
			`2. Setup a test database and import the schema:`

			`createdb vulcanize_private`

			`psql vulcanize_private < db/schema.sql`
			3. `go test ./...` to run all tests.