Commit Graph

238 Commits

Author SHA1 Message Date
Ian Norden
1d50a0ace0 fixed tests for getting balanceOf and allowance, added balanceOf and allowance to the repo and repo tests and updated sql schema with new token_balance and token_allowance tables to accept this data 2018-11-03 13:49:23 -05:00
Rob Mulholand
05186634bd Add light sync command
- Only syncs block headers (excludes block bodies, transactions, receipts, and logs)
- Modifies validation window to include the most recent block
- Isolates validation window to the variable defined in the cmd directory (blocks
  have a separate variable defined in the block_repository for determining when
  to set a block as final)
2018-11-03 13:49:23 -05:00
Elizabeth Engelman
b6f93e735f Add ERC20 token watcher example
- starting with the totalSupply function
- sets contract config on transformer by passing it into the transformer
initializer
- handles block records with the same number for different nodes for both creating token_supply records, and finding missing blocks
2018-11-03 13:49:23 -05:00
Elizabeth Engelman
929643132a Squash price feed trigger update into original migration 2018-11-01 15:12:04 -05:00
Elizabeth Engelman
99dc12c1fd Squash changes to tend and dent tables 2018-11-01 15:03:08 -05:00
Elizabeth Engelman
21239a986d Checking in new schema for reference 2018-11-01 14:56:36 -05:00
Elizabeth Engelman
8ce6ac9b7c Update vow_flog migration for consistency 2018-11-01 14:55:56 -05:00
Elizabeth Engelman
aedcafe488 Update create_vat_heal down migration 2018-11-01 14:13:07 -05:00
Elizabeth Engelman
504cb0ec2d Change timestamp on create_checked_headers_table migration 2018-11-01 14:12:58 -05:00
Edvard
78ad436228 Rename flog -> vow_flog 2018-10-26 15:42:18 +02:00
Edvard
77c7c9b453 Squash frob checked_headers 2018-10-26 15:26:36 +02:00
Edvard
6610821833 Squash cat_file checked_headers 2018-10-26 15:25:27 +02:00
Edvard
d87371cb11 Squash tend checked_headers 2018-10-26 15:24:03 +02:00
Edvard
058ab85518 Squash flip_kick checked_headers 2018-10-26 15:22:27 +02:00
Edvard
393c5297b2 Squash bite checked_headers 2018-10-26 15:21:04 +02:00
Edvard
000c504603 Remove _table from combined vat_init migration 2018-10-26 15:19:44 +02:00
Edvard
48525203f3 Squash dent checked_headers 2018-10-26 15:17:30 +02:00
Edvard
07a0c4f097 Squash deal checked_headers 2018-10-26 15:16:07 +02:00
Edvard
8eb6767da9 Squash drip_drip checked_headers 2018-10-26 15:14:17 +02:00
Edvard
60fec86881 Squash drip_file checked_headers 2018-10-26 15:12:41 +02:00
Edvard
a865aa1aea Squash pit_file checked_headers 2018-10-26 15:09:16 +02:00
Edvard
46a26f5bd9 Squash vat_init checked_headers 2018-10-26 15:08:58 +02:00
Edvard Hübinette
be249437c2 VDB-117: Add Vow.flog transformer (#80)
* Add DB migrations for flog

* Add Vow addresses and constants

* Add err logging to ABI signature getter, test for flog sig

* Add flog transformer

* Add tests+data for flog transformer

* Prepend DB tables and columns with vow_

* Panic on failure to get method sig
2018-10-25 12:19:46 +02:00
Elizabeth
5682ee988b Add log index to various events (#78)
* Add log_index field to flip kick

* Add log index to flop kick

* Add repo tests for Cat.file

* Add log_index to vat_flux

* Add log_index to vat_slip

* Add log_index to vat_toll

* Add log_index to vat_tune

* Add test to repos for updating checked headers
2018-10-23 16:33:04 -05:00
Rob Mulholand
42d60c3cbd Add log index to Vat move 2018-10-23 15:25:21 -05:00
Rob Mulholand
4ac896c246 Add log index to Vat init 2018-10-23 15:25:21 -05:00
Rob Mulholand
702f90367a Add log index to Vat heal 2018-10-23 15:25:21 -05:00
Rob Mulholand
650bbf5874 Add log index to Vat grab 2018-10-23 15:25:21 -05:00
Rob Mulholand
1e74c73591 Add log index to Vat fold 2018-10-23 15:25:21 -05:00
Rob Mulholand
db52955669 Add log index to Tend 2018-10-23 15:25:21 -05:00
Rob Mulholand
e0253cce55 Add log index to Price Feed events 2018-10-23 15:25:21 -05:00
Rob Mulholand
a0a86871b5 Add log index to Frob 2018-10-23 15:25:21 -05:00
Rob Mulholand
b3d00fe3c5 Add log index to Pit file events 2018-10-23 15:25:21 -05:00
Edvard
7d3bf1f0f6 Fixes after rebase 2018-10-23 10:45:48 +02:00
Elizabeth
954fcf2378 integration tests (#75)
* Deal integration test

* Add LogIndex to CatFileChopLump to update unique constraint

* Add LogIndex to cat_file_pit_vow records

* Add integration tests for Cat.file transformers

* Add log index to Cat.file flip
2018-10-22 13:28:42 -05:00
Rob Mulholand
d87fb14c6b Add log index to Drip file events 2018-10-22 12:14:43 -05:00
Rob Mulholand
de294ad8ea Add log index to Drip drip 2018-10-22 12:14:43 -05:00
Rob Mulholand
e5ba0496a6 Add log index to Dent 2018-10-22 12:14:43 -05:00
Rob Mulholand
1aafb7201e Add log index to Deal 2018-10-22 12:14:43 -05:00
Rob Mulholand
495f39abb8 Add log index to Bite 2018-10-22 12:14:43 -05:00
Rob Mulholand
fdbe0b0b44 Add Vat heal integration test 2018-10-19 10:56:28 -05:00
Rob Mulholand
130d11d0aa Add Vat slip transformer 2018-10-17 13:30:35 -05:00
Taka Goto
4d99e8a324 migration for vat flux 2018-10-17 10:53:36 -05:00
Rob Mulholand
d0226dbce5 Convert Bite event field types to numeric
- Enables arithmetic operations when constructing views
- Also removes application-level references to DB ID
2018-10-16 15:27:13 -05:00
Rob Mulholand
d3df685c46 Add timestamp to block header
- Enables quickly deriving timestamp from header for log events
2018-10-12 12:00:02 -05:00
Rob Mulholand
5f67161f41 Convert raw header to JSON
- Enables parsing values off of header via Postgres migration
2018-10-12 12:00:02 -05:00
Edvard Hübinette
be58dd4ac8 Add vat_move transformer (#47)
* Add vat_move transformer base

* Add vat_move migrations

* Add test data for vat_move

* Add vat_move transformer to initialisers

* Add numeric cast to psql insert of Rad

* Add new db schema

* Dependency update

* Expand abbreviation in repository

* Add test suite for vat_move

* Add header checking to transformer and mock repository

* Remove trailing zero in test data

* Fix minor mishaps

* Go fmt nitpicking

* Refactoring in tests

* Add tests covering checked headers stuff (and fix revealed bugs)

* go fmt fixes

* Implement batching behaviour of transformer

* Small fixes after review

* Go fmt
2018-10-12 16:13:13 +02:00
Elizabeth
8fe92edd26 Remove premature urn view migrations (#59) 2018-10-11 14:44:29 -05:00
Elizabeth
6c77f369d9 Vat heal (#56)
* VatHeal Converter

* Add VatHeal repository

* Add VatHeal transformer

* Add VatHeal to continuousLogSync command

* Mark vat_init_checked as true when creating vat init records

* Update urn and v converting

* Return error if Repository.MarkCheckedHeader fails

* Add deleting vat heal from test cleanup method
2018-10-10 11:56:06 -05:00
David Terry
b1745a6403 Merge branch 'staging' into VDB-101-vat-fold-transformer
Conflicts:
	cmd/continuousLogSync.go
	db/schema.sql
	pkg/transformers/shared/constants.go
	pkg/transformers/transformers.go
	test_config/test_config.go
2018-10-10 14:09:14 +03:00
David Terry
f05f2e3ae9 vat.fold: correct table name in down migration 2018-10-10 13:37:31 +03:00
Rob Mulholand
e935f2f553 Add Vat grab transformer 2018-10-09 16:48:19 -05:00
Rob Mulholand
61dca314c4 Add Vat tune transformer 2018-10-09 14:37:26 -05:00
Elizabeth
ed1ed979b3 Remove bid_id unique constraint for Tend and Dent (#48) 2018-10-09 14:32:23 -05:00
Rob Mulholand
8ee106e14d Add Vat toll transformer 2018-10-09 11:40:35 -05:00
David Terry
09770b2e50 Merge branch 'staging' into VDB-101-vat-fold-transformer 2018-10-09 14:15:07 +03:00
Taka Goto
347a339687 update frob and bite converters and repository 2018-10-07 22:02:14 -05:00
Taka Goto
7080c0a74f migration to update columns 2018-10-04 13:56:44 -05:00
David Terry
b3e2218eb5 Merge branch 'staging' into VDB-101-vat-fold-transformer 2018-10-04 18:49:38 +03:00
David Terry
f79a017b48 Vat.fold: store ilk and urn as text instead of bytea 2018-10-04 18:48:06 +03:00
David Terry
2feb7ea546 Vat.fold: add migrations 2018-10-03 12:48:11 +02:00
Rob Mulholand
bf44ec614a Log checked headers for MCD events (#43)
* Log checked headers for Bite events

* Log checked headers for Drip drip events

* Log checked headers for Drip file events

- mark headers checked for 3 implementations of file on the Drip contract

* Log checked headers for Frob frob events

* Log checked headers for Pit file events

- mark headers checked for 3 implementations of file on the Pit contract

* Log checked headers for Vat init events
2018-10-02 17:25:38 -05:00
Rob Mulholand
58f17a24ae Add transformers for Cat file events
- transform logs from 3 implementations of file on the Cat contract
2018-09-27 10:32:08 -05:00
Elizabeth
b0cd852aa3 Flop deal (#39)
*Refactor deal transformer to take in several contract addresses
* Add flop address to deal transformer
2018-09-27 10:12:29 -05:00
Taka Goto
369939ad84 update schema for new price feed subscription migration 2018-09-26 11:45:57 -05:00
Taka Goto
aaec3adf9a update price feed trigger to return new price feed 2018-09-26 11:32:24 -05:00
Elizabeth
31516ea87e Flop kick (#28)
* FlopKick transformer
2018-09-26 09:42:52 -05:00
Rob Mulholand
ac362650db Log checked headers for tend events 2018-09-25 14:11:35 -05:00
Rob Mulholand
009cbb8aa0 Log checked headers for flip kick events 2018-09-25 14:11:35 -05:00
Rob Mulholand
d8b1a61ff5 Log checked headers for dent events 2018-09-25 14:11:35 -05:00
Rob Mulholand
c3f1fcf796 Log checked headers for deal events
- mark header as checked if no matching logs
- mark header as checked if all matching logs persisted successfully
- batch convert and insert matching logs to enable transactional persist
- enable skipping previously checked block headers when restarting a sync
2018-09-25 14:11:35 -05:00
Elizabeth Engelman
6a112e29c4 Update migration names to use unix timestamp 2018-09-19 13:42:15 -05:00
Elizabeth
60901d9095 Update tend, flipkick and dent tables for consistency (#24) 2018-09-18 09:27:14 -05:00
Rob Mulholand
06881db350 Sync unchecked headers for price feeds
- Indicate that a header has been checked for logs if no logs returned
- Indicate that a header has been checked for logs when persisting a log
- Fetch headers that need to be checked based on absence of the above
2018-09-17 14:14:52 -05:00
Takayuki Goto
a6cf173fe7 Merge pull request #11 from 8thlight/add_postgraphile_subscription
Add Postgraphile subscription exposure for price feed
2018-09-17 11:53:47 -05:00
Rob Mulholand
f5595bee78 Add Drip drip transformer 2018-09-13 16:35:10 -05:00
Elizabeth
939ead0c82 Deal transformer (#19) 2018-09-13 09:06:26 -05:00
Rob Mulholand
d69063636e Add raw log to price feeds 2018-09-12 11:35:32 -05:00
Rob Mulholand
b2ba7ee1e3 Add transaction index to flip kick 2018-09-12 11:35:32 -05:00
Rob Mulholand
a843de5eb7 Add drip file transformers
- for all three file functions on the drip contract
2018-09-12 11:35:32 -05:00
Takayuki Goto
c8b002deaa Merge branch 'master' into add_postgraphile_subscription 2018-09-12 09:48:00 -05:00
Taka Goto
13b19c0870 add price feed trigger 2018-09-12 09:43:23 -05:00
Rob Mulholand
ff110592bd Add Vat init transformer 2018-09-11 16:30:29 -05:00
Elizabeth
9abe3ffa68 Dent (#10)
Convert and persist LogNote events as dent records
2018-09-11 14:01:45 -05:00
Rob Mulholand
d48cf38070 Add Pit file debt ceiling event 2018-09-05 13:51:18 -05:00
Rob Mulholand
e1f609528d Add Pit file stability fee event
- file event on the pit contract is overloaded and each implementation
  logs a note
- existing Pit file transformer now specified as dealing with ilks
2018-09-05 11:36:13 -05:00
Rob Mulholand
8f5bc0a956 Add raw log to Frob record in DB
- Enables parsing additional values off the log if necessary
- Also update Frob to reflect DSS changes (lad => urn)
- Move Frob constants to shared directory
2018-09-04 16:11:54 -05:00
Rob Mulholand
db4b891398 Backfill Pit.file log events 2018-09-04 16:07:35 -05:00
Elizabeth
53635ec362 Update flip kick and tend transformers due to contract changes
* Update FlipperAddress to new local ganache address

* Update flip_kick table

* Update flipkick transformer to handle new signature and abi

* Update tend table

* Update tend converter
2018-09-04 15:50:29 -05:00
Takayuki Goto
807b7accd6 Merge pull request #12 from 8thlight/bite-event
bite event transformer
2018-09-04 09:58:51 -05:00
Taka Goto
adc7c47244 update tests and import order per PR feedback 2018-08-30 15:06:14 -05:00
Taka Goto
985fa49178 migration file to create bite; create bite repository
add transaction index and raw log to bite table
work on converter for bite event
update bite repository, replace guy with 32byte lad; create bite converter to entity
update field type for bite event; start on bite transformer
finish bite event transformer
2018-08-30 08:38:43 -05:00
Rob Mulholand
1916d585fa Incorporate DSS updates (#8)
* Incorporate DSS updates
- Lad renamed to Guy
- Dink and Dart added to Frob
* update test chain data
* Remove Mom field from FlipKick
* Update Flip ABI and sample flip kick data
* Incorporate updates to Frob event
- Guy renamed to Lad
- Era and Gem removed, iArt added
- Also turn off integration tests that read from test chain while
events are actively under development
2018-08-28 14:16:39 -05:00
Elizabeth
66ad7e3021 Transform and persist Tend log events 2018-08-16 16:36:35 -05:00
Rob Mulholand
634604d0b5 Combine price feed transformers
- fetches logs from all three price feeds in one query
- assumes eth/usd price feed will be updated to include LogValue event
- updates transformers to run separate from header sync
2018-08-16 11:22:16 -05:00
Rob Mulholand
6e68dc4a92 Add Rep price feed 2018-08-16 11:22:16 -05:00
Rob Mulholand
2949996d22 Add Pip price feed 2018-08-16 11:22:16 -05:00
Rob Mulholand
0551147453 Add Pep price feed 2018-08-16 11:22:16 -05:00
Rob Mulholand
ac63e43543 Backfill Frob log events 2018-08-14 10:47:43 -05:00
Elizabeth
c617cd9c9d Add FlipKick log events transformer 2018-08-07 10:51:34 -05:00
Rob Mulholand
1355271011 Add light sync command
- Only syncs block headers (excludes block bodies, transactions, receipts, and logs)
- Modifies validation window to include the most recent block
- Isolates validation window to the variable defined in the cmd directory (blocks
  have a separate variable defined in the block_repository for determining when
  to set a block as final)
2018-07-18 16:34:12 -05:00
Elizabeth Engelman
7ee253b2a3 Add ERC20 token watcher example
- starting with the totalSupply function
- sets contract config on transformer by passing it into the transformer
initializer
- handles block records with the same number for different nodes for both creating token_supply records, and finding missing blocks
2018-06-27 14:41:51 -05:00
Rob Mulholand
d5c2ab33fc Sync only missing blocks on cold import
= Add eth_node_fingerprint to block that can be imitated by both hot and cold imports
- Only sync missing blocks (blocks that are missing or don't share a fingerprint) on cold import
- Set block is_final status after import
2018-05-07 16:38:04 -05:00
Rob Mulholand
462f94d84a Associate receipts with blocks instead of transactions
- Allows us to fetch receipts by block and persist even when
associated transaction is unknown. (The associated transaction
can still be derived from the tx_hash column at query time, but
is an expensive operation to require for inserts).
2018-05-04 10:52:28 -05:00
Elizabeth Engelman
115b26a76a Add updated schema 2018-04-11 11:02:29 -05:00
Matt Krump
1b4e57a5b4 Rename node table and node column 2018-03-21 13:24:13 -05:00
Matt K
5a652190d9 Allow Parity as ingest node (#36)
* Upgrade go-ethereum to v1.8

* Add Node Info for parity nodes

* Upgrade start_private_blockchain to use v1.8
2018-03-07 15:29:21 -06:00
Matt K
605b0a96ae Add graphql server (#27)
* Add graphql server

* Update Makefile

* Update log_filters constraint

* Add GetLogFilter to repo

* Update travis (use Makefile, go fmt, go vet)

* Add logFilter schema and resolvers

* Add GetWatchedEvent to watched_events_repo

* Add watchedEventLog schema and resolvers
2018-02-08 10:12:08 -06:00
Matt K
d5852654bb Update table columns (#26)
* Update block table names

* Update transaction table names
2018-02-02 16:12:14 -06:00
Matt K
aea9c7b5e2 Separate repositories (#25)
* Separate Repository into multiple Repositories

* Use struct scan for transactions

* Use struct scan for blocks

* Remove unused block repo methods

* Update naming

* Rename / Cleanup repository related fields
2018-02-02 15:53:16 -06:00
Matt Krump
afdeadeca1 Update deps, schema and ci 2018-01-25 19:06:06 -06:00
Matt K
d09c2ae9bb Remove godo (#136) 2018-01-25 15:46:55 -06:00
Matt K
572023cdf5 Commandline (#135)
* Add cmd line tool and Makefile

* Add shared utils pkg

* Add cmdline README

* Update godo for new structure
2018-01-25 13:21:55 -06:00
Matt K
c00b8a5a98 Add Filters (#133)
* Add LogFilter struct

* Add log_filters table

* Add view for events watching

* Add cmd line "add_filter" to mimic eventual endpoint

* Allow multiple filters in config
2018-01-23 12:43:35 -06:00
Matt Krump
6583ce72b8 Remove log_uc
* Logs now are attached to receipt, so removing block + index unique
constraint
2018-01-16 09:44:46 -06:00
Matt Krump
08993cc6a4 Add back schema dump 2018-01-16 09:11:23 -06:00
Matt Krump
82c39a2c1f Add fk constraint on logs 2018-01-16 08:58:11 -06:00
Matt K
a9bea4f492 Allow for multiple Geth nodes (#128) 2018-01-10 15:54:36 -06:00
Matt K
6decf0b54b Remove pubsub and replace w/ polling head of chain (#122)
* Rename geth package structs to not be prefaced with package name

* No longer need to dump schema since Travis uses migrate

* Rearrange history package

* Removed double request for receipt from block rewards

* Remove Listener + Observers and Replace w/ Polling Head

* Potential Short term Issue w/ Infura (ignore these tests for now)
2018-01-05 11:55:00 -06:00
Matt K
4fabe3e917 Add receipts (#119)
* Conversion between Geth Receipt and core.Receipt

* Add receipt to DB

* Insert receipts with transactions

* Update Travis CI to use dep for dependencies
2018-01-03 11:23:43 -06:00
Matt Krump
351d315a4c Add tx fields 2017-12-28 17:23:56 -06:00
Matt Krump
8b024bade9 Add block rewards to db 2017-12-28 11:58:26 -06:00
Matt Krump
cb4e745464 Add extra data field 2017-12-27 12:10:08 -06:00
Matt Krump
c992186846 Add Block Miner 2017-12-27 10:50:56 -06:00
Matt K
a786241c8c Add indices for tx_to and tx_from (#112) 2017-12-20 16:58:37 -06:00
Matt K
24bc83a448 Block categorization (#110)
* Add block categorization (is_final=)

* Add godo task for vulcanizeDB (Example of how everything could work together)

* Add unique constraint on block_number and node

* Add index on block_id for transactions_table

* Add node_id index on blocks table

* Sort transactions returned from FindBlock by tx_hash

* lowercase tx_to, tx_from like etherscan
2017-12-20 14:06:22 -06:00
Matt K
266c9587c8 Canonical blocks (#108)
* Update Block w/ newest Block

* Add cascading delete to blocks and transactions tables

* Add handling for new conflicting blocks

* Command line version of sliding window n behind HEAD
2017-12-19 14:14:41 -06:00
Matt K
a68f277066 Save Logs to DB (#100)
* Save logs to database

* Save multiple logs to db

* Add block number parameter to `FindLogs`
2017-12-12 15:55:26 -06:00
Matt K
921bde1089 Update BlockChain to record NodeInfo (#95) 2017-12-07 13:32:16 -06:00
Matt K
18163f970e Get ABI via etherscan API (#96)
- Added ABI request
- Add unique constraint on contract hash for watched contracts
2017-12-07 09:58:06 -06:00
Eric Meyer
fa2766b64d Store contract ABI on watched_contracts 2017-12-04 14:13:15 -06:00
Matt Krump
aa3318451b Updated to use contracts derived Attributes 2017-11-28 13:43:08 -06:00
Matt K
f0f086e48d Merge pull request #69 from 8thlight/watch_contract
* Add contracts table / start building out watch contracts
2017-11-13 10:55:55 -06:00
Matt Krump
e510154c81 mend 2017-11-13 10:37:09 -06:00
Matt Krump
4ad1d531a8 Update contract naming per Eric PR review 2017-11-13 10:11:27 -06:00
Matt Krump
c7bd6de7da Added From field to transactions 2017-11-09 16:51:22 -06:00
Matt Krump
30fadffb14 Add contracts table / start building out watch contracts 2017-11-09 15:06:01 -06:00
Eric Meyer
aeb1fe8862 Nest migrations under db directory 2017-11-06 13:12:57 -06:00