Commit Graph

48 Commits

Author SHA1 Message Date
Ian Norden
decc2a3caf remove pkg/transformers; extract shared files needed to
libraries/shared; work on automated db migration management
2019-02-24 21:38:47 -06:00
Ian Norden
6c2d895023 composeAndExecute command that loads and executes over arbitrary transformer set exported from a go plugin generated according to config file; test for plugin generation, loading, and execution; work on plugins README 2019-02-24 15:23:35 -06:00
Rob Mulholand
dbdd16d3a2 (VDB-380) Create separate table for ilks
- reference ilk by foreign key every place it's used
2019-02-21 11:23:00 -06:00
Rob Mulholand
09abcb841a Persist Urns in raw bytes32 format
- Enables simple usage in contract storage lookups
2019-02-08 11:41:17 -06:00
Takayuki Goto
0a024d429d * update transformer to able to recheck headers (#4)
* update transformer to able to recheck headers
* put cap on rechecking header
* integration test for recheck headers
* use enum for recheck headers param; make recheck cap configurable
* update integration tests with new test config
* update omni pkg with new recheck header column type
* update migration with new migration tool and final tweaks needed to accommodate changes in omni pkg
2019-02-08 10:35:46 -06:00
Rob Mulholand
867f92c431 (VDB-298) Consume Pit contract storage diffs
- Continuously parse storage diffs CSV data to read Pit contract state
- Convert ilks in database to raw bytes32 value for use in generating
  storage keys dynamically
- Persist storage diffs with block number and hash for validation
2019-02-06 10:31:46 -06:00
Ian Norden
8749c94994 update license to AGPL-3.0 2019-01-24 14:41:30 -06:00
Konstantin Zolotarev
65cb6e9092 VDB-205 Fix external configuration values (#121)
* Updated loading config from /Users/konstantinzolotarev/.vulcanizedb for addreses

* Updated contract addresses loading from env variables.

* Implement getter chain for addresses, configs and transformers

Solves the issue with declaring stuff as vars when the environment file hasn't been loaded yet. Instead, this data is fetched dynamically when the root commands are called.

* Update test data to use constant Kovan addresses

* Decouple integration tests from config file IO

* Add ABI to environment file, convert getter chains for related values

* Decouple tests suites from ABI config file IO

* Add startingBlock to environment and implement getter chain

* Exit when called without config file

* Add missing deployment blocks to staging.toml

* Remove comment
2019-01-23 19:44:09 +01:00
Andrew J Yao
c8aa3dc275 Replace standard log with logrus import 2019-01-11 10:07:00 -08:00
yaoandrew
513513e93b refactor error handling in chop_lump 2019-01-11 10:07:00 -08:00
yaoandrew
78b302eab6 refactor error handling in bite transformer 2019-01-11 10:07:00 -08:00
Edvard
833dde62cf Remove injection of fetcher and repository to watcher 2019-01-07 15:19:31 +01:00
Edvard
d93817f346 Interface-ify shared repository 2018-12-11 11:35:13 +01:00
Edvard
47c75d055b Update repository tests for absence of MissingHeaders 2018-12-04 17:53:46 +01:00
Edvard
38c745e8c3 Refactor fetching out from repositories to log_fetcher and watcher 2018-12-04 16:04:13 +01:00
Taka Goto
e8be96a4e7 use logrus for logging 2018-11-21 10:14:11 -06:00
Elizabeth
128d20c9bf Vdb 269- fetch logs by hash (#122)
* Upgrade geth from 1.8.15 to 1.8.18

* Update vat_tune to use shared repository methods

* Query blockchain by block hash instead of block number range

* Remove hash validation from repositories

* Fix vow flog integration test

* Update README Travis build sticker

* Update constants formatting per go fmt

* Update EthPublicKeyParser.ParsePublicKey to use discv5.PubkeyID method

* Address PR comments
2018-11-21 09:30:00 -06:00
Edvard
8f4648a9e3 Add support for never-ending transformer 2018-11-14 10:11:16 -06:00
Rob Mulholand
82fd73ba3f (VDB-68) Verify log block hash matches header block hash
- Delete header on conflict to prompt data refresh (cascade deletes all
  data associated with that block)
- Derive header hash from rpc payload rather than computing it from data
  (prevents hash mismatch between blockchain and cache)
2018-11-13 14:51:39 -06:00
Elizabeth
6a43cd4287 Vdb 207 the rest of the repos (#111)
* Use shared repository test examples for frob

* Use shared repository methods for frob

* Use shared repository test examples for flop kick

* Use shared repository methods for flop kick

* Use shared repository test examples for flip kick

* Use shared repository methods for flip kick

* Use shared repository test examples for bite

* Use shared repository methods for bite

* Alphabetize check headers constants
2018-11-12 10:50:51 -06:00
Rob Mulholand
8c168ed332 Extract constants package
- Reduce clutter in the shared package
2018-11-08 13:08:59 -06:00
Rob Mulholand
b2a78f1da4 Remove unused transformer config struct 2018-11-07 10:52:55 -06:00
Elizabeth
bfe4531d7e Common transformer tests (#97)
* Move chop lump transformer test to a generic log_note_transformer_test

* Add test cases to transformer test

- cat file flip
- cat file pit vow
- deal
- dent

* Move transformer tests over to transformer factor test file

- drip drip
- drip file ilk/repo/vow
- pit file deb ceiling/ilk/stability fee
- price feeds

* Move LogNoteTransformer test into factory package and remove log specific transformer tests

* Move Transformer test into factory package and remove log event specific transformer tests

* Respond to PR comments
2018-11-06 10:30:11 -06:00
Elizabeth
ab4e904713 Frob transformer factory (#95)
* Updates to previous transformers

* Use transformer factory for Frob

* Move bite integration test to integrationt est pkg
2018-10-29 15:08:00 -05:00
Rob Mulholand
d1ee501f12 Use transformer factory for Bite (#89)
* Use transformer factory for Bite

- introduces separate transformer factory for non-LogNote events
- converter includes `ToEntities` for events defined via ABI

* Updates after rebasing with staging
2018-10-26 13:26:10 -05:00
Rob Mulholand
495f39abb8 Add log index to Bite 2018-10-22 12:14:43 -05:00
Rob Mulholand
0ce3e3d829 Add Bite integration test 2018-10-19 14:21:57 -05:00
Rob Mulholand
d0226dbce5 Convert Bite event field types to numeric
- Enables arithmetic operations when constructing views
- Also removes application-level references to DB ID
2018-10-16 15:27:13 -05:00
Rob Mulholand
d3df685c46 Add timestamp to block header
- Enables quickly deriving timestamp from header for log events
2018-10-12 12:00:02 -05:00
Rob Mulholand
5f67161f41 Convert raw header to JSON
- Enables parsing values off of header via Postgres migration
2018-10-12 12:00:02 -05:00
Taka Goto
347a339687 update frob and bite converters and repository 2018-10-07 22:02:14 -05:00
Taka Goto
7e6302c662 update bite to take in string for urn and ilk 2018-10-04 16:35:51 -05:00
Rob Mulholand
bf44ec614a Log checked headers for MCD events (#43)
* Log checked headers for Bite events

* Log checked headers for Drip drip events

* Log checked headers for Drip file events

- mark headers checked for 3 implementations of file on the Drip contract

* Log checked headers for Frob frob events

* Log checked headers for Pit file events

- mark headers checked for 3 implementations of file on the Pit contract

* Log checked headers for Vat init events
2018-10-02 17:25:38 -05:00
Rob Mulholand
58f17a24ae Add transformers for Cat file events
- transform logs from 3 implementations of file on the Cat contract
2018-09-27 10:32:08 -05:00
Elizabeth
b0cd852aa3 Flop deal (#39)
*Refactor deal transformer to take in several contract addresses
* Add flop address to deal transformer
2018-09-27 10:12:29 -05:00
Elizabeth
31516ea87e Flop kick (#28)
* FlopKick transformer
2018-09-26 09:42:52 -05:00
Rob Mulholand
56c45b6841 Add logging for every transformer
- Also add drip file transformer initializer
2018-09-19 11:00:20 -05:00
Rob Mulholand
a735698f77 Update config for Kovan 2018-09-19 11:00:20 -05:00
Rob Mulholand
042f1142c0 Handle headers from POA chain
- Fetching headers from Kovan requires custom type without mixHash/Nonce
2018-09-19 11:00:20 -05:00
Rob Mulholand
dc9bda7d68 Extract addresses to shared constants file 2018-09-11 16:30:29 -05:00
Rob Mulholand
ceeaa0f8d3 Update licenses and imports
- Verify license added to each file
- Order imports by group and separate with line breaks
2018-09-11 16:30:29 -05:00
Elizabeth
9abe3ffa68 Dent (#10)
Convert and persist LogNote events as dent records
2018-09-11 14:01:45 -05:00
Rob Mulholand
a30e5036fd generate log event signatures from functions 2018-09-04 16:19:03 -05:00
Rob Mulholand
db4b891398 Backfill Pit.file log events 2018-09-04 16:07:35 -05:00
Taka Goto
415ec3d97c update from PR feedback 2018-08-31 10:37:02 -05:00
Taka Goto
adc7c47244 update tests and import order per PR feedback 2018-08-30 15:06:14 -05:00
Taka Goto
4ca7c10f92 fix compile error 2018-08-30 14:27:28 -05:00
Taka Goto
985fa49178 migration file to create bite; create bite repository
add transaction index and raw log to bite table
work on converter for bite event
update bite repository, replace guy with 32byte lad; create bite converter to entity
update field type for bite event; start on bite transformer
finish bite event transformer
2018-08-30 08:38:43 -05:00