Commit Graph
171 Commits
Author SHA1 Message Date
Elizabeth Engelman f7d520c933 Update header sync transformer alias in ContractWatcher 2019-05-08 13:42:43 -05:00
Rob Mulholand 36c4da372b (VDB-570) Handle duplicate storage diffs
- If processing a new diff for a row that already exists in the DB,
  ignore the error without logging or queueing
- If processing a queued diff for a row that already exists, remove
  it from the queue
2019-05-02 11:39:16 -05:00
Rob Mulholand 6716c3b92a Make queue recheck interval configurable via CLI 2019-05-01 12:32:39 -05:00
Rob Mulholand 6a86de87b4 (VDB-371) Recheck queued storage
- Iterate through queued storage at defined interval, popping rows
  from the queue if successfully persisted
2019-05-01 12:30:37 -05:00
Rob Mulholand 2d684c5aec Extract storage diff fetching behind an interface
- Replaces directly reading from a CSV
- Simplifies testing
- Should hopefully make it easier to plug in other sources for storage
  diffs (e.g. differently formatted CSVs, JSON RPC, etc)
2019-05-01 12:30:36 -05:00
Gabe Laughlin a344432156 (VDB-560) Rename lightSync to headerSync 2019-05-01 12:12:55 -05:00
Ian Norden 1aa849bcb4 PR fixes; remove all infura token references and setup travis to use encrypted env
variable; rest of the ethjson_rpc dependent tests extracted to integration_test
2019-03-21 18:36:51 -05:00
Ian Norden d3e172d9ab remove references to infura token except for in infura.toml so that tests will continue to pass on CI 2019-03-21 18:36:49 -05:00
Ian Norden 37fc038605 fixes for review comments 2019-03-21 18:36:49 -05:00
Ian Norden 24879a85aa mv pkg/omni pkg/contract_watcher 2019-03-21 18:33:56 -05:00
Ian Norden 073378de6d update © to 2019 2019-03-21 18:33:56 -05:00
Ian Norden e4e092f542 changes to plugin and commands to accomodate changes 2019-03-21 18:33:56 -05:00
Ian Norden 8174ce4aee refactor omni code; rename omniWatcher => contractWatcher; use config instead of excessive number of CLI flags 2019-03-21 18:30:48 -05:00
Ian Norden ff55e3ba7a TransformerInitializer => EventTransformerInitializer reorganization 2019-03-14 11:59:39 -05:00
Ian Norden 9f8c50e3ab tests for migration path ordering and errors 2019-03-13 11:42:30 -05:00
Ian Norden 05cb06c2d3 config example fixes 2019-03-13 11:42:29 -05:00
Ian Norden b3ce1ecd78 update readme and cmd usage comments 2019-03-13 11:42:29 -05:00
Ian Norden 84aa0a7eba plugin migration order specified in config 2019-03-13 11:42:29 -05:00
Rob Mulholand 5eff2618ed Assorted cleanups
- Remove DataDog
- Remove token_supply table
- Drop from more tables when cleaning DB for tests
2019-03-08 11:35:10 -06:00
Ian Norden 8bd397e2eb Merge pull request #29 from vulcanize/fix
Fix for Travis issue at https://github.com/vulcanize/mcd_transformers/pull/6
2019-03-06 11:27:43 -06:00
Ian Norden 5fd9a59027 fix for issue with Travis building plugin from plugin repo 2019-03-05 08:57:54 -06:00
Edvard Hübinette 6ff67f26d1 Fix logrus append (#24) 2019-03-05 11:30:22 +01:00
Ian Norden 83f7daf37d compose and execute as separate commands; update README and add git checkout vendor/github.com/ethereum/go-ethereum/accounts/abi to make build 2019-02-28 18:15:16 -06:00
Ian Norden 2c5ddd03ba core repo migrations in fixed form 2019-02-26 00:52:54 -06:00
Ian Norden 17f4d3dfa5 move maker plugin generator test to mcd_transformers 2019-02-24 21:53:04 -06:00
Ian Norden f2072561a7 removing mcd/maker references; delete test_data; update READMEs; use
logrus
2019-02-24 21:53:04 -06:00
Ian Norden 3d34a9e7c9 remove maker migrations and convert back to timestamps and fix bug in
composeAndExecute command
2019-02-24 21:52:23 -06:00
Ian Norden b449193b16 addressing review comments; still need to reorg migrations and add helper tests 2019-02-24 21:38:48 -06:00
Ian Norden 2868cf2f73 goose changes; update plugin readme; use rel paths for transformers in config; more commenting in pkg/plugin; fix PluginWriter to work with different transformer types; reopen generator tests 2019-02-24 21:38:48 -06:00
Ian Norden 90e67d6da9 move storage mappings pkg into shared 2019-02-24 21:38:48 -06:00
Ian Norden 03a7379617 work on porting storage watcher; watcher type is defined for
transformers in the config, storage transformers are exported from
plugin like event transformers
2019-02-24 21:38:47 -06:00
Ian Norden a322372713 refactoring plugin generating code 2019-02-24 21:38:47 -06:00
Ian Norden decc2a3caf remove pkg/transformers; extract shared files needed to
libraries/shared; work on automated db migration management
2019-02-24 21:38:47 -06:00
Ian Norden 55fa9b8364 auto management of dependencies temporarily needed to build plugin and transformer
migrations; new test showing it is working with external transformers runs
properly when migrations were manually performed but still need
to test automated migration management
2019-02-24 15:23:35 -06:00
Ian Norden 6c2d895023 composeAndExecute command that loads and executes over arbitrary transformer set exported from a go plugin generated according to config file; test for plugin generation, loading, and execution; work on plugins README 2019-02-24 15:23:35 -06:00
Edvard cd6611d2ec Add Cat storage transformer 2019-02-20 14:37:46 +01:00
Edvard Hübinette 53789c82ab Merge pull request #12 from vulcanize/VDB-392-fix-io-error-propagation
VDB-392 Fix IO error propagation
2019-02-20 13:58:00 +01:00
Edvard 6cd4e5ea95 Improve I/O error propagation 2019-02-14 16:03:57 +01:00
Taka Goto c2c21325b5 vow contract storage diffing excluding sin 2019-02-11 15:28:54 -06:00
Edvard Hübinette f73623789e Merge pull request #6 from 8thlight/VDB-337-Dockerize-vDB
VDB-337 Dockerize vDB
2019-02-11 12:37:56 +01:00
Edvard 0bf8bd4183 Add char replacer when searching for env variables 2019-02-11 11:27:33 +01:00
Edvard e4968c4a75 Add back override with env variables 2019-02-11 11:26:12 +01:00
Rob Mulholand c05d3b7836 (VDB-280) Consume vat contract storage diffs 2019-02-08 13:36:30 -06:00
Takayuki Goto 0a024d429d * update transformer to able to recheck headers (#4)
* update transformer to able to recheck headers
* put cap on rechecking header
* integration test for recheck headers
* use enum for recheck headers param; make recheck cap configurable
* update integration tests with new test config
* update omni pkg with new recheck header column type
* update migration with new migration tool and final tweaks needed to accommodate changes in omni pkg
2019-02-08 10:35:46 -06:00
Rob Mulholand 867f92c431 (VDB-298) Consume Pit contract storage diffs
- Continuously parse storage diffs CSV data to read Pit contract state
- Convert ilks in database to raw bytes32 value for use in generating
  storage keys dynamically
- Persist storage diffs with block number and hash for validation
2019-02-06 10:31:46 -06:00
Edvard Hübinette 29f2537130 Merge pull request #7 from vulcanize/VDB-347-Squelch-lightSync
VDB-347 Squelch lightSync in stdout
2019-02-01 11:02:48 +01:00
Edvard 8eea1b00ed Redirect validation window status to logrus instead of stdout 2019-01-31 15:55:28 +01:00
Ian Norden 8749c94994 update license to AGPL-3.0 2019-01-24 14:41:30 -06:00
Ian Norden 1f95eb6443 Merge branch 'staging' into pub_merge 2019-01-24 09:47:12 -06:00
Ian Norden dc87f39889 change omniwatcher database test helpers so that it doesn't interfere with downstream maker tests 2019-01-23 14:55:20 -06:00