Commit Graph

98 Commits

Author SHA1 Message Date
Ian Norden
decc2a3caf remove pkg/transformers; extract shared files needed to
libraries/shared; work on automated db migration management
2019-02-24 21:38:47 -06:00
Ian Norden
6c2d895023 composeAndExecute command that loads and executes over arbitrary transformer set exported from a go plugin generated according to config file; test for plugin generation, loading, and execution; work on plugins README 2019-02-24 15:23:35 -06:00
Rob Mulholand
dbdd16d3a2 (VDB-380) Create separate table for ilks
- reference ilk by foreign key every place it's used
2019-02-21 11:23:00 -06:00
Rob Mulholand
ac9375ae47 Discard logs created during tests
- Make sure output is being set to discard for logrus to reduce clutter in the console
2019-02-12 11:18:39 -06:00
Edvard Hübinette
f73623789e Merge pull request #6 from 8thlight/VDB-337-Dockerize-vDB
VDB-337 Dockerize vDB
2019-02-11 12:37:56 +01:00
Edvard
9b986c4c14 Nuke OpenRC from docker image, update readme's 2019-02-11 11:27:33 +01:00
Edvard
553fdbfa09 Default config starting block to 0 2019-02-11 11:27:33 +01:00
Takayuki Goto
0a024d429d * update transformer to able to recheck headers (#4)
* update transformer to able to recheck headers
* put cap on rechecking header
* integration test for recheck headers
* use enum for recheck headers param; make recheck cap configurable
* update integration tests with new test config
* update omni pkg with new recheck header column type
* update migration with new migration tool and final tweaks needed to accommodate changes in omni pkg
2019-02-08 10:35:46 -06:00
Rob Mulholand
867f92c431 (VDB-298) Consume Pit contract storage diffs
- Continuously parse storage diffs CSV data to read Pit contract state
- Convert ilks in database to raw bytes32 value for use in generating
  storage keys dynamically
- Persist storage diffs with block number and hash for validation
2019-02-06 10:31:46 -06:00
Ian Norden
8749c94994 update license to AGPL-3.0 2019-01-24 14:41:30 -06:00
Konstantin Zolotarev
65cb6e9092 VDB-205 Fix external configuration values (#121)
* Updated loading config from /Users/konstantinzolotarev/.vulcanizedb for addreses

* Updated contract addresses loading from env variables.

* Implement getter chain for addresses, configs and transformers

Solves the issue with declaring stuff as vars when the environment file hasn't been loaded yet. Instead, this data is fetched dynamically when the root commands are called.

* Update test data to use constant Kovan addresses

* Decouple integration tests from config file IO

* Add ABI to environment file, convert getter chains for related values

* Decouple tests suites from ABI config file IO

* Add startingBlock to environment and implement getter chain

* Exit when called without config file

* Add missing deployment blocks to staging.toml

* Remove comment
2019-01-23 19:44:09 +01:00
Taka Goto
24cf0661b0 lowercase address on map to get transformers 2019-01-16 14:24:00 -06:00
yaoandrew
e2b864baa8 Add rollback if GetTicInTx fails and move GetTicInTx to shared/repository.go 2019-01-11 10:07:00 -08:00
Edvard
50cc0c2459 Update tests for watcher and repository 2019-01-07 15:20:05 +01:00
Edvard
833dde62cf Remove injection of fetcher and repository to watcher 2019-01-07 15:19:31 +01:00
Edvard
e54699c039 Remove todo on chunker non-match (it's basically a filter) 2018-12-14 15:22:46 +01:00
Edvard
35f7f719e6 Swap strings.Builder for bytes.Buffer for go 1.9 support 2018-12-13 16:08:57 +01:00
Edvard
129964f3bc Add tests for now shared MissingHeaders 2018-12-13 13:58:16 +01:00
Edvard
d30fcfed80 Use new config getter on shared.Transformer <3 2018-12-13 12:39:57 +01:00
Edvard
882a6dd7a2 Update tests with new adding of transformers 2018-12-12 15:41:44 +01:00
Edvard
dc43547612 Improve process of adding transformers 2018-12-12 15:41:29 +01:00
Edvard
a86a2ec5c6 Improve checked header column detection 2018-12-12 12:13:23 +01:00
Edvard
9000c6e9f9 Add test for watcher log delegation 2018-12-11 15:02:32 +01:00
Edvard
5d4239de05 Finish watcher and watcher tests (+fmt) 2018-12-11 13:52:02 +01:00
Edvard
d93817f346 Interface-ify shared repository 2018-12-11 11:35:13 +01:00
Edvard
83593cb5f2 Cleanup fetcher and add test todos 2018-12-10 21:12:55 +01:00
Edvard
2d81861ae5 Add chunker tests 2018-12-10 21:11:25 +01:00
Edvard
e1ba7ac5b4 WIP update tests for aggregate fetching 2018-12-07 18:10:36 +01:00
Edvard
9789648cc2 Delegate log chunks to respective transformers 2018-12-04 17:40:39 +01:00
Edvard
8bebcdc064 Change transformer.Execute to single header 2018-12-04 17:05:34 +01:00
Edvard
38c745e8c3 Refactor fetching out from repositories to log_fetcher and watcher 2018-12-04 16:04:13 +01:00
Edvard
c26736dc9e WIP continue on aggregate fetching architecture 2018-11-30 17:28:52 +01:00
Edvard
45a087f5d1 WIP: First steps in refactoring to aggregate fetching 2018-11-28 17:00:04 +01:00
Edvard
423fdf01b5 Add log_chunker 2018-11-28 16:59:06 +01:00
Elizabeth
128d20c9bf Vdb 269- fetch logs by hash (#122)
* Upgrade geth from 1.8.15 to 1.8.18

* Update vat_tune to use shared repository methods

* Query blockchain by block hash instead of block number range

* Remove hash validation from repositories

* Fix vow flog integration test

* Update README Travis build sticker

* Update constants formatting per go fmt

* Update EthPublicKeyParser.ParsePublicKey to use discv5.PubkeyID method

* Address PR comments
2018-11-21 09:30:00 -06:00
Edvard
7cd512ae91 Remove PFSF label 2018-11-15 12:14:59 -06:00
Rob Mulholand
854236fd99 (VDB-267) Remove pit file stability fee
- Method is no longer present on the Pit contract
2018-11-15 12:14:59 -06:00
Edvard Hübinette
40408e8300 VDB-120 include tic in dent and tend (#118)
* Add constants for TTL and tic for fakeHeader

* Add tic to tend transformer

* Update tests for tend

* Fix string conversion bug in fakes

* Fix tend integration tests after staging rebase

* Add tic to dent transformer

* Update dent tests

* Change integration tests to use hardcoded block timestamp
2018-11-15 12:57:29 +01:00
Edvard
8f4648a9e3 Add support for never-ending transformer 2018-11-14 10:11:16 -06:00
Rob Mulholand
82fd73ba3f (VDB-68) Verify log block hash matches header block hash
- Delete header on conflict to prompt data refresh (cascade deletes all
  data associated with that block)
- Derive header hash from rpc payload rather than computing it from data
  (prevents hash mismatch between blockchain and cache)
2018-11-13 14:51:39 -06:00
Elizabeth
6a43cd4287 Vdb 207 the rest of the repos (#111)
* Use shared repository test examples for frob

* Use shared repository methods for frob

* Use shared repository test examples for flop kick

* Use shared repository methods for flop kick

* Use shared repository test examples for flip kick

* Use shared repository methods for flip kick

* Use shared repository test examples for bite

* Use shared repository methods for bite

* Alphabetize check headers constants
2018-11-12 10:50:51 -06:00
Edvard Hübinette
4f295b9aec VDB-207 refactor init move split toll tune flog (#110)
* Use shared repo tests for vat_init

* Use shared methods in vat_init repository

* Use shared repo tests for vat_move

* Use shared methods in vat_move repository

* Use shared repo tests for vat_slip

* Use shared methods in vat_slip repository

* Use shared repo tests for vat_toll

* Use shared methods in vat_toll repository

* Use shared methods in vat_tune repository

* Use shared repo tests for vow_flog (+small namespace fix)

* Use shared methods in vow_flog repository

* Go vet + tiny oopsie
2018-11-12 14:17:58 +01:00
Edvard Hübinette
571f300392 Add rendering of available transformer names to continuousLogSync help (#109) 2018-11-12 14:01:07 +01:00
Elizabeth
672269effa VDB 207 refactor more repo tests (#107)
* Use shared repo tests for vat flux

* Use shared repo methods for vat flux

* Use shared repo test examples for vat fold

* Use shared repository methods for vat fold

* Use shared repo test examples for vat grab

* Use shared repo methods for vat grab

* Use shared repo test examples for vat heal

* Use shared repo methods for vat heal
2018-11-09 14:54:22 -06:00
Elizabeth
c1e10f09fb Vdb 67 convert numbers to ray and wad (#105)
* Add method to convert values to ray or wad units

* Convert data to ray or wad for cat_file_chop_lump

* Use shared convert functions in price feed conversion

* Pull common ray/wad values into vars

* Fix after rebase with staging
2018-11-09 14:53:20 -06:00
Rob Mulholand
0b58efb56a Extract common repository methods
- Inject column names to reduce duplication across common behavior
- Extract checked headers column names to constants
2018-11-08 13:08:59 -06:00
Rob Mulholand
8c168ed332 Extract constants package
- Reduce clutter in the shared package
2018-11-08 13:08:59 -06:00
Rob Mulholand
b2a78f1da4 Remove unused transformer config struct 2018-11-07 10:52:55 -06:00
Rob Mulholand
cb141f7cb7 (VDB-204) Add transformer for Flap kick 2018-11-07 10:52:55 -06:00
Rob Mulholand
78b8173b37 Improve test coverage for event method signature lookups 2018-10-26 15:35:18 -05:00