Rob Mulholand
670741616e
Use logrus for logging in populating blocks
...
- Removes logging from test output
2018-12-07 11:24:06 -06:00
Rob Mulholand
2a518ad518
(VDB-254) Convert numerical values for Pit.file ilk
...
- convert data to ray when what == "spot" and wad when what == "line"
2018-12-07 11:24:06 -06:00
Rob Mulholand
087af252c9
(VDB-253) Update numerical values for Pit.file debt ceiling
...
- convert Line to wad
2018-12-07 11:24:06 -06:00
Rob Mulholand
4a5765b20d
(VDB-249) Update numerical values for Vat.fold
...
- convert rate to ray
2018-12-07 11:24:06 -06:00
Edvard
e1ba7ac5b4
WIP update tests for aggregate fetching
2018-12-07 18:10:36 +01:00
Edvard
47c75d055b
Update repository tests for absence of MissingHeaders
2018-12-04 17:53:46 +01:00
Edvard
9789648cc2
Delegate log chunks to respective transformers
2018-12-04 17:40:39 +01:00
Edvard
8bebcdc064
Change transformer.Execute to single header
2018-12-04 17:05:34 +01:00
Edvard
38c745e8c3
Refactor fetching out from repositories to log_fetcher and watcher
2018-12-04 16:04:13 +01:00
Ian Norden
e02b33547d
finishing porting omni watcher to work with light sync; split into full, light,
...
and shared directories and refactor as much into shared; finish
lightSync omni watcher tests
2018-11-30 13:33:48 -06:00
Edvard
c26736dc9e
WIP continue on aggregate fetching architecture
2018-11-30 17:28:52 +01:00
Ian Norden
975f13b969
reorganizing omni directory and beginning light watcher work
2018-11-29 20:33:21 -06:00
Ian Norden
817bd76713
refactoring event converter and repository so that event logs are
...
persisted as they are converted, preventing build up in memory, and finishing method polling (for full sync)
2018-11-29 20:33:21 -06:00
Ian Norden
390a60f7f6
port over lightSync updates from maker repo
2018-11-29 20:33:21 -06:00
Elizabeth
8ce901d1af
Convert drip_file_ilk tax field to ray ( #117 )
2018-11-29 14:10:12 -06:00
Edvard
45a087f5d1
WIP: First steps in refactoring to aggregate fetching
2018-11-28 17:00:04 +01:00
Edvard
423fdf01b5
Add log_chunker
2018-11-28 16:59:06 +01:00
Edvard
f7ecab9dc2
Fix typo
2018-11-28 15:39:46 +01:00
Taka Goto
e8be96a4e7
use logrus for logging
2018-11-21 10:14:11 -06:00
Elizabeth
128d20c9bf
Vdb 269- fetch logs by hash ( #122 )
...
* Upgrade geth from 1.8.15 to 1.8.18
* Update vat_tune to use shared repository methods
* Query blockchain by block hash instead of block number range
* Remove hash validation from repositories
* Fix vow flog integration test
* Update README Travis build sticker
* Update constants formatting per go fmt
* Update EthPublicKeyParser.ParsePublicKey to use discv5.PubkeyID method
* Address PR comments
2018-11-21 09:30:00 -06:00
Elizabeth
b19e77e419
Add integration test for Vow.flog ( #114 )
2018-11-15 16:44:49 -06:00
Ian Norden
417b18ec6a
Edits to address PR issues; change license from apache to AGPL; and work
...
towards generic method polling and reposito;y; config settings to
filterevents/methods by account address; refactoring some stuff out of
repo and into converter; remove fetcher and instead call
blockchain's FetchContractData directly; finishing tests
2018-11-15 12:32:52 -06:00
Edvard
7cd512ae91
Remove PFSF label
2018-11-15 12:14:59 -06:00
Rob Mulholand
854236fd99
(VDB-267) Remove pit file stability fee
...
- Method is no longer present on the Pit contract
2018-11-15 12:14:59 -06:00
Edvard Hübinette
40408e8300
VDB-120 include tic in dent and tend ( #118 )
...
* Add constants for TTL and tic for fakeHeader
* Add tic to tend transformer
* Update tests for tend
* Fix string conversion bug in fakes
* Fix tend integration tests after staging rebase
* Add tic to dent transformer
* Update dent tests
* Change integration tests to use hardcoded block timestamp
2018-11-15 12:57:29 +01:00
Edvard
8f4648a9e3
Add support for never-ending transformer
2018-11-14 10:11:16 -06:00
Rob Mulholand
82fd73ba3f
(VDB-68) Verify log block hash matches header block hash
...
- Delete header on conflict to prompt data refresh (cascade deletes all
data associated with that block)
- Derive header hash from rpc payload rather than computing it from data
(prevents hash mismatch between blockchain and cache)
2018-11-13 14:51:39 -06:00
Elizabeth
6a43cd4287
Vdb 207 the rest of the repos ( #111 )
...
* Use shared repository test examples for frob
* Use shared repository methods for frob
* Use shared repository test examples for flop kick
* Use shared repository methods for flop kick
* Use shared repository test examples for flip kick
* Use shared repository methods for flip kick
* Use shared repository test examples for bite
* Use shared repository methods for bite
* Alphabetize check headers constants
2018-11-12 10:50:51 -06:00
Edvard Hübinette
4f295b9aec
VDB-207 refactor init move split toll tune flog ( #110 )
...
* Use shared repo tests for vat_init
* Use shared methods in vat_init repository
* Use shared repo tests for vat_move
* Use shared methods in vat_move repository
* Use shared repo tests for vat_slip
* Use shared methods in vat_slip repository
* Use shared repo tests for vat_toll
* Use shared methods in vat_toll repository
* Use shared methods in vat_tune repository
* Use shared repo tests for vow_flog (+small namespace fix)
* Use shared methods in vow_flog repository
* Go vet + tiny oopsie
2018-11-12 14:17:58 +01:00
Edvard Hübinette
571f300392
Add rendering of available transformer names to continuousLogSync help ( #109 )
2018-11-12 14:01:07 +01:00
Elizabeth
672269effa
VDB 207 refactor more repo tests ( #107 )
...
* Use shared repo tests for vat flux
* Use shared repo methods for vat flux
* Use shared repo test examples for vat fold
* Use shared repository methods for vat fold
* Use shared repo test examples for vat grab
* Use shared repo methods for vat grab
* Use shared repo test examples for vat heal
* Use shared repo methods for vat heal
2018-11-09 14:54:22 -06:00
Elizabeth
c1e10f09fb
Vdb 67 convert numbers to ray and wad ( #105 )
...
* Add method to convert values to ray or wad units
* Convert data to ray or wad for cat_file_chop_lump
* Use shared convert functions in price feed conversion
* Pull common ray/wad values into vars
* Fix after rebase with staging
2018-11-09 14:53:20 -06:00
Rob Mulholand
0b58efb56a
Extract common repository methods
...
- Inject column names to reduce duplication across common behavior
- Extract checked headers column names to constants
2018-11-08 13:08:59 -06:00
Rob Mulholand
8c168ed332
Extract constants package
...
- Reduce clutter in the shared package
2018-11-08 13:08:59 -06:00
Rob Mulholand
8c5192f82f
(VDB-210) Refactor tend repository tests
2018-11-08 13:08:59 -06:00
Rob Mulholand
1c1a9c699a
(VDB-209) Refactor price feed repository tests
2018-11-08 13:08:59 -06:00
Rob Mulholand
af8bbe5ce9
(VDB-208) Refactor pit file repository tests
2018-11-08 13:08:59 -06:00
Rob Mulholand
a1c4d70a77
(VDB-99) Transform Flap deal events
...
- Add flapper address to deal transformer config
2018-11-07 10:52:55 -06:00
Rob Mulholand
9cc9d4d3b6
(VDB-95) Transform Flop dent events
...
- Add flopper address to dent config
2018-11-07 10:52:55 -06:00
Rob Mulholand
52bbedd837
(VDB-98) Transform Flap tend events
...
- Add Flapper contract address to tend transformer config
2018-11-07 10:52:55 -06:00
Rob Mulholand
b2a78f1da4
Remove unused transformer config struct
2018-11-07 10:52:55 -06:00
Rob Mulholand
cb141f7cb7
(VDB-204) Add transformer for Flap kick
2018-11-07 10:52:55 -06:00
Taka Goto
627ae288bd
use self hosted parity kovan for staging
2018-11-06 12:58:45 -06:00
Takayuki Goto
7afd978cf0
Merge pull request #100 from 8thlight/update-integration-tests
...
integration tests hits self hosted remote parity kovan node:
2018-11-06 12:08:48 -06:00
Elizabeth
0ab5ea1b25
Refactor more repo tests ( #101 )
...
* Use shared repository tests for deal
* Small updates to repo tests
* Use shared repository test examples for dent
* Use shared repository test examples for drip_drip
* Use shared repository test examples for drip_file_ilk
* Use shared repository test examples for drip_file_repo
* Use shared repository test examples for drip_file_vow
* Update formatting
2018-11-06 10:51:26 -06:00
Elizabeth
a4abf3781e
Refactor repository tests ( #99 )
...
* Add shared test examples for repositories
- Create
- MissingHeaders
- MarkHeaderChecked
* Use shared repository tests for cat_file_chop_lump
* Use shared repository test for cat_file_flip
* Use shared repository test for cat_file_pit_vow
* Updates to shared repository tests
* Use shared repository tests for deal
* Small updates to repo tests
2018-11-06 10:30:31 -06:00
Elizabeth
bfe4531d7e
Common transformer tests ( #97 )
...
* Move chop lump transformer test to a generic log_note_transformer_test
* Add test cases to transformer test
- cat file flip
- cat file pit vow
- deal
- dent
* Move transformer tests over to transformer factor test file
- drip drip
- drip file ilk/repo/vow
- pit file deb ceiling/ilk/stability fee
- price feeds
* Move LogNoteTransformer test into factory package and remove log specific transformer tests
* Move Transformer test into factory package and remove log event specific transformer tests
* Respond to PR comments
2018-11-06 10:30:11 -06:00
Taka Goto
2cc9826be9
create integration toml file
2018-11-06 09:25:08 -06:00
Ian Norden
b459cf35ed
beginning work on method polling; first need to generate list of token holder address in a completely generic/contratc agnostic fashion. Created address retriever that can iterate through any given contract's watched events, finding the inputs/arguments with address type, and generate a list from those values. Edit: Contract objects now cache every event emitted address as its event logs are transformed into the repo to grow a list of contract associated addresses as we go
2018-11-04 21:37:31 -06:00
Ian Norden
e9dbd771e5
tests and fixes for fetcher, parser, retriever, converter, and repository; update cmd and watcher
2018-11-04 01:49:11 -05:00
Ian Norden
8ce75fe5ad
Generic watcher that takes a contract address, grabs the contract abi and starting block number, creates custom event filters, and extracts and transforms event data into postgres. Can configure to look at only a subset of events through CLI flag. Building but needs testing.
2018-11-03 14:00:25 -05:00
Ian Norden
9b41000b88
Moved fetcher to generic directory (methods have to remain public since it is in seperate package now), added FetchHash method, created ERC20 and generic getters which call the fetcher with specific contract methods (GetTotalSupply, GetBalance, GetAllowance for ERC20 getter, and GetOwner, GetStoppedStatus, GetStringName, GetHashName, GetStringSymbol, GetHashSymbol, and GetDecimals for generic getter). Getter tests cover all but GetBalance and GetAllowance, and also cover all of the Fetcher methods- but with only nil methodArgs. GetAllowance and GetBalance tests are not working against infura and these are the only contract method calls with arguments passed in so I suspect this might be where the issue lies. Have tested GetBalance using previous version of FetchContractData without the variadic input to the Pack method and it fails with the same error so I don’t think it is due to those changes.
2018-11-03 13:49:23 -05:00
Ian Norden
ccd21f73c9
adjusted FetchContractData to work with variable number of contract method arguments, changed fetcher's FetchSupply method to more general FetchBigInt method and added FetchBool and FetchAddress methods
2018-11-03 13:49:23 -05:00
Rob Mulholand
ba071ef13f
Consolidate test doubles
...
- Migrate various mocks of core namespaces to shared version in `fakes` pkg
- Err on the side of making test doubles less sophisticated
- Don't pull over mocks of namespaces that are only used in example code
2018-11-03 13:49:23 -05:00
Rob Mulholand
5fe6394406
Add tests for pkg/geth/blockchain
...
- inject dependencies instead of initializing them in the constructor
2018-11-03 13:49:23 -05:00
Rob Mulholand
05186634bd
Add light sync command
...
- Only syncs block headers (excludes block bodies, transactions, receipts, and logs)
- Modifies validation window to include the most recent block
- Isolates validation window to the variable defined in the cmd directory (blocks
have a separate variable defined in the block_repository for determining when
to set a block as final)
2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
7b507667f8
Added user and password to db config
2018-11-03 13:49:23 -05:00
Takayuki Goto
0d325afbfb
flop kick transformer factory ( #98 )
...
* flop kick transformer factory
* add tests for wrong model/entity
2018-11-02 09:13:10 -05:00
Elizabeth
981393d0a7
Use transformer factory for FlipKick ( #96 )
2018-11-02 09:12:51 -05:00
Taka Goto
3b722c132e
integration tests hits self hosted parity kovan node:
2018-11-01 10:24:03 -05:00
Elizabeth
ab4e904713
Frob transformer factory ( #95 )
...
* Updates to previous transformers
* Use transformer factory for Frob
* Move bite integration test to integrationt est pkg
2018-10-29 15:08:00 -05:00
Rob Mulholand
78b8173b37
Improve test coverage for event method signature lookups
2018-10-26 15:35:18 -05:00
Rob Mulholand
602802e5f4
Don't test order of DB results
...
- reduce flakiness by enabling tests to pass regardless of result order
2018-10-26 15:35:18 -05:00
Rob Mulholand
ed624330f6
Rename Flog pack to Vow flog
...
- enables differentiation if another contract implements flog
- also deletes flog-specific mocks
2018-10-26 15:35:18 -05:00
Rob Mulholand
1b680d2a4b
Use transformer factory for Drip drip
2018-10-26 15:35:18 -05:00
Edvard Hübinette
f3a82e6ba1
VDB-184 Refactor Dent ( #91 )
...
* Factorise dent transformer
* Update dent transformer tests
* Add additional tests to dent repository test
* Update dent to use LogNoteTransformer factory
2018-10-26 13:56:49 -05:00
Rob Mulholand
d1ee501f12
Use transformer factory for Bite ( #89 )
...
* Use transformer factory for Bite
- introduces separate transformer factory for non-LogNote events
- converter includes `ToEntities` for events defined via ABI
* Updates after rebasing with staging
2018-10-26 13:26:10 -05:00
Rob Mulholand
0e5b23993a
Discard log output from Vat heal during tests
2018-10-26 12:02:56 -05:00
Rob Mulholand
45a52a59f9
Add repository tests for Deal
2018-10-26 12:02:56 -05:00
Edvard
0e1675354f
Update tests for deal transformer
2018-10-26 12:02:56 -05:00
Edvard
35a8009410
Refactor deal transformer to use factory
2018-10-26 12:02:56 -05:00
Elizabeth
d522d33531
Update vat heal to use transformer factory ( #88 )
...
* Update vat heal to use transformer factory
2018-10-26 11:40:24 -05:00
Rob Mulholand
5de949f4c5
Add cat file repository tests
2018-10-26 11:36:02 -05:00
Edvard
00bb48bafb
Fix whitespace errors
2018-10-26 11:36:02 -05:00
Edvard
e6e2ed3308
Update cat file transformer tests
2018-10-26 11:36:02 -05:00
Edvard
28ab5f92ab
Factorise cat_file transformers
2018-10-26 11:36:02 -05:00
Rob Mulholand
47a3c35938
Use transformer factory for Price Feed events
2018-10-26 11:25:16 -05:00
Elizabeth
bd428e752a
Update vat grab to use transformer factory ( #85 )
...
* Update vat grab to use transformer factory
2018-10-26 11:02:39 -05:00
Rob Mulholand
745b3a460e
Add test for updating existing checked header row
2018-10-26 10:45:49 -05:00
Rob Mulholand
e37447ae9a
Use transformer factory for Vat fold
2018-10-26 10:45:49 -05:00
Takayuki Goto
0e7f316568
flip kick integration test ( #83 )
...
* flip kick integration test
* use timestamp instead to compare date
* fix after rebase staging
2018-10-26 10:26:22 -05:00
Rob Mulholand
3ec5b4afab
Remove unused param in integration test helper
2018-10-25 10:02:06 -05:00
Rob Mulholand
8831586fba
Extract shared mock converter and repository
2018-10-25 10:02:06 -05:00
Rob Mulholand
f5b9a21a8c
Use transformer factory for Vat slip
2018-10-25 10:02:06 -05:00
Rob Mulholand
1e7ee08af5
Use transformer factory for Vat toll
2018-10-25 10:02:06 -05:00
Rob Mulholand
6d662a6ce7
Add test for can't create Vat tune of wrong type
2018-10-25 10:02:06 -05:00
Rob Mulholand
a0cae99437
Use transformer factory for Vat flux
2018-10-25 10:02:06 -05:00
Edvard Hübinette
be249437c2
VDB-117: Add Vow.flog
transformer ( #80 )
...
* Add DB migrations for flog
* Add Vow addresses and constants
* Add err logging to ABI signature getter, test for flog sig
* Add flog transformer
* Add tests+data for flog transformer
* Prepend DB tables and columns with vow_
* Panic on failure to get method sig
2018-10-25 12:19:46 +02:00
Rob Mulholand
78d746bb5b
Use transformer factory for Vat tune
2018-10-24 15:49:19 -05:00
Elizabeth
5682ee988b
Add log index to various events ( #78 )
...
* Add log_index field to flip kick
* Add log index to flop kick
* Add repo tests for Cat.file
* Add log_index to vat_flux
* Add log_index to vat_slip
* Add log_index to vat_toll
* Add log_index to vat_tune
* Add test to repos for updating checked headers
2018-10-23 16:33:04 -05:00
Takayuki Goto
28f8861f81
Merge pull request #71 from 8thlight/VDB-150-vat-flux-integration-test
...
integration test for vat flux
2018-10-23 16:01:42 -05:00
Rob Mulholand
42d60c3cbd
Add log index to Vat move
2018-10-23 15:25:21 -05:00
Rob Mulholand
4ac896c246
Add log index to Vat init
2018-10-23 15:25:21 -05:00
Rob Mulholand
702f90367a
Add log index to Vat heal
2018-10-23 15:25:21 -05:00
Rob Mulholand
650bbf5874
Add log index to Vat grab
2018-10-23 15:25:21 -05:00
Rob Mulholand
1e74c73591
Add log index to Vat fold
2018-10-23 15:25:21 -05:00
Rob Mulholand
db52955669
Add log index to Tend
2018-10-23 15:25:21 -05:00
Rob Mulholand
e0253cce55
Add log index to Price Feed events
2018-10-23 15:25:21 -05:00
Rob Mulholand
a0a86871b5
Add log index to Frob
2018-10-23 15:25:21 -05:00
Rob Mulholand
b3d00fe3c5
Add log index to Pit file events
2018-10-23 15:25:21 -05:00
Edvard
a0deb5eb62
Add and use transformer labels in shared constants
2018-10-23 11:23:31 +02:00
Edvard
f55449d0e5
Replace GinkgoRandomSeed with proper randomisation
2018-10-23 10:47:01 +02:00
Edvard
794a7e5f99
Nuke some leftover converter interfaces
2018-10-23 10:47:01 +02:00
Edvard
6d9547a888
Fix invalidated test for tend repository
2018-10-23 10:47:01 +02:00
Edvard
7ee3329baf
Privatise DB repository field and add transformer test
2018-10-23 10:47:01 +02:00
Edvard
cb74cbe990
Add cast error checking to refactored transformers
2018-10-23 10:47:01 +02:00
Edvard
b7f8432e65
Fix git whitespace warnings
2018-10-23 10:47:00 +02:00
Edvard
bb204b6b14
Remove unnecessary type washing in factory transformer
2018-10-23 10:45:48 +02:00
Edvard
e7e511764e
Implement skip to next header on empty matchingLogs
2018-10-23 10:45:48 +02:00
Edvard
33f1f62c11
Implement SingleTransformerConfig
2018-10-23 10:45:48 +02:00
Edvard
7d3bf1f0f6
Fixes after rebase
2018-10-23 10:45:48 +02:00
Edvard
70722fc9e3
Add todo for changing to single topic in shared transformer config
2018-10-23 10:45:48 +02:00
Edvard
2a9fafca08
Update drip_file tests
2018-10-23 10:45:48 +02:00
Edvard
dc4eb009c1
Refactor drip_file transformers
2018-10-23 10:45:48 +02:00
Edvard
ccca6e0229
Update old pit file shared config references
2018-10-23 10:43:03 +02:00
Edvard
d9b6603109
Update pit file debt ceiling tests
2018-10-23 10:43:03 +02:00
Edvard
24199cb67e
Refactor pit file debt ceiling transformer
2018-10-23 10:43:03 +02:00
Edvard
0239c4daf6
Fix sneaky missing asterisk
2018-10-23 10:43:03 +02:00
Edvard
399b3b78a0
Update tests for pit file ilk transformer
2018-10-23 10:43:03 +02:00
Edvard
98062a7970
Refactor pit file ilk transfomer
2018-10-23 10:43:03 +02:00
Edvard
837a777c8a
Update transformer name for stability fee
2018-10-23 10:43:03 +02:00
Edvard
3412866182
Update tests for stability_fee
2018-10-23 10:43:03 +02:00
Edvard
2cc303a2f5
Refactor stability_fee transfomer
2018-10-23 10:43:03 +02:00
Edvard
57b3744085
Update tend tests
2018-10-23 10:43:03 +02:00
Edvard
a4df8f348d
Refactor Tend transformer using factory
2018-10-23 10:43:03 +02:00
Edvard
4d36f6200d
Further refactor move and init tests
2018-10-23 10:43:03 +02:00
Edvard
c4d3783f8b
Expand error logging in transformer
2018-10-23 10:43:03 +02:00
Edvard
719ffd5819
Update vat init tests for factory
2018-10-23 10:43:03 +02:00
Edvard
fedbce80f6
Refactor vat init transformer to use factory
2018-10-23 10:43:03 +02:00
Edvard
177d212cc4
Add SetDB test for vat move
2018-10-23 10:43:03 +02:00
Edvard
32654ffabc
Add transformer name to config for logging in factory transformer
2018-10-23 10:43:03 +02:00
Edvard
46d8ea8d97
Snag transformer topics from config in factory
2018-10-23 10:43:03 +02:00
Edvard
155569a1d8
Update vat_move tests
2018-10-23 10:43:03 +02:00
Edvard
af1dfbc489
Implement vat_move transformer using factory
2018-10-23 10:43:03 +02:00
Edvard
15e33db33d
Fix transformer factory
2018-10-23 10:43:03 +02:00
Edvard
1b9e6491da
Add transformer factory
2018-10-23 10:43:03 +02:00
Elizabeth
954fcf2378
integration tests ( #75 )
...
* Deal integration test
* Add LogIndex to CatFileChopLump to update unique constraint
* Add LogIndex to cat_file_pit_vow records
* Add integration tests for Cat.file transformers
* Add log index to Cat.file flip
2018-10-22 13:28:42 -05:00
Rob Mulholand
d87fb14c6b
Add log index to Drip file events
2018-10-22 12:14:43 -05:00
Rob Mulholand
de294ad8ea
Add log index to Drip drip
2018-10-22 12:14:43 -05:00
Rob Mulholand
e5ba0496a6
Add log index to Dent
2018-10-22 12:14:43 -05:00
Rob Mulholand
1aafb7201e
Add log index to Deal
2018-10-22 12:14:43 -05:00
Rob Mulholand
495f39abb8
Add log index to Bite
2018-10-22 12:14:43 -05:00
Rob Mulholand
ea3de51792
Add Flop kick integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
fa828cf041
Add Tend integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
cbb5ffa3c7
Add Pit file stability fee integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
3449a1e246
Add Pit file debt ceiling integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
a83bf5ed0d
Add Pit file ilk integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
677e78891d
Add Drip file vow integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
0ce3e3d829
Add Bite integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
ddc7ee5143
Add Drip drip integration test
2018-10-19 10:56:28 -05:00