cerc-io/ipld-eth-server
18
0
Fork 1
Code Issues 10 Pull Requests 1 Actions Packages Projects Releases 37 Wiki Activity
820 Commits 12 Branches 52 Tags 80 MiB
37fc038605
Commit Graph

478 Commits

Author SHA1 Message Date
Rob Mulholand
ee113668a5 Add first draft of storage transformers readme 
- Explain process of creating a new storage transformer
 2019-02-12 15:58:28 -06:00
Rob Mulholand
ac9375ae47 Discard logs created during tests 
- Make sure output is being set to discard for logrus to reduce clutter in the console
 2019-02-12 11:18:39 -06:00
Taka Goto
4fe2ff4104 repository methods: 2019-02-11 16:34:02 -06:00
Taka Goto
c2c21325b5 vow contract storage diffing excluding sin 2019-02-11 15:28:54 -06:00
Edvard Hübinette
f73623789e Merge pull request #6 from 8thlight/VDB-337-Dockerize-vDB 
VDB-337 Dockerize vDB
 2019-02-11 12:37:56 +01:00
Edvard
9b986c4c14 Nuke OpenRC from docker image, update readme's 2019-02-11 11:27:33 +01:00
Edvard
553fdbfa09 Default config starting block to 0 2019-02-11 11:27:33 +01:00
Edvard
e6e66d58dd Add container service files and docker README 2019-02-11 11:26:12 +01:00
Rob Mulholand
c05d3b7836 (VDB-280) Consume vat contract storage diffs 2019-02-08 13:36:30 -06:00
Rob Mulholand
dba4e48a3e Use map for key in storage value metadata 
- Enables inserting multiple keys for one storage row (e.g. ilk and urn)
 2019-02-08 11:41:17 -06:00
Rob Mulholand
09abcb841a Persist Urns in raw bytes32 format 
- Enables simple usage in contract storage lookups
 2019-02-08 11:41:17 -06:00
Takayuki Goto
0a024d429d * update transformer to able to recheck headers (#4) 
* update transformer to able to recheck headers
* put cap on rechecking header
* integration test for recheck headers
* use enum for recheck headers param; make recheck cap configurable
* update integration tests with new test config
* update omni pkg with new recheck header column type
* update migration with new migration tool and final tweaks needed to accommodate changes in omni pkg
 2019-02-08 10:35:46 -06:00
Rob Mulholand
867f92c431 (VDB-298) Consume Pit contract storage diffs 
- Continuously parse storage diffs CSV data to read Pit contract state
- Convert ilks in database to raw bytes32 value for use in generating
  storage keys dynamically
- Persist storage diffs with block number and hash for validation
 2019-02-06 10:31:46 -06:00
Edvard Hübinette
29f2537130 Merge pull request #7 from vulcanize/VDB-347-Squelch-lightSync 
VDB-347 Squelch lightSync in stdout
 2019-02-01 11:02:48 +01:00
Edvard
a1fa21cbec Remove test based on window buffer stuff 2019-01-31 16:39:29 +01:00
Edvard
8eea1b00ed Redirect validation window status to logrus instead of stdout 2019-01-31 15:55:28 +01:00
Ian Norden
8749c94994 update license to AGPL-3.0 2019-01-24 14:41:30 -06:00
Ian Norden
ab371a2024 Merge remote-tracking branch 'maker/staging' into staging 2019-01-24 10:22:22 -06:00
Ian Norden
1f95eb6443 Merge branch 'staging' into pub_merge 2019-01-24 09:47:12 -06:00
Taka Goto
c5d513e96d fix index out of range on lightSync; hopefully the new integration tests will catch them next time :) 2019-01-23 15:25:44 -06:00
Ian Norden
00c2052da3 for some reason an error is thrown when the ipc string is declared in integration_tests_suite_test.go but not when moved to a different file in the integration_tests package; I think this is due to switching to go1.11 2019-01-23 15:23:23 -06:00
Ian Norden
dc87f39889 change omniwatcher database test helpers so that it doesn't interfere with downstream maker tests 2019-01-23 14:55:20 -06:00
Konstantin Zolotarev
65cb6e9092 VDB-205 Fix external configuration values (#121) 
* Updated loading config from /Users/konstantinzolotarev/.vulcanizedb for addreses

* Updated contract addresses loading from env variables.

* Implement getter chain for addresses, configs and transformers

Solves the issue with declaring stuff as vars when the environment file hasn't been loaded yet. Instead, this data is fetched dynamically when the root commands are called.

* Update test data to use constant Kovan addresses

* Decouple integration tests from config file IO

* Add ABI to environment file, convert getter chains for related values

* Decouple tests suites from ABI config file IO

* Add startingBlock to environment and implement getter chain

* Exit when called without config file

* Add missing deployment blocks to staging.toml

* Remove comment
 2019-01-23 19:44:09 +01:00
Ian Norden
46b898f690 merge public vulcanizedb/master 2019-01-23 00:37:26 -06:00
Taka Goto
3595771825 use batchElem from rpc client 2019-01-21 14:52:37 -06:00
Takayuki Goto
54d771cb06 Merge branch 'staging' into batch-historical-headers 2019-01-16 15:56:36 -06:00
Taka Goto
24cf0661b0 lowercase address on map to get transformers 2019-01-16 14:24:00 -06:00
Taka Goto
07da9d720b fix get headers when less than 100 fetched 2019-01-16 10:16:06 -06:00
Taka Goto
074140b9ac iterate for loop by headers instead of blocknumbers 2019-01-15 16:34:20 -06:00
Ian Norden
aa4e698240 move interface_getter.go from ens_watcher to vDB pkging- used to construct a custom ABI based on results from calls to supportsInterface; also moved fetcher.go from examples to pkg 2019-01-15 15:49:18 -06:00
Takayuki Goto
f1ba21b505 Merge branch 'staging' into batch-historical-headers 2019-01-13 21:29:58 -06:00
yaoandrew
e2b864baa8 Add rollback if GetTicInTx fails and move GetTicInTx to shared/repository.go 2019-01-11 10:07:00 -08:00
Andrew J Yao
c8aa3dc275 Replace standard log with logrus import 2019-01-11 10:07:00 -08:00
yaoandrew
bc8b7a101c refactor error handling in vat, cat, vow 2019-01-11 10:07:00 -08:00
yaoandrew
525fada3d7 refactor error handling in frob, pit, price, tend 2019-01-11 10:07:00 -08:00
yaoandrew
657aaccff3 refactor error handling in drip, flap, flop 2019-01-11 10:07:00 -08:00
yaoandrew
05351a8ef4 refactor error handling in dent 2019-01-11 10:07:00 -08:00
yaoandrew
08ac7f18a5 refactor error handling in flip_deal 2019-01-11 10:07:00 -08:00
yaoandrew
f655997fde refactor error handling in pit_vow 2019-01-11 10:07:00 -08:00
yaoandrew
843d30c88e refactor error handling in flip 2019-01-11 10:07:00 -08:00
yaoandrew
513513e93b refactor error handling in chop_lump 2019-01-11 10:07:00 -08:00
yaoandrew
78b302eab6 refactor error handling in bite transformer 2019-01-11 10:07:00 -08:00
Edvard Hübinette
ebca338b1e VDB-302 Sleep when no missing blocks in lightSync (#129) 
* Sleep when no missing blocks in lightSync

* Fix tests and error propagation

* Correct geth.log to logrus
 2019-01-11 10:58:41 +01:00
Takayuki Goto
ac430ddf54 Merge branch 'staging' into batch-historical-headers 2019-01-08 19:06:53 -06:00
Taka Goto
aa99c861a8 batch headers by 100s 2019-01-08 16:53:26 -06:00
Ian Norden
59cdaa05e6 moving some memory allocation to init function rather than within 
execution cycles and fixing test db tear down so that it drops
checked_headers table instead of individual columns (droping columns
only hides them, they remain in memory and continue to contribute to the
1600 column limit which causes us to eventually exceed it)
 2019-01-07 13:07:31 -06:00
Edvard
2292c94e99 Merge branch 'staging' into VDB-125-aggregate-log-fetching 2019-01-07 15:25:37 +01:00
Edvard
50cc0c2459 Update tests for watcher and repository 2019-01-07 15:20:05 +01:00
Edvard
833dde62cf Remove injection of fetcher and repository to watcher 2019-01-07 15:19:31 +01:00
Ian Norden
2cbe6e7a70 method piping 2018-12-21 11:14:54 -06:00
Ian Norden
456c735087 fetch headers and logs in batches across all contracts and events 2018-12-21 10:33:31 -06:00
Ian Norden
d188329661 fetch event logs in batches- all events for a contract address at a given block height are fetched together, different contract events still fetched separately 2018-12-21 10:33:31 -06:00
Ian Norden
e390a97502 finish method polling with hash or address type argument values 
collected from watched events; 'MissingMethodsCheckedEventsIntersection'
method to find headers which have been checked for each event of
interest but methods have not yet been polled at that header; tests for
new features; travis ci go version 1.9 -> 1.11 ; consolidate omniWatcher and lightOmniWatcher into single command with light as default
 2018-12-21 10:33:31 -06:00
Ian Norden
0a59f06cac begin work on: Add checked_headers column for methods that are polled so taht we don’t duplicate; Add batching of method polling so that we arent generating a rediculously large account address list before using it to poll methods (or persist the list in pg?); User passed ABI and other ways to get ABI; Add ability to collect []byte and hashes from events and use them in method polling same manner as addresses; Event filter addrs => only those event’s addresses/hashes are used for polling; Option to persist seen address/hash/bytes lists into pg; Only generate lists of addresses, []byte, or hashes if a method will use them later 2018-12-21 10:33:31 -06:00
Edvard
e54699c039 Remove todo on chunker non-match (it's basically a filter) 2018-12-14 15:22:46 +01:00
Edvard Hübinette
a3e8633aff VDB-104 improve lightsync ui (#125) 
* Return PopulateMissingHeaders early if the sync is at the head of the chain

* Squelch logging if no blocks to sync

* Fix broken test

* Refactor repository tests
 2018-12-14 12:11:48 +01:00
Edvard
7cf694f5e7 Update transformer documentation 2018-12-14 10:30:08 +01:00
Edvard
35f7f719e6 Swap strings.Builder for bytes.Buffer for go 1.9 support 2018-12-13 16:08:57 +01:00
Edvard
129964f3bc Add tests for now shared MissingHeaders 2018-12-13 13:58:16 +01:00
Edvard
e5db740ae4 Move MockTransformer to test_data 2018-12-13 13:03:51 +01:00
Edvard
d30fcfed80 Use new config getter on shared.Transformer <3 2018-12-13 12:39:57 +01:00
Edvard
882a6dd7a2 Update tests with new adding of transformers 2018-12-12 15:41:44 +01:00
Edvard
dc43547612 Improve process of adding transformers 2018-12-12 15:41:29 +01:00
Edvard
a86a2ec5c6 Improve checked header column detection 2018-12-12 12:13:23 +01:00
Edvard
4e089c363e Fixes after merging staging 2018-12-11 15:19:27 +01:00
Edvard
ff47e2e06e Merge branch 'staging' into VDB-125-aggregate-log-fetching 2018-12-11 15:03:21 +01:00
Edvard
9000c6e9f9 Add test for watcher log delegation 2018-12-11 15:02:32 +01:00
Edvard
5d4239de05 Finish watcher and watcher tests (+fmt) 2018-12-11 13:52:02 +01:00
Edvard
d93817f346 Interface-ify shared repository 2018-12-11 11:35:13 +01:00
Edvard
83593cb5f2 Cleanup fetcher and add test todos 2018-12-10 21:12:55 +01:00
Edvard
2d81861ae5 Add chunker tests 2018-12-10 21:11:25 +01:00
Edvard
bc9fd17793 Fix remaining integration tests 2018-12-10 17:12:51 +01:00
Edvard
3cb8fe60d4 Fix persistHeader not returning proper header ID 2018-12-10 16:58:24 +01:00
Edvard
637623b752 Fix test suites! 2018-12-10 15:40:45 +01:00
Rob Mulholand
670741616e Use logrus for logging in populating blocks 
- Removes logging from test output
 2018-12-07 11:24:06 -06:00
Rob Mulholand
2a518ad518 (VDB-254) Convert numerical values for Pit.file ilk 
- convert data to ray when what == "spot" and wad when what == "line"
 2018-12-07 11:24:06 -06:00
Rob Mulholand
087af252c9 (VDB-253) Update numerical values for Pit.file debt ceiling 
- convert Line to wad
 2018-12-07 11:24:06 -06:00
Rob Mulholand
4a5765b20d (VDB-249) Update numerical values for Vat.fold 
- convert rate to ray
 2018-12-07 11:24:06 -06:00
Edvard
e1ba7ac5b4 WIP update tests for aggregate fetching 2018-12-07 18:10:36 +01:00
Edvard
47c75d055b Update repository tests for absence of MissingHeaders 2018-12-04 17:53:46 +01:00
Edvard
9789648cc2 Delegate log chunks to respective transformers 2018-12-04 17:40:39 +01:00
Edvard
8bebcdc064 Change transformer.Execute to single header 2018-12-04 17:05:34 +01:00
Edvard
38c745e8c3 Refactor fetching out from repositories to log_fetcher and watcher 2018-12-04 16:04:13 +01:00
Ian Norden
e02b33547d finishing porting omni watcher to work with light sync; split into full, light, 
and shared directories and refactor as much into shared; finish
lightSync omni watcher tests
 2018-11-30 13:33:48 -06:00
Edvard
c26736dc9e WIP continue on aggregate fetching architecture 2018-11-30 17:28:52 +01:00
Ian Norden
975f13b969 reorganizing omni directory and beginning light watcher work 2018-11-29 20:33:21 -06:00
Ian Norden
817bd76713 refactoring event converter and repository so that event logs are 
persisted as they are converted, preventing build up in memory, and finishing  method polling (for full sync)
 2018-11-29 20:33:21 -06:00
Ian Norden
390a60f7f6 port over lightSync updates from maker repo 2018-11-29 20:33:21 -06:00
Elizabeth
8ce901d1af Convert drip_file_ilk tax field to ray (#117) 2018-11-29 14:10:12 -06:00
Edvard
45a087f5d1 WIP: First steps in refactoring to aggregate fetching 2018-11-28 17:00:04 +01:00
Edvard
423fdf01b5 Add log_chunker 2018-11-28 16:59:06 +01:00
Edvard
f7ecab9dc2 Fix typo 2018-11-28 15:39:46 +01:00
Taka Goto
e8be96a4e7 use logrus for logging 2018-11-21 10:14:11 -06:00
Elizabeth
128d20c9bf Vdb 269- fetch logs by hash (#122) 
* Upgrade geth from 1.8.15 to 1.8.18

* Update vat_tune to use shared repository methods

* Query blockchain by block hash instead of block number range

* Remove hash validation from repositories

* Fix vow flog integration test

* Update README Travis build sticker

* Update constants formatting per go fmt

* Update EthPublicKeyParser.ParsePublicKey to use discv5.PubkeyID method

* Address PR comments
 2018-11-21 09:30:00 -06:00
Elizabeth
b19e77e419 Add integration test for Vow.flog (#114) 2018-11-15 16:44:49 -06:00
Ian Norden
417b18ec6a Edits to address PR issues; change license from apache to AGPL; and work 
towards generic method polling and reposito;y; config settings to
filterevents/methods by account address; refactoring some stuff out of
repo and into converter; remove fetcher and instead call
blockchain's FetchContractData directly; finishing tests
 2018-11-15 12:32:52 -06:00
Edvard
7cd512ae91 Remove PFSF label 2018-11-15 12:14:59 -06:00
Rob Mulholand
854236fd99 (VDB-267) Remove pit file stability fee 
- Method is no longer present on the Pit contract
 2018-11-15 12:14:59 -06:00
Edvard Hübinette
40408e8300 VDB-120 include tic in dent and tend (#118) 
* Add constants for TTL and tic for fakeHeader

* Add tic to tend transformer

* Update tests for tend

* Fix string conversion bug in fakes

* Fix tend integration tests after staging rebase

* Add tic to dent transformer

* Update dent tests

* Change integration tests to use hardcoded block timestamp
 2018-11-15 12:57:29 +01:00
Edvard
8f4648a9e3 Add support for never-ending transformer 2018-11-14 10:11:16 -06:00