Taka Goto
aa99c861a8
batch headers by 100s
2019-01-08 16:53:26 -06:00
Edvard Hübinette
77209d3b62
Merge pull request #128 from 8thlight/VDB-125-aggregate-log-fetching
...
VDB-125 Aggregate log fetching
2019-01-08 10:51:42 +01:00
Ian Norden
59cdaa05e6
moving some memory allocation to init function rather than within
...
execution cycles and fixing test db tear down so that it drops
checked_headers table instead of individual columns (droping columns
only hides them, they remain in memory and continue to contribute to the
1600 column limit which causes us to eventually exceed it)
2019-01-07 13:07:31 -06:00
Edvard
2292c94e99
Merge branch 'staging' into VDB-125-aggregate-log-fetching
2019-01-07 15:25:37 +01:00
Edvard
50cc0c2459
Update tests for watcher and repository
2019-01-07 15:20:05 +01:00
Edvard
833dde62cf
Remove injection of fetcher and repository to watcher
2019-01-07 15:19:31 +01:00
Ian Norden
c5d4e8e7af
fix conflicts and merge branch documentation
2018-12-21 11:23:22 -06:00
Ian Norden
2cbe6e7a70
method piping
2018-12-21 11:14:54 -06:00
Ian Norden
456c735087
fetch headers and logs in batches across all contracts and events
2018-12-21 10:33:31 -06:00
Ian Norden
d188329661
fetch event logs in batches- all events for a contract address at a given block height are fetched together, different contract events still fetched separately
2018-12-21 10:33:31 -06:00
Ian Norden
e390a97502
finish method polling with hash or address type argument values
...
collected from watched events; 'MissingMethodsCheckedEventsIntersection'
method to find headers which have been checked for each event of
interest but methods have not yet been polled at that header; tests for
new features; travis ci go version 1.9 -> 1.11 ; consolidate omniWatcher and lightOmniWatcher into single command with light as default
2018-12-21 10:33:31 -06:00
Ian Norden
0a59f06cac
begin work on: Add checked_headers column for methods that are polled so taht we don’t duplicate; Add batching of method polling so that we arent generating a rediculously large account address list before using it to poll methods (or persist the list in pg?); User passed ABI and other ways to get ABI; Add ability to collect []byte and hashes from events and use them in method polling same manner as addresses; Event filter addrs => only those event’s addresses/hashes are used for polling; Option to persist seen address/hash/bytes lists into pg; Only generate lists of addresses, []byte, or hashes if a method will use them later
2018-12-21 10:33:31 -06:00
Ian Norden
8c5b1b4dbe
remove accidental duplicate vendor lib that was imported by goland
2018-12-21 10:33:21 -06:00
Takayuki Goto
8723575521
Merge pull request #130 from 8thlight/update-staging-config
...
update staging/integration ipc path to 20c's kovan node
2018-12-17 13:16:59 -06:00
Taka Goto
f787e7ac0d
update staging/integration ipc path to 20c's kovan node
2018-12-17 08:37:35 -06:00
Edvard
e54699c039
Remove todo on chunker non-match (it's basically a filter)
2018-12-14 15:22:46 +01:00
Edvard Hübinette
a3e8633aff
VDB-104 improve lightsync ui ( #125 )
...
* Return PopulateMissingHeaders early if the sync is at the head of the chain
* Squelch logging if no blocks to sync
* Fix broken test
* Refactor repository tests
2018-12-14 12:11:48 +01:00
Edvard
7cf694f5e7
Update transformer documentation
2018-12-14 10:30:08 +01:00
Edvard
35f7f719e6
Swap strings.Builder for bytes.Buffer for go 1.9 support
2018-12-13 16:08:57 +01:00
Edvard
129964f3bc
Add tests for now shared MissingHeaders
2018-12-13 13:58:16 +01:00
Edvard
a3737c46f6
Remove duplicate entry in initializer map
2018-12-13 13:16:49 +01:00
Edvard
e5db740ae4
Move MockTransformer to test_data
2018-12-13 13:03:51 +01:00
Edvard
d30fcfed80
Use new config getter on shared.Transformer <3
2018-12-13 12:39:57 +01:00
Edvard
882a6dd7a2
Update tests with new adding of transformers
2018-12-12 15:41:44 +01:00
Edvard
dc43547612
Improve process of adding transformers
2018-12-12 15:41:29 +01:00
Edvard
a86a2ec5c6
Improve checked header column detection
2018-12-12 12:13:23 +01:00
Edvard
4e089c363e
Fixes after merging staging
2018-12-11 15:19:27 +01:00
Edvard
ff47e2e06e
Merge branch 'staging' into VDB-125-aggregate-log-fetching
2018-12-11 15:03:21 +01:00
Edvard
9000c6e9f9
Add test for watcher log delegation
2018-12-11 15:02:32 +01:00
Edvard
5d4239de05
Finish watcher and watcher tests (+fmt)
2018-12-11 13:52:02 +01:00
Edvard
d93817f346
Interface-ify shared repository
2018-12-11 11:35:13 +01:00
Edvard
83593cb5f2
Cleanup fetcher and add test todos
2018-12-10 21:12:55 +01:00
Edvard
e03ccb094f
Add error propagation and test todos for watcher
2018-12-10 21:12:31 +01:00
Edvard
2d81861ae5
Add chunker tests
2018-12-10 21:11:25 +01:00
Edvard
bc9fd17793
Fix remaining integration tests
2018-12-10 17:12:51 +01:00
Edvard
3cb8fe60d4
Fix persistHeader not returning proper header ID
2018-12-10 16:58:24 +01:00
Edvard
587d2219d8
Remove bc references and cleanup pointers
2018-12-10 16:50:13 +01:00
Edvard
637623b752
Fix test suites!
2018-12-10 15:40:45 +01:00
Rob Mulholand
670741616e
Use logrus for logging in populating blocks
...
- Removes logging from test output
2018-12-07 11:24:06 -06:00
Rob Mulholand
2a518ad518
(VDB-254) Convert numerical values for Pit.file ilk
...
- convert data to ray when what == "spot" and wad when what == "line"
2018-12-07 11:24:06 -06:00
Rob Mulholand
087af252c9
(VDB-253) Update numerical values for Pit.file debt ceiling
...
- convert Line to wad
2018-12-07 11:24:06 -06:00
Rob Mulholand
4a5765b20d
(VDB-249) Update numerical values for Vat.fold
...
- convert rate to ray
2018-12-07 11:24:06 -06:00
Edvard
e1ba7ac5b4
WIP update tests for aggregate fetching
2018-12-07 18:10:36 +01:00
Takayuki Goto
b922e73bf9
Merge pull request #127 from 8thlight/use-kovan-archive
...
run integration tests on kovan archive instead of full
2018-12-07 10:11:03 -06:00
Ian Norden
ce428199eb
finishing porting omni watcher to work with light sync; split into full, light,
...
and shared directories and refactor as much into shared; finish
lightSync omni watcher tests
2018-12-07 09:54:53 -06:00
Taka Goto
75fe3186fd
run integration tests on kovan archive instead of full
2018-12-06 11:00:17 -06:00
Edvard
47c75d055b
Update repository tests for absence of MissingHeaders
2018-12-04 17:53:46 +01:00
Edvard
9789648cc2
Delegate log chunks to respective transformers
2018-12-04 17:40:39 +01:00
Edvard
8bebcdc064
Change transformer.Execute to single header
2018-12-04 17:05:34 +01:00
Edvard
38c745e8c3
Refactor fetching out from repositories to log_fetcher and watcher
2018-12-04 16:04:13 +01:00