Commit Graph

1038 Commits

Author SHA1 Message Date
Elizabeth Engelman
8e166f2a56 Update after rebasing with staging 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
639c561fdf Remove unncessary pointer receiver from NewGethRpcStorageFetcher.FetchStorageDiffs 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
9226e53e0b Panic if connecting to geth subscription fails 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
0cc90a1f80 Factor out an IStorageWatcher interface 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
18f47b7b6a Memoizing keccak hhash of address => transformer 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
045d78be25 Rename StorageDiffRow -> StorageDiff 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
87252f42b9 Add additional logging 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
e637024cce Update vendor dir 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
7cd66b8775 Use a for loop for getting diffs off the payload channel 2019-09-25 16:32:30 -05:00
Elizabeth Engelman
7cbad1d89f Override protobuf to version 3 2019-09-25 16:32:30 -05:00
Elizabeth Engelman
963479fdc0 Remove ipfs and eth-block-extractor as dependencies 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
b09b8a8735 Use geth state diff source in execute command 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
70fcc22a00 Apply go formatting 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
a577811e0a Use geth state diff source in composeAndExecute 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
8c4a4d6587 Handle different contract address format for geth vs csv 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
ee244ac6f5 Apply go fmt 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
dc06991605 Wire up the streamer with a fetcher 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
1b4a901892 Rename storage_fetcher -> csv_tail_storage_fetcher 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
9de631f3a7 Add a statediff streamer 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
34f8e16c11 Add a Subscribe method to rpc client interface 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
36533f7c3f Update vendor directory and make necessary code changes
Fixes for new geth version
2019-09-25 16:32:27 -05:00
20ce0ab852 Revert "option to write the plugin .go file and run the plugin migrations without building the .so file so that it can be done from the shell instead to afford complete error messages"
This reverts commit 9bb2f27a69.
2019-09-24 23:54:25 +00:00
Ian Norden
9bb2f27a69 option to write the plugin .go file and run the plugin migrations without building the .so file so that it can be done from the shell instead to afford complete error messages 2019-09-24 18:44:04 -05:00
Rob Mulholand
ba4e79fc63
Merge pull request #143 from vulcanize/contract-watcher-missing-headers
Correctly identify missing headers after contract watcher restart
2019-09-24 08:55:25 -05:00
Rob Mulholand
c21c069b69 Replace magic numbers with variables 2019-09-23 16:13:45 -05:00
Rob Mulholand
eea0b0bdca Return missing headers after starting block number
- Previously, we required that the first missing header match the
  configured starting block number. This helps guarantee that we have
  the necessary data for method polling in memory, but prevents the
  process from moving forward if restarted after the starting block
  has already been checked.
2019-09-23 16:13:45 -05:00
Rob Mulholand
a596966b7c
Merge pull request #141 from vulcanize/contract-watcher-duplicate-inserts
Do nothing on duplicate log insert in contract watcher
2019-09-23 16:13:18 -05:00
Rob Mulholand
37e2673a8d Do nothing on duplicate log insert
- Prevents duplicate key constraint violation from blocking the process
  from moving forward on restart.
- If header_id, log_idx, and tx_idx are the same, we can safely do
  nothing since it's definitely the same log - a reorg would cause the
  original header to be replaced with a new ID.
2019-09-23 16:02:07 -05:00
Rob Mulholand
dc30099a7c
Merge pull request #139 from vulcanize/contract-watcher-uint8
Correctly parse uint8 values in contract watcher
2019-09-23 15:59:38 -05:00
Rob Mulholand
3466a51b4d Correctly parse uint8 values in contract watcher
- calling string(uint8) => insert errors on numeric columns
2019-09-20 21:37:59 -05:00
Rob Mulholand
8bbd65eabd
Merge pull request #138 from vulcanize/contract-watcher-bytes32-parsing
Parse bytes32 vals in Contract Watcher
2019-09-20 14:23:29 -05:00
Rob Mulholand
749db40b96 Parse bytes32 vals in Contract Watcher 2019-09-20 13:48:43 -05:00
Rob Mulholand
2b151c2ba3
Merge pull request #131 from vulcanize/vdb-699-logs-table
(VDB-699) Decouple Log extraction vs delegation to transformers
2019-09-20 10:52:49 -05:00
Rob Mulholand
3897b288af Update Geth dep to 1.9.5 2019-09-20 10:38:16 -05:00
Rob Mulholand
b96a1ee1c6 Put secondary processes to sleep in event watcher tests
- Prevent extract/delegate from spinning when other side is being
  simulated
2019-09-18 22:17:02 -05:00
Rob Mulholand
f83e996ab8 Update comments in checked headers repository 2019-09-18 21:39:34 -05:00
Rob Mulholand
4fa19be90a Return error when no logs/headers available
- Replaces bool and moots question of error/bool ordering
- Also make event watcher execution synchronous
2019-09-18 20:55:15 -05:00
Rob Mulholand
2b798e00e0 Cap random fake timestamp 2019-09-13 11:20:13 -05:00
Rob Mulholand
3f9b034c4c Rename checked_logs => watched_logs
- We're logging that a given log has been included in any fetch calls
  for checked headers, rather than that we have already checked for
  that log
2019-09-10 21:22:14 -05:00
Rob Mulholand
13d503b851 Distinguish between missing and unchecked headers
- Missing == not in DB
- Unchecked == logs haven't been fetched
2019-09-10 14:39:45 -05:00
Rob Mulholand
c568fedd89 Remove unused functions on mock event repository 2019-09-10 14:20:47 -05:00
Rob Mulholand
3d6c973f6e Remove extraneous db constraint 2019-09-10 13:58:58 -05:00
Rob Mulholand
ce91b0d9e6 Simplify checked header repository tests
- Use assertions instead of comments to document expectations
- Also randomize the fake timestamp in test data
2019-09-10 13:47:43 -05:00
Rob Mulholand
b9f3b9f946 Reference header sync logs address via foreign key 2019-08-28 10:41:34 -05:00
Rob Mulholand
5ac76eee74 Make check_count a column on public.headers
- Don't need to maintain it on public.checked_headers if we're not
  adding additional columns to that table
2019-08-28 09:50:17 -05:00
Rob Mulholand
222252f89a Remove extraneous migration formatting 2019-08-28 09:26:13 -05:00
Rob Mulholand
666ea1c325 Update checked headers for new transformers
- If a header was marked as checked before a transformer was added to
  the watcher, mark all headers since the new transformer's starting
  block number as unchecked.
2019-08-28 09:25:14 -05:00
Rob Mulholand
d76be4962b Remove unnecessary async from the event watcher
- extract and delegate logs synchronously after initial goroutine fired
2019-08-28 09:25:13 -05:00
Rob Mulholand
1883a11ab1 Update comment in log chunker
Co-Authored-By: Edvard Hübinette <edvard@hubinette.me>
2019-08-28 09:25:13 -05:00
Rob Mulholand
63dabbb051 Extract and delegate logs concurrently 2019-08-28 09:25:13 -05:00