Commit Graph

1256 Commits

Author SHA1 Message Date
Rob Mulholand
2b151c2ba3
Merge pull request #131 from vulcanize/vdb-699-logs-table
(VDB-699) Decouple Log extraction vs delegation to transformers
2019-09-20 10:52:49 -05:00
Rob Mulholand
3897b288af Update Geth dep to 1.9.5 2019-09-20 10:38:16 -05:00
Rob Mulholand
b96a1ee1c6 Put secondary processes to sleep in event watcher tests
- Prevent extract/delegate from spinning when other side is being
  simulated
2019-09-18 22:17:02 -05:00
Rob Mulholand
f83e996ab8 Update comments in checked headers repository 2019-09-18 21:39:34 -05:00
Rob Mulholand
4fa19be90a Return error when no logs/headers available
- Replaces bool and moots question of error/bool ordering
- Also make event watcher execution synchronous
2019-09-18 20:55:15 -05:00
Rob Mulholand
2b798e00e0 Cap random fake timestamp 2019-09-13 11:20:13 -05:00
Rob Mulholand
3f9b034c4c Rename checked_logs => watched_logs
- We're logging that a given log has been included in any fetch calls
  for checked headers, rather than that we have already checked for
  that log
2019-09-10 21:22:14 -05:00
Rob Mulholand
13d503b851 Distinguish between missing and unchecked headers
- Missing == not in DB
- Unchecked == logs haven't been fetched
2019-09-10 14:39:45 -05:00
Rob Mulholand
c568fedd89 Remove unused functions on mock event repository 2019-09-10 14:20:47 -05:00
Rob Mulholand
3d6c973f6e Remove extraneous db constraint 2019-09-10 13:58:58 -05:00
Rob Mulholand
ce91b0d9e6 Simplify checked header repository tests
- Use assertions instead of comments to document expectations
- Also randomize the fake timestamp in test data
2019-09-10 13:47:43 -05:00
Rob Mulholand
b9f3b9f946 Reference header sync logs address via foreign key 2019-08-28 10:41:34 -05:00
Rob Mulholand
5ac76eee74 Make check_count a column on public.headers
- Don't need to maintain it on public.checked_headers if we're not
  adding additional columns to that table
2019-08-28 09:50:17 -05:00
Rob Mulholand
222252f89a Remove extraneous migration formatting 2019-08-28 09:26:13 -05:00
Rob Mulholand
666ea1c325 Update checked headers for new transformers
- If a header was marked as checked before a transformer was added to
  the watcher, mark all headers since the new transformer's starting
  block number as unchecked.
2019-08-28 09:25:14 -05:00
Rob Mulholand
d76be4962b Remove unnecessary async from the event watcher
- extract and delegate logs synchronously after initial goroutine fired
2019-08-28 09:25:13 -05:00
Rob Mulholand
1883a11ab1 Update comment in log chunker
Co-Authored-By: Edvard Hübinette <edvard@hubinette.me>
2019-08-28 09:25:13 -05:00
Rob Mulholand
63dabbb051 Extract and delegate logs concurrently 2019-08-28 09:25:13 -05:00
Rob Mulholand
d496dad33c Decouple log extraction from transformer delegation
- limit missing headers results set to 100 so that extraction doesn't
  excessively block delegation
- wrap checked headers functions in repository struct
- move storage repository to factory, to correspond with event
  repository path
- remove unused files
- reformat sql
- remove line breaks in imports
2019-08-28 09:25:13 -05:00
Rob Mulholand
cb819fa9a6 Write event logs to database before transforming
- enables decoupling event extraction/persistence from transformation
- modifies event transformer, converter, and log chunker to accept
  payload that includes internal log database ID with log data
- remove alias for transformer pkg as shared_t
- remove unused mock watcher repository
2019-08-28 09:13:44 -05:00
Rob Mulholand
66a4e20b20 Update event transformer to take header ID
- rather than header
- enables executing transformers without full header lookup
2019-08-28 09:11:06 -05:00
Rob Mulholand
3693ed905f Rename logs to full sync logs
- Enable creating new table for logs used in event watching based on
  header sync
2019-08-28 09:11:06 -05:00
Elizabeth
b6a2a278ce
Merge pull request #126 from vulcanize/vdb-585-create-address-table
Vdb-585 create address table
2019-08-26 15:13:07 -05:00
Elizabeth Engelman
edc0bdf668 Address more PR feedback 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
1b3786338f Address PR feedback 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
58c23c6632 Update Copyrights 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
38f6847ff2 Create a HeaderSyncReceiptRepository 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
7e38764618 Rename ReceiptRepository -> FullSyncReceiptRepository 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
4e40e892d2 Updating header_sync_receipts to have FK reference to addresses 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
bcd6d14fcd Updating full_sync_receipts to have FK reference to addresses 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
258035833b Add GetAddressById 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
4b61c87b55 Get or create address record in a transaction 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
1373fe83a1 Pass a db into GetOrCreateAddress 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
b13eefbce7 Add AddressRepository 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
3005a1ad50 Move add address migration table up in the migration order 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
750754caa7 Add an addresses table 2019-08-23 10:11:37 -05:00
Rob Mulholand
e093ad351f
Merge pull request #130 from vulcanize/update-deps
Update dependencies
2019-08-21 16:04:55 -05:00
Rob Mulholand
33e7651096 Update dependencies 2019-08-20 16:17:25 -05:00
Gabe Laughlin
baea9740c1
Merge pull request #125 from vulcanize/decode-packed-addresses
(VDB-768) Allow addresses in packed slots to be decoded
2019-08-05 21:40:15 -05:00
Gabe Laughlin
4f8b057726
(VDB-768) Allow addresses in packed slots to be decoded 2019-08-01 22:40:50 -05:00
Andrew J Yao
65e762f80b
Merge pull request #124 from vulcanize/refactor-mcd-configs
Removes _checked from column name query
2019-07-30 18:18:42 -07:00
Andrew J Yao
dc8d52a58d Removes _checked from column name query 2019-07-30 18:07:25 -07:00
Rob Mulholand
f2547b5a42
Merge pull request #117 from vulcanize/geth-1.9.0
Update Geth to 1.9.0
2019-07-29 15:04:37 -05:00
Rob Mulholand
62e454b7fe Update docs to reflect using go mod 2019-07-23 15:28:08 -05:00
Rob Mulholand
ca4451d914 Update Travis to enable go mod
- set GO111MODULE=on
- use go 1.12
2019-07-23 15:28:08 -05:00
Rob Mulholand
f51f5c64a0 Replace dep with go mod 2019-07-23 15:28:08 -05:00
Rob Mulholand
987abd4b2e Update Geth to 1.9.0 2019-07-23 15:26:18 -05:00
Andrew J Yao
a3b55bfd56
Merge pull request #120 from vulcanize/logging-enhancement
Logging enhancement
2019-07-23 13:22:58 -07:00
Andrew J Yao
ee77fc6521 Remove injection of SubCommand for logs 2019-07-23 13:11:20 -07:00
Elizabeth
85aba29cf2
Merge pull request #121 from vulcanize/allow-for-packed-storage-slots
VDB-318 Allow for packed storage slots
2019-07-23 13:48:36 -05:00