Commit Graph
181 Commits
Author SHA1 Message Date
Elizabeth Engelman 3fb8e13979 Update Mappings interface to take a diff as an argument to Lookup 2019-09-25 16:34:31 -05:00
Elizabeth Engelman 66d695d93b Update keccak of address to be a common.Hash instead of a common.Address 2019-09-25 16:34:31 -05:00
Elizabeth Engelman 639c561fdf Remove unncessary pointer receiver from NewGethRpcStorageFetcher.FetchStorageDiffs 2019-09-25 16:32:31 -05:00
Elizabeth Engelman 9226e53e0b Panic if connecting to geth subscription fails 2019-09-25 16:32:31 -05:00
Elizabeth Engelman 0cc90a1f80 Factor out an IStorageWatcher interface 2019-09-25 16:32:31 -05:00
Elizabeth Engelman 18f47b7b6a Memoizing keccak hhash of address => transformer 2019-09-25 16:32:31 -05:00
Elizabeth Engelman 045d78be25 Rename StorageDiffRow -> StorageDiff 2019-09-25 16:32:31 -05:00
Elizabeth Engelman 87252f42b9 Add additional logging 2019-09-25 16:32:31 -05:00
Elizabeth Engelman 7cd66b8775 Use a for loop for getting diffs off the payload channel 2019-09-25 16:32:30 -05:00
Elizabeth Engelman 70fcc22a00 Apply go formatting 2019-09-25 16:32:27 -05:00
Elizabeth Engelman a577811e0a Use geth state diff source in composeAndExecute 2019-09-25 16:32:27 -05:00
Elizabeth Engelman 8c4a4d6587 Handle different contract address format for geth vs csv 2019-09-25 16:32:27 -05:00
Elizabeth Engelman ee244ac6f5 Apply go fmt 2019-09-25 16:32:27 -05:00
Elizabeth Engelman dc06991605 Wire up the streamer with a fetcher 2019-09-25 16:32:27 -05:00
Elizabeth Engelman 1b4a901892 Rename storage_fetcher -> csv_tail_storage_fetcher 2019-09-25 16:32:27 -05:00
Elizabeth Engelman 9de631f3a7 Add a statediff streamer 2019-09-25 16:32:27 -05:00
Rob Mulholand b96a1ee1c6 Put secondary processes to sleep in event watcher tests
- Prevent extract/delegate from spinning when other side is being
  simulated
2019-09-18 22:17:02 -05:00
Rob Mulholand 4fa19be90a Return error when no logs/headers available
- Replaces bool and moots question of error/bool ordering
- Also make event watcher execution synchronous
2019-09-18 20:55:15 -05:00
Rob Mulholand 3f9b034c4c Rename checked_logs => watched_logs
- We're logging that a given log has been included in any fetch calls
  for checked headers, rather than that we have already checked for
  that log
2019-09-10 21:22:14 -05:00
Rob Mulholand 13d503b851 Distinguish between missing and unchecked headers
- Missing == not in DB
- Unchecked == logs haven't been fetched
2019-09-10 14:39:45 -05:00
Rob Mulholand c568fedd89 Remove unused functions on mock event repository 2019-09-10 14:20:47 -05:00
Rob Mulholand 5ac76eee74 Make check_count a column on public.headers
- Don't need to maintain it on public.checked_headers if we're not
  adding additional columns to that table
2019-08-28 09:50:17 -05:00
Rob Mulholand 666ea1c325 Update checked headers for new transformers
- If a header was marked as checked before a transformer was added to
  the watcher, mark all headers since the new transformer's starting
  block number as unchecked.
2019-08-28 09:25:14 -05:00
Rob Mulholand d76be4962b Remove unnecessary async from the event watcher
- extract and delegate logs synchronously after initial goroutine fired
2019-08-28 09:25:13 -05:00
Rob MulholandandEdvard Hübinette 1883a11ab1 Update comment in log chunker
Co-Authored-By: Edvard Hübinette <edvard@hubinette.me>
2019-08-28 09:25:13 -05:00
Rob Mulholand 63dabbb051 Extract and delegate logs concurrently 2019-08-28 09:25:13 -05:00
Rob Mulholand d496dad33c Decouple log extraction from transformer delegation
- limit missing headers results set to 100 so that extraction doesn't
  excessively block delegation
- wrap checked headers functions in repository struct
- move storage repository to factory, to correspond with event
  repository path
- remove unused files
- reformat sql
- remove line breaks in imports
2019-08-28 09:25:13 -05:00
Rob Mulholand cb819fa9a6 Write event logs to database before transforming
- enables decoupling event extraction/persistence from transformation
- modifies event transformer, converter, and log chunker to accept
  payload that includes internal log database ID with log data
- remove alias for transformer pkg as shared_t
- remove unused mock watcher repository
2019-08-28 09:13:44 -05:00
Rob Mulholand 66a4e20b20 Update event transformer to take header ID
- rather than header
- enables executing transformers without full header lookup
2019-08-28 09:11:06 -05:00
Gabe Laughlin 4f8b057726 (VDB-768) Allow addresses in packed slots to be decoded 2019-08-01 22:40:50 -05:00
Andrew J Yao dc8d52a58d Removes _checked from column name query 2019-07-30 18:07:25 -07:00
Elizabeth 85aba29cf2 Merge pull request #121 from vulcanize/allow-for-packed-storage-slots
VDB-318 Allow for packed storage slots
2019-07-23 13:48:36 -05:00
Elizabeth Engelman 2c66afbcd7 Address PR feedback
- factor out bitsPerByte constant
- panic if the necessary args aren't passed to GetStorageValueMetadataForPackedSlot
- only have one decode integer method in the deocoder file
2019-07-23 13:36:58 -05:00
Elizabeth Engelman 69ad521482 Apply go fmt changes and cleanup 2019-07-19 10:44:36 -05:00
Elizabeth Engelman d30b4ea4f2 Update GetStorageValueMetadata 2019-07-19 10:13:30 -05:00
Elizabeth Engelman 51915f7506 Pass storage values to repo.create as a map 2019-07-18 18:40:38 -05:00
Elizabeth Engelman ebfb4965c7 Add tests for decoding packed storage 2019-07-18 16:21:57 -05:00
Elizabeth Engelman b0fff9a1dd First pass at decoded packed storage slots 2019-07-18 13:30:04 -05:00
Edvard Hübinette 2c092e8d04 [VDB-751 VDB-754] Bugfix null pointer panic and improve logging (#119)
* VDB-751 VDB-754 Bugfix null pointer panic and improve logging

* Fix typo
2019-07-18 09:21:40 +02:00
Andrew J Yao 11a65c525c Adds ability to set log level from toml or CLI args 2019-06-27 11:14:11 -07:00
Rob Mulholand f0a7a7d999 Merge pull request #103 from vulcanize/remove-queued-storage-duplicates
Don't duplicate queued storage diffs
2019-06-17 15:55:35 -05:00
Rob Mulholand 7d6629e5b4 Merge pull request #102 from vulcanize/fix-mappings-util
Fix mappings util
2019-06-17 15:45:55 -05:00
Elizabeth Engelman 948246cf83 Address PR comments 2019-06-17 14:49:25 -05:00
Rob Mulholand e11f2c8c59 Don't duplicate queued storage diffs
- currently, if we don't recognize the same diff several times (e.g.
  if you restart the storage diff watcher pointed at the same file),
  we'll add the same row to the queue on each run.
- these changes assure we only queue an unrecognized diff once.
2019-06-14 11:26:15 -05:00
Rob Mulholand 95fa6280c5 Fix mappings util
- Remove hex prefix so that the function can accept keys with or
  without a hex prefix and return the same result
2019-06-13 18:33:22 -05:00
Elizabeth Engelman fd126c7bea Remove unused RecheckHeaders method and update Repo tests 2019-06-13 15:19:23 -05:00
Elizabeth Engelman f292888195 Refactor checked column specific SQL 2019-06-13 12:36:29 -05:00
Elizabeth Engelman adf91df23a Remove unused RecheckHeaders method from the events Repository interface 2019-06-13 10:33:53 -05:00
Elizabeth Engelman ede6ad1c8f Remove unused MissingHeaders method from the events Repository interface 2019-06-13 10:33:48 -05:00
Elizabeth Engelman ee13d5c1df Remove unused recheckHeaders bool argument from Transformer.Execute 2019-06-13 10:33:16 -05:00