deep-stack/ipld-eth-server
6
0
Fork 0
forked from cerc-io/ipld-eth-server
Code Pull Requests Activity
1,022 Commits 12 Branches 50 Tags 64 MiB
dc06991605
Commit Graph

118 Commits

Author SHA1 Message Date
Elizabeth Engelman
dc06991605 Wire up the streamer with a fetcher 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
1b4a901892 Rename storage_fetcher -> csv_tail_storage_fetcher 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
9de631f3a7 Add a statediff streamer 2019-09-25 16:32:27 -05:00
Rob Mulholand
b96a1ee1c6 Put secondary processes to sleep in event watcher tests 
- Prevent extract/delegate from spinning when other side is being
  simulated
 2019-09-18 22:17:02 -05:00
Rob Mulholand
4fa19be90a Return error when no logs/headers available 
- Replaces bool and moots question of error/bool ordering
- Also make event watcher execution synchronous
 2019-09-18 20:55:15 -05:00
Rob Mulholand
3f9b034c4c Rename checked_logs => watched_logs 
- We're logging that a given log has been included in any fetch calls
  for checked headers, rather than that we have already checked for
  that log
 2019-09-10 21:22:14 -05:00
Rob Mulholand
13d503b851 Distinguish between missing and unchecked headers 
- Missing == not in DB
- Unchecked == logs haven't been fetched
 2019-09-10 14:39:45 -05:00
Rob Mulholand
c568fedd89 Remove unused functions on mock event repository 2019-09-10 14:20:47 -05:00
Rob Mulholand
5ac76eee74 Make check_count a column on public.headers 
- Don't need to maintain it on public.checked_headers if we're not
  adding additional columns to that table
 2019-08-28 09:50:17 -05:00
Rob Mulholand
666ea1c325 Update checked headers for new transformers 
- If a header was marked as checked before a transformer was added to
  the watcher, mark all headers since the new transformer's starting
  block number as unchecked.
 2019-08-28 09:25:14 -05:00
Rob Mulholand
d76be4962b Remove unnecessary async from the event watcher 
- extract and delegate logs synchronously after initial goroutine fired
 2019-08-28 09:25:13 -05:00
Rob Mulholand
1883a11ab1 Update comment in log chunker 
Co-Authored-By: Edvard Hübinette <edvard@hubinette.me>
 2019-08-28 09:25:13 -05:00
Rob Mulholand
63dabbb051 Extract and delegate logs concurrently 2019-08-28 09:25:13 -05:00
Rob Mulholand
d496dad33c Decouple log extraction from transformer delegation 
- limit missing headers results set to 100 so that extraction doesn't
  excessively block delegation
- wrap checked headers functions in repository struct
- move storage repository to factory, to correspond with event
  repository path
- remove unused files
- reformat sql
- remove line breaks in imports
 2019-08-28 09:25:13 -05:00
Rob Mulholand
cb819fa9a6 Write event logs to database before transforming 
- enables decoupling event extraction/persistence from transformation
- modifies event transformer, converter, and log chunker to accept
  payload that includes internal log database ID with log data
- remove alias for transformer pkg as shared_t
- remove unused mock watcher repository
 2019-08-28 09:13:44 -05:00
Rob Mulholand
66a4e20b20 Update event transformer to take header ID 
- rather than header
- enables executing transformers without full header lookup
 2019-08-28 09:11:06 -05:00
Gabe Laughlin
4f8b057726
(VDB-768) Allow addresses in packed slots to be decoded 2019-08-01 22:40:50 -05:00
Andrew J Yao
dc8d52a58d Removes _checked from column name query 2019-07-30 18:07:25 -07:00
Elizabeth
85aba29cf2
Merge pull request #121 from vulcanize/allow-for-packed-storage-slots 
VDB-318 Allow for packed storage slots
 2019-07-23 13:48:36 -05:00
Elizabeth Engelman
2c66afbcd7 Address PR feedback 
- factor out bitsPerByte constant
- panic if the necessary args aren't passed to GetStorageValueMetadataForPackedSlot
- only have one decode integer method in the deocoder file
 2019-07-23 13:36:58 -05:00
Elizabeth Engelman
69ad521482 Apply go fmt changes and cleanup 2019-07-19 10:44:36 -05:00
Elizabeth Engelman
d30b4ea4f2 Update GetStorageValueMetadata 2019-07-19 10:13:30 -05:00
Elizabeth Engelman
51915f7506 Pass storage values to repo.create as a map 2019-07-18 18:40:38 -05:00
Elizabeth Engelman
ebfb4965c7 Add tests for decoding packed storage 2019-07-18 16:21:57 -05:00
Elizabeth Engelman
b0fff9a1dd First pass at decoded packed storage slots 2019-07-18 13:30:04 -05:00
Edvard Hübinette
2c092e8d04
[VDB-751 VDB-754] Bugfix null pointer panic and improve logging (#119) 
* VDB-751 VDB-754 Bugfix null pointer panic and improve logging

* Fix typo
 2019-07-18 09:21:40 +02:00
Andrew J Yao
11a65c525c Adds ability to set log level from toml or CLI args 2019-06-27 11:14:11 -07:00
Rob Mulholand
f0a7a7d999
Merge pull request #103 from vulcanize/remove-queued-storage-duplicates 
Don't duplicate queued storage diffs
 2019-06-17 15:55:35 -05:00
Rob Mulholand
7d6629e5b4
Merge pull request #102 from vulcanize/fix-mappings-util 
Fix mappings util
 2019-06-17 15:45:55 -05:00
Elizabeth Engelman
948246cf83 Address PR comments 2019-06-17 14:49:25 -05:00
Rob Mulholand
e11f2c8c59 Don't duplicate queued storage diffs 
- currently, if we don't recognize the same diff several times (e.g.
  if you restart the storage diff watcher pointed at the same file),
  we'll add the same row to the queue on each run.
- these changes assure we only queue an unrecognized diff once.
 2019-06-14 11:26:15 -05:00
Rob Mulholand
95fa6280c5 Fix mappings util 
- Remove hex prefix so that the function can accept keys with or
  without a hex prefix and return the same result
 2019-06-13 18:33:22 -05:00
Elizabeth Engelman
fd126c7bea Remove unused RecheckHeaders method and update Repo tests 2019-06-13 15:19:23 -05:00
Elizabeth Engelman
f292888195 Refactor checked column specific SQL 2019-06-13 12:36:29 -05:00
Elizabeth Engelman
adf91df23a Remove unused RecheckHeaders method from the events Repository interface 2019-06-13 10:33:53 -05:00
Elizabeth Engelman
ede6ad1c8f Remove unused MissingHeaders method from the events Repository interface 2019-06-13 10:33:48 -05:00
Elizabeth Engelman
ee13d5c1df Remove unused recheckHeaders bool argument from Transformer.Execute 2019-06-13 10:33:16 -05:00
Rob Mulholand
e2909797fc Remove handling of duplicate storage diffs in watcher 
- Can push this responsibility down to the transformers
- Update docs to reflect that transformers should handle duplicates
 2019-05-20 13:29:09 -05:00
Rob Mulholand
3b531950f2
Merge pull request #93 from vulcanize/vdb-570-storage-duplicates 
(VDB-570) Handle duplicate storage diffs
 2019-05-02 13:40:44 -05:00
Rob Mulholand
36c4da372b (VDB-570) Handle duplicate storage diffs 
- If processing a new diff for a row that already exists in the DB,
  ignore the error without logging or queueing
- If processing a queued diff for a row that already exists, remove
  it from the queue
 2019-05-02 11:39:16 -05:00
Elizabeth
77b7ee9e4c
Merge pull request #91 from vulcanize/vdb-558-remove-log-note-transformer 
Remove LogNote Transformer and Converter
 2019-05-02 11:26:10 -05:00
Rob Mulholand
b036053937 Queue storage diffs if transformer execution fails 
- For any error, not just if key isn't recognized
- Means we don't lose track of diffs on random ephemeral errors
 2019-05-01 12:32:39 -05:00
Rob Mulholand
76ab914bdc Add license 2019-05-01 12:32:39 -05:00
Rob Mulholand
d77f3fe012 Don't pass empty row to channel on error 2019-05-01 12:30:37 -05:00
Rob Mulholand
6a86de87b4 (VDB-371) Recheck queued storage 
- Iterate through queued storage at defined interval, popping rows
  from the queue if successfully persisted
 2019-05-01 12:30:37 -05:00
Rob Mulholand
bf4b1687a0 Add get/delete functions to storage queue 2019-05-01 12:30:37 -05:00
Rob Mulholand
2d684c5aec Extract storage diff fetching behind an interface 
- Replaces directly reading from a CSV
- Simplifies testing
- Should hopefully make it easier to plug in other sources for storage
  diffs (e.g. differently formatted CSVs, JSON RPC, etc)
 2019-05-01 12:30:36 -05:00
Gabe Laughlin
a344432156
(VDB-560) Rename lightSync to headerSync 2019-05-01 12:12:55 -05:00
Elizabeth Engelman
eb1a9cb381 Remove LogNote Transformer and Converter 2019-04-30 12:25:51 -05:00
Rob Mulholand
3fd6269b78 Don't lookup transactions if no log events 
- Prevents EOF error on transactions lookup
 2019-04-18 09:53:18 -05:00