Commit Graph

205 Commits

Author SHA1 Message Date
Ian Norden
d702cb720c update dependencies to work with update eth-block-extractor 2019-12-02 13:24:49 -06:00
Ian Norden
31a9017c4f index all cids in a payload in a single atomic tx; misc fixes; comment additions 2019-12-02 13:24:46 -06:00
Ian Norden
79efaeb089 syncAndPublish command 2019-12-02 13:24:46 -06:00
Ian Norden
6434a7279d integrate backfill into storage watcher; documentation for storage backfill 2019-12-02 13:24:45 -06:00
Ian Norden
db0f024088 review fixes 2019-12-02 11:26:44 -06:00
Ian Norden
a834e55b9f command edits; external pkg for finding min deployment block; gofmt 2019-12-02 11:22:18 -06:00
Ian Norden
37f4a2d603 integrate backfill into storage watcher; documentation for storage backfill 2019-12-02 11:06:28 -06:00
Ian Norden
a59bd06a37 goimports -w; comments; refactor test 2019-12-02 11:03:36 -06:00
Rob Mulholand
e1236b4072 Prefer all caps for initialisms and acronyms 2019-10-31 13:42:19 -05:00
Edvard Hübinette
3fff2896aa
Rename geth to eth, signifying client independence (#161) 2019-10-28 12:30:24 +01:00
Elizabeth Engelman
f0d2741dea Allow storageDiff source to be set through an env variable 2019-10-01 10:24:53 -05:00
Rob Mulholand
d06dddbfaa Always hash storage diff contract addresses
- Enables syncing Geth and Parity diffs with same transformer lookup
- Maybe worth always hashing the storage key so we don't need a hashed
  and not-hashed version in the key lookups?
2019-09-25 16:36:08 -05:00
Elizabeth Engelman
639c561fdf Remove unncessary pointer receiver from NewGethRpcStorageFetcher.FetchStorageDiffs 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
0cc90a1f80 Factor out an IStorageWatcher interface 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
b09b8a8735 Use geth state diff source in execute command 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
a577811e0a Use geth state diff source in composeAndExecute 2019-09-25 16:32:27 -05:00
20ce0ab852 Revert "option to write the plugin .go file and run the plugin migrations without building the .so file so that it can be done from the shell instead to afford complete error messages"
This reverts commit 9bb2f27a69.
2019-09-24 23:54:25 +00:00
Ian Norden
9bb2f27a69 option to write the plugin .go file and run the plugin migrations without building the .so file so that it can be done from the shell instead to afford complete error messages 2019-09-24 18:44:04 -05:00
Rob Mulholand
4fa19be90a Return error when no logs/headers available
- Replaces bool and moots question of error/bool ordering
- Also make event watcher execution synchronous
2019-09-18 20:55:15 -05:00
Rob Mulholand
13d503b851 Distinguish between missing and unchecked headers
- Missing == not in DB
- Unchecked == logs haven't been fetched
2019-09-10 14:39:45 -05:00
Rob Mulholand
666ea1c325 Update checked headers for new transformers
- If a header was marked as checked before a transformer was added to
  the watcher, mark all headers since the new transformer's starting
  block number as unchecked.
2019-08-28 09:25:14 -05:00
Rob Mulholand
63dabbb051 Extract and delegate logs concurrently 2019-08-28 09:25:13 -05:00
Elizabeth Engelman
1b3786338f Address PR feedback 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
58c23c6632 Update Copyrights 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
7e38764618 Rename ReceiptRepository -> FullSyncReceiptRepository 2019-08-23 10:11:37 -05:00
Andrew J Yao
ee77fc6521 Remove injection of SubCommand for logs 2019-07-23 13:11:20 -07:00
Andrew J Yao
a188e1dd79 Incorporate changes from Edvard PR 2019-07-23 08:10:58 -07:00
Andrew J Yao
92d153b010 Add subCommand name to log fields 2019-07-23 08:09:57 -07:00
Andrew J Yao
d7a82e353b Add calling method to debug level log messages 2019-07-23 08:08:15 -07:00
Edvard Hübinette
2c092e8d04
[VDB-751 VDB-754] Bugfix null pointer panic and improve logging (#119)
* VDB-751 VDB-754 Bugfix null pointer panic and improve logging

* Fix typo
2019-07-18 09:21:40 +02:00
Rob Mulholand
62cca62432 Log error on failure to read config file
- include actual error to highlight the specific issue
2019-06-27 13:58:58 -05:00
Andrew J Yao
11a65c525c Adds ability to set log level from toml or CLI args 2019-06-27 11:14:11 -07:00
Rob Mulholand
e2909797fc Remove handling of duplicate storage diffs in watcher
- Can push this responsibility down to the transformers
- Update docs to reflect that transformers should handle duplicates
2019-05-20 13:29:09 -05:00
Elizabeth Engelman
d947c8f30a Rename sync command to fullSync 2019-05-13 16:14:56 -05:00
Elizabeth Engelman
f7d520c933 Update header sync transformer alias in ContractWatcher 2019-05-08 13:42:43 -05:00
Rob Mulholand
36c4da372b (VDB-570) Handle duplicate storage diffs
- If processing a new diff for a row that already exists in the DB,
  ignore the error without logging or queueing
- If processing a queued diff for a row that already exists, remove
  it from the queue
2019-05-02 11:39:16 -05:00
Rob Mulholand
6716c3b92a Make queue recheck interval configurable via CLI 2019-05-01 12:32:39 -05:00
Rob Mulholand
6a86de87b4 (VDB-371) Recheck queued storage
- Iterate through queued storage at defined interval, popping rows
  from the queue if successfully persisted
2019-05-01 12:30:37 -05:00
Rob Mulholand
2d684c5aec Extract storage diff fetching behind an interface
- Replaces directly reading from a CSV
- Simplifies testing
- Should hopefully make it easier to plug in other sources for storage
  diffs (e.g. differently formatted CSVs, JSON RPC, etc)
2019-05-01 12:30:36 -05:00
Gabe Laughlin
a344432156
(VDB-560) Rename lightSync to headerSync 2019-05-01 12:12:55 -05:00
Ian Norden
1aa849bcb4 PR fixes; remove all infura token references and setup travis to use encrypted env
variable; rest of the ethjson_rpc dependent tests extracted to integration_test
2019-03-21 18:36:51 -05:00
Ian Norden
d3e172d9ab remove references to infura token except for in infura.toml so that tests will continue to pass on CI 2019-03-21 18:36:49 -05:00
Ian Norden
37fc038605 fixes for review comments 2019-03-21 18:36:49 -05:00
Ian Norden
24879a85aa mv pkg/omni pkg/contract_watcher 2019-03-21 18:33:56 -05:00
Ian Norden
073378de6d update © to 2019 2019-03-21 18:33:56 -05:00
Ian Norden
e4e092f542 changes to plugin and commands to accomodate changes 2019-03-21 18:33:56 -05:00
Ian Norden
8174ce4aee refactor omni code; rename omniWatcher => contractWatcher; use config instead of excessive number of CLI flags 2019-03-21 18:30:48 -05:00
Ian Norden
ff55e3ba7a TransformerInitializer => EventTransformerInitializer reorganization 2019-03-14 11:59:39 -05:00
Ian Norden
9f8c50e3ab tests for migration path ordering and errors 2019-03-13 11:42:30 -05:00
Ian Norden
05cb06c2d3 config example fixes 2019-03-13 11:42:29 -05:00