Commit Graph

38 Commits

Author SHA1 Message Date
Rob Mulholand
56ce8bdb41 (VDB-950) Write raw diffs before transforming
- Raw field we can reference by FK for related data
- Enables replay for unwatched or mistransformed diffs
2019-12-03 14:51:17 -06:00
yaoandrew
0b9f29c300 Removes import alias for logrus 2019-12-02 23:08:48 -06:00
Andrew J Yao
e26fa12af0 Adds logging for transformer start, log conversions, log persistance 2019-12-02 22:59:32 -06:00
Ian Norden
a834e55b9f command edits; external pkg for finding min deployment block; gofmt 2019-12-02 11:22:18 -06:00
Ian Norden
a59bd06a37 goimports -w; comments; refactor test 2019-12-02 11:03:36 -06:00
Rob Mulholand
e1236b4072 Prefer all caps for initialisms and acronyms 2019-10-31 13:42:19 -05:00
Rob Mulholand
b8fec5e4e3 (VDB-929) Minimize storage key lookup bespoke code
- Extract shared namespace for looking up and hashing keys
- Require storage transformers only to implement a loader that
  associates known keys with metadata
- Move key loader/lookup utils to utils directory to avoid
  multiple "storage" packages in imports
2019-10-28 14:29:09 -05:00
Edvard Hübinette
f7c4a6736d
VDB-919 Generalise converter (#152)
* Generalise transformer stack to use InsertionModel

* Add tests for event repository

* Restrict accepted values in InsertionModel

* Add call to repository.SetDB

* Improve error propagation/clarity on GetABI()

* Remove maker references in example

* Please golint

* refactor rollback error handling in repository

* Cleaner errors in repository, refactor tests
2019-10-28 11:48:31 +01:00
Elizabeth Engelman
5c0e5592ab Updates after rebasing 2019-09-25 16:52:42 -05:00
Elizabeth Engelman
9c6182c356 Rename StorageDiff field from KeccakOfContractAddress to HashedAddress 2019-09-25 16:36:44 -05:00
Rob Mulholand
2ff88de859 Extract helper for converting hex to keccak256 hash
- Also prefer crypto.Keccak256Hash(x) to common.BytesToHash(crypto.Keccak256(x))
2019-09-25 16:36:08 -05:00
Rob Mulholand
d06dddbfaa Always hash storage diff contract addresses
- Enables syncing Geth and Parity diffs with same transformer lookup
- Maybe worth always hashing the storage key so we don't need a hashed
  and not-hashed version in the key lookups?
2019-09-25 16:36:08 -05:00
Rob Mulholand
f574407bb6 Revert "Update Mappings interface to take a diff as an argument to Lookup"
This reverts commit cda646b78ceb6714b132891ea4489a8ff4fc95e9.
2019-09-25 16:34:31 -05:00
Elizabeth Engelman
3fb8e13979 Update Mappings interface to take a diff as an argument to Lookup 2019-09-25 16:34:31 -05:00
Elizabeth Engelman
045d78be25 Rename StorageDiffRow -> StorageDiff 2019-09-25 16:32:31 -05:00
Rob Mulholand
d496dad33c Decouple log extraction from transformer delegation
- limit missing headers results set to 100 so that extraction doesn't
  excessively block delegation
- wrap checked headers functions in repository struct
- move storage repository to factory, to correspond with event
  repository path
- remove unused files
- reformat sql
- remove line breaks in imports
2019-08-28 09:25:13 -05:00
Rob Mulholand
cb819fa9a6 Write event logs to database before transforming
- enables decoupling event extraction/persistence from transformation
- modifies event transformer, converter, and log chunker to accept
  payload that includes internal log database ID with log data
- remove alias for transformer pkg as shared_t
- remove unused mock watcher repository
2019-08-28 09:13:44 -05:00
Rob Mulholand
66a4e20b20 Update event transformer to take header ID
- rather than header
- enables executing transformers without full header lookup
2019-08-28 09:11:06 -05:00
Elizabeth Engelman
69ad521482 Apply go fmt changes and cleanup 2019-07-19 10:44:36 -05:00
Elizabeth Engelman
51915f7506 Pass storage values to repo.create as a map 2019-07-18 18:40:38 -05:00
Elizabeth Engelman
adf91df23a Remove unused RecheckHeaders method from the events Repository interface 2019-06-13 10:33:53 -05:00
Elizabeth Engelman
ede6ad1c8f Remove unused MissingHeaders method from the events Repository interface 2019-06-13 10:33:48 -05:00
Elizabeth Engelman
ee13d5c1df Remove unused recheckHeaders bool argument from Transformer.Execute 2019-06-13 10:33:16 -05:00
Rob Mulholand
e2909797fc Remove handling of duplicate storage diffs in watcher
- Can push this responsibility down to the transformers
- Update docs to reflect that transformers should handle duplicates
2019-05-20 13:29:09 -05:00
Elizabeth
77b7ee9e4c
Merge pull request #91 from vulcanize/vdb-558-remove-log-note-transformer
Remove LogNote Transformer and Converter
2019-05-02 11:26:10 -05:00
Gabe Laughlin
a344432156
(VDB-560) Rename lightSync to headerSync 2019-05-01 12:12:55 -05:00
Elizabeth Engelman
eb1a9cb381 Remove LogNote Transformer and Converter 2019-04-30 12:25:51 -05:00
Rob Mulholand
39471e1d04 Remove links to Maker 2019-04-01 16:03:35 -05:00
Rob Mulholand
8b5b5b8f17 Remove links to 8th Light 2019-04-01 15:15:08 -05:00
Ian Norden
1aa849bcb4 PR fixes; remove all infura token references and setup travis to use encrypted env
variable; rest of the ethjson_rpc dependent tests extracted to integration_test
2019-03-21 18:36:51 -05:00
Ian Norden
ff55e3ba7a TransformerInitializer => EventTransformerInitializer reorganization 2019-03-14 11:59:39 -05:00
Ian Norden
9f8c50e3ab tests for migration path ordering and errors 2019-03-13 11:42:30 -05:00
Ian Norden
35c5808a07 readme fixes 2019-03-12 12:54:36 -05:00
Ian Norden
1d6219cc83 event transformers readme and update main readme to remove last links to
mcd_transformers
2019-03-12 12:45:29 -05:00
Rob Mulholand
cdcb90c170 Merge pull request #34 from vulcanize/redsquirrel-patch-3
Fixing some typos
2019-03-08 10:56:38 -06:00
Dave Hoover
f03329247d Fixing some typos 2019-03-07 15:33:18 -06:00
Dave Hoover
789477bfbf Minor Solidity suggestion
Just reading through this and figured I'd suggest a more concise snippet.
2019-03-07 15:18:27 -06:00
Ian Norden
708425c4d6 rebase; extract factories and the mocks they are dependent on to
libraries/shared; adjust omni test_helpers to drop and recreate
checked_headers table to avoid reaching 1600 column limit after repeated
tests (dropping columns doesn't actually remove them from contributing
to that limit)
2019-02-25 01:34:38 -06:00