Commit Graph

1074 Commits

Author SHA1 Message Date
Rob Mulholand
e252229b8a Add constraint to prevent duplicate headers
- Disallow inserts of headers with the same number, hash, and node
  fingerprint, since it will enable duplicate log fetching for the
  same header
2019-10-28 14:57:13 -05:00
Rob Mulholand
62e1378e0c
Merge pull request #163 from vulcanize/vdb-929-storage-key-lookup-cleanup
(VDB-929) Minimize storage key lookup bespoke code
2019-10-28 14:56:26 -05:00
Rob Mulholand
b8fec5e4e3 (VDB-929) Minimize storage key lookup bespoke code
- Extract shared namespace for looking up and hashing keys
- Require storage transformers only to implement a loader that
  associates known keys with metadata
- Move key loader/lookup utils to utils directory to avoid
  multiple "storage" packages in imports
2019-10-28 14:29:09 -05:00
Ian Norden
b7675316b4
Merge pull request #158 from vulcanize/missed_marked_checked_headers
Fix for issue #146
2019-10-28 12:34:47 -05:00
Ian Norden
a2d249ca9d review fixes 2019-10-28 11:40:32 -05:00
Ian Norden
4fbde836d4 log sql.ErrNoRows which I suspect is what is leading to the flaky test 2019-10-28 09:37:21 -05:00
Ian Norden
65808998b3 goimports -w; golinting, remove some unused code 2019-10-28 09:37:21 -05:00
Ian Norden
11b5efbfe3 fix for issue #146; mark header checked for contract if it doesnt have
any logs at that header but other contracts do; test
2019-10-28 09:34:42 -05:00
Edvard Hübinette
3fff2896aa
Rename geth to eth, signifying client independence (#161) 2019-10-28 12:30:24 +01:00
Edvard Hübinette
f7c4a6736d
VDB-919 Generalise converter (#152)
* Generalise transformer stack to use InsertionModel

* Add tests for event repository

* Restrict accepted values in InsertionModel

* Add call to repository.SetDB

* Improve error propagation/clarity on GetABI()

* Remove maker references in example

* Please golint

* refactor rollback error handling in repository

* Cleaner errors in repository, refactor tests
2019-10-28 11:48:31 +01:00
Rob Mulholand
6c055a9e12 Pin to urfave/cli version in go.mod (#154)
* Pin to urfave/cli version in go.mod

- Attempting to fix error: github.com/vulcanize/vulcanizedb@v0.0.8
  requires gopkg.in/urfave/cli.v1@v1.0.0-00010101000000-000000000000:
  invalid version: unknown revision 000000000000
2019-10-22 05:58:26 +09:00
Rob Mulholand
7be070fcea
Merge pull request #156 from vulcanize/contract-watcher-init
Enable contractWatcher without prior headerSync
2019-10-10 08:37:48 +09:00
Rob Mulholand
2800e6df36 Enable contractWatcher without prior headerSync
- Previous setup would fail if there were no headers in the db. This
  makes sense because we need headers that haven't been checked for
  logs to exist so that we can fetch logs for those headers. But it
  also prevents us from kicking off the headerSync and contractWatcher
  processes concurrently. These changes enable kicking off both
  processes at the same time with the idea that we will have unchecked
  headers upon transformer execution.
2019-10-04 16:00:13 -05:00
Elizabeth
f6ab9382b2
Address repo updates (#134)
* Factor out get or create address into one sql string

* Factor out getChecksumAddress method in address repo

* Update address repo methods to not need a receiver

* Move address repository to libraries/shared
2019-10-03 11:17:08 -05:00
Rob Mulholand
031043130e
Merge pull request #145 from vulcanize/contract-watcher-logging
Increase logging in contract watcher
2019-10-02 16:30:17 -05:00
Rob Mulholand
4505382590 Increase logging in contract watcher
- Focus on header mode
- Add context to errors, trace guard clauses, warn on non-returned
  errors
- Give errors distinct names so compiler will recognize if unchecked
- Remove redundant type declarations/fix typos
2019-10-02 15:14:42 -05:00
Elizabeth
b4e16c4af5
Merge pull request #116 from vulcanize/storage-diffs-over-rpc
Storage diffs over rpc
2019-10-02 09:56:29 -05:00
Elizabeth Engelman
f0d2741dea Allow storageDiff source to be set through an env variable 2019-10-01 10:24:53 -05:00
Elizabeth Engelman
267de00f99 Update dependencies
specifically golang.org/x/crypto/sha3
2019-09-26 10:41:12 -05:00
Elizabeth Engelman
5c0e5592ab Updates after rebasing 2019-09-25 16:52:42 -05:00
Elizabeth Engelman
9c6182c356 Rename StorageDiff field from KeccakOfContractAddress to HashedAddress 2019-09-25 16:36:44 -05:00
Elizabeth Engelman
6672ecf547 Add trace debugging after parsing statediff 2019-09-25 16:36:44 -05:00
Elizabeth Engelman
f315988507 Factor out a bad storage diff for testing 2019-09-25 16:36:44 -05:00
Elizabeth Engelman
6869330bd3 Decode storage value RLP after fetching from statediffs 2019-09-25 16:36:44 -05:00
Elizabeth Engelman
2931edc317 Pin urfave/cli version to 1.20 2019-09-25 16:36:44 -05:00
Elizabeth Engelman
4166fb24dd Replace any geth version with the vdb statediff patch 2019-09-25 16:36:08 -05:00
Elizabeth Engelman
e1de3afdfc Small changes to helper method names/arg names 2019-09-25 16:36:08 -05:00
Rob Mulholand
2ff88de859 Extract helper for converting hex to keccak256 hash
- Also prefer crypto.Keccak256Hash(x) to common.BytesToHash(crypto.Keccak256(x))
2019-09-25 16:36:08 -05:00
Rob Mulholand
d06dddbfaa Always hash storage diff contract addresses
- Enables syncing Geth and Parity diffs with same transformer lookup
- Maybe worth always hashing the storage key so we don't need a hashed
  and not-hashed version in the key lookups?
2019-09-25 16:36:08 -05:00
Rob Mulholand
f574407bb6 Revert "Update Mappings interface to take a diff as an argument to Lookup"
This reverts commit cda646b78ceb6714b132891ea4489a8ff4fc95e9.
2019-09-25 16:34:31 -05:00
Rob Mulholand
8111f4ec5e Add function for adding hashed keys to mapping
- Tool to facilitate parsing diffs from Geth patch that emits hashed
  versions of storage keys
2019-09-25 16:34:31 -05:00
Elizabeth Engelman
3fb8e13979 Update Mappings interface to take a diff as an argument to Lookup 2019-09-25 16:34:31 -05:00
Elizabeth Engelman
deb0315a49 Remove Gopkg files 2019-09-25 16:34:31 -05:00
Elizabeth Engelman
66d695d93b Update keccak of address to be a common.Hash instead of a common.Address 2019-09-25 16:34:31 -05:00
Elizabeth Engelman
a8a8fe4ac2 Remove vendor dir 2019-09-25 16:34:31 -05:00
Elizabeth Engelman
9831badc2a Use geth patch in go mod 2019-09-25 16:33:58 -05:00
Elizabeth Engelman
8e166f2a56 Update after rebasing with staging 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
639c561fdf Remove unncessary pointer receiver from NewGethRpcStorageFetcher.FetchStorageDiffs 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
9226e53e0b Panic if connecting to geth subscription fails 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
0cc90a1f80 Factor out an IStorageWatcher interface 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
18f47b7b6a Memoizing keccak hhash of address => transformer 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
045d78be25 Rename StorageDiffRow -> StorageDiff 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
87252f42b9 Add additional logging 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
e637024cce Update vendor dir 2019-09-25 16:32:31 -05:00
Elizabeth Engelman
7cd66b8775 Use a for loop for getting diffs off the payload channel 2019-09-25 16:32:30 -05:00
Elizabeth Engelman
7cbad1d89f Override protobuf to version 3 2019-09-25 16:32:30 -05:00
Elizabeth Engelman
963479fdc0 Remove ipfs and eth-block-extractor as dependencies 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
b09b8a8735 Use geth state diff source in execute command 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
70fcc22a00 Apply go formatting 2019-09-25 16:32:27 -05:00
Elizabeth Engelman
a577811e0a Use geth state diff source in composeAndExecute 2019-09-25 16:32:27 -05:00