Commit Graph

28 Commits

Author SHA1 Message Date
Ian Norden
a69ee0b441 fix mock blockchain; golint; bump patch 2020-08-14 14:58:57 -05:00
Ian Norden
97454eb6fe minor changes/fixes 2020-05-21 15:07:19 -05:00
Ian Norden
7ba6efa59c test for state trie builder 2020-05-19 14:39:25 -05:00
Ian Norden
44aefa849b add endpoint to fetch all state and storage nodes at a given blockheight; useful for generating a recent atate cache/snapshot that we can diff forward from rather than needing to collect all diffs from genesis 2020-05-18 20:44:17 -05:00
Ian Norden
f572e8d716 finish testing known edge cases 2020-05-18 01:27:58 -05:00
Ian Norden
859e18aea4 cleanup chain maker 2020-05-14 22:46:21 -05:00
Ian Norden
c4529a1139 storage leafkey filter test 2020-05-14 17:39:51 -05:00
Ian Norden
e7b78dae41 update api_test.go 2020-05-14 16:14:40 -05:00
Ian Norden
347a7ba0e8 update top level tests 2020-05-13 14:03:45 -05:00
Ian Norden
afd3a61bf0 remove unneeded bc methods 2020-05-13 14:03:40 -05:00
Ian Norden
b7c412d303 diff type for representing deleted accounts 2020-05-10 15:44:21 -05:00
Ian Norden
5c3e0d6437 complete tests for early mainnet blocks 2020-05-08 16:41:11 -05:00
Ian Norden
d07dc803d2 adjust statediff builder tests to changes and extend to test intermediate nodes; golint 2020-03-11 13:25:36 -05:00
Ian Norden
c2c80f32d9 fix state diff builder: emit actual leaf nodes instead of value nodes; diff on the leaf not on the value; emit correct path for intermediate nodes 2020-03-11 13:25:27 -05:00
Ian Norden
c7bfb7eda3 include total difficulty to the payload 2020-03-05 10:12:09 -06:00
Ian Norden
6e05b78bd0 fix linter issues 2020-03-05 10:12:09 -06:00
Ian Norden
8eb2e64435 review fixes 2020-03-05 10:11:31 -06:00
Ian Norden
a8b456883f rpc method to get statediff at specific block; requires archival node or the block be within the pruning range 2020-03-05 10:11:31 -06:00
Ian Norden
b3f10c9e69 relay receipts with the rest of the data + review fixes/changes 2020-03-05 10:11:31 -06:00
Ian Norden
ad87bad4de review fixes; proper method signature for api; adjust service so that statediff processing is halted/paused until there is at least one subscriber listening for the results 2020-03-05 10:11:31 -06:00
Ian Norden
35538aea05 cli parameter to limit statediffing to select account addresses + test 2020-03-05 10:11:31 -06:00
Ian Norden
19489d3b34 make state diff rlp serializable 2020-03-05 10:11:31 -06:00
Ian Norden
5c62d0bcb1 make proofs and paths optional + compress service loop into single for loop (may be missing something here) 2020-03-05 10:11:31 -06:00
Ian Norden
71b41b5c77 refactoring state diff service and adding api which allows for streaming state diff payloads over an rpc websocket subscription 2020-03-05 10:11:31 -06:00
Ian Norden
3e2450bac4 refactoring/reorganizing packages 2020-03-05 10:11:31 -06:00
Elizabeth
4c284ea5c1 Statediff for full node (#6)
* Open a trie from the in-memory database

* Use a node's LeafKey as an identifier instead of the address

It was proving difficult to find look the address up from a given path
with a full node (sometimes the value wouldn't exist in the disk db).
So, instead, for now we are using the node's LeafKey with is a Keccak256
hash of the address, so if we know the address we can figure out which
LeafKey it matches up to.

* Make sure that statediff has been processed before pruning

* Use blockchain stateCache.OpenTrie for storage diffs

* Clean up log lines and remove unnecessary fields from builder

* Apply go fmt changes

* Add a sleep to the blockchain test

* Address PR comments

* Address PR comments
2020-03-05 10:11:31 -06:00
Elizabeth
18c0083e10 Gracefully exit geth command(#4) 2020-03-05 10:10:50 -06:00
Elizabeth
39c2a55942 Write state diff to CSV (#2)
* port statediff from 9b7fd9af80/statediff/statediff.go; minor fixes

* integrating state diff extracting, building, and persisting into geth processes

* work towards persisting created statediffs in ipfs; based off github.com/vulcanize/eth-block-extractor

* Add a state diff service

* Remove diff extractor from blockchain

* Update imports

* Move statediff on/off check to geth cmd config

* Update starting state diff service

* Add debugging logs for creating diff

* Add statediff extractor and builder tests and small refactoring

* Start to write statediff to a CSV

* Restructure statediff directory

* Pull CSV publishing methods into their own file

* Reformatting due to go fmt

* Add gomega to vendor dir

* Remove testing focuses

* Update statediff tests to use golang test pkg

instead of ginkgo

- builder_test
- extractor_test
- publisher_test

* Use hexutil.Encode instead of deprecated common.ToHex

* Remove OldValue from DiffBigInt and DiffUint64 fields

* Update builder test

* Remove old storage value from updated accounts

* Remove old values from created/deleted accounts

* Update publisher to account for only storing current account values

* Update service loop and fetching previous block

* Update testing

- remove statediff ginkgo test suite file
- move mocks to their own dir

* Updates per go fmt

* Updates to tests

* Pass statediff mode and path in through cli

* Return filename from publisher

* Remove some duplication in builder

* Remove code field from state diff output

this is the contract byte code, and it can still be obtained by querying
the db by the codeHash

* Consolidate acct diff structs for updated & updated/deleted accts

* Include block number in csv filename

* Clean up error logging

* Cleanup formatting, spelling, etc

* Address PR comments

* Add contract address and storage value to csv

* Refactor accumulating account row in csv publisher

* Add DiffStorage struct

* Add storage key to csv

* Address PR comments

* Fix publisher to include rows for accounts that don't have store updates

* Update builder test after merging in release/1.8

* Update test contract to include storage on contract intialization

- so that we're able to test that storage diffing works for created and
deleted accounts (not just updated accounts).

* Factor out a common trie iterator method in builder
2020-03-05 10:10:50 -06:00