Commit Graph

16 Commits

Author SHA1 Message Date
Ian Norden
e3b6d3df1c tests for eip-158 account removal and storage value deletions; there is one edge case left to test where we remove 1 account when only two exist such that the remaining account is moved up and replaces the root branch node 2020-05-17 23:04:31 -05:00
Ian Norden
9b6e486799 use new contract in unit tests that has self-destruct ability, so we can test eip-158 since simply moving an account to new path doesn't count as 'touchin' it 2020-05-17 15:14:53 -05:00
Ian Norden
face00ca01 found an account whose creation causes the empty account to be moved to a new path; this should count as 'touching; the empty account and cause it to be removed according to eip-158... but it doesn't 2020-05-15 21:12:17 -05:00
Ian Norden
3c514837f3 adjust chain maker for tests to add an empty account in block1 and switch to EIP-158 afterwards (now we just need to generate enough accounts until one causes the empty account to be touched and removed post-EIP-158 so we can simulate and test that process...); also added 2 new blocks where more contract storage is set and old slots are set to zero so they are removed so we can test that 2020-05-15 20:06:06 -05:00
Ian Norden
5c3e0d6437 complete tests for early mainnet blocks 2020-05-08 16:41:11 -05:00
Ian Norden
b5a9ea52e0 builder test on mainnet blocks 2020-05-07 22:48:55 -05:00
Ian Norden
d07dc803d2 adjust statediff builder tests to changes and extend to test intermediate nodes; golint 2020-03-11 13:25:36 -05:00
Ian Norden
c2c80f32d9 fix state diff builder: emit actual leaf nodes instead of value nodes; diff on the leaf not on the value; emit correct path for intermediate nodes 2020-03-11 13:25:27 -05:00
Ian Norden
2364e07975 fixes after rebase 2020-03-05 10:11:31 -06:00
Ian Norden
19489d3b34 make state diff rlp serializable 2020-03-05 10:11:31 -06:00
Ian Norden
71b41b5c77 refactoring state diff service and adding api which allows for streaming state diff payloads over an rpc websocket subscription 2020-03-05 10:11:31 -06:00
Ian Norden
3e2450bac4 refactoring/reorganizing packages 2020-03-05 10:11:31 -06:00
Elizabeth
4c284ea5c1 Statediff for full node (#6)
* Open a trie from the in-memory database

* Use a node's LeafKey as an identifier instead of the address

It was proving difficult to find look the address up from a given path
with a full node (sometimes the value wouldn't exist in the disk db).
So, instead, for now we are using the node's LeafKey with is a Keccak256
hash of the address, so if we know the address we can figure out which
LeafKey it matches up to.

* Make sure that statediff has been processed before pruning

* Use blockchain stateCache.OpenTrie for storage diffs

* Clean up log lines and remove unnecessary fields from builder

* Apply go fmt changes

* Add a sleep to the blockchain test

* Address PR comments

* Address PR comments
2020-03-05 10:11:31 -06:00
Elizabeth Engelman
95f16994fa Apply goimports to statediff 2020-03-05 10:10:50 -06:00
Elizabeth Engelman
6b93154c3d Apply go fmt changes to statediff 2020-03-05 10:10:50 -06:00
Elizabeth
39c2a55942 Write state diff to CSV (#2)
* port statediff from 9b7fd9af80/statediff/statediff.go; minor fixes

* integrating state diff extracting, building, and persisting into geth processes

* work towards persisting created statediffs in ipfs; based off github.com/vulcanize/eth-block-extractor

* Add a state diff service

* Remove diff extractor from blockchain

* Update imports

* Move statediff on/off check to geth cmd config

* Update starting state diff service

* Add debugging logs for creating diff

* Add statediff extractor and builder tests and small refactoring

* Start to write statediff to a CSV

* Restructure statediff directory

* Pull CSV publishing methods into their own file

* Reformatting due to go fmt

* Add gomega to vendor dir

* Remove testing focuses

* Update statediff tests to use golang test pkg

instead of ginkgo

- builder_test
- extractor_test
- publisher_test

* Use hexutil.Encode instead of deprecated common.ToHex

* Remove OldValue from DiffBigInt and DiffUint64 fields

* Update builder test

* Remove old storage value from updated accounts

* Remove old values from created/deleted accounts

* Update publisher to account for only storing current account values

* Update service loop and fetching previous block

* Update testing

- remove statediff ginkgo test suite file
- move mocks to their own dir

* Updates per go fmt

* Updates to tests

* Pass statediff mode and path in through cli

* Return filename from publisher

* Remove some duplication in builder

* Remove code field from state diff output

this is the contract byte code, and it can still be obtained by querying
the db by the codeHash

* Consolidate acct diff structs for updated & updated/deleted accts

* Include block number in csv filename

* Clean up error logging

* Cleanup formatting, spelling, etc

* Address PR comments

* Add contract address and storage value to csv

* Refactor accumulating account row in csv publisher

* Add DiffStorage struct

* Add storage key to csv

* Address PR comments

* Fix publisher to include rows for accounts that don't have store updates

* Update builder test after merging in release/1.8

* Update test contract to include storage on contract intialization

- so that we're able to test that storage diffing works for created and
deleted accounts (not just updated accounts).

* Factor out a common trie iterator method in builder
2020-03-05 10:10:50 -06:00