Commit Graph

11562 Commits

Author SHA1 Message Date
Ian Norden
e37ce73971 handle storage deletions 2020-05-17 20:43:23 -05:00
Ian Norden
9b6e486799 use new contract in unit tests that has self-destruct ability, so we can test eip-158 since simply moving an account to new path doesn't count as 'touchin' it 2020-05-17 15:14:53 -05:00
Ian Norden
face00ca01 found an account whose creation causes the empty account to be moved to a new path; this should count as 'touching; the empty account and cause it to be removed according to eip-158... but it doesn't 2020-05-15 21:12:17 -05:00
Ian Norden
3c514837f3 adjust chain maker for tests to add an empty account in block1 and switch to EIP-158 afterwards (now we just need to generate enough accounts until one causes the empty account to be touched and removed post-EIP-158 so we can simulate and test that process...); also added 2 new blocks where more contract storage is set and old slots are set to zero so they are removed so we can test that 2020-05-15 20:06:06 -05:00
Ian Norden
859e18aea4 cleanup chain maker 2020-05-14 22:46:21 -05:00
Ian Norden
c4529a1139 storage leafkey filter test 2020-05-14 17:39:51 -05:00
Ian Norden
e7b78dae41 update api_test.go 2020-05-14 16:14:40 -05:00
Ian Norden
ea9a8feb81 update mainnet blocks test 2020-05-14 16:14:29 -05:00
Ian Norden
2ad905ff75 comments; explain logic 2020-05-14 16:03:36 -05:00
Ian Norden
aa7c5b0869 add ability to watch specific storage slots (leaf keys) only 2020-05-13 14:28:35 -05:00
Ian Norden
347a7ba0e8 update top level tests 2020-05-13 14:03:45 -05:00
Ian Norden
93781fba83 update service and api; statediffing params are now defined by user through api rather than by service provider by cli 2020-05-13 14:03:45 -05:00
Ian Norden
afd3a61bf0 remove unneeded bc methods 2020-05-13 14:03:40 -05:00
Ian Norden
e5fee80511 remove cli params; moving them to subscriber defined 2020-05-13 12:42:07 -05:00
Ian Norden
b7048a3d66 fix builder so that we handle account deletions properly and properly diff storage when an account is moved to a new path; update params 2020-05-13 12:40:19 -05:00
Ian Norden
b7c412d303 diff type for representing deleted accounts 2020-05-10 15:44:21 -05:00
Ian Norden
5c3e0d6437 complete tests for early mainnet blocks 2020-05-08 16:41:11 -05:00
Ian Norden
5598d67872 common.BytesToHash(path) => crypto.Keaccak256(hash) in builder; BytesToHash produces same hash for e.g. []byte{} and []byte{\x00} - prefix \x00 steps are inconsequential to the hash result 2020-05-08 11:58:22 -05:00
Ian Norden
b5a9ea52e0 builder test on mainnet blocks 2020-05-07 22:48:55 -05:00
Ian Norden
0e0ebd7cb3 rlp files for mainnet blocks 0-3, for tests 2020-05-07 22:48:55 -05:00
Ian Norden
8c9ac32245 add genesis block to test; handle block 0 in StateDiffAt 2020-05-07 22:48:49 -05:00
Ian Norden
d07dc803d2 adjust statediff builder tests to changes and extend to test intermediate nodes; golint 2020-03-11 13:25:36 -05:00
Ian Norden
c2c80f32d9 fix state diff builder: emit actual leaf nodes instead of value nodes; diff on the leaf not on the value; emit correct path for intermediate nodes 2020-03-11 13:25:27 -05:00
Ian Norden
c7bfb7eda3 include total difficulty to the payload 2020-03-05 10:12:09 -06:00
Ian Norden
6e05b78bd0 fix linter issues 2020-03-05 10:12:09 -06:00
Ian Norden
2395e70a61 statediff verison meta 2020-03-05 10:12:09 -06:00
Ian Norden
2364e07975 fixes after rebase 2020-03-05 10:11:31 -06:00
Ian Norden
8eb2e64435 review fixes 2020-03-05 10:11:31 -06:00
Ian Norden
a8b456883f rpc method to get statediff at specific block; requires archival node or the block be within the pruning range 2020-03-05 10:11:31 -06:00
Ian Norden
b3f10c9e69 relay receipts with the rest of the data + review fixes/changes 2020-03-05 10:11:31 -06:00
Ian Norden
f3e4e85cd7 adjust buffering to improve stability; doc.go; fix notifier
err handling
2020-03-05 10:11:31 -06:00
Ian Norden
ad87bad4de review fixes; proper method signature for api; adjust service so that statediff processing is halted/paused until there is at least one subscriber listening for the results 2020-03-05 10:11:31 -06:00
Ian Norden
8543581b5d review fixes and fixes for issues ran into in integration 2020-03-05 10:11:31 -06:00
Ian Norden
35538aea05 cli parameter to limit statediffing to select account addresses + test 2020-03-05 10:11:31 -06:00
Ian Norden
19489d3b34 make state diff rlp serializable 2020-03-05 10:11:31 -06:00
Ian Norden
e7487003f2 option to process intermediate nodes 2020-03-05 10:11:31 -06:00
Ian Norden
5c62d0bcb1 make proofs and paths optional + compress service loop into single for loop (may be missing something here) 2020-03-05 10:11:31 -06:00
Ian Norden
71b41b5c77 refactoring state diff service and adding api which allows for streaming state diff payloads over an rpc websocket subscription 2020-03-05 10:11:31 -06:00
Ian Norden
bf5a25478e refactoring statediff builder and types and adjusted to relay proofs and paths (still need to make this optional) 2020-03-05 10:11:31 -06:00
Ian Norden
3e2450bac4 refactoring/reorganizing packages 2020-03-05 10:11:31 -06:00
Elizabeth
4c284ea5c1 Statediff for full node (#6)
* Open a trie from the in-memory database

* Use a node's LeafKey as an identifier instead of the address

It was proving difficult to find look the address up from a given path
with a full node (sometimes the value wouldn't exist in the disk db).
So, instead, for now we are using the node's LeafKey with is a Keccak256
hash of the address, so if we know the address we can figure out which
LeafKey it matches up to.

* Make sure that statediff has been processed before pruning

* Use blockchain stateCache.OpenTrie for storage diffs

* Clean up log lines and remove unnecessary fields from builder

* Apply go fmt changes

* Add a sleep to the blockchain test

* Address PR comments

* Address PR comments
2020-03-05 10:11:31 -06:00
Elizabeth
18c0083e10 Gracefully exit geth command(#4) 2020-03-05 10:10:50 -06:00
Elizabeth Engelman
8c01c9a6b8 Apply gosimple changes to statediff 2020-03-05 10:10:50 -06:00
Elizabeth Engelman
95f16994fa Apply goimports to statediff 2020-03-05 10:10:50 -06:00
Elizabeth Engelman
6b93154c3d Apply go fmt changes to statediff 2020-03-05 10:10:50 -06:00
Elizabeth
39c2a55942 Write state diff to CSV (#2)
* port statediff from 9b7fd9af80/statediff/statediff.go; minor fixes

* integrating state diff extracting, building, and persisting into geth processes

* work towards persisting created statediffs in ipfs; based off github.com/vulcanize/eth-block-extractor

* Add a state diff service

* Remove diff extractor from blockchain

* Update imports

* Move statediff on/off check to geth cmd config

* Update starting state diff service

* Add debugging logs for creating diff

* Add statediff extractor and builder tests and small refactoring

* Start to write statediff to a CSV

* Restructure statediff directory

* Pull CSV publishing methods into their own file

* Reformatting due to go fmt

* Add gomega to vendor dir

* Remove testing focuses

* Update statediff tests to use golang test pkg

instead of ginkgo

- builder_test
- extractor_test
- publisher_test

* Use hexutil.Encode instead of deprecated common.ToHex

* Remove OldValue from DiffBigInt and DiffUint64 fields

* Update builder test

* Remove old storage value from updated accounts

* Remove old values from created/deleted accounts

* Update publisher to account for only storing current account values

* Update service loop and fetching previous block

* Update testing

- remove statediff ginkgo test suite file
- move mocks to their own dir

* Updates per go fmt

* Updates to tests

* Pass statediff mode and path in through cli

* Return filename from publisher

* Remove some duplication in builder

* Remove code field from state diff output

this is the contract byte code, and it can still be obtained by querying
the db by the codeHash

* Consolidate acct diff structs for updated & updated/deleted accts

* Include block number in csv filename

* Clean up error logging

* Cleanup formatting, spelling, etc

* Address PR comments

* Add contract address and storage value to csv

* Refactor accumulating account row in csv publisher

* Add DiffStorage struct

* Add storage key to csv

* Address PR comments

* Fix publisher to include rows for accounts that don't have store updates

* Update builder test after merging in release/1.8

* Update test contract to include storage on contract intialization

- so that we're able to test that storage diffing works for created and
deleted accounts (not just updated accounts).

* Factor out a common trie iterator method in builder
2020-03-05 10:10:50 -06:00
Péter Szilágyi
6a62fe399b
params: release Geth v1.9.11 stable 2020-02-18 13:26:00 +02:00
Felix Lange
dae3aee5ff
les: add bootstrap nodes as initial discoveries (#20688) 2020-02-18 13:24:05 +02:00
Péter Szilágyi
05ccbb5edd
Merge pull request #20687 from karalabe/cht-1.9.11
params: update CHTs for the v1.9.11 release
2020-02-18 10:57:05 +02:00
Péter Szilágyi
4f55e24c02
params: update CHTs for the v1.9.11 release 2020-02-18 10:55:44 +02:00