Commit Graph

7 Commits

Author SHA1 Message Date
51d20540ff Implement WriteStateDiffAt
Writes state diffs directly to postgres

Adds CLI flags to configure PG

Refactors builder output with callbacks

Copies refactored postgres handling code from ipld-eth-indexer
2020-10-23 21:47:44 +08:00
Ian Norden
44aefa849b add endpoint to fetch all state and storage nodes at a given blockheight; useful for generating a recent atate cache/snapshot that we can diff forward from rather than needing to collect all diffs from genesis 2020-05-18 20:44:17 -05:00
Ian Norden
347a7ba0e8 update top level tests 2020-05-13 14:03:45 -05:00
Ian Norden
19489d3b34 make state diff rlp serializable 2020-03-05 10:11:31 -06:00
Ian Norden
71b41b5c77 refactoring state diff service and adding api which allows for streaming state diff payloads over an rpc websocket subscription 2020-03-05 10:11:31 -06:00
Ian Norden
3e2450bac4 refactoring/reorganizing packages 2020-03-05 10:11:31 -06:00
Elizabeth
39c2a55942 Write state diff to CSV (#2)
* port statediff from 9b7fd9af80/statediff/statediff.go; minor fixes

* integrating state diff extracting, building, and persisting into geth processes

* work towards persisting created statediffs in ipfs; based off github.com/vulcanize/eth-block-extractor

* Add a state diff service

* Remove diff extractor from blockchain

* Update imports

* Move statediff on/off check to geth cmd config

* Update starting state diff service

* Add debugging logs for creating diff

* Add statediff extractor and builder tests and small refactoring

* Start to write statediff to a CSV

* Restructure statediff directory

* Pull CSV publishing methods into their own file

* Reformatting due to go fmt

* Add gomega to vendor dir

* Remove testing focuses

* Update statediff tests to use golang test pkg

instead of ginkgo

- builder_test
- extractor_test
- publisher_test

* Use hexutil.Encode instead of deprecated common.ToHex

* Remove OldValue from DiffBigInt and DiffUint64 fields

* Update builder test

* Remove old storage value from updated accounts

* Remove old values from created/deleted accounts

* Update publisher to account for only storing current account values

* Update service loop and fetching previous block

* Update testing

- remove statediff ginkgo test suite file
- move mocks to their own dir

* Updates per go fmt

* Updates to tests

* Pass statediff mode and path in through cli

* Return filename from publisher

* Remove some duplication in builder

* Remove code field from state diff output

this is the contract byte code, and it can still be obtained by querying
the db by the codeHash

* Consolidate acct diff structs for updated & updated/deleted accts

* Include block number in csv filename

* Clean up error logging

* Cleanup formatting, spelling, etc

* Address PR comments

* Add contract address and storage value to csv

* Refactor accumulating account row in csv publisher

* Add DiffStorage struct

* Add storage key to csv

* Address PR comments

* Fix publisher to include rows for accounts that don't have store updates

* Update builder test after merging in release/1.8

* Update test contract to include storage on contract intialization

- so that we're able to test that storage diffing works for created and
deleted accounts (not just updated accounts).

* Factor out a common trie iterator method in builder
2020-03-05 10:10:50 -06:00