Port iterator pkg here #276

Merged
telackey merged 51 commits from port_iterator into v1.10.26-statediff-v4 2023-01-24 21:10:13 +00:00
51 Commits
Author SHA1 Message Date
i-norden 2b033f2f3c remove logrus, use geth logging pkg 2023-01-24 14:48:26 -06:00
i-norden eb482ee783 Merge branch 'v1.10.26-statediff-v4' into port_iterator 2023-01-24 14:13:32 -06:00
i-norden 1385209eda update go mod 2022-09-13 07:31:06 -05:00
i-norden 37f60b6300 move pkg 2022-09-09 15:14:54 -05:00
i-norden c5076e7d37 Merge branch 'master' of /Users/iannorden/go/src/github.com/cerc-io/go-eth-state-node-iterator into port_iterator 2022-09-09 15:12:34 -05:00
i-norden ba680f44ce repkg 2022-09-08 22:47:31 -05:00
roysc 3d57aa34fa Merge pull request #18 from roysc/add-tracker
Add tracker package
2022-09-06 07:40:12 -05:00
roysc d72e3214c1 Add tracker package
Moved from snapshot repo
2022-09-06 14:38:09 +02:00
Michael 2508990948 Merge pull request #19 from vulcanize/rebase-1.10.23-wip
dependency update to 1.10.23 of vulcanize statediffing geth
2022-09-01 16:51:53 -04:00
Michael Shaw 71af7dfd2a dependency update to 1.10.23 of vulcanize statediffing geth 2022-09-01 16:19:23 -04:00
MichaelandAbdul Rabbani 918a30dcc9 V1.10.23 statediff v4 wip (#275)
* Statediff Geth

Handle conflicts (#244)

* Handle conflicts

* Update go mod file versions

* Make lint changes

Disassociate block number from the indexer object

Update ipld-eth-db ref

Refactor builder code to make it reusable

Use prefix comparison for account selective statediffing

Update builder unit tests

Add mode to write to CSV files in statediff file writer (#249)

* Change file writing mode to csv files

* Implement writer interface for file indexer

* Implement option for csv or sql in file mode

* Close files in CSV writer

* Add tests for CSV file mode

* Implement CSV file for watched addresses

* Separate test configs for CSV and SQL

* Refactor common code for file indexer tests

Update indexer to include block hash in receipts and logs (#256)

* Update indexer to include block hash in receipts and logs

* Upgrade ipld-eth-db image in docker-compose to run tests

Use watched addresses from direct indexing params by default while serving statediff APIs (#262)

* Use watched addresses from direct indexing params in statediff APIs by default

* Avoid using indexer object when direct indexing is off

* Add nil check before accessing watched addresses from direct indexing params

Rebase missed these changes needed at 1.10.20

Flags cleanup for CLI changes and linter complaints

Linter appeasements to achieve perfection

enforce go 1.18 for check (#267)

* enforce go 1.18 for check

* tests on 1.18 as well

* adding db yml for possible change in docker-compose behavior in yml parsing

Add indexer tests for handling non canonical blocks (#254)

* Add indexer tests for header and transactions in a non canonical block

* Add indexer tests for receipts in a non-canonical block and refactor

* Add indexer tests for logs in a non-canonical block

* Add indexer tests for state and storage nodes in a non-canonical block

* Add indexer tests for non-canonical block at another height

* Avoid passing address of a pointer

* Update refs in GitHub workflow

* Add genesis file path to stack-orchestrator config in GitHub workflow

* Add descriptive comments

fix non-deterministic ordering in unit tests

Refactor indexer tests to avoid duplicate code (#270)

* Refactor indexer tests to avoid duplicate code

* Refactor file mode indexer tests

* Fix expected db stats for sqlx after tx closure

* Refactor indexer tests for legacy block

* Refactor mainnet indexer tests

* Refactor tests for watched addressess methods

* Fix query in legacy indexer test

rebase and resolve onto 1.10.23... still error out of index related to GetLeafKeys

changed trie.Commit behavior was subtle about not not flushing to disk without an Update

* no merge nodeset throws nil

* linter appeasement

Co-authored-by: Abdul Rabbani <abdulrabbani00@gmail.com>
2022-09-01 14:34:58 -04:00
prathamesh0 b40114b739 Store start path in prefix bound iterator object (#17) 2022-08-03 16:12:45 +05:30
Michael Shaw c6ac14bf08 go mody tidy for geth 1.10.21 2022-07-29 15:10:02 -04:00
prathamesh0 9fde91f0a0 Merge pull request #16 from vulcanize/release-v1.1.2
go-ethereum 1.10.20 update with go mod tidy
2022-07-29 16:55:52 +05:30
Michael Shaw 2de2187a00 go-ethereum 1.10.20 update with go mod tidy 2022-07-18 18:35:34 -04:00
erikdies 58b943f532 Create issues-notion-sync.yml 2022-06-29 09:12:43 -04:00
Abdul Rabbani 833627c276 Merge pull request #15 from vulcanize/feature/update-geth-v.1.10.19
Update geth version
2022-06-17 08:20:53 -04:00
Abdul Rabbani e9a8abe8e9 Update geth version
```
08:19:37:~/GitHub/cerc/go-eth-state-node-iterator go test ./... -v
=== RUN   TestMakePaths
--- PASS: TestMakePaths (0.00s)
=== RUN   TestIterator
=== RUN   TestIterator/in_bounds
=== RUN   TestIterator/in_bounds/case
=== RUN   TestIterator/in_bounds/case#01
=== RUN   TestIterator/in_bounds/case#02
=== RUN   TestIterator/in_bounds/case#03
=== RUN   TestIterator/in_bounds/case#04
=== RUN   TestIterator/in_bounds/case#05
=== RUN   TestIterator/trie_is_covered
=== RUN   TestIterator/trie_is_covered/1_bins
=== RUN   TestIterator/trie_is_covered/2_bins
=== RUN   TestIterator/trie_is_covered/4_bins
=== RUN   TestIterator/trie_is_covered/8_bins
=== RUN   TestIterator/trie_is_covered/16_bins
=== RUN   TestIterator/trie_is_covered/32_bins
--- PASS: TestIterator (0.15s)
    --- PASS: TestIterator/in_bounds (0.00s)
        --- PASS: TestIterator/in_bounds/case (0.00s)
        --- PASS: TestIterator/in_bounds/case#01 (0.00s)
        --- PASS: TestIterator/in_bounds/case#02 (0.00s)
        --- PASS: TestIterator/in_bounds/case#03 (0.00s)
        --- PASS: TestIterator/in_bounds/case#04 (0.00s)
        --- PASS: TestIterator/in_bounds/case#05 (0.00s)
    --- PASS: TestIterator/trie_is_covered (0.02s)
        --- PASS: TestIterator/trie_is_covered/1_bins (0.00s)
        --- PASS: TestIterator/trie_is_covered/2_bins (0.00s)
        --- PASS: TestIterator/trie_is_covered/4_bins (0.00s)
        --- PASS: TestIterator/trie_is_covered/8_bins (0.00s)
        --- PASS: TestIterator/trie_is_covered/16_bins (0.01s)
        --- PASS: TestIterator/trie_is_covered/32_bins (0.00s)
PASS
ok      github.com/vulcanize/go-eth-state-node-iterator 0.325s
?       github.com/vulcanize/go-eth-state-node-iterator/fixture [no test files]
```
2022-06-17 08:20:04 -04:00
nabarun 81a7fff289 Merge pull request #14 from roysc/fix-hanging-ctor
Fix hanging constructor
2022-06-10 12:47:26 +05:30
roysc 4fbaa45d8e Fix tests for revert 2022-06-08 18:04:44 +08:00
roysc 30472ce3c7 Revert "fix edge behavior and constructor"
This reverts commit c134d25de5.
2022-06-08 18:04:44 +08:00
Abdul Rabbani 8f27a1392a Merge pull request #13 from vulcanize/release-v1.0.3
Update go and geth
2022-05-31 07:36:53 -04:00
Abdul Rabbani 03a6bd4aed Update go and geth 2022-05-27 15:01:28 -04:00
nikugogoi a4f22bc253 Upgrade geth version and remove vulcanize geth dependency 2022-05-17 19:06:49 +05:30
Ian Norden fb86725ef2 Merge pull request #10 from roysc/fixes-tests
Fix edge behavior and expand tests
2022-03-03 08:22:45 -06:00
roysc 08614e0941 cleanup 2022-02-26 12:09:06 +08:00
roysc f6d9fa992d comments 2022-02-23 18:40:37 +08:00
roysc 87c5033edf add test cases 2022-02-23 18:29:41 +08:00
roysc ff21f3cbcb add fixture/chaindata 2022-02-23 18:29:37 +08:00
roysc c134d25de5 fix edge behavior and constructor 2022-02-22 16:16:02 +08:00
roysc c532c17b05 export end path field 2022-02-21 16:30:26 +08:00
roysc f25ee8319c simplify and clean up 2022-02-21 16:30:26 +08:00
Ian Norden 8e349dfbaa Merge pull request #9 from roysc/dev
Fix and update go.mod
2022-01-26 11:55:09 -06:00
i-norden 25f9744744 bump to v1.10.15-statediff-2.0.0 2022-01-26 11:53:32 -06:00
roysc 6bba614243 Patches, test, comments 2022-01-27 01:34:29 +08:00
Arijit Das 89b3064c00 Merge pull request #8 from vulcanize/upgrade-geth
Upgrade geth to v1.10.9.
2021-10-19 11:36:12 +05:30
Arijit Das d23d01ed81 Upgrade geth to v1.10.9. 2021-10-14 12:19:06 +05:30
Ian Norden f282c17770 Merge pull request #4 from vulcanize/dev
Concurrent traversal utilities
2020-09-09 12:05:17 -05:00
roysc c6a1534e7f move to root path 2020-09-08 18:56:00 -05:00
roysc fb1768b7a9 go fmt, cleanup 2020-09-06 12:57:14 -05:00
roysc 3715d87161 Fix termination behavior
Boundary must be (a, b] to conform to NodeIterator behavior (stop _at_
next valid key rather than before)
2020-09-06 12:54:43 -05:00
roysc 321f5fe368 don't prepend (nil, 0], make first bin start at nil 2020-09-06 12:54:43 -05:00
roysc deb8b6b71c refactor into slice generator, and factory 2020-09-06 02:32:34 -05:00
roysc 70b374787a iterator slice instead of callback 2020-09-06 02:32:34 -05:00
roysc 8b8f718505 fill nodeiterator interface 2020-09-06 02:32:34 -05:00
roysc a3595be72b doc 2020-08-31 22:22:40 -05:00
roysc d82b49b8ba improve prefix gen
use stateful iterator, and allow splitting into powers of 2 rather than 16
2020-08-31 11:09:21 -05:00
roysc c3c3195991 fmt 2020-08-30 21:59:15 -05:00
roysc f8962c6c58 cleanup 2020-08-23 22:11:58 -05:00
roysc 6d20ef7c3b iterator package
* stripped down Nodeiterator interface

* prefix-bound iterator for traversing a specific range of keys

* visitor over subtries divided at a specific depth
2020-08-23 21:28:42 -05:00
Ian Norden 10f5d0ed93 init 2020-08-14 09:49:35 -05:00