Commit Graph

48 Commits

Author SHA1 Message Date
i-norden
8bd9e967ba fix PGX direct database writing tests 2023-03-21 12:02:30 -05:00
i-norden
297e88890a fix TestSQLXIndexerNonCanonical test 2023-03-21 11:57:29 -05:00
i-norden
02bcb1add6 []byte{} val for removed storage nodes to make pgx vs sqlx results more consistent 2023-03-21 11:57:15 -05:00
i-norden
25839ddd54 fix direct database writing mode tests 2023-03-20 17:28:04 -05:00
i-norden
2c50615b81 insert node_ids pq.StringArray 2023-03-18 13:27:10 -05:00
i-norden
23b0efd3eb remove access list 2023-03-18 13:06:16 -05:00
i-norden
396399bf9b fix for remaining builder bug, related to issue identified by prathamesh 2023-03-16 08:06:30 -05:00
i-norden
64d865740b remove silly debug statements 2023-03-15 18:43:21 -05:00
i-norden
9e5a5a237f update statediff api 2023-03-15 18:09:16 -05:00
i-norden
ca5a4c472f update statediff types 2023-03-15 18:09:06 -05:00
i-norden
1eef72d1e9 update statediffing service 2023-03-15 18:08:55 -05:00
i-norden
8a5befde44 update indexer interfaces and shared constants 2023-03-15 18:08:43 -05:00
i-norden
288408a2b9 update database/sql indexer 2023-03-15 18:08:19 -05:00
i-norden
4d53681a27 update database/file/csv and database/file/sql indexers 2023-03-15 18:08:03 -05:00
i-norden
6e8ee099e3 updating database/dump indexer 2023-03-15 18:07:41 -05:00
i-norden
6d5a0157b7 finish builder tests and some builder fixes 2023-03-15 18:07:15 -05:00
i-norden
86480b834b finish updating builder and builder tests 2023-03-15 12:05:34 -05:00
i-norden
7574909281 remove now unused trie helpers 2023-03-14 10:07:55 -05:00
i-norden
85ee920c8b rewriting expeccted test objects and refactoring builder tests 2023-02-21 22:00:31 -06:00
i-norden
7f1b630b3e refactor statediff/service.go to work with new builder and to remove some unused functionality 2023-02-16 16:54:37 -06:00
i-norden
714e1f58a4 refactor (minimize) internal ipld package 2023-02-16 16:53:44 -06:00
i-norden
d81184b548 remove now unused fields from builder config struct 2023-02-16 15:52:21 -06:00
i-norden
6daab10ca4 update types to remove unused fields 2023-02-16 15:52:07 -06:00
i-norden
50d08bcb36 export Keccak256ToCID funtion 2023-02-16 15:49:56 -06:00
i-norden
ae5ab3d9b0 update statediff builder to process internalized leaf node values (storage trie) 2023-02-16 15:49:18 -06:00
i-norden
1e574d4df8 update types, models, and indexers and writers to match removal of partial_path columns from state_cids and storage_cids 2023-02-15 15:20:50 -06:00
i-norden
583de67b10 remove unused full-trie code 2023-02-14 16:03:10 -06:00
i-norden
85279baa0e update statediff builder to process internalized leaf node values 2023-02-14 16:02:52 -06:00
i-norden
530fed2614 adjust indexers for tx, rct, and log trie changes 2023-02-13 15:04:08 -06:00
i-norden
f05d6b6baa refactor eth parser to not process rct, tx, and log tries 2023-02-13 14:48:02 -06:00
i-norden
b568bb49e9 match DB model changes in writers 2023-02-13 13:20:10 -06:00
i-norden
f513ac95d7 match DB model changes in indexers 2023-02-13 12:52:15 -06:00
i-norden
73e148f759 update models 2023-02-10 10:06:53 -06:00
i-norden
57b7df4918 remove unused types 2023-02-01 20:48:09 -06:00
i-norden
9d7bb5797e fix uncle processing for dump indexer 2023-01-24 20:26:14 -06:00
i-norden
edb2e77c3a fix uncle processing for sql indexer 2023-01-24 20:24:05 -06:00
i-norden
531e37ccfa fix uncle processing for file indexer 2023-01-24 20:20:38 -06:00
i-norden
c8dd85f417 updated db models 2023-01-24 20:20:15 -06:00
ba297e4a35
Change behavior of WriteStateDiffAt to async (#306)
- returns job ID and provides status update stream
- won't start redundant jobs
2023-01-18 09:40:17 -06:00
907bf3d2f1
Move message from Debug to Trace. (#308) 2023-01-13 15:37:04 -06:00
339d7bfe7e
Add --statediff.db.logstatements option. (#307)
* Add --statediff.db.logstatements option.
2023-01-13 14:59:18 -06:00
998e4f5917
Additional statediff metrics and logging. (#304) 2023-01-12 11:19:38 -06:00
i-norden
52c66b31e5 logging tweaks 2023-01-11 23:26:16 -06:00
i-norden
5fb5b3f8e4 remove known_gaps integration 2023-01-11 18:53:55 -06:00
i-norden
47297d6724 integrate upsert mode toggle into CLI 2023-01-10 15:16:12 -06:00
i-norden
248b558e96 indexer/database/postgres.DB updates to toggle no-UPSERT mode 2023-01-10 14:53:35 -06:00
3373b0c763 statediff: Use delayed Tx
A lazy Tx object that caches statements, then builds/commits the whole underlying Tx at Commit()
2023-01-06 01:33:20 -06:00
Michael
2ddad81c1a V1.10.25 statediff v4 wip (#275)
* Statediff Geth

Handle conflicts (#244)

* Handle conflicts

* Update go mod file versions

* Make lint changes

Disassociate block number from the indexer object

Update ipld-eth-db ref

Refactor builder code to make it reusable

Use prefix comparison for account selective statediffing

Update builder unit tests

Add mode to write to CSV files in statediff file writer (#249)

* Change file writing mode to csv files

* Implement writer interface for file indexer

* Implement option for csv or sql in file mode

* Close files in CSV writer

* Add tests for CSV file mode

* Implement CSV file for watched addresses

* Separate test configs for CSV and SQL

* Refactor common code for file indexer tests

Update indexer to include block hash in receipts and logs (#256)

* Update indexer to include block hash in receipts and logs

* Upgrade ipld-eth-db image in docker-compose to run tests

Use watched addresses from direct indexing params by default while serving statediff APIs (#262)

* Use watched addresses from direct indexing params in statediff APIs by default

* Avoid using indexer object when direct indexing is off

* Add nil check before accessing watched addresses from direct indexing params

Rebase missed these changes needed at 1.10.20

Flags cleanup for CLI changes and linter complaints

Linter appeasements to achieve perfection

enforce go 1.18 for check (#267)

* enforce go 1.18 for check

* tests on 1.18 as well

* adding db yml for possible change in docker-compose behavior in yml parsing

Add indexer tests for handling non canonical blocks (#254)

* Add indexer tests for header and transactions in a non canonical block

* Add indexer tests for receipts in a non-canonical block and refactor

* Add indexer tests for logs in a non-canonical block

* Add indexer tests for state and storage nodes in a non-canonical block

* Add indexer tests for non-canonical block at another height

* Avoid passing address of a pointer

* Update refs in GitHub workflow

* Add genesis file path to stack-orchestrator config in GitHub workflow

* Add descriptive comments

fix non-deterministic ordering in unit tests

Refactor indexer tests to avoid duplicate code (#270)

* Refactor indexer tests to avoid duplicate code

* Refactor file mode indexer tests

* Fix expected db stats for sqlx after tx closure

* Refactor indexer tests for legacy block

* Refactor mainnet indexer tests

* Refactor tests for watched addressess methods

* Fix query in legacy indexer test

rebase and resolve onto 1.10.23... still error out of index related to GetLeafKeys

changed trie.Commit behavior was subtle about not not flushing to disk without an Update

* no merge nodeset throws nil

* linter appeasement

Co-authored-by: Abdul Rabbani <abdulrabbani00@gmail.com>

Cerc refactor (#281)

* first pass cerc refactor in cicd

* 1st attempt to publish binary to git.vdb.to from github release

* docker build step mangled

* docker build step mangled

* wrong username for docker login... which still succeeded

* circcicd is not cerccicd

* bad hostname
2022-09-23 11:57:11 -04:00