Commit Graph
13683 Commits
Author SHA1 Message Date
i-norden f513ac95d7 match DB model changes in indexers 2023-02-13 12:52:15 -06:00
i-norden 73e148f759 update models 2023-02-10 10:06:53 -06:00
i-norden 57b7df4918 remove unused types 2023-02-01 20:48:09 -06:00
i-norden 9d7bb5797e fix uncle processing for dump indexer 2023-01-24 20:26:14 -06:00
i-norden edb2e77c3a fix uncle processing for sql indexer 2023-01-24 20:24:05 -06:00
i-norden 531e37ccfa fix uncle processing for file indexer 2023-01-24 20:20:38 -06:00
i-norden c8dd85f417 updated db models 2023-01-24 20:20:15 -06:00
Ian Norden e03b722623 Merge pull request #276 from cerc-io/port_iterator
Port iterator pkg here
2023-01-24 15:10:12 -06:00
i-norden 2b033f2f3c remove logrus, use geth logging pkg 2023-01-24 14:48:26 -06:00
i-norden eb482ee783 Merge branch 'v1.10.26-statediff-v4' into port_iterator 2023-01-24 14:13:32 -06:00
roysc ba297e4a35 Change behavior of WriteStateDiffAt to async (#306)
- returns job ID and provides status update stream
- won't start redundant jobs
2023-01-18 09:40:17 -06:00
telackey fe49b8a23c Add driver detail to flag. (#310) 2023-01-13 16:32:45 -06:00
telackey 907bf3d2f1 Move message from Debug to Trace. (#308) 2023-01-13 15:37:04 -06:00
telackey 339d7bfe7e Add --statediff.db.logstatements option. (#307)
* Add --statediff.db.logstatements option.
2023-01-13 14:59:18 -06:00
telackey 998e4f5917 Additional statediff metrics and logging. (#304) 2023-01-12 11:19:38 -06:00
Ian Norden 60a54c4639 Merge pull request #305 from cerc-io/v1.10.26-statediff-4.3.3-iannorden-0.0.1
logging tweaks
2023-01-11 23:53:50 -06:00
i-norden 52c66b31e5 logging tweaks 2023-01-11 23:26:16 -06:00
Ian Norden 6810cf526d Merge pull request #303 from cerc-io/v1.10.26-statediff-4.3.3-iannorden-0.0.1
remove known_gaps integration
2023-01-11 22:25:56 -06:00
i-norden 5fb5b3f8e4 remove known_gaps integration 2023-01-11 18:53:55 -06:00
Ian Norden fe8814e246 Merge pull request #301 from cerc-io/release-v1.10.26-statediff-4.3.3-alpha
bump statediff patch version
2023-01-10 20:54:08 -06:00
i-norden c3e6c00b7d bump statediff patch version 2023-01-10 20:50:46 -06:00
Ian Norden 175482d34c Merge pull request #300 from cerc-io/ian_dev
No-UPSERT mode
2023-01-10 18:23:31 -06:00
i-norden 47297d6724 integrate upsert mode toggle into CLI 2023-01-10 15:16:12 -06:00
i-norden 248b558e96 indexer/database/postgres.DB updates to toggle no-UPSERT mode 2023-01-10 14:53:35 -06:00
Ian Norden 1b379116b5 Merge pull request #297 from cerc-io/v1.10.26-statediff-4.3.2-alpha
bump minor statediff version
2023-01-06 12:13:37 -06:00
i-norden d23731db9b bump minor statediff version 2023-01-06 11:48:09 -06:00
Ian Norden 01cbc5182d Merge pull request #295 from cerc-io/statediff-lazy-tx
Implement delayed Tx
2023-01-06 11:32:18 -06:00
roysc 3373b0c763 statediff: Use delayed Tx
A lazy Tx object that caches statements, then builds/commits the whole underlying Tx at Commit()
2023-01-06 01:33:20 -06:00
Michael Shaw 52f8c92b17 bad tag variable in publish workflow 2022-11-09 11:03:40 -05:00
fda1723d02 V1.10.26 rebase wip (#289)
* eth/protocols/snap: fix problems due to idle-but-busy peers (#25651)

* eth/protocols/snap: throttle trie heal requests when peers DoS us (#25666)

* eth/protocols/snap: throttle trie heal requests when peers DoS us

* eth/protocols/snap: lower heal throttle log to debug

Co-authored-by: Martin Holst Swende <martin@swende.se>

* eth/protocols/snap: fix comment

Co-authored-by: Martin Holst Swende <martin@swende.se>

* trie: check childrens' existence concurrently for snap heal (#25694)

* eth: fix a rare datarace on CHT challenge reply / shutdown (#25831)

* eth/filters: change filter block to be by-ref (#26054)

This PR changes the block field in the filter to be a pointer, to disambiguate between empty hash and no hash

* rpc: handle wrong HTTP batch response length (#26064)

* params: release geth v1.10.26 stable

* V1.10.25 statediff v4 wip (#275)

* Statediff Geth

Handle conflicts (#244)

* Handle conflicts

* Update go mod file versions

* Make lint changes

Disassociate block number from the indexer object

Update ipld-eth-db ref

Refactor builder code to make it reusable

Use prefix comparison for account selective statediffing

Update builder unit tests

Add mode to write to CSV files in statediff file writer (#249)

* Change file writing mode to csv files

* Implement writer interface for file indexer

* Implement option for csv or sql in file mode

* Close files in CSV writer

* Add tests for CSV file mode

* Implement CSV file for watched addresses

* Separate test configs for CSV and SQL

* Refactor common code for file indexer tests

Update indexer to include block hash in receipts and logs (#256)

* Update indexer to include block hash in receipts and logs

* Upgrade ipld-eth-db image in docker-compose to run tests

Use watched addresses from direct indexing params by default while serving statediff APIs (#262)

* Use watched addresses from direct indexing params in statediff APIs by default

* Avoid using indexer object when direct indexing is off

* Add nil check before accessing watched addresses from direct indexing params

Rebase missed these changes needed at 1.10.20

Flags cleanup for CLI changes and linter complaints

Linter appeasements to achieve perfection

enforce go 1.18 for check (#267)

* enforce go 1.18 for check

* tests on 1.18 as well

* adding db yml for possible change in docker-compose behavior in yml parsing

Add indexer tests for handling non canonical blocks (#254)

* Add indexer tests for header and transactions in a non canonical block

* Add indexer tests for receipts in a non-canonical block and refactor

* Add indexer tests for logs in a non-canonical block

* Add indexer tests for state and storage nodes in a non-canonical block

* Add indexer tests for non-canonical block at another height

* Avoid passing address of a pointer

* Update refs in GitHub workflow

* Add genesis file path to stack-orchestrator config in GitHub workflow

* Add descriptive comments

fix non-deterministic ordering in unit tests

Refactor indexer tests to avoid duplicate code (#270)

* Refactor indexer tests to avoid duplicate code

* Refactor file mode indexer tests

* Fix expected db stats for sqlx after tx closure

* Refactor indexer tests for legacy block

* Refactor mainnet indexer tests

* Refactor tests for watched addressess methods

* Fix query in legacy indexer test

rebase and resolve onto 1.10.23... still error out of index related to GetLeafKeys

changed trie.Commit behavior was subtle about not not flushing to disk without an Update

* no merge nodeset throws nil

* linter appeasement

Cerc refactor (#281)

* first pass cerc refactor in cicd

* 1st attempt to publish binary to git.vdb.to from github release

* docker build step mangled

* docker build step mangled

* wrong username for docker login... which still succeeded

* circcicd is not cerccicd

* bad hostname

adding manual override of binary publish to git.vdb.to for development/emergency (#282)

Cerc io publish fix (#284)

* adding manual override of binary publish to git.vdb.to for development/emergency

* Create manual_binary_publish.yaml (#283)

* github did not pick up workflow added outside of its UI and I still cannot spell cerc right

rawdb helper functions for cold levelDB sync export

Jenkins reborn (#285)

* initial build and output testing...  lots of trial and error

* clean up for working (but failing) unit test geth with ubuntu foundation image

* linter problem on comments in version

* trying linter appeasement with gofmt output on versions.go

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: Jordan Krage <jmank88@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-11-04 13:46:38 -04:00
Michael 70d6dbb012 Jenkins reborn (#285)
* initial build and output testing...  lots of trial and error

* clean up for working (but failing) unit test geth with ubuntu foundation image
2022-10-17 18:08:11 -04:00
Ian Norden c9beb11f8f Merge pull request #279 from cerc-io/ian_updates
rawdb helper functions
2022-09-29 08:21:36 -05:00
i-norden 9f669f8ae6 rawdb helper functions for cold levelDB sync export 2022-09-29 08:20:42 -05:00
Michael 7fbb589fb4 Cerc io publish fix (#284)
* adding manual override of binary publish to git.vdb.to for development/emergency

* Create manual_binary_publish.yaml (#283)

* github did not pick up workflow added outside of its UI and I still cannot spell cerc right
2022-09-26 22:56:37 -04:00
Michael f43e17f051 adding manual override of binary publish to git.vdb.to for development/emergency (#282) 2022-09-26 11:06:07 -04:00
MichaelandAbdul Rabbani 2ddad81c1a V1.10.25 statediff v4 wip (#275)
* Statediff Geth

Handle conflicts (#244)

* Handle conflicts

* Update go mod file versions

* Make lint changes

Disassociate block number from the indexer object

Update ipld-eth-db ref

Refactor builder code to make it reusable

Use prefix comparison for account selective statediffing

Update builder unit tests

Add mode to write to CSV files in statediff file writer (#249)

* Change file writing mode to csv files

* Implement writer interface for file indexer

* Implement option for csv or sql in file mode

* Close files in CSV writer

* Add tests for CSV file mode

* Implement CSV file for watched addresses

* Separate test configs for CSV and SQL

* Refactor common code for file indexer tests

Update indexer to include block hash in receipts and logs (#256)

* Update indexer to include block hash in receipts and logs

* Upgrade ipld-eth-db image in docker-compose to run tests

Use watched addresses from direct indexing params by default while serving statediff APIs (#262)

* Use watched addresses from direct indexing params in statediff APIs by default

* Avoid using indexer object when direct indexing is off

* Add nil check before accessing watched addresses from direct indexing params

Rebase missed these changes needed at 1.10.20

Flags cleanup for CLI changes and linter complaints

Linter appeasements to achieve perfection

enforce go 1.18 for check (#267)

* enforce go 1.18 for check

* tests on 1.18 as well

* adding db yml for possible change in docker-compose behavior in yml parsing

Add indexer tests for handling non canonical blocks (#254)

* Add indexer tests for header and transactions in a non canonical block

* Add indexer tests for receipts in a non-canonical block and refactor

* Add indexer tests for logs in a non-canonical block

* Add indexer tests for state and storage nodes in a non-canonical block

* Add indexer tests for non-canonical block at another height

* Avoid passing address of a pointer

* Update refs in GitHub workflow

* Add genesis file path to stack-orchestrator config in GitHub workflow

* Add descriptive comments

fix non-deterministic ordering in unit tests

Refactor indexer tests to avoid duplicate code (#270)

* Refactor indexer tests to avoid duplicate code

* Refactor file mode indexer tests

* Fix expected db stats for sqlx after tx closure

* Refactor indexer tests for legacy block

* Refactor mainnet indexer tests

* Refactor tests for watched addressess methods

* Fix query in legacy indexer test

rebase and resolve onto 1.10.23... still error out of index related to GetLeafKeys

changed trie.Commit behavior was subtle about not not flushing to disk without an Update

* no merge nodeset throws nil

* linter appeasement

Co-authored-by: Abdul Rabbani <abdulrabbani00@gmail.com>

Cerc refactor (#281)

* first pass cerc refactor in cicd

* 1st attempt to publish binary to git.vdb.to from github release

* docker build step mangled

* docker build step mangled

* wrong username for docker login... which still succeeded

* circcicd is not cerccicd

* bad hostname
2022-09-23 11:57:11 -04:00
Péter Szilágyi 69568c5548 params: release Geth v1.10.25 v1.10.25 2022-09-15 17:55:58 +02:00
Marius van der WijdenandMartin Holst Swende 8f61fc8b73 params: set TerminalTotalDifficultyPassed to true (#25769)
* params: set TerminalTotalDifficultyPassed to true

* Update params/config.go

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-15 17:54:48 +02:00
Péter Szilágyi 972007a517 Release Geth v1.10.24 v1.10.24 2022-09-14 11:07:10 +02:00
Sina Mahmoodi 3b41be695e graphql: fixes missing tx logs (#25745)
* graphql: fix tx logs

* graphql: refactor test service setup

* graphql: add test for tx logs
2022-09-13 23:03:49 +02:00
Sina Mahmoodi d0dc349fd3 graphql: return correct logs for tx (#25612)
* graphql: fix tx logs

* minor

* Use optimized search for selecting tx logs
2022-09-13 23:03:21 +02:00
i-norden 1385209eda update go mod 2022-09-13 07:31:06 -05:00
i-norden 37f60b6300 move pkg 2022-09-09 15:14:54 -05:00
i-norden c5076e7d37 Merge branch 'master' of /Users/iannorden/go/src/github.com/cerc-io/go-eth-state-node-iterator into port_iterator 2022-09-09 15:12:34 -05:00
i-norden ba680f44ce repkg 2022-09-08 22:47:31 -05:00
roysc 3d57aa34fa Merge pull request #18 from roysc/add-tracker
Add tracker package
2022-09-06 07:40:12 -05:00
roysc d72e3214c1 Add tracker package
Moved from snapshot repo
2022-09-06 14:38:09 +02:00
Michael 2508990948 Merge pull request #19 from vulcanize/rebase-1.10.23-wip
dependency update to 1.10.23 of vulcanize statediffing geth
2022-09-01 16:51:53 -04:00
Michael Shaw 71af7dfd2a dependency update to 1.10.23 of vulcanize statediffing geth 2022-09-01 16:19:23 -04:00
MichaelandAbdul Rabbani 918a30dcc9 V1.10.23 statediff v4 wip (#275)
* Statediff Geth

Handle conflicts (#244)

* Handle conflicts

* Update go mod file versions

* Make lint changes

Disassociate block number from the indexer object

Update ipld-eth-db ref

Refactor builder code to make it reusable

Use prefix comparison for account selective statediffing

Update builder unit tests

Add mode to write to CSV files in statediff file writer (#249)

* Change file writing mode to csv files

* Implement writer interface for file indexer

* Implement option for csv or sql in file mode

* Close files in CSV writer

* Add tests for CSV file mode

* Implement CSV file for watched addresses

* Separate test configs for CSV and SQL

* Refactor common code for file indexer tests

Update indexer to include block hash in receipts and logs (#256)

* Update indexer to include block hash in receipts and logs

* Upgrade ipld-eth-db image in docker-compose to run tests

Use watched addresses from direct indexing params by default while serving statediff APIs (#262)

* Use watched addresses from direct indexing params in statediff APIs by default

* Avoid using indexer object when direct indexing is off

* Add nil check before accessing watched addresses from direct indexing params

Rebase missed these changes needed at 1.10.20

Flags cleanup for CLI changes and linter complaints

Linter appeasements to achieve perfection

enforce go 1.18 for check (#267)

* enforce go 1.18 for check

* tests on 1.18 as well

* adding db yml for possible change in docker-compose behavior in yml parsing

Add indexer tests for handling non canonical blocks (#254)

* Add indexer tests for header and transactions in a non canonical block

* Add indexer tests for receipts in a non-canonical block and refactor

* Add indexer tests for logs in a non-canonical block

* Add indexer tests for state and storage nodes in a non-canonical block

* Add indexer tests for non-canonical block at another height

* Avoid passing address of a pointer

* Update refs in GitHub workflow

* Add genesis file path to stack-orchestrator config in GitHub workflow

* Add descriptive comments

fix non-deterministic ordering in unit tests

Refactor indexer tests to avoid duplicate code (#270)

* Refactor indexer tests to avoid duplicate code

* Refactor file mode indexer tests

* Fix expected db stats for sqlx after tx closure

* Refactor indexer tests for legacy block

* Refactor mainnet indexer tests

* Refactor tests for watched addressess methods

* Fix query in legacy indexer test

rebase and resolve onto 1.10.23... still error out of index related to GetLeafKeys

changed trie.Commit behavior was subtle about not not flushing to disk without an Update

* no merge nodeset throws nil

* linter appeasement

Co-authored-by: Abdul Rabbani <abdulrabbani00@gmail.com>
v1.10.23-statediff-v4-unstable
2022-09-01 14:34:58 -04:00