Commit Graph

20 Commits

Author SHA1 Message Date
80dcf3019d refactor fixtures 2023-09-27 02:01:18 +08:00
12b1b2df3d refactor tests 2023-09-02 19:36:53 +08:00
25962de254 refactor to use statediff plugin 2023-09-02 19:36:29 +08:00
i-norden
891fca89bd fix overshadowing of snap.Tx 2023-05-17 08:16:50 -05:00
i-norden
ea4c1042c4 update service_test with new fixtures 2023-05-12 10:07:28 -05:00
Michael
989bd1c0f1
Cerc refactor (#64)
* cerc refactor waiting on unpublished dependencies

* more cerc_refactor with published ipld-eth-db unstable

* TearDownDB refactor

* missed second publisher_test TearDownDB
2022-09-20 13:47:34 -04:00
Ian Norden
05aeeab581
Account selective snapshot (#46)
* snapshotter ignores nodes not along a path along those derived from a list of account addresses if one is provided

* config and env updates

* cmd update

* Encode watched address path bytes to hex for comparison

* actually ignore the subtries that are not along the paths of interest

* Fixes for account selective snapshot

* Use non-concurrent iterator when having a single worker

* Only index root node when starting path of an iterator is nil

* Upgrade deps

* Avoid tracking iterators and skip recovery test

* Fix recovery mechanism, use sync Map instead of buffered channels

* Add test for account selective snapshot

* Continue traversal with concurrent iterators with starting path nil

* Use errgroup to simplify error handling with concurrent iterators

* Check if all the nodes are indexed in the recovery test

* Use concurrency safe sync Map in account selective snapshot test

* Only track concurrent iterators and refactor code

* Fix node and recovered path comparison

* Revert back to using buffered channels for tracking iterators

* Add a metric to monitor number of active iterators

* Update docs

* Update seeked path after node is processed

* Return error on context cancellation from subtrie iteration

* Add tests for account selective snapshot recovery

* Explicity enforce concurrent iterator bounds to avoid duplicate nodes

* Update full snapshot test to check nodes being indexed

* Refactor code to simplify snapshot logic

* Remove unnecessary function argument

* Use ctx cancellation for handling signals

* Add descriptive comments

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2022-08-03 17:05:04 +05:30
prathamesh0
ac00ee280f
Upgrade geth (#51)
* Fix Makefile for installing mockgen tool

* Upgrade geth

* Add sleep in the failing recovery test

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-07-13 11:39:36 +05:30
baef795834 Merge changes to fix iterator and update 2022-06-10 14:12:34 +05:30
3e24972a89 try to fix hanging iterator 2022-06-08 20:05:38 +08:00
fc9e82f717 Index block number using string in file mode 2022-05-13 14:00:40 +05:30
26ae3ab008 Updates to use v4 schema 2022-04-19 18:14:25 +05:30
i-norden
060eb5cf67 add some logs and guards, update module name, update readme 2022-03-30 18:57:30 -05:00
b5d0010581 implement job recovery; fix traversal & iterator 2022-03-03 17:12:29 +08:00
c4b4f232e9 create csv file publisher; update geth and schema 2022-02-14 07:22:31 +08:00
4f53299e9a clean up [need storage node] 2022-01-16 17:55:35 -06:00
159f93902e tests - mock publisher with fixtures
* go structs for header, etc.
* refactor: pkg/types

* add fixture chaindata
2022-01-14 21:43:00 -06:00
2bb06e0a8d split ethdb out of service 2022-01-10 23:37:27 -06:00
1fab8ac1c8 rf config, mock publisher 2022-01-10 19:08:45 -06:00
802a5908df set up test 2022-01-10 18:59:15 -06:00