In passing, I noticed that these options were no longer being bound to their defined ENV variables.
Reviewed-on: #9
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
* Refactors to replace most of the code with the statediff plugin.
* Adds basic CI test workflows for Gitea
* Refactors fixtures to use https://git.vdb.to/cerc-io/eth-testing
* Renames env vars for consistency with flags and other services:
- LOGRUS_{LEVEL,FILE} => LOG_LEVEL, etc.
- LVL_DB_PATH => LEVELDB_PATH
- ANCIENT_DB_PATH => LEVELDB_ANCIENT
- These will need to be updated wherever they are used
Reviewed-on: #1
* Add a progress counter by checking the distance already traversed from the startPath to endPath in a bounded iterator vs the estimated number of iterations.
* snapshotter ignores nodes not along a path along those derived from a list of account addresses if one is provided
* config and env updates
* cmd update
* Encode watched address path bytes to hex for comparison
* actually ignore the subtries that are not along the paths of interest
* Fixes for account selective snapshot
* Use non-concurrent iterator when having a single worker
* Only index root node when starting path of an iterator is nil
* Upgrade deps
* Avoid tracking iterators and skip recovery test
* Fix recovery mechanism, use sync Map instead of buffered channels
* Add test for account selective snapshot
* Continue traversal with concurrent iterators with starting path nil
* Use errgroup to simplify error handling with concurrent iterators
* Check if all the nodes are indexed in the recovery test
* Use concurrency safe sync Map in account selective snapshot test
* Only track concurrent iterators and refactor code
* Fix node and recovered path comparison
* Revert back to using buffered channels for tracking iterators
* Add a metric to monitor number of active iterators
* Update docs
* Update seeked path after node is processed
* Return error on context cancellation from subtrie iteration
* Add tests for account selective snapshot recovery
* Explicity enforce concurrent iterator bounds to avoid duplicate nodes
* Update full snapshot test to check nodes being indexed
* Refactor code to simplify snapshot logic
* Remove unnecessary function argument
* Use ctx cancellation for handling signals
* Add descriptive comments
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
* Add command to take an in-place snapshot
* Add test data for in place snapshot unit test
* Implement unit test for inplace snapshot
* Add check for storage IPLD
* Run unit tests sequentially
* Add github workflow for unit test
* Add missing checks for state and storage cid fields
* Add more storage nodes to test
* Update ipld-eth-db version for tests
* Add comments for inplace snapshot test data
* Add in-place snapshot cmd in readme
* Implement defer pattern for db transaction
* Log transaction commit or rollback error
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
