Michael
989bd1c0f1
Cerc refactor ( #64 )
...
* cerc refactor waiting on unpublished dependencies
* more cerc_refactor with published ipld-eth-db unstable
* TearDownDB refactor
* missed second publisher_test TearDownDB
2022-09-20 13:47:34 -04:00
Michael Shaw
321e37584d
updates for geth rebase on 1.10.23
2022-09-02 15:21:29 -04:00
Ian Norden
05aeeab581
Account selective snapshot ( #46 )
...
* snapshotter ignores nodes not along a path along those derived from a list of account addresses if one is provided
* config and env updates
* cmd update
* Encode watched address path bytes to hex for comparison
* actually ignore the subtries that are not along the paths of interest
* Fixes for account selective snapshot
* Use non-concurrent iterator when having a single worker
* Only index root node when starting path of an iterator is nil
* Upgrade deps
* Avoid tracking iterators and skip recovery test
* Fix recovery mechanism, use sync Map instead of buffered channels
* Add test for account selective snapshot
* Continue traversal with concurrent iterators with starting path nil
* Use errgroup to simplify error handling with concurrent iterators
* Check if all the nodes are indexed in the recovery test
* Use concurrency safe sync Map in account selective snapshot test
* Only track concurrent iterators and refactor code
* Fix node and recovered path comparison
* Revert back to using buffered channels for tracking iterators
* Add a metric to monitor number of active iterators
* Update docs
* Update seeked path after node is processed
* Return error on context cancellation from subtrie iteration
* Add tests for account selective snapshot recovery
* Explicity enforce concurrent iterator bounds to avoid duplicate nodes
* Update full snapshot test to check nodes being indexed
* Refactor code to simplify snapshot logic
* Remove unnecessary function argument
* Use ctx cancellation for handling signals
* Add descriptive comments
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2022-08-03 17:05:04 +05:30
prathamesh0
5002c82038
Upgrade dependencies ( #56 )
2022-08-03 16:11:38 +05:30
Michael Shaw
257fac1a0c
update for go-ethereum 1.10.20
2022-07-19 15:12:44 -04:00
prathamesh0
ac00ee280f
Upgrade geth ( #51 )
...
* Fix Makefile for installing mockgen tool
* Upgrade geth
* Add sleep in the failing recovery test
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-07-13 11:39:36 +05:30
baef795834
Merge changes to fix iterator and update
2022-06-10 14:12:34 +05:30
59011cba3c
Update go-eth-state-node-iterator version
2022-06-10 14:02:13 +05:30
838dabcba6
update iterator mod
2022-06-08 20:12:22 +08:00
Abdul Rabbani
8aab6a5137
Update geth versions
2022-06-01 09:20:45 -04:00
Abdul Rabbani
a04777cb78
Updating Go and Geth Version
...
Ran Tests:
```
09:07:50:~/GitHub/cerc/ipld-eth-state-snapshot go test ./... -v
# github.com/vulcanize/ipld-eth-state-snapshot/cmd
cmd/stateSnapshot.go:47:3: (*github.com/sirupsen/logrus.Entry).Fatal call has possible formatting directive %v
? github.com/vulcanize/ipld-eth-state-snapshot [no test files]
=== RUN TestCreateSnapshot
=== RUN TestCreateSnapshot/case
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN TestCreateSnapshot/case#01
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN TestCreateSnapshot/case#02
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN TestCreateSnapshot/case#03
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
--- PASS: TestCreateSnapshot (0.20s)
--- PASS: TestCreateSnapshot/case (0.15s)
--- PASS: TestCreateSnapshot/case#01 (0.01s)
--- PASS: TestCreateSnapshot/case#02 (0.02s)
--- PASS: TestCreateSnapshot/case#03 (0.03s)
=== RUN TestRecovery
=== RUN TestRecovery/case
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
time="2022-06-01T09:08:03-04:00" level=info msg="Dumping recovery state to: /var/folders/xf/cg688d9j7f39ktmd_4ttwxqh0000gn/T/TestRecoverycase3574218035/001/recover.csv"
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN TestRecovery/case#01
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
time="2022-06-01T09:08:03-04:00" level=info msg="Dumping recovery state to: /var/folders/xf/cg688d9j7f39ktmd_4ttwxqh0000gn/T/TestRecoverycase#011814952517/001/recover.csv"
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN TestRecovery/case#02
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
time="2022-06-01T09:08:03-04:00" level=info msg="Dumping recovery state to: /var/folders/xf/cg688d9j7f39ktmd_4ttwxqh0000gn/T/TestRecoverycase#021650290358/001/recover.csv"
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
--- PASS: TestRecovery (0.40s)
--- PASS: TestRecovery/case (0.12s)
--- PASS: TestRecovery/case#01 (0.19s)
--- PASS: TestRecovery/case#02 (0.09s)
PASS
ok github.com/vulcanize/ipld-eth-state-snapshot/pkg/snapshot 1.109s
=== RUN TestWriting
--- PASS: TestWriting (0.00s)
=== RUN TestPgCopy
publisher_test.go:84: set TEST_WITH_DB to enable test
--- SKIP: TestPgCopy (0.00s)
PASS
ok github.com/vulcanize/ipld-eth-state-snapshot/pkg/snapshot/file 0.279s
=== RUN TestBasic
publisher_test.go:47: set TEST_WITH_DB to enable test
--- SKIP: TestBasic (0.00s)
PASS
ok github.com/vulcanize/ipld-eth-state-snapshot/pkg/snapshot/pg 0.647s
```
2022-06-01 09:09:19 -04:00
prathamesh0
cce45bda57
Add prometheus metrics collection ( #33 )
...
* Upgrade geth
* Add prometheus metrics collection
* Update README
2022-05-23 16:56:48 +05:30
i-norden
060eb5cf67
add some logs and guards, update module name, update readme
2022-03-30 18:57:30 -05:00
56acb77829
update iterator mod
2022-03-14 23:31:48 +08:00
c4b4f232e9
create csv file publisher; update geth and schema
2022-02-14 07:22:31 +08:00
i-norden
1919738fec
update go mod
2022-01-26 12:07:52 -06:00
159f93902e
tests - mock publisher with fixtures
...
* go structs for header, etc.
* refactor: pkg/types
* add fixture chaindata
2022-01-14 21:43:00 -06:00
i-norden
b5d31748e0
bump geth version
2021-12-28 23:30:17 -06:00
Arijit Das
ba8e32ce97
Update geth and implement transaction batching.
2021-12-13 20:31:32 +05:30
Ian Norden
e50dd95c29
go modules
2020-07-01 14:10:44 -05:00
