cerc-io/ipld-eth-state-snapshot
16
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity
109 Commits 10 Branches 27 Tags 7.2 MiB
ac00ee280f
Commit Graph

109 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
prathamesh0
ac00ee280f
Upgrade geth (#51) 
* Fix Makefile for installing mockgen tool

* Upgrade geth

* Add sleep in the failing recovery test

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
 2022-07-13 11:39:36 +05:30
prathamesh0
b241bf05eb
Add command to take an in-place snapshot (#42) 
* Add command to take an in-place snapshot

* Add test data for in place snapshot unit test

* Implement unit test for inplace snapshot

* Add check for storage IPLD

* Run unit tests sequentially

* Add github workflow for unit test

* Add missing checks for state and storage cid fields

* Add more storage nodes to test

* Update ipld-eth-db version for tests

* Add comments for inplace snapshot test data

* Add in-place snapshot cmd in readme

* Implement defer pattern for db transaction

* Log transaction commit or rollback error

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
 2022-06-15 12:51:26 +05:30
Ashwin Phatak
c2ecea6d61
Merge pull request #45 from vulcanize/ng-merge-fix-iterator 
Merge changes to fix iterator and update
 2022-06-10 14:24:33 +05:30
nabarun
baef795834 Merge changes to fix iterator and update 2022-06-10 14:12:34 +05:30
nikugogoi
4979e4090d
Merge pull request #43 from roysc/fix-iterator-and-update 
Fix iterator and update
 2022-06-10 14:03:43 +05:30
nabarun
59011cba3c Update go-eth-state-node-iterator version 2022-06-10 14:02:13 +05:30
Roy Crihfield
8371344756 clean up logging 2022-06-08 20:12:22 +08:00
Roy Crihfield
ef391d8d31 simplify tracker, prevent send to closed chan 2022-06-08 20:12:22 +08:00
Roy Crihfield
838dabcba6 update iterator mod 2022-06-08 20:12:22 +08:00
Roy Crihfield
8572c36ee7 decrement path from restored iterator 
to cover node gaps
 2022-06-08 20:12:22 +08:00
Ashwin Phatak
74f82a641d Fix dependency 2022-06-08 20:05:38 +08:00
Roy Crihfield
3e24972a89 try to fix hanging iterator 2022-06-08 20:05:38 +08:00
Ashwin Phatak
7129044eae Log progress info 2022-06-08 18:32:10 +08:00
Abdul Rabbani
c9d398b5af
Merge pull request #40 from vulcanize/feature/update-go-geth-sharding 
Update go and geth versions
 2022-06-01 09:22:04 -04:00
Abdul Rabbani
8aab6a5137 Update geth versions 2022-06-01 09:20:45 -04:00
Abdul Rabbani
cd2559e434 Merge branch 'master' into feature/update-go-geth-sharding 2022-06-01 09:19:18 -04:00
Abdul Rabbani
d213644ff4
Merge pull request #39 from vulcanize/feature/update-go-geth 
Updating Go and Geth Version
 2022-06-01 09:10:35 -04:00
Abdul Rabbani
a04777cb78 Updating Go and Geth Version 
Ran Tests:

```
09:07:50:~/GitHub/cerc/ipld-eth-state-snapshot go test ./... -v
# github.com/vulcanize/ipld-eth-state-snapshot/cmd
cmd/stateSnapshot.go:47:3: (*github.com/sirupsen/logrus.Entry).Fatal call has possible formatting directive %v
?       github.com/vulcanize/ipld-eth-state-snapshot    [no test files]
=== RUN   TestCreateSnapshot
=== RUN   TestCreateSnapshot/case
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN   TestCreateSnapshot/case#01
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN   TestCreateSnapshot/case#02
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN   TestCreateSnapshot/case#03
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
--- PASS: TestCreateSnapshot (0.20s)
    --- PASS: TestCreateSnapshot/case (0.15s)
    --- PASS: TestCreateSnapshot/case#01 (0.01s)
    --- PASS: TestCreateSnapshot/case#02 (0.02s)
    --- PASS: TestCreateSnapshot/case#03 (0.03s)
=== RUN   TestRecovery
=== RUN   TestRecovery/case
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
time="2022-06-01T09:08:03-04:00" level=info msg="Dumping recovery state to: /var/folders/xf/cg688d9j7f39ktmd_4ttwxqh0000gn/T/TestRecoverycase3574218035/001/recover.csv"
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN   TestRecovery/case#01
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
time="2022-06-01T09:08:03-04:00" level=info msg="Dumping recovery state to: /var/folders/xf/cg688d9j7f39ktmd_4ttwxqh0000gn/T/TestRecoverycase#011814952517/001/recover.csv"
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
=== RUN   TestRecovery/case#02
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
time="2022-06-01T09:08:03-04:00" level=info msg="Dumping recovery state to: /var/folders/xf/cg688d9j7f39ktmd_4ttwxqh0000gn/T/TestRecoverycase#021650290358/001/recover.csv"
time="2022-06-01T09:08:03-04:00" level=info msg="Creating snapshot at height 1"
time="2022-06-01T09:08:03-04:00" level=info msg="head hash: 0xa7684ac44d48494670b2e0d9085b7750e7341620f0a271db146ed5e70c1db854 head height: 1"
--- PASS: TestRecovery (0.40s)
    --- PASS: TestRecovery/case (0.12s)
    --- PASS: TestRecovery/case#01 (0.19s)
    --- PASS: TestRecovery/case#02 (0.09s)
PASS
ok      github.com/vulcanize/ipld-eth-state-snapshot/pkg/snapshot       1.109s
=== RUN   TestWriting
--- PASS: TestWriting (0.00s)
=== RUN   TestPgCopy
    publisher_test.go:84: set TEST_WITH_DB to enable test
--- SKIP: TestPgCopy (0.00s)
PASS
ok      github.com/vulcanize/ipld-eth-state-snapshot/pkg/snapshot/file  0.279s
=== RUN   TestBasic
    publisher_test.go:47: set TEST_WITH_DB to enable test
--- SKIP: TestBasic (0.00s)
PASS
ok      github.com/vulcanize/ipld-eth-state-snapshot/pkg/snapshot/pg    0.647s
```
 2022-06-01 09:09:19 -04:00
Ashwin Phatak
c785c12274
Merge pull request #35 from vulcanize/ng-file-ethereum-nodeId 
Add ethereum config to fix db import in file mode
 2022-05-25 18:38:09 +05:30
nabarun
23c4d22885 Add ethereum config to fix db import in file mode 2022-05-25 18:15:08 +05:30
Ashwin Phatak
d3a149899b
Merge pull request #34 from deep-stack/pm-merge-metrics 
Merge changes to add env bindings and prometheus metrics
 2022-05-23 18:14:06 +05:30
prathamesh0
dc27487b66 Merge changes to add env bindings and prometheus metrics 2022-05-23 17:11:51 +05:30
prathamesh0
cce45bda57
Add prometheus metrics collection (#33) 
* Upgrade geth

* Add prometheus metrics collection

* Update README
 2022-05-23 16:56:48 +05:30
Ian Norden
751e064f8f
Merge pull request #28 from vulcanize/release-v3.0.2 
add env bindings
 2022-05-17 22:36:08 -05:00
i-norden
004f3e6665 constants for CLI binding strings 2022-05-17 22:33:20 -05:00
prathamesh0
0a1ea3ae4a
Fix insert statements and upgrade dependencies (#31) 2022-05-17 19:19:52 +05:30
Ashwin Phatak
b3af4e9e75
Merge pull request #30 from deep-stack/pm-upgrade-geth 
Upgrade vulcanize-geth to v4
 2022-05-16 14:05:28 +05:30
prathamesh0
67ec456e88 Upgrade vulcanize-geth to v4 2022-05-16 13:48:26 +05:30
Ashwin Phatak
04b57ca186
Merge pull request #29 from deep-stack/pm-db-sharding 
Updates to use v4 schema
 2022-05-13 16:06:09 +05:30
prathamesh0
fc9e82f717 Index block number using string in file mode 2022-05-13 14:00:40 +05:30
prathamesh0
44d48b8644 Change ON CONFLICT clauses to do nothing and upgrade geth 2022-05-13 09:28:36 +05:30
prathamesh0
26ae3ab008 Updates to use v4 schema 2022-04-19 18:14:25 +05:30
i-norden
31105a009c add env bindings 2022-04-04 14:35:18 -05:00
Ian Norden
560199f930
Merge pull request #25 from vulcanize/release-v3.0.1 
fix minor bug
 2022-03-31 10:34:36 -05:00
i-norden
060eb5cf67 add some logs and guards, update module name, update readme 2022-03-30 18:57:30 -05:00
i-norden
de39842049 fix minor bug 2022-03-30 14:09:32 -05:00
Ian Norden
9e768dc2ba
Merge pull request #22 from vulcanize/recovery 
Service state recovery
 2022-03-17 07:09:25 -05:00
Roy Crihfield
8d1e79d22e handle sigterm 2022-03-16 18:48:51 +08:00
Roy Crihfield
b732ffc5f2 fix race condition 2022-03-14 23:34:26 +08:00
Roy Crihfield
2481d0a28d recovery - handle signals 2022-03-14 23:34:26 +08:00
Roy Crihfield
02488e2b79 logging tweaks 2022-03-14 23:32:00 +08:00
Roy Crihfield
7a2ef4f542 fix fixtures 2022-03-14 23:32:00 +08:00
Roy Crihfield
56acb77829 update iterator mod 2022-03-14 23:31:48 +08:00
Roy Crihfield
7a31f6f007 add chaindata/.gitignore 2022-03-03 17:12:29 +08:00
Roy Crihfield
b5d0010581 implement job recovery; fix traversal & iterator 2022-03-03 17:12:29 +08:00
Ian Norden
a99d0a5ad6
Merge pull request #21 from vulcanize/file-output 
Create CSV file publisher with updated schema
 2022-03-01 06:35:10 -06:00
Roy Crihfield
e0df3399e1 clanup 2022-02-17 14:35:20 +08:00
Roy Crihfield
1a65d9e022 string, not ptr, as map index 2022-02-17 14:34:54 +08:00
Roy Crihfield
ef4ae3a0d4 revisions 2022-02-16 19:27:02 +08:00
Roy Crihfield
3fda647ee9 clean up 2022-02-16 11:13:51 +08:00