Commit Graph

83 Commits

Author SHA1 Message Date
39f8b6ec79
Fix getLogs query (avoid repeated execution, separate tx, and improve performance) (#250)
* Avoid repeated queries in getLogs.

* Rewrite query for speed and simplicity.

* Make default limit 500
2023-06-16 11:49:56 -05:00
i-norden
175c840bac fix err overshadowing in defers 2023-05-31 17:57:42 -05:00
6d7487152c Upgrade to v5 schema
Now uses:
* ipld direct_by_leaf StateDB for basic queries
* trie_by_cid StateDB for trie slice and proof queries

Also:
* vulcanize => cerc refactor
* Backend method to close dbs
* state tests are in multiple packages, to allow separate ginkgo suites
* removes gap-filler module
* integration tests and github workflows
* run stack-orchestrator for testnet
* fix various issues with tests, hardhat server, dockerfile
* fix cmd flags / env vars
* fix flaky tests and clean up code
* remove unused code, scripts
* remove outdated docs
* update version
2023-05-25 21:39:48 +08:00
i-norden
b410c0876a interface fixes for v1.11.5 2023-04-14 14:49:39 +08:00
i-norden
20b787e5dc missed moving some sql into consolidated file 2023-03-14 10:04:41 -05:00
i-norden
4678b6a733 ipld-eth-statedb initialization/instantiation 2023-03-13 18:23:49 -05:00
i-norden
df8bec1ad0 adapt uncle retrieval methods to work for v5 2023-03-13 12:48:25 -05:00
i-norden
aab5a2ef0b use ipld-eth-statedb for eth_call and graphql endpoints 2023-03-13 11:25:19 -05:00
i-norden
ba5cdecf93 new backend methods using ipld_eth_statedb.StateDB 2023-03-13 07:51:03 -05:00
i-norden
4c29841c21 internalize some node type utils from geth as they are no longer used over there in v5 2023-03-07 21:49:38 -06:00
i-norden
6f1bfc7fce combine ipld and cid retriever into one struct. the original reason for the separation of concerns was because we used to fetch cids directly from the database but then use those to fetch iplds using an ipfs.BlockService abstraction ontop of our database (and/or a ipfs.BlockExchange). But now they are both accessed directly in the local DB. Additionally, some of the queries should be further refined/refactored/combined as we no longer need to take two trips but can retrieve cid and ipld in the same query. 2023-03-07 21:49:38 -06:00
i-norden
907f15f640 mh_key => cid; leaf_cid => cid; refactor GetReceipts 2023-03-02 17:59:13 -06:00
i-norden
87013110f9 public.blocks => ipld.blocks 2023-03-02 17:41:44 -06:00
i-norden
8ec203ec3d refactor out remaining unused bits 2023-02-21 19:50:58 -06:00
190d0d7ac9
Improved logging and metrics. (#227)
1. Improve logging to include API method, user ID, etc in the output. We do this by intercepting the request details in the "middleware" and adding them to the request context, as well as adding a wrapper to logrus that simplifies including the fields in the output.

2. Breakdown API metrics by method. This will allow us to differentiate call counts and durations by API method (eg, eth_call vs eth_getStorageAt).
2023-01-20 19:39:26 -06:00
prathamesh0
721a728d4b
Implement getSlice API (#206)
* Implement getSlice API for state nodes

* Implement getSlice API for storage nodes

* Fix the helper function to create a slice of required paths

* Fix query to get state leaf key for given storage root

* Add a test to get state slice for root path

* Add checks in queries to get canonical data

* Add tests to get state slice

* Add a todo for using an iterator

* Avoid filtering out removed nodes

* Add tests to get storage slice

* Remove logs

* Populate extra contract leaves field in the response

* Update tests

* Avoid EOAs in additional data in response

* Use iterator based approach for getSlice

* Skip undesired nodes from stem and head iterators

* Update storage slice tests

* Fix meta data updates

* Use state trie to get stem nodes directly using paths

* Bugfix - Continue processing other trie nodes on encountering a leaf

* Remove unnecessary TODO
2022-12-19 14:12:23 +05:30
prathamesh0
cc17368c0d
Optimize GQL and getBlock APIs (#201)
* Add optional block number filter to getLogs API

* Use block number while fetching block with transactions

* Fix for failing tests

* Use block number when fetching block objects

* Use block number when fetching a tx by hash

* Add back methods to get block objects by block hash

* Update run_unit_test.sh

Removing echo command, as it seems to be only change in CICD.

* Increase wait time for db setup in unit test job

Co-authored-by: Michael <michael@abastionofsanity.com>
2022-11-04 10:02:09 +05:30
prathamesh0
2c03c8cbd0
Support debug_traceCall API (#192)
* Implement debug_traceCall API

* Use API implementation from geth with custom backend

* Update refs in GitHub workflow
2022-09-21 17:15:03 +05:30
Michael
fc0d7a6dd6
Cerc refactor (#193)
* cerc refactor waiting on unpublished dependencies

* cerc refactor updates for dependencies

* Describe imports got removed

* cleaning up more vulcanize refs in ci/cd

* another test lost Describe imports

* another test lost Describe imports... not caught in go build -a???

* more cerc-io migrations to utilize new git.vdb.to gitea instance

* switching back to github for running unit test

* first try at git.vdb.to as conatiner repository

* targeted tag was incomplete
2022-09-20 11:52:06 -04:00
Michael Shaw
0e91e42128 first pass updating for statediffing geth 1.10.23... filters changed 2022-09-01 18:37:51 -04:00
prathamesh0
4e9f4bbad6
Re-order uncles if calculated uncle hash doesn't match (#188)
* Re-order uncles if calculated uncle hash doesn't match

* Add a spec to test out of order uncles
2022-08-25 15:54:32 +05:30
prathamesh0
e40e8a7335
Optimize eth_getBlockByNumber and eth_getBlockByHash implementations (#187)
* Avoid two step queries for creating a block object

* Use a db tx while fetching block objects

* Upgrade ipfs-ethdb
2022-08-23 12:42:19 +05:30
prathamesh0
6cb54ca790
Update transactions, receipts and logs queries (#173)
* Update queries to fetch data from cids tables

* Fix eth_getTransactionByHash to return tx from canonical block

* Fetch transaction by hash in a single query

* Update queries to fetch data from IPLD blocks table

* Avoid NULL result from canonical block hash query

* Update GitHub workflow and script to run unit tests

* Avoid usage of LIMIT 1 in queries for GQL API

* Fetch IPLD data separately when retrieving latest account data or storage value

* Upgrade geth dependency

* Update GitHub workflow
2022-07-12 13:40:45 +05:30
07a991d3e4 Add misssing error logs 2022-06-28 16:40:07 +05:30
Abdul Rabbani
4121863a4c Update Geth and CICD 2022-06-17 10:45:54 -04:00
ace12fbc35 Update module path for v4 2022-05-20 18:50:50 +05:30
36e590c092 Delete from all tables when tearing down db for tests 2022-05-18 20:23:40 +05:30
384d939dc2 Upgrade dependencies 2022-05-18 14:53:09 +05:30
24a45a3f00 Update unit tests and queries to fetch data from cids tables 2022-05-18 14:38:09 +05:30
705835512e Update module path for v3 2022-05-18 13:35:16 +05:30
233fa29740 Upgrade eth-ipfs-state-validator to v3 2022-05-04 13:06:07 +05:30
4883590d85 Upgrade ipfs-ethdb to v3 2022-05-04 13:06:07 +05:30
3aa5cb36ef Fixes for unit tests 2022-04-19 14:01:24 +05:30
8df8b50cb1 Update to use new schema 2022-04-19 14:01:24 +05:30
072ba1edcc Use sqlx for db connection 2022-04-19 14:01:24 +05:30
i-norden
907c7132f4 needed type assertion 2021-12-29 14:57:21 -06:00
Arijit Das
1973e8032d Bump up daptools image tag and geth version. 2021-10-07 15:07:23 +05:30
Arijit Das
cc6822f7a6 Update eth_call code. 2021-09-29 10:57:11 +05:30
Arijit Das
5772d52eb1 Change receipt CID and MHKey to point trie leaf cid and mkhey. 2021-09-21 17:51:38 +05:30
Ashwin Phatak
2de9c5bd48
Use groupcache pool for state db access (#91)
* Use groupcache pool for state db access

* Group cache config and logging stats on timer

* Integrate state validator into server

* Use tagged ipfs-ethdb

* groupcache config for tests

* Work around duplicate registration of groupcache error in tests

* Use tagged version of eth-ipfs-state-validator

* State validation command.

* Validator for static replicas to keep cache warm

* Update docker go-version and go.mod.

* Address comments and self review.

* Fix ipfs-ethdb version.

Co-authored-by: Arijit Das <arijitad.in@gmail.com>
2021-09-21 17:40:55 +05:30
Arijit Das
ed4171a1ab Revert "Add status field for pre-byzantium blocks."
This reverts commit 04a0f9a751.
2021-09-15 17:14:25 +05:30
Arijit Das
04a0f9a751 Add status field for pre-byzantium blocks. 2021-09-14 17:25:48 +05:30
Arijit Das
71837c4b24 Self review. 2021-08-31 18:10:41 +05:30
Arijit Das
a28892f1d3 Fix get log API to use log_cids table. 2021-08-30 21:25:39 +05:30
Arijit Das
d8a5358a70 Remove ipld-eth-indexer dependency. 2021-08-12 11:56:29 +05:30
Arijit Das
e00e602098
Merge pull request #83 from vulcanize/marshall-binary
Update RPC api and backend to support the EIP-2930 and EIP-1559.
2021-08-05 11:04:57 +05:30
Arijit Das
924d0be0b9 Update RPC api and backend to support the EIP-2930 and EIP-1559. 2021-07-27 17:37:50 +05:30
Ashwin Phatak
70f7face75
getLogs API changes to return txHash, make contract arg optional. (#81)
* getLogs API changes to return txHash, make contract arg optional.

* Populate log index.

* Add test for txn hash in GetLogs request.

* Convert tx string to common.Hash after fetching.

Co-authored-by: Arijit Das <arijitad.in@gmail.com>
2021-07-26 15:43:38 +05:30
Arijit Das
9a5581b543 Fix failing unit tests. 2021-06-29 12:59:37 +05:30
Ashwin Phatak
a284a566d5 Get storage API, with storage leaf CID and raw IPLD block. 2021-06-28 12:51:35 +05:30