Commit Graph

27 Commits

Author SHA1 Message Date
8a720ef175
Wait for events processing to complete before continuing historical blocks processing (#450)
* Wait for events queue to be empty before continuing historical processing

* Make historicalLogsBlockRange and historicalMaxFetchAhead configurable

* Perform single RPC request for multiple addresses
2023-11-06 11:34:48 +05:30
c06330dd06
Implement rpc-eth-client for Ethereum compatible JSON-RPC endpoint (#398)
* Implement rpc-eth-client with getStorageAt method

* Add test for comparing RPC and GQL eth-client getStorageAt method

* Add getBlockWithTransactions and getBlocks method

* Implement getFullBlocks with RLP encoded data

* Implement getFullTransaction method with raw tx

* Implement getBlockByHash and getLogs methods

* Add flag and interface to switch between RPC and GQL eth clients

* Fix getBlocks to return empty array when block not present

* Return empty array in getBlocks for missing block and use blockNumber in getLogs

* Fix getRawTransaction method for zero signature.v value

* Remove duplicate util from rpc-eth-client
2023-08-08 18:48:55 +05:30
76522afa8d
Implement CLI to create checkpoint state from existing GQL endpoint (#382)
* Add util method for preparing state from GQL response entity

* Refactor method to create or update state data

* Fix typeorm bigint transformer and convert to checksum contract address

* Skip resetting to previous block in job-runner if isComplete set to true

* Fix creating subgraph event with struct params

* Use quotes for table names in custom queries

* Fix indexer prepareStateEntry method

* Fix toEthereumValue method when used with ethereum.decode

* Add CLI for creating snapshot checkpoint state from GQL endpoint

* Skip import-state if block is already indexed

* Review changes
2023-05-15 12:10:27 +05:30
49d67899be
Support basic array return types of contract methods in codegen (#363)
* Fix ValueTransformers to check for undefined value

* Create Result types if needed

* Update codegen to include array return type queries in schema

* Update codegen to include array return type queries in resolvers

* Update codegen to include array return type queries in indexers

* Codegen skip db caching methods for queries returning unhandled types

* Codegen handle bigint array in indexer

* Update codegen to handle basic array types

* Remove redundant code and refactor

* Codegen map Solidity types directly to GraphQL types

* Codegen get TS from GQL types

* Codegen handle ethers BigNumber array in indexer template

* Fix GQL type returned in client queries

* Codegen unused method isElementaryType should not be exported

* Codegen update README with solc version in config

* Codegen update README with continue-on-error flag

* Codegen add continue-on-error flag for storage mode

* Codegen generate gitignore and license for watchers

* Codegen refactor gitignore and license generation utils

---------

Co-authored-by: Dhruv Srivastava <dhruvdhs.ds@gmail.com>
2023-04-20 13:31:41 +05:30
b66dcb4af9
Refactor graph-node database and move to util (#259)
* Move graph-database from graph-node to util

* Refactor and remove graph-node dependency from cli package

* Modify dependencies using depcheck

* Implement CLI refactoring changes in other watchers

* Review changes to remove eden comment and fix local import in util

* Import GraphDatabase from util instead of graph-node

* Move graph-node non assemblyscript code to util package

* Implement CLI refactoring changes in codegen

* Fix graph-node tests after refactoring

* Move fromStateEntityValues to graph state utils
2022-11-25 15:54:35 +05:30
prathamesh0
f53371e17b
Accomodate GQL requests caching in code generator (#237)
* Accomodate GQL requests caching in code generator

* Add GQL API request queuing
2022-11-17 12:02:08 +05:30
prathamesh0
a52bdf64b1
Add config for GQL requests cache (#228)
* Add config and a method to set cache control settings for GQL requests

* Add option for max size in GQL cache config

* Fix failing tests
2022-11-15 14:56:08 +05:30
prathamesh0
266442a6d9
Update getFullBlocks usage (#213)
* Update getFullBlocks usage to add block number arg

* Add skipStateFieldsUpdate config for uniswap-watcher

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-11-03 16:49:41 +05:30
prathamesh0
2517f110ea
Update GQL and getBlock API usage (#210)
* Update getLogs and getBlockWithTransactions usage to add block number arg

* Update getFullTransaction usage to add block number arg
2022-11-03 14:14:58 +05:30
prathamesh0
306bbb73ca
Use prefetching of blocks with events in watchers and codegen (#206)
* Avoid refetching block while fetching events

* Prefetch a batch of blocks with events while indexing

* Update mock indexer used in graph-node testing

* Process available blocks while prefetching

* Refactor events fetching to a method in util

* Move method to get GQL event query result to util
2022-10-20 18:46:56 +05:30
prathamesh0
5af90bd388
Refactor state creation code (#204)
* Remove support for pushing state to IPFS

* Move job handlers for state creation to util

* Rename state creation related methods and objects

* Update mock indexer used in graph-node testing

* Fetch and merge diffs in batches while creating a state checkpoint

* Fix timing logs while for state checkpoint creation

* Refactor method to get state query result to util

* Accept contracts for state verification in compare CLI config

* Make method to update state status map synchronous
2022-10-19 15:24:14 +05:30
f3230e3822
Change package name prefixes to cerc-io (#177) 2022-09-09 17:13:01 +05:30
73ca225779
Asynchronous prefetching of block size with eth_getBlockByNumber (#173)
* Prefetch block size from eth_blockByHash

* Fix updating of blockSizeMapLatestHeight

* Move block size caching to separate file

* Remove timer logs from graph-node store set and get
2022-09-07 18:29:04 +05:30
8af7417df6
Implement query for multiple entities and nested relation fields in eden-watcher (#166)
* Implement query for multiple entities in eden-watcher

* Implement nested relation queries

* Implement GQL query params first, skip, orderBy, orderDirection

* Add blockNumber index to subgraph entities

* Add logs for timing eth-calls and storage calls

* Add prometheus metrics to monitor GQL queries

* Fix default limit and order of 1-N related field in GQL entitiy query

* Add timer logs for block processing

* Run transpiled js in all watchers

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2022-09-01 14:17:43 +05:30
97e88ab5f0
Fixes and improvements for eden-watcher job-runner and compare CLI (#165)
* Compare IPLD state entity without derived fields

* Apply default limit to array relation fields in IPLD state entity

* Mark block as complete after processing of block handler

* Avoid re processing of block handler

* Use LIMIT 1 in the query to get latest IPLD block

* Replace eth_calls in eden-watcher with getStorageValue

* Add checkpoint verification to export state CLI

* Fix get diff blocks query when creating checkpoint

* Fix subgraph staker sort and remove entities sequentially in reset CLI

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2022-08-26 12:02:39 +05:30
5077abc90f
Compute gas price for EIP-1559 transaction (#157) 2022-08-09 10:50:25 +05:30
58bbf4d756
Handle getStorage calls for old blocks (#145)
* Handle getStorage calls for old blocks

* Refactor getStorage code in mobymask watcher

* Fix returning proof as null
2022-07-20 10:37:17 +05:30
3cee10607e
Parse batched transactions for mobymask-watcher events (#141)
* Remove using graph-node from mobymask-watcher

* Parse batched transactions for MobyMask events

* Update to use same ethers package version

* Add CLI for processing block out of order

* Fix job-runner for already processed blocks out of order
2022-07-11 11:29:33 +05:30
prathamesh0
71d34331e9
Deprecate postgraphile usage (#122)
* Deprecate postgraphile usage

* Change endpoint in watcher config

* Remove subscription queries from eth-client

* Remove postgraphile config and client from watchers

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-06-08 12:13:52 +05:30
eca09ad66b
Change postgraphile queries to work with v3 schema (#112) 2022-04-28 17:13:32 +05:30
9b1aa29afd
Get missing fields in transaction data for subgraph event handlers (#102)
* Get missing fields in event transaction

* Cache transaction data for event handlers in subgraph
2021-12-29 13:21:39 +05:30
c75be57146 Get block size using JSON RPC API from ipld-eth-server (#68) 2021-12-28 16:08:05 +05:30
86dcee12f7 Get author field for block data 2021-12-28 16:08:05 +05:30
prathamesh0
ec3a8a31a7 Get missing block fields (#51)
* Decode header and get missing block fields

* Move method to get block data to misc in util

* Remove unnecessary encoding of header data
2021-12-28 16:08:05 +05:30
nikugogoi
800ad79baf Handle subgraph schema entity array type and relation fields (#49)
* Handle relation entities in subgraph

* Modify eden-watcher entities to handle subgraph schema data types

* Modify schema gql to match eden subgraph gql shape

* Handle array type fields in subgraph schema

* Fix store get api for array type fields in subgraph entities

* Handle array type in eden-watcher and format await used in params
2021-12-28 16:08:05 +05:30
prathamesh0
08c712d766
Handle bigint and decimal types for entities in watchers (#295)
* Add a GraphDecimal class to replace Decimal with

* Replace Decimal in uni-info-watcher with GraphDecimal

* Add definition for custom scalar BigDecimal in uni-info-watcher

* Use pg-type numeric and transformer for bigint columns in typeorm entities
2021-12-02 13:22:29 +05:30
9d95e49ec9
Change block processing to be pull based (#288)
* Implement pull based watcher for uni-watcher

* Fix same block processed multiple times

* Implement wait time for fetching block from config

* Use blockProgress event to fetch and process next block

* Rename utils index to misc
2021-10-26 17:36:21 +05:30