Commit Graph

41 Commits

Author SHA1 Message Date
5077abc90f
Compute gas price for EIP-1559 transaction (#157) 2022-08-09 10:50:25 +05:30
1a903fccc6
CLI to compare eden-watcher entities and fix mapping code (#149)
* Make sumStaked variable local in eden network mapping

* Implement compare CLI to fetch and query by ids

* Set filterLogs to true for eden-watcher

* Use varchar for bigint array type in eden-watcher

* Store subgraph entities by id in IPLD state

* Store bigint vales as string in IPLD state

* Update eden watcher hook to store single Block entity in IPLD checkpoint

* Fix entity enum type property

* Fix parsing big numbers in event params

* Fix event bigint params parsing in all watchers

* Set default limit to query result and process block after events
2022-08-08 15:58:13 +05:30
6990eb892a
Handle array and struct type event params in codegen (#150)
* Handle array and struct type event params

* Refactor parse event in watcher indexer code
2022-08-04 17:09:30 +05:30
a15305450c
Add prometheus metrics support for watchers (#152)
* Add prometheus metrics endpoint in watcher

* Add event, sync status and DB size metrics

* Fix subgraph watchers DB entities directory path

* Make watcher metrics optional using config
2022-08-03 16:26:51 +05:30
1bcabd64f2
Update codegen with changes implemented in mobymask watcher (#148)
* Update codegen with index-block CLI and remove graph-node

* Add filter logs by contract flag

* Skip generating GQL API for immutable variables

* Add config for maxEventsBlockRange

* Add new flags in existing watchers
2022-07-22 13:17:56 +05:30
58bbf4d756
Handle getStorage calls for old blocks (#145)
* Handle getStorage calls for old blocks

* Refactor getStorage code in mobymask watcher

* Fix returning proof as null
2022-07-20 10:37:17 +05:30
e2668690b5
Add filterLogs flag to fetch logs by contract (#144) 2022-07-18 14:04:59 +05:30
3cee10607e
Parse batched transactions for mobymask-watcher events (#141)
* Remove using graph-node from mobymask-watcher

* Parse batched transactions for MobyMask events

* Update to use same ethers package version

* Add CLI for processing block out of order

* Fix job-runner for already processed blocks out of order
2022-07-11 11:29:33 +05:30
prathamesh0
71d34331e9
Deprecate postgraphile usage (#122)
* Deprecate postgraphile usage

* Change endpoint in watcher config

* Remove subscription queries from eth-client

* Remove postgraphile config and client from watchers

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-06-08 12:13:52 +05:30
e4b9596be7
Update ipfs daemon version in readme (#121) 2022-05-26 20:01:31 +05:30
168689a7c7
Fix codegen for non subgraph watchers (#120)
* Fix codegen for non subgraph watchers

* Remove graphWatcher.init from generated non subgraph watcher
2022-05-26 18:00:17 +05:30
ce6568aad2 Add demo for graph-watcher IPLD statediff and checkpointing 2022-05-06 15:43:45 +05:30
aabf9f8e15
Implement data source templates (#106)
* Implement data source templates

* Remove optional param from wasm instantiate

* Add all types for solidity to typescript mapping

* Set contract address in context for data source templates

* Implement block handlers for data source templates
2022-01-06 18:32:08 +05:30
prathamesh0
3638d56787 Add checks for and create hooks, checkpoint and IPFS jobs in job-runner (#99)
* Add fields for checkpoint and IPFS in IpldStatus table

* Mark block processing job complete in watchers
2021-12-28 16:08:05 +05:30
dee517e444 Implement IpldStatus table for IPLD related jobs (#98) 2021-12-28 16:08:05 +05:30
198e49e5a0 Implement WASM instance restart to handle out of memory error (#81)
* Test case for wasm out of memory error

* Restart wasm instance after N blocks

* Handle out of memory error and re instantiate WASM

* Remove old instance from map before reinstantiating WASM
2021-12-28 16:08:05 +05:30
68bc1c00db Refactor watchers for util package changes 2021-12-28 16:08:05 +05:30
prathamesh0
560df57ac7 Performance improvements for IPLDBlocks in eden watcher (#80)
* Avoid fetching contracts while creating a checkpoint

* Use enum for state kind and cache ipld status

* Avoid fetching block twice while finalizing a staged diff IPLDBlock

* Create checkpoints at fixed block numbers and refactor checkpointing code

* Avoid calling block handler until start block is reached

* Use delete while finalizing staged diff IPLDBlocks

* Add a check to ensure hooks job is created only once

* Avoid check for initial state while creating a checkpoint
2021-12-28 16:08:05 +05:30
prathamesh0
952f68ee58 Add time logs for eden-watcher (#78) 2021-12-28 16:08:05 +05:30
f4d16afa6c Refactor watchers according to changes in util package 2021-12-28 16:08:05 +05:30
768a4d0818 Use Promise all while fetching events and watch contracts only once 2021-12-28 16:08:05 +05:30
ba890e7d9a Add method in watchers to cache contract and fixes for running tests 2021-12-28 16:08:05 +05:30
prathamesh0
cb2fe2aa45 Store initial state and refactor IPLDBlocks related code (#76)
* Process hooks for the first block in the watchers

* Store initial state from a hook in an IPLDBlock for eden-watcher

* Extract watcher method to prepare IPLDBlocks in util

* Extract common IPLDBlocks related code in util

* Move IPFSClient to util

* Use constants for state kind
2021-12-28 16:08:05 +05:30
prathamesh0
3e0c84b333 Use entity column type map to create entity in store get API (#72)
* Use typeof to distinguish between BigInt and BigDecimal in store get API

* Use entity column type map to create entity in store get API

* Add entity column type map in eden-watcher
2021-12-28 16:08:05 +05:30
prathamesh0
5f03ad5029 Use pg-type numeric for bigint in entities and use custom decimal in graph-node (#71)
* Use pg-type numeric for bigint columns in typeorm entities

* Use custom decimal implementation from util in graph-node
2021-12-28 16:08:05 +05:30
c75be57146 Get block size using JSON RPC API from ipld-eth-server (#68) 2021-12-28 16:08:05 +05:30
30f3c9e694 Fix block processing job created twice when processing missing parent block (#67)
* Fix block processing job created twice in watcher

* Fix block processing job for missing parent blocks
2021-12-28 16:08:05 +05:30
prathamesh0
94e9182dd3 Handle BigNumber event params and customize Decimal (#63)
* Handle BigNumber event params in watchers

* Customize decimal according to limits of IEEE-754 decimal128

* Add definition for custom scalar BigDecimal
2021-12-28 16:08:05 +05:30
238ad21189 Handle subgraph schema field with derivedFrom directive (#60)
* Handle subgraph schema field with derivedFrom directive

* Handle derivedFrom directive in eden-watcher

* Fix 1 to N relation error by removing limit from query

* Order by id for derivedFrom relations to match graph-node

* Refactor example subgraph schema entities

* Fix watcher queries to return correct relation field values

* Fix hierarchical query for getting two entities at same block
2021-12-28 16:08:05 +05:30
prathamesh0
129ba694fb Create external checkpoints after hook processing and clear subgraph entities on reset (#61)
* Create checkpoint using CLI after the hooks have been processed

* Push checkpoints created while exporting state to IPFS

* Clear subgraph entities on reset state

* Remove unnecessary staged IPLD blocks created while importing state
2021-12-28 16:08:05 +05:30
prathamesh0
d979e51723 Add a CLI to compare entity query results from two GQL endpoints (#57)
* Add CLI to compare entities from two GQL endpoints

* Print out result diffs in compare-entity CLI

* Get the colorized result diff in compare-entity CLI

* Read query dir from config file or as an arg

* Make config file arg required

* Make queries in Example schema similar to that in graph-node

* Get non-colorized output on choosing raw-json diff option

* Make queries in eden-watcher similar to that in graph-node
2021-12-28 16:08:05 +05:30
f52467f724 Implement gql queries for relation entities similar to subgraph (#53)
* Add implementation for one to one relation

* Implement one to many relation in gql queries

* Make changes for gql relation queries in eden-watcher

* Implement subgraph gql relation queries with joins
2021-12-28 16:08:05 +05:30
prathamesh0
44b3fd59e8 Add watched contract from subgraph yaml on startup (#56) 2021-12-28 16:08:05 +05:30
nikugogoi
d76268e506 Call subgraph block handlers and fix eden-watcher dependency (#52)
* Call subgraph block handler in watchers

* Fix eden-watcher dependency for build

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-12-28 16:08:05 +05:30
nikugogoi
800ad79baf Handle subgraph schema entity array type and relation fields (#49)
* Handle relation entities in subgraph

* Modify eden-watcher entities to handle subgraph schema data types

* Modify schema gql to match eden subgraph gql shape

* Handle array type fields in subgraph schema

* Fix store get api for array type fields in subgraph entities

* Handle array type in eden-watcher and format await used in params
2021-12-28 16:08:05 +05:30
nikugogoi
47cef8d5d7 Add support for subgraph schema data types (#46)
* Fix packages build

* Add support for boolean type in subgraph schema

* Add BigDecimal fromString and toString implementation

* Add implementation for subgraph Bytes type

* Implement enum type for subgraph schema

* Add implementation for subgraph schema BigDecimal type

* JSON stringify and parse entity data for auto-diff
2021-12-28 16:08:05 +05:30
prathamesh0
73dcb882a6 Update graph-test-watcher (#45) 2021-12-28 16:08:05 +05:30
prathamesh0
16bb955213 Invoke subgraph block handlers (#43)
* Add a test case to eden test to call the block handler

* Add a block handler in example subgraph and call it in a watcher

* Use an array map to call all the block handlers for a contract

* Await on all the promises returned by block handlers map
2021-12-28 16:08:05 +05:30
83775608ec Implement subgraph store host API (#35)
* Implement store get api without blockHash and blockNumber

* Pass database instance to GraphWatcher

* Implement store set without block data

* Store blockHash and blockNumber in database entity table

* Implement getting entity in subgraph from store.get

* Add block data present in postgraphile

* Pass db and context to instantiate method in tests

* GQL API in graph-test-watcher to test store.set

* Remove contract address from subgraph file

* Fix block in dummy event data

* Pass just blockHash to get an entity from the database

* Review changes and add TODOs

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-12-28 16:08:05 +05:30
43d64f9e4b Invoke subgraph handler in watcher event processing (#34)
* Invoke subgraph handler in watcher event processing

* Fix error when invoking subgraph handler

* Parse events using event signature specified in subgraph yaml

* Use contract abi to parse event params

* Invoke event handler based on event signature

* Fill event with block and transaction data

* Comment missing fields in block and transaction data
2021-12-28 16:08:05 +05:30
6cca55a1ab Integrate generated watcher to invoke handlers in graph-node (#33)
* Invoke handlers based on watcher-ts events

* Read subgraph yaml and listen to events from watcher-ts

* Create GraphWatcher class to use in generated example contract watcher

* Call graph-node event handler from generated watcher
2021-12-28 16:08:05 +05:30