Commit Graph

60 Commits

Author SHA1 Message Date
ce182bce85
Clear cache of latest entities on event processing error (#201)
* Clear cache of latest entities on event processing error

* Remove lighthouse-watcher and update ethersjs version

* Handle GraphDecimal type in state entity

* Add option for comparing all entities using paginate

* Clear pruned cached entities at intervals

* Move ipld-demo to graph-node package and remove reset-dbs script

* Implement changes in all watchers and codegen
2022-10-19 14:26:10 +05:30
a8fdcca866
Update watcher import CLI to create entities from checkpoint (#186)
* Import entities from checkpoint

* Fix IPLD state when updating subgraph entity

* Changes in codegen and other watchers

* Update IPLD state with all Block entities

* Add verify and create sub commands to checkpoint

* Add option for specifying snapshot block in export state CLI
2022-09-22 15:26:06 +05:30
8af7417df6
Implement query for multiple entities and nested relation fields in eden-watcher (#166)
* Implement query for multiple entities in eden-watcher

* Implement nested relation queries

* Implement GQL query params first, skip, orderBy, orderDirection

* Add blockNumber index to subgraph entities

* Add logs for timing eth-calls and storage calls

* Add prometheus metrics to monitor GQL queries

* Fix default limit and order of 1-N related field in GQL entitiy query

* Add timer logs for block processing

* Run transpiled js in all watchers

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2022-09-01 14:17:43 +05:30
7238f614c0
Update CLI to compare only updated entities and verify IPLD state (#161)
* Change compare CLI to verify only updated entities

* Implement IPLD state verification in compare CLI

* Changes to IPLD state to match with GQL result entity
2022-08-17 19:11:40 +05:30
prathamesh0
80682e2755
Remove Uniswap watchers and related artifacts (#163)
* Remove Uniswap watchers and related code

* Remove Uniswap artifacts from graph-node

* Remove Uniswap artifacts from graph-test-watcher
2022-08-17 14:04:20 +05:30
1a903fccc6
CLI to compare eden-watcher entities and fix mapping code (#149)
* Make sumStaked variable local in eden network mapping

* Implement compare CLI to fetch and query by ids

* Set filterLogs to true for eden-watcher

* Use varchar for bigint array type in eden-watcher

* Store subgraph entities by id in IPLD state

* Store bigint vales as string in IPLD state

* Update eden watcher hook to store single Block entity in IPLD checkpoint

* Fix entity enum type property

* Fix parsing big numbers in event params

* Fix event bigint params parsing in all watchers

* Set default limit to query result and process block after events
2022-08-08 15:58:13 +05:30
a15305450c
Add prometheus metrics support for watchers (#152)
* Add prometheus metrics endpoint in watcher

* Add event, sync status and DB size metrics

* Fix subgraph watchers DB entities directory path

* Make watcher metrics optional using config
2022-08-03 16:26:51 +05:30
3cee10607e
Parse batched transactions for mobymask-watcher events (#141)
* Remove using graph-node from mobymask-watcher

* Parse batched transactions for MobyMask events

* Update to use same ethers package version

* Add CLI for processing block out of order

* Fix job-runner for already processed blocks out of order
2022-07-11 11:29:33 +05:30
ba73cd24c2 Generate MobyMask watcher using codegen 2022-06-29 14:53:57 +05:30
ce6568aad2 Add demo for graph-watcher IPLD statediff and checkpointing 2022-05-06 15:43:45 +05:30
prathamesh0
5b12db541b Support artifacts generation for multiple contracts (#82) 2021-12-28 16:08:05 +05:30
prathamesh0
d979e51723 Add a CLI to compare entity query results from two GQL endpoints (#57)
* Add CLI to compare entities from two GQL endpoints

* Print out result diffs in compare-entity CLI

* Get the colorized result diff in compare-entity CLI

* Read query dir from config file or as an arg

* Make config file arg required

* Make queries in Example schema similar to that in graph-node

* Get non-colorized output on choosing raw-json diff option

* Make queries in eden-watcher similar to that in graph-node
2021-12-28 16:08:05 +05:30
prathamesh0
44b3fd59e8 Add watched contract from subgraph yaml on startup (#56) 2021-12-28 16:08:05 +05:30
prathamesh0
73dcb882a6 Update graph-test-watcher (#45) 2021-12-28 16:08:05 +05:30
prathamesh0
06ba24e38f Generate GQL API for subgraph entities and auto-diff based on store set (#38)
* Add subgraph schema types to the generated schema

* Add queries for subgraph entities

* Add entity generation for subgraph entities

* Call subgraph event handler in indexer

* Refactor subgraph schema and entity generation

* Add resolvers generation for subgraph entities

* Get event signature in the event

* Add NonNullType check for field type in entity generation

* Auto-diff based on store set

* Use contract address from data source in loader

* Change subgraph-schema arg to subgraph-path arg
2021-12-28 16:08:05 +05:30
83775608ec Implement subgraph store host API (#35)
* Implement store get api without blockHash and blockNumber

* Pass database instance to GraphWatcher

* Implement store set without block data

* Store blockHash and blockNumber in database entity table

* Implement getting entity in subgraph from store.get

* Add block data present in postgraphile

* Pass db and context to instantiate method in tests

* GQL API in graph-test-watcher to test store.set

* Remove contract address from subgraph file

* Fix block in dummy event data

* Pass just blockHash to get an entity from the database

* Review changes and add TODOs

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-12-28 16:08:05 +05:30
6cca55a1ab Integrate generated watcher to invoke handlers in graph-node (#33)
* Invoke handlers based on watcher-ts events

* Read subgraph yaml and listen to events from watcher-ts

* Create GraphWatcher class to use in generated example contract watcher

* Call graph-node event handler from generated watcher
2021-12-28 16:08:05 +05:30
1c15c1eedb Test case for EdenNetwork event handler (#31)
* Test case for calling EdenNetwork event handler

* Test cases for calling EdenNetwork event handlers

* Test cases for calling event handlers in EdenNetworkDistribution

* Test cases for calling event handlers in EdenNetworkGovernance

* Perform eth-calls according to contract name in param

* Pass function params in eth-call host API

* Return value from eth-call to wasm

* Add instructions for running tests

* Install assemblyscript from github packages

* Add instrcutions for installing private github packages

* Use vulcanize assemblyscript package
2021-12-28 16:08:05 +05:30
f34d83c04b Use async functions in wasm imports (#28)
* Use async functions in wasm imports

* Complete implementing asyncify with assemblyscript loader
2021-12-28 16:08:05 +05:30
Ashwin Phatak
889e96572d Host API stubs (#12) 2021-12-28 16:08:04 +05:30
Ashwin Phatak
1421ba5a9b WASM instance setup (#11)
* Code for running wasm file.

* Add test for wasm exported function.

* Use target names for build files.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-12-28 16:08:04 +05:30
prathamesh0
a2ad139769
Use node v16 and pprof to generate profile data for watchers (#299)
* Use pprof to generate profile data for watchers

* Update docs and github workflows to use node v16
2021-12-09 10:39:35 +05:30
prathamesh0
d3971b5258
Generate GQL client (#259)
* Add GQL client generation

* Add subscription, events, eventsInRange in client gen

* Add mutation in client gen and return event tx info from indexer

* Capitalize class names for storage-mode entities
2021-10-04 11:04:06 +05:30
Ashwin Phatak
338cef9954
Generate entities from YAML templates and lint support in generated watchers (#253)
* Add lint files generation.

* Fix lint errors in generated code.

* Load default entities from yaml files.

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-27 18:03:04 +05:30
Ashwin Phatak
92b7967895
Generating eth_call based lazy watcher (#249)
* Add entity generation.

* Add resolvers generation.

* Add queries in resolvers generation.

* Add indexer generation.

* Extract helper code in utils.

* Add server and artifacts generation.

* Fix solidity-flattener issue.

* Update readme and cleanup misc files.

* Add queries to entity generation.

* Add database generation.

* Use snakecase in database.

* Add readme generation.

* Change template file names.

* Add method descriptions.

* Change mode to eth_call in readme.

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-23 16:55:46 +05:30
Ashwin Phatak
4f8f1d8cd7
Update code for log_cids change in vdb core (#243)
* Remove fields no longer present in schema.

* Refactor code to work with log_cids change in eth-server.

* Refactor process event to save events from logs.

* Use constants for erc20 event names.

* Implement watch blocks in erc20-watcher similar to uni-watcher.

* Moved common methods to util.

* Implement eventsInRange query in erc20-watcher.

* Filter unknown event in database query.

* Change dependencies version to be same in all packages.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-21 16:43:55 +05:30
Ashwin Phatak
0a96bbd94d
Change dependencies version to be same in all packages. (#250)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-21 10:27:34 +05:30
Ashwin Phatak
c973f3b343
Add husky for setting lint in commit hook. (#248)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-20 16:25:54 +05:30
Ashwin Phatak
6560639a68
Use flatten function and modify ResultEvent in schema. (#247)
Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-17 17:10:08 +05:30
Ashwin Phatak
f9934675b2
CLI to flatten and generate GQL schema from Solidity file (#245)
* Add schema generation.

* Use yargs and add a script to generate schema.

* Add command to generate schema to README.md

* Remove use of arrow functions in visitor.

* Add function to export schema in visitor.

* Create Event union while adding to it.

* Add bool to typemappings.

* Add method descriptions and remove output type mappings.

* Add storage mode.

* Add URL support and a flattening script.

* Fix parameter for flattening in README.md.

* Add setup and references to README.md.

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-16 17:06:10 +05:30
Ashwin Phatak
06bed1c13b
Enable source maps for transpiled files (#242)
* Rename .spec files to .test files.

* Enable source maps.

* Pass enable-source-maps flag for transpiled files.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-07 16:01:15 +05:30
Ashwin Phatak
d3a50d2b18
Fix package versions after cleaning dependencies. (#241)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-02 16:13:55 +05:30
Ashwin Phatak
2d493b3838
Run services from transpiled ES5 (#239)
* Transpile ES5 code.

* Use transpiled ES5 code in all packages.

* Refactor tracing-client to transpile ES5 code.

* Order scripts in package json.

* Update readme in packages with new scripts.

* Refactor code to use entity config programatically.

* Remove unused depencdencies.

* Use DEBUG environment for scripts.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-02 10:27:56 +05:30
Ashwin Phatak
cf0713eda7
Tasks in erc20-watcher to deploy and transfer tokens. (#233)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-27 17:47:23 +05:30
Ashwin Phatak
9a3ac28a5d
uni-watcher smoke test (#172)
* Created a smoke test for uni-watcher.

* Added test for createPool.

* Added more pool tests.

* Using ethers instead of hardhat-ethers in smoke test.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-07-30 19:50:56 +05:30
Ashwin Phatak
3a6af9f9cc
Handle mint event (#130)
* Handle mint event and update Factory fields.

* Integrate decimal.js with typeorm.

* Update transaction in Mint event.

* Update day and hour data.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-13 12:01:54 +05:30
Ashwin Phatak
7f5229bf2f
Watch pool burn and swap events (#128)
* Remove test scripts, moved to vulcanize/uniswap-v3-core#watcher-ts.

* Move uniswap analysis docs.

* Watch pool burn and swap events.
2021-07-09 12:30:50 +05:30
Ashwin Phatak
da758aceaa
Script to deploy uniswap contracts locally for testing (#118)
* Initial setup with hardhat.

* Deploy Factory contract.

* Deploy tokens and create pool using factory contract.

* Deploy contract to private network.

* Implement separate scripts for deploying Factory, Token and Pool.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-07-05 11:05:45 +05:30
Ashwin Phatak
b7ffb7c672
uniswap-info-watcher package with frontend schema (#114)
* Implement mock bundle query in watcher.

* Use chance js for generating mock data.

* Add bundles query.

* Setup resolvers with uniswap info schema.

* Add mock query for Burn entity.

* Add mock query for Factory entity.

* Add mock queries for entities Mint, Pool and Transaction.

* Add mock query for Swap entity.

* Add mock queries for day data entities.

* Implment mock ticks query.

* Add mock tokenHourDatas query.

* Add mock queries for uni-info-watcher to readme.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-07-02 16:26:32 +05:30
Ashwin Phatak
5de555a4e5
Uniswap watcher package with event schema (#113)
* Uniswap watcher package skeleton.

* Uniswap events.
2021-07-01 16:42:37 +05:30
Ashwin Phatak
69f4cf3513
Uniswap queries/mapping analysis (#91)
* List down view method in uniswap core contracts.

* List queries used in uniswap subgraph frontend.

* Analyse uniswap subgraph entities with mappings.

* Generate schema graphql file for uniswap subgraph.

* Schema graphql for queries used in subgraph frontend.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-29 10:39:32 +05:30
Ashwin Phatak
07805b6ae9
Tracing job queue (#89)
* eth-client API to get transaction in block.

* Job queue to process tracing requests.

* Request traces from job queue runner.

* Max trace completion lag time config for downstream events.
2021-06-25 16:35:47 +05:30
Ashwin Phatak
2adc5e9c34
Push address events to downstream subscribers (#85)
* Push address event to downstream subscribers.

* Get addresses from trace - tests and fixes.
2021-06-22 14:04:48 +05:30
Ashwin Phatak
c9bf002675
Test decoder using hardhat RPC, geth-statediff RPC and ipld-eth-server GQL endpoints for fetching storage slots (#77)
* Implement test for getStorageValue with geth server.

* Wait for transaction to complete in tests.

* Implement tests with ipld-eth-client using graphql endpoint.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-18 18:09:50 +05:30
Ashwin Phatak
4e08f359fc
Call tracer with address tracing support (#65)
* Tracing client package, basic utils.

* Start of address tracer.

* Rename trace CLI.

* Call tracer API.

* Tracing timeout, more traces.

* Integrate address tracer into call address tracer.
2021-06-16 16:14:35 +05:30
Ashwin Phatak
7035a513b1
Downstream event subscription support. (#47)
* Downstream event subscription support.

* Logging changes.
2021-06-09 15:11:05 +05:30
Ashwin Phatak
fc44617db3
Use solidity mapper to get value for mapping and nested mapping (balance and allowance) (#48)
* Implement getting value for basic mapping type.

* Add test for basic mapping type.

* Implement getting value for nested mapping type.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-09 10:18:19 +05:30
Ashwin Phatak
84e1927402
Watch upstream ERC20 events to trigger indexing (#43)
* Move to apollo client, enables subscriptions.

* Watch logs and trigger other indexer methods.

* Refactoring config loading, watched contracts table.

* Check event sync progress inside transaction.

* Refactoring server startup.
2021-06-08 16:07:52 +05:30
nikugogoi
a13a909a85 Implement getting storage value for uint256 type in watcher. 2021-06-08 12:49:00 +05:30
nikugogoi
8d697ab9d1 Check and remove unused dependecies. 2021-06-07 11:42:03 +05:30