Commit Graph

54 Commits

Author SHA1 Message Date
a15305450c
Add prometheus metrics support for watchers (#152)
* Add prometheus metrics endpoint in watcher

* Add event, sync status and DB size metrics

* Fix subgraph watchers DB entities directory path

* Make watcher metrics optional using config
2022-08-03 16:26:51 +05:30
3cee10607e
Parse batched transactions for mobymask-watcher events (#141)
* Remove using graph-node from mobymask-watcher

* Parse batched transactions for MobyMask events

* Update to use same ethers package version

* Add CLI for processing block out of order

* Fix job-runner for already processed blocks out of order
2022-07-11 11:29:33 +05:30
ba73cd24c2 Generate MobyMask watcher using codegen 2022-06-29 14:53:57 +05:30
ce6568aad2 Add demo for graph-watcher IPLD statediff and checkpointing 2022-05-06 15:43:45 +05:30
prathamesh0
5b12db541b Support artifacts generation for multiple contracts (#82) 2021-12-28 16:08:05 +05:30
prathamesh0
d979e51723 Add a CLI to compare entity query results from two GQL endpoints (#57)
* Add CLI to compare entities from two GQL endpoints

* Print out result diffs in compare-entity CLI

* Get the colorized result diff in compare-entity CLI

* Read query dir from config file or as an arg

* Make config file arg required

* Make queries in Example schema similar to that in graph-node

* Get non-colorized output on choosing raw-json diff option

* Make queries in eden-watcher similar to that in graph-node
2021-12-28 16:08:05 +05:30
prathamesh0
44b3fd59e8 Add watched contract from subgraph yaml on startup (#56) 2021-12-28 16:08:05 +05:30
prathamesh0
73dcb882a6 Update graph-test-watcher (#45) 2021-12-28 16:08:05 +05:30
prathamesh0
06ba24e38f Generate GQL API for subgraph entities and auto-diff based on store set (#38)
* Add subgraph schema types to the generated schema

* Add queries for subgraph entities

* Add entity generation for subgraph entities

* Call subgraph event handler in indexer

* Refactor subgraph schema and entity generation

* Add resolvers generation for subgraph entities

* Get event signature in the event

* Add NonNullType check for field type in entity generation

* Auto-diff based on store set

* Use contract address from data source in loader

* Change subgraph-schema arg to subgraph-path arg
2021-12-28 16:08:05 +05:30
83775608ec Implement subgraph store host API (#35)
* Implement store get api without blockHash and blockNumber

* Pass database instance to GraphWatcher

* Implement store set without block data

* Store blockHash and blockNumber in database entity table

* Implement getting entity in subgraph from store.get

* Add block data present in postgraphile

* Pass db and context to instantiate method in tests

* GQL API in graph-test-watcher to test store.set

* Remove contract address from subgraph file

* Fix block in dummy event data

* Pass just blockHash to get an entity from the database

* Review changes and add TODOs

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-12-28 16:08:05 +05:30
6cca55a1ab Integrate generated watcher to invoke handlers in graph-node (#33)
* Invoke handlers based on watcher-ts events

* Read subgraph yaml and listen to events from watcher-ts

* Create GraphWatcher class to use in generated example contract watcher

* Call graph-node event handler from generated watcher
2021-12-28 16:08:05 +05:30
1c15c1eedb Test case for EdenNetwork event handler (#31)
* Test case for calling EdenNetwork event handler

* Test cases for calling EdenNetwork event handlers

* Test cases for calling event handlers in EdenNetworkDistribution

* Test cases for calling event handlers in EdenNetworkGovernance

* Perform eth-calls according to contract name in param

* Pass function params in eth-call host API

* Return value from eth-call to wasm

* Add instructions for running tests

* Install assemblyscript from github packages

* Add instrcutions for installing private github packages

* Use vulcanize assemblyscript package
2021-12-28 16:08:05 +05:30
f34d83c04b Use async functions in wasm imports (#28)
* Use async functions in wasm imports

* Complete implementing asyncify with assemblyscript loader
2021-12-28 16:08:05 +05:30
Ashwin Phatak
889e96572d Host API stubs (#12) 2021-12-28 16:08:04 +05:30
Ashwin Phatak
1421ba5a9b WASM instance setup (#11)
* Code for running wasm file.

* Add test for wasm exported function.

* Use target names for build files.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-12-28 16:08:04 +05:30
prathamesh0
a2ad139769
Use node v16 and pprof to generate profile data for watchers (#299)
* Use pprof to generate profile data for watchers

* Update docs and github workflows to use node v16
2021-12-09 10:39:35 +05:30
prathamesh0
d3971b5258
Generate GQL client (#259)
* Add GQL client generation

* Add subscription, events, eventsInRange in client gen

* Add mutation in client gen and return event tx info from indexer

* Capitalize class names for storage-mode entities
2021-10-04 11:04:06 +05:30
Ashwin Phatak
338cef9954
Generate entities from YAML templates and lint support in generated watchers (#253)
* Add lint files generation.

* Fix lint errors in generated code.

* Load default entities from yaml files.

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-27 18:03:04 +05:30
Ashwin Phatak
92b7967895
Generating eth_call based lazy watcher (#249)
* Add entity generation.

* Add resolvers generation.

* Add queries in resolvers generation.

* Add indexer generation.

* Extract helper code in utils.

* Add server and artifacts generation.

* Fix solidity-flattener issue.

* Update readme and cleanup misc files.

* Add queries to entity generation.

* Add database generation.

* Use snakecase in database.

* Add readme generation.

* Change template file names.

* Add method descriptions.

* Change mode to eth_call in readme.

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-23 16:55:46 +05:30
Ashwin Phatak
4f8f1d8cd7
Update code for log_cids change in vdb core (#243)
* Remove fields no longer present in schema.

* Refactor code to work with log_cids change in eth-server.

* Refactor process event to save events from logs.

* Use constants for erc20 event names.

* Implement watch blocks in erc20-watcher similar to uni-watcher.

* Moved common methods to util.

* Implement eventsInRange query in erc20-watcher.

* Filter unknown event in database query.

* Change dependencies version to be same in all packages.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-21 16:43:55 +05:30
Ashwin Phatak
0a96bbd94d
Change dependencies version to be same in all packages. (#250)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-21 10:27:34 +05:30
Ashwin Phatak
c973f3b343
Add husky for setting lint in commit hook. (#248)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-20 16:25:54 +05:30
Ashwin Phatak
6560639a68
Use flatten function and modify ResultEvent in schema. (#247)
Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-17 17:10:08 +05:30
Ashwin Phatak
f9934675b2
CLI to flatten and generate GQL schema from Solidity file (#245)
* Add schema generation.

* Use yargs and add a script to generate schema.

* Add command to generate schema to README.md

* Remove use of arrow functions in visitor.

* Add function to export schema in visitor.

* Create Event union while adding to it.

* Add bool to typemappings.

* Add method descriptions and remove output type mappings.

* Add storage mode.

* Add URL support and a flattening script.

* Fix parameter for flattening in README.md.

* Add setup and references to README.md.

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-16 17:06:10 +05:30
Ashwin Phatak
06bed1c13b
Enable source maps for transpiled files (#242)
* Rename .spec files to .test files.

* Enable source maps.

* Pass enable-source-maps flag for transpiled files.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-07 16:01:15 +05:30
Ashwin Phatak
d3a50d2b18
Fix package versions after cleaning dependencies. (#241)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-02 16:13:55 +05:30
Ashwin Phatak
2d493b3838
Run services from transpiled ES5 (#239)
* Transpile ES5 code.

* Use transpiled ES5 code in all packages.

* Refactor tracing-client to transpile ES5 code.

* Order scripts in package json.

* Update readme in packages with new scripts.

* Refactor code to use entity config programatically.

* Remove unused depencdencies.

* Use DEBUG environment for scripts.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-02 10:27:56 +05:30
Ashwin Phatak
cf0713eda7
Tasks in erc20-watcher to deploy and transfer tokens. (#233)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-27 17:47:23 +05:30
Ashwin Phatak
9a3ac28a5d
uni-watcher smoke test (#172)
* Created a smoke test for uni-watcher.

* Added test for createPool.

* Added more pool tests.

* Using ethers instead of hardhat-ethers in smoke test.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-07-30 19:50:56 +05:30
Ashwin Phatak
3a6af9f9cc
Handle mint event (#130)
* Handle mint event and update Factory fields.

* Integrate decimal.js with typeorm.

* Update transaction in Mint event.

* Update day and hour data.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-13 12:01:54 +05:30
Ashwin Phatak
7f5229bf2f
Watch pool burn and swap events (#128)
* Remove test scripts, moved to vulcanize/uniswap-v3-core#watcher-ts.

* Move uniswap analysis docs.

* Watch pool burn and swap events.
2021-07-09 12:30:50 +05:30
Ashwin Phatak
da758aceaa
Script to deploy uniswap contracts locally for testing (#118)
* Initial setup with hardhat.

* Deploy Factory contract.

* Deploy tokens and create pool using factory contract.

* Deploy contract to private network.

* Implement separate scripts for deploying Factory, Token and Pool.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-07-05 11:05:45 +05:30
Ashwin Phatak
b7ffb7c672
uniswap-info-watcher package with frontend schema (#114)
* Implement mock bundle query in watcher.

* Use chance js for generating mock data.

* Add bundles query.

* Setup resolvers with uniswap info schema.

* Add mock query for Burn entity.

* Add mock query for Factory entity.

* Add mock queries for entities Mint, Pool and Transaction.

* Add mock query for Swap entity.

* Add mock queries for day data entities.

* Implment mock ticks query.

* Add mock tokenHourDatas query.

* Add mock queries for uni-info-watcher to readme.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-07-02 16:26:32 +05:30
Ashwin Phatak
5de555a4e5
Uniswap watcher package with event schema (#113)
* Uniswap watcher package skeleton.

* Uniswap events.
2021-07-01 16:42:37 +05:30
Ashwin Phatak
69f4cf3513
Uniswap queries/mapping analysis (#91)
* List down view method in uniswap core contracts.

* List queries used in uniswap subgraph frontend.

* Analyse uniswap subgraph entities with mappings.

* Generate schema graphql file for uniswap subgraph.

* Schema graphql for queries used in subgraph frontend.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-29 10:39:32 +05:30
Ashwin Phatak
07805b6ae9
Tracing job queue (#89)
* eth-client API to get transaction in block.

* Job queue to process tracing requests.

* Request traces from job queue runner.

* Max trace completion lag time config for downstream events.
2021-06-25 16:35:47 +05:30
Ashwin Phatak
2adc5e9c34
Push address events to downstream subscribers (#85)
* Push address event to downstream subscribers.

* Get addresses from trace - tests and fixes.
2021-06-22 14:04:48 +05:30
Ashwin Phatak
c9bf002675
Test decoder using hardhat RPC, geth-statediff RPC and ipld-eth-server GQL endpoints for fetching storage slots (#77)
* Implement test for getStorageValue with geth server.

* Wait for transaction to complete in tests.

* Implement tests with ipld-eth-client using graphql endpoint.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-18 18:09:50 +05:30
Ashwin Phatak
4e08f359fc
Call tracer with address tracing support (#65)
* Tracing client package, basic utils.

* Start of address tracer.

* Rename trace CLI.

* Call tracer API.

* Tracing timeout, more traces.

* Integrate address tracer into call address tracer.
2021-06-16 16:14:35 +05:30
Ashwin Phatak
7035a513b1
Downstream event subscription support. (#47)
* Downstream event subscription support.

* Logging changes.
2021-06-09 15:11:05 +05:30
Ashwin Phatak
fc44617db3
Use solidity mapper to get value for mapping and nested mapping (balance and allowance) (#48)
* Implement getting value for basic mapping type.

* Add test for basic mapping type.

* Implement getting value for nested mapping type.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-09 10:18:19 +05:30
Ashwin Phatak
84e1927402
Watch upstream ERC20 events to trigger indexing (#43)
* Move to apollo client, enables subscriptions.

* Watch logs and trigger other indexer methods.

* Refactoring config loading, watched contracts table.

* Check event sync progress inside transaction.

* Refactoring server startup.
2021-06-08 16:07:52 +05:30
nikugogoi
a13a909a85 Implement getting storage value for uint256 type in watcher. 2021-06-08 12:49:00 +05:30
nikugogoi
8d697ab9d1 Check and remove unused dependecies. 2021-06-07 11:42:03 +05:30
Ashwin Phatak
3aaa9dd7f8
Use postgres instead of sqlite. (#38) 2021-06-04 15:19:30 +05:30
Ashwin Phatak
4e0ef7c852
Implement typescript strict mode and ESLint in ipld-eth-client and cache packages (#37)
* Set up typescript build.

* Setup eslint in cache package.

* Automatic lint fixes.

* Fix typescript return types.

* Fix typescript argument type warnings.

* Set up typescript build and eslint.

* Automatic lint fixes.

* Fix typescript explicit any warnings.

* Add argument types.

* Fix return type warnings.

* Fix typescript errors.

* Implement declaration in types directory.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-04 12:04:12 +05:30
Ashwin Phatak
13ec526ebd Create index for ERC20 data. 2021-05-31 16:24:45 +05:30
Ashwin Phatak
a0aae09f83
Get slot for ERC20 variable from storage layout (#13)
* Get slot for ERC20 variable from storage layout.

* Fix solidity-mapper build for importing library functions.

* Implement lint command in solidity-mapper package.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-05-31 14:50:05 +05:30
Ashwin Phatak
72ca980198
Solidity data mapper/parser (#12)
* Initial setup with hardhat and typescript.

* Add test for integer type.

* Add test for unsigned integer type.

* Add test for boolean type.

* Add test for address type.

* Add test for string type.

* Setup building library with typescript.

* Remove hardhat dependency from getStorageValue library function.

* Move contracts to test and remove deploy script.

* Add readme for running tests.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-05-31 11:07:11 +05:30
Ashwin Phatak
7213a1dc6d
Lazy ERC20 watcher (#11)
* Storage mapping utils.

* Resolver factories.

* Get ERC20 balance from upstream ipld-eth-server slot.

* Get ERC20 allowance from storage slot.

* Parse ERC20 events from block events.

* Cache GQL requests in leveldb.

* Refactor fetch/cache from upstream into eth-loader class.

* Refactoring.

* Refactor eth-client, extract cache class.

* Cache config settings.

* Debug logs.

* Rename server to watcher.

* Start local server by default.

* Split into multiple packages.

* eth-client API methods.

* Update docs.
2021-05-28 16:56:40 +05:30