Commit Graph

305 Commits

Author SHA1 Message Date
f34d83c04b Use async functions in wasm imports (#28)
* Use async functions in wasm imports

* Complete implementing asyncify with assemblyscript loader
2021-12-28 16:08:05 +05:30
Ashwin Phatak
bf54e85d05 eth_call and invoking event handler in WASM (#25)
* Perform eth_call and get result in js import

* Use dummy value for eth_call

* Called handler in wasm with event param

* Implement passing event to subgraph handler function

* Use generated event class Test for passing to handler

* Pass event params to handler

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-12-28 16:08:05 +05:30
2bc40896b0 Point graph-cli to use fork for automatic graph-ts exports (#22) 2021-12-28 16:08:05 +05:30
e10f61ba61 Implement typeConversion APIs bytesToHex, bigIntToString and stringToH160 (#19)
* Implement host api typeConversion.bytesToHex

* Complete host api bigIntToString.

* Create TypeId for assemblyscript loader.

* Implement host apis bigInt fromString, plus and minus
2021-12-28 16:08:04 +05:30
Ashwin Phatak
d247815ce2 Example subgraph, ethCall test to drive implementation (#18)
* Setup example subgraph.

* Implement eth_call in subgraph.

* eth-call test stub

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-12-28 16:08:04 +05:30
Ashwin Phatak
f078db688a Eden subgraph artifacts, host API stub (#15)
* Skip AssemblyScript during TS transpilation

* Eden subgraph build files

* Eden subgraph host API stubs
2021-12-28 16:08:04 +05:30
Ashwin Phatak
9e6f6b4519 AssemblyScript loader experiments (#13) 2021-12-28 16:08:04 +05:30
Ashwin Phatak
889e96572d Host API stubs (#12) 2021-12-28 16:08:04 +05:30
Ashwin Phatak
1421ba5a9b WASM instance setup (#11)
* Code for running wasm file.

* Add test for wasm exported function.

* Use target names for build files.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-12-28 16:08:04 +05:30
Ashwin Phatak
987457992a Create graph-node package. (#10)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-12-28 16:08:04 +05:30
62521d7ccc Use postgraphile client to fetch blocks 2021-12-28 16:08:04 +05:30
prathamesh0
bb1345c696 Refactor code to initialize clients (#290)
* Refactor CLI code

* Refactor code to initialize clients

* Don't return config from function to initialize clients
2021-12-28 16:08:04 +05:30
prathamesh0
a846917f1d Export-import watcher state and IPFS integration (#289)
* Add export-watcher CLI generation

* Export snapshot block and add import-watcher CLI gen

* Update codegen docs with export and import commands

* Use different function to get latest checkpoint in export CLI

* Update generated watcher docs with import state instructions

* Push IPLDBlocks to IPFS

* Change ts type for data column to Buffer

* Add checkpoint hook

* Push CLI checkpoint to IPFS

* Fix latest IPLDBlock query to filter out irrelevant blocks

* Add CLI to inspect a CID

* Decode data before exporting

* Encode data in JSON for getState GQL API

* Use go-ipfs and export cbor-encoded buffer

* Remove empty post-block methods from existing watchers

* Call hooks and checkpoint job complete handlers in fill

* Remove subscription to hooks queue from existing watchers
2021-12-28 16:08:04 +05:30
prathamesh0
f2b150995b Boolean state param to indexer methods and IPLDBlocks only in the pruned region (#286)
* Remove checkpoint option for state param. to indexer methods

* Change state param. to indexer methods to boolean

* Use hierarchical query to get the last checkpoint of a contract

* Filter out blocks before prev checkpoint for a new checkpoint

* Add checkpointing job on completion of chain pruning

* Update codegen docs with reset commands

* Create initial checkpoint in the checkpoint queue

* Handle initial checkpoint along with other checkpoints

* Create initial checkpoint job in indexer

* Create IPLDBlocks only in the pruned region

* Create staged diff IPLDBlock for default state
2021-12-28 16:08:04 +05:30
prathamesh0
27104f9d74 Optional server port arg. for watcher generation (#284) 2021-12-28 16:08:04 +05:30
c349b23f29 Add reset CLI generation 2021-12-28 16:08:04 +05:30
prathamesh0
4ddb8c4af6 Auto generating state from indexer methods (#277)
* Generate default derived state for Mapping type variables

* Update IPLDBlock in storage methods instead of using a private class variable

* Default state flag for indexer storage methods

* Helper functions to update state objects

* Add checkpoint flag in Contract table and corresponding changes in existing watchers

* Update codegen docs

* Add examples to generated docs

* Turn default state off by default

* Make state parameter to indexer storage methods default to none

* Add method to get prev. state in indexer
2021-12-28 16:08:04 +05:30
prathamesh0
2aa0234da5 Change initial checkpoint hook and hook status entity naming (#269) 2021-12-28 16:08:04 +05:30
prathamesh0
421e7498d3 Ensuring chronological execution of hooks and checkpointing (#264)
* Subscribe to hooks queue in existing watchers

* Change naming strategy for generated get and save functions

* Push checkpointing job after post-block hook job completed

* Using hooks status to ensure their chronological execution

* Add default indices to IPLDBlock table

* Add kind parameter to getState GQL API

* Add checkpoint CLI

* Add blockHash arg to checkpoint CLI and update codegen docs

* Print out block hash for checkpoint CLI

* Use log from debug for logging

* Filter using contract at start in hierarchical query

* Make kind argument to prepare IPLDBlock required
2021-12-28 16:08:04 +05:30
prathamesh0
51b200709b Generate IPLD blocks table and related GQL API (#260)
* Add ipld-blocks entity generation

* Populate ipld-blocks table

* Rename ipld-block entity and update after each event

* Move ipld-hook to hooks.ts

* Change IPLD block structure

* Add cid field in blocks

* Fetch prev. IPLDBlock for a contract

* GQL API to query IPLDBlock by CID

* Save cid in blocks in existing watchers

* Update codegen docs

* GQL API for getting last derived state (#3)

* GQL API for getting last derived state

* Rename query to getState

* Change query names to getState and getStateByCid

* Save BigInt as string

* Move function to prepare IPLDBlock to indexer

* Refactor IPLDBlock hook

* Add genesis hook

* Call post-block hook after a block is marked as complete

* Add IPLDBlock checkpointing

* Use queryRunner instead of a new repo for queries

* Add a query to get block in ipld-eth-client

* Get latest checkpoints for all contracts for checkpointing.

* Call post-block hook in a queue

* Pass server config to Indexer in watch-contract cli

Co-authored-by: nikugogoi <nabarun@deepstacksoft.com>
2021-12-28 16:08:04 +05:30
7b6f6e468f
Move update sync status outside job-runner (#324) 2021-12-21 15:46:05 +05:30
prathamesh0
2bcf579859
Remove unknown events of parent block when processing a block (#323) 2021-12-20 17:00:14 +05:30
prathamesh0
5b632a72aa
Store contract addresses in uni-info-watcher entities in lowercase (#322) 2021-12-17 17:55:41 +05:30
5dc6582ec8
Log time taken for processing single block inside job-runner (#321) 2021-12-17 15:20:33 +05:30
prathamesh0
01fa6e2184
Fix asserts for nullable values in uni-info-watcher mapping code (#320) 2021-12-17 11:57:32 +05:30
cfd293f11b
Fix job-runner stop and restart in uni-watcher (#319)
* Fix job-runner stop and restart in uni-watcher

* Fix blocks processed twice after processing missing parent block
2021-12-17 11:57:09 +05:30
prathamesh0
d087667177
Update sync status after pruning before processing next block (#318)
* Update sync status after pruning before processing next block

* Exit gracefully on error while prefetching blocks
2021-12-17 10:23:04 +05:30
105b26d6a3
Performance improvements for fill CLI (#314)
* Fix fill cli to work with watcher server

* Add createdAt column and insert events in batches

* Implement prefetch to fill block and events in parallel

* Fix getPrevEntity and increase fill prefetch default batch size

* Fix watcher creating mulitple jobs for a block
2021-12-16 17:16:48 +05:30
b06d931054
Add time logs for uniswap watchers (#310)
* Log time taken in watchers

* Log time for all events and for each event batch

Co-authored-by: prathamesh0 <prathamesh@deepstacksoft.com>
2021-12-14 14:48:29 +05:30
b345d25bb8
Optimize and reduce queries in block processing job (#308)
* Improvement TODOs

* Get parent and current block in single query

* Remove block progress queries and events query

* Reduce queries in fetching batch events by using query builder

* Implement changes in codegen package
2021-12-13 15:38:34 +05:30
f56f7a823f
Process events in singe job and avoid block progress query to improve performance (#306)
* Avoid database query by passing event directly to job-queue

* Avoid block progress query by returning from update query

* Process batch of events for a block in a single job

* Fix smoke test for subscribed events and use teamSize for job queue
2021-12-10 10:44:10 +05:30
prathamesh0
a2ad139769
Use node v16 and pprof to generate profile data for watchers (#299)
* Use pprof to generate profile data for watchers

* Update docs and github workflows to use node v16
2021-12-09 10:39:35 +05:30
6f98166c49
Changes to improve performance for event processing job (#304)
* Perf improvement TODOs

* Move watch contract to util package

* Get block progress from event instead of querying

* Use index field in event to check order of processing

* Use watched contract map to avoid querying database

* Use update query for blockProgress entity

Co-authored-by: Ashwin Phatak <ashwinpphatak@gmail.com>
2021-12-08 11:11:29 +05:30
prathamesh0
f89a7a07aa
Use eth client for getStorageAt query (#302) 2021-12-06 17:22:09 +05:30
ec586216db
Fix events out of order in job-queue (#300) 2021-12-06 17:17:55 +05:30
cda55646d2
Update uni-info-watcher with changes in uniswap subgraph mapping (#298)
* Update mapping code with latest subgraph changes

* Add mapping code for TickDayData entity
2021-12-03 16:24:13 +05:30
63ce6fd55f
Handle missing blocks in postgraphile by making RPC call (#296) 2021-12-03 16:23:11 +05:30
32fea1f2cb
Fix uni-watcher to handle mainnet data (#293)
* Fix watchers to handle mainnet data

* Tweak jobs fetched per interval to reduce event processing time
2021-12-02 15:28:03 +05:30
prathamesh0
08c712d766
Handle bigint and decimal types for entities in watchers (#295)
* Add a GraphDecimal class to replace Decimal with

* Replace Decimal in uni-info-watcher with GraphDecimal

* Add definition for custom scalar BigDecimal in uni-info-watcher

* Use pg-type numeric and transformer for bigint columns in typeorm entities
2021-12-02 13:22:29 +05:30
9d95e49ec9
Change block processing to be pull based (#288)
* Implement pull based watcher for uni-watcher

* Fix same block processed multiple times

* Implement wait time for fetching block from config

* Use blockProgress event to fetch and process next block

* Rename utils index to misc
2021-10-26 17:36:21 +05:30
137f7d1a21
Fix eth_calls with blockHash (#287)
* Fix eth_calls with blockHash

* Use blockHash in erc20 eth_calls
2021-10-25 19:51:16 +05:30
737d9a9b6e
Handle missing blocks by making eth call to ipld-eth-server (#285) 2021-10-22 15:20:11 +05:30
6f3b8029b2
Use blockHash for eth_calls in watchers (#283) 2021-10-21 15:23:46 +05:30
7953ba2949
Replace storage calls in uni-info-watcher with eth_calls (#281)
* Replace storage calls in uni-info-watcher with eth_calls

* Handle eth_call reverts and catch them in uni-info-watcher
2021-10-21 13:24:46 +05:30
5cbcd455d2
Purge unknown events after block processing complete (#280) 2021-10-20 17:49:44 +05:30
47b9e6bbbd
Implement reset job queue and state CLI in watchers (#276)
* Implement clean jobs CLI in watchers

* Pull common code cleanJobs to util

* Implement commands for reset job-queue and state

* Reset command for job-queues and watcher state

* Reset sync status on reset state

* Fix using cli from built js

* Implement skipCheck to reset syncStatus table and skip complete handler on fail

* Check for block isComplete on reset state

* Set default value for force param to update syncStatus to false

* Reset tables in erc20 watcher

* Push job in fill after previos block is complete
2021-10-20 16:06:03 +05:30
f00ea0c1f0
Implement lazy watcher mode in erc20-watcher (#275) 2021-10-18 13:03:55 +05:30
3b27177389
Fix query when filtering by bigint type field (#268) 2021-10-14 17:19:25 +05:30
e0933f10b2
Resolve nested relations required by frontend app (#267) 2021-10-14 12:30:23 +05:30
bc7588599d
Increase sleep in smoke test to process events (#263) 2021-10-13 15:34:37 +05:30
08c0668ddb
Get endpoint URL from config in tests (#262) 2021-10-13 11:01:36 +05:30
prathamesh0
d3971b5258
Generate GQL client (#259)
* Add GQL client generation

* Add subscription, events, eventsInRange in client gen

* Add mutation in client gen and return event tx info from indexer

* Capitalize class names for storage-mode entities
2021-10-04 11:04:06 +05:30
Ashwin Phatak
a9067c4374
Update codegen docs (#258)
Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-30 10:44:30 +05:30
d456a3f676
Include NFPM collect event in smoke tests. (#257) 2021-09-29 17:14:16 +05:30
prathamesh0
278fe30a2c
Add GQL mutations support (#256) 2021-09-29 13:49:01 +05:30
prathamesh0
40574cf3d9
Support GQL subscriptions, fill and custom hook for indexing on event (#255)
* Custom hook support for indexing on events.

* Add fill support.

* Process GQL subscriptions.

* Add hooks example.

* Update hooks example.
2021-09-29 10:04:09 +05:30
prathamesh0
8e3093c684
Codegen flag for watcher kind and eventsInRange query support (#254)
* Add watcher kind argument.

* Process eventsInRange query.
2021-09-28 10:22:21 +05:30
Ashwin Phatak
338cef9954
Generate entities from YAML templates and lint support in generated watchers (#253)
* Add lint files generation.

* Fix lint errors in generated code.

* Load default entities from yaml files.

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-27 18:03:04 +05:30
Ashwin Phatak
11d16b9870
Implement active watcher. (#252)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-27 15:24:17 +05:30
Ashwin Phatak
482be8cfaa
Support storage mode for code generator (#251)
* Implement storage mode in code generator.

* Fix indexer template for eth_call.

* Add support for both eth_call and storage modes.

* Add eventsInRange, events file gen and default entites gen.

* Use static property to column maps in database.

* Process events query.

* Avoid adding duplicate events in indexer.

* Fix generated watcher query with bigint arguments.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-27 10:13:50 +05:30
Ashwin Phatak
92b7967895
Generating eth_call based lazy watcher (#249)
* Add entity generation.

* Add resolvers generation.

* Add queries in resolvers generation.

* Add indexer generation.

* Extract helper code in utils.

* Add server and artifacts generation.

* Fix solidity-flattener issue.

* Update readme and cleanup misc files.

* Add queries to entity generation.

* Add database generation.

* Use snakecase in database.

* Add readme generation.

* Change template file names.

* Add method descriptions.

* Change mode to eth_call in readme.

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-23 16:55:46 +05:30
Ashwin Phatak
4f8f1d8cd7
Update code for log_cids change in vdb core (#243)
* Remove fields no longer present in schema.

* Refactor code to work with log_cids change in eth-server.

* Refactor process event to save events from logs.

* Use constants for erc20 event names.

* Implement watch blocks in erc20-watcher similar to uni-watcher.

* Moved common methods to util.

* Implement eventsInRange query in erc20-watcher.

* Filter unknown event in database query.

* Change dependencies version to be same in all packages.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-21 16:43:55 +05:30
Ashwin Phatak
0a96bbd94d
Change dependencies version to be same in all packages. (#250)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-21 10:27:34 +05:30
Ashwin Phatak
6560639a68
Use flatten function and modify ResultEvent in schema. (#247)
Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-17 17:10:08 +05:30
Ashwin Phatak
f9934675b2
CLI to flatten and generate GQL schema from Solidity file (#245)
* Add schema generation.

* Use yargs and add a script to generate schema.

* Add command to generate schema to README.md

* Remove use of arrow functions in visitor.

* Add function to export schema in visitor.

* Create Event union while adding to it.

* Add bool to typemappings.

* Add method descriptions and remove output type mappings.

* Add storage mode.

* Add URL support and a flattening script.

* Fix parameter for flattening in README.md.

* Add setup and references to README.md.

Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-16 17:06:10 +05:30
Ashwin Phatak
a9dc34f704
Fix null parent. (#244)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-09 10:51:58 +05:30
Ashwin Phatak
06bed1c13b
Enable source maps for transpiled files (#242)
* Rename .spec files to .test files.

* Enable source maps.

* Pass enable-source-maps flag for transpiled files.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
Co-authored-by: prathamesh <prathamesh.musale0@gmail.com>
2021-09-07 16:01:15 +05:30
Ashwin Phatak
d3a50d2b18
Fix package versions after cleaning dependencies. (#241)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-02 16:13:55 +05:30
Ashwin Phatak
2d493b3838
Run services from transpiled ES5 (#239)
* Transpile ES5 code.

* Use transpiled ES5 code in all packages.

* Refactor tracing-client to transpile ES5 code.

* Order scripts in package json.

* Update readme in packages with new scripts.

* Refactor code to use entity config programatically.

* Remove unused depencdencies.

* Use DEBUG environment for scripts.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-09-02 10:27:56 +05:30
Ashwin Phatak
fc4f49113b
Bug fixes from smoke testing (#238)
* Fix update for entity relations.

* Implement demo values in uni-info-watcher indexer code.

* amend

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-31 18:05:29 +05:30
Ashwin Phatak
f1f70ad7d4
Move chain pruning to block processing queue with higher priority (#235)
* Move chain pruning to block processing queue.

* Refactor common code in util.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-30 11:16:54 +05:30
Ashwin Phatak
458956a3a1
Fix ts-node watch for file changes. (#234)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-27 18:14:23 +05:30
Ashwin Phatak
cf0713eda7
Tasks in erc20-watcher to deploy and transfer tokens. (#233)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-27 17:47:23 +05:30
Ashwin Phatak
8fa6229562
Fix queries, demo mode to display all tokens (#231)
* Fix queries for frontend info app.

* Fix query for nested relations.

* Add demo mode in uni-info-watcher.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-26 15:36:16 +05:30
Ashwin Phatak
5941a929d2
Implement health query for uniswap-info. (#228)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-25 17:12:23 +05:30
Ashwin Phatak
afd7c954a2
Refactoring pruning and reorg handling code (#227)
* Refactor db methods getPrevEntityVersion and getFrothyRegion.

* Filter out entities from pruned blocks.

* Pull saveContract and getModelEntities to util.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-25 12:47:53 +05:30
Ashwin Phatak
3ff2fdf11b
Refactoring, extract uni-info-watcher GQL client (#226)
* Refactor common code in uni-watcher and uni-info-watcher.

* Implement client.ts in uni-info-watcher for queries and use in smoke-test.

* Disable cache in graphql client.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-24 11:55:29 +05:30
Ashwin Phatak
885a55e513
Implement hierarchical query for checking ancestor block during pruning (#225)
* Implement hierarchical query for checking ancestor block.

* Prune blocks together in single query.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-23 17:06:35 +05:30
Ashwin Phatak
2ddedfefe7
Refactoring common code into util (#224)
* Refactor and pull common database code to util.

* Refactor and pull common indexer code to util.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-20 19:02:57 +05:30
8489b5632a
Tests for pruning abandonded branches (#223)
* Create dummy test blocks and refactor test code.

* Intial test for pruning block in a chain without branches.

* Add test for pruning with multiple branches.

* Add test for pruning at frothy region.

* Test for pruning block at height more than max reorg depth.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-20 12:26:37 +05:30
Ashwin Phatak
35068b2c3d
Implement chain pruning in uni-info-watcher (#222)
* Refactor code for chain pruning.

* Implement chain pruning in uni-info-watcher.

* Refactor pruning code to util.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-19 13:27:32 +05:30
Ashwin Phatak
b8b216ea5b
Extract block filler into common package, implement in uni-info-watcher (#220)
* Implement block filler for uni-info-watcher.

* Pull common code for Block filler to util.

* Watch for job events in block filler command.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-18 15:50:44 +05:30
Ashwin Phatak
23bfcc02dc
Implement heirarchical query for getting previous entity (#216)
* Implement heirarchical query for getting previous entity.

* Check entity id in hierarchical query to limit number of recursions.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-16 17:55:23 +05:30
Ashwin Phatak
9e1ed70d18
Tests for getPrevEntityVersion (#217)
* Insert and remove a dummy BlockProgress from db.

* Test to fetch a Token entity from pruned region.

* Check if db is empty initially.

* Test to fetch a Token entity from frothy region.

* Test to fetch a Token from frothy region (same block num).

* Test to fetch a Token entity from another branch.

* Tests to fetch a Token entity (multiple tokens).

* Test to fetch updated Token entity in pruned region.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-16 15:42:34 +05:30
Ashwin Phatak
ec92779c4e
Link to original code/license for reused source (#214) 2021-08-13 15:23:42 +05:30
Ashwin Phatak
eb912b7f68
Blocks GQL API (#213)
* Add schema for blocks query.

* Implement resolver for blocks query.

* Fix issue with multiple query filters for one entity field.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-13 15:06:58 +05:30
Ashwin Phatak
a7ec3d8da8
Add APGL license and copyright notices (#212)
* Add license & copyright declarations for add-watcher.

* Add copyright declarations for cache.

* Add copyright declarations for erc20-watcher.

* Add copyright declarations for ipld-eth-client.

* Add copyright declarations for tracing-client.

* Add copyright declarations for uni-watcher.

* Add copyright declarations for solidity-mapper.

* Add copyright declarations for uni-info-watcher.

* Add copyright declarations for util.

* Add copyright declarations for lighthouse-watcher.

* Change license identifier in .sol files.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-12 15:28:13 +05:30
Ashwin Phatak
43107b9ed7
Remove unused settings from lighthouse config. (#210)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-12 10:33:23 +05:30
Ashwin Phatak
4fb0d95135
Lighthouse watcher (#205)
* Implement watcher for Lighthouse StorageRequest event.

* Add smoke test for lighthouse watcher.

* Add fileCost as value to Lighthouse store call.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-11 17:27:49 +05:30
Ashwin Phatak
053695a3e8
Add test:init in uni-watcher. (#204)
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-11 16:27:42 +05:30
Ashwin Phatak
cbc8dce88b
Implement transaction for uni-info-watcher blocks and events update. (#203)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-11 15:23:44 +05:30
Ashwin Phatak
1372fc74ef
Test for entities after CollectEvent. (#202)
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-10 17:31:50 +05:30
Ashwin Phatak
ed9577b830
Test for entities after DecreaseLiquidityEvent. (#199)
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-10 16:32:35 +05:30
Ashwin Phatak
6c05600b41
Test for entities after IncreaseLiquidityEvent. (#198)
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-10 15:53:59 +05:30
Ashwin Phatak
0c2efde028
Test for entities after TransferEvent (#196)
* Add resolver for Position entity in uni-info-watcher.

* Test for entities after TransferEvent.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-10 12:09:00 +05:30
nikugogoi
abd3175c09
Import test utils separately. (#194)
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-09 21:34:55 +05:30
Ashwin Phatak
16b041c5cf
Test for Transaction entity after Mint, Burn & Swap events. (#193)
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-09 12:08:37 +05:30
Ashwin Phatak
a9d9b39c37
Fix transaction query runner in event handlers. (#192)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-09 11:30:43 +05:30
Ashwin Phatak
88121f9390
Test for entities after SwapEvent. (#191)
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-06 19:10:25 +05:30
Ashwin Phatak
91463136d7
Implement transactions in uni-watcher. (#190)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-06 17:53:55 +05:30
Ashwin Phatak
63620f0a0a
Smoke test for entities after BurnEvent (#189)
* Test for entities after BurnEvent.

* Fix event value types in uni-info-watcher.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-06 17:21:30 +05:30
Ashwin Phatak
95f830178d
Test for all entities after MintEvent. (#188)
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-06 12:12:52 +05:30
Ashwin Phatak
0487c05ee1
Event handlers should update db atomically (#186)
* Run db operations inside event handlers atomically using transaction.

* Implement database transaction for Pool Initialize event.

* Implement typeorm transaction without callback.

* Implement transaction for NFPM event handlers.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-06 10:25:56 +05:30
Ashwin Phatak
b6fe8a8c47
Import TICK_MIN from util and fix lint errors. (#187)
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-05 18:33:47 +05:30
Ashwin Phatak
df85f961db
Smoke test for entities after Mint event (#185)
* Test for Token & Factory entities after MintEvent.

* Test for Pool, Mint & Tick entities after MintEvent.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-05 17:12:11 +05:30
Ashwin Phatak
a9d411c6df
Smoke test for uni-info-watcher (#184)
* Smoke test for uni-info-watcher and refactor token deployment code.

* Test for Token entity after PoolCreated event.

* Test for Pool entity after PoolCreated event.

* Tests for entities after InitializeEvent.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-04 18:57:44 +05:30
Ashwin Phatak
73e0475dfa
Misc changes for uni-info-watcher resolvers (#183)
* Implement transaction from to set entity origin field.

* Add all transaction fields specified in schema and add delay in uni-info block processing.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-04 18:42:59 +05:30
Ashwin Phatak
eea3a5864b
Implement filters for uni-info-watcher resolvers. (#180)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-03 11:56:25 +05:30
Ashwin Phatak
d9366017da
Smoke test for NFPM events (#178)
* Added NFPM tests.

* Refactor smoke test.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-08-02 16:13:56 +05:30
Ashwin Phatak
f1a96e8ba9
Resolver for Ticks and Token(s) methods (#177)
* Add query resolvers for Tick and Token entities.

* Implemented query for orderBy.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-08-02 15:46:21 +05:30
Ashwin Phatak
9a3ac28a5d
uni-watcher smoke test (#172)
* Created a smoke test for uni-watcher.

* Added test for createPool.

* Added more pool tests.

* Using ethers instead of hardhat-ethers in smoke test.

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2021-07-30 19:50:56 +05:30
Ashwin Phatak
18ebed0b36
Resolver for Mint, Pool, PoolDayData and Swap entities (#173)
* Use getEntities to get result for queries.

* Implement resolver for Mint entity.

* Implement resolvers for Pool and PoolDayData entities.

* Add query resolver for Swap entity.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-30 17:49:22 +05:30
Ashwin Phatak
d4a19d15a9
Implemented query resolvers for entities Burn and Factory. (#171)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-29 18:12:35 +05:30
Ashwin Phatak
175fa48d71
Prune abandoned branches (#166)
* Prune chain queue, refactoring job runner.

* Prune blocks on non-canonical branches.

* Prune blocks not reachable from head, update canonical block.

* Note to move pruning to block processing queue.
2021-07-29 14:54:40 +05:30
Ashwin Phatak
df01b6539b
Implement resolvers for querying Bundle entity. (#170)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-29 14:45:38 +05:30
Ashwin Phatak
f93c0e3cb6
Track last processed event index (#168)
* Implement last processed event in uni-info-watcher.

* Check if block or event is already processed.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-28 17:36:43 +05:30
Ashwin Phatak
c677e5942c
ERC20 watcher based on eth_call (#165)
* Implement eth_call for ERC20 totalSupply.

* Use eth_call for erc20-watcher.

* Implement fallback for ERC20 symbol method call.

* Implement fallback for ERC20 name and totalSupply.

* implement fallback for erc20 decimals method.

* Lint fixes.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-28 10:04:07 +05:30
Ashwin Phatak
a5d6bfa285
Get/create entity based on block hash and parent traversal (#163)
* Watch all blocks and fetch events for each block.

* Implement getPrevVersionEntity in database.

* Implement getPrevVersionEntity for PoolCreated event.

* Implement getPrevEntityVersion for Pool events.

* Implement getPrevEntityVersion for NFPM events.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-23 19:22:55 +05:30
Ashwin Phatak
f8335aad03
Add block hash to entities, split get and set methods (#161)
* Split get and set entity for PoolCreated and Initialize events.

* Split get/set entity for Mint, Burn and Swap events.

* Split get/set entity for IncreaseLiquidity and DecreaseLiquidity events.

* Split get/set for NFPM Transfer and Collect events.

* Add blockHash to entities.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-23 11:00:40 +05:30
Ashwin Phatak
26965f372f
Handle reorgs upto known canonical block hash (#157)
* Handle reorgs upto known canonical block hash.

* Process block events strictly in order.

* Support filling old blocks, process in-order.

* Always publish block progress.
2021-07-22 16:32:39 +05:30
Ashwin Phatak
34de3d51b3
Handle collect event (#158)
* Handle NFPM Collect event.

* Handle NFPM Transfer event.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-22 11:45:21 +05:30
Ashwin Phatak
0d39363467
Move uni-info-watcher event procesing to job queue. (#156)
* Move event procesing to job queue.

* Store block properties in blocks table.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-22 10:02:06 +05:30
Ashwin Phatak
95486d6553
Save block parent hash (#155)
* Save block parent hash.

* Store block properties in blocks table.
2021-07-21 13:00:26 +05:30
Ashwin Phatak
10e7d37fa8
Update pgboss queue params. (#154) 2021-07-20 15:46:35 +05:30
Ashwin Phatak
c1ef96ba2e
Handle Position Manager DecreaseLiquidity event. (#148)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-19 10:43:29 +05:30
Ashwin Phatak
3477366458
Handle increase liquidity event (#143)
* Implement handler for NFPM IncreaseLiquidity event.

* Get contract values by querying uni-watcher.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-16 18:34:51 +05:30
Ashwin Phatak
ae13edb99a
Update docs. (#142) 2021-07-16 16:35:49 +05:30
Ashwin Phatak
29c8c1f80f
Implement event watchers for NonFungiblePositionManager events. (#141)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-15 17:27:10 +05:30
Ashwin Phatak
70e88b1004
Block filler for uni-watcher. (#140) 2021-07-15 13:10:07 +05:30
Ashwin Phatak
7151521c3b
Get uniswap events in block range (#139)
* Get events in block range.

* Get uniswap events in block range.
2021-07-15 10:48:06 +05:30
Ashwin Phatak
ecfa3ed386
Handle swap event (#138)
* Create Swap entity.

* Update day and hour interval data in Swap event handler.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-15 10:45:35 +05:30
Ashwin Phatak
03ceb95a1b
Job queue to process events (#137)
* Job queue to process events.

* Event queue processing changes.
2021-07-14 18:00:26 +05:30
Ashwin Phatak
a4f5d43bc5
Extra info for handling events (tx, block) (#136)
* Extra info in events for downstream processing.

* Changes in uni-info-watcher after change in uni-watcher event schema.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-13 17:32:57 +05:30
Ashwin Phatak
208b0f7f4f
Handle Pool burn event. (#134)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-13 12:36:10 +05:30
Ashwin Phatak
3a6af9f9cc
Handle mint event (#130)
* Handle mint event and update Factory fields.

* Integrate decimal.js with typeorm.

* Update transaction in Mint event.

* Update day and hour data.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-13 12:01:54 +05:30
Ashwin Phatak
d71557e963
Event processing/sync changes to maintain processing order. (#131) 2021-07-12 17:06:33 +05:30
Ashwin Phatak
69c68b365f
Include txHash in event sent to GQL subscribers. (#129) 2021-07-09 17:01:36 +05:30
Ashwin Phatak
61f211f2d5
Handle pool initialize event (#127)
* Move getConfig to util package.

* Handle Pool initialize event.

* Update Bundle entity ethPriceUSD.

* Update Pool day and hour data.

* Update token derivedETH and complete handleInitialize.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-09 12:38:25 +05:30
Ashwin Phatak
7f5229bf2f
Watch pool burn and swap events (#128)
* Remove test scripts, moved to vulcanize/uniswap-v3-core#watcher-ts.

* Move uniswap analysis docs.

* Watch pool burn and swap events.
2021-07-09 12:30:50 +05:30
Ashwin Phatak
538e4b7591
Watch pool mint event. (#125) 2021-07-08 17:40:40 +05:30
Ashwin Phatak
f7df56bb75
Handle pool created event (#124)
* Update Token entities.

* Complete handlePoolCreated in events.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-07 11:10:10 +05:30
Ashwin Phatak
194d079d5e
Watch pool initialize event (#123)
* Script to init pool.

* Watch pool initialize event.
2021-07-06 20:05:40 +05:30
Ashwin Phatak
aec9281fb8
Factory PoolCreated event handler (#120)
* Setup handler for PoolCreated event.

* Create Pool entity.

* Subscribe to uni-watcher for watching events.

* Refactor code to create GraphQLClient in ipld-eth-client.

Co-authored-by: nikugogoi <95nikass@gmail.com>
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-06 16:55:11 +05:30
Ashwin Phatak
9c60895352
Save watched contracts in db. (#122) 2021-07-06 15:58:03 +05:30
Ashwin Phatak
0b6a33561b
Include block number in GQL event. (#121) 2021-07-06 11:04:08 +05:30
Ashwin Phatak
8d2a4c6b14
Watch for Factory PoolCreated event. (#119) 2021-07-05 16:15:54 +05:30
Ashwin Phatak
da758aceaa
Script to deploy uniswap contracts locally for testing (#118)
* Initial setup with hardhat.

* Deploy Factory contract.

* Deploy tokens and create pool using factory contract.

* Deploy contract to private network.

* Implement separate scripts for deploying Factory, Token and Pool.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-07-05 11:05:45 +05:30
Ashwin Phatak
b7ffb7c672
uniswap-info-watcher package with frontend schema (#114)
* Implement mock bundle query in watcher.

* Use chance js for generating mock data.

* Add bundles query.

* Setup resolvers with uniswap info schema.

* Add mock query for Burn entity.

* Add mock query for Factory entity.

* Add mock queries for entities Mint, Pool and Transaction.

* Add mock query for Swap entity.

* Add mock queries for day data entities.

* Implment mock ticks query.

* Add mock tokenHourDatas query.

* Add mock queries for uni-info-watcher to readme.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-07-02 16:26:32 +05:30
Ashwin Phatak
1b6d4511f7
Minimal schema required by info-watcher. (#115) 2021-07-02 12:47:15 +05:30
Ashwin Phatak
5de555a4e5
Uniswap watcher package with event schema (#113)
* Uniswap watcher package skeleton.

* Uniswap events.
2021-07-01 16:42:37 +05:30