Commit Graph
604 Commits
Author SHA1 Message Date
prathamesh0 4bfb007a7e Refactor job-runner CLI to cli package (#255)
* Fix eden-watcher server initialization

* Add an indexer method to watch subgraph contracts

* Refactor job-runner CLI to cli package

* Move watcher reset commands to refactored code
2022-11-23 18:20:16 +05:30
nabarun cc28474537 Accommodate GQL optimizations in codegen (#254)
* Prune optional methods in indexer and database interfaces

* Implement GQL optimization changes in codegen

* Fix graph-node test indexer

* Add demos to codegen package
2022-11-23 17:42:25 +05:30
prathamesh0 0b33cc98c9 Refactor server CLI to cli package (#253)
* Refactor server CLI to cli package

* Use server CLI from cli package in all watchers
2022-11-23 12:57:59 +05:30
prathamesh0 122a64c2f9 Refactor export-state and verify checkpoint CLIs to cli package (#252)
* Refactor export-state CLI to cli package

* Refactor verify checkpoint CLI to cli package

* Update mock indexer object
2022-11-22 18:27:49 +05:30
nabarun ace52d9da3 Add GQL metrics to ERC20 and ERC721 watchers (#251)
* Add changes in erc20-watcher

* Add changes in erc721-watcher

* Add changes in mobymask-watcher

* Add reset state CLI to all watchers
2022-11-22 17:04:38 +05:30
prathamesh0 6737ec756c Refactor import-state CLI to cli package (#250)
* Remove unnecessary upstream config arg to event watcher

* Initialize event watcher in CLI package

* Refactor import-state CLI to cli package
2022-11-22 16:41:15 +05:30
nabarun 6622d0874e Accommodate GQL optimizations in graph-test-watcher (#249)
* Use separate subgraph entities list

* Implement changes in graph-test-watcher

* Reset latest entity tables and use cache directives in schema GQL
2022-11-22 15:20:44 +05:30
prathamesh0 570640d4bc Refactor inspect-cid CLI to cli package (#248) 2022-11-22 11:45:45 +05:30
prathamesh0 03050e5323 Refactor create checkpoint CLI to cli package (#247)
* Refactor create checkpoint CLI to cli package

* Update mock indexer
2022-11-22 10:38:23 +05:30
nabarunandprathamesh 06e5c62fa8 Remove saveBlockAndFetchEvents method callback pattern (#244)
* Remove saveBlockAndFetchEvents method callback pattern

* Accomodate change in code generator

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2022-11-21 15:54:28 +04:00
prathamesh0 1e639c1af3 Refactor reset CLIs to cli package (#246)
* Refactor reset CLIs to cli package

* Use reset CLIs from cli package in watchers

* Fix method to canonicalize latest entities

* Use composition with common code for refactored CLIs
2022-11-21 15:14:10 +04:00
prathamesh0andnabarun 7520e9012c Refactor watch-contract CLI to cli package (#245)
* Refactor watch-contract CLI to cli package

* Use watch-contract from cli in eden and erc721 watchers

* Use watch-contract from cli in all watchers

* Pass an object containing clients to refactored CLI

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-11-21 11:47:23 +04:00
nabarun a0ba657d17 Rename variables and fix comments (#243)
* Rename entity to entityType

* Rename variables and fix comments
2022-11-18 19:16:42 +05:30
nabarun 662c79a5e7 Refactor event-watcher and move code to util (#242) 2022-11-18 17:23:33 +05:30
prathamesh0 f3c65cbd64 Refactor util code to be reused (#241)
* Ignore watch contract jobs in event processing complete handler

* Update job-queue config and handle errors on job completion hook

* Update graph decimal implementation

* Return generic type from method to read watcher config

* Export fill prefetch batch size default value
2022-11-18 16:29:06 +05:30
nabarun cc8fcffaa1 Fix use of prefetchBlocksInMem flag in block processing (#240)
* Fix use of prefetchBlocksInMem flag in block processing

* Rename prefetchedBlocksMap to blockAndEventsMap
2022-11-18 11:01:09 +05:30
prathamesh0 92fd3cac03 Upgrade client libraries (#239) 2022-11-17 19:06:07 +05:30
nabarun 635b26aa0c Use entity is_pruned flag in queries (#238) 2022-11-17 13:55:53 +05:30
prathamesh0 f53371e17b Accomodate GQL requests caching in code generator (#237)
* Accomodate GQL requests caching in code generator

* Add GQL API request queuing
2022-11-17 12:02:08 +05:30
nabarun 79e903b396 Reset latest and frothy entity tables on watcher reset (#236) 2022-11-17 10:14:59 +05:30
nabarun 693c23d192 Update latest entity tables on chain pruning (#235) 2022-11-16 20:16:48 +05:30
prathamesh0 1ad223db4d Add GQL requests caching in eden-watcher (#234) 2022-11-16 20:01:18 +05:30
nabarun 74741184ee Use lateral query for time travel queries using latest entity tables (#233)
* Avoid joins in id only queries

* Use lateral query for time travel queries using latest entity tables
2022-11-16 18:57:04 +05:30
nabarun a084b4e40c Latest entity queries to optimize frontend app GQL requests (#232)
* Add latest entity table query pattern

* Add subscriber config to typeorm
2022-11-16 18:11:27 +05:30
prathamesh0 408a3927c0 Add a table for entites in frothy region for subgraph watchers (#231)
* Add a table for entites in frothy region and update it in a subscriber

* Accommodate changes to other watchers and codegen
2022-11-16 17:12:54 +05:30
nabarun 7e5974ccf7 Denormalize is_pruned flag in eden-watcher (#230)
* Add is_pruned flag to all entities

* Mark entities as pruned once the corresponding block gets pruned
2022-11-16 15:30:40 +05:30
prathamesh0 62c57d8005 Upgrade Apollo server to v3 (#229)
* Upgrade apollo server to v3 in eden-watcher

* Upgrade apollo server to v3 in other watchers and codegen

* Refactor common server code to util

* Pass express application object in the method to create server

* Remove unnecessary import
2022-11-16 11:14:54 +05:30
prathamesh0 a52bdf64b1 Add config for GQL requests cache (#228)
* Add config and a method to set cache control settings for GQL requests

* Add option for max size in GQL cache config

* Fix failing tests
2022-11-15 14:56:08 +05:30
nabarun 6f8ededd52 Add flag to load relations sequentially or concurrently (#225)
* Add flag to load relations sequentially or concurrently

* Fix database init in graph-node test

* Fix graph-node watcher query method and add changes to codegen
2022-11-14 14:23:46 +05:30
prathamesh0 13edff143b Update cerc-io packages version in watcher generation (#224) 2022-11-11 15:00:48 +05:30
prathamesh0 861420a10f Add a flag to enable state creation (#222)
* Add a flag to enable state creation

* Update flag in mock config for testing
2022-11-10 16:56:06 +05:30
prathamesh0 8d3c68873b Avoid creating a separate base db instance for graph database (#221)
* Avoid creating a separate base db instance for graph database

* Fix failing test file build

* Use getter method to get base database
2022-11-10 14:51:18 +05:30
nabarun ecd8b2474a Remove eventName filter in events count query (#220) 2022-11-10 12:09:17 +05:30
prathamesh0 3862f9ce2e Use FindManyOptions to get entities (#219) 2022-11-09 14:01:39 +05:30
nabarun 772fdbec25 Add unique entities query without checking pruned block (#218) 2022-11-09 13:31:26 +05:30
nabarun 7a4b459b1c Add DB queries without checking pruned block (#217) 2022-11-08 17:01:35 +05:30
Ashwin Phatak 99c6823564 GQL API request queue params (#216) 2022-11-04 12:01:07 +05:30
nabarun 0e464385c0 Remove contracts on reset watcher (#215) 2022-11-04 10:41:48 +05:30
prathamesh0andnabarun 266442a6d9 Update getFullBlocks usage (#213)
* Update getFullBlocks usage to add block number arg

* Add skipStateFieldsUpdate config for uniswap-watcher

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-11-03 16:49:41 +05:30
nabarun be65ca45ea Load relations sequentially (#212) 2022-11-03 14:32:34 +05:30
prathamesh0 2517f110ea Update GQL and getBlock API usage (#210)
* Update getLogs and getBlockWithTransactions usage to add block number arg

* Update getFullTransaction usage to add block number arg
2022-11-03 14:14:58 +05:30
nabarunandprathamesh 52c42f4e84 Reset watcher to previous indexed block on start (#207)
* Reset watcher to previous indexed block before start

* Implement changes in other watchers

* Save successfully fetched blocks and events to prefetch cache

* Add unique query for transaction table

* Check db for blocks before fetching from eth-server

* Show all mismatches at a block

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2022-11-03 14:01:10 +05:30
prathamesh0 306bbb73ca Use prefetching of blocks with events in watchers and codegen (#206)
* Avoid refetching block while fetching events

* Prefetch a batch of blocks with events while indexing

* Update mock indexer used in graph-node testing

* Process available blocks while prefetching

* Refactor events fetching to a method in util

* Move method to get GQL event query result to util
2022-10-20 18:46:56 +05:30
nabarun 668875b3a0 Refactor code in graph-node to use in uniswap watcher (#205)
* Refactor code in graph-node to use in uniswap

* Move over database methods to fetch entities for GQL query
2022-10-20 14:39:32 +05:30
prathamesh0 5af90bd388 Refactor state creation code (#204)
* Remove support for pushing state to IPFS

* Move job handlers for state creation to util

* Rename state creation related methods and objects

* Update mock indexer used in graph-node testing

* Fetch and merge diffs in batches while creating a state checkpoint

* Fix timing logs while for state checkpoint creation

* Refactor method to get state query result to util

* Accept contracts for state verification in compare CLI config

* Make method to update state status map synchronous
2022-10-19 15:24:14 +05:30
nabarun ce182bce85 Clear cache of latest entities on event processing error (#201)
* Clear cache of latest entities on event processing error

* Remove lighthouse-watcher and update ethersjs version

* Handle GraphDecimal type in state entity

* Add option for comparing all entities using paginate

* Clear pruned cached entities at intervals

* Move ipld-demo to graph-node package and remove reset-dbs script

* Implement changes in all watchers and codegen
2022-10-19 14:26:10 +05:30
prathamesh0 74747ce49a Add scripts for ERC721 token interaction (#203) v0.2.14 2022-10-17 15:38:59 +05:30
prathamesh0 45111e68ac Add an option to run compare CLI in batched intervals (#202) 2022-10-17 12:07:21 +05:30
Michael 5ffe13b723 Npm gitea publish (#200)
* update gitignore for goland and first attempt at git.vdb.to npm publish

* whitespace yaml nonsense

* missing run entry

* lerna is not cicd friendly, yet

* try from-git arg in lerna.json

* attempt default lerna publish with full checkout

* explicit cli args... lerna.json ???

* explicit cli args... from-package is not --arg

* make publish yaml match manual workflow
2022-10-13 15:12:24 -04:00
prathamesh0 bad5ec754a Prefetch blocks with events in memory (#199)
* Prefetch blocks along with events in memory

* Fix prefetched block destructuring

* Wait while blocks are being prefetched

* Retry only if block not returned by upstream eth-server

* Log time taken to prefetch blocks

* Avoid creating a separate job for prefetching blocks

* Skip syncStatus check while fetching blocks

* Convert block number string to number for comparison

* Prefetch blocks in a batch only when required

* Keep existing pattern for watchers in this repo

* Fix logging while fetching a batch of blocks
2022-10-13 17:07:46 +05:30