Commit Graph

130 Commits

Author SHA1 Message Date
prathamesh0
7069a7f455
Upgrade version to 0.2.24 (#299) 2023-01-19 15:48:38 +05:30
prathamesh0
38daa31029
Add a method to send messages on the network using pubsub (#293)
* Add a method to send messages using pubsub on a topic

* Update version to 0.2.22
2023-01-11 17:19:21 +05:30
55884a8ecd
Fix publish failing for react-peer package (#292)
* Remove empty repository from package.json

* Update version to 0.2.21
2023-01-10 22:11:40 +05:30
4f82dfbf74
Update version to 0.2.20 (#291) 2023-01-10 20:32:05 +05:30
cd29b47ecc
Implement peer package to send messages between peers (#279)
* Initial implementation of class and discoving other peers

* Implement peer connection and sending messages between peers

* Add react app and use peer package for broadcasting

* Maintain stream for each remote peer

* Refactor code in peer package

* Add serve package for react-app

* Add readme for running react app

* Add peer package readme

* Add logs for events with details

* Add a chat CLI using peer package (#280)

* Add a flag to instantiate Peer for nodejs

* Add a basic chat CLI using peer

* Add a signal server arg to chat CLI

* Add instructions for chat CLI

* Fix typescript and ESM issues after adding peer package (#282)

* Fix build issues in util package

* Update eslint TS plugins

* Scope react app package name

* Convert cli package back to CJS and dynamically import ESM

* Upgrade ts-node version

* Fix tests

* Setup a relay node and pubsub based discovery (#284)

* Add a script to setup a relay node

* Use pubsub based peer discovery

* Add peer multiaddr to connection log

* Catch relay node dial errors

* Increase discovery interval and dial all mutiaddr

* Add UI to display self peer ID and multiaddrs

* Add UI for displaying live remote connections

* Send js objects in peer broadcast messages

* Add react-peer package for using peer in react app

* Reduce disconnect frequency between peers (#287)

* Restrict number of max concurrent dials per peer

* Increase hop relay timeout to 1 day

* Self review changes

* Review changes

* Increase pubsub discovery interval and add steps to create a peer id  (#290)

* Increase pubsub discovery interval

* Disable autodial to avoid peer dials without protocol on a reconnect

* Add steps to create a peer id and use for relay node

* Add back dependency to run signalling server

* Avoid bootstrapping and dial to relay node directly

Co-authored-by: prathamesh0 <42446521+prathamesh0@users.noreply.github.com>
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2023-01-10 20:10:27 +05:30
aa4a954330
Use graph-ts and graph-cli packages from gitea (#283)
* Use published packages from gitea

* Remove token used for github packages
2022-12-29 16:59:42 +05:30
9566dff466
Fix job-runner to process missing parent block (#274)
* Fix job-runner to process missing parent block

* Upgrade package versions to 0.2.18
2022-12-13 11:59:55 +05:30
cc66f44bab
Use assemblyscript from gitea packages (#272)
* Use assemblyscript from gitea packages

* Add back github registry for vulcanize/graph-ts

* Use cerc-io scoped assemblyscript package
2022-12-07 15:13:41 +05:30
c5908c4c0e
Upgrade package versions to 0.2.17 (#269) 2022-11-28 19:10:36 +05:30
prathamesh0
2417e3feb1
Fix linter warnings and dependencies (#264)
* Fix minor linting errors

* Fix CLI getters linting errors

* Fix resolvers template in codegen

* Fix dependencies
2022-11-25 20:03:58 +05:30
e47aab2ed7
Upgrade package versions to 0.2.16 (#263) 2022-11-25 16:09:35 +05:30
b66dcb4af9
Refactor graph-node database and move to util (#259)
* Move graph-database from graph-node to util

* Refactor and remove graph-node dependency from cli package

* Modify dependencies using depcheck

* Implement CLI refactoring changes in other watchers

* Review changes to remove eden comment and fix local import in util

* Import GraphDatabase from util instead of graph-node

* Move graph-node non assemblyscript code to util package

* Implement CLI refactoring changes in codegen

* Fix graph-node tests after refactoring

* Move fromStateEntityValues to graph state utils
2022-11-25 15:54:35 +05:30
prathamesh0
63a2c5804e
Accommodate CLI refactoring changes to codegen (#261)
* Use watch-contract CLI from cli package in codegen

* Use reset CLIs from cli package in codegen

* Use create checkpoint CLI from cli package in codegen

* Use inspect-cid CLI from cli package in codegen

* Use import state CLI from cli package in codegen

* Use export-state and verify-checkpoint CLI from cli package in codegen

* Use server CLI from cli package in codegen

* Use job-runner CLI from cli package in codegen

* Use index-block CLI from cli package in codegen

* Use fill CLI from cli package in codegen

* Minor codegen fixes

* Update watcher instructions to run job-runner before server
2022-11-25 11:31:20 +05:30
prathamesh0
aba0c665f3
Refactor fill and fill-state CLIs to cli package (#257)
* Refactor fill CLI to cli package

* Refactor method to fill-state to graph-node

* Refactor fill-state CLI to cli package

* Move subgraph state utils to a separate file

* Refactor subgraph state helper methods to graph-node

* Update mock indexer

* Move watcher job-runner to util

* Remove mock server and data from erc20-watcher

* Import watcher job-runner from util
2022-11-24 15:28:38 +05:30
b08fcebe54
Upgrade package versions (#256) 2022-11-23 18:23:26 +05:30
cc28474537
Accommodate GQL optimizations in codegen (#254)
* Prune optional methods in indexer and database interfaces

* Implement GQL optimization changes in codegen

* Fix graph-node test indexer

* Add demos to codegen package
2022-11-23 17:42:25 +05:30
prathamesh0
122a64c2f9
Refactor export-state and verify checkpoint CLIs to cli package (#252)
* Refactor export-state CLI to cli package

* Refactor verify checkpoint CLI to cli package

* Update mock indexer object
2022-11-22 18:27:49 +05:30
prathamesh0
6737ec756c
Refactor import-state CLI to cli package (#250)
* Remove unnecessary upstream config arg to event watcher

* Initialize event watcher in CLI package

* Refactor import-state CLI to cli package
2022-11-22 16:41:15 +05:30
prathamesh0
570640d4bc
Refactor inspect-cid CLI to cli package (#248) 2022-11-22 11:45:45 +05:30
prathamesh0
03050e5323
Refactor create checkpoint CLI to cli package (#247)
* Refactor create checkpoint CLI to cli package

* Update mock indexer
2022-11-22 10:38:23 +05:30
prathamesh0
1e639c1af3
Refactor reset CLIs to cli package (#246)
* Refactor reset CLIs to cli package

* Use reset CLIs from cli package in watchers

* Fix method to canonicalize latest entities

* Use composition with common code for refactored CLIs
2022-11-21 15:14:10 +04:00
prathamesh0
7520e9012c
Refactor watch-contract CLI to cli package (#245)
* Refactor watch-contract CLI to cli package

* Use watch-contract from cli in eden and erc721 watchers

* Use watch-contract from cli in all watchers

* Pass an object containing clients to refactored CLI

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-11-21 11:47:23 +04:00
a0ba657d17
Rename variables and fix comments (#243)
* Rename entity to entityType

* Rename variables and fix comments
2022-11-18 19:16:42 +05:30
prathamesh0
f3c65cbd64
Refactor util code to be reused (#241)
* Ignore watch contract jobs in event processing complete handler

* Update job-queue config and handle errors on job completion hook

* Update graph decimal implementation

* Return generic type from method to read watcher config

* Export fill prefetch batch size default value
2022-11-18 16:29:06 +05:30
635b26aa0c
Use entity is_pruned flag in queries (#238) 2022-11-17 13:55:53 +05:30
79e903b396
Reset latest and frothy entity tables on watcher reset (#236) 2022-11-17 10:14:59 +05:30
693c23d192
Update latest entity tables on chain pruning (#235) 2022-11-16 20:16:48 +05:30
74741184ee
Use lateral query for time travel queries using latest entity tables (#233)
* Avoid joins in id only queries

* Use lateral query for time travel queries using latest entity tables
2022-11-16 18:57:04 +05:30
a084b4e40c
Latest entity queries to optimize frontend app GQL requests (#232)
* Add latest entity table query pattern

* Add subscriber config to typeorm
2022-11-16 18:11:27 +05:30
prathamesh0
408a3927c0
Add a table for entites in frothy region for subgraph watchers (#231)
* Add a table for entites in frothy region and update it in a subscriber

* Accommodate changes to other watchers and codegen
2022-11-16 17:12:54 +05:30
7e5974ccf7
Denormalize is_pruned flag in eden-watcher (#230)
* Add is_pruned flag to all entities

* Mark entities as pruned once the corresponding block gets pruned
2022-11-16 15:30:40 +05:30
prathamesh0
a52bdf64b1
Add config for GQL requests cache (#228)
* Add config and a method to set cache control settings for GQL requests

* Add option for max size in GQL cache config

* Fix failing tests
2022-11-15 14:56:08 +05:30
6f8ededd52
Add flag to load relations sequentially or concurrently (#225)
* Add flag to load relations sequentially or concurrently

* Fix database init in graph-node test

* Fix graph-node watcher query method and add changes to codegen
2022-11-14 14:23:46 +05:30
prathamesh0
861420a10f
Add a flag to enable state creation (#222)
* Add a flag to enable state creation

* Update flag in mock config for testing
2022-11-10 16:56:06 +05:30
prathamesh0
8d3c68873b
Avoid creating a separate base db instance for graph database (#221)
* Avoid creating a separate base db instance for graph database

* Fix failing test file build

* Use getter method to get base database
2022-11-10 14:51:18 +05:30
ecd8b2474a
Remove eventName filter in events count query (#220) 2022-11-10 12:09:17 +05:30
772fdbec25
Add unique entities query without checking pruned block (#218) 2022-11-09 13:31:26 +05:30
7a4b459b1c
Add DB queries without checking pruned block (#217) 2022-11-08 17:01:35 +05:30
prathamesh0
266442a6d9
Update getFullBlocks usage (#213)
* Update getFullBlocks usage to add block number arg

* Add skipStateFieldsUpdate config for uniswap-watcher

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-11-03 16:49:41 +05:30
be65ca45ea
Load relations sequentially (#212) 2022-11-03 14:32:34 +05:30
prathamesh0
2517f110ea
Update GQL and getBlock API usage (#210)
* Update getLogs and getBlockWithTransactions usage to add block number arg

* Update getFullTransaction usage to add block number arg
2022-11-03 14:14:58 +05:30
52c42f4e84
Reset watcher to previous indexed block on start (#207)
* Reset watcher to previous indexed block before start

* Implement changes in other watchers

* Save successfully fetched blocks and events to prefetch cache

* Add unique query for transaction table

* Check db for blocks before fetching from eth-server

* Show all mismatches at a block

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2022-11-03 14:01:10 +05:30
prathamesh0
306bbb73ca
Use prefetching of blocks with events in watchers and codegen (#206)
* Avoid refetching block while fetching events

* Prefetch a batch of blocks with events while indexing

* Update mock indexer used in graph-node testing

* Process available blocks while prefetching

* Refactor events fetching to a method in util

* Move method to get GQL event query result to util
2022-10-20 18:46:56 +05:30
668875b3a0
Refactor code in graph-node to use in uniswap watcher (#205)
* Refactor code in graph-node to use in uniswap

* Move over database methods to fetch entities for GQL query
2022-10-20 14:39:32 +05:30
prathamesh0
5af90bd388
Refactor state creation code (#204)
* Remove support for pushing state to IPFS

* Move job handlers for state creation to util

* Rename state creation related methods and objects

* Update mock indexer used in graph-node testing

* Fetch and merge diffs in batches while creating a state checkpoint

* Fix timing logs while for state checkpoint creation

* Refactor method to get state query result to util

* Accept contracts for state verification in compare CLI config

* Make method to update state status map synchronous
2022-10-19 15:24:14 +05:30
ce182bce85
Clear cache of latest entities on event processing error (#201)
* Clear cache of latest entities on event processing error

* Remove lighthouse-watcher and update ethersjs version

* Handle GraphDecimal type in state entity

* Add option for comparing all entities using paginate

* Clear pruned cached entities at intervals

* Move ipld-demo to graph-node package and remove reset-dbs script

* Implement changes in all watchers and codegen
2022-10-19 14:26:10 +05:30
prathamesh0
45111e68ac
Add an option to run compare CLI in batched intervals (#202) 2022-10-17 12:07:21 +05:30
521f6ed3fa
v0.2.13 (#197) 2022-10-11 16:12:06 +05:30
e1aef1a7e9
Changes to use packages in uniswap-watcher (#196)
* Changes to use util from uniswap-watcher

* Refactor ResultIPLDBlock to util

* Verify state in compare CLI for uniswap multiple queries

* Prepare watcher-ts for publishing packages

* Fix verify state in compare CLI for multiple entities query

* Fix codegen util imports
2022-10-11 13:41:26 +05:30
18861eaf79
Implement cache for latest updated entities to be used in mapping code (#194)
* Load relations according to GQL query

* Implement cache for latest entities to used in mapping code

* Add metrics for cache hit and fix caching pruned entities

* Changes in codegen and graph-test-watcher

* Remove entity load counter reset to zero
2022-10-04 13:31:29 +05:30