Commit Graph

102 Commits

Author SHA1 Message Date
prathamesh0
48e7785367
Upgrade nitro-node package (#415)
* Upgrade nitro-node package

* Update package versions
2023-09-21 18:32:03 +05:30
77312a2f95
Add yamux stream multiplexer in peer package (#413)
* Add yamux muxer to peer

* Upgrade version to 0.2.58

---------

Co-authored-by: neeraj <neeraj.rtly@gmail.com>
2023-09-15 12:19:51 +05:30
prathamesh0
bd73dae1b1
Implement consensus mechanism using mokka (#412)
* Add Consensus class with placeholders

* Implement Consensus constructor

* Move PubsubType to peer package

* Implement remaining methods in Consensus class

* Use existing consensus stream if it exists

* Setup send and receive pipes on consensus streams

* Refactor P2P and consensus setup in server command

* Add Nitro node initialization in server command

* Return objects from server initializations

* Use dynamic imports for ES modules in util package

* Fix util deps

* Change target to es6 to allow creating a Mokka instance

* Set moduleResolution to node16 in util for dynamic imports

* Upgrade @cerc-io/nitro-node package

* Implement retries while sending consensus messages

* Use bunyan for consensus logs and subscribe to state changes

* Use debug for logging state change events

* Handle empty watcher party file path

* Return object from initP2P

* Upgrade @cerc-io/nitro-node package

* Update package versions
2023-09-14 13:47:31 +05:30
prathamesh0
c80e4d0ab7
Add support for gossipsub (#410)
* Replace floodsub with gossipsub

* Add a pubsub flag to support both floodsub and gossipsub

* Fix deps

* Handle empty string values

* Upgrade @cerc-io/libp2p dependency

* Upgrade package versions
2023-09-05 17:54:52 +05:30
1d3bbf89c8
Update graph-cli version to fix concat-stream dependency issue (#405)
* Update yarn lock

* Upgrade @cerc-io/graph-cli version to 0.22.4-watcher-ts-0.1.2

---------

Co-authored-by: neeraj <neeraj.rtly@gmail.com>
2023-08-25 14:26:00 +05:30
444901eb87
Upgrade package versions to 0.2.54 (#403)
Co-authored-by: neeraj <neeraj.rtly@gmail.com>
2023-08-16 19:07:30 +05:30
prathamesh0
198d3e65db
Add payments config and charge according to configured rates (#395)
* Track amount paid on payment channels and charge as per query cost

* Add payments manager config

* Fix received payment checks

* Separate out payment not received and amount insufficient errors

* Serve a query for free if its rate is not configured

* Load open payment channels from the stored state

* Take list of free queries from config

* Take mutation rates from config

* Rename query rates config fields

* Take cache settings and request timeout from config

* Upgrade ts-nitro packages

* Upgrade package versions
2023-07-27 15:52:36 +05:30
prathamesh0
aa10a2dc05
Cleanup dead connections in peer heartbeat checker (#394)
* Cleanup dead connections in peer heartbeat checker

* Remove yamux muxer
2023-07-26 13:37:16 +05:30
prathamesh0
6731a640a2
Use a single payment header in payments plugin (#393)
* Add client address field to payments manager

* Use a single payment header in payments plugin

* Upgrade ts-nitro packages

* Rename method to authenticateVoucherForSender

* Upgrade package versions

---------

Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
2023-07-25 13:34:02 +05:30
prathamesh0
eec1965980
Add a payments plugin to GQL server (#391)
* Add a custom payments plugin in server process

* Fix voucher channel read in voucher subscription

* Use voucher hash and signature for payment authentication

* Use a LRU hash map for storing received vouchers

* Avoid hoisting level types to resolve build errors

* Upgrade ts-nitro packages

* Add payments related logs

* Upgrade package versions

* Always allow latestBlock query

* Upgrade ts-nitro packages
2023-07-20 17:22:48 +05:30
1b6c2a486a
Add peer config options for transports and peerDiscovery (#387)
* Add peer config options for transports and peerDiscovery

* Add yamux muxer to peer libp2p
2023-06-28 11:25:03 +05:30
prathamesh0
c5cbe33059
Pin typeorm version to avoid discrepancies with generated watchers (#385) 2023-05-15 17:55:36 +05:30
7f91fa7ed8
Remove watchers after moving them out to their own repos (#365)
* Remove all watchers after moving them out

* Add missing dependencies
2023-04-14 17:22:44 +05:30
3f2dc56261
Fix json-bigint package version in yarn.lock file (#361)
* Fix json-bigint package version in lock file

* Upgrade package versions
2023-04-13 17:38:50 +05:30
25df38ace2
Add log for handling muliple returned values (#359)
Co-authored-by: Dhruv Srivastava <dhruvdhs.ds@gmail.com>
2023-04-12 14:46:43 +05:30
prathamesh0
91816db6f3
Upgrade js-libp2p (#346)
* Upgrade js-libp2p

* Upgrade package version
2023-03-29 09:37:06 +05:30
9e53b1a498
Upgrade dependency versions to remove vulnerabilities from dependabot (#343)
* Dependency patch updates

* Minor update @apollo/client 3.3.19 -> 3.7.10

* Minor update @nomiclabs/hardhat-ethers 2.0.2 -> 2.2.2

* Minor update @openzeppelin/contracts 4.3.2 -> 4.8.2

* Minor update @types/chai 4.2.19 -> 4.3.4

* Minor update decimal 10.3.1 -> 10.4.3

* Minor update @graphql-tools/utils 9.1.1 -> 9.2.1

* Minor update @graphql, graphql-ws, graphql-request

* Minor update yargs

* Minor update @types/node

* Minor update graph-ts

* Minor updates graphql-tools, multiaddr

* Update dependencies with minor updates

* Minor updates to eslint and related dependencies

* Update solidity compiler version for erc721-watcher

* Fix ESLint indent rule

---------

Co-authored-by: Dhruv Srivastava <dhruvdhs.ds@gmail.com>
2023-03-28 13:17:53 +05:30
ee63e33cc5
Upgrade Typescript and ESLint (#337) (#342)
* Upgrade eslint and additional lint changes

* Upgrade typescript version to v5

---------

Co-authored-by: Dhruv Srivastava <dhruvdhs.ds@gmail.com>
2023-03-24 10:31:33 +04:00
prathamesh0
0f239c360e
Use websockets transport with WebRTC signalling (#341)
* Use websockets as the base transport

* Update yarn lock file

* Upgrade cerc-io/js-libp2p

* Update package version
2023-03-23 16:58:11 +05:30
prathamesh0
787991c432
Add a basic test to demonstrate a p2p connection (#336)
* Add a test to demonstrate a p2p connection

* Use debug for logs

* Perform peer intialization in a separate test
2023-03-09 17:36:27 +05:30
90d60f54a6
Improve connection check with peers by retrying ping on failures (#330)
* Improve connection check with peers by retrying ping on failures

* Use DEFAULT_PING_INTERVAL in cli

* Log only error message

* Add config option to pass peer id file path

* Finish retrying pings before connection check interval duration

* Handle duplicate connections to relay nodes

* Increase default max dial retries from one relay to another

* Update connection manager config for relay nodes

* Use debug for logs in relay node

---------

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2023-02-27 12:47:49 +05:30
054600ccc4
Use browser metrics package in peer (#322)
* Use browser metrics package

* Get latency from heartbeat check

* Use metrics in nodejs CLI

* Avoid disconnect from primary relay on reaching limit

* Fix relay connections count
2023-02-20 11:19:57 +05:30
prathamesh0
83ad5d80a7
Add a method to get a deterministic pseudonym for given peer id (#326) 2023-02-20 09:45:18 +05:30
prathamesh0
e47f864966
Add an option to use a domain name for a announce address (#320)
* Add an option to use a domain name for a announce address

* Update chat CLI readme

* Set port range for relay node

* Add descriptive comment

* Upgrade webrtc-direct version
2023-02-15 18:09:21 +05:30
prathamesh0
582a8e10e3
Pass node type in libp2p transport config (#316) 2023-02-09 19:53:49 +05:30
40fd5f8ab8
Add heartbeat checks in relay node to hangup disconnected peers (#314)
* Close connections from relay node to disconnected peers

* Review changes and update libp2p version

* Update package version
2023-02-07 20:04:41 +08:00
prathamesh0
a1af962d99 Use webrtc-direct protocol with peer signalling integrated into relay node (#307)
* Use webrtc-direct transport with pubsub based discovery

* Use changes to integrate signalling using relay node

* Add an indicator for peer connection type (direct/relayed)

* Catch errors while creting a libp2p node

* Catch errors while dialling peers

* Catch errors when dialling from relay node

* Pass CLI arg for port to listen on for relay node

* Subscribe chat CLI to pubsub topic

* Update yarn lockfile

* Update webrtc-direct dependency in package json

* Update webrtc-direct version

* Update yarn lockfile
2023-02-02 19:50:27 +05:30
28023d834a
Remove peer-test-app react app (#312) 2023-02-02 12:39:55 +05:30
30dd2521e3
Fix yarn lock file libp2p package (#310) 2023-02-01 16:16:40 +05:30
dfc3f5a63e
Move out react-peer package to its own repo (#308) 2023-02-01 09:54:52 +05:30
prathamesh0
ff64dcb43c
Upgrade js-libp2p and related dependencies (#298)
* Upgrade js-libp2p and related dependencies

* Dial all multiaddr of a discovered peer in parallel

* Dial using peer id instead of dialling individual multiaddr

* Log peer id on discovery
2023-01-20 10:34:55 +05:30
cd29b47ecc
Implement peer package to send messages between peers (#279)
* Initial implementation of class and discoving other peers

* Implement peer connection and sending messages between peers

* Add react app and use peer package for broadcasting

* Maintain stream for each remote peer

* Refactor code in peer package

* Add serve package for react-app

* Add readme for running react app

* Add peer package readme

* Add logs for events with details

* Add a chat CLI using peer package (#280)

* Add a flag to instantiate Peer for nodejs

* Add a basic chat CLI using peer

* Add a signal server arg to chat CLI

* Add instructions for chat CLI

* Fix typescript and ESM issues after adding peer package (#282)

* Fix build issues in util package

* Update eslint TS plugins

* Scope react app package name

* Convert cli package back to CJS and dynamically import ESM

* Upgrade ts-node version

* Fix tests

* Setup a relay node and pubsub based discovery (#284)

* Add a script to setup a relay node

* Use pubsub based peer discovery

* Add peer multiaddr to connection log

* Catch relay node dial errors

* Increase discovery interval and dial all mutiaddr

* Add UI to display self peer ID and multiaddrs

* Add UI for displaying live remote connections

* Send js objects in peer broadcast messages

* Add react-peer package for using peer in react app

* Reduce disconnect frequency between peers (#287)

* Restrict number of max concurrent dials per peer

* Increase hop relay timeout to 1 day

* Self review changes

* Review changes

* Increase pubsub discovery interval and add steps to create a peer id  (#290)

* Increase pubsub discovery interval

* Disable autodial to avoid peer dials without protocol on a reconnect

* Add steps to create a peer id and use for relay node

* Add back dependency to run signalling server

* Avoid bootstrapping and dial to relay node directly

Co-authored-by: prathamesh0 <42446521+prathamesh0@users.noreply.github.com>
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2023-01-10 20:10:27 +05:30
cc66f44bab
Use assemblyscript from gitea packages (#272)
* Use assemblyscript from gitea packages

* Add back github registry for vulcanize/graph-ts

* Use cerc-io scoped assemblyscript package
2022-12-07 15:13:41 +05:30
05bfd57278
Add isPruned flag to erc721-watcher custom transferCount entity (#266)
* Remove block number param from job-queue reset CLI in readme

* Add isPruned flag to erc721-watcher custom transferCount entity
2022-11-28 14:14:02 +05:30
prathamesh0
2417e3feb1
Fix linter warnings and dependencies (#264)
* Fix minor linting errors

* Fix CLI getters linting errors

* Fix resolvers template in codegen

* Fix dependencies
2022-11-25 20:03:58 +05:30
prathamesh0
63a2c5804e
Accommodate CLI refactoring changes to codegen (#261)
* Use watch-contract CLI from cli package in codegen

* Use reset CLIs from cli package in codegen

* Use create checkpoint CLI from cli package in codegen

* Use inspect-cid CLI from cli package in codegen

* Use import state CLI from cli package in codegen

* Use export-state and verify-checkpoint CLI from cli package in codegen

* Use server CLI from cli package in codegen

* Use job-runner CLI from cli package in codegen

* Use index-block CLI from cli package in codegen

* Use fill CLI from cli package in codegen

* Minor codegen fixes

* Update watcher instructions to run job-runner before server
2022-11-25 11:31:20 +05:30
ace52d9da3
Add GQL metrics to ERC20 and ERC721 watchers (#251)
* Add changes in erc20-watcher

* Add changes in erc721-watcher

* Add changes in mobymask-watcher

* Add reset state CLI to all watchers
2022-11-22 17:04:38 +05:30
prathamesh0
7520e9012c
Refactor watch-contract CLI to cli package (#245)
* Refactor watch-contract CLI to cli package

* Use watch-contract from cli in eden and erc721 watchers

* Use watch-contract from cli in all watchers

* Pass an object containing clients to refactored CLI

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2022-11-21 11:47:23 +04:00
prathamesh0
92fd3cac03
Upgrade client libraries (#239) 2022-11-17 19:06:07 +05:30
prathamesh0
f53371e17b
Accomodate GQL requests caching in code generator (#237)
* Accomodate GQL requests caching in code generator

* Add GQL API request queuing
2022-11-17 12:02:08 +05:30
prathamesh0
1ad223db4d
Add GQL requests caching in eden-watcher (#234) 2022-11-16 20:01:18 +05:30
prathamesh0
62c57d8005
Upgrade Apollo server to v3 (#229)
* Upgrade apollo server to v3 in eden-watcher

* Upgrade apollo server to v3 in other watchers and codegen

* Refactor common server code to util

* Pass express application object in the method to create server

* Remove unnecessary import
2022-11-16 11:14:54 +05:30
ce182bce85
Clear cache of latest entities on event processing error (#201)
* Clear cache of latest entities on event processing error

* Remove lighthouse-watcher and update ethersjs version

* Handle GraphDecimal type in state entity

* Add option for comparing all entities using paginate

* Clear pruned cached entities at intervals

* Move ipld-demo to graph-node package and remove reset-dbs script

* Implement changes in all watchers and codegen
2022-10-19 14:26:10 +05:30
a8fdcca866
Update watcher import CLI to create entities from checkpoint (#186)
* Import entities from checkpoint

* Fix IPLD state when updating subgraph entity

* Changes in codegen and other watchers

* Update IPLD state with all Block entities

* Add verify and create sub commands to checkpoint

* Add option for specifying snapshot block in export state CLI
2022-09-22 15:26:06 +05:30
8af7417df6
Implement query for multiple entities and nested relation fields in eden-watcher (#166)
* Implement query for multiple entities in eden-watcher

* Implement nested relation queries

* Implement GQL query params first, skip, orderBy, orderDirection

* Add blockNumber index to subgraph entities

* Add logs for timing eth-calls and storage calls

* Add prometheus metrics to monitor GQL queries

* Fix default limit and order of 1-N related field in GQL entitiy query

* Add timer logs for block processing

* Run transpiled js in all watchers

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
2022-09-01 14:17:43 +05:30
7238f614c0
Update CLI to compare only updated entities and verify IPLD state (#161)
* Change compare CLI to verify only updated entities

* Implement IPLD state verification in compare CLI

* Changes to IPLD state to match with GQL result entity
2022-08-17 19:11:40 +05:30
prathamesh0
80682e2755
Remove Uniswap watchers and related artifacts (#163)
* Remove Uniswap watchers and related code

* Remove Uniswap artifacts from graph-node

* Remove Uniswap artifacts from graph-test-watcher
2022-08-17 14:04:20 +05:30
1a903fccc6
CLI to compare eden-watcher entities and fix mapping code (#149)
* Make sumStaked variable local in eden network mapping

* Implement compare CLI to fetch and query by ids

* Set filterLogs to true for eden-watcher

* Use varchar for bigint array type in eden-watcher

* Store subgraph entities by id in IPLD state

* Store bigint vales as string in IPLD state

* Update eden watcher hook to store single Block entity in IPLD checkpoint

* Fix entity enum type property

* Fix parsing big numbers in event params

* Fix event bigint params parsing in all watchers

* Set default limit to query result and process block after events
2022-08-08 15:58:13 +05:30
a15305450c
Add prometheus metrics support for watchers (#152)
* Add prometheus metrics endpoint in watcher

* Add event, sync status and DB size metrics

* Fix subgraph watchers DB entities directory path

* Make watcher metrics optional using config
2022-08-03 16:26:51 +05:30
3cee10607e
Parse batched transactions for mobymask-watcher events (#141)
* Remove using graph-node from mobymask-watcher

* Parse batched transactions for MobyMask events

* Update to use same ethers package version

* Add CLI for processing block out of order

* Fix job-runner for already processed blocks out of order
2022-07-11 11:29:33 +05:30