* Fix duplicate historical processing jobs created on template create
* Fix switch from historical to realtime processing when template create block is near head
* Keep return value for singular subgraph entity queries nullable
* Prioritize block hash over number in time travel queries
* Throw error when block with given hash doesn't exist in db
* Set gzip true in ethersjs provider
* Add timer logs and use StaticJsonRpcProvider
* Fetch block data in historical processing and cache in map
* Fetch txs required for event logs in historical processing
* Process events with prefetched block and txs data in realtime processing
* Clear old TODOs
* Reset to latest processed block on restarting job-runner
* Update sync status during historical processing in job-runner
* Codegen changes
* Use sync status latest processed block for subgraph _meta GQL query
* Set job per interval for subscribing events queue to 1
* Fix events processing skipped for blocks after template create
* Move method to apply block filter to base database
* Handle nested entity sorting
* Handle nested entity sorting for distinct on type queries
* Handle nested entity sorting for all query types
* Add a method to get meta data for watcher indexing status
* Add a flag indicating indexing error to sync status
* Codegen changes
* Clear indexing error on job-runner startup
* Fix lint errors
* Wait for events queue to be empty before continuing historical processing
* Make historicalLogsBlockRange and historicalMaxFetchAhead configurable
* Perform single RPC request for multiple addresses
* Handle logical operators in where input
* Handle logical filter operators while building the database query
* Refactor code to build where clause for database query
* Handle subgraph template create when filterLogsByAddresses set to true
* Fix historical processing stopping after running for multiple batches
* Add new method in graph-node test dummy indexer
* Handle nested filters for array type fields
* Transform Decimal param values to strings for db queries
* Handle nested filters for derived fields
* Handle filters for array type fields
* Handle nested filters while building the database query
* Build nested filters for relational fields
* Avoid queries to get canonical block hashes for loading relations
* Handle nested filters in all query types
* Give precedence to block hash over number in time travel queries
* Fix async block size caching for missing blocks in historical processing
* Start historical block processing only if filter logs is set to true
* Fetch filtered logs by topics and save required blocks
* Fix realtime processing start block after historical processing
* Avoid publishing events and blocks in historical processing
* Add new method to graph-node test indexer
* Get full block data for subgraph block handler only if configured
* Add useBlockRanges flag for switching between historical and realtime processing
* Fix bigint values transformation
* Fix starts and ends filter operator resolution
* Support case insensitive filters for string fields
* Add support for global filter _change_block
* Handle _change_block filter in all query types
* Add TODOs for historical blocks processing
* Add new job for historical blocks processing
* Handle historical job completion
* Fetch latest block in chain and start historical block processing
* Fix starting realtime block processing from latest canonical block
* Refactor historical block processing method and add logs
* Add dummy indexer methods in graph-node to pass test
* Changes in codegen for historical processing in generated watcher
* Support filtering by topics when fetching logs
* Include event signatures from all contracts in topics list
* Refactor common code
* Update package versions
* Skip log index order in events processing
* Use json-bigint stringify in processBatchEvents
* Set eventSignature in event processing for contracts watched later
* Check for null block in async caching of block sizes
* Handle null blocks in chain pruning
* Skip FEVM null blocks while prefetching
* Fix BaseFee assignment while fetching block in rpc-eth-client
* Convert blockNumber field to number type before saving block to db
* Avoid saving blocks after null block twice
* Update subgraph readme to run fill before job-runner
* Fix getContractEntitiesMap incase of template data sources
* Use rpcSupportsBlockHashParam flag to use blockNumber for rpc client eth_call
* Fix optional baseFeePerGas in rpc-eth-client
* Fix graph-node tests after changes
* Remove completed TODO
* Support fetching logs for a block range in rpc-eth-client
* Add a method to fetch block events for multiple blocks at once
* Add a method to save blocks with fetched events in a block range
* Fix transactions destructuring
* Fix get logs call args
* Add a separate ETH client method to get logs in a block range
* Codegen changes
* Add validation for config file
* Add support for websocket endpoints to be checked for contract deployment
* Add flag to toggle validation
* Update log message for paid rpc method validation
* Update log message for validation
* Rename enableValidation to enableConfigValidation
* Check deployment only if contract address is in a valid format
* Update httpValidateEndpoint method to check if endpoint is up instead of response as ok
* Update log messages
* Update log messages
---------
Co-authored-by: Shreerang Kale <shreerangkale@gmail.com>
* Upgrade @cerc-io/nitro-node dependency
* Add a separate config option for chain URL used by Nitro node
* Avoid adding ETH RPC method to request URL as no longer needed
* Update package versions
* Add config option for upstream RPC mutation endpoint
* Use mutation endpoint as chain URL for Nitro node
* Add a method to setup an upstream payment channel
* Add a method to send payments to upstream Nitro node
* Upgrade package versions
* Add support for setting gossipsub direct peers
* Use string array for direct peers config
* Handle empty nitro and consensus config
* Increase consensus write retry attempts
* Rename config option for watcher party peers file
* Handle directPeers array being passed undefined
* Upgrade package versions
---------
Co-authored-by: Shreerang Kale <shreerangkale@gmail.com>
* Add Consensus class with placeholders
* Implement Consensus constructor
* Move PubsubType to peer package
* Implement remaining methods in Consensus class
* Use existing consensus stream if it exists
* Setup send and receive pipes on consensus streams
* Refactor P2P and consensus setup in server command
* Add Nitro node initialization in server command
* Return objects from server initializations
* Use dynamic imports for ES modules in util package
* Fix util deps
* Change target to es6 to allow creating a Mokka instance
* Set moduleResolution to node16 in util for dynamic imports
* Upgrade @cerc-io/nitro-node package
* Implement retries while sending consensus messages
* Use bunyan for consensus logs and subscribe to state changes
* Use debug for logging state change events
* Handle empty watcher party file path
* Return object from initP2P
* Upgrade @cerc-io/nitro-node package
* Update package versions
* Replace floodsub with gossipsub
* Add a pubsub flag to support both floodsub and gossipsub
* Fix deps
* Handle empty string values
* Upgrade @cerc-io/libp2p dependency
* Upgrade package versions
* Avoid gasPrice while serializing tx in getRawTransaction
* Convert base fee to bigint when fetching block data
* Fix methods to get block data in rpc-eth-client
* Fix error checking in rpc-eth-client
* Continue processig after missing null block in FEVM
* Self review fix
* Upgrade package versions
---------
Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
* Implement rpc-eth-client with getStorageAt method
* Add test for comparing RPC and GQL eth-client getStorageAt method
* Add getBlockWithTransactions and getBlocks method
* Implement getFullBlocks with RLP encoded data
* Implement getFullTransaction method with raw tx
* Implement getBlockByHash and getLogs methods
* Add flag and interface to switch between RPC and GQL eth clients
* Fix getBlocks to return empty array when block not present
* Return empty array in getBlocks for missing block and use blockNumber in getLogs
* Fix getRawTransaction method for zero signature.v value
* Remove duplicate util from rpc-eth-client
* Track amount paid on payment channels and charge as per query cost
* Add payments manager config
* Fix received payment checks
* Separate out payment not received and amount insufficient errors
* Serve a query for free if its rate is not configured
* Load open payment channels from the stored state
* Take list of free queries from config
* Take mutation rates from config
* Rename query rates config fields
* Take cache settings and request timeout from config
* Upgrade ts-nitro packages
* Upgrade package versions
* Add client address field to payments manager
* Use a single payment header in payments plugin
* Upgrade ts-nitro packages
* Rename method to authenticateVoucherForSender
* Upgrade package versions
---------
Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
* Wait for receiving voucher on GQL query
* Add timeout while authenticating voucher and remove listener channel
* Clear timeout after payment received
* Rename variable to paymentListener
* Upgrade package versions
* Add rpcSupportsBlockHashParam flag in config
* Add a custom payments plugin in server process
* Fix voucher channel read in voucher subscription
* Use voucher hash and signature for payment authentication
* Use a LRU hash map for storing received vouchers
* Avoid hoisting level types to resolve build errors
* Upgrade ts-nitro packages
* Add payments related logs
* Upgrade package versions
* Always allow latestBlock query
* Upgrade ts-nitro packages
* Add util method for preparing state from GQL response entity
* Refactor method to create or update state data
* Fix typeorm bigint transformer and convert to checksum contract address
* Skip resetting to previous block in job-runner if isComplete set to true
* Fix creating subgraph event with struct params
* Use quotes for table names in custom queries
* Fix indexer prepareStateEntry method
* Fix toEthereumValue method when used with ethereum.decode
* Add CLI for creating snapshot checkpoint state from GQL endpoint
* Skip import-state if block is already indexed
* Review changes
* Block connections from blacklisted multiaddrs in relay node
* Block connections from blacklisted multiaddrs in peer nodes
* Block dials for blacklisted multiaddrs
* Update package version
* Fix ValueTransformers to check for undefined value
* Create Result types if needed
* Update codegen to include array return type queries in schema
* Update codegen to include array return type queries in resolvers
* Update codegen to include array return type queries in indexers
* Codegen skip db caching methods for queries returning unhandled types
* Codegen handle bigint array in indexer
* Update codegen to handle basic array types
* Remove redundant code and refactor
* Codegen map Solidity types directly to GraphQL types
* Codegen get TS from GQL types
* Codegen handle ethers BigNumber array in indexer template
* Fix GQL type returned in client queries
* Codegen unused method isElementaryType should not be exported
* Codegen update README with solc version in config
* Codegen update README with continue-on-error flag
* Codegen add continue-on-error flag for storage mode
* Codegen generate gitignore and license for watchers
* Codegen refactor gitignore and license generation utils
---------
Co-authored-by: Dhruv Srivastava <dhruvdhs.ds@gmail.com>
* Refactor discovery handler
* Broadcast peer info over floodsub on requests
* Broadcast peer info from relay nodes
* Make debug reponse handler optional
* Register debug info request handler on peer init
* Move debug info types to types dir
* Return method to unsubscribe from the debug topic
* Make debug info flag optional for relay nodes
* Restructure peer connection info data
* Refactor getting peer info to be used in react app
* Refactor duplicate code to utils
* Rename peer methods
---------
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
* Improve connection check with peers by retrying ping on failures
* Use DEFAULT_PING_INTERVAL in cli
* Log only error message
* Add config option to pass peer id file path
* Finish retrying pings before connection check interval duration
* Handle duplicate connections to relay nodes
* Increase default max dial retries from one relay to another
* Update connection manager config for relay nodes
* Use debug for logs in relay node
---------
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
* Pass initialization options when starting a peer
* Update config used for relay and peer nodes in watcher
* Rename types
* Refactor mobymask libp2p message parsing
* Enable laconic debug logs in server command
* Add an option to run relay node with watcher server
* Add a v2 mobymask-watcher
* Add an option to run peer node with watcher server
* Ignore ts error when importing types from peer package
* Rename mobymask v2 watcher
* Parse mobymask libp2p messages
* Refactor and add comments
* Add a note in peer package about nodejs exports
* Update copyright
* Initial implementation of class and discoving other peers
* Implement peer connection and sending messages between peers
* Add react app and use peer package for broadcasting
* Maintain stream for each remote peer
* Refactor code in peer package
* Add serve package for react-app
* Add readme for running react app
* Add peer package readme
* Add logs for events with details
* Add a chat CLI using peer package (#280)
* Add a flag to instantiate Peer for nodejs
* Add a basic chat CLI using peer
* Add a signal server arg to chat CLI
* Add instructions for chat CLI
* Fix typescript and ESM issues after adding peer package (#282)
* Fix build issues in util package
* Update eslint TS plugins
* Scope react app package name
* Convert cli package back to CJS and dynamically import ESM
* Upgrade ts-node version
* Fix tests
* Setup a relay node and pubsub based discovery (#284)
* Add a script to setup a relay node
* Use pubsub based peer discovery
* Add peer multiaddr to connection log
* Catch relay node dial errors
* Increase discovery interval and dial all mutiaddr
* Add UI to display self peer ID and multiaddrs
* Add UI for displaying live remote connections
* Send js objects in peer broadcast messages
* Add react-peer package for using peer in react app
* Reduce disconnect frequency between peers (#287)
* Restrict number of max concurrent dials per peer
* Increase hop relay timeout to 1 day
* Self review changes
* Review changes
* Increase pubsub discovery interval and add steps to create a peer id (#290)
* Increase pubsub discovery interval
* Disable autodial to avoid peer dials without protocol on a reconnect
* Add steps to create a peer id and use for relay node
* Add back dependency to run signalling server
* Avoid bootstrapping and dial to relay node directly
Co-authored-by: prathamesh0 <42446521+prathamesh0@users.noreply.github.com>
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
* Move graph-database from graph-node to util
* Refactor and remove graph-node dependency from cli package
* Modify dependencies using depcheck
* Implement CLI refactoring changes in other watchers
* Review changes to remove eden comment and fix local import in util
* Import GraphDatabase from util instead of graph-node
* Move graph-node non assemblyscript code to util package
* Implement CLI refactoring changes in codegen
* Fix graph-node tests after refactoring
* Move fromStateEntityValues to graph state utils
* Refactor fill CLI to cli package
* Refactor method to fill-state to graph-node
* Refactor fill-state CLI to cli package
* Move subgraph state utils to a separate file
* Refactor subgraph state helper methods to graph-node
* Update mock indexer
* Move watcher job-runner to util
* Remove mock server and data from erc20-watcher
* Import watcher job-runner from util
* Refactor reset CLIs to cli package
* Use reset CLIs from cli package in watchers
* Fix method to canonicalize latest entities
* Use composition with common code for refactored CLIs
* Refactor watch-contract CLI to cli package
* Use watch-contract from cli in eden and erc721 watchers
* Use watch-contract from cli in all watchers
* Pass an object containing clients to refactored CLI
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
* Ignore watch contract jobs in event processing complete handler
* Update job-queue config and handle errors on job completion hook
* Update graph decimal implementation
* Return generic type from method to read watcher config
* Export fill prefetch batch size default value
* Upgrade apollo server to v3 in eden-watcher
* Upgrade apollo server to v3 in other watchers and codegen
* Refactor common server code to util
* Pass express application object in the method to create server
* Remove unnecessary import
* Add flag to load relations sequentially or concurrently
* Fix database init in graph-node test
* Fix graph-node watcher query method and add changes to codegen
* Reset watcher to previous indexed block before start
* Implement changes in other watchers
* Save successfully fetched blocks and events to prefetch cache
* Add unique query for transaction table
* Check db for blocks before fetching from eth-server
* Show all mismatches at a block
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
* Avoid refetching block while fetching events
* Prefetch a batch of blocks with events while indexing
* Update mock indexer used in graph-node testing
* Process available blocks while prefetching
* Refactor events fetching to a method in util
* Move method to get GQL event query result to util
