* Add export-watcher CLI generation
* Export snapshot block and add import-watcher CLI gen
* Update codegen docs with export and import commands
* Use different function to get latest checkpoint in export CLI
* Update generated watcher docs with import state instructions
* Push IPLDBlocks to IPFS
* Change ts type for data column to Buffer
* Add checkpoint hook
* Push CLI checkpoint to IPFS
* Fix latest IPLDBlock query to filter out irrelevant blocks
* Add CLI to inspect a CID
* Decode data before exporting
* Encode data in JSON for getState GQL API
* Use go-ipfs and export cbor-encoded buffer
* Remove empty post-block methods from existing watchers
* Call hooks and checkpoint job complete handlers in fill
* Remove subscription to hooks queue from existing watchers
* Subscribe to hooks queue in existing watchers
* Change naming strategy for generated get and save functions
* Push checkpointing job after post-block hook job completed
* Using hooks status to ensure their chronological execution
* Add default indices to IPLDBlock table
* Add kind parameter to getState GQL API
* Add checkpoint CLI
* Add blockHash arg to checkpoint CLI and update codegen docs
* Print out block hash for checkpoint CLI
* Use log from debug for logging
* Filter using contract at start in hierarchical query
* Make kind argument to prepare IPLDBlock required
* Add ipld-blocks entity generation
* Populate ipld-blocks table
* Rename ipld-block entity and update after each event
* Move ipld-hook to hooks.ts
* Change IPLD block structure
* Add cid field in blocks
* Fetch prev. IPLDBlock for a contract
* GQL API to query IPLDBlock by CID
* Save cid in blocks in existing watchers
* Update codegen docs
* GQL API for getting last derived state (#3)
* GQL API for getting last derived state
* Rename query to getState
* Change query names to getState and getStateByCid
* Save BigInt as string
* Move function to prepare IPLDBlock to indexer
* Refactor IPLDBlock hook
* Add genesis hook
* Call post-block hook after a block is marked as complete
* Add IPLDBlock checkpointing
* Use queryRunner instead of a new repo for queries
* Add a query to get block in ipld-eth-client
* Get latest checkpoints for all contracts for checkpointing.
* Call post-block hook in a queue
* Pass server config to Indexer in watch-contract cli
Co-authored-by: nikugogoi <nabarun@deepstacksoft.com>
* Fix fill cli to work with watcher server
* Add createdAt column and insert events in batches
* Implement prefetch to fill block and events in parallel
* Fix getPrevEntity and increase fill prefetch default batch size
* Fix watcher creating mulitple jobs for a block
* Improvement TODOs
* Get parent and current block in single query
* Remove block progress queries and events query
* Reduce queries in fetching batch events by using query builder
* Implement changes in codegen package
* Avoid database query by passing event directly to job-queue
* Avoid block progress query by returning from update query
* Process batch of events for a block in a single job
* Fix smoke test for subscribed events and use teamSize for job queue
* Perf improvement TODOs
* Move watch contract to util package
* Get block progress from event instead of querying
* Use index field in event to check order of processing
* Use watched contract map to avoid querying database
* Use update query for blockProgress entity
Co-authored-by: Ashwin Phatak <ashwinpphatak@gmail.com>
* Add a GraphDecimal class to replace Decimal with
* Replace Decimal in uni-info-watcher with GraphDecimal
* Add definition for custom scalar BigDecimal in uni-info-watcher
* Use pg-type numeric and transformer for bigint columns in typeorm entities
* Implement pull based watcher for uni-watcher
* Fix same block processed multiple times
* Implement wait time for fetching block from config
* Use blockProgress event to fetch and process next block
* Rename utils index to misc
* Implement clean jobs CLI in watchers
* Pull common code cleanJobs to util
* Implement commands for reset job-queue and state
* Reset command for job-queues and watcher state
* Reset sync status on reset state
* Fix using cli from built js
* Implement skipCheck to reset syncStatus table and skip complete handler on fail
* Check for block isComplete on reset state
* Set default value for force param to update syncStatus to false
* Reset tables in erc20 watcher
* Push job in fill after previos block is complete
* Remove fields no longer present in schema.
* Refactor code to work with log_cids change in eth-server.
* Refactor process event to save events from logs.
* Use constants for erc20 event names.
* Implement watch blocks in erc20-watcher similar to uni-watcher.
* Moved common methods to util.
* Implement eventsInRange query in erc20-watcher.
* Filter unknown event in database query.
* Change dependencies version to be same in all packages.
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
* Transpile ES5 code.
* Use transpiled ES5 code in all packages.
* Refactor tracing-client to transpile ES5 code.
* Order scripts in package json.
* Update readme in packages with new scripts.
* Refactor code to use entity config programatically.
* Remove unused depencdencies.
* Use DEBUG environment for scripts.
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
* Refactor db methods getPrevEntityVersion and getFrothyRegion.
* Filter out entities from pruned blocks.
* Pull saveContract and getModelEntities to util.
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
* Refactor common code in uni-watcher and uni-info-watcher.
* Implement client.ts in uni-info-watcher for queries and use in smoke-test.
* Disable cache in graphql client.
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
* Implement hierarchical query for checking ancestor block.
* Prune blocks together in single query.
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
* Create dummy test blocks and refactor test code.
* Intial test for pruning block in a chain without branches.
* Add test for pruning with multiple branches.
* Add test for pruning at frothy region.
* Test for pruning block at height more than max reorg depth.
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
* Implement block filler for uni-info-watcher.
* Pull common code for Block filler to util.
* Watch for job events in block filler command.
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
* Implement heirarchical query for getting previous entity.
* Check entity id in hierarchical query to limit number of recursions.
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
* Insert and remove a dummy BlockProgress from db.
* Test to fetch a Token entity from pruned region.
* Check if db is empty initially.
* Test to fetch a Token entity from frothy region.
* Test to fetch a Token from frothy region (same block num).
* Test to fetch a Token entity from another branch.
* Tests to fetch a Token entity (multiple tokens).
* Test to fetch updated Token entity in pruned region.
Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
