cerc-io/watcher-ts
16
1
Fork 0
mirror of https://github.com/cerc-io/watcher-ts synced 2024-11-19 12:26:19 +00:00
Code Issues Packages Projects Releases Wiki Activity

combine demo & readme, graph-test-watcher; start docs/cli

Browse Source
This commit is contained in:
zramsay 2023-01-10 07:39:29 -05:00
parent 4ee889eaa7
commit 85452a870f
No known key found for this signature in database
GPG Key ID: FECD2DBFAE9DBE63
4 changed files with 269 additions and 326 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

136
docs/cli.md
View File

@ -1,3 +1,137 @@
## Watcher CLI commands (yarn) ## Watcher CLI commands (yarn)
spot to explain core commands and flags so they aren't repeated in every tutorial * Run the server:
```bash
yarn server
```
GQL console: http://localhost:3008/graphql
* If the watcher is an `active` watcher:
* Run the job-runner:
```bash
yarn job-runner
```
* Run the server:
```bash
yarn server
```
GQL console: http://localhost:3008/graphql
* To watch a contract:
```bash
yarn watch:contract --address <contract-address> --kind <contract-kind> --checkpoint <true | false> --starting-block [block-number]
```
* `address`: Address or identifier of the contract to be watched.
* `kind`: Kind of the contract.
* `checkpoint`: Turn checkpointing on (`true` | `false`).
* `starting-block`: Starting block for the contract (default: `1`).
Examples:
Watch a contract with its address and checkpointing on:
```bash
yarn watch:contract --address 0x1F78641644feB8b64642e833cE4AFE93DD6e7833 --kind ERC20 --checkpoint true
```
Watch a contract with its identifier and checkpointing on:
```bash
yarn watch:contract --address MyProtocol --kind protocol --checkpoint true
```
* To fill a block range:
```bash
yarn fill --start-block <from-block> --end-block <to-block>
```
* `start-block`: Block number to start filling from.
* `end-block`: Block number till which to fill.
* To create a checkpoint for a contract:
```bash
yarn checkpoint create --address <contract-address> --block-hash [block-hash]
```
* `address`: Address or identifier of the contract for which to create a checkpoint.
* `block-hash`: Hash of a block (in the pruned region) at which to create the checkpoint (default: latest canonical block hash).
* To verify a checkpoint:
```bash
yarn checkpoint verify --cid <checkpoint-cid>
```
`cid`: CID of the checkpoint for which to verify.
* To reset the watcher to a previous block number:
* Reset watcher:
```bash
yarn reset watcher --block-number <previous-block-number>
```
* Reset job-queue:
```bash
yarn reset job-queue
```
* Reset state:
```bash
yarn reset state --block-number <previous-block-number>
```
* `block-number`: Block number to which to reset the watcher.
* To export and import the watcher state:
* In source watcher, export watcher state:
```bash
yarn export-state --export-file [export-file-path] --block-number [snapshot-block-height]
```
* `export-file`: Path of file to which to export the watcher data.
* `block-number`: Block height at which to take snapshot for export.
* In target watcher, run job-runner:
```bash
yarn job-runner
```
* Import watcher state:
```bash
yarn import-state --import-file <import-file-path>
```
* `import-file`: Path of file from which to import the watcher data.
* Run server:
```bash
yarn server
```
* To inspect a CID:
```bash
yarn inspect-cid --cid <cid>
```
* `cid`: CID to be inspected.

2
packages/erc20-watcher/README.md
View File

@ -31,7 +31,7 @@ yarn server
``` ```
For development or to specify the config file: For development or to specify the config file:
```basj ```bash
yarn server:dev yarn server:dev
yarn server -f environments/local.toml yarn server -f environments/local.toml
``` ```

275
packages/graph-test-watcher/README.md
View File

@ -2,12 +2,143 @@
## Setup ## Setup
* Run the following command to install required packages: First try the [stack orchestrator](https://github.com/cerc-io/stack-orchestrator) to quickly get started. Advanced users can see [here](/docs/README.md) for instructions on setting up a local environment by hand.
Run the following command to install required packages:
```bash
yarn
```
## Run
* In [packages/graph-node](../graph-node/), deploy an `Example` contract:
```bash ```bash
yarn yarn example:deploy
``` ```
* Set the returned address to the variable `$EXAMPLE_ADDRESS`:
```bash
EXAMPLE_ADDRESS=<EXAMPLE_ADDRESS>
```
* In [packages/graph-node/test/subgraph/example1/subgraph.yaml](../graph-node/test/subgraph/example1/subgraph.yaml):
* Set the source address for `Example1` datasource to the `EXAMPLE_ADDRESS`.
* Set the `startBlock` less than or equal to the latest mined block.
* Build the example subgraph:
```bash
yarn build:example
```
* Run the job-runner:
```bash
yarn job-runner
```
* Run the watcher:
```bash
yarn server
```
* The output from the block handler in the mapping code should be visible in the `job-runner` for each block.
* Run the following GQL subscription at the graphql endpoint http://127.0.0.1:3008/graphql
```graphql
subscription {
onEvent {
event {
__typename
... on TestEvent {
param1
param2
},
},
block {
number
hash
}
}
}
```
* In [packages/graph-node](../graph-node/), trigger the `Test` event by calling a example contract method:
```bash
yarn example:test --address $EXAMPLE_ADDRESS
```
* A `Test` event shall be visible in the subscription at endpoint.
* The subgraph entity `Category` should be updated in the database.
* An auto-generated `diff-staged` entry `State` should be added.
* Run the query for entity in at the endpoint:
```graphql
query {
category(
block: { hash: "EVENT_BLOCK_HASH" },
id: "1"
) {
__typename
id
count
name
}
}
```
* Run the `getState` query at the endpoint to get the latest `State` for `EXAMPLE_ADDRESS`:
```graphql
query {
getState (
blockHash: "EVENT_BLOCK_HASH"
contractAddress: "EXAMPLE_ADDRESS"
# kind: "checkpoint"
# kind: "diff"
kind: "diff_staged"
) {
cid
block {
cid
hash
number
timestamp
parentHash
}
contractAddress
data
}
}
```
* `diff` states get created corresponding to the `diff_staged` states when their respective blocks reach the pruned region.
* In [packages/graph-test-watcher](./):
* After the `diff` state has been created, create a `checkpoint`:
```bash
yarn checkpoint create --address $EXAMPLE_ADDRESS
```
* A `checkpoint` state should be created at the latest canonical block hash.
* Run the `getState` query again at the endpoint with the output `blockHash` and kind `checkpoint`.
* All the `State` entries can be seen in `pg-admin` in table `state`.
## Customize ## Customize
* Indexing on an event: * Indexing on an event:
@ -25,143 +156,3 @@
* Edit the custom hook function `createStateCheckpoint` (triggered just before default and CLI checkpoint) in [hooks.ts](./src/hooks.ts) to save the state in a `checkpoint` `State` using the `Indexer` object. * Edit the custom hook function `createStateCheckpoint` (triggered just before default and CLI checkpoint) in [hooks.ts](./src/hooks.ts) to save the state in a `checkpoint` `State` using the `Indexer` object.
* The existing example hooks in [hooks.ts](./src/hooks.ts) are for an `ERC20` contract. * The existing example hooks in [hooks.ts](./src/hooks.ts) are for an `ERC20` contract.
## Run
Follow the steps below or follow the [Demo](./demo.md)
* Run the server:
```bash
yarn server
```
GQL console: http://localhost:3008/graphql
* If the watcher is an `active` watcher:
* Run the job-runner:
```bash
yarn job-runner
```
* Run the server:
```bash
yarn server
```
GQL console: http://localhost:3008/graphql
* To watch a contract:
```bash
yarn watch:contract --address <contract-address> --kind <contract-kind> --checkpoint <true | false> --starting-block [block-number]
```
* `address`: Address or identifier of the contract to be watched.
* `kind`: Kind of the contract.
* `checkpoint`: Turn checkpointing on (`true` | `false`).
* `starting-block`: Starting block for the contract (default: `1`).
Examples:
Watch a contract with its address and checkpointing on:
```bash
yarn watch:contract --address 0x1F78641644feB8b64642e833cE4AFE93DD6e7833 --kind ERC20 --checkpoint true
```
Watch a contract with its identifier and checkpointing on:
```bash
yarn watch:contract --address MyProtocol --kind protocol --checkpoint true
```
* To fill a block range:
```bash
yarn fill --start-block <from-block> --end-block <to-block>
```
* `start-block`: Block number to start filling from.
* `end-block`: Block number till which to fill.
* To create a checkpoint for a contract:
```bash
yarn checkpoint create --address <contract-address> --block-hash [block-hash]
```
* `address`: Address or identifier of the contract for which to create a checkpoint.
* `block-hash`: Hash of a block (in the pruned region) at which to create the checkpoint (default: latest canonical block hash).
* To verify a checkpoint:
```bash
yarn checkpoint verify --cid <checkpoint-cid>
```
`cid`: CID of the checkpoint for which to verify.
* To reset the watcher to a previous block number:
* Reset watcher:
```bash
yarn reset watcher --block-number <previous-block-number>
```
* Reset job-queue:
```bash
yarn reset job-queue
```
* Reset state:
```bash
yarn reset state --block-number <previous-block-number>
```
* `block-number`: Block number to which to reset the watcher.
* To export and import the watcher state:
* In source watcher, export watcher state:
```bash
yarn export-state --export-file [export-file-path] --block-number [snapshot-block-height]
```
* `export-file`: Path of file to which to export the watcher data.
* `block-number`: Block height at which to take snapshot for export.
* In target watcher, run job-runner:
```bash
yarn job-runner
```
* Import watcher state:
```bash
yarn import-state --import-file <import-file-path>
```
* `import-file`: Path of file from which to import the watcher data.
* Run server:
```bash
yarn server
```
* To inspect a CID:
```bash
yarn inspect-cid --cid <cid>
```
* `cid`: CID to be inspected.

182
packages/graph-test-watcher/demo.md
View File

@ -1,182 +0,0 @@
# Demo
* The following core services need to be running for the demo:
* [ipld-eth-db](https://github.com/cerc-io/ipld-eth-db)
* Version: [v4.2.3-alpha](https://github.com/cerc-io/ipld-eth-db/releases/tag/v4.2.3-alpha)
* [geth](https://github.com/cerc-io/go-ethereum)
* State diffing service should use `ipld-eth-db` for database.
* Version: [v1.10.26-statediff-4.2.2-alpha](https://github.com/cerc-io/go-ethereum/releases/tag/v1.10.26-statediff-4.2.2-alpha)
* Endpoint: http://127.0.0.1:8545
* [ipld-eth-server](https://github.com/cerc-io/ipld-eth-server)
* Should use `ipld-eth-db` for database.
* Version: [v4.2.3-alpha](https://github.com/cerc-io/ipld-eth-server/releases/tag/v4.2.3-alpha)
* Endpoints:
* GQL: http://127.0.0.1:8082/graphql
* RPC: http://127.0.0.1:8081
* Create a postgres12 database for the watcher:
```bash
sudo su - postgres
# If database already exists
# dropdb graph-test-watcher
createdb graph-test-watcher
```
* Create database for the job queue and enable the `pgcrypto` extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):
```bash
# If database already exists
# dropdb graph-test-watcher-job-queue
createdb graph-test-watcher-job-queue
```
```
postgres@tesla:~$ psql -U postgres -h localhost graph-test-watcher-job-queue
Password for user postgres:
psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
graph-test-watcher-job-queue=# CREATE EXTENSION pgcrypto;
CREATE EXTENSION
graph-test-watcher-job-queue=# exit
```
* In the [config file](./environments/local.toml) update the `database` connection settings.
* In `watcher-ts` repo, follow the instructions in [Setup](../../README.md#setup) for installing and building packages.
```bash
# After setup
yarn && yarn build
```
* In [packages/graph-node](../graph-node/), deploy an `Example` contract:
```bash
yarn example:deploy
```
* Set the returned address to the variable `$EXAMPLE_ADDRESS`:
```bash
EXAMPLE_ADDRESS=<EXAMPLE_ADDRESS>
```
* In [packages/graph-node/test/subgraph/example1/subgraph.yaml](../graph-node/test/subgraph/example1/subgraph.yaml):
* Set the source address for `Example1` datasource to the `EXAMPLE_ADDRESS`.
* Set the `startBlock` less than or equal to the latest mined block.
* Build the example subgraph:
```bash
yarn build:example
```
* Run the job-runner:
```bash
yarn job-runner
```
* Run the watcher:
```bash
yarn server
```
* The output from the block handler in the mapping code should be visible in the `job-runner` for each block.
* Run the following GQL subscription at the graphql endpoint http://127.0.0.1:3008/graphql
```graphql
subscription {
onEvent {
event {
__typename
... on TestEvent {
param1
param2
},
},
block {
number
hash
}
}
}
```
* In [packages/graph-node](../graph-node/), trigger the `Test` event by calling a example contract method:
```bash
yarn example:test --address $EXAMPLE_ADDRESS
```
* A `Test` event shall be visible in the subscription at endpoint.
* The subgraph entity `Category` should be updated in the database.
* An auto-generated `diff-staged` entry `State` should be added.
* Run the query for entity in at the endpoint:
```graphql
query {
category(
block: { hash: "EVENT_BLOCK_HASH" },
id: "1"
) {
__typename
id
count
name
}
}
```
* Run the `getState` query at the endpoint to get the latest `State` for `EXAMPLE_ADDRESS`:
```graphql
query {
getState (
blockHash: "EVENT_BLOCK_HASH"
contractAddress: "EXAMPLE_ADDRESS"
# kind: "checkpoint"
# kind: "diff"
kind: "diff_staged"
) {
cid
block {
cid
hash
number
timestamp
parentHash
}
contractAddress
data
}
}
```
* `diff` states get created corresponding to the `diff_staged` states when their respective blocks reach the pruned region.
* In [packages/graph-test-watcher](./):
* After the `diff` state has been created, create a `checkpoint`:
```bash
yarn checkpoint create --address $EXAMPLE_ADDRESS
```
* A `checkpoint` state should be created at the latest canonical block hash.
* Run the `getState` query again at the endpoint with the output `blockHash` and kind `checkpoint`.
* All the `State` entries can be seen in `pg-admin` in table `state`.