combine demo & readme, mobymask-watcher

This commit is contained in:
zramsay 2023-01-04 14:55:23 -05:00
parent d70b89d19a
commit 4080e2ded7
No known key found for this signature in database
GPG Key ID: FECD2DBFAE9DBE63
5 changed files with 298 additions and 379 deletions

View File

@ -121,6 +121,36 @@ Ensure that watcher is of active kind. Update the kind in `server` config to act
* Update the `server` config with state checkpoint settings.
## mm watcher
* Create a postgres12 database for the watcher:
```bash
sudo su - postgres
createdb mobymask-watcher
```
* If the watcher is an `active` watcher:
Create database for the job queue and enable the `pgcrypto` extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):
```
createdb mobymask-watcher-job-queue
```
```
postgres@tesla:~$ psql -U postgres -h localhost mobymask-watcher-job-queue
Password for user postgres:
psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
mobymask-watcher-job-queue=# CREATE EXTENSION pgcrypto;
CREATE EXTENSION
mobymask-watcher-job-queue=# exit
```
## Watcher CLI commands (yarn)
spot to explain core commands and flags so they aren't repeated in every tutorial

View File

@ -1,6 +1,6 @@
# ERC20 Watcher
To setup the stack required to run this demo, either [use stack-orchestrator](https://github.com/cerc-io/stack-orchestrator) or [do it manually](../../docs/README.md)
To setup the stack required to run this demo, either [use stack-orchestrator](https://github.com/cerc-io/stack-orchestrator) or [do it manually](../../docs/README.md).
## Build

View File

@ -1,5 +1,7 @@
# erc721-watcher
To setup the stack required to run this demo, either [use stack-orchestrator](https://github.com/cerc-io/stack-orchestrator) or [do it manually](../../docs/README.md).
## Setup
Run the following command to install required packages:

View File

@ -1,47 +1,284 @@
# mobymask-watcher
# MobyMask Watcher
To setup the stack required to run this demo, either [use stack-orchestrator](https://github.com/cerc-io/stack-orchestrator) or [do it manually](../../docs/README.md).
## Setup
* Run the following command to install required packages:
Run the following command to install required packages:
```bash
yarn
```bash
yarn && yarn build
```
If the watcher is "lazy", run the server:
```bash
yarn server
```
GQL console: http://localhost:3010/graphql
If the watcher is "active", run the job-runner:
```bash
yarn job-runner
```
then the server:
```bash
yarn server
```
Next, clone the MobyMask repo and checkout this branch:
```bash
git clone https://github.com/cerc-io/MobyMask && cd MobyMask
git checkout use-laconic-watcher-as-hosted-index
```
Install the packages:
```bash
yarn
```
Deploy the contract:
```bash
cd packages/hardhat
yarn deploy
# deploying "PhisherRegistry" (tx: 0xaebeb2e883ece1f679304ec46f5dc61ca74f9e168427268a7dfa8802195b8de0)...: deployed at 0xMobyAddress with 2306221 gas
# $ hardhat run scripts/publish.js
# ✅ Published contracts to the subgraph package.
# Done in 14.28s.
```
Export the address of the deployed contract to a shell variable for later use:
```bash
export MOBY_ADDRESS="0xMobyAddress"
```
Run the following GQL mutation in watcher GraphQL endpoint http://127.0.0.1:3010/graphql
```graphql
mutation {
watchContract(
address: "MOBY_ADDRESS"
kind: "PhisherRegistry"
checkpoint: true
)
}
```
Get the latest block
```graphql
query {
latestBlock {
hash
number
}
}
```
* Create a postgres12 database for the watcher:
Run the following GQL query in GraphQL endpoint
```bash
sudo su - postgres
createdb mobymask-watcher
```
```graphql
query {
isPhisher(
blockHash: "LATEST_BLOCK_HASH"
contractAddress: "MOBY_ADDRESS"
key0: "TWT:phishername"
) {
value
proof {
data
}
}
isMember(
blockHash: "LATEST_BLOCK_HASH"
contractAddress: "MOBY_ADDRESS"
key0: "TWT:membername"
) {
value
proof {
data
}
}
}
```
* If the watcher is an `active` watcher:
Run the following GQL subscription in generated watcher GraphQL endpoint:
Create database for the job queue and enable the `pgcrypto` extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):
```graphql
subscription {
onEvent {
event {
__typename
... on PhisherStatusUpdatedEvent {
entity
isPhisher
},
... on MemberStatusUpdatedEvent {
entity
isMember
}
},
block {
number
hash
}
}
}
```
```
createdb mobymask-watcher-job-queue
```
Update isPhiser and isMember lists with names
```
postgres@tesla:~$ psql -U postgres -h localhost mobymask-watcher-job-queue
Password for user postgres:
psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
```bash
yarn claimPhisher --contract $MOBY_ADDRESS --name phisherName
```
mobymask-watcher-job-queue=# CREATE EXTENSION pgcrypto;
CREATE EXTENSION
mobymask-watcher-job-queue=# exit
```
```bash
yarn claimMember --contract $MOBY_ADDRESS --name memberName
```
* In the [config file](./environments/local.toml):
- The events should be visible in the subscription at GQL endpoint. Note down the event blockHash from result.
* Update the database connection settings.
- The isMember and isPhisher lists should be indexed. Check the database (mobymask-watcher) tables `is_phisher` and `is_member`, there should be entries at the event blockHash and the value should be true. The data is indexed in `handleEvent` method in the [hooks file](./src/hooks.ts).
* Update the `upstream` config and provide the `ipld-eth-server` GQL API endpoint.
Update the the previous query with event blockHash and check isPhisher and isMember in GraphQL playground
* Update the `server` config with state checkpoint settings.
```graphql
query {
isPhisher(
blockHash: "EVENT_BLOCK_HASH"
contractAddress: "MOBY_ADDRESS",
key0: "TWT:phishername"
) {
value
proof {
data
}
}
isMember(
blockHash: "EVENT_BLOCK_HASH"
contractAddress: "MOBY_ADDRESS",
key0: "TWT:membername"
) {
value
proof {
data
}
}
}
```
The data is fetched from watcher database as it is already indexed.
## Additional Commands
To watch a contract, run:
```bash
yarn watch:contract --address <contract-address> --kind <contract-kind> --checkpoint <true | false> --starting-block [block-number]
```
where:
- `address`: Address or identifier of the contract to be watched.
- `kind`: Kind of the contract.
- `checkpoint`: Turn checkpointing on (`true` | `false`).
- `starting-block`: Starting block for the contract (default: `1`).
Examples:
Watch a contract with its address and checkpointing on:
```bash
yarn watch:contract --address 0x1F78641644feB8b64642e833cE4AFE93DD6e7833 --kind ERC20 --checkpoint true
```
Watch a contract with its identifier and checkpointing on:
```bash
yarn watch:contract --address MyProtocol --kind protocol --checkpoint true
```
To fill a block range:
```bash
yarn fill --start-block <from-block> --end-block <to-block>
```
* `start-block`: Block number to start filling from.
* `end-block`: Block number till which to fill.
To create a checkpoint for a contract:
```bash
yarn checkpoint create --address <contract-address> --block-hash [block-hash]
```
* `address`: Address or identifier of the contract for which to create a checkpoint.
* `block-hash`: Hash of a block (in the pruned region) at which to create the checkpoint (default: latest canonical block hash).
To reset the watcher to a previous block number:
```bash
yarn reset watcher --block-number <previous-block-number>
```
Reset job-queue:
```bash
yarn reset job-queue
```
Reset state:
```bash
yarn reset state --block-number <previous-block-number>
```
* `block-number`: Block number to which to reset the watcher.
To export and import the watcher state:
In the source watcher, export watcher state:
```bash
yarn export-state --export-file [export-file-path] --block-number [snapshot-block-height]
```
* `export-file`: Path of file to which to export the watcher data.
* `block-number`: Block height at which to take snapshot for export.
In the target watcher, run job-runner:
```bash
yarn job-runner
```
Import watcher state:
```bash
yarn import-state --import-file <import-file-path>
```
* `import-file`: Path of file from which to import the watcher data.
Run server:
```bash
yarn server
```
To inspect a CID:
```bash
yarn inspect-cid --cid <cid>
```
* `cid`: CID to be inspected.
## Customize
@ -59,134 +296,3 @@
* Edit the custom hook function `createStateCheckpoint` (triggered just before default and CLI checkpoint) in [hooks.ts](./src/hooks.ts) to save the state in a `checkpoint` `State` using the `Indexer` object.
## Run
Follow the steps below or follow the [Demo](./demo.md)
* Run the server:
```bash
yarn server
```
GQL console: http://localhost:3010/graphql
* If the watcher is an `active` watcher:
* Run the job-runner:
```bash
yarn job-runner
```
* Run the server:
```bash
yarn server
```
GQL console: http://localhost:3010/graphql
* To watch a contract:
```bash
yarn watch:contract --address <contract-address> --kind <contract-kind> --checkpoint <true | false> --starting-block [block-number]
```
* `address`: Address or identifier of the contract to be watched.
* `kind`: Kind of the contract.
* `checkpoint`: Turn checkpointing on (`true` | `false`).
* `starting-block`: Starting block for the contract (default: `1`).
Examples:
Watch a contract with its address and checkpointing on:
```bash
yarn watch:contract --address 0x1F78641644feB8b64642e833cE4AFE93DD6e7833 --kind ERC20 --checkpoint true
```
Watch a contract with its identifier and checkpointing on:
```bash
yarn watch:contract --address MyProtocol --kind protocol --checkpoint true
```
* To fill a block range:
```bash
yarn fill --start-block <from-block> --end-block <to-block>
```
* `start-block`: Block number to start filling from.
* `end-block`: Block number till which to fill.
* To create a checkpoint for a contract:
```bash
yarn checkpoint create --address <contract-address> --block-hash [block-hash]
```
* `address`: Address or identifier of the contract for which to create a checkpoint.
* `block-hash`: Hash of a block (in the pruned region) at which to create the checkpoint (default: latest canonical block hash).
* To reset the watcher to a previous block number:
* Reset watcher:
```bash
yarn reset watcher --block-number <previous-block-number>
```
* Reset job-queue:
```bash
yarn reset job-queue
```
* Reset state:
```bash
yarn reset state --block-number <previous-block-number>
```
* `block-number`: Block number to which to reset the watcher.
* To export and import the watcher state:
* In source watcher, export watcher state:
```bash
yarn export-state --export-file [export-file-path] --block-number [snapshot-block-height]
```
* `export-file`: Path of file to which to export the watcher data.
* `block-number`: Block height at which to take snapshot for export.
* In target watcher, run job-runner:
```bash
yarn job-runner
```
* Import watcher state:
```bash
yarn import-state --import-file <import-file-path>
```
* `import-file`: Path of file from which to import the watcher data.
* Run server:
```bash
yarn server
```
* To inspect a CID:
```bash
yarn inspect-cid --cid <cid>
```
* `cid`: CID to be inspected.

View File

@ -1,219 +0,0 @@
# Demo
* The following core services need to be running for the demo:
* [ipld-eth-db](https://github.com/cerc-io/ipld-eth-db)
* Version: [v4.2.3-alpha](https://github.com/cerc-io/ipld-eth-db/releases/tag/v4.2.3-alpha)
* [geth](https://github.com/cerc-io/go-ethereum)
* State diffing service should use `ipld-eth-db` for database.
* Version: [v1.10.26-statediff-4.2.2-alpha](https://github.com/cerc-io/go-ethereum/releases/tag/v1.10.26-statediff-4.2.2-alpha)
* Endpoint: http://127.0.0.1:8545
* [ipld-eth-server](https://github.com/cerc-io/ipld-eth-server)
* Should use `ipld-eth-db` for database.
* Version: [v4.2.3-alpha](https://github.com/cerc-io/ipld-eth-server/releases/tag/v4.2.3-alpha)
* Endpoints:
* GQL: http://127.0.0.1:8082/graphql
* RPC: http://127.0.0.1:8081
* Create a postgres12 database for the watcher:
```bash
sudo su - postgres
# If database already exists
# dropdb mobymask-watcher
createdb mobymask-watcher
```
* Create database for the job queue and enable the `pgcrypto` extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):
```bash
# If database already exists
# dropdb mobymask-watcher-job-queue
createdb mobymask-watcher-job-queue
```
```
postgres@tesla:~$ psql -U postgres -h localhost mobymask-watcher-job-queue
Password for user postgres:
psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
mobymask-watcher-job-queue=# CREATE EXTENSION pgcrypto;
CREATE EXTENSION
mobymask-watcher-job-queue=# exit
```
* In the [config file](./environments/local.toml) update the `database` connection settings.
* In `watcher-ts` repo, follow the instructions in [Setup](../../README.md#setup) for installing and building packages.
```bash
# After setup
yarn && yarn build
```
* Run the job-runner:
```bash
yarn job-runner
```
* Change directory to `packages/mobymask-watcher/` and run the watcher:
```bash
yarn server
```
* Clone the [MobyMask](https://github.com/cerc-io/MobyMask) repo.
* Checkout to the branch with changes for using this watcher:
```bash
# In MobyMask repo.
git checkout use-laconic-watcher-as-hosted-index
```
* Run yarn to install the packages
```bash
yarn
```
* Deploy the contract:
```bash
cd packages/hardhat
yarn deploy
# deploying "PhisherRegistry" (tx: 0xaebeb2e883ece1f679304ec46f5dc61ca74f9e168427268a7dfa8802195b8de0)...: deployed at <MOBY_ADDRESS> with 2306221 gas
# $ hardhat run scripts/publish.js
# ✅ Published contracts to the subgraph package.
# Done in 14.28s.
```
Export the address of the deployed contract to a shell variable for later use:
```bash
export MOBY_ADDRESS="<MOBY_ADDRESS>"
```
* Run the following GQL mutation in watcher GraphQL endpoint http://127.0.0.1:3010/graphql
```graphql
mutation {
watchContract(
address: "MOBY_ADDRESS"
kind: "PhisherRegistry"
checkpoint: true
)
}
```
* Get the latest block
```graphql
query {
latestBlock {
hash
number
}
}
```
* Run the following GQL query in GraphQL endpoint
```graphql
query {
isPhisher(
blockHash: "LATEST_BLOCK_HASH"
contractAddress: "MOBY_ADDRESS"
key0: "TWT:phishername"
) {
value
proof {
data
}
}
isMember(
blockHash: "LATEST_BLOCK_HASH"
contractAddress: "MOBY_ADDRESS"
key0: "TWT:membername"
) {
value
proof {
data
}
}
}
```
* Run the following GQL subscription in generated watcher GraphQL endpoint:
```graphql
subscription {
onEvent {
event {
__typename
... on PhisherStatusUpdatedEvent {
entity
isPhisher
},
... on MemberStatusUpdatedEvent {
entity
isMember
}
},
block {
number
hash
}
}
}
```
* Update isPhiser and isMember lists with names
```bash
yarn claimPhisher --contract $MOBY_ADDRESS --name phisherName
```
```bash
yarn claimMember --contract $MOBY_ADDRESS --name memberName
```
* The events should be visible in the subscription at GQL endpoint. Note down the event blockHash from result.
* The isMember and isPhisher lists should be indexed. Check the database (mobymask-watcher) tables `is_phisher` and `is_member`, there should be entries at the event blockHash and the value should be true. The data is indexed in `handleEvent` method in the [hooks file](./src/hooks.ts).
* Update the the previous query with event blockHash and check isPhisher and isMember in GraphQL playground
```graphql
query {
isPhisher(
blockHash: "EVENT_BLOCK_HASH"
contractAddress: "MOBY_ADDRESS",
key0: "TWT:phishername"
) {
value
proof {
data
}
}
isMember(
blockHash: "EVENT_BLOCK_HASH"
contractAddress: "MOBY_ADDRESS",
key0: "TWT:membername"
) {
value
proof {
data
}
}
}
```
The data is fetched from watcher database as it is already indexed.