combine demo & readme, mobymask-watcher

docs/README.md

@@ -121,6 +121,36 @@ Ensure that watcher is of active kind. Update the kind in `server` config to act
 
   * Update the `server` config with state checkpoint settings.
 
+## mm watcher
+
+* Create a postgres12 database for the watcher:
+
+  ```bash
+  sudo su - postgres
+  createdb mobymask-watcher
+  ```
+
+* If the watcher is an `active` watcher:
+
+  Create database for the job queue and enable the `pgcrypto` extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):
+
+  ```
+  createdb mobymask-watcher-job-queue
+  ```
+
+  ```
+  postgres@tesla:~$ psql -U postgres -h localhost mobymask-watcher-job-queue
+  Password for user postgres:
+  psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
+  SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
+  Type "help" for help.
+
+  mobymask-watcher-job-queue=# CREATE EXTENSION pgcrypto;
+  CREATE EXTENSION
+  mobymask-watcher-job-queue=# exit
+  ```
+
+
 ## Watcher CLI commands (yarn)
 
 spot to explain core commands and flags so they aren't repeated in every tutorial

packages/erc20-watcher/README.md

@@ -1,6 +1,6 @@
 # ERC20 Watcher
 
-To setup the stack required to run this demo, either [use stack-orchestrator](https://github.com/cerc-io/stack-orchestrator) or [do it manually](../../docs/README.md)
+To setup the stack required to run this demo, either [use stack-orchestrator](https://github.com/cerc-io/stack-orchestrator) or [do it manually](../../docs/README.md).
 
 ## Build 
 

packages/erc721-watcher/README.md

@@ -1,5 +1,7 @@
 # erc721-watcher
 
+To setup the stack required to run this demo, either [use stack-orchestrator](https://github.com/cerc-io/stack-orchestrator) or [do it manually](../../docs/README.md).
+
 ## Setup
 
 Run the following command to install required packages:

packages/mobymask-watcher/README.md

@@ -1,47 +1,284 @@
-# mobymask-watcher
+# MobyMask Watcher 
+
+To setup the stack required to run this demo, either [use stack-orchestrator](https://github.com/cerc-io/stack-orchestrator) or [do it manually](../../docs/README.md).
 
 ## Setup
 
-* Run the following command to install required packages:
+Run the following command to install required packages:
 
-  ```bash
+```bash
-  yarn
+yarn && yarn build
+```
+
+If the watcher is "lazy", run the server:
+
+```bash
+yarn server
+```
+
+GQL console: http://localhost:3010/graphql
+
+If the watcher is "active", run the job-runner:
+
+```bash
+yarn job-runner
+```
+then the server:
+
+```bash
+yarn server
+```
+
+Next, clone the MobyMask repo and checkout this branch:
+
+```bash
+git clone https://github.com/cerc-io/MobyMask && cd MobyMask
+git checkout use-laconic-watcher-as-hosted-index
+```
+
+Install the packages:
+```bash
+yarn
+```
+
+Deploy the contract:
+```bash
+cd packages/hardhat
+
+yarn deploy
+# deploying "PhisherRegistry" (tx: 0xaebeb2e883ece1f679304ec46f5dc61ca74f9e168427268a7dfa8802195b8de0)...: deployed at 0xMobyAddress with 2306221 gas
+# $ hardhat run scripts/publish.js
+# ✅  Published contracts to the subgraph package.
+# Done in 14.28s.
+```
+
+Export the address of the deployed contract to a shell variable for later use:
+
+```bash
+export MOBY_ADDRESS="0xMobyAddress"
+```
+
+Run the following GQL mutation in watcher GraphQL endpoint http://127.0.0.1:3010/graphql
+
+```graphql
+mutation {
+  watchContract(
+    address: "MOBY_ADDRESS"
+    kind: "PhisherRegistry"
+    checkpoint: true
+  )
+}
+```
+
+Get the latest block
+
+  ```graphql
+  query {
+    latestBlock {
+      hash
+      number
+    }
+  }
   ```
 
-* Create a postgres12 database for the watcher:
+Run the following GQL query in GraphQL endpoint
 
-  ```bash
+```graphql
-  sudo su - postgres
+query {
-  createdb mobymask-watcher
+  isPhisher(
-  ```
+    blockHash: "LATEST_BLOCK_HASH"
+    contractAddress: "MOBY_ADDRESS"
+    key0: "TWT:phishername"
+  ) {
+    value
+    proof {
+      data
+    }
+  }
+  isMember(
+    blockHash: "LATEST_BLOCK_HASH"
+    contractAddress: "MOBY_ADDRESS"
+    key0: "TWT:membername"
+  ) {
+    value
+    proof {
+      data
+    }
+  }
+}
+```
 
-* If the watcher is an `active` watcher:
+Run the following GQL subscription in generated watcher GraphQL endpoint:
 
-  Create database for the job queue and enable the `pgcrypto` extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):
+```graphql
+subscription {
+  onEvent {
+    event {
+      __typename
+      ... on PhisherStatusUpdatedEvent {
+        entity
+        isPhisher
+      },
+      ... on MemberStatusUpdatedEvent {
+        entity
+        isMember
+      }
+    },
+    block {
+      number
+      hash
+    }
+  }
+}
+```
 
-  ```
+Update isPhiser and isMember lists with names
-  createdb mobymask-watcher-job-queue
-  ```
 
-  ```
+```bash
-  postgres@tesla:~$ psql -U postgres -h localhost mobymask-watcher-job-queue
+yarn claimPhisher --contract $MOBY_ADDRESS --name phisherName 
-  Password for user postgres:
+```
-  psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
-  SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
-  Type "help" for help.
 
-  mobymask-watcher-job-queue=# CREATE EXTENSION pgcrypto;
+```bash
-  CREATE EXTENSION
+yarn claimMember --contract $MOBY_ADDRESS --name memberName
-  mobymask-watcher-job-queue=# exit
+```
-  ```
 
-* In the [config file](./environments/local.toml):
+- The events should be visible in the subscription at GQL endpoint. Note down the event blockHash from result.
 
-  * Update the database connection settings.
+- The isMember and isPhisher lists should be indexed. Check the database (mobymask-watcher) tables `is_phisher` and `is_member`, there should be entries at the event blockHash and the value should be true. The data is indexed in `handleEvent` method in the [hooks file](./src/hooks.ts).
 
-  * Update the `upstream` config and provide the `ipld-eth-server` GQL API endpoint.
+Update the the previous query with event blockHash and check isPhisher and isMember in GraphQL playground
 
-  * Update the `server` config with state checkpoint settings.
+```graphql
+query {
+  isPhisher(
+    blockHash: "EVENT_BLOCK_HASH"
+    contractAddress: "MOBY_ADDRESS",
+    key0: "TWT:phishername"
+  ) {
+    value
+    proof {
+      data
+    }
+  }
+  
+  isMember(
+    blockHash: "EVENT_BLOCK_HASH"
+    contractAddress: "MOBY_ADDRESS",
+    key0: "TWT:membername"
+  ) {
+    value
+    proof {
+      data
+    }
+  }
+}
+```
+
+The data is fetched from watcher database as it is already indexed.
+
+## Additional Commands
+
+To watch a contract, run:
+
+```bash
+yarn watch:contract --address <contract-address> --kind <contract-kind> --checkpoint <true | false> --starting-block [block-number]
+```
+where:
+- `address`: Address or identifier of the contract to be watched.
+- `kind`: Kind of the contract.
+- `checkpoint`: Turn checkpointing on (`true` | `false`).
+- `starting-block`: Starting block for the contract (default: `1`).
+
+Examples:
+
+Watch a contract with its address and checkpointing on:
+
+```bash
+yarn watch:contract --address 0x1F78641644feB8b64642e833cE4AFE93DD6e7833 --kind ERC20 --checkpoint true
+```
+
+Watch a contract with its identifier and checkpointing on:
+
+```bash
+yarn watch:contract --address MyProtocol --kind protocol --checkpoint true
+```
+
+To fill a block range:
+
+```bash
+yarn fill --start-block <from-block> --end-block <to-block>
+```
+
+* `start-block`: Block number to start filling from.
+* `end-block`: Block number till which to fill.
+
+To create a checkpoint for a contract:
+
+```bash
+yarn checkpoint create --address <contract-address> --block-hash [block-hash]
+```
+
+* `address`: Address or identifier of the contract for which to create a checkpoint.
+* `block-hash`: Hash of a block (in the pruned region) at which to create the checkpoint (default: latest canonical block hash).
+
+To reset the watcher to a previous block number:
+
+```bash
+yarn reset watcher --block-number <previous-block-number>
+```
+
+Reset job-queue:
+
+```bash
+yarn reset job-queue
+```
+
+Reset state:
+
+```bash
+yarn reset state --block-number <previous-block-number>
+```
+
+* `block-number`: Block number to which to reset the watcher.
+
+To export and import the watcher state:
+
+In the source watcher, export watcher state:
+
+```bash
+yarn export-state --export-file [export-file-path] --block-number [snapshot-block-height]
+```
+
+* `export-file`: Path of file to which to export the watcher data.
+* `block-number`: Block height at which to take snapshot for export.
+
+In the target watcher, run job-runner:
+
+```bash
+yarn job-runner
+```
+
+Import watcher state:
+
+```bash
+yarn import-state --import-file <import-file-path>
+```
+
+* `import-file`: Path of file from which to import the watcher data.
+
+Run server:
+
+```bash
+yarn server
+```
+
+To inspect a CID:
+
+```bash
+yarn inspect-cid --cid <cid>
+```
+
+* `cid`: CID to be inspected.
 
 ## Customize
 
@@ -59,134 +296,3 @@
 
   * Edit the custom hook function `createStateCheckpoint` (triggered just before default and CLI checkpoint) in [hooks.ts](./src/hooks.ts) to save the state in a `checkpoint` `State` using the `Indexer` object.
 
-## Run
-
-Follow the steps below or follow the [Demo](./demo.md)
-
-  * Run the server:
-
-    ```bash
-    yarn server
-    ```
-
-    GQL console: http://localhost:3010/graphql
-
-* If the watcher is an `active` watcher:
-
-  * Run the job-runner:
-
-    ```bash
-    yarn job-runner
-    ```
-
-  * Run the server:
-
-    ```bash
-    yarn server
-    ```
-
-    GQL console: http://localhost:3010/graphql
-
-  * To watch a contract:
-
-    ```bash
-    yarn watch:contract --address <contract-address> --kind <contract-kind> --checkpoint <true | false> --starting-block [block-number]
-    ```
-
-    * `address`: Address or identifier of the contract to be watched.
-    * `kind`: Kind of the contract.
-    * `checkpoint`: Turn checkpointing on (`true` | `false`).
-    * `starting-block`: Starting block for the contract (default: `1`).
-
-    Examples:
-
-    Watch a contract with its address and checkpointing on:
-
-    ```bash
-    yarn watch:contract --address 0x1F78641644feB8b64642e833cE4AFE93DD6e7833 --kind ERC20 --checkpoint true
-    ```
-
-    Watch a contract with its identifier and checkpointing on:
-
-    ```bash
-    yarn watch:contract --address MyProtocol --kind protocol --checkpoint true
-    ```
-
-  * To fill a block range:
-
-    ```bash
-    yarn fill --start-block <from-block> --end-block <to-block>
-    ```
-
-    * `start-block`: Block number to start filling from.
-    * `end-block`: Block number till which to fill.
-
-  * To create a checkpoint for a contract:
-
-    ```bash
-    yarn checkpoint create --address <contract-address> --block-hash [block-hash]
-    ```
-
-    * `address`: Address or identifier of the contract for which to create a checkpoint.
-    * `block-hash`: Hash of a block (in the pruned region) at which to create the checkpoint (default: latest canonical block hash).
-
-  * To reset the watcher to a previous block number:
-
-    * Reset watcher:
-
-      ```bash
-      yarn reset watcher --block-number <previous-block-number>
-      ```
-
-    * Reset job-queue:
-
-      ```bash
-      yarn reset job-queue
-      ```
-
-    * Reset state:
-
-      ```bash
-      yarn reset state --block-number <previous-block-number>
-      ```
-
-    * `block-number`: Block number to which to reset the watcher.
-
-  * To export and import the watcher state:
-
-    * In source watcher, export watcher state:
-
-      ```bash
-      yarn export-state --export-file [export-file-path] --block-number [snapshot-block-height]
-      ```
-
-      * `export-file`: Path of file to which to export the watcher data.
-      * `block-number`: Block height at which to take snapshot for export.
-
-    * In target watcher, run job-runner:
-
-      ```bash
-      yarn job-runner
-      ```
-
-    * Import watcher state:
-
-      ```bash
-      yarn import-state --import-file <import-file-path>
-      ```
-
-      * `import-file`: Path of file from which to import the watcher data.
-
-    * Run server:
-
-      ```bash
-      yarn server
-      ```
-
-  * To inspect a CID:
-
-    ```bash
-    yarn inspect-cid --cid <cid>
-    ```
-
-    * `cid`: CID to be inspected.

packages/mobymask-watcher/demo.md

@@ -1,219 +0,0 @@
-# Demo
-
-* The following core services need to be running for the demo:
-  * [ipld-eth-db](https://github.com/cerc-io/ipld-eth-db)
-    * Version: [v4.2.3-alpha](https://github.com/cerc-io/ipld-eth-db/releases/tag/v4.2.3-alpha)
-  * [geth](https://github.com/cerc-io/go-ethereum)
-    * State diffing service should use `ipld-eth-db` for database.
-    * Version: [v1.10.26-statediff-4.2.2-alpha](https://github.com/cerc-io/go-ethereum/releases/tag/v1.10.26-statediff-4.2.2-alpha)
-    * Endpoint: http://127.0.0.1:8545
-  * [ipld-eth-server](https://github.com/cerc-io/ipld-eth-server)
-    * Should use `ipld-eth-db` for database.
-    * Version: [v4.2.3-alpha](https://github.com/cerc-io/ipld-eth-server/releases/tag/v4.2.3-alpha)
-    * Endpoints:
-      * GQL: http://127.0.0.1:8082/graphql
-      * RPC: http://127.0.0.1:8081
-
-* Create a postgres12 database for the watcher:
-
-  ```bash
-  sudo su - postgres
-  
-  # If database already exists
-  # dropdb mobymask-watcher
-
-  createdb mobymask-watcher
-  ```
-
-* Create database for the job queue and enable the `pgcrypto` extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):
-
-  ```bash
-  # If database already exists
-  # dropdb mobymask-watcher-job-queue
-
-  createdb mobymask-watcher-job-queue
-  ```
-
-  ```
-  postgres@tesla:~$ psql -U postgres -h localhost mobymask-watcher-job-queue
-  Password for user postgres:
-  psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
-  SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
-  Type "help" for help.
-
-  mobymask-watcher-job-queue=# CREATE EXTENSION pgcrypto;
-  CREATE EXTENSION
-  mobymask-watcher-job-queue=# exit
-  ```
-
-* In the [config file](./environments/local.toml) update the `database` connection settings.
-
-* In `watcher-ts` repo, follow the instructions in [Setup](../../README.md#setup) for installing and building packages.
-
-  ```bash
-  # After setup
-  yarn && yarn build
-  ```
-
-* Run the job-runner:
-
-  ```bash
-  yarn job-runner
-  ```
-
-* Change directory to `packages/mobymask-watcher/` and run the watcher:
-
-  ```bash
-  yarn server
-  ```
-
-* Clone the [MobyMask](https://github.com/cerc-io/MobyMask) repo.
-
-* Checkout to the branch with changes for using this watcher:
-
-  ```bash
-  # In MobyMask repo.
-  git checkout use-laconic-watcher-as-hosted-index
-  ```
-
-* Run yarn to install the packages
-
-  ```bash
-  yarn
-  ```
-
-* Deploy the contract:
-
-  ```bash
-  cd packages/hardhat
-
-  yarn deploy
-  # deploying "PhisherRegistry" (tx: 0xaebeb2e883ece1f679304ec46f5dc61ca74f9e168427268a7dfa8802195b8de0)...: deployed at <MOBY_ADDRESS> with 2306221 gas
-  # $ hardhat run scripts/publish.js
-  # ✅  Published contracts to the subgraph package.
-  # Done in 14.28s.
-  ```
-  
-  Export the address of the deployed contract to a shell variable for later use:
-
-  ```bash
-  export MOBY_ADDRESS="<MOBY_ADDRESS>"
-  ```
-
-* Run the following GQL mutation in watcher GraphQL endpoint http://127.0.0.1:3010/graphql
-
-  ```graphql
-  mutation {
-    watchContract(
-      address: "MOBY_ADDRESS"
-      kind: "PhisherRegistry"
-      checkpoint: true
-    )
-  }
-  ```
-
-* Get the latest block
-
-    ```graphql
-    query {
-      latestBlock {
-        hash
-        number
-      }
-    }
-    ```
-
-* Run the following GQL query in GraphQL endpoint
-
-  ```graphql
-  query {
-    isPhisher(
-      blockHash: "LATEST_BLOCK_HASH"
-      contractAddress: "MOBY_ADDRESS"
-      key0: "TWT:phishername"
-    ) {
-      value
-      proof {
-        data
-      }
-    }
-    isMember(
-      blockHash: "LATEST_BLOCK_HASH"
-      contractAddress: "MOBY_ADDRESS"
-      key0: "TWT:membername"
-    ) {
-      value
-      proof {
-        data
-      }
-    }
-  }
-  ```
-
-* Run the following GQL subscription in generated watcher GraphQL endpoint:
-
-  ```graphql
-  subscription {
-    onEvent {
-      event {
-        __typename
-        ... on PhisherStatusUpdatedEvent {
-          entity
-          isPhisher
-        },
-        ... on MemberStatusUpdatedEvent {
-          entity
-          isMember
-        }
-      },
-      block {
-        number
-        hash
-      }
-    }
-  }
-  ```
-
-* Update isPhiser and isMember lists with names
-
-  ```bash
-  yarn claimPhisher --contract $MOBY_ADDRESS --name phisherName 
-  ```
-
-  ```bash
-  yarn claimMember --contract $MOBY_ADDRESS --name memberName
-  ```
-
-* The events should be visible in the subscription at GQL endpoint. Note down the event blockHash from result.
-
-* The isMember and isPhisher lists should be indexed. Check the database (mobymask-watcher) tables `is_phisher` and `is_member`, there should be entries at the event blockHash and the value should be true. The data is indexed in `handleEvent` method in the [hooks file](./src/hooks.ts).
-
-* Update the the previous query with event blockHash and check isPhisher and isMember in GraphQL playground
-
-  ```graphql
-  query {
-    isPhisher(
-      blockHash: "EVENT_BLOCK_HASH"
-      contractAddress: "MOBY_ADDRESS",
-      key0: "TWT:phishername"
-    ) {
-      value
-      proof {
-        data
-      }
-    }
-    
-    isMember(
-      blockHash: "EVENT_BLOCK_HASH"
-      contractAddress: "MOBY_ADDRESS",
-      key0: "TWT:membername"
-    ) {
-      value
-      proof {
-        data
-      }
-    }
-  }
-  ```
-
-  The data is fetched from watcher database as it is already indexed.