watcher-ts/packages/codegen/subgraph-demo.md

Subgraph watcher demo

• The following core services need to be running for the demo:

  • ipld-eth-db
    • Version: v4.2.3-alpha
  • geth
    • State diffing service should use ipld-eth-db for database.
    • Version: v1.10.26-statediff-4.2.2-alpha
    • Endpoint: http://127.0.0.1:8545
  • ipld-eth-server
    • Should use ipld-eth-db for database.
    • Version: v4.2.3-alpha
    • Endpoints:
      • GQL: http://127.0.0.1:8082/graphql
      • RPC: http://127.0.0.1:8081

• For this demo, an example subgraph will be used

  • In package.json, the graph-ts and graph-cli dependencies are replaced by their respective cerc-io forked dependencies

  • This needs to be done for running any subgraph project with watcher-ts

• In watcher-ts packages/graph-node, deploy an Example contract:

  yarn example:deploy
  

• Set the returned address to the variable $EXAMPLE_ADDRESS:

  export EXAMPLE_ADDRESS=<EXAMPLE_ADDRESS>
  

• In packages/graph-node/test/subgraph/example1/subgraph.yaml, set the source address for Example1 datasource to the EXAMPLE_ADDRESS. Then in packages/graph-node run:

  yarn build:example
  

  • This will run yarn && yarn codegen && yarn build script in the example subgraph directory

• In packages/codegen, create a config.yaml file:

  # Example config.yaml
  # Contracts to watch (required).
  # Can pass empty array ([]) when using subgraphPath.
  contracts: []
  
  # Output folder path (logs output using `stdout` if not provided).
  outputFolder: ../test-watcher
  
  # Code generation mode [eth_call | storage | all | none] (default: none).
  mode: none
  
  # Kind of watcher [lazy | active] (default: active).
  kind: active
  
  # Watcher server port (default: 3008).
  port: 3008
  
  # Flatten the input contract file(s) [true | false] (default: true).
  flatten: true
  
  # Path to the subgraph build (optional).
  # Can set empty contracts array when using subgraphPath.
  # Subgraph WASM files should be compiled using @cerc-io/graph-cli
  subgraphPath: ../graph-node/test/subgraph/example1/build
  

• Run codegen to generate watcher:

  yarn codegen --config-file ./config.yaml
  

  The watcher should be generated in packages/test-watcher

• Create a postgres12 database for the watcher:

  sudo su - postgres
  
  # If database already exists
  # dropdb test-watcher
  
  createdb test-watcher
  

• Create database for the job queue and enable the pgcrypto extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):

  # If database already exists
  # dropdb test-watcher-job-queue
  
  createdb test-watcher-job-queue
  

  postgres@tesla:~$ psql -U postgres -h localhost test-watcher-job-queue
  Password for user postgres:
  psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
  SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
  Type "help" for help.
  
  test-watcher-job-queue=# CREATE EXTENSION pgcrypto;
  CREATE EXTENSION
  test-watcher-job-queue=# exit
  

• In watcher-ts repo, follow the instructions in Setup for installing and building packages.

  # After setup
  yarn && yarn build
  

• In packages/test-watcher, run fill for the subgraph start block:

  yarn fill --start-block 10 --end-block 10
  

  • Subgraph start block is the lowest startBlock in example subgraph.yaml

• Run the job-runner:

  yarn job-runner
  

• Run the watcher:

  yarn server
  

Operations

• Run the following GQL subscription at the graphql endpoint:

  subscription {
    onEvent {
      event {
        __typename
        ... on TestEvent {
          param1
          param2
        },
      },
      block {
        number
        hash
      }
    }
  }
  

• In packages/graph-node, trigger the Test event by calling a example contract method:

  yarn example:test --address $EXAMPLE_ADDRESS
  

  • A Test event shall be visible in the subscription at endpoint.

  • The subgraph entity Category should be updated in the database.

  • An auto-generated diff-staged entry State should be added.

• Run the query for entity in at the endpoint:

  query {
    category(
      block: { hash: "EVENT_BLOCK_HASH" },
      id: "1"
    ) {
      __typename
      id
      count
      name
    }
  }
  

• Run the getState query at the endpoint to get the latest State for EXAMPLE_ADDRESS:

  query {
    getState (
      blockHash: "EVENT_BLOCK_HASH"
      contractAddress: "EXAMPLE_ADDRESS"
      # kind: "checkpoint"
      # kind: "diff"
      kind: "diff_staged"
    ) {
      cid
      block {
        cid
        hash
        number
        timestamp
        parentHash
      }
      contractAddress
      data
    }
  }
  

• diff states get created corresponding to the diff_staged states when their respective blocks reach the pruned region.

• In packages/test-watcher:

  • After the diff state has been created, create a checkpoint:

    yarn checkpoint create --address $EXAMPLE_ADDRESS
    

    • A checkpoint state should be created at the latest canonical block hash.

    • Run the getState query again at the endpoint with the output blockHash and kind checkpoint.

• All the State entries can be seen in pg-admin in table state.