watcher-ts/packages/codegen/subgraph-demo.md
nikugogoi cc28474537
Accommodate GQL optimizations in codegen (#254)
* Prune optional methods in indexer and database interfaces

* Implement GQL optimization changes in codegen

* Fix graph-node test indexer

* Add demos to codegen package
2022-11-23 17:42:25 +05:30

6.4 KiB

Subgraph watcher demo

  • Clone the stack-orchestrator repo.

    git clone https://github.com/vulcanize/stack-orchestrator
    
  • Create a config.sh file.

    cd stack-orchestrator/helper-scripts
    ./create-config.sh
    
  • Setup the required repositories.

    ./setup-repositories.sh -p ssh
    
  • Checkout v4 release in go-ethereum repo. The path for go-ethereum is specified by vulcanize_go_ethereum variable in config.sh file created in stack-orchestrator repo.

    # In go-ethereum repo.
    git checkout v1.10.26-statediff-4.2.2-alpha
    
  • Update to use latest images for ipld-eth-db and ipld-eth-server

  • To run the stack-orchestrator, the docker-compose version used is:

    docker-compose version
    
    # docker-compose version 1.29.2, build 5becea4c
    
  • Run the stack-orchestrator

    cd stack-orchestrator/helper-scripts
    
    ./wrapper.sh -f true \
      -m true \
      -s v4 \
      -l latest \
      -v remove \
      -p ../config.sh
    
  • In watcher-ts packages/graph-node, deploy an Example contract:

    yarn example:deploy
    
  • Set the returned address to the variable $EXAMPLE_ADDRESS:

    export EXAMPLE_ADDRESS=<EXAMPLE_ADDRESS>
    
  • In packages/graph-node/test/subgraph/example1/subgraph.yaml, set the source address for Example1 datasource to the EXAMPLE_ADDRESS. Then in packages/graph-node run:

    yarn build:example
    
  • In packages/codegen, create a config.yaml file:

    # Example config.yaml
    # Contracts to watch (required).
    # Can pass empty array ([]) when using subgraphPath.
    contracts:
        # Contract name.
      - name: Example
        # Contract file path or an url.
        path: ../graph-node/test/contracts/Example.sol
        # Contract kind (should match that in {subgraphPath}/subgraph.yaml if subgraphPath provided)
        kind: Example1
    
    # Output folder path (logs output using `stdout` if not provided).
    outputFolder: ../test-watcher
    
    # Code generation mode [eth_call | storage | all | none] (default: none).
    mode: none
    
    # Kind of watcher [lazy | active] (default: active).
    kind: active
    
    # Watcher server port (default: 3008).
    port: 3008
    
    # Flatten the input contract file(s) [true | false] (default: true).
    flatten: true
    
    # Path to the subgraph build (optional).
    # Can set empty contracts array when using subgraphPath.
    subgraphPath: ../graph-node/test/subgraph/example1/build
    
  • Run codegen to generate watcher:

    yarn codegen --config-file ./config.yaml
    

    The watcher should be generated in packages/test-watcher

  • Create a postgres12 database for the watcher:

    sudo su - postgres
    
    # If database already exists
    # dropdb test-watcher
    
    createdb test-watcher
    
  • Create database for the job queue and enable the pgcrypto extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):

    # If database already exists
    # dropdb test-watcher-job-queue
    
    createdb test-watcher-job-queue
    
    postgres@tesla:~$ psql -U postgres -h localhost test-watcher-job-queue
    Password for user postgres:
    psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
    SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
    Type "help" for help.
    
    test-watcher-job-queue=# CREATE EXTENSION pgcrypto;
    CREATE EXTENSION
    test-watcher-job-queue=# exit
    
  • In watcher-ts repo, follow the instructions in Setup for installing and building packages.

    # After setup
    yarn && yarn build
    
  • In packages/test-watcher, run the job-runner:

    yarn job-runner
    
  • Run the watcher:

    yarn server
    

Operations

  • Run the following GQL subscription at the graphql endpoint:

    subscription {
      onEvent {
        event {
          __typename
          ... on TestEvent {
            param1
            param2
          },
        },
        block {
          number
          hash
        }
      }
    }
    
  • In packages/graph-node, trigger the Test event by calling a example contract method:

    yarn example:test --address $EXAMPLE_ADDRESS
    
    • A Test event shall be visible in the subscription at endpoint.

    • The subgraph entity Category should be updated in the database.

    • An auto-generated diff-staged entry State should be added.

  • Run the query for entity in at the endpoint:

    query {
      category(
        block: { hash: "EVENT_BLOCK_HASH" },
        id: "1"
      ) {
        __typename
        id
        count
        name
      }
    }
    
  • Run the getState query at the endpoint to get the latest State for EXAMPLE_ADDRESS:

    query {
      getState (
        blockHash: "EVENT_BLOCK_HASH"
        contractAddress: "EXAMPLE_ADDRESS"
        # kind: "checkpoint"
        # kind: "diff"
        kind: "diff_staged"
      ) {
        cid
        block {
          cid
          hash
          number
          timestamp
          parentHash
        }
        contractAddress
        data
      }
    }
    
  • diff states get created corresponding to the diff_staged states when their respective blocks reach the pruned region.

  • In packages/test-watcher:

    • After the diff state has been created, create a checkpoint:

      yarn checkpoint create --address $EXAMPLE_ADDRESS
      
      • A checkpoint state should be created at the latest canonical block hash.

      • Run the getState query again at the endpoint with the output blockHash and kind checkpoint.

  • All the State entries can be seen in pg-admin in table state.