cerc-io/watcher-ts
16
1
Fork 0
mirror of https://github.com/cerc-io/watcher-ts synced 2025-01-08 12:28:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add eden-watcher generated using codegen (#47)

Browse Source 
* Add a watcher for EdenNetwork contract

* Add events from MerkleDistributor contract to eden-watcher

* Add ERC721 Transfer and Approval events to eden-watcher schema

* Add artifacts for DistributorGovernance and MerkleDistributor contracts
This commit is contained in:
prathamesh0 2021-11-16 14:47:03 +05:30 committed by nabarun
parent 47cef8d5d7
commit b7ae655c20
79 changed files with 8336 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
packages/eden-watcher/.eslintignore Normal file
View File

@ -0,0 +1,2 @@
# Don't lint build output.
dist

27
packages/eden-watcher/.eslintrc.json Normal file
View File

@ -0,0 +1,27 @@
{
"env": {
"browser": true,
"es2021": true
},
"extends": [
"semistandard",
"plugin:@typescript-eslint/recommended"
],
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 12,
"sourceType": "module"
},
"plugins": [
"@typescript-eslint"
],
"rules": {
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/explicit-module-boundary-types": [
"warn",
{
"allowArgumentsExplicitlyTypedAsAny": true
}
]
}
}

191
packages/eden-watcher/README.md Normal file
View File

@ -0,0 +1,191 @@
# EdenNetwork Watcher
## Setup
* Run the following command to install required packages:
```bash
yarn
```
* Run the IPFS (go-ipfs version 0.9.0) daemon:
```bash
ipfs daemon
```
* Create a postgres12 database for the watcher:
```bash
sudo su - postgres
createdb eden-watcher
```
* If the watcher is an `active` watcher:
Create database for the job queue and enable the `pgcrypto` extension on them (https://github.com/timgit/pg-boss/blob/master/docs/usage.md#intro):
```
createdb eden-watcher-job-queue
```
```
postgres@tesla:~$ psql -U postgres -h localhost eden-watcher-job-queue
Password for user postgres:
psql (12.7 (Ubuntu 12.7-1.pgdg18.04+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
eden-watcher-job-queue=# CREATE EXTENSION pgcrypto;
CREATE EXTENSION
eden-watcher-job-queue=# exit
```
* In the [config file](./environments/local.toml):
* Update the database connection settings.
* Update the `upstream` config and provide the `ipld-eth-server` GQL API and the `indexer-db` postgraphile endpoints.
* Update the `server` config with state checkpoint settings and provide the IPFS API address.
## Customize
* Indexing on an event:
* Edit the custom hook function `handleEvent` (triggered on an event) in [hooks.ts](./src/hooks.ts) to perform corresponding indexing using the `Indexer` object.
* While using the indexer storage methods for indexing, pass `diff` as true if default state is desired to be generated using the state variables being indexed.
* Generating state:
* Edit the custom hook function `createInitialCheckpoint` (triggered on watch-contract, checkpoint: `true`) in [hooks.ts](./src/hooks.ts) to save an initial checkpoint `IPLDBlock` using the `Indexer` object.
* Edit the custom hook function `createStateDiff` (triggered on a block) in [hooks.ts](./src/hooks.ts) to save the state in a `diff` `IPLDBlock` using the `Indexer` object. The default state (if exists) is updated.
* Edit the custom hook function `createStateCheckpoint` (triggered just before default and CLI checkpoint) in [hooks.ts](./src/hooks.ts) to save the state in a `checkpoint` `IPLDBlock` using the `Indexer` object.
* The existing example hooks in [hooks.ts](./src/hooks.ts) are for an `ERC20` contract.
## Run
* Run the watcher:
```bash
yarn server
```
GQL console: http://localhost:3008/graphql
* If the watcher is an `active` watcher:
* Run the job-runner:
```bash
yarn job-runner
```
* To watch a contract:
```bash
yarn watch:contract --address <contract-address> --kind <contract-kind> --checkpoint <true | false> --starting-block [block-number]
```
* `address`: Address or identifier of the contract to be watched.
* `kind`: Kind of the contract.
* `checkpoint`: Turn checkpointing on (`true` | `false`).
* `starting-block`: Starting block for the contract (default: `1`).
Examples:
Watch a contract with its address and checkpointing on:
```bash
yarn watch:contract --address 0x1F78641644feB8b64642e833cE4AFE93DD6e7833 --kind ERC20 --checkpoint true
```
Watch a contract with its identifier and checkpointing on:
```bash
yarn watch:contract --address MyProtocol --kind protocol --checkpoint true
```
* To fill a block range:
```bash
yarn fill --start-block <from-block> --end-block <to-block>
```
* `start-block`: Block number to start filling from.
* `end-block`: Block number till which to fill.
* To create a checkpoint for a contract:
```bash
yarn checkpoint --address <contract-address> --block-hash [block-hash]
```
* `address`: Address or identifier of the contract for which to create a checkpoint.
* `block-hash`: Hash of a block (in the pruned region) at which to create the checkpoint (default: latest canonical block hash).
* To reset the watcher to a previous block number:
* Reset state:
```bash
yarn reset state --block-number <previous-block-number>
```
* Reset job-queue:
```bash
yarn reset job-queue --block-number <previous-block-number>
```
* `block-number`: Block number to which to reset the watcher.
* To export and import the watcher state:
* In source watcher, export watcher state:
```bash
yarn export-state --export-file [export-file-path]
```
* `export-file`: Path of JSON file to which to export the watcher data.
* In target watcher, run job-runner:
```bash
yarn job-runner
```
* Import watcher state:
```bash
yarn import-state --import-file <import-file-path>
```
* `import-file`: Path of JSON file from which to import the watcher data.
* Run fill:
```bash
yarn fill --start-block <snapshot-block> --end-block <to-block>
```
* `snapshot-block`: Block number at which the watcher state was exported.
* Run server:
```bash
yarn server
```
* To inspect a CID:
```bash
yarn inspect-cid --cid <cid>
```
* `cid`: CID to be inspected.

42
packages/eden-watcher/environments/local.toml Normal file
View File

@ -0,0 +1,42 @@
[server]
host = "127.0.0.1"
port = 3012
kind = "active"
# Checkpointing state.
checkpointing = true
# Checkpoint interval in number of blocks.
checkpointInterval = 2000
# IPFS API address (can be taken from the output on running the IPFS daemon).
ipfsApiAddr = "/ip4/127.0.0.1/tcp/5001"
subgraphPath = "../graph-node/test/subgraph/eden"
[database]
type = "postgres"
host = "localhost"
port = 5432
database = "eden-watcher"
username = "postgres"
password = "postgres"
synchronize = true
logging = false
[upstream]
[upstream.ethServer]
gqlApiEndpoint = "http://127.0.0.1:8082/graphql"
gqlPostgraphileEndpoint = "http://127.0.0.1:5000/graphql"
rpcProviderEndpoint = "http://127.0.0.1:8081"
blockDelayInMilliSecs = 2000
[upstream.cache]
name = "requests"
enabled = false
deleteOnStart = false
[jobQueue]
dbConnectionString = "postgres://postgres:postgres@localhost/eden-watcher-job-queue"
maxCompletionLagInSecs = 300
jobDelayInMilliSecs = 100

69
packages/eden-watcher/package.json Normal file
View File

@ -0,0 +1,69 @@
{
"name": "@vulcanize/eden-watcher",
"version": "0.1.0",
"description": "eden-watcher",
"private": true,
"main": "dist/index.js",
"scripts": {
"lint": "eslint .",
"build": "tsc",
"server": "DEBUG=vulcanize:* ts-node src/server.ts",
"job-runner": "DEBUG=vulcanize:* ts-node src/job-runner.ts",
"watch:contract": "DEBUG=vulcanize:* ts-node src/cli/watch-contract.ts",
"fill": "DEBUG=vulcanize:* ts-node src/fill.ts",
"reset": "DEBUG=vulcanize:* ts-node src/cli/reset.ts",
"checkpoint": "DEBUG=vulcanize:* ts-node src/cli/checkpoint.ts",
"export-state": "DEBUG=vulcanize:* ts-node src/cli/export-state.ts",
"import-state": "DEBUG=vulcanize:* ts-node src/cli/import-state.ts",
"inspect-cid": "DEBUG=vulcanize:* ts-node src/cli/inspect-cid.ts"
},
"repository": {
"type": "git",
"url": "git+https://github.com/vulcanize/watcher-ts.git"
},
"author": "",
"license": "AGPL-3.0",
"bugs": {
"url": "https://github.com/vulcanize/watcher-ts/issues"
},
"homepage": "https://github.com/vulcanize/watcher-ts#readme",
"dependencies": {
"@apollo/client": "^3.3.19",
"@ethersproject/providers": "5.3.0",
"@ipld/dag-cbor": "^6.0.12",
"@vulcanize/cache": "^0.1.0",
"@vulcanize/ipld-eth-client": "^0.1.0",
"@vulcanize/solidity-mapper": "^0.1.0",
"@vulcanize/util": "^0.1.0",
"apollo-server-express": "^2.25.0",
"apollo-type-bigint": "^0.1.3",
"debug": "^4.3.1",
"ethers": "^5.2.0",
"express": "^4.17.1",
"graphql": "^15.5.0",
"graphql-import-node": "^0.0.4",
"ipfs-http-client": "^53.0.1",
"json-bigint": "^1.0.0",
"lodash": "^4.17.21",
"multiformats": "^9.4.8",
"reflect-metadata": "^0.1.13",
"typeorm": "^0.2.32",
"yargs": "^17.0.1"
},
"devDependencies": {
"@ethersproject/abi": "^5.3.0",
"@types/express": "^4.17.11",
"@types/yargs": "^17.0.0",
"@typescript-eslint/eslint-plugin": "^4.25.0",
"@typescript-eslint/parser": "^4.25.0",
"eslint": "^7.27.0",
"eslint-config-semistandard": "^15.0.1",
"eslint-config-standard": "^16.0.3",
"eslint-plugin-import": "^2.23.3",
"eslint-plugin-node": "^11.1.0",
"eslint-plugin-promise": "^5.1.0",
"eslint-plugin-standard": "^5.0.0",
"ts-node": "^10.0.0",
"typescript": "^4.3.2"
}
}

723
packages/eden-watcher/src/artifacts/DistributorGovernance.json Normal file
View File

@ -0,0 +1,723 @@
{
"abi": [
{
"inputs": [
{
"internalType": "address",
"name": "_admin",
"type": "address"
},
{
"internalType": "address[]",
"name": "_blockProducers",
"type": "address[]"
},
{
"internalType": "address[]",
"name": "_collectors",
"type": "address[]"
}
],
"stateMutability": "nonpayable",
"type": "constructor"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "address",
"name": "producer",
"type": "address"
}
],
"name": "BlockProducerAdded",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "address",
"name": "producer",
"type": "address"
}
],
"name": "BlockProducerRemoved",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "address",
"name": "producer",
"type": "address"
},
{
"indexed": true,
"internalType": "address",
"name": "collector",
"type": "address"
}
],
"name": "BlockProducerRewardCollectorChanged",
"type": "event"
},
{
"anonymous": false,
"inputs": [],
"name": "RewardScheduleChanged",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
},
{
"indexed": true,
"internalType": "bytes32",
"name": "previousAdminRole",
"type": "bytes32"
},
{
"indexed": true,
"internalType": "bytes32",
"name": "newAdminRole",
"type": "bytes32"
}
],
"name": "RoleAdminChanged",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
},
{
"indexed": true,
"internalType": "address",
"name": "account",
"type": "address"
},
{
"indexed": true,
"internalType": "address",
"name": "sender",
"type": "address"
}
],
"name": "RoleGranted",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
},
{
"indexed": true,
"internalType": "address",
"name": "account",
"type": "address"
},
{
"indexed": true,
"internalType": "address",
"name": "sender",
"type": "address"
}
],
"name": "RoleRevoked",
"type": "event"
},
{
"inputs": [],
"name": "DEFAULT_ADMIN_ROLE",
"outputs": [
{
"internalType": "bytes32",
"name": "",
"type": "bytes32"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [],
"name": "DELEGATOR_ROLE",
"outputs": [
{
"internalType": "bytes32",
"name": "",
"type": "bytes32"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [],
"name": "GOV_ROLE",
"outputs": [
{
"internalType": "bytes32",
"name": "",
"type": "bytes32"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [],
"name": "REWARD_SCHEDULE_ENTRY_LENGTH",
"outputs": [
{
"internalType": "uint256",
"name": "",
"type": "uint256"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "address",
"name": "producer",
"type": "address"
}
],
"name": "add",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "address[]",
"name": "producers",
"type": "address[]"
}
],
"name": "addBatch",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "address",
"name": "",
"type": "address"
}
],
"name": "blockProducer",
"outputs": [
{
"internalType": "bool",
"name": "",
"type": "bool"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "address",
"name": "producer",
"type": "address"
},
{
"internalType": "address",
"name": "collector",
"type": "address"
}
],
"name": "delegate",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "address[]",
"name": "producers",
"type": "address[]"
},
{
"internalType": "address[]",
"name": "collectors",
"type": "address[]"
}
],
"name": "delegateBatch",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
}
],
"name": "getRoleAdmin",
"outputs": [
{
"internalType": "bytes32",
"name": "",
"type": "bytes32"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
},
{
"internalType": "uint256",
"name": "index",
"type": "uint256"
}
],
"name": "getRoleMember",
"outputs": [
{
"internalType": "address",
"name": "",
"type": "address"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
}
],
"name": "getRoleMemberCount",
"outputs": [
{
"internalType": "uint256",
"name": "",
"type": "uint256"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
},
{
"internalType": "address",
"name": "account",
"type": "address"
}
],
"name": "grantRole",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
},
{
"internalType": "address",
"name": "account",
"type": "address"
}
],
"name": "hasRole",
"outputs": [
{
"internalType": "bool",
"name": "",
"type": "bool"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "address",
"name": "producer",
"type": "address"
}
],
"name": "remove",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "address[]",
"name": "producers",
"type": "address[]"
}
],
"name": "removeBatch",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
},
{
"internalType": "address",
"name": "account",
"type": "address"
}
],
"name": "renounceRole",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "bytes32",
"name": "role",
"type": "bytes32"
},
{
"internalType": "address",
"name": "account",
"type": "address"
}
],
"name": "revokeRole",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "address",
"name": "",
"type": "address"
}
],
"name": "rewardCollector",
"outputs": [
{
"internalType": "address",
"name": "",
"type": "address"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [],
"name": "rewardScheduleEntries",
"outputs": [
{
"internalType": "uint256",
"name": "",
"type": "uint256"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint256",
"name": "index",
"type": "uint256"
}
],
"name": "rewardScheduleEntry",
"outputs": [
{
"components": [
{
"internalType": "uint64",
"name": "startTime",
"type": "uint64"
},
{
"internalType": "uint64",
"name": "epochDuration",
"type": "uint64"
},
{
"internalType": "uint128",
"name": "rewardsPerEpoch",
"type": "uint128"
}
],
"internalType": "struct IGovernance.RewardScheduleEntry",
"name": "",
"type": "tuple"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "bytes",
"name": "set",
"type": "bytes"
}
],
"name": "setRewardSchedule",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "bytes4",
"name": "interfaceId",
"type": "bytes4"
}
],
"name": "supportsInterface",
"outputs": [
{
"internalType": "bool",
"name": "",
"type": "bool"
}
],
"stateMutability": "view",
"type": "function"
}
],
"storageLayout": {
"storage": [
{
"astId": 380,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "_roles",
"offset": 0,
"slot": "0",
"type": "t_mapping(t_bytes32,t_struct(RoleData)375_storage)"
},
{
"astId": 1210,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "_roleMembers",
"offset": 0,
"slot": "1",
"type": "t_mapping(t_bytes32,t_struct(AddressSet)969_storage)"
},
{
"astId": 1721,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "rewardCollector",
"offset": 0,
"slot": "2",
"type": "t_mapping(t_address,t_address)"
},
{
"astId": 1727,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "blockProducer",
"offset": 0,
"slot": "3",
"type": "t_mapping(t_address,t_bool)"
},
{
"astId": 1730,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "_rewardSchedule",
"offset": 0,
"slot": "4",
"type": "t_bytes_storage"
}
],
"types": {
"t_address": {
"encoding": "inplace",
"label": "address",
"numberOfBytes": "20"
},
"t_array(t_bytes32)dyn_storage": {
"base": "t_bytes32",
"encoding": "dynamic_array",
"label": "bytes32[]",
"numberOfBytes": "32"
},
"t_bool": {
"encoding": "inplace",
"label": "bool",
"numberOfBytes": "1"
},
"t_bytes32": {
"encoding": "inplace",
"label": "bytes32",
"numberOfBytes": "32"
},
"t_bytes_storage": {
"encoding": "bytes",
"label": "bytes",
"numberOfBytes": "32"
},
"t_mapping(t_address,t_address)": {
"encoding": "mapping",
"key": "t_address",
"label": "mapping(address => address)",
"numberOfBytes": "32",
"value": "t_address"
},
"t_mapping(t_address,t_bool)": {
"encoding": "mapping",
"key": "t_address",
"label": "mapping(address => bool)",
"numberOfBytes": "32",
"value": "t_bool"
},
"t_mapping(t_bytes32,t_struct(AddressSet)969_storage)": {
"encoding": "mapping",
"key": "t_bytes32",
"label": "mapping(bytes32 => struct EnumerableSet.AddressSet)",
"numberOfBytes": "32",
"value": "t_struct(AddressSet)969_storage"
},
"t_mapping(t_bytes32,t_struct(RoleData)375_storage)": {
"encoding": "mapping",
"key": "t_bytes32",
"label": "mapping(bytes32 => struct AccessControl.RoleData)",
"numberOfBytes": "32",
"value": "t_struct(RoleData)375_storage"
},
"t_mapping(t_bytes32,t_uint256)": {
"encoding": "mapping",
"key": "t_bytes32",
"label": "mapping(bytes32 => uint256)",
"numberOfBytes": "32",
"value": "t_uint256"
},
"t_struct(AddressSet)969_storage": {
"encoding": "inplace",
"label": "struct EnumerableSet.AddressSet",
"members": [
{
"astId": 968,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "_inner",
"offset": 0,
"slot": "0",
"type": "t_struct(Set)698_storage"
}
],
"numberOfBytes": "64"
},
"t_struct(RoleData)375_storage": {
"encoding": "inplace",
"label": "struct AccessControl.RoleData",
"members": [
{
"astId": 372,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "members",
"offset": 0,
"slot": "0",
"type": "t_mapping(t_address,t_bool)"
},
{
"astId": 374,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "adminRole",
"offset": 0,
"slot": "1",
"type": "t_bytes32"
}
],
"numberOfBytes": "64"
},
"t_struct(Set)698_storage": {
"encoding": "inplace",
"label": "struct EnumerableSet.Set",
"members": [
{
"astId": 693,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "_values",
"offset": 0,
"slot": "0",
"type": "t_array(t_bytes32)dyn_storage"
},
{
"astId": 697,
"contract": "DistributorGovernance.sol:DistributorGovernance",
"label": "_indexes",
"offset": 0,
"slot": "1",
"type": "t_mapping(t_bytes32,t_uint256)"
}
],
"numberOfBytes": "64"
},
"t_uint256": {
"encoding": "inplace",
"label": "uint256",
"numberOfBytes": "32"
}
}
}
}

942
packages/eden-watcher/src/artifacts/EdenNetwork.json Normal file
View File

@ -0,0 +1,942 @@
{
"abi": [
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "address",
"name": "newAdmin",
"type": "address"
},
{
"indexed": true,
"internalType": "address",
"name": "oldAdmin",
"type": "address"
}
],
"name": "AdminUpdated",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "uint8",
"name": "slot",
"type": "uint8"
},
{
"indexed": true,
"internalType": "address",
"name": "owner",
"type": "address"
},
{
"indexed": true,
"internalType": "address",
"name": "delegate",
"type": "address"
},
{
"indexed": false,
"internalType": "uint128",
"name": "newBidAmount",
"type": "uint128"
},
{
"indexed": false,
"internalType": "uint128",
"name": "oldBidAmount",
"type": "uint128"
},
{
"indexed": false,
"internalType": "uint16",
"name": "taxNumerator",
"type": "uint16"
},
{
"indexed": false,
"internalType": "uint16",
"name": "taxDenominator",
"type": "uint16"
}
],
"name": "SlotClaimed",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "uint8",
"name": "slot",
"type": "uint8"
},
{
"indexed": true,
"internalType": "address",
"name": "owner",
"type": "address"
},
{
"indexed": true,
"internalType": "address",
"name": "newDelegate",
"type": "address"
},
{
"indexed": false,
"internalType": "address",
"name": "oldDelegate",
"type": "address"
}
],
"name": "SlotDelegateUpdated",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "address",
"name": "staker",
"type": "address"
},
{
"indexed": false,
"internalType": "uint256",
"name": "stakeAmount",
"type": "uint256"
}
],
"name": "Stake",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": false,
"internalType": "uint16",
"name": "newNumerator",
"type": "uint16"
},
{
"indexed": false,
"internalType": "uint16",
"name": "newDenominator",
"type": "uint16"
},
{
"indexed": false,
"internalType": "uint16",
"name": "oldNumerator",
"type": "uint16"
},
{
"indexed": false,
"internalType": "uint16",
"name": "oldDenominator",
"type": "uint16"
}
],
"name": "TaxRateUpdated",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "address",
"name": "staker",
"type": "address"
},
{
"indexed": false,
"internalType": "uint256",
"name": "unstakedAmount",
"type": "uint256"
}
],
"name": "Unstake",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"internalType": "address",
"name": "withdrawer",
"type": "address"
},
{
"indexed": false,
"internalType": "uint256",
"name": "withdrawalAmount",
"type": "uint256"
}
],
"name": "Withdraw",
"type": "event"
},
{
"inputs": [],
"name": "MIN_BID",
"outputs": [
{
"internalType": "uint128",
"name": "",
"type": "uint128"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [],
"name": "admin",
"outputs": [
{
"internalType": "address",
"name": "",
"type": "address"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "slot",
"type": "uint8"
},
{
"internalType": "uint128",
"name": "bid",
"type": "uint128"
},
{
"internalType": "address",
"name": "delegate",
"type": "address"
}
],
"name": "claimSlot",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "slot",
"type": "uint8"
},
{
"internalType": "uint128",
"name": "bid",
"type": "uint128"
},
{
"internalType": "address",
"name": "delegate",
"type": "address"
},
{
"internalType": "uint256",
"name": "deadline",
"type": "uint256"
},
{
"internalType": "uint8",
"name": "v",
"type": "uint8"
},
{
"internalType": "bytes32",
"name": "r",
"type": "bytes32"
},
{
"internalType": "bytes32",
"name": "s",
"type": "bytes32"
}
],
"name": "claimSlotWithPermit",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "contract IERC20Extended",
"name": "_token",
"type": "address"
},
{
"internalType": "contract ILockManager",
"name": "_lockManager",
"type": "address"
},
{
"internalType": "address",
"name": "_admin",
"type": "address"
},
{
"internalType": "uint16",
"name": "_taxNumerator",
"type": "uint16"
},
{
"internalType": "uint16",
"name": "_taxDenominator",
"type": "uint16"
}
],
"name": "initialize",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [],
"name": "lockManager",
"outputs": [
{
"internalType": "contract ILockManager",
"name": "",
"type": "address"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "address",
"name": "",
"type": "address"
}
],
"name": "lockedBalance",
"outputs": [
{
"internalType": "uint128",
"name": "",
"type": "uint128"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "address",
"name": "newAdmin",
"type": "address"
}
],
"name": "setAdmin",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "slot",
"type": "uint8"
},
{
"internalType": "address",
"name": "delegate",
"type": "address"
}
],
"name": "setSlotDelegate",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint16",
"name": "numerator",
"type": "uint16"
},
{
"internalType": "uint16",
"name": "denominator",
"type": "uint16"
}
],
"name": "setTaxRate",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "slot",
"type": "uint8"
}
],
"name": "slotBalance",
"outputs": [
{
"internalType": "uint128",
"name": "balance",
"type": "uint128"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "",
"type": "uint8"
}
],
"name": "slotBid",
"outputs": [
{
"internalType": "address",
"name": "bidder",
"type": "address"
},
{
"internalType": "uint16",
"name": "taxNumerator",
"type": "uint16"
},
{
"internalType": "uint16",
"name": "taxDenominator",
"type": "uint16"
},
{
"internalType": "uint64",
"name": "periodStart",
"type": "uint64"
},
{
"internalType": "uint128",
"name": "bidAmount",
"type": "uint128"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "slot",
"type": "uint8"
}
],
"name": "slotCost",
"outputs": [
{
"internalType": "uint128",
"name": "",
"type": "uint128"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "slot",
"type": "uint8"
}
],
"name": "slotDelegate",
"outputs": [
{
"internalType": "address",
"name": "",
"type": "address"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "",
"type": "uint8"
}
],
"name": "slotExpiration",
"outputs": [
{
"internalType": "uint64",
"name": "",
"type": "uint64"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "slot",
"type": "uint8"
}
],
"name": "slotForeclosed",
"outputs": [
{
"internalType": "bool",
"name": "",
"type": "bool"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint8",
"name": "slot",
"type": "uint8"
}
],
"name": "slotOwner",
"outputs": [
{
"internalType": "address",
"name": "",
"type": "address"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint128",
"name": "amount",
"type": "uint128"
}
],
"name": "stake",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint128",
"name": "amount",
"type": "uint128"
},
{
"internalType": "uint256",
"name": "deadline",
"type": "uint256"
},
{
"internalType": "uint8",
"name": "v",
"type": "uint8"
},
{
"internalType": "bytes32",
"name": "r",
"type": "bytes32"
},
{
"internalType": "bytes32",
"name": "s",
"type": "bytes32"
}
],
"name": "stakeWithPermit",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "address",
"name": "",
"type": "address"
}
],
"name": "stakedBalance",
"outputs": [
{
"internalType": "uint128",
"name": "",
"type": "uint128"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [],
"name": "taxDenominator",
"outputs": [
{
"internalType": "uint16",
"name": "",
"type": "uint16"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [],
"name": "taxNumerator",
"outputs": [
{
"internalType": "uint16",
"name": "",
"type": "uint16"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [],
"name": "token",
"outputs": [
{
"internalType": "contract IERC20Extended",
"name": "",
"type": "address"
}
],
"stateMutability": "view",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint128",
"name": "amount",
"type": "uint128"
}
],
"name": "unstake",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
},
{
"inputs": [
{
"internalType": "uint128",
"name": "amount",
"type": "uint128"
}
],
"name": "withdraw",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
}
],
"storageLayout": {
"storage": [
{
"astId": 322,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "_initialized",
"offset": 0,
"slot": "0",
"type": "t_bool"
},
{
"astId": 325,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "_initializing",
"offset": 1,
"slot": "0",
"type": "t_bool"
},
{
"astId": 381,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "slotExpiration",
"offset": 0,
"slot": "1",
"type": "t_mapping(t_uint8,t_uint64)"
},
{
"astId": 386,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "_slotDelegate",
"offset": 0,
"slot": "2",
"type": "t_mapping(t_uint8,t_address)"
},
{
"astId": 391,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "_slotOwner",
"offset": 0,
"slot": "3",
"type": "t_mapping(t_uint8,t_address)"
},
{
"astId": 397,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "slotBid",
"offset": 0,
"slot": "4",
"type": "t_mapping(t_uint8,t_struct(Bid)376_storage)"
},
{
"astId": 402,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "stakedBalance",
"offset": 0,
"slot": "5",
"type": "t_mapping(t_address,t_uint128)"
},
{
"astId": 407,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "lockedBalance",
"offset": 0,
"slot": "6",
"type": "t_mapping(t_address,t_uint128)"
},
{
"astId": 411,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "token",
"offset": 0,
"slot": "7",
"type": "t_contract(IERC20Extended)318"
},
{
"astId": 415,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "lockManager",
"offset": 0,
"slot": "8",
"type": "t_contract(ILockManager)133"
},
{
"astId": 418,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "admin",
"offset": 0,
"slot": "9",
"type": "t_address"
},
{
"astId": 421,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "taxNumerator",
"offset": 20,
"slot": "9",
"type": "t_uint16"
},
{
"astId": 424,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "taxDenominator",
"offset": 22,
"slot": "9",
"type": "t_uint16"
},
{
"astId": 427,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "MIN_BID",
"offset": 0,
"slot": "10",
"type": "t_uint128"
},
{
"astId": 430,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "_NOT_ENTERED",
"offset": 0,
"slot": "11",
"type": "t_uint256"
},
{
"astId": 433,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "_ENTERED",
"offset": 0,
"slot": "12",
"type": "t_uint256"
},
{
"astId": 436,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "_status",
"offset": 0,
"slot": "13",
"type": "t_uint256"
}
],
"types": {
"t_address": {
"encoding": "inplace",
"label": "address",
"numberOfBytes": "20"
},
"t_bool": {
"encoding": "inplace",
"label": "bool",
"numberOfBytes": "1"
},
"t_contract(IERC20Extended)318": {
"encoding": "inplace",
"label": "contract IERC20Extended",
"numberOfBytes": "20"
},
"t_contract(ILockManager)133": {
"encoding": "inplace",
"label": "contract ILockManager",
"numberOfBytes": "20"
},
"t_mapping(t_address,t_uint128)": {
"encoding": "mapping",
"key": "t_address",
"label": "mapping(address => uint128)",
"numberOfBytes": "32",
"value": "t_uint128"
},
"t_mapping(t_uint8,t_address)": {
"encoding": "mapping",
"key": "t_uint8",
"label": "mapping(uint8 => address)",
"numberOfBytes": "32",
"value": "t_address"
},
"t_mapping(t_uint8,t_struct(Bid)376_storage)": {
"encoding": "mapping",
"key": "t_uint8",
"label": "mapping(uint8 => struct EdenNetwork.Bid)",
"numberOfBytes": "32",
"value": "t_struct(Bid)376_storage"
},
"t_mapping(t_uint8,t_uint64)": {
"encoding": "mapping",
"key": "t_uint8",
"label": "mapping(uint8 => uint64)",
"numberOfBytes": "32",
"value": "t_uint64"
},
"t_struct(Bid)376_storage": {
"encoding": "inplace",
"label": "struct EdenNetwork.Bid",
"members": [
{
"astId": 367,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "bidder",
"offset": 0,
"slot": "0",
"type": "t_address"
},
{
"astId": 369,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "taxNumerator",
"offset": 20,
"slot": "0",
"type": "t_uint16"
},
{
"astId": 371,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "taxDenominator",
"offset": 22,
"slot": "0",
"type": "t_uint16"
},
{
"astId": 373,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "periodStart",
"offset": 24,
"slot": "0",
"type": "t_uint64"
},
{
"astId": 375,
"contract": "EdenNetwork.sol:EdenNetwork",
"label": "bidAmount",
"offset": 0,
"slot": "1",
"type": "t_uint128"
}
],
"numberOfBytes": "64"
},
"t_uint128": {
"encoding": "inplace",
"label": "uint128",
"numberOfBytes": "16"
},
"t_uint16": {
"encoding": "inplace",
"label": "uint16",
"numberOfBytes": "2"
},
"t_uint256": {
"encoding": "inplace",
"label": "uint256",
"numberOfBytes": "32"
},
"t_uint64": {
"encoding": "inplace",
"label": "uint64",
"numberOfBytes": "8"
},
"t_uint8": {
"encoding": "inplace",
"label": "uint8",
"numberOfBytes": "1"
}
}
}
}

1482
packages/eden-watcher/src/artifacts/MerkleDistributor.json Normal file
View File

File diff suppressed because it is too large Load Diff

69
packages/eden-watcher/src/cli/checkpoint.ts Normal file
View File

@ -0,0 +1,69 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import path from 'path';
import yargs from 'yargs';
import 'reflect-metadata';
import debug from 'debug';
import { Config, DEFAULT_CONFIG_PATH, getConfig, initClients } from '@vulcanize/util';
import { GraphWatcher, Database as GraphDatabase } from '@vulcanize/graph-node';
import { Database } from '../database';
import { Indexer } from '../indexer';
const log = debug('vulcanize:checkpoint');
const main = async (): Promise<void> => {
const argv = await yargs.parserConfiguration({
'parse-numbers': false
}).options({
configFile: {
alias: 'f',
type: 'string',
require: true,
demandOption: true,
describe: 'Configuration file path (toml)',
default: DEFAULT_CONFIG_PATH
},
address: {
type: 'string',
require: true,
demandOption: true,
describe: 'Contract address to create the checkpoint for.'
},
blockHash: {
type: 'string',
describe: 'Blockhash at which to create the checkpoint.'
}
}).argv;
const config: Config = await getConfig(argv.configFile);
const { ethClient, postgraphileClient, ethProvider } = await initClients(config);
const db = new Database(config.database);
await db.init();
const graphDb = new GraphDatabase(config.database, path.resolve(__dirname, 'entity/*'));
await graphDb.init();
const graphWatcher = new GraphWatcher(graphDb, postgraphileClient, config.server.subgraphPath);
const indexer = new Indexer(config.server, db, ethClient, postgraphileClient, ethProvider, graphWatcher);
graphWatcher.setIndexer(indexer);
await graphWatcher.init();
const blockHash = await indexer.processCLICheckpoint(argv.address, argv.blockHash);
log(`Created a checkpoint for contract ${argv.address} at block-hash ${blockHash}`);
await db.close();
};
main().catch(err => {
log(err);
}).finally(() => {
process.exit(0);
});

119
packages/eden-watcher/src/cli/export-state.ts Normal file
View File

@ -0,0 +1,119 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import assert from 'assert';
import yargs from 'yargs';
import 'reflect-metadata';
import debug from 'debug';
import fs from 'fs';
import path from 'path';
import { Config, DEFAULT_CONFIG_PATH, getConfig, initClients } from '@vulcanize/util';
import { GraphWatcher, Database as GraphDatabase } from '@vulcanize/graph-node';
import * as codec from '@ipld/dag-cbor';
import { Database } from '../database';
import { Indexer } from '../indexer';
const log = debug('vulcanize:export-state');
const main = async (): Promise<void> => {
const argv = await yargs.parserConfiguration({
'parse-numbers': false
}).options({
configFile: {
alias: 'f',
type: 'string',
require: true,
demandOption: true,
describe: 'Configuration file path (toml)',
default: DEFAULT_CONFIG_PATH
},
exportFile: {
alias: 'o',
type: 'string',
describe: 'Export file path'
}
}).argv;
const config: Config = await getConfig(argv.configFile);
const { ethClient, postgraphileClient, ethProvider } = await initClients(config);
const db = new Database(config.database);
await db.init();
const graphDb = new GraphDatabase(config.database, path.resolve(__dirname, 'entity/*'));
await graphDb.init();
const graphWatcher = new GraphWatcher(graphDb, postgraphileClient, config.server.subgraphPath);
const indexer = new Indexer(config.server, db, ethClient, postgraphileClient, ethProvider, graphWatcher);
graphWatcher.setIndexer(indexer);
await graphWatcher.init();
const exportData: any = {
snapshotBlock: {},
contracts: [],
ipldCheckpoints: []
};
const contracts = await db.getContracts({});
// Get latest canonical block.
const block = await indexer.getLatestCanonicalBlock();
assert(block);
// Export snapshot block.
exportData.snapshotBlock = {
blockNumber: block.blockNumber,
blockHash: block.blockHash
};
// Export contracts and checkpoints.
for (const contract of contracts) {
exportData.contracts.push({
address: contract.address,
kind: contract.kind,
checkpoint: contract.checkpoint,
startingBlock: block.blockNumber
});
// Create and export checkpoint if checkpointing is on for the contract.
if (contract.checkpoint) {
await indexer.createCheckpoint(contract.address, block.blockHash);
const ipldBlock = await indexer.getLatestIPLDBlock(contract.address, 'checkpoint', block.blockNumber);
assert(ipldBlock);
const data = codec.decode(Buffer.from(ipldBlock.data)) as any;
exportData.ipldCheckpoints.push({
contractAddress: ipldBlock.contractAddress,
cid: ipldBlock.cid,
kind: ipldBlock.kind,
data
});
}
}
if (argv.exportFile) {
const encodedExportData = codec.encode(exportData);
const filePath = path.resolve(argv.exportFile);
const fileDir = path.dirname(filePath);
if (!fs.existsSync(fileDir)) fs.mkdirSync(fileDir, { recursive: true });
fs.writeFileSync(filePath, encodedExportData);
} else {
log(exportData);
}
};
main().catch(err => {
log(err);
}).finally(() => {
process.exit(0);
});

118
packages/eden-watcher/src/cli/import-state.ts Normal file
View File

@ -0,0 +1,118 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import assert from 'assert';
import 'reflect-metadata';
import yargs from 'yargs';
import { hideBin } from 'yargs/helpers';
import debug from 'debug';
import { PubSub } from 'apollo-server-express';
import fs from 'fs';
import path from 'path';
import { getConfig, fillBlocks, JobQueue, DEFAULT_CONFIG_PATH, Config, initClients } from '@vulcanize/util';
import { GraphWatcher, Database as GraphDatabase } from '@vulcanize/graph-node';
import * as codec from '@ipld/dag-cbor';
import { Database } from '../database';
import { Indexer } from '../indexer';
import { EventWatcher } from '../events';
import { IPLDBlock } from '../entity/IPLDBlock';
const log = debug('vulcanize:import-state');
export const main = async (): Promise<any> => {
const argv = await yargs(hideBin(process.argv)).parserConfiguration({
'parse-numbers': false
}).options({
configFile: {
alias: 'f',
type: 'string',
demandOption: true,
describe: 'configuration file path (toml)',
default: DEFAULT_CONFIG_PATH
},
importFile: {
alias: 'i',
type: 'string',
demandOption: true,
describe: 'Import file path (JSON)'
}
}).argv;
const config: Config = await getConfig(argv.configFile);
const { ethClient, postgraphileClient, ethProvider } = await initClients(config);
const db = new Database(config.database);
await db.init();
const graphDb = new GraphDatabase(config.database, path.resolve(__dirname, 'entity/*'));
await graphDb.init();
const graphWatcher = new GraphWatcher(graphDb, postgraphileClient, config.server.subgraphPath);
// Note: In-memory pubsub works fine for now, as each watcher is a single process anyway.
// Later: https://www.apollographql.com/docs/apollo-server/data/subscriptions/#production-pubsub-libraries
const pubsub = new PubSub();
const indexer = new Indexer(config.server, db, ethClient, postgraphileClient, ethProvider, graphWatcher);
graphWatcher.setIndexer(indexer);
await graphWatcher.init();
const jobQueueConfig = config.jobQueue;
assert(jobQueueConfig, 'Missing job queue config');
const { dbConnectionString, maxCompletionLagInSecs } = jobQueueConfig;
assert(dbConnectionString, 'Missing job queue db connection string');
const jobQueue = new JobQueue({ dbConnectionString, maxCompletionLag: maxCompletionLagInSecs });
await jobQueue.start();
const eventWatcher = new EventWatcher(config.upstream, ethClient, postgraphileClient, indexer, pubsub, jobQueue);
// Import data.
const importFilePath = path.resolve(argv.importFile);
const encodedImportData = fs.readFileSync(importFilePath);
const importData = codec.decode(Buffer.from(encodedImportData)) as any;
// Fill the snapshot block.
await fillBlocks(
jobQueue,
indexer,
postgraphileClient,
eventWatcher,
config.upstream.ethServer.blockDelayInMilliSecs,
{
startBlock: importData.snapshotBlock.blockNumber,
endBlock: importData.snapshotBlock.blockNumber
}
);
// Fill the Contracts.
for (const contract of importData.contracts) {
await db.saveContract(contract.address, contract.kind, contract.checkpoint, contract.startingBlock);
}
// Get the snapshot block.
const block = await indexer.getBlockProgress(importData.snapshotBlock.blockHash);
assert(block);
// Fill the IPLDBlocks.
for (const checkpoint of importData.ipldCheckpoints) {
let ipldBlock = new IPLDBlock();
ipldBlock = Object.assign(ipldBlock, checkpoint);
ipldBlock.block = block;
ipldBlock.data = Buffer.from(codec.encode(ipldBlock.data));
await db.saveOrUpdateIPLDBlock(ipldBlock);
}
};
main().catch(err => {
log(err);
}).finally(() => {
process.exit(0);
});

68
packages/eden-watcher/src/cli/inspect-cid.ts Normal file
View File

@ -0,0 +1,68 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import path from 'path';
import assert from 'assert';
import yargs from 'yargs';
import 'reflect-metadata';
import debug from 'debug';
import util from 'util';
import { Config, DEFAULT_CONFIG_PATH, getConfig, initClients } from '@vulcanize/util';
import { GraphWatcher, Database as GraphDatabase } from '@vulcanize/graph-node';
import { Database } from '../database';
import { Indexer } from '../indexer';
const log = debug('vulcanize:inspect-cid');
const main = async (): Promise<void> => {
const argv = await yargs.parserConfiguration({
'parse-numbers': false
}).options({
configFile: {
alias: 'f',
type: 'string',
require: true,
demandOption: true,
describe: 'Configuration file path (toml)',
default: DEFAULT_CONFIG_PATH
},
cid: {
alias: 'c',
type: 'string',
demandOption: true,
describe: 'CID to be inspected'
}
}).argv;
const config: Config = await getConfig(argv.configFile);
const { ethClient, postgraphileClient, ethProvider } = await initClients(config);
const db = new Database(config.database);
await db.init();
const graphDb = new GraphDatabase(config.database, path.resolve(__dirname, 'entity/*'));
await graphDb.init();
const graphWatcher = new GraphWatcher(graphDb, postgraphileClient, config.server.subgraphPath);
const indexer = new Indexer(config.server, db, ethClient, postgraphileClient, ethProvider, graphWatcher);
graphWatcher.setIndexer(indexer);
await graphWatcher.init();
const ipldBlock = await indexer.getIPLDBlockByCid(argv.cid);
assert(ipldBlock, 'IPLDBlock for the provided CID doesn\'t exist.');
const ipldData = await indexer.getIPLDData(ipldBlock);
log(util.inspect(ipldData, false, null));
};
main().catch(err => {
log(err);
}).finally(() => {
process.exit(0);
});

22
packages/eden-watcher/src/cli/reset-cmds/job-queue.ts Normal file
View File

@ -0,0 +1,22 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import debug from 'debug';
import { getConfig, resetJobs } from '@vulcanize/util';
const log = debug('vulcanize:reset-job-queue');
export const command = 'job-queue';
export const desc = 'Reset job queue';
export const builder = {};
export const handler = async (argv: any): Promise<void> => {
const config = await getConfig(argv.configFile);
await resetJobs(config);
log('Job queue reset successfully');
};

91
packages/eden-watcher/src/cli/reset-cmds/state.ts Normal file
View File

@ -0,0 +1,91 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import path from 'path';
import debug from 'debug';
import { MoreThan } from 'typeorm';
import assert from 'assert';
import { getConfig, initClients, resetJobs } from '@vulcanize/util';
import { GraphWatcher, Database as GraphDatabase } from '@vulcanize/graph-node';
import { Database } from '../../database';
import { Indexer } from '../../indexer';
import { BlockProgress } from '../../entity/BlockProgress';
const log = debug('vulcanize:reset-state');
export const command = 'state';
export const desc = 'Reset state to block number';
export const builder = {
blockNumber: {
type: 'number'
}
};
export const handler = async (argv: any): Promise<void> => {
const config = await getConfig(argv.configFile);
await resetJobs(config);
const { ethClient, postgraphileClient, ethProvider } = await initClients(config);
// Initialize database.
const db = new Database(config.database);
await db.init();
const graphDb = new GraphDatabase(config.database, path.resolve(__dirname, 'entity/*'));
await graphDb.init();
const graphWatcher = new GraphWatcher(graphDb, postgraphileClient, config.server.subgraphPath);
const indexer = new Indexer(config.server, db, ethClient, postgraphileClient, ethProvider, graphWatcher);
graphWatcher.setIndexer(indexer);
await graphWatcher.init();
const syncStatus = await indexer.getSyncStatus();
assert(syncStatus, 'Missing syncStatus');
const hooksStatus = await indexer.getHookStatus();
assert(hooksStatus, 'Missing hooksStatus');
const blockProgresses = await indexer.getBlocksAtHeight(argv.blockNumber, false);
assert(blockProgresses.length, `No blocks at specified block number ${argv.blockNumber}`);
assert(!blockProgresses.some(block => !block.isComplete), `Incomplete block at block number ${argv.blockNumber} with unprocessed events`);
const [blockProgress] = blockProgresses;
const dbTx = await db.createTransactionRunner();
try {
const entities = [BlockProgress];
const removeEntitiesPromise = entities.map(async entityClass => {
return db.removeEntities<any>(dbTx, entityClass, { blockNumber: MoreThan(argv.blockNumber) });
});
await Promise.all(removeEntitiesPromise);
if (syncStatus.latestIndexedBlockNumber > blockProgress.blockNumber) {
await indexer.updateSyncStatusIndexedBlock(blockProgress.blockHash, blockProgress.blockNumber, true);
}
if (syncStatus.latestCanonicalBlockNumber > blockProgress.blockNumber) {
await indexer.updateSyncStatusCanonicalBlock(blockProgress.blockHash, blockProgress.blockNumber, true);
}
if (hooksStatus.latestProcessedBlockNumber > blockProgress.blockNumber) {
await indexer.updateHookStatusProcessedBlock(blockProgress.blockNumber, true);
}
dbTx.commitTransaction();
} catch (error) {
await dbTx.rollbackTransaction();
throw error;
} finally {
await dbTx.release();
}
log('Reset state successfully');
};

24
packages/eden-watcher/src/cli/reset.ts Normal file
View File

@ -0,0 +1,24 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import 'reflect-metadata';
import debug from 'debug';
import { getResetYargs } from '@vulcanize/util';
const log = debug('vulcanize:reset');
const main = async () => {
return getResetYargs()
.commandDir('reset-cmds', { extensions: ['ts', 'js'], exclude: /([a-zA-Z0-9\s_\\.\-:])+(.d.ts)$/ })
.demandCommand(1)
.help()
.argv;
};
main().then(() => {
process.exit();
}).catch(err => {
log(err);
});

79
packages/eden-watcher/src/cli/watch-contract.ts Normal file
View File

@ -0,0 +1,79 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import path from 'path';
import yargs from 'yargs';
import 'reflect-metadata';
import debug from 'debug';
import { Config, DEFAULT_CONFIG_PATH, getConfig, initClients } from '@vulcanize/util';
import { GraphWatcher, Database as GraphDatabase } from '@vulcanize/graph-node';
import { Database } from '../database';
import { Indexer } from '../indexer';
const log = debug('vulcanize:watch-contract');
const main = async (): Promise<void> => {
const argv = await yargs.parserConfiguration({
'parse-numbers': false
}).options({
configFile: {
alias: 'f',
type: 'string',
require: true,
demandOption: true,
describe: 'Configuration file path (toml)',
default: DEFAULT_CONFIG_PATH
},
address: {
type: 'string',
require: true,
demandOption: true,
describe: 'Address of the deployed contract'
},
kind: {
type: 'string',
require: true,
demandOption: true,
describe: 'Kind of contract'
},
checkpoint: {
type: 'boolean',
require: true,
demandOption: true,
describe: 'Turn checkpointing on'
},
startingBlock: {
type: 'number',
describe: 'Starting block'
}
}).argv;
const config: Config = await getConfig(argv.configFile);
const { ethClient, postgraphileClient, ethProvider } = await initClients(config);
const db = new Database(config.database);
await db.init();
const graphDb = new GraphDatabase(config.database, path.resolve(__dirname, 'entity/*'));
await graphDb.init();
const graphWatcher = new GraphWatcher(graphDb, postgraphileClient, config.server.subgraphPath);
const indexer = new Indexer(config.server, db, ethClient, postgraphileClient, ethProvider, graphWatcher);
graphWatcher.setIndexer(indexer);
await graphWatcher.init();
await indexer.watchContract(argv.address, argv.kind, argv.checkpoint, argv.startingBlock);
await db.close();
};
main().catch(err => {
log(err);
}).finally(() => {
process.exit(0);
});

55
packages/eden-watcher/src/client.ts Normal file
View File

@ -0,0 +1,55 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { gql } from '@apollo/client/core';
import { GraphQLClient, GraphQLConfig } from '@vulcanize/ipld-eth-client';
import { queries, mutations, subscriptions } from './gql';
export class Client {
_config: GraphQLConfig;
_client: GraphQLClient;
constructor (config: GraphQLConfig) {
this._config = config;
this._client = new GraphQLClient(config);
}
async getEvents (blockHash: string, contractAddress: string, name: string): Promise<any> {
const { events } = await this._client.query(
gql(queries.events),
{ blockHash, contractAddress, name }
);
return events;
}
async getEventsInRange (fromBlockNumber: number, toBlockNumber: number): Promise<any> {
const { eventsInRange } = await this._client.query(
gql(queries.eventsInRange),
{ fromBlockNumber, toBlockNumber }
);
return eventsInRange;
}
async watchContract (contractAddress: string, startingBlock?: number): Promise<any> {
const { watchContract } = await this._client.mutate(
gql(mutations.watchContract),
{ contractAddress, startingBlock }
);
return watchContract;
}
async watchEvents (onNext: (value: any) => void): Promise<ZenObservable.Subscription> {
return this._client.subscribe(
gql(subscriptions.onEvent),
({ data }) => {
onNext(data.onEvent);
}
);
}
}

313
packages/eden-watcher/src/database.ts Normal file
View File

@ -0,0 +1,313 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import assert from 'assert';
import { Connection, ConnectionOptions, DeepPartial, FindConditions, QueryRunner, FindManyOptions, MoreThan } from 'typeorm';
import path from 'path';
import { Database as BaseDatabase, MAX_REORG_DEPTH, DatabaseInterface } from '@vulcanize/util';
import { Contract } from './entity/Contract';
import { Event } from './entity/Event';
import { SyncStatus } from './entity/SyncStatus';
import { HookStatus } from './entity/HookStatus';
import { BlockProgress } from './entity/BlockProgress';
import { IPLDBlock } from './entity/IPLDBlock';
export class Database implements DatabaseInterface {
_config: ConnectionOptions;
_conn!: Connection;
_baseDatabase: BaseDatabase;
constructor (config: ConnectionOptions) {
assert(config);
this._config = {
...config,
entities: [path.join(__dirname, 'entity/*')]
};
this._baseDatabase = new BaseDatabase(this._config);
}
async init (): Promise<void> {
this._conn = await this._baseDatabase.init();
}
async close (): Promise<void> {
return this._baseDatabase.close();
}
async getIPLDBlocks (where: FindConditions<IPLDBlock>): Promise<IPLDBlock[]> {
const repo = this._conn.getRepository(IPLDBlock);
return repo.find({ where, relations: ['block'] });
}
async getLatestIPLDBlock (contractAddress: string, kind: string | null, blockNumber?: number): Promise<IPLDBlock | undefined> {
const repo = this._conn.getRepository(IPLDBlock);
let queryBuilder = repo.createQueryBuilder('ipld_block')
.leftJoinAndSelect('ipld_block.block', 'block')
.where('block.is_pruned = false')
.andWhere('ipld_block.contract_address = :contractAddress', { contractAddress })
.orderBy('block.block_number', 'DESC');
// Filter out blocks after the provided block number.
if (blockNumber) {
queryBuilder.andWhere('block.block_number <= :blockNumber', { blockNumber });
}
// Filter using kind if specified else order by id to give preference to checkpoint.
queryBuilder = kind
? queryBuilder.andWhere('ipld_block.kind = :kind', { kind })
: queryBuilder.andWhere('ipld_block.kind != :kind', { kind: 'diff_staged' })
.addOrderBy('ipld_block.id', 'DESC');
return queryBuilder.getOne();
}
async getPrevIPLDBlock (queryRunner: QueryRunner, blockHash: string, contractAddress: string, kind?: string): Promise<IPLDBlock | undefined> {
const heirerchicalQuery = `
WITH RECURSIVE cte_query AS
(
SELECT
b.block_hash,
b.block_number,
b.parent_hash,
1 as depth,
i.id,
i.kind
FROM
block_progress b
LEFT JOIN
ipld_block i ON i.block_id = b.id
AND i.contract_address = $2
WHERE
b.block_hash = $1
UNION ALL
SELECT
b.block_hash,
b.block_number,
b.parent_hash,
c.depth + 1,
i.id,
i.kind
FROM
block_progress b
LEFT JOIN
ipld_block i
ON i.block_id = b.id
AND i.contract_address = $2
INNER JOIN
cte_query c ON c.parent_hash = b.block_hash
WHERE
c.depth < $3
)
SELECT
block_number, id, kind
FROM
cte_query
ORDER BY block_number DESC, id DESC
`;
// Fetching block and id for previous IPLDBlock in frothy region.
const queryResult = await queryRunner.query(heirerchicalQuery, [blockHash, contractAddress, MAX_REORG_DEPTH]);
const latestRequiredResult = kind
? queryResult.find((obj: any) => obj.kind === kind)
: queryResult.find((obj: any) => obj.id);
let result: IPLDBlock | undefined;
if (latestRequiredResult) {
result = await queryRunner.manager.findOne(IPLDBlock, { id: latestRequiredResult.id }, { relations: ['block'] });
} else {
// If IPLDBlock not found in frothy region get latest IPLDBlock in the pruned region.
// Filter out IPLDBlocks from pruned blocks.
const canonicalBlockNumber = queryResult.pop().block_number + 1;
let queryBuilder = queryRunner.manager.createQueryBuilder(IPLDBlock, 'ipld_block')
.leftJoinAndSelect('ipld_block.block', 'block')
.where('block.is_pruned = false')
.andWhere('ipld_block.contract_address = :contractAddress', { contractAddress })
.andWhere('block.block_number <= :canonicalBlockNumber', { canonicalBlockNumber })
.orderBy('block.block_number', 'DESC');
// Filter using kind if specified else order by id to give preference to checkpoint.
queryBuilder = kind
? queryBuilder.andWhere('ipld_block.kind = :kind', { kind })
: queryBuilder.addOrderBy('ipld_block.id', 'DESC');
result = await queryBuilder.getOne();
}
return result;
}
// Fetch all diff IPLDBlocks after the specified checkpoint.
async getDiffIPLDBlocksByCheckpoint (contractAddress: string, checkpointBlockNumber: number): Promise<IPLDBlock[]> {
const repo = this._conn.getRepository(IPLDBlock);
return repo.find({
relations: ['block'],
where: {
contractAddress,
kind: 'diff',
block: {
isPruned: false,
blockNumber: MoreThan(checkpointBlockNumber)
}
},
order: {
block: 'ASC'
}
});
}
async saveOrUpdateIPLDBlock (ipldBlock: IPLDBlock): Promise<IPLDBlock> {
const repo = this._conn.getRepository(IPLDBlock);
return repo.save(ipldBlock);
}
async getHookStatus (queryRunner: QueryRunner): Promise<HookStatus | undefined> {
const repo = queryRunner.manager.getRepository(HookStatus);
return repo.findOne();
}
async updateHookStatusProcessedBlock (queryRunner: QueryRunner, blockNumber: number, force?: boolean): Promise<HookStatus> {
const repo = queryRunner.manager.getRepository(HookStatus);
let entity = await repo.findOne();
if (!entity) {
entity = repo.create({
latestProcessedBlockNumber: blockNumber
});
}
if (force || blockNumber > entity.latestProcessedBlockNumber) {
entity.latestProcessedBlockNumber = blockNumber;
}
return repo.save(entity);
}
async getContracts (where: FindConditions<Contract>): Promise<Contract[]> {
const repo = this._conn.getRepository(Contract);
return repo.find({ where });
}
async getContract (address: string): Promise<Contract | undefined> {
const repo = this._conn.getRepository(Contract);
return this._baseDatabase.getContract(repo, address);
}
async createTransactionRunner (): Promise<QueryRunner> {
return this._baseDatabase.createTransactionRunner();
}
async getProcessedBlockCountForRange (fromBlockNumber: number, toBlockNumber: number): Promise<{ expected: number, actual: number }> {
const repo = this._conn.getRepository(BlockProgress);
return this._baseDatabase.getProcessedBlockCountForRange(repo, fromBlockNumber, toBlockNumber);
}
async getEventsInRange (fromBlockNumber: number, toBlockNumber: number): Promise<Array<Event>> {
const repo = this._conn.getRepository(Event);
return this._baseDatabase.getEventsInRange(repo, fromBlockNumber, toBlockNumber);
}
async saveEventEntity (queryRunner: QueryRunner, entity: Event): Promise<Event> {
const repo = queryRunner.manager.getRepository(Event);
return this._baseDatabase.saveEventEntity(repo, entity);
}
async getBlockEvents (blockHash: string, where: FindConditions<Event>): Promise<Event[]> {
const repo = this._conn.getRepository(Event);
return this._baseDatabase.getBlockEvents(repo, blockHash, where);
}
async saveEvents (queryRunner: QueryRunner, block: DeepPartial<BlockProgress>, events: DeepPartial<Event>[]): Promise<void> {
const blockRepo = queryRunner.manager.getRepository(BlockProgress);
const eventRepo = queryRunner.manager.getRepository(Event);
return this._baseDatabase.saveEvents(blockRepo, eventRepo, block, events);
}
async saveContract (address: string, kind: string, checkpoint: boolean, startingBlock: number): Promise<void> {
await this._conn.transaction(async (tx) => {
const repo = tx.getRepository(Contract);
return this._baseDatabase.saveContract(repo, address, kind, checkpoint, startingBlock);
});
}
async updateSyncStatusIndexedBlock (queryRunner: QueryRunner, blockHash: string, blockNumber: number, force = false): Promise<SyncStatus> {
const repo = queryRunner.manager.getRepository(SyncStatus);
return this._baseDatabase.updateSyncStatusIndexedBlock(repo, blockHash, blockNumber, force);
}
async updateSyncStatusCanonicalBlock (queryRunner: QueryRunner, blockHash: string, blockNumber: number, force = false): Promise<SyncStatus> {
const repo = queryRunner.manager.getRepository(SyncStatus);
return this._baseDatabase.updateSyncStatusCanonicalBlock(repo, blockHash, blockNumber, force);
}
async updateSyncStatusChainHead (queryRunner: QueryRunner, blockHash: string, blockNumber: number): Promise<SyncStatus> {
const repo = queryRunner.manager.getRepository(SyncStatus);
return this._baseDatabase.updateSyncStatusChainHead(repo, blockHash, blockNumber);
}
async getSyncStatus (queryRunner: QueryRunner): Promise<SyncStatus | undefined> {
const repo = queryRunner.manager.getRepository(SyncStatus);
return this._baseDatabase.getSyncStatus(repo);
}
async getEvent (id: string): Promise<Event | undefined> {
const repo = this._conn.getRepository(Event);
return this._baseDatabase.getEvent(repo, id);
}
async getBlocksAtHeight (height: number, isPruned: boolean): Promise<BlockProgress[]> {
const repo = this._conn.getRepository(BlockProgress);
return this._baseDatabase.getBlocksAtHeight(repo, height, isPruned);
}
async markBlocksAsPruned (queryRunner: QueryRunner, blocks: BlockProgress[]): Promise<void> {
const repo = queryRunner.manager.getRepository(BlockProgress);
return this._baseDatabase.markBlocksAsPruned(repo, blocks);
}
async getBlockProgress (blockHash: string): Promise<BlockProgress | undefined> {
const repo = this._conn.getRepository(BlockProgress);
return this._baseDatabase.getBlockProgress(repo, blockHash);
}
async updateBlockProgress (queryRunner: QueryRunner, blockHash: string, lastProcessedEventIndex: number): Promise<void> {
const repo = queryRunner.manager.getRepository(BlockProgress);
return this._baseDatabase.updateBlockProgress(repo, blockHash, lastProcessedEventIndex);
}
async removeEntities<Entity> (queryRunner: QueryRunner, entity: new () => Entity, findConditions?: FindManyOptions<Entity> | FindConditions<Entity>): Promise<void> {
return this._baseDatabase.removeEntities(queryRunner, entity, findConditions);
}
async getAncestorAtDepth (blockHash: string, depth: number): Promise<string> {
return this._baseDatabase.getAncestorAtDepth(blockHash, depth);
}
_getPropertyColumnMapForEntity (entityName: string): Map<string, string> {
return this._conn.getMetadata(entityName).ownColumns.reduce((acc, curr) => {
return acc.set(curr.propertyName, curr.databaseName);
}, new Map<string, string>());
}
}

32
packages/eden-watcher/src/entity/Account.ts Normal file
View File

@ -0,0 +1,32 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Claim } from './Claim';
import { Slash } from './Slash';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Account {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('bigint', { transformer: bigintTransformer })
totalClaimed!: bigint;
@Column('bigint', { transformer: bigintTransformer })
totalSlashed!: bigint;
@ManyToOne(() => Claim)
claims!: Claim;
@ManyToOne(() => Slash)
slashes!: Slash;
}

63
packages/eden-watcher/src/entity/Block.ts Normal file
View File

@ -0,0 +1,63 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column } from 'typeorm';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Block {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('boolean')
fromActiveProducer!: boolean;
@Column('varchar')
hash!: string;
@Column('varchar')
parentHash!: string;
@Column('varchar')
unclesHash!: string;
@Column('varchar')
author!: string;
@Column('varchar')
stateRoot!: string;
@Column('varchar')
transactionsRoot!: string;
@Column('varchar')
receiptsRoot!: string;
@Column('bigint', { transformer: bigintTransformer })
number!: bigint;
@Column('bigint', { transformer: bigintTransformer })
gasUsed!: bigint;
@Column('bigint', { transformer: bigintTransformer })
gasLimit!: bigint;
@Column('bigint', { transformer: bigintTransformer })
timestamp!: bigint;
@Column('bigint', { transformer: bigintTransformer })
difficulty!: bigint;
@Column('bigint', { transformer: bigintTransformer })
totalDifficulty!: bigint;
@Column('bigint', { nullable: true, transformer: bigintTransformer })
size!: bigint;
}

45
packages/eden-watcher/src/entity/BlockProgress.ts Normal file
View File

@ -0,0 +1,45 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryGeneratedColumn, Column, Index } from 'typeorm';
import { BlockProgressInterface } from '@vulcanize/util';
@Entity()
@Index(['blockHash'], { unique: true })
@Index(['blockNumber'])
@Index(['parentHash'])
export class BlockProgress implements BlockProgressInterface {
@PrimaryGeneratedColumn()
id!: number;
@Column('varchar')
cid!: string;
@Column('varchar', { length: 66 })
blockHash!: string;
@Column('varchar', { length: 66 })
parentHash!: string;
@Column('integer')
blockNumber!: number;
@Column('integer')
blockTimestamp!: number;
@Column('integer')
numEvents!: number;
@Column('integer')
numProcessedEvents!: number;
@Column('integer')
lastProcessedEventIndex!: number;
@Column('boolean')
isComplete!: boolean;
@Column('boolean', { default: false })
isPruned!: boolean;
}

34
packages/eden-watcher/src/entity/Claim.ts Normal file
View File

@ -0,0 +1,34 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Account } from './Account';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Claim {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('bigint', { transformer: bigintTransformer })
timestamp!: bigint;
@Column('bigint', { transformer: bigintTransformer })
index!: bigint;
@ManyToOne(() => Account)
account!: Account;
@Column('bigint', { transformer: bigintTransformer })
totalEarned!: bigint;
@Column('bigint', { transformer: bigintTransformer })
claimed!: bigint;
}

24
packages/eden-watcher/src/entity/Contract.ts Normal file
View File

@ -0,0 +1,24 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryGeneratedColumn, Column, Index } from 'typeorm';
@Entity()
@Index(['address'], { unique: true })
export class Contract {
@PrimaryGeneratedColumn()
id!: number;
@Column('varchar', { length: 42 })
address!: string;
@Column('varchar')
kind!: string;
@Column('boolean')
checkpoint!: boolean;
@Column('integer')
startingBlock!: number;
}

34
packages/eden-watcher/src/entity/Distribution.ts Normal file
View File

@ -0,0 +1,34 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Distributor } from './Distributor';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Distribution {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@ManyToOne(() => Distributor)
distributor!: Distributor;
@Column('bigint', { transformer: bigintTransformer })
timestamp!: bigint;
@Column('bigint', { transformer: bigintTransformer })
distributionNumber!: bigint;
@Column('varchar')
merkleRoot!: string;
@Column('varchar')
metadataURI!: string;
}

21
packages/eden-watcher/src/entity/Distributor.ts Normal file
View File

@ -0,0 +1,21 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Distribution } from './Distribution';
@Entity()
export class Distributor {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@ManyToOne(() => Distribution, { nullable: true })
currentDistribution!: Distribution;
}

44
packages/eden-watcher/src/entity/Epoch.ts Normal file
View File

@ -0,0 +1,44 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Block } from './Block';
import { ProducerEpoch } from './ProducerEpoch';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Epoch {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('boolean')
finalized!: boolean;
@Column('bigint', { transformer: bigintTransformer })
epochNumber!: bigint;
@ManyToOne(() => Block, { nullable: true })
startBlock!: Block;
@ManyToOne(() => Block, { nullable: true })
endBlock!: Block;
@Column('bigint', { transformer: bigintTransformer })
producerBlocks!: bigint;
@Column('bigint', { transformer: bigintTransformer })
allBlocks!: bigint;
@Column('varchar')
producerBlocksRatio!: string;
@ManyToOne(() => ProducerEpoch)
producerRewards!: ProducerEpoch;
}

38
packages/eden-watcher/src/entity/Event.ts Normal file
View File

@ -0,0 +1,38 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryGeneratedColumn, Column, Index, ManyToOne } from 'typeorm';
import { BlockProgress } from './BlockProgress';
@Entity()
@Index(['block', 'contract'])
@Index(['block', 'contract', 'eventName'])
export class Event {
@PrimaryGeneratedColumn()
id!: number;
@ManyToOne(() => BlockProgress, { onDelete: 'CASCADE' })
block!: BlockProgress;
@Column('varchar', { length: 66 })
txHash!: string;
@Column('integer')
index!: number;
@Column('varchar', { length: 42 })
contract!: string;
@Column('varchar', { length: 256 })
eventName!: string;
@Column('text')
eventInfo!: string;
@Column('text')
extraInfo!: string;
@Column('text')
proof!: string;
}

14
packages/eden-watcher/src/entity/HookStatus.ts Normal file
View File

@ -0,0 +1,14 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryGeneratedColumn, Column } from 'typeorm';
@Entity()
export class HookStatus {
@PrimaryGeneratedColumn()
id!: number;
@Column('integer')
latestProcessedBlockNumber!: number;
}

30
packages/eden-watcher/src/entity/IPLDBlock.ts Normal file
View File

@ -0,0 +1,30 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryGeneratedColumn, Column, Index, ManyToOne } from 'typeorm';
import { BlockProgress } from './BlockProgress';
@Entity()
@Index(['cid'], { unique: true })
@Index(['block', 'contractAddress'])
@Index(['block', 'contractAddress', 'kind'], { unique: true })
export class IPLDBlock {
@PrimaryGeneratedColumn()
id!: number;
@ManyToOne(() => BlockProgress, { onDelete: 'CASCADE' })
block!: BlockProgress;
@Column('varchar', { length: 42 })
contractAddress!: string;
@Column('varchar')
cid!: string;
@Column('varchar')
kind!: string;
@Column('bytea')
data!: Buffer;
}

41
packages/eden-watcher/src/entity/Network.ts Normal file
View File

@ -0,0 +1,41 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Slot } from './Slot';
import { Staker } from './Staker';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Network {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@ManyToOne(() => Slot, { nullable: true })
slot0!: Slot;
@ManyToOne(() => Slot, { nullable: true })
slot1!: Slot;
@ManyToOne(() => Slot, { nullable: true })
slot2!: Slot;
@ManyToOne(() => Staker)
stakers!: Staker;
@Column('bigint', { nullable: true, transformer: bigintTransformer })
numStakers!: bigint;
@Column('bigint', { transformer: bigintTransformer })
totalStaked!: bigint;
@Column('bigint', { array: true })
stakedPercentiles!: bigint[];
}

33
packages/eden-watcher/src/entity/Producer.ts Normal file
View File

@ -0,0 +1,33 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column } from 'typeorm';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Producer {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('boolean')
active!: boolean;
@Column('varchar', { nullable: true })
rewardCollector!: string;
@Column('bigint', { transformer: bigintTransformer })
rewards!: bigint;
@Column('bigint', { transformer: bigintTransformer })
confirmedBlocks!: bigint;
@Column('bigint', { transformer: bigintTransformer })
pendingEpochBlocks!: bigint;
}

34
packages/eden-watcher/src/entity/ProducerEpoch.ts Normal file
View File

@ -0,0 +1,34 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Epoch } from './Epoch';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class ProducerEpoch {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('varchar')
address!: string;
@ManyToOne(() => Epoch)
epoch!: Epoch;
@Column('bigint', { transformer: bigintTransformer })
totalRewards!: bigint;
@Column('bigint', { transformer: bigintTransformer })
blocksProduced!: bigint;
@Column('varchar')
blocksProducedRatio!: string;
}

23
packages/eden-watcher/src/entity/ProducerRewardCollectorChange.ts Normal file
View File

@ -0,0 +1,23 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column } from 'typeorm';
@Entity()
export class ProducerRewardCollectorChange {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('varchar')
producer!: string;
@Column('varchar')
rewardCollector!: string;
}

21
packages/eden-watcher/src/entity/ProducerSet.ts Normal file
View File

@ -0,0 +1,21 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Producer } from './Producer';
@Entity()
export class ProducerSet {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@ManyToOne(() => Producer)
producers!: Producer;
}

28
packages/eden-watcher/src/entity/ProducerSetChange.ts Normal file
View File

@ -0,0 +1,28 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column } from 'typeorm';
enum ProducerSetChangeType {
Added,
Removed
}
@Entity()
export class ProducerSetChange {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('varchar')
producer!: string;
@Column('integer')
changeType!: ProducerSetChangeType;
}

31
packages/eden-watcher/src/entity/RewardSchedule.ts Normal file
View File

@ -0,0 +1,31 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { RewardScheduleEntry } from './RewardScheduleEntry';
import { Epoch } from './Epoch';
@Entity()
export class RewardSchedule {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@ManyToOne(() => RewardScheduleEntry)
rewardScheduleEntries!: RewardScheduleEntry;
@ManyToOne(() => Epoch, { nullable: true })
lastEpoch!: Epoch;
@ManyToOne(() => Epoch, { nullable: true })
pendingEpoch!: Epoch;
@ManyToOne(() => RewardScheduleEntry, { nullable: true })
activeRewardScheduleEntry!: RewardScheduleEntry;
}

27
packages/eden-watcher/src/entity/RewardScheduleEntry.ts Normal file
View File

@ -0,0 +1,27 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column } from 'typeorm';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class RewardScheduleEntry {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('bigint', { transformer: bigintTransformer })
startTime!: bigint;
@Column('bigint', { transformer: bigintTransformer })
epochDuration!: bigint;
@Column('bigint', { transformer: bigintTransformer })
rewardsPerEpoch!: bigint;
}

28
packages/eden-watcher/src/entity/Slash.ts Normal file
View File

@ -0,0 +1,28 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Account } from './Account';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Slash {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('bigint', { transformer: bigintTransformer })
timestamp!: bigint;
@ManyToOne(() => Account)
account!: Account;
@Column('bigint', { transformer: bigintTransformer })
slashed!: bigint;
}

43
packages/eden-watcher/src/entity/Slot.ts Normal file
View File

@ -0,0 +1,43 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { SlotClaim } from './SlotClaim';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Slot {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('varchar')
owner!: string;
@Column('varchar')
delegate!: string;
@Column('bigint', { transformer: bigintTransformer })
winningBid!: bigint;
@Column('bigint', { transformer: bigintTransformer })
oldBid!: bigint;
@Column('bigint', { transformer: bigintTransformer })
startTime!: bigint;
@Column('bigint', { transformer: bigintTransformer })
expirationTime!: bigint;
@Column('varchar')
taxRatePerDay!: string;
@ManyToOne(() => SlotClaim)
claims!: SlotClaim;
}

40
packages/eden-watcher/src/entity/SlotClaim.ts Normal file
View File

@ -0,0 +1,40 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column, ManyToOne } from 'typeorm';
import { Slot } from './Slot';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class SlotClaim {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@ManyToOne(() => Slot)
slot!: Slot;
@Column('varchar')
owner!: string;
@Column('bigint', { transformer: bigintTransformer })
winningBid!: bigint;
@Column('bigint', { transformer: bigintTransformer })
oldBid!: bigint;
@Column('bigint', { transformer: bigintTransformer })
startTime!: bigint;
@Column('bigint', { transformer: bigintTransformer })
expirationTime!: bigint;
@Column('varchar')
taxRatePerDay!: string;
}

24
packages/eden-watcher/src/entity/Staker.ts Normal file
View File

@ -0,0 +1,24 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryColumn, Column } from 'typeorm';
import { bigintTransformer } from '@vulcanize/util';
@Entity()
export class Staker {
@PrimaryColumn('varchar')
id!: string;
@PrimaryColumn('varchar', { length: 66 })
blockHash!: string;
@Column('integer')
blockNumber!: number;
@Column('bigint', { transformer: bigintTransformer })
staked!: bigint;
@Column('bigint', { nullable: true, transformer: bigintTransformer })
rank!: bigint;
}

30
packages/eden-watcher/src/entity/SyncStatus.ts Normal file
View File

@ -0,0 +1,30 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { Entity, PrimaryGeneratedColumn, Column } from 'typeorm';
import { SyncStatusInterface } from '@vulcanize/util';
@Entity()
export class SyncStatus implements SyncStatusInterface {
@PrimaryGeneratedColumn()
id!: number;
@Column('varchar', { length: 66 })
chainHeadBlockHash!: string;
@Column('integer')
chainHeadBlockNumber!: number;
@Column('varchar', { length: 66 })
latestIndexedBlockHash!: string;
@Column('integer')
latestIndexedBlockNumber!: number;
@Column('varchar', { length: 66 })
latestCanonicalBlockHash!: string;
@Column('integer')
latestCanonicalBlockNumber!: number;
}

180
packages/eden-watcher/src/events.ts Normal file
View File

@ -0,0 +1,180 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import assert from 'assert';
import debug from 'debug';
import { PubSub } from 'apollo-server-express';
import { EthClient } from '@vulcanize/ipld-eth-client';
import {
JobQueue,
EventWatcher as BaseEventWatcher,
EventWatcherInterface,
QUEUE_BLOCK_PROCESSING,
QUEUE_EVENT_PROCESSING,
QUEUE_BLOCK_CHECKPOINT,
QUEUE_HOOKS,
QUEUE_IPFS,
UNKNOWN_EVENT_NAME,
UpstreamConfig,
JOB_KIND_PRUNE
} from '@vulcanize/util';
import { Indexer } from './indexer';
import { Event } from './entity/Event';
const EVENT = 'event';
const log = debug('vulcanize:events');
export class EventWatcher implements EventWatcherInterface {
_ethClient: EthClient
_indexer: Indexer
_subscription: ZenObservable.Subscription | undefined
_baseEventWatcher: BaseEventWatcher
_pubsub: PubSub
_jobQueue: JobQueue
constructor (upstreamConfig: UpstreamConfig, ethClient: EthClient, postgraphileClient: EthClient, indexer: Indexer, pubsub: PubSub, jobQueue: JobQueue) {
assert(ethClient);
assert(indexer);
this._ethClient = ethClient;
this._indexer = indexer;
this._pubsub = pubsub;
this._jobQueue = jobQueue;
this._baseEventWatcher = new BaseEventWatcher(upstreamConfig, this._ethClient, postgraphileClient, this._indexer, this._pubsub, this._jobQueue);
}
getEventIterator (): AsyncIterator<any> {
return this._pubsub.asyncIterator([EVENT]);
}
getBlockProgressEventIterator (): AsyncIterator<any> {
return this._baseEventWatcher.getBlockProgressEventIterator();
}
async start (): Promise<void> {
assert(!this._subscription, 'subscription already started');
await this.initBlockProcessingOnCompleteHandler();
await this.initEventProcessingOnCompleteHandler();
await this.initHooksOnCompleteHandler();
await this.initBlockCheckpointOnCompleteHandler();
this._baseEventWatcher.startBlockProcessing();
}
async stop (): Promise<void> {
this._baseEventWatcher.stop();
}
async initBlockProcessingOnCompleteHandler (): Promise<void> {
this._jobQueue.onComplete(QUEUE_BLOCK_PROCESSING, async (job) => {
const { id, data: { failed, request: { data: { kind } } } } = job;
if (failed) {
log(`Job ${id} for queue ${QUEUE_BLOCK_PROCESSING} failed`);
return;
}
await this._baseEventWatcher.blockProcessingCompleteHandler(job);
await this.createHooksJob(kind);
});
}
async initEventProcessingOnCompleteHandler (): Promise<void> {
await this._jobQueue.onComplete(QUEUE_EVENT_PROCESSING, async (job) => {
const { id, data: { request, failed, state, createdOn } } = job;
if (failed) {
log(`Job ${id} for queue ${QUEUE_EVENT_PROCESSING} failed`);
return;
}
const dbEvent = await this._baseEventWatcher.eventProcessingCompleteHandler(job);
const timeElapsedInSeconds = (Date.now() - Date.parse(createdOn)) / 1000;
log(`Job onComplete event ${request.data.id} publish ${!!request.data.publish}`);
if (!failed && state === 'completed' && request.data.publish) {
// Check for max acceptable lag time between request and sending results to live subscribers.
if (timeElapsedInSeconds <= this._jobQueue.maxCompletionLag) {
await this.publishEventToSubscribers(dbEvent, timeElapsedInSeconds);
} else {
log(`event ${request.data.id} is too old (${timeElapsedInSeconds}s), not broadcasting to live subscribers`);
}
}
});
}
async initHooksOnCompleteHandler (): Promise<void> {
this._jobQueue.onComplete(QUEUE_HOOKS, async (job) => {
const { data: { request: { data: { blockNumber, blockHash } } } } = job;
await this._indexer.updateHookStatusProcessedBlock(blockNumber);
// Create a checkpoint job after completion of a hook job.
await this.createCheckpointJob(blockHash, blockNumber);
});
}
async initBlockCheckpointOnCompleteHandler (): Promise<void> {
this._jobQueue.onComplete(QUEUE_BLOCK_CHECKPOINT, async (job) => {
const { data: { request: { data: { blockHash } } } } = job;
if (this._indexer.isIPFSConfigured()) {
await this.createIPFSPutJob(blockHash);
}
});
}
async publishEventToSubscribers (dbEvent: Event, timeElapsedInSeconds: number): Promise<void> {
if (dbEvent && dbEvent.eventName !== UNKNOWN_EVENT_NAME) {
const resultEvent = this._indexer.getResultEvent(dbEvent);
log(`pushing event to GQL subscribers (${timeElapsedInSeconds}s elapsed): ${resultEvent.event.__typename}`);
// Publishing the event here will result in pushing the payload to GQL subscribers for `onEvent`.
await this._pubsub.publish(EVENT, {
onEvent: resultEvent
});
}
}
async createHooksJob (kind: string): Promise<void> {
// If it's a pruning job: Create a hook job for the latest canonical block.
if (kind === JOB_KIND_PRUNE) {
const latestCanonicalBlock = await this._indexer.getLatestCanonicalBlock();
assert(latestCanonicalBlock);
await this._jobQueue.pushJob(
QUEUE_HOOKS,
{
blockHash: latestCanonicalBlock.blockHash,
blockNumber: latestCanonicalBlock.blockNumber
}
);
}
}
async createCheckpointJob (blockHash: string, blockNumber: number): Promise<void> {
await this._jobQueue.pushJob(
QUEUE_BLOCK_CHECKPOINT,
{
blockHash,
blockNumber
}
);
}
async createIPFSPutJob (blockHash: string): Promise<void> {
const ipldBlocks = await this._indexer.getIPLDBlocksByHash(blockHash);
for (const ipldBlock of ipldBlocks) {
const data = this._indexer.getIPLDData(ipldBlock);
await this._jobQueue.pushJob(QUEUE_IPFS, { data });
}
}
}

82
packages/eden-watcher/src/fill.ts Normal file
View File

@ -0,0 +1,82 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import path from 'path';
import assert from 'assert';
import 'reflect-metadata';
import yargs from 'yargs';
import { hideBin } from 'yargs/helpers';
import debug from 'debug';
import { PubSub } from 'apollo-server-express';
import { Config, getConfig, fillBlocks, JobQueue, DEFAULT_CONFIG_PATH, initClients } from '@vulcanize/util';
import { GraphWatcher, Database as GraphDatabase } from '@vulcanize/graph-node';
import { Database } from './database';
import { Indexer } from './indexer';
import { EventWatcher } from './events';
const log = debug('vulcanize:server');
export const main = async (): Promise<any> => {
const argv = await yargs(hideBin(process.argv)).parserConfiguration({
'parse-numbers': false
}).options({
configFile: {
alias: 'f',
type: 'string',
demandOption: true,
describe: 'configuration file path (toml)',
default: DEFAULT_CONFIG_PATH
},
startBlock: {
type: 'number',
demandOption: true,
describe: 'Block number to start processing at'
},
endBlock: {
type: 'number',
demandOption: true,
describe: 'Block number to stop processing at'
}
}).argv;
const config: Config = await getConfig(argv.configFile);
const { ethClient, postgraphileClient, ethProvider } = await initClients(config);
const db = new Database(config.database);
await db.init();
const graphDb = new GraphDatabase(config.database, path.resolve(__dirname, 'entity/*'));
await graphDb.init();
const graphWatcher = new GraphWatcher(graphDb, postgraphileClient, config.server.subgraphPath);
// Note: In-memory pubsub works fine for now, as each watcher is a single process anyway.
// Later: https://www.apollographql.com/docs/apollo-server/data/subscriptions/#production-pubsub-libraries
const pubsub = new PubSub();
const indexer = new Indexer(config.server, db, ethClient, postgraphileClient, ethProvider, graphWatcher);
graphWatcher.setIndexer(indexer);
await graphWatcher.init();
const jobQueueConfig = config.jobQueue;
assert(jobQueueConfig, 'Missing job queue config');
const { dbConnectionString, maxCompletionLagInSecs } = jobQueueConfig;
assert(dbConnectionString, 'Missing job queue db connection string');
const jobQueue = new JobQueue({ dbConnectionString, maxCompletionLag: maxCompletionLagInSecs });
await jobQueue.start();
const eventWatcher = new EventWatcher(config.upstream, ethClient, postgraphileClient, indexer, pubsub, jobQueue);
await fillBlocks(jobQueue, indexer, postgraphileClient, eventWatcher, config.upstream.ethServer.blockDelayInMilliSecs, argv);
};
main().catch(err => {
log(err);
}).finally(() => {
process.exit();
});

3
packages/eden-watcher/src/gql/index.ts Normal file
View File

@ -0,0 +1,3 @@
export * as mutations from './mutations';
export * as queries from './queries';
export * as subscriptions from './subscriptions';

4
packages/eden-watcher/src/gql/mutations/index.ts Normal file
View File

@ -0,0 +1,4 @@
import fs from 'fs';
import path from 'path';
export const watchContract = fs.readFileSync(path.join(__dirname, 'watchContract.gql'), 'utf8');

3
packages/eden-watcher/src/gql/mutations/watchContract.gql Normal file
View File

@ -0,0 +1,3 @@
mutation watchContract($address: String!, $kind: String!, $checkpoint: Boolean!, $startingBlock: Int){
watchContract(address: $address, kind: $kind, checkpoint: $checkpoint, startingBlock: $startingBlock)
}

29
packages/eden-watcher/src/gql/queries/account.gql Normal file
View File

@ -0,0 +1,29 @@
query account($id: String!, $blockHash: String!){
account(id: $id, blockHash: $blockHash){
id
totalClaimed
totalSlashed
claims{
id
timestamp
index
account{
id
totalClaimed
totalSlashed
slashes{
id
timestamp
account{
id
totalClaimed
totalSlashed
}
slashed
}
}
totalEarned
claimed
}
}
}

36
packages/eden-watcher/src/gql/queries/claim.gql Normal file
View File

@ -0,0 +1,36 @@
query claim($id: String!, $blockHash: String!){
claim(id: $id, blockHash: $blockHash){
id
timestamp
index
account{
id
totalClaimed
totalSlashed
claims{
id
timestamp
index
account{
id
totalClaimed
totalSlashed
slashes{
id
timestamp
account{
id
totalClaimed
totalSlashed
}
slashed
}
}
totalEarned
claimed
}
}
totalEarned
claimed
}
}

22
packages/eden-watcher/src/gql/queries/distribution.gql Normal file
View File

@ -0,0 +1,22 @@
query distribution($id: String!, $blockHash: String!){
distribution(id: $id, blockHash: $blockHash){
id
distributor{
id
currentDistribution{
id
distributor{
id
}
timestamp
distributionNumber
merkleRoot
metadataURI
}
}
timestamp
distributionNumber
merkleRoot
metadataURI
}
}

15
packages/eden-watcher/src/gql/queries/distributor.gql Normal file
View File

@ -0,0 +1,15 @@
query distributor($id: String!, $blockHash: String!){
distributor(id: $id, blockHash: $blockHash){
id
currentDistribution{
id
distributor{
id
}
timestamp
distributionNumber
merkleRoot
metadataURI
}
}
}

39
packages/eden-watcher/src/gql/queries/epoch.gql Normal file
View File

@ -0,0 +1,39 @@
query epoch($id: String!, $blockHash: String!){
epoch(id: $id, blockHash: $blockHash){
id
finalized
epochNumber
startBlock{
cid
hash
number
timestamp
parentHash
}
endBlock{
cid
hash
number
timestamp
parentHash
}
producerBlocks
allBlocks
producerBlocksRatio
producerRewards{
id
address
epoch{
id
finalized
epochNumber
producerBlocks
allBlocks
producerBlocksRatio
}
totalRewards
blocksProduced
blocksProducedRatio
}
}
}

75
packages/eden-watcher/src/gql/queries/events.gql Normal file
View File

@ -0,0 +1,75 @@
query events($blockHash: String!, $contractAddress: String!, $name: String){
events(blockHash: $blockHash, contractAddress: $contractAddress, name: $name){
block{
cid
hash
number
timestamp
parentHash
}
tx{
hash
index
from
to
}
contract
eventIndex
event{
... on TransferEvent {
from
to
value
}
... on ApprovalEvent {
owner
spender
value
}
... on AuthorizationUsedEvent {
authorizer
nonce
}
... on AdminUpdatedEvent {
newAdmin
oldAdmin
}
... on TaxRateUpdatedEvent {
newNumerator
newDenominator
oldNumerator
oldDenominator
}
... on SlotClaimedEvent {
slot
owner
delegate
newBidAmount
oldBidAmount
taxNumerator
taxDenominator
}
... on SlotDelegateUpdatedEvent {
slot
owner
newDelegate
oldDelegate
}
... on StakeEvent {
staker
stakeAmount
}
... on UnstakeEvent {
staker
unstakedAmount
}
... on WithdrawEvent {
withdrawer
withdrawalAmount
}
}
proof{
data
}
}
}

75
packages/eden-watcher/src/gql/queries/eventsInRange.gql Normal file
View File

@ -0,0 +1,75 @@
query eventsInRange($fromBlockNumber: Int!, $toBlockNumber: Int!){
eventsInRange(fromBlockNumber: $fromBlockNumber, toBlockNumber: $toBlockNumber){
block{
cid
hash
number
timestamp
parentHash
}
tx{
hash
index
from
to
}
contract
eventIndex
event{
... on TransferEvent {
from
to
value
}
... on ApprovalEvent {
owner
spender
value
}
... on AuthorizationUsedEvent {
authorizer
nonce
}
... on AdminUpdatedEvent {
newAdmin
oldAdmin
}
... on TaxRateUpdatedEvent {
newNumerator
newDenominator
oldNumerator
oldDenominator
}
... on SlotClaimedEvent {
slot
owner
delegate
newBidAmount
oldBidAmount
taxNumerator
taxDenominator
}
... on SlotDelegateUpdatedEvent {
slot
owner
newDelegate
oldDelegate
}
... on StakeEvent {
staker
stakeAmount
}
... on UnstakeEvent {
staker
unstakedAmount
}
... on WithdrawEvent {
withdrawer
withdrawalAmount
}
}
proof{
data
}
}
}

15
packages/eden-watcher/src/gql/queries/getState.gql Normal file
View File

@ -0,0 +1,15 @@
query getState($blockHash: String!, $contractAddress: String!, $kind: String){
getState(blockHash: $blockHash, contractAddress: $contractAddress, kind: $kind){
block{
cid
hash
number
timestamp
parentHash
}
contractAddress
cid
kind
data
}
}

15
packages/eden-watcher/src/gql/queries/getStateByCID.gql Normal file
View File

@ -0,0 +1,15 @@
query getStateByCID($cid: String!){
getStateByCID(cid: $cid){
block{
cid
hash
number
timestamp
parentHash
}
contractAddress
cid
kind
data
}
}

24
packages/eden-watcher/src/gql/queries/index.ts Normal file
View File

@ -0,0 +1,24 @@
import fs from 'fs';
import path from 'path';
export const events = fs.readFileSync(path.join(__dirname, 'events.gql'), 'utf8');
export const eventsInRange = fs.readFileSync(path.join(__dirname, 'eventsInRange.gql'), 'utf8');
export const producer = fs.readFileSync(path.join(__dirname, 'producer.gql'), 'utf8');
export const producerSet = fs.readFileSync(path.join(__dirname, 'producerSet.gql'), 'utf8');
export const producerSetChange = fs.readFileSync(path.join(__dirname, 'producerSetChange.gql'), 'utf8');
export const producerRewardCollectorChange = fs.readFileSync(path.join(__dirname, 'producerRewardCollectorChange.gql'), 'utf8');
export const rewardScheduleEntry = fs.readFileSync(path.join(__dirname, 'rewardScheduleEntry.gql'), 'utf8');
export const rewardSchedule = fs.readFileSync(path.join(__dirname, 'rewardSchedule.gql'), 'utf8');
export const producerEpoch = fs.readFileSync(path.join(__dirname, 'producerEpoch.gql'), 'utf8');
export const epoch = fs.readFileSync(path.join(__dirname, 'epoch.gql'), 'utf8');
export const slotClaim = fs.readFileSync(path.join(__dirname, 'slotClaim.gql'), 'utf8');
export const slot = fs.readFileSync(path.join(__dirname, 'slot.gql'), 'utf8');
export const staker = fs.readFileSync(path.join(__dirname, 'staker.gql'), 'utf8');
export const network = fs.readFileSync(path.join(__dirname, 'network.gql'), 'utf8');
export const distributor = fs.readFileSync(path.join(__dirname, 'distributor.gql'), 'utf8');
export const distribution = fs.readFileSync(path.join(__dirname, 'distribution.gql'), 'utf8');
export const claim = fs.readFileSync(path.join(__dirname, 'claim.gql'), 'utf8');
export const slash = fs.readFileSync(path.join(__dirname, 'slash.gql'), 'utf8');
export const account = fs.readFileSync(path.join(__dirname, 'account.gql'), 'utf8');
export const getStateByCID = fs.readFileSync(path.join(__dirname, 'getStateByCID.gql'), 'utf8');
export const getState = fs.readFileSync(path.join(__dirname, 'getState.gql'), 'utf8');

89
packages/eden-watcher/src/gql/queries/network.gql Normal file
View File

@ -0,0 +1,89 @@
query network($id: String!, $blockHash: String!){
network(id: $id, blockHash: $blockHash){
id
slot0{
id
owner
delegate
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
claims{
id
slot{
id
owner
delegate
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
claims{
id
owner
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
}
}
owner
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
}
}
slot1{
id
owner
delegate
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
claims{
id
owner
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
}
}
slot2{
id
owner
delegate
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
claims{
id
owner
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
}
}
stakers{
id
staked
rank
}
numStakers
totalStaked
stakedPercentiles
}
}

10
packages/eden-watcher/src/gql/queries/producer.gql Normal file
View File

@ -0,0 +1,10 @@
query producer($id: String!, $blockHash: String!){
producer(id: $id, blockHash: $blockHash){
id
active
rewardCollector
rewards
confirmedBlocks
pendingEpochBlocks
}
}

46
packages/eden-watcher/src/gql/queries/producerEpoch.gql Normal file
View File

@ -0,0 +1,46 @@
query producerEpoch($id: String!, $blockHash: String!){
producerEpoch(id: $id, blockHash: $blockHash){
id
address
epoch{
id
finalized
epochNumber
startBlock{
cid
hash
number
timestamp
parentHash
}
endBlock{
cid
hash
number
timestamp
parentHash
}
producerBlocks
allBlocks
producerBlocksRatio
producerRewards{
id
address
epoch{
id
finalized
epochNumber
producerBlocks
allBlocks
producerBlocksRatio
}
totalRewards
blocksProduced
blocksProducedRatio
}
}
totalRewards
blocksProduced
blocksProducedRatio
}
}

8
packages/eden-watcher/src/gql/queries/producerRewardCollectorChange.gql Normal file
View File

@ -0,0 +1,8 @@
query producerRewardCollectorChange($id: String!, $blockHash: String!){
producerRewardCollectorChange(id: $id, blockHash: $blockHash){
id
blockNumber
producer
rewardCollector
}
}

13
packages/eden-watcher/src/gql/queries/producerSet.gql Normal file
View File

@ -0,0 +1,13 @@
query producerSet($id: String!, $blockHash: String!){
producerSet(id: $id, blockHash: $blockHash){
id
producers{
id
active
rewardCollector
rewards
confirmedBlocks
pendingEpochBlocks
}
}
}

8
packages/eden-watcher/src/gql/queries/producerSetChange.gql Normal file
View File

@ -0,0 +1,8 @@
query producerSetChange($id: String!, $blockHash: String!){
producerSetChange(id: $id, blockHash: $blockHash){
id
blockNumber
producer
changeType
}
}

104
packages/eden-watcher/src/gql/queries/rewardSchedule.gql Normal file
View File

@ -0,0 +1,104 @@
query rewardSchedule($id: String!, $blockHash: String!){
rewardSchedule(id: $id, blockHash: $blockHash){
id
rewardScheduleEntries{
id
startTime
epochDuration
rewardsPerEpoch
}
lastEpoch{
id
finalized
epochNumber
startBlock{
cid
hash
number
timestamp
parentHash
}
endBlock{
cid
hash
number
timestamp
parentHash
}
producerBlocks
allBlocks
producerBlocksRatio
producerRewards{
id
address
epoch{
id
finalized
epochNumber
startBlock{
cid
hash
number
timestamp
parentHash
}
endBlock{
cid
hash
number
timestamp
parentHash
}
producerBlocks
allBlocks
producerBlocksRatio
producerRewards{
id
address
totalRewards
blocksProduced
blocksProducedRatio
}
}
totalRewards
blocksProduced
blocksProducedRatio
}
}
pendingEpoch{
id
finalized
epochNumber
startBlock{
cid
hash
number
timestamp
parentHash
}
endBlock{
cid
hash
number
timestamp
parentHash
}
producerBlocks
allBlocks
producerBlocksRatio
producerRewards{
id
address
totalRewards
blocksProduced
blocksProducedRatio
}
}
activeRewardScheduleEntry{
id
startTime
epochDuration
rewardsPerEpoch
}
}
}

8
packages/eden-watcher/src/gql/queries/rewardScheduleEntry.gql Normal file
View File

@ -0,0 +1,8 @@
query rewardScheduleEntry($id: String!, $blockHash: String!){
rewardScheduleEntry(id: $id, blockHash: $blockHash){
id
startTime
epochDuration
rewardsPerEpoch
}
}

34
packages/eden-watcher/src/gql/queries/slash.gql Normal file
View File

@ -0,0 +1,34 @@
query slash($id: String!, $blockHash: String!){
slash(id: $id, blockHash: $blockHash){
id
timestamp
account{
id
totalClaimed
totalSlashed
claims{
id
timestamp
index
account{
id
totalClaimed
totalSlashed
slashes{
id
timestamp
account{
id
totalClaimed
totalSlashed
}
slashed
}
}
totalEarned
claimed
}
}
slashed
}
}

31
packages/eden-watcher/src/gql/queries/slot.gql Normal file
View File

@ -0,0 +1,31 @@
query slot($id: String!, $blockHash: String!){
slot(id: $id, blockHash: $blockHash){
id
owner
delegate
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
claims{
id
slot{
id
owner
delegate
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
}
owner
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
}
}
}

40
packages/eden-watcher/src/gql/queries/slotClaim.gql Normal file
View File

@ -0,0 +1,40 @@
query slotClaim($id: String!, $blockHash: String!){
slotClaim(id: $id, blockHash: $blockHash){
id
slot{
id
owner
delegate
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
claims{
id
slot{
id
owner
delegate
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
}
owner
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
}
}
owner
winningBid
oldBid
startTime
expirationTime
taxRatePerDay
}
}

7
packages/eden-watcher/src/gql/queries/staker.gql Normal file
View File

@ -0,0 +1,7 @@
query staker($id: String!, $blockHash: String!){
staker(id: $id, blockHash: $blockHash){
id
staked
rank
}
}

4
packages/eden-watcher/src/gql/subscriptions/index.ts Normal file
View File

@ -0,0 +1,4 @@
import fs from 'fs';
import path from 'path';
export const onEvent = fs.readFileSync(path.join(__dirname, 'onEvent.gql'), 'utf8');

75
packages/eden-watcher/src/gql/subscriptions/onEvent.gql Normal file
View File

@ -0,0 +1,75 @@
subscription onEvent{
onEvent{
block{
cid
hash
number
timestamp
parentHash
}
tx{
hash
index
from
to
}
contract
eventIndex
event{
... on TransferEvent {
from
to
value
}
... on ApprovalEvent {
owner
spender
value
}
... on AuthorizationUsedEvent {
authorizer
nonce
}
... on AdminUpdatedEvent {
newAdmin
oldAdmin
}
... on TaxRateUpdatedEvent {
newNumerator
newDenominator
oldNumerator
oldDenominator
}
... on SlotClaimedEvent {
slot
owner
delegate
newBidAmount
oldBidAmount
taxNumerator
taxDenominator
}
... on SlotDelegateUpdatedEvent {
slot
owner
newDelegate
oldDelegate
}
... on StakeEvent {
staker
stakeAmount
}
... on UnstakeEvent {
staker
unstakedAmount
}
... on WithdrawEvent {
withdrawer
withdrawalAmount
}
}
proof{
data
}
}
}

38
packages/eden-watcher/src/hooks.ts Normal file
View File

@ -0,0 +1,38 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import assert from 'assert';
import { Indexer, ResultEvent } from './indexer';
export async function createInitialCheckpoint (indexer: Indexer, contractAddress: string, blockHash: string): Promise<void> {
assert(indexer);
assert(blockHash);
assert(contractAddress);
// Store an empty state in an IPLDBlock.
const ipldBlockData: any = {
state: {}
};
await indexer.createCheckpoint(contractAddress, blockHash, ipldBlockData);
}
export async function createStateDiff (indexer: Indexer, blockHash: string): Promise<void> {
assert(indexer);
assert(blockHash);
}
export async function createStateCheckpoint (indexer: Indexer, contractAddress: string, blockHash: string): Promise<boolean> {
assert(indexer);
assert(blockHash);
assert(contractAddress);
return false;
}
export async function handleEvent (indexer: Indexer, eventData: ResultEvent): Promise<void> {
assert(indexer);
assert(eventData);
}

1031
packages/eden-watcher/src/indexer.ts Normal file
View File

File diff suppressed because it is too large Load Diff

17
packages/eden-watcher/src/ipfs.ts Normal file
View File

@ -0,0 +1,17 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import { create, IPFSHTTPClient } from 'ipfs-http-client';
export class IPFSClient {
_client: IPFSHTTPClient
constructor (url: string) {
this._client = create({ url });
}
async push (data: any): Promise<void> {
await this._client.dag.put(data, { format: 'dag-cbor', hashAlg: 'sha2-256' });
}
}

163
packages/eden-watcher/src/job-runner.ts Normal file
View File

@ -0,0 +1,163 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import path from 'path';
import assert from 'assert';
import 'reflect-metadata';
import yargs from 'yargs';
import { hideBin } from 'yargs/helpers';
import debug from 'debug';
import {
getConfig,
Config,
JobQueue,
JobRunner as BaseJobRunner,
QUEUE_BLOCK_PROCESSING,
QUEUE_EVENT_PROCESSING,
QUEUE_BLOCK_CHECKPOINT,
QUEUE_HOOKS,
QUEUE_IPFS,
JobQueueConfig,
DEFAULT_CONFIG_PATH,
initClients
} from '@vulcanize/util';
import { GraphWatcher, Database as GraphDatabase } from '@vulcanize/graph-node';
import { Indexer } from './indexer';
import { Database } from './database';
const log = debug('vulcanize:job-runner');
export class JobRunner {
_indexer: Indexer
_jobQueue: JobQueue
_baseJobRunner: BaseJobRunner
_jobQueueConfig: JobQueueConfig
constructor (jobQueueConfig: JobQueueConfig, indexer: Indexer, jobQueue: JobQueue) {
this._jobQueueConfig = jobQueueConfig;
this._indexer = indexer;
this._jobQueue = jobQueue;
this._baseJobRunner = new BaseJobRunner(this._jobQueueConfig, this._indexer, this._jobQueue);
}
async start (): Promise<void> {
await this.subscribeBlockProcessingQueue();
await this.subscribeEventProcessingQueue();
await this.subscribeBlockCheckpointQueue();
await this.subscribeHooksQueue();
await this.subscribeIPFSQueue();
}
async subscribeBlockProcessingQueue (): Promise<void> {
await this._jobQueue.subscribe(QUEUE_BLOCK_PROCESSING, async (job) => {
// TODO Call pre-block hook here (Directly or indirectly (Like done through indexer.processEvent for events)).
await this._baseJobRunner.processBlock(job);
await this._jobQueue.markComplete(job);
});
}
async subscribeEventProcessingQueue (): Promise<void> {
await this._jobQueue.subscribe(QUEUE_EVENT_PROCESSING, async (job) => {
const event = await this._baseJobRunner.processEvent(job);
const watchedContract = await this._indexer.isWatchedContract(event.contract);
if (watchedContract) {
await this._indexer.processEvent(event);
}
await this._jobQueue.markComplete(job);
});
}
async subscribeHooksQueue (): Promise<void> {
await this._jobQueue.subscribe(QUEUE_HOOKS, async (job) => {
const { data: { blockNumber } } = job;
const hookStatus = await this._indexer.getHookStatus();
if (hookStatus && hookStatus.latestProcessedBlockNumber < (blockNumber - 1)) {
const message = `Hooks for blockNumber ${blockNumber - 1} not processed yet, aborting`;
log(message);
throw new Error(message);
}
await this._indexer.processCanonicalBlock(job);
await this._jobQueue.markComplete(job);
});
}
async subscribeBlockCheckpointQueue (): Promise<void> {
await this._jobQueue.subscribe(QUEUE_BLOCK_CHECKPOINT, async (job) => {
await this._indexer.processCheckpoint(job);
await this._jobQueue.markComplete(job);
});
}
async subscribeIPFSQueue (): Promise<void> {
await this._jobQueue.subscribe(QUEUE_IPFS, async (job) => {
const { data: { data } } = job;
await this._indexer.pushToIPFS(data);
await this._jobQueue.markComplete(job);
});
}
}
export const main = async (): Promise<any> => {
const argv = await yargs(hideBin(process.argv))
.option('f', {
alias: 'config-file',
demandOption: true,
describe: 'configuration file path (toml)',
type: 'string',
default: DEFAULT_CONFIG_PATH
})
.argv;
const config: Config = await getConfig(argv.f);
const { ethClient, postgraphileClient, ethProvider } = await initClients(config);
const db = new Database(config.database);
await db.init();
const graphDb = new GraphDatabase(config.database, path.resolve(__dirname, 'entity/*'));
await graphDb.init();
const graphWatcher = new GraphWatcher(graphDb, postgraphileClient, config.server.subgraphPath);
const indexer = new Indexer(config.server, db, ethClient, postgraphileClient, ethProvider, graphWatcher);
graphWatcher.setIndexer(indexer);
await graphWatcher.init();
const jobQueueConfig = config.jobQueue;
assert(jobQueueConfig, 'Missing job queue config');
const { dbConnectionString, maxCompletionLagInSecs } = jobQueueConfig;
assert(dbConnectionString, 'Missing job queue db connection string');
const jobQueue = new JobQueue({ dbConnectionString, maxCompletionLag: maxCompletionLagInSecs });
await jobQueue.start();
const jobRunner = new JobRunner(jobQueueConfig, indexer, jobQueue);
await jobRunner.start();
};
main().then(() => {
log('Starting job runner...');
}).catch(err => {
log(err);
});
process.on('uncaughtException', err => {
log('uncaughtException', err);
});

210
packages/eden-watcher/src/resolvers.ts Normal file
View File

@ -0,0 +1,210 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import assert from 'assert';
import BigInt from 'apollo-type-bigint';
import debug from 'debug';
import { Indexer } from './indexer';
import { EventWatcher } from './events';
import { Producer } from './entity/Producer';
import { ProducerSet } from './entity/ProducerSet';
import { ProducerSetChange } from './entity/ProducerSetChange';
import { ProducerRewardCollectorChange } from './entity/ProducerRewardCollectorChange';
import { RewardScheduleEntry } from './entity/RewardScheduleEntry';
import { RewardSchedule } from './entity/RewardSchedule';
import { ProducerEpoch } from './entity/ProducerEpoch';
import { Epoch } from './entity/Epoch';
import { SlotClaim } from './entity/SlotClaim';
import { Slot } from './entity/Slot';
import { Staker } from './entity/Staker';
import { Network } from './entity/Network';
import { Distributor } from './entity/Distributor';
import { Distribution } from './entity/Distribution';
import { Claim } from './entity/Claim';
import { Slash } from './entity/Slash';
import { Account } from './entity/Account';
const log = debug('vulcanize:resolver');
export const createResolvers = async (indexer: Indexer, eventWatcher: EventWatcher): Promise<any> => {
assert(indexer);
return {
BigInt: new BigInt('bigInt'),
Event: {
__resolveType: (obj: any) => {
assert(obj.__typename);
return obj.__typename;
}
},
Subscription: {
onEvent: {
subscribe: () => eventWatcher.getEventIterator()
}
},
Mutation: {
watchContract: (_: any, { address, kind, checkpoint, startingBlock }: { address: string, kind: string, checkpoint: boolean, startingBlock: number }): Promise<boolean> => {
log('watchContract', address, kind, checkpoint, startingBlock);
return indexer.watchContract(address, kind, checkpoint, startingBlock);
}
},
Query: {
producer: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Producer | undefined> => {
log('producer', id, blockHash);
return indexer.getSubgraphEntity(Producer, id, blockHash);
},
producerSet: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<ProducerSet | undefined> => {
log('producerSet', id, blockHash);
return indexer.getSubgraphEntity(ProducerSet, id, blockHash);
},
producerSetChange: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<ProducerSetChange | undefined> => {
log('producerSetChange', id, blockHash);
return indexer.getSubgraphEntity(ProducerSetChange, id, blockHash);
},
producerRewardCollectorChange: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<ProducerRewardCollectorChange | undefined> => {
log('producerRewardCollectorChange', id, blockHash);
return indexer.getSubgraphEntity(ProducerRewardCollectorChange, id, blockHash);
},
rewardScheduleEntry: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<RewardScheduleEntry | undefined> => {
log('rewardScheduleEntry', id, blockHash);
return indexer.getSubgraphEntity(RewardScheduleEntry, id, blockHash);
},
rewardSchedule: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<RewardSchedule | undefined> => {
log('rewardSchedule', id, blockHash);
return indexer.getSubgraphEntity(RewardSchedule, id, blockHash);
},
producerEpoch: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<ProducerEpoch | undefined> => {
log('producerEpoch', id, blockHash);
return indexer.getSubgraphEntity(ProducerEpoch, id, blockHash);
},
// block: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Block | undefined> => {
// log('block', id, blockHash);
// return indexer.getSubgraphEntity(Block, id, blockHash);
// },
epoch: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Epoch | undefined> => {
log('epoch', id, blockHash);
return indexer.getSubgraphEntity(Epoch, id, blockHash);
},
slotClaim: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<SlotClaim | undefined> => {
log('slotClaim', id, blockHash);
return indexer.getSubgraphEntity(SlotClaim, id, blockHash);
},
slot: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Slot | undefined> => {
log('slot', id, blockHash);
return indexer.getSubgraphEntity(Slot, id, blockHash);
},
staker: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Staker | undefined> => {
log('staker', id, blockHash);
return indexer.getSubgraphEntity(Staker, id, blockHash);
},
network: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Network | undefined> => {
log('network', id, blockHash);
return indexer.getSubgraphEntity(Network, id, blockHash);
},
distributor: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Distributor | undefined> => {
log('distributor', id, blockHash);
return indexer.getSubgraphEntity(Distributor, id, blockHash);
},
distribution: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Distribution | undefined> => {
log('distribution', id, blockHash);
return indexer.getSubgraphEntity(Distribution, id, blockHash);
},
claim: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Claim | undefined> => {
log('claim', id, blockHash);
return indexer.getSubgraphEntity(Claim, id, blockHash);
},
slash: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Slash | undefined> => {
log('slash', id, blockHash);
return indexer.getSubgraphEntity(Slash, id, blockHash);
},
account: async (_: any, { id, blockHash }: { id: string, blockHash: string }): Promise<Account | undefined> => {
log('account', id, blockHash);
return indexer.getSubgraphEntity(Account, id, blockHash);
},
events: async (_: any, { blockHash, contractAddress, name }: { blockHash: string, contractAddress: string, name?: string }) => {
log('events', blockHash, contractAddress, name);
const block = await indexer.getBlockProgress(blockHash);
if (!block || !block.isComplete) {
throw new Error(`Block hash ${blockHash} number ${block?.blockNumber} not processed yet`);
}
const events = await indexer.getEventsByFilter(blockHash, contractAddress, name);
return events.map(event => indexer.getResultEvent(event));
},
eventsInRange: async (_: any, { fromBlockNumber, toBlockNumber }: { fromBlockNumber: number, toBlockNumber: number }) => {
log('eventsInRange', fromBlockNumber, toBlockNumber);
const { expected, actual } = await indexer.getProcessedBlockCountForRange(fromBlockNumber, toBlockNumber);
if (expected !== actual) {
throw new Error(`Range not available, expected ${expected}, got ${actual} blocks in range`);
}
const events = await indexer.getEventsInRange(fromBlockNumber, toBlockNumber);
return events.map(event => indexer.getResultEvent(event));
},
getStateByCID: async (_: any, { cid }: { cid: string }) => {
log('getStateByCID', cid);
const ipldBlock = await indexer.getIPLDBlockByCid(cid);
return ipldBlock && ipldBlock.block.isComplete ? indexer.getResultIPLDBlock(ipldBlock) : undefined;
},
getState: async (_: any, { blockHash, contractAddress, kind = 'diff' }: { blockHash: string, contractAddress: string, kind: string }) => {
log('getState', blockHash, contractAddress, kind);
const ipldBlock = await indexer.getPrevIPLDBlock(blockHash, contractAddress, kind);
return ipldBlock && ipldBlock.block.isComplete ? indexer.getResultIPLDBlock(ipldBlock) : undefined;
}
}
};
};

391
packages/eden-watcher/src/schema.gql Normal file
View File

@ -0,0 +1,391 @@
scalar BigInt
type Proof {
data: String!
}
type ResultBoolean {
value: Boolean!
proof: Proof
}
type ResultString {
value: String!
proof: Proof
}
type ResultInt {
value: Int!
proof: Proof
}
type ResultBigInt {
value: BigInt!
proof: Proof
}
type Block {
cid: String!
hash: String!
number: Int!
timestamp: Int!
parentHash: String!
}
type Transaction {
hash: String!
index: Int!
from: String!
to: String!
}
type ResultEvent {
block: Block!
tx: Transaction!
contract: String!
eventIndex: Int!
event: Event!
proof: Proof
}
union Event = TransferERC20Event | ApprovalERC20Event | AuthorizationUsedEvent | AdminUpdatedEvent | TaxRateUpdatedEvent | SlotClaimedEvent | SlotDelegateUpdatedEvent | StakeEvent | UnstakeEvent | WithdrawEvent | TransferERC721Event | ApprovalERC721Event | ApprovalForAllEvent | BlockProducerAddedEvent | BlockProducerRemovedEvent | BlockProducerRewardCollectorChangedEvent | ClaimedEvent | SlashedEvent | MerkleRootUpdatedEvent | AccountUpdatedEvent | PermanentURIEvent | GovernanceChangedEvent | UpdateThresholdChangedEvent | RoleAdminChangedEvent | RoleGrantedEvent | RoleRevokedEvent | RewardScheduleChangedEvent
type TransferERC20Event {
from: String!
to: String!
value: BigInt!
}
type ApprovalERC20Event {
owner: String!
spender: String!
value: BigInt!
}
type AuthorizationUsedEvent {
authorizer: String!
nonce: String!
}
type AdminUpdatedEvent {
newAdmin: String!
oldAdmin: String!
}
type TaxRateUpdatedEvent {
newNumerator: Int!
newDenominator: Int!
oldNumerator: Int!
oldDenominator: Int!
}
type SlotClaimedEvent {
slot: Int!
owner: String!
delegate: String!
newBidAmount: Int!
oldBidAmount: Int!
taxNumerator: Int!
taxDenominator: Int!
}
type SlotDelegateUpdatedEvent {
slot: Int!
owner: String!
newDelegate: String!
oldDelegate: String!
}
type StakeEvent {
staker: String!
stakeAmount: BigInt!
}
type UnstakeEvent {
staker: String!
unstakedAmount: BigInt!
}
type WithdrawEvent {
withdrawer: String!
withdrawalAmount: BigInt!
}
type TransferERC721Event {
from: String!
to: String!
tokenId: BigInt!
}
type ApprovalERC721Event {
owner: String!
approved: String!
tokenId: BigInt!
}
type ApprovalForAllEvent {
owner: String!
operator: String!
approved: Boolean!
}
type BlockProducerAddedEvent {
producer: String!
}
type BlockProducerRemovedEvent {
producer: String!
}
type BlockProducerRewardCollectorChangedEvent {
producer: String!
collector: String!
}
type RewardScheduleChangedEvent {
# Note: dummy property added as server throws an error for type without any fields.
dummy: String
}
type ClaimedEvent {
index: BigInt!
totalEarned: BigInt!
account: String!
claimed: BigInt!
}
type SlashedEvent {
account: String!
slashed: BigInt!
}
type MerkleRootUpdatedEvent {
merkleRoot: String!
distributionNumber: BigInt!
metadataURI: String!
}
type AccountUpdatedEvent {
account: String!
totalClaimed: BigInt!
totalSlashed: BigInt!
}
type PermanentURIEvent {
value: String!
id: BigInt!
}
type GovernanceChangedEvent {
from: String!
to: String!
}
type UpdateThresholdChangedEvent {
updateThreshold: BigInt!
}
type RoleAdminChangedEvent {
role: String!
previousAdminRole: String!
newAdminRole: String!
}
type RoleGrantedEvent {
role: String!
account: String!
sender: String!
}
type RoleRevokedEvent {
role: String!
account: String!
sender: String!
}
type ResultIPLDBlock {
block: Block!
contractAddress: String!
cid: String!
kind: String!
data: String!
}
type Query {
events(blockHash: String!, contractAddress: String!, name: String): [ResultEvent!]
eventsInRange(fromBlockNumber: Int!, toBlockNumber: Int!): [ResultEvent!]
producer(id: String!, blockHash: String!): Producer!
producerSet(id: String!, blockHash: String!): ProducerSet!
producerSetChange(id: String!, blockHash: String!): ProducerSetChange!
producerRewardCollectorChange(id: String!, blockHash: String!): ProducerRewardCollectorChange!
rewardScheduleEntry(id: String!, blockHash: String!): RewardScheduleEntry!
rewardSchedule(id: String!, blockHash: String!): RewardSchedule!
producerEpoch(id: String!, blockHash: String!): ProducerEpoch!
epoch(id: String!, blockHash: String!): Epoch!
slotClaim(id: String!, blockHash: String!): SlotClaim!
slot(id: String!, blockHash: String!): Slot!
staker(id: String!, blockHash: String!): Staker!
network(id: String!, blockHash: String!): Network!
distributor(id: String!, blockHash: String!): Distributor!
distribution(id: String!, blockHash: String!): Distribution!
claim(id: String!, blockHash: String!): Claim!
slash(id: String!, blockHash: String!): Slash!
account(id: String!, blockHash: String!): Account!
getStateByCID(cid: String!): ResultIPLDBlock
getState(blockHash: String!, contractAddress: String!, kind: String): ResultIPLDBlock
}
type Producer {
id: ID!
active: Boolean!
rewardCollector: String
rewards: BigInt!
confirmedBlocks: BigInt!
pendingEpochBlocks: BigInt!
}
type ProducerSet {
id: ID!
producers: [Producer!]!
}
type ProducerSetChange {
id: ID!
blockNumber: BigInt!
producer: String!
changeType: ProducerSetChangeType!
}
enum ProducerSetChangeType {
Added
Removed
}
type ProducerRewardCollectorChange {
id: ID!
blockNumber: BigInt!
producer: String!
rewardCollector: String!
}
type RewardScheduleEntry {
id: ID!
startTime: BigInt!
epochDuration: BigInt!
rewardsPerEpoch: BigInt!
}
type RewardSchedule {
id: ID!
rewardScheduleEntries: [RewardScheduleEntry!]!
lastEpoch: Epoch
pendingEpoch: Epoch
activeRewardScheduleEntry: RewardScheduleEntry
}
type Epoch {
id: ID!
finalized: Boolean!
epochNumber: BigInt!
startBlock: Block
endBlock: Block
producerBlocks: BigInt!
allBlocks: BigInt!
producerBlocksRatio: String!
producerRewards: [ProducerEpoch!]!
}
type ProducerEpoch {
id: ID!
address: String!
epoch: Epoch!
totalRewards: BigInt!
blocksProduced: BigInt!
blocksProducedRatio: String!
}
type SlotClaim {
id: ID!
slot: Slot!
owner: String!
winningBid: BigInt!
oldBid: BigInt!
startTime: BigInt!
expirationTime: BigInt!
taxRatePerDay: String!
}
type Slot {
id: ID!
owner: String!
delegate: String!
winningBid: BigInt!
oldBid: BigInt!
startTime: BigInt!
expirationTime: BigInt!
taxRatePerDay: String!
claims: [SlotClaim!]!
}
type Staker {
id: ID!
staked: BigInt!
rank: BigInt
}
type Network {
id: ID!
slot0: Slot
slot1: Slot
slot2: Slot
stakers: [Staker!]!
numStakers: BigInt
totalStaked: BigInt!
stakedPercentiles: [BigInt!]!
}
type Distributor {
id: ID!
currentDistribution: Distribution
}
type Distribution {
id: ID!
distributor: Distributor!
timestamp: BigInt!
distributionNumber: BigInt!
merkleRoot: String!
metadataURI: String!
}
type Claim {
id: ID!
timestamp: BigInt!
index: BigInt!
account: Account!
totalEarned: BigInt!
claimed: BigInt!
}
type Account {
id: ID!
totalClaimed: BigInt!
totalSlashed: BigInt!
claims: [Claim!]!
slashes: [Slash!]!
}
type Slash {
id: ID!
timestamp: BigInt!
account: Account!
slashed: BigInt!
}
type Mutation {
watchContract(address: String!, kind: String!, checkpoint: Boolean!, startingBlock: Int): Boolean!
}
type Subscription {
onEvent: ResultEvent!
}

100
packages/eden-watcher/src/server.ts Normal file
View File

@ -0,0 +1,100 @@
//
// Copyright 2021 Vulcanize, Inc.
//
import fs from 'fs';
import path from 'path';
import assert from 'assert';
import 'reflect-metadata';
import express, { Application } from 'express';
import { ApolloServer, PubSub } from 'apollo-server-express';
import yargs from 'yargs';
import { hideBin } from 'yargs/helpers';
import debug from 'debug';
import 'graphql-import-node';
import { createServer } from 'http';
import { DEFAULT_CONFIG_PATH, getConfig, Config, JobQueue, KIND_ACTIVE, initClients } from '@vulcanize/util';
import { GraphWatcher, Database as GraphDatabase } from '@vulcanize/graph-node';
import { createResolvers } from './resolvers';
import { Indexer } from './indexer';
import { Database } from './database';
import { EventWatcher } from './events';
const log = debug('vulcanize:server');
export const main = async (): Promise<any> => {
const argv = await yargs(hideBin(process.argv))
.option('f', {
alias: 'config-file',
demandOption: true,
describe: 'configuration file path (toml)',
type: 'string',
default: DEFAULT_CONFIG_PATH
})
.argv;
const config: Config = await getConfig(argv.f);
const { ethClient, postgraphileClient, ethProvider } = await initClients(config);
const { host, port, kind: watcherKind } = config.server;
const db = new Database(config.database);
await db.init();
const graphDb = new GraphDatabase(config.database, path.resolve(__dirname, 'entity/*'));
await graphDb.init();
const graphWatcher = new GraphWatcher(graphDb, postgraphileClient, config.server.subgraphPath);
// Note: In-memory pubsub works fine for now, as each watcher is a single process anyway.
// Later: https://www.apollographql.com/docs/apollo-server/data/subscriptions/#production-pubsub-libraries
const pubsub = new PubSub();
const indexer = new Indexer(config.server, db, ethClient, postgraphileClient, ethProvider, graphWatcher);
graphWatcher.setIndexer(indexer);
await graphWatcher.init();
const jobQueueConfig = config.jobQueue;
assert(jobQueueConfig, 'Missing job queue config');
const { dbConnectionString, maxCompletionLagInSecs } = jobQueueConfig;
assert(dbConnectionString, 'Missing job queue db connection string');
const jobQueue = new JobQueue({ dbConnectionString, maxCompletionLag: maxCompletionLagInSecs });
const eventWatcher = new EventWatcher(config.upstream, ethClient, postgraphileClient, indexer, pubsub, jobQueue);
if (watcherKind === KIND_ACTIVE) {
await jobQueue.start();
await eventWatcher.start();
}
const resolvers = await createResolvers(indexer, eventWatcher);
const app: Application = express();
const typeDefs = fs.readFileSync(path.join(__dirname, 'schema.gql')).toString();
const server = new ApolloServer({
typeDefs,
resolvers
});
await server.start();
server.applyMiddleware({ app });
const httpServer = createServer(app);
server.installSubscriptionHandlers(httpServer);
httpServer.listen(port, host, () => {
log(`Server is listening on host ${host} port ${port}`);
});
return { app, server };
};
main().then(() => {
log('Starting server...');
}).catch(err => {
log(err);
});

74
packages/eden-watcher/tsconfig.json Normal file
View File

@ -0,0 +1,74 @@
{
"compilerOptions": {
/* Visit https://aka.ms/tsconfig.json to read more about this file */
/* Basic Options */
// "incremental": true, /* Enable incremental compilation */
"target": "es5", /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019', 'ES2020', 'ES2021', or 'ESNEXT'. */
"module": "commonjs", /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', 'es2020', or 'ESNext'. */
// "lib": [], /* Specify library files to be included in the compilation. */
// "allowJs": true, /* Allow javascript files to be compiled. */
// "checkJs": true, /* Report errors in .js files. */
// "jsx": "preserve", /* Specify JSX code generation: 'preserve', 'react-native', 'react', 'react-jsx' or 'react-jsxdev'. */
// "declaration": true, /* Generates corresponding '.d.ts' file. */
// "declarationMap": true, /* Generates a sourcemap for each corresponding '.d.ts' file. */
// "sourceMap": true, /* Generates corresponding '.map' file. */
// "outFile": "./", /* Concatenate and emit output to single file. */
"outDir": "dist", /* Redirect output structure to the directory. */
// "rootDir": "./", /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */
// "composite": true, /* Enable project compilation */
// "tsBuildInfoFile": "./", /* Specify file to store incremental compilation information */
// "removeComments": true, /* Do not emit comments to output. */
// "noEmit": true, /* Do not emit outputs. */
// "importHelpers": true, /* Import emit helpers from 'tslib'. */
// "downlevelIteration": true, /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */
// "isolatedModules": true, /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */
/* Strict Type-Checking Options */
"strict": true, /* Enable all strict type-checking options. */
// "noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */
// "strictNullChecks": true, /* Enable strict null checks. */
// "strictFunctionTypes": true, /* Enable strict checking of function types. */
// "strictBindCallApply": true, /* Enable strict 'bind', 'call', and 'apply' methods on functions. */
// "strictPropertyInitialization": true, /* Enable strict checking of property initialization in classes. */
// "noImplicitThis": true, /* Raise error on 'this' expressions with an implied 'any' type. */
// "alwaysStrict": true, /* Parse in strict mode and emit "use strict" for each source file. */
/* Additional Checks */
// "noUnusedLocals": true, /* Report errors on unused locals. */
// "noUnusedParameters": true, /* Report errors on unused parameters. */
// "noImplicitReturns": true, /* Report error when not all code paths in function return a value. */
// "noFallthroughCasesInSwitch": true, /* Report errors for fallthrough cases in switch statement. */
// "noUncheckedIndexedAccess": true, /* Include 'undefined' in index signature results */
// "noImplicitOverride": true, /* Ensure overriding members in derived classes are marked with an 'override' modifier. */
// "noPropertyAccessFromIndexSignature": true, /* Require undeclared properties from index signatures to use element accesses. */
/* Module Resolution Options */
// "moduleResolution": "node", /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */
// "baseUrl": "./", /* Base directory to resolve non-absolute module names. */
// "paths": {}, /* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */
// "rootDirs": [], /* List of root folders whose combined content represents the structure of the project at runtime. */
// "typeRoots": [], /* List of folders to include type definitions from. */
// "types": [], /* Type declaration files to be included in compilation. */
// "allowSyntheticDefaultImports": true, /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */
"esModuleInterop": true, /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
// "preserveSymlinks": true, /* Do not resolve the real path of symlinks. */
// "allowUmdGlobalAccess": true, /* Allow accessing UMD globals from modules. */
/* Source Map Options */
// "sourceRoot": "", /* Specify the location where debugger should locate TypeScript files instead of source locations. */
// "mapRoot": "", /* Specify the location where debugger should locate map files instead of generated locations. */
// "inlineSourceMap": true, /* Emit a single file with source maps instead of having a separate file. */
// "inlineSources": true, /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */
/* Experimental Options */
"experimentalDecorators": true, /* Enables experimental support for ES7 decorators. */
// "emitDecoratorMetadata": true, /* Enables experimental support for emitting type metadata for decorators. */
/* Advanced Options */
"skipLibCheck": true, /* Skip type checking of declaration files. */
"forceConsistentCasingInFileNames": true, /* Disallow inconsistently-cased references to the same file. */
"resolveJsonModule": true /* Enabling the option allows importing JSON, and validating the types in that JSON file. */
},
"include": ["src/**/*"]
}