Parse events for multiple contracts in the generated code (#95)

* Parse events for multiple contracts in the generated code

* Use contract wise artifacts in the generated indexer methods

* Update codegen docs to use config file to generate a watcher

* Add watcher generation config to eden-watcher
This commit is contained in:
prathamesh0 2021-12-23 17:26:03 +05:30 committed by nabarun
parent e5faba8e68
commit e883463aa6
10 changed files with 307 additions and 167 deletions

View File

@ -6,68 +6,69 @@
* Install required packages: * Install required packages:
```bash ```bash
yarn yarn
``` ```
* Build files: * Build files:
```bash ```bash
yarn build yarn build
``` ```
## Run ## Run
* Run the following command to generate a watcher from a contract file: * Create a `.yaml` config file in the following format for generating a watcher:
```bash ```yaml
yarn codegen --input-files <input-file-paths> --contract-names <contract-names> --output-folder [output-folder] --mode [eth_call | storage | all | none] --flatten [true | false] --kind [lazy | active] --port [server-port] --subgraph-path [subgraph-build-path] # Example config.yaml
# Contracts to watch (required).
contracts:
# Contract name.
- name: Example
# Contract file path or an url.
path: ../graph-node/test/contracts/Example.sol
# Contract kind (should match that in {subgraphPath}/subgraph.yaml if subgraphPath provided)
kind: Example1
# Output folder path (logs output using `stdout` if not provided).
outputFolder: ../test-watcher
# Code generation mode [eth_call | storage | all | none] (default: all).
mode: all
# Kind of watcher [lazy | active] (default: active).
kind: active
# Watcher server port (default: 3008).
port: 3008
# Flatten the input contract file(s) [true | false] (default: true).
flatten: true
# Path to the subgraph build (optional).
subgraphPath: ../graph-node/test/subgraph/example1/build
# NOTE: When passed an *URL* as contract path, it is assumed that it points to an already flattened contract file.
``` ```
* `input-files`(alias: `i`): Input contract file path(s) or URL(s) (required). * Run the following command to generate a watcher from contract(s):
* `contract-names`(alias: `c`): Contract name(s) (in order of `input-files`) (required).
* `output-folder`(alias: `o`): Output folder path. (logs output using `stdout` if not provided).
* `mode`(alias: `m`): Code generation mode (default: `all`).
* `flatten`(alias: `f`): Flatten the input contract file (default: `true`).
* `kind` (alias: `k`): Kind of watcher (default: `active`).
* `port` (alias: `p`): Server port (default: `3008`).
* `subgraph-path` (alias: `s`): Path to the subgraph build.
**Note**: When passed an *URL* in `input-files`, it is assumed that it points to an already flattened contract file.
Examples:
Generate code in `storage` mode, `lazy` kind.
```bash ```bash
yarn codegen --input-files ./test/examples/contracts/ERC721.sol --contract-names ERC721 --output-folder ../my-erc721-watcher --mode storage --kind lazy yarn codegen --config-file <config-file-path>
``` ```
Generate code in `eth_call` mode using a contract provided by an URL. * `config-file`(alias: `c`): Watcher generation config file path (yaml) (required).
```bash Example:
yarn codegen --input-files https://git.io/Jupci --contract-names ERC721 --output-folder ../my-erc721-watcher --mode eth_call
```
Generate code for `ERC721` in both `eth_call` and `storage` mode, `active` kind. * Generate code using a config file `config.yaml`:
```bash ```bash
yarn codegen --input-files ../../node_modules/@openzeppelin/contracts/token/ERC721/ERC721.sol --contract-names ERC721 --output-folder ../demo-erc721-watcher --mode all --kind active yarn codegen --config-file ./config.yaml
``` ```
Generate code for `ERC20` contract in both `eth_call` and `storage` mode, `active` kind: This will create a folder containing the generated code at the path provided in config. Follow the steps in [Run Generated Watcher](#run-generated-watcher) to setup and run the generated watcher.
```bash
yarn codegen --input-files ../../node_modules/@openzeppelin/contracts/token/ERC20/ERC20.sol --contract-names ERC20 --output-folder ../demo-erc20-watcher --mode all --kind active
```
This will create a folder called `demo-erc20-watcher` containing the generated code at the specified path. Follow the steps in [Run Generated Watcher](#run-generated-watcher) to setup and run the generated watcher.
Generate code for `Eden` contracts in `none` mode, `active` kind:
```bash
yarn codegen --input-files ~/vulcanize/governance/contracts/EdenNetwork.sol ~/vulcanize/governance/contracts/MerkleDistributor.sol ~/vulcanize/governance/contracts/DistributorGovernance.sol --contract-names EdenNetwork MerkleDistributor DistributorGovernance --output-folder ../demo-eden-watcher --mode none --kind active --subgraph-path ~/vulcanize/eden-data/packages/subgraph/build
```
## Run Generated Watcher ## Run Generated Watcher
@ -103,14 +104,12 @@
* Generating state: * Generating state:
* Edit the custom hook function `createInitialCheckpoint` (triggered on watch-contract, checkpoint: `true`) in `src/hooks.ts` to save an initial checkpoint `IPLDBlock` using the `Indexer` object. * Edit the custom hook function `createInitialState` (triggered if the watcher passes the start block, checkpoint: `true`) in `src/hooks.ts` to save an initial state `IPLDBlock` using the `Indexer` object.
* Edit the custom hook function `createStateDiff` (triggered on a block) in `src/hooks.ts` to save the state in a `diff` `IPLDBlock` using the `Indexer` object. The default state (if exists) is updated. * Edit the custom hook function `createStateDiff` (triggered on a block) in `src/hooks.ts` to save the state in a `diff` `IPLDBlock` using the `Indexer` object. The default state (if exists) is updated.
* Edit the custom hook function `createStateCheckpoint` (triggered just before default and CLI checkpoint) in `src/hooks.ts` to save the state in a `checkpoint` `IPLDBlock` using the `Indexer` object. * Edit the custom hook function `createStateCheckpoint` (triggered just before default and CLI checkpoint) in `src/hooks.ts` to save the state in a `checkpoint` `IPLDBlock` using the `Indexer` object.
* The existing example hooks in `src/hooks.ts` are for an `ERC20` contract.
### Run ### Run
* Run lint: * Run lint:

View File

@ -9,12 +9,14 @@ import yargs from 'yargs';
import { hideBin } from 'yargs/helpers'; import { hideBin } from 'yargs/helpers';
import assert from 'assert'; import assert from 'assert';
import { Writable } from 'stream'; import { Writable } from 'stream';
import yaml from 'js-yaml';
import os from 'os';
import { flatten } from '@poanet/solidity-flattener'; import { flatten } from '@poanet/solidity-flattener';
import { parse, visit } from '@solidity-parser/parser'; import { parse, visit } from '@solidity-parser/parser';
import { KIND_ACTIVE, KIND_LAZY } from '@vulcanize/util'; import { KIND_ACTIVE, KIND_LAZY } from '@vulcanize/util';
import { MODE_ETH_CALL, MODE_STORAGE, MODE_ALL, MODE_NONE } from './utils/constants'; import { MODE_ETH_CALL, MODE_STORAGE, MODE_ALL, MODE_NONE, DEFAULT_PORT } from './utils/constants';
import { Visitor } from './visitor'; import { Visitor } from './visitor';
import { exportServer } from './server'; import { exportServer } from './server';
import { exportConfig } from './config'; import { exportConfig } from './config';
@ -33,86 +35,50 @@ import { exportCheckpoint } from './checkpoint';
import { exportState } from './export-state'; import { exportState } from './export-state';
import { importState } from './import-state'; import { importState } from './import-state';
import { exportInspectCID } from './inspect-cid'; import { exportInspectCID } from './inspect-cid';
import { getContractKindList } from './utils/subgraph';
const main = async (): Promise<void> => { const main = async (): Promise<void> => {
const argv = await yargs(hideBin(process.argv)) const argv = await yargs(hideBin(process.argv))
.option('input-files', { .option('config-file', {
alias: 'i',
demandOption: true,
describe: 'Input contract file path(s) or url(s).',
type: 'array'
})
.option('contract-names', {
alias: 'c', alias: 'c',
demandOption: true, demandOption: true,
describe: 'Contract name(s).', describe: 'Watcher generation config file path (yaml)',
type: 'array'
})
.option('output-folder', {
alias: 'o',
describe: 'Output folder path.',
type: 'string'
})
.option('mode', {
alias: 'm',
describe: 'Code generation mode.',
type: 'string',
default: MODE_ALL,
choices: [MODE_ETH_CALL, MODE_STORAGE, MODE_ALL, MODE_NONE]
})
.option('kind', {
alias: 'k',
describe: 'Watcher kind.',
type: 'string',
default: KIND_ACTIVE,
choices: [KIND_ACTIVE, KIND_LAZY]
})
.option('port', {
alias: 'p',
describe: 'Server port.',
type: 'number',
default: 3008
})
.option('flatten', {
alias: 'f',
describe: 'Flatten the input contract file.',
type: 'boolean',
default: true
})
.option('subgraph-path', {
alias: 's',
describe: 'Path to the subgraph build.',
type: 'string' type: 'string'
}) })
.argv; .argv;
// Create an array of flattened contract strings. const config = getConfig(path.resolve(argv['config-file']));
const contractStrings: string[] = [];
for (const inputFile of argv['input-files']) { // Create an array of flattened contract strings.
assert(typeof inputFile === 'string', 'Input file path should be a string'); const contracts: any = [];
for (const contract of config.contracts) {
const inputFile = contract.path;
assert(typeof inputFile === 'string', 'Contract input file path should be a string');
let contractString;
if (inputFile.startsWith('http')) { if (inputFile.startsWith('http')) {
// Assume flattened file in case of URL. // Assume flattened file in case of URL.
const response = await fetch(inputFile); const response = await fetch(inputFile);
const contractString = await response.text(); contractString = await response.text();
contractStrings.push(contractString);
} else { } else {
contractStrings.push(argv.flatten contractString = config.flatten
? await flatten(path.resolve(inputFile)) ? await flatten(path.resolve(inputFile))
: fs.readFileSync(path.resolve(inputFile)).toString() : fs.readFileSync(path.resolve(inputFile)).toString();
);
} }
contracts.push({ contractString, contractName: contract.name, contractKind: contract.kind });
} }
const visitor = new Visitor(); const visitor = new Visitor();
parseAndVisit(contractStrings, visitor, argv.mode); parseAndVisit(visitor, contracts, config.mode);
generateWatcher(contractStrings, visitor, argv); generateWatcher(visitor, contracts, config);
}; };
function parseAndVisit (contractStrings: string[], visitor: Visitor, mode: string) { function parseAndVisit (visitor: Visitor, contracts: any[], mode: string) {
const eventDefinitionVisitor = visitor.eventDefinitionVisitor.bind(visitor); const eventDefinitionVisitor = visitor.eventDefinitionVisitor.bind(visitor);
let functionDefinitionVisitor; let functionDefinitionVisitor;
let stateVariableDeclarationVisitor; let stateVariableDeclarationVisitor;
@ -127,13 +93,15 @@ function parseAndVisit (contractStrings: string[], visitor: Visitor, mode: strin
stateVariableDeclarationVisitor = visitor.stateVariableDeclarationVisitor.bind(visitor); stateVariableDeclarationVisitor = visitor.stateVariableDeclarationVisitor.bind(visitor);
} }
for (const contractString of contractStrings) { for (const contract of contracts) {
// Get the abstract syntax tree for the flattened contract. // Get the abstract syntax tree for the flattened contract.
const ast = parse(contractString); const ast = parse(contract.contractString);
// Filter out library nodes. // Filter out library nodes.
ast.children = ast.children.filter(child => !(child.type === 'ContractDefinition' && child.kind === 'library')); ast.children = ast.children.filter(child => !(child.type === 'ContractDefinition' && child.kind === 'library'));
visitor.setContract(contract.contractName, contract.contractKind);
visit(ast, { visit(ast, {
FunctionDefinition: functionDefinitionVisitor, FunctionDefinition: functionDefinitionVisitor,
StateVariableDeclaration: stateVariableDeclarationVisitor, StateVariableDeclaration: stateVariableDeclarationVisitor,
@ -142,11 +110,11 @@ function parseAndVisit (contractStrings: string[], visitor: Visitor, mode: strin
} }
} }
function generateWatcher (contractStrings: string[], visitor: Visitor, argv: any) { function generateWatcher (visitor: Visitor, contracts: any[], config: any) {
// Prepare directory structure for the watcher. // Prepare directory structure for the watcher.
let outputDir = ''; let outputDir = '';
if (argv['output-folder']) { if (config.outputFolder) {
outputDir = path.resolve(argv['output-folder']); outputDir = path.resolve(config.outputFolder);
if (!fs.existsSync(outputDir)) fs.mkdirSync(outputDir, { recursive: true }); if (!fs.existsSync(outputDir)) fs.mkdirSync(outputDir, { recursive: true });
const environmentsFolder = path.join(outputDir, 'environments'); const environmentsFolder = path.join(outputDir, 'environments');
@ -164,30 +132,26 @@ function generateWatcher (contractStrings: string[], visitor: Visitor, argv: any
let outStream: Writable; let outStream: Writable;
const contractNames = argv['contract-names'];
const inputFileNames: string[] = [];
// Export artifacts for the contracts. // Export artifacts for the contracts.
argv['input-files'].forEach((inputFile: string, index: number) => { config.contracts.forEach((contract: any, index: number) => {
const inputFileName = path.basename(inputFile, '.sol'); const inputFileName = path.basename(contract.path, '.sol');
inputFileNames.push(inputFileName);
outStream = outputDir outStream = outputDir
? fs.createWriteStream(path.join(outputDir, 'src/artifacts/', `${inputFileName}.json`)) ? fs.createWriteStream(path.join(outputDir, 'src/artifacts/', `${contract.name}.json`))
: process.stdout; : process.stdout;
exportArtifacts( exportArtifacts(
outStream, outStream,
contractStrings[index], contracts[index].contractString,
`${inputFileName}.sol`, `${inputFileName}.sol`,
contractNames[index] contract.name
); );
}); });
// Register the handlebar helpers to be used in the templates. // Register the handlebar helpers to be used in the templates.
registerHandlebarHelpers(); registerHandlebarHelpers();
visitor.visitSubgraph(argv['subgraph-path']); visitor.visitSubgraph(config.subgraphPath);
outStream = outputDir outStream = outputDir
? fs.createWriteStream(path.join(outputDir, 'src/schema.gql')) ? fs.createWriteStream(path.join(outputDir, 'src/schema.gql'))
@ -202,7 +166,7 @@ function generateWatcher (contractStrings: string[], visitor: Visitor, argv: any
outStream = outputDir outStream = outputDir
? fs.createWriteStream(path.join(outputDir, 'src/indexer.ts')) ? fs.createWriteStream(path.join(outputDir, 'src/indexer.ts'))
: process.stdout; : process.stdout;
visitor.exportIndexer(outStream, inputFileNames); visitor.exportIndexer(outStream, config.contracts);
outStream = outputDir outStream = outputDir
? fs.createWriteStream(path.join(outputDir, 'src/server.ts')) ? fs.createWriteStream(path.join(outputDir, 'src/server.ts'))
@ -212,7 +176,7 @@ function generateWatcher (contractStrings: string[], visitor: Visitor, argv: any
outStream = outputDir outStream = outputDir
? fs.createWriteStream(path.join(outputDir, 'environments/local.toml')) ? fs.createWriteStream(path.join(outputDir, 'environments/local.toml'))
: process.stdout; : process.stdout;
exportConfig(argv.kind, argv.port, path.basename(outputDir), outStream, argv['subgraph-path']); exportConfig(config.kind, config.port, path.basename(outputDir), outStream, config.subgraphPath);
outStream = outputDir outStream = outputDir
? fs.createWriteStream(path.join(outputDir, 'src/database.ts')) ? fs.createWriteStream(path.join(outputDir, 'src/database.ts'))
@ -237,7 +201,7 @@ function generateWatcher (contractStrings: string[], visitor: Visitor, argv: any
outStream = outputDir outStream = outputDir
? fs.createWriteStream(path.join(outputDir, 'README.md')) ? fs.createWriteStream(path.join(outputDir, 'README.md'))
: process.stdout; : process.stdout;
exportReadme(path.basename(outputDir), argv['contract-name'], outStream); exportReadme(path.basename(outputDir), config.port, outStream);
outStream = outputDir outStream = outputDir
? fs.createWriteStream(path.join(outputDir, 'src/events.ts')) ? fs.createWriteStream(path.join(outputDir, 'src/events.ts'))
@ -321,6 +285,61 @@ function generateWatcher (contractStrings: string[], visitor: Visitor, argv: any
exportInspectCID(outStream); exportInspectCID(outStream);
} }
function getConfig (configFile: string): any {
assert(fs.existsSync(configFile), `Config file not found at ${configFile}`);
// Read config.
const inputConfig = yaml.load(fs.readFileSync(configFile, 'utf8')) as any;
// Run validations on config fields.
if (inputConfig.mode) {
assert([MODE_ETH_CALL, MODE_STORAGE, MODE_ALL, MODE_NONE].includes(inputConfig.mode), 'Invalid code generation mode');
}
if (inputConfig.kind) {
assert([KIND_ACTIVE, KIND_LAZY].includes(inputConfig.kind), 'Invalid watcher kind');
}
if (inputConfig.port) {
assert(typeof inputConfig.port === 'number', 'Invalid watcher server port');
}
// Check that every input contract kind is present in the subgraph config.
let subgraphPath;
if (inputConfig.subgraphPath) {
// Resolve path.
subgraphPath = inputConfig.subgraphPath.replace(/^~/, os.homedir());
const subgraphKinds: string[] = getContractKindList(subgraphPath);
const inputKinds: string[] = inputConfig.contracts.map((contract: any) => contract.kind);
assert(
inputKinds.every((inputKind: string) => subgraphKinds.includes(inputKind)),
'Input contract kind not available in the subgraph.'
);
}
const inputFlatten = inputConfig.flatten;
const flatten = (inputFlatten === undefined || inputFlatten === null) ? true : inputFlatten;
// Resolve paths.
const contracts = inputConfig.contracts.map((contract: any) => {
contract.path = contract.path.replace(/^~/, os.homedir());
return contract;
});
return {
contracts,
outputFolder: inputConfig.outputFolder,
mode: inputConfig.mode || MODE_ALL,
kind: inputConfig.kind || KIND_ACTIVE,
port: inputConfig.port || DEFAULT_PORT,
flatten,
subgraphPath
};
}
main().catch(err => { main().catch(err => {
console.error(err); console.error(err);
}); });

View File

@ -37,7 +37,7 @@ export class Indexer {
* @param returnType Return type for the query. * @param returnType Return type for the query.
* @param stateVariableTypeName Type of the state variable in case of state variable query. * @param stateVariableTypeName Type of the state variable in case of state variable query.
*/ */
addQuery (mode: string, name: string, params: Array<Param>, returnType: string, stateVariableType?: string): void { addQuery (contract: string, mode: string, name: string, params: Array<Param>, returnType: string, stateVariableType?: string): void {
// Check if the query is already added. // Check if the query is already added.
if (this._queries.some(query => query.name === name)) { if (this._queries.some(query => query.name === name)) {
return; return;
@ -50,7 +50,8 @@ export class Indexer {
params: _.cloneDeep(params), params: _.cloneDeep(params),
returnType, returnType,
mode, mode,
stateVariableType stateVariableType,
contract
}; };
if (name.charAt(0) === '_') { if (name.charAt(0) === '_') {
@ -81,15 +82,16 @@ export class Indexer {
this._queries.push(queryObject); this._queries.push(queryObject);
} }
addEvent (name: string, params: Array<Param>): void { addEvent (name: string, params: Array<Param>, contractKind: string): void {
// Check if the event is already added. // Check if the event is already added.
if (this._events.some(event => event.name === name)) { if (this._events.some(event => event.name === name && event.kind === contractKind)) {
return; return;
} }
const eventObject = { const eventObject = {
name, name,
params: _.cloneDeep(params) params: _.cloneDeep(params),
kind: contractKind
}; };
eventObject.params = eventObject.params.map((param) => { eventObject.params = eventObject.params.map((param) => {
@ -169,20 +171,23 @@ export class Indexer {
/** /**
* Writes the indexer file generated from a template to a stream. * Writes the indexer file generated from a template to a stream.
* @param outStream A writable output stream to write the indexer file to. * @param outStream A writable output stream to write the indexer file to.
* @param inputFileName Input contract file name to be passed to the template. * @param contracts Input contracts to be passed to the template.
*/ */
exportIndexer (outStream: Writable, inputFileNames: string[]): void { exportIndexer (outStream: Writable, contracts: any[]): void {
const template = Handlebars.compile(this._templateString); const template = Handlebars.compile(this._templateString);
const eventNames = this._events.map((event: any) => event.name);
const obj = { const obj = {
inputFileNames, contracts,
queries: this._queries, queries: this._queries,
subgraphEntities: this._subgraphEntities, subgraphEntities: this._subgraphEntities,
constants: { constants: {
MODE_ETH_CALL, MODE_ETH_CALL,
MODE_STORAGE MODE_STORAGE
}, },
events: this._events events: this._events,
uniqueEvents: new Set(eventNames)
}; };
const indexer = template(obj); const indexer = template(obj);

View File

@ -12,15 +12,15 @@ const TEMPLATE_FILE = './templates/readme-template.handlebars';
/** /**
* Writes the README.md file generated from a template to a stream. * Writes the README.md file generated from a template to a stream.
* @param folderName Watcher folder name to be passed to the template. * @param folderName Watcher folder name to be passed to the template.
* @param contractName Input contract name given as title of the README. * @param port Watcher server port.
* @param outStream A writable output stream to write the README.md file to. * @param outStream A writable output stream to write the README.md file to.
*/ */
export function exportReadme (folderName: string, contractName: string, outStream: Writable): void { export function exportReadme (folderName: string, port: number, outStream: Writable): void {
const templateString = fs.readFileSync(path.resolve(__dirname, TEMPLATE_FILE)).toString(); const templateString = fs.readFileSync(path.resolve(__dirname, TEMPLATE_FILE)).toString();
const template = Handlebars.compile(templateString); const template = Handlebars.compile(templateString);
const readmeString = template({ const readmeString = template({
folderName, folderName,
contractName port
}); });
outStream.write(readmeString); outStream.write(readmeString);
} }

View File

@ -30,8 +30,8 @@ import {
} from '@vulcanize/util'; } from '@vulcanize/util';
import { GraphWatcher } from '@vulcanize/graph-node'; import { GraphWatcher } from '@vulcanize/graph-node';
{{#each inputFileNames as | inputFileName |}} {{#each contracts as | contract |}}
import artifacts from './artifacts/{{inputFileName}}.json'; import {{contract.name}}Artifacts from './artifacts/{{contract.name}}.json';
{{/each}} {{/each}}
import { Database } from './database'; import { Database } from './database';
import { createInitialState, handleEvent, createStateDiff, createStateCheckpoint } from './hooks'; import { createInitialState, handleEvent, createStateDiff, createStateCheckpoint } from './hooks';
@ -48,8 +48,12 @@ import { {{subgraphEntity.className}} } from './entity/{{subgraphEntity.classNam
const log = debug('vulcanize:indexer'); const log = debug('vulcanize:indexer');
{{#each events as | event |}} {{#each contracts as | contract |}}
const {{capitalize event.name}}_EVENT = '{{event.name}}'; const KIND_{{capitalize contract.name}} = '{{contract.kind}}';
{{/each}}
{{#each uniqueEvents as | event |}}
const {{capitalize event}}_EVENT = '{{event}}';
{{/each}} {{/each}}
export type ResultEvent = { export type ResultEvent = {
@ -99,9 +103,9 @@ export class Indexer implements IndexerInterface {
_serverConfig: ServerConfig _serverConfig: ServerConfig
_graphWatcher: GraphWatcher; _graphWatcher: GraphWatcher;
_abi: JsonFragment[] _abiMap: Map<string, JsonFragment[]>
_storageLayout: StorageLayout _storageLayoutMap: Map<string, StorageLayout>
_contract: ethers.utils.Interface _contractMap: Map<string, ethers.utils.Interface>
_ipfsClient: IPFSClient _ipfsClient: IPFSClient
@ -122,16 +126,22 @@ export class Indexer implements IndexerInterface {
this._baseIndexer = new BaseIndexer(this._serverConfig, this._db, this._ethClient, this._postgraphileClient, this._ethProvider, jobQueue, this._ipfsClient); this._baseIndexer = new BaseIndexer(this._serverConfig, this._db, this._ethClient, this._postgraphileClient, this._ethProvider, jobQueue, this._ipfsClient);
this._graphWatcher = graphWatcher; this._graphWatcher = graphWatcher;
const { abi, storageLayout } = artifacts; this._abiMap = new Map();
this._storageLayoutMap = new Map();
this._contractMap = new Map();
assert(abi); {{#each contracts as | contract |}}
assert(storageLayout); const { abi: {{contract.name}}ABI, storageLayout: {{contract.name}}StorageLayout } = {{contract.name}}Artifacts;
assert({{contract.name}}ABI);
assert({{contract.name}}StorageLayout);
this._abi = abi; {{/each}}
this._storageLayout = storageLayout; {{#each contracts as | contract |}}
this._abiMap.set(KIND_{{capitalize contract.name}}, {{contract.name}}ABI);
this._contract = new ethers.utils.Interface(this._abi); this._storageLayoutMap.set(KIND_{{capitalize contract.name}}, {{contract.name}}StorageLayout);
this._contractMap.set(KIND_{{capitalize contract.name}}, new ethers.utils.Interface({{contract.name}}ABI));
{{/each}}
this._entityTypesMap = new Map(); this._entityTypesMap = new Map();
this._populateEntityTypesMap(); this._populateEntityTypesMap();
@ -224,7 +234,10 @@ export class Indexer implements IndexerInterface {
const blockNumber = ethers.BigNumber.from(number).toNumber(); const blockNumber = ethers.BigNumber.from(number).toNumber();
{{#if (compare query.mode @root.constants.MODE_ETH_CALL)}} {{#if (compare query.mode @root.constants.MODE_ETH_CALL)}}
const contract = new ethers.Contract(contractAddress, this._abi, this._ethProvider); const abi = this._abiMap.get(KIND_{{capitalize query.contract}});
assert(abi);
const contract = new ethers.Contract(contractAddress, abi, this._ethProvider);
{{#if (compare query.returnType 'bigint')}} {{#if (compare query.returnType 'bigint')}}
let value = await contract.{{query.name}}( let value = await contract.{{query.name}}(
{{~#each query.params}}{{this.name}}, {{/each}}{ blockTag: blockHash }); {{~#each query.params}}{{this.name}}, {{/each}}{ blockTag: blockHash });
@ -239,8 +252,11 @@ export class Indexer implements IndexerInterface {
{{/if}} {{/if}}
{{~#if (compare query.mode @root.constants.MODE_STORAGE)}} {{~#if (compare query.mode @root.constants.MODE_STORAGE)}}
const storageLayout = this._storageLayoutMap.get(KIND_{{capitalize query.contract}});
assert(storageLayout);
const result = await this._baseIndexer.getStorageValue( const result = await this._baseIndexer.getStorageValue(
this._storageLayout, storageLayout,
blockHash, blockHash,
contractAddress, contractAddress,
'{{query.name}}'{{#if query.params.length}},{{/if}} '{{query.name}}'{{#if query.params.length}},{{/if}}
@ -413,10 +429,37 @@ export class Indexer implements IndexerInterface {
let eventInfo = {}; let eventInfo = {};
const { topics, data } = logObj; const { topics, data } = logObj;
const logDescription = this._contract.parseLog({ data, topics });
const contract = this._contractMap.get(kind);
assert(contract);
const logDescription = contract.parseLog({ data, topics });
switch (kind) {
{{#each contracts as | contract |}}
case KIND_{{capitalize contract.name}}: {
({ eventName, eventInfo } = this.parse{{contract.name}}Event(logDescription));
break;
}
{{/each}}
}
return {
eventName,
eventInfo,
eventSignature: logDescription.signature
};
}
{{#each contracts as | contract |}}
parse{{contract.name}}Event (logDescription: ethers.utils.LogDescription): { eventName: string, eventInfo: any } {
let eventName = UNKNOWN_EVENT_NAME;
let eventInfo = {};
switch (logDescription.name) { switch (logDescription.name) {
{{#each events as | event |}} {{#each ../events as | event |}}
{{#if (compare contract.kind event.kind)}}
case {{capitalize event.name}}_EVENT: { case {{capitalize event.name}}_EVENT: {
eventName = logDescription.name; eventName = logDescription.name;
{{#if event.params}} {{#if event.params}}
@ -435,16 +478,17 @@ export class Indexer implements IndexerInterface {
break; break;
} }
{{/if}}
{{/each}} {{/each}}
} }
return { return {
eventName, eventName,
eventInfo, eventInfo
eventSignature: logDescription.signature
}; };
} }
{{/each}}
async getHookStatus (): Promise<HookStatus | undefined> { async getHookStatus (): Promise<HookStatus | undefined> {
return this._db.getHookStatus(); return this._db.getHookStatus();
} }

View File

@ -1,4 +1,4 @@
# {{contractName}} Watcher # {{folderName}}
## Setup ## Setup
@ -59,14 +59,12 @@
* Generating state: * Generating state:
* Edit the custom hook function `createInitialCheckpoint` (triggered on watch-contract, checkpoint: `true`) in [hooks.ts](./src/hooks.ts) to save an initial checkpoint `IPLDBlock` using the `Indexer` object. * Edit the custom hook function `createInitialState` (triggered if the watcher passes the start block, checkpoint: `true`) in [hooks.ts](./src/hooks.ts) to save an initial state `IPLDBlock` using the `Indexer` object.
* Edit the custom hook function `createStateDiff` (triggered on a block) in [hooks.ts](./src/hooks.ts) to save the state in a `diff` `IPLDBlock` using the `Indexer` object. The default state (if exists) is updated. * Edit the custom hook function `createStateDiff` (triggered on a block) in [hooks.ts](./src/hooks.ts) to save the state in a `diff` `IPLDBlock` using the `Indexer` object. The default state (if exists) is updated.
* Edit the custom hook function `createStateCheckpoint` (triggered just before default and CLI checkpoint) in [hooks.ts](./src/hooks.ts) to save the state in a `checkpoint` `IPLDBlock` using the `Indexer` object. * Edit the custom hook function `createStateCheckpoint` (triggered just before default and CLI checkpoint) in [hooks.ts](./src/hooks.ts) to save the state in a `checkpoint` `IPLDBlock` using the `Indexer` object.
* The existing example hooks in [hooks.ts](./src/hooks.ts) are for an `ERC20` contract.
## Run ## Run
* Run the watcher: * Run the watcher:
@ -75,7 +73,7 @@
yarn server yarn server
``` ```
GQL console: http://localhost:3008/graphql GQL console: http://localhost:{{port}}/graphql
* If the watcher is an `active` watcher: * If the watcher is an `active` watcher:

View File

@ -6,3 +6,5 @@ export const MODE_ETH_CALL = 'eth_call';
export const MODE_STORAGE = 'storage'; export const MODE_STORAGE = 'storage';
export const MODE_ALL = 'all'; export const MODE_ALL = 'all';
export const MODE_NONE = 'none'; export const MODE_NONE = 'none';
export const DEFAULT_PORT = 3008;

View File

@ -1,13 +1,14 @@
import path from 'path'; import path from 'path';
import assert from 'assert'; import assert from 'assert';
import fs from 'fs'; import fs from 'fs';
import yaml from 'js-yaml';
import { loadFilesSync } from '@graphql-tools/load-files'; import { loadFilesSync } from '@graphql-tools/load-files';
export function parseSubgraphSchema (subgraphPath: string): any { export function parseSubgraphSchema (subgraphPath: string): any {
const subgraphSchemaPath = path.join(path.resolve(subgraphPath), '/schema.graphql'); const subgraphSchemaPath = path.join(path.resolve(subgraphPath), '/schema.graphql');
assert(fs.existsSync(subgraphSchemaPath)); assert(fs.existsSync(subgraphSchemaPath), `Schema file not found at ${subgraphSchemaPath}`);
const typesArray = loadFilesSync(subgraphSchemaPath); const typesArray = loadFilesSync(subgraphSchemaPath);
// Get a subgraph-schema DocumentNode with existing types. // Get a subgraph-schema DocumentNode with existing types.
@ -44,6 +45,19 @@ export function getFieldType (typeNode: any): { typeName: string, array: boolean
return { typeName: typeNode.name.value, array: false, nullable: true }; return { typeName: typeNode.name.value, array: false, nullable: true };
} }
export function getContractKindList (subgraphPath: string): string[] {
const subgraphConfigPath = path.join(path.resolve(subgraphPath), '/subgraph.yaml');
assert(fs.existsSync(subgraphConfigPath), `Subgraph config file not found at ${subgraphConfigPath}`);
const subgraph = yaml.load(fs.readFileSync(subgraphConfigPath, 'utf8')) as any;
const contractKinds: string[] = subgraph.dataSources.map((dataSource: any) => {
return dataSource.name;
});
return contractKinds;
}
function parseType (typeNode: any): any { function parseType (typeNode: any): any {
// Check if 'NamedType' is reached. // Check if 'NamedType' is reached.
if (typeNode.kind !== 'NamedType') { if (typeNode.kind !== 'NamedType') {

View File

@ -3,6 +3,7 @@
// //
import { Writable } from 'stream'; import { Writable } from 'stream';
import assert from 'assert';
import { Database } from './database'; import { Database } from './database';
import { Entity } from './entity'; import { Entity } from './entity';
@ -26,6 +27,8 @@ export class Visitor {
_reset: Reset; _reset: Reset;
_types: Types; _types: Types;
_contract?: { name: string, kind: string };
constructor () { constructor () {
this._schema = new Schema(); this._schema = new Schema();
this._resolvers = new Resolvers(); this._resolvers = new Resolvers();
@ -37,6 +40,13 @@ export class Visitor {
this._types = new Types(); this._types = new Types();
} }
setContract (name: string, kind: string): void {
this._contract = {
name,
kind
};
}
/** /**
* Visitor function for function definitions. * Visitor function for function definitions.
* @param node ASTNode for a function definition. * @param node ASTNode for a function definition.
@ -61,11 +71,13 @@ export class Visitor {
this._schema.addQuery(name, params, returnType); this._schema.addQuery(name, params, returnType);
this._resolvers.addQuery(name, params, returnType); this._resolvers.addQuery(name, params, returnType);
this._indexer.addQuery(MODE_ETH_CALL, name, params, returnType);
this._entity.addQuery(name, params, returnType); this._entity.addQuery(name, params, returnType);
this._database.addQuery(name, params, returnType); this._database.addQuery(name, params, returnType);
this._client.addQuery(name, params, returnType); this._client.addQuery(name, params, returnType);
this._reset.addQuery(name); this._reset.addQuery(name);
assert(this._contract);
this._indexer.addQuery(this._contract.name, MODE_ETH_CALL, name, params, returnType);
} }
} }
@ -103,11 +115,13 @@ export class Visitor {
this._schema.addQuery(name, params, returnType); this._schema.addQuery(name, params, returnType);
this._resolvers.addQuery(name, params, returnType); this._resolvers.addQuery(name, params, returnType);
this._indexer.addQuery(MODE_STORAGE, name, params, returnType, stateVariableType);
this._entity.addQuery(name, params, returnType); this._entity.addQuery(name, params, returnType);
this._database.addQuery(name, params, returnType); this._database.addQuery(name, params, returnType);
this._client.addQuery(name, params, returnType); this._client.addQuery(name, params, returnType);
this._reset.addQuery(name); this._reset.addQuery(name);
assert(this._contract);
this._indexer.addQuery(this._contract.name, MODE_STORAGE, name, params, returnType, stateVariableType);
} }
/** /**
@ -121,7 +135,9 @@ export class Visitor {
}); });
this._schema.addEventType(name, params); this._schema.addEventType(name, params);
this._indexer.addEvent(name, params);
assert(this._contract);
this._indexer.addEvent(name, params, this._contract.kind);
} }
visitSubgraph (subgraphPath?: string): void { visitSubgraph (subgraphPath?: string): void {
@ -160,10 +176,10 @@ export class Visitor {
/** /**
* Writes the indexer file generated from a template to a stream. * Writes the indexer file generated from a template to a stream.
* @param outStream A writable output stream to write the indexer file to. * @param outStream A writable output stream to write the indexer file to.
* @param inputFileName Input contract file names to be passed to the template. * @param contracts Input contracts to be passed to the template.
*/ */
exportIndexer (outStream: Writable, inputFileNames: string[]): void { exportIndexer (outStream: Writable, contracts: any[]): void {
this._indexer.exportIndexer(outStream, inputFileNames); this._indexer.exportIndexer(outStream, contracts);
} }
/** /**

View File

@ -0,0 +1,43 @@
# Config to generate eden-watcher using codegen.
# Contracts to watch (required).
contracts:
# Contract name.
- name: EdenNetwork
# Contract file path or an url.
path: ~/eden/governance/contracts/EdenNetwork.sol
# Contract kind (should match that in {subgraphPath}/subgraph.yaml if subgraphPath provided)
kind: EdenNetwork
# Contract name.
- name: MerkleDistributor
# Contract file path or an url.
path: ~/eden/governance/contracts/MerkleDistributor.sol
# Contract kind (should match that in {subgraphPath}/subgraph.yaml if subgraphPath provided)
kind: EdenNetworkDistribution
# Contract name.
- name: DistributorGovernance
# Contract file path or an url.
path: ~/eden/governance/contracts/DistributorGovernance.sol
# Contract kind (should match that in {subgraphPath}/subgraph.yaml if subgraphPath provided)
kind: EdenNetworkGovernance
# Output folder path (logs output using `stdout` if not provided).
outputFolder: ../demo-eden-watcher
# Code generation mode [eth_call | storage | all | none] (default: all).
mode: none
# Kind of watcher [lazy | active] (default: active).
kind: active
# Watcher server port (default: 3008).
port: 3012
# Flatten the input contract file(s) [true | false] (default: true).
flatten: true
# Path to the subgraph build (optional).
subgraphPath: ~/eden/eden-data/packages/subgraph/build
# NOTE: When passed an *URL* as contract path, it is assumed that it points to an already flattened contract file.