Commit Graph

390 Commits

Author SHA1 Message Date
Ashwin Phatak
f93c0e3cb6
Track last processed event index (#168)
* Implement last processed event in uni-info-watcher.

* Check if block or event is already processed.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-28 17:36:43 +05:30
Ashwin Phatak
c677e5942c
ERC20 watcher based on eth_call (#165)
* Implement eth_call for ERC20 totalSupply.

* Use eth_call for erc20-watcher.

* Implement fallback for ERC20 symbol method call.

* Implement fallback for ERC20 name and totalSupply.

* implement fallback for erc20 decimals method.

* Lint fixes.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-28 10:04:07 +05:30
Ashwin Phatak
a5d6bfa285
Get/create entity based on block hash and parent traversal (#163)
* Watch all blocks and fetch events for each block.

* Implement getPrevVersionEntity in database.

* Implement getPrevVersionEntity for PoolCreated event.

* Implement getPrevEntityVersion for Pool events.

* Implement getPrevEntityVersion for NFPM events.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-23 19:22:55 +05:30
Ashwin Phatak
f8335aad03
Add block hash to entities, split get and set methods (#161)
* Split get and set entity for PoolCreated and Initialize events.

* Split get/set entity for Mint, Burn and Swap events.

* Split get/set entity for IncreaseLiquidity and DecreaseLiquidity events.

* Split get/set for NFPM Transfer and Collect events.

* Add blockHash to entities.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-23 11:00:40 +05:30
Ashwin Phatak
26965f372f
Handle reorgs upto known canonical block hash (#157)
* Handle reorgs upto known canonical block hash.

* Process block events strictly in order.

* Support filling old blocks, process in-order.

* Always publish block progress.
2021-07-22 16:32:39 +05:30
Ashwin Phatak
34de3d51b3
Handle collect event (#158)
* Handle NFPM Collect event.

* Handle NFPM Transfer event.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-22 11:45:21 +05:30
Ashwin Phatak
0d39363467
Move uni-info-watcher event procesing to job queue. (#156)
* Move event procesing to job queue.

* Store block properties in blocks table.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-22 10:02:06 +05:30
Ashwin Phatak
95486d6553
Save block parent hash (#155)
* Save block parent hash.

* Store block properties in blocks table.
2021-07-21 13:00:26 +05:30
Ashwin Phatak
10e7d37fa8
Update pgboss queue params. (#154) 2021-07-20 15:46:35 +05:30
Ashwin Phatak
c1ef96ba2e
Handle Position Manager DecreaseLiquidity event. (#148)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-19 10:43:29 +05:30
Ashwin Phatak
3477366458
Handle increase liquidity event (#143)
* Implement handler for NFPM IncreaseLiquidity event.

* Get contract values by querying uni-watcher.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-16 18:34:51 +05:30
Ashwin Phatak
ae13edb99a
Update docs. (#142) 2021-07-16 16:35:49 +05:30
Ashwin Phatak
29c8c1f80f
Implement event watchers for NonFungiblePositionManager events. (#141)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-15 17:27:10 +05:30
Ashwin Phatak
70e88b1004
Block filler for uni-watcher. (#140) 2021-07-15 13:10:07 +05:30
Ashwin Phatak
7151521c3b
Get uniswap events in block range (#139)
* Get events in block range.

* Get uniswap events in block range.
2021-07-15 10:48:06 +05:30
Ashwin Phatak
ecfa3ed386
Handle swap event (#138)
* Create Swap entity.

* Update day and hour interval data in Swap event handler.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-15 10:45:35 +05:30
Ashwin Phatak
03ceb95a1b
Job queue to process events (#137)
* Job queue to process events.

* Event queue processing changes.
2021-07-14 18:00:26 +05:30
Ashwin Phatak
a4f5d43bc5
Extra info for handling events (tx, block) (#136)
* Extra info in events for downstream processing.

* Changes in uni-info-watcher after change in uni-watcher event schema.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-13 17:32:57 +05:30
Ashwin Phatak
208b0f7f4f
Handle Pool burn event. (#134)
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-13 12:36:10 +05:30
Ashwin Phatak
3a6af9f9cc
Handle mint event (#130)
* Handle mint event and update Factory fields.

* Integrate decimal.js with typeorm.

* Update transaction in Mint event.

* Update day and hour data.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-13 12:01:54 +05:30
Ashwin Phatak
d71557e963
Event processing/sync changes to maintain processing order. (#131) 2021-07-12 17:06:33 +05:30
Ashwin Phatak
69c68b365f
Include txHash in event sent to GQL subscribers. (#129) 2021-07-09 17:01:36 +05:30
Ashwin Phatak
61f211f2d5
Handle pool initialize event (#127)
* Move getConfig to util package.

* Handle Pool initialize event.

* Update Bundle entity ethPriceUSD.

* Update Pool day and hour data.

* Update token derivedETH and complete handleInitialize.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-09 12:38:25 +05:30
Ashwin Phatak
7f5229bf2f
Watch pool burn and swap events (#128)
* Remove test scripts, moved to vulcanize/uniswap-v3-core#watcher-ts.

* Move uniswap analysis docs.

* Watch pool burn and swap events.
2021-07-09 12:30:50 +05:30
Ashwin Phatak
538e4b7591
Watch pool mint event. (#125) 2021-07-08 17:40:40 +05:30
Ashwin Phatak
f7df56bb75
Handle pool created event (#124)
* Update Token entities.

* Complete handlePoolCreated in events.

Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-07 11:10:10 +05:30
Ashwin Phatak
194d079d5e
Watch pool initialize event (#123)
* Script to init pool.

* Watch pool initialize event.
2021-07-06 20:05:40 +05:30
Ashwin Phatak
aec9281fb8
Factory PoolCreated event handler (#120)
* Setup handler for PoolCreated event.

* Create Pool entity.

* Subscribe to uni-watcher for watching events.

* Refactor code to create GraphQLClient in ipld-eth-client.

Co-authored-by: nikugogoi <95nikass@gmail.com>
Co-authored-by: nabarun <nabarun@deepstacksoft.com>
2021-07-06 16:55:11 +05:30
Ashwin Phatak
9c60895352
Save watched contracts in db. (#122) 2021-07-06 15:58:03 +05:30
Ashwin Phatak
0b6a33561b
Include block number in GQL event. (#121) 2021-07-06 11:04:08 +05:30
Ashwin Phatak
8d2a4c6b14
Watch for Factory PoolCreated event. (#119) 2021-07-05 16:15:54 +05:30
Ashwin Phatak
da758aceaa
Script to deploy uniswap contracts locally for testing (#118)
* Initial setup with hardhat.

* Deploy Factory contract.

* Deploy tokens and create pool using factory contract.

* Deploy contract to private network.

* Implement separate scripts for deploying Factory, Token and Pool.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-07-05 11:05:45 +05:30
Ashwin Phatak
b7ffb7c672
uniswap-info-watcher package with frontend schema (#114)
* Implement mock bundle query in watcher.

* Use chance js for generating mock data.

* Add bundles query.

* Setup resolvers with uniswap info schema.

* Add mock query for Burn entity.

* Add mock query for Factory entity.

* Add mock queries for entities Mint, Pool and Transaction.

* Add mock query for Swap entity.

* Add mock queries for day data entities.

* Implment mock ticks query.

* Add mock tokenHourDatas query.

* Add mock queries for uni-info-watcher to readme.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-07-02 16:26:32 +05:30
Ashwin Phatak
1b6d4511f7
Minimal schema required by info-watcher. (#115) 2021-07-02 12:47:15 +05:30
Ashwin Phatak
5de555a4e5
Uniswap watcher package with event schema (#113)
* Uniswap watcher package skeleton.

* Uniswap events.
2021-07-01 16:42:37 +05:30
Ashwin Phatak
d5608f637a
Aggregation in entities (#109)
* Analyse entity aggregations in uniswap subgraph.

* Analyse entity aggregation for Day data and Hour data entities.

* Design notes.

* Reorg, split analysis files.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-30 17:16:50 +05:30
Ashwin Phatak
69f4cf3513
Uniswap queries/mapping analysis (#91)
* List down view method in uniswap core contracts.

* List queries used in uniswap subgraph frontend.

* Analyse uniswap subgraph entities with mappings.

* Generate schema graphql file for uniswap subgraph.

* Schema graphql for queries used in subgraph frontend.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-29 10:39:32 +05:30
Ashwin Phatak
a654b79df3
Address watcher block filler (#105)
* Address watcher block filler.

* Update docs.
2021-06-28 17:12:53 +05:30
Ashwin Phatak
e9880693ad
Fill task queue for block range. (#90) 2021-06-25 17:56:58 +05:30
Ashwin Phatak
07805b6ae9
Tracing job queue (#89)
* eth-client API to get transaction in block.

* Job queue to process tracing requests.

* Request traces from job queue runner.

* Max trace completion lag time config for downstream events.
2021-06-25 16:35:47 +05:30
Ashwin Phatak
27b732fef7
Fix getting negative values from signed integers. (#88)
Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-23 15:23:59 +05:30
Ashwin Phatak
fc30290685
Assert proof data when running against GQL endpoint (#86)
* Assert proof data for intial test cases of value type variables.

* Assert proof data for value type variables integers and bytes.

* Assert proof for dynamic byte arrays and strings.

* Assert proof for fixed array types.

* Assert proof for dynamic array types.

* Assert proofs for nested arrays.

* Assert proof for struct type variables.

* Assert proof for mapping type variables.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-23 09:33:42 +05:30
Ashwin Phatak
2adc5e9c34
Push address events to downstream subscribers (#85)
* Push address event to downstream subscribers.

* Get addresses from trace - tests and fixes.
2021-06-22 14:04:48 +05:30
Ashwin Phatak
816352c9ff
Refactor initial test cases for transaction wait. (#84)
Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-22 10:10:16 +05:30
Ashwin Phatak
7bbc920fda
Group transactions to decrease overall test suite run time (#81)
* Refactor transaction wait for value types and arrays.

* Refactor transaction wait for reference type variables.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-21 19:13:58 +05:30
Ashwin Phatak
38a189d74a
Watch tx to trigger indexing. (#80) 2021-06-21 18:55:13 +05:30
Ashwin Phatak
eea69fe4d4
Indexing to get traces given address (#79)
* Build reverse index from address to traces.

* Create reverse index from address to traces.
2021-06-21 15:38:36 +05:30
Ashwin Phatak
c9bf002675
Test decoder using hardhat RPC, geth-statediff RPC and ipld-eth-server GQL endpoints for fetching storage slots (#77)
* Implement test for getStorageValue with geth server.

* Wait for transaction to complete in tests.

* Implement tests with ipld-eth-client using graphql endpoint.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-18 18:09:50 +05:30
Ashwin Phatak
6a33c704b8
Basic GQL API to get traces, save to db. (#76) 2021-06-18 17:04:02 +05:30
Ashwin Phatak
8507bc8b9c
Address watcher package scaffolding (#75)
* Rename erc20 watcher folder.

* Update repo URLs.

* Address watcher package scaffolding.
2021-06-17 17:56:38 +05:30
Ashwin Phatak
56fb7c69c0
Address tracer perf improvements, more mainnet traces, capture opcodes (#74)
* Maintain cache of previous address resolution.

* Indent using tabs in call_address_tracker to keep diffs sane.

* Undo aggressive falsifying of potential addresses.

* Capture opcode from prev step, more mainnet tx traces.
2021-06-17 13:20:04 +05:30
Ashwin Phatak
e3ef17d547
Fix getting values for arrays. (#73)
Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-17 11:59:53 +05:30
nikugogoi
7d609f9a2b Tests for nested dynamic array. 2021-06-16 18:30:41 +05:30
nikugogoi
7c4fef153e Tests for dynamically sized byte arrays. 2021-06-16 18:19:30 +05:30
Ashwin Phatak
b505332970
Tests for mapping with reference type values. (#70)
Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-16 18:16:55 +05:30
Ashwin Phatak
4e08f359fc
Call tracer with address tracing support (#65)
* Tracing client package, basic utils.

* Start of address tracer.

* Rename trace CLI.

* Call tracer API.

* Tracing timeout, more traces.

* Integrate address tracer into call address tracer.
2021-06-16 16:14:35 +05:30
Ashwin Phatak
909242a827
Tests for fixed size array of reference type values (#68)
* Tests for fixed size array of enum type.

* Tests for fixed size array of reference type values.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-16 12:11:44 +05:30
Ashwin Phatak
876dfa0616
Add tests for dynamic array of reference type values. (#67)
Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-16 10:08:39 +05:30
Ashwin Phatak
d4db1f5d28
Get value for dynamic arrays (#66)
* Implement getting values of dynamic arrays.

* Add tests for dynamic arrays of value types.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-15 18:20:12 +05:30
Ashwin Phatak
3439dd4041
Get value for nested array, mapping with address keys and struct value (#64)
* Add test for mapping with address keys and struct value.

* Implement getting value for nested array.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-15 15:46:39 +05:30
Ashwin Phatak
3815853f7b
Add tests for nested struct. (#59)
Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-14 19:57:29 +05:30
Ashwin Phatak
eb20708faf
Tests for maps with struct type value and double nested maps (#58)
* Tests for fixed array of struct type.

* Add tests for maps with struct type value and double nested maps.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-14 16:23:41 +05:30
Ashwin Phatak
f5c1a22fdc
Tests for getting array and string type member value in struct (#57)
* Add tests for structs with fixed array and string type
 members.

* Refactor functions in solidity-mapper storage.

* Add tests for getting array and string type member value in struct.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-14 16:12:42 +05:30
nikugogoi
2fcfadecc2 Implement getting value of struct by member name. 2021-06-11 16:45:12 +05:30
nikugogoi
b33ce0b640 Get array element by specifying index. 2021-06-11 16:44:27 +05:30
Ashwin Phatak
8851882144
Get value of struct type (#54)
* Implement getting value for struct types.

* Add tests for structs with value type memebers.

* Add tests for verifying proof in struct type.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-11 11:19:20 +05:30
Ashwin Phatak
5316b19fbf
Get fixed size array of type address (#53)
* Add test for fixed array of address type.

* Add tests for verifying proof data.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-11 11:16:02 +05:30
Ashwin Phatak
df025433ec
Tests for getStorageValue on mapping and nested mappings with different key types (#52)
* Tests for mapping with value type keys.

* Add test for mapping with string type keys.

* Add test for mapping with dynamically-sized byte array as keys.

* Add tests for nested mapping.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-10 11:22:03 +05:30
Ashwin Phatak
c916e61a9b GQL mutation to start watching contract. 2021-06-09 15:54:10 +05:30
Ashwin Phatak
7035a513b1
Downstream event subscription support. (#47)
* Downstream event subscription support.

* Logging changes.
2021-06-09 15:11:05 +05:30
Ashwin Phatak
fc44617db3
Use solidity mapper to get value for mapping and nested mapping (balance and allowance) (#48)
* Implement getting value for basic mapping type.

* Add test for basic mapping type.

* Implement getting value for nested mapping type.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-09 10:18:19 +05:30
Ashwin Phatak
84e1927402
Watch upstream ERC20 events to trigger indexing (#43)
* Move to apollo client, enables subscriptions.

* Watch logs and trigger other indexer methods.

* Refactoring config loading, watched contracts table.

* Check event sync progress inside transaction.

* Refactoring server startup.
2021-06-08 16:07:52 +05:30
nikugogoi
a13a909a85 Implement getting storage value for uint256 type in watcher. 2021-06-08 12:49:00 +05:30
nikugogoi
8d697ab9d1 Check and remove unused dependecies. 2021-06-07 11:42:03 +05:30
Ashwin Phatak
945cba9392
Implement typescript strict mode and ESLint in watcher package (#40)
* Set up typescript build and eslint.

* Automatic lint fixes.

* Fix argument type warnings.

* Fix function return types.

* Fix typescript errors in watcher.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-04 18:08:26 +05:30
Ashwin Phatak
3aaa9dd7f8
Use postgres instead of sqlite. (#38) 2021-06-04 15:19:30 +05:30
Ashwin Phatak
4e0ef7c852
Implement typescript strict mode and ESLint in ipld-eth-client and cache packages (#37)
* Set up typescript build.

* Setup eslint in cache package.

* Automatic lint fixes.

* Fix typescript return types.

* Fix typescript argument type warnings.

* Set up typescript build and eslint.

* Automatic lint fixes.

* Fix typescript explicit any warnings.

* Add argument types.

* Fix return type warnings.

* Fix typescript errors.

* Implement declaration in types directory.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-04 12:04:12 +05:30
Ashwin Phatak
47ce52c112
Sqlite based index for ERC20 data (#31)
* Save/load balance and allowance data to/from RDBMS.

* Event entities.

* Pass artifacts to indexer.

* Refactoring, comments.

* Sync events to database.

* Absorb typeorm config file settings into our own config file.
2021-06-03 17:31:21 +05:30
Ashwin Phatak
23f9a9db41
Get value for array of integer and boolean types in solidity mapper (#30)
* Implement getting value for array type.

* Add subtypes in solidit mapper readme.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-03 15:03:39 +05:30
Ashwin Phatak
00eb129536
Use getStorageValue to parse value for string type variable (#29)
* Get string value of variable name.

* Use getStorageValue to parse value for string type variable.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-03 11:52:23 +05:30
Ashwin Phatak
b243025ca8
Test cases in solidity-mapper for contract, fixed-size byte arrays and enum types (#26)
* Add tests for getStorageInfo and getEventNameTopics.

* Lint solidity-mapper package code.

* Add test for contract type.

* Add test for fixed size byte arrays.

* Add test for Enum types.

* Add tests for variables packed together and using single slot.

* Fix comments in test contracts.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-06-02 11:23:33 +05:30
Ashwin Phatak
f6870d88dc
Support GQL query for name, symbol, totalSupply (#24)
* Update mock server and tests for additional ERC20 state vars.

* Refactor eth-client getStorageAt, impl totalSupply.

* Impl ERC20 API for name, symbol.
2021-06-01 18:13:41 +05:30
nikugogoi
7d7cf26f81 Add method to get event name topics from abi. 2021-05-31 16:24:45 +05:30
Ashwin Phatak
13ec526ebd Create index for ERC20 data. 2021-05-31 16:24:45 +05:30
Ashwin Phatak
a0aae09f83
Get slot for ERC20 variable from storage layout (#13)
* Get slot for ERC20 variable from storage layout.

* Fix solidity-mapper build for importing library functions.

* Implement lint command in solidity-mapper package.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-05-31 14:50:05 +05:30
Ashwin Phatak
72ca980198
Solidity data mapper/parser (#12)
* Initial setup with hardhat and typescript.

* Add test for integer type.

* Add test for unsigned integer type.

* Add test for boolean type.

* Add test for address type.

* Add test for string type.

* Setup building library with typescript.

* Remove hardhat dependency from getStorageValue library function.

* Move contracts to test and remove deploy script.

* Add readme for running tests.

Co-authored-by: nikugogoi <95nikass@gmail.com>
2021-05-31 11:07:11 +05:30
Ashwin Phatak
7213a1dc6d
Lazy ERC20 watcher (#11)
* Storage mapping utils.

* Resolver factories.

* Get ERC20 balance from upstream ipld-eth-server slot.

* Get ERC20 allowance from storage slot.

* Parse ERC20 events from block events.

* Cache GQL requests in leveldb.

* Refactor fetch/cache from upstream into eth-loader class.

* Refactoring.

* Refactor eth-client, extract cache class.

* Cache config settings.

* Debug logs.

* Rename server to watcher.

* Start local server by default.

* Split into multiple packages.

* eth-client API methods.

* Update docs.
2021-05-28 16:56:40 +05:30
Ashwin Phatak
95eb1ac0d1 Data driven tests for GQL endpoints. 2021-05-13 19:20:43 +05:30
Ashwin Phatak
11fbe0534d Refactor mock server. 2021-05-13 16:06:38 +05:30
Ashwin Phatak
c84b481385 Move server to yarn workspace package. 2021-05-13 16:06:38 +05:30