Commit Graph

263 Commits

Author SHA1 Message Date
Ian Norden
58b34548f3 vendor updates to work with go 1.11 and geth 1.8.18 2018-11-29 20:33:21 -06:00
Ian Norden
975f13b969 reorganizing omni directory and beginning light watcher work 2018-11-29 20:33:21 -06:00
Ian Norden
817bd76713 refactoring event converter and repository so that event logs are
persisted as they are converted, preventing build up in memory, and finishing  method polling (for full sync)
2018-11-29 20:33:21 -06:00
Ian Norden
390a60f7f6 port over lightSync updates from maker repo 2018-11-29 20:33:21 -06:00
Ian Norden
26aaa8319b Merge pull request #68 from vulcanize/omni
Generic Watcher -- Work Towards Epics 1/3
2018-11-15 12:51:43 -06:00
Ian Norden
417b18ec6a Edits to address PR issues; change license from apache to AGPL; and work
towards generic method polling and reposito;y; config settings to
filterevents/methods by account address; refactoring some stuff out of
repo and into converter; remove fetcher and instead call
blockchain's FetchContractData directly; finishing tests
2018-11-15 12:32:52 -06:00
Ian Norden
b459cf35ed beginning work on method polling; first need to generate list of token holder address in a completely generic/contratc agnostic fashion. Created address retriever that can iterate through any given contract's watched events, finding the inputs/arguments with address type, and generate a list from those values. Edit: Contract objects now cache every event emitted address as its event logs are transformed into the repo to grow a list of contract associated addresses as we go 2018-11-04 21:37:31 -06:00
Ian Norden
e9dbd771e5 tests and fixes for fetcher, parser, retriever, converter, and repository; update cmd and watcher 2018-11-04 01:49:11 -05:00
Ian Norden
5307de2b97 modify go-etheruem vendor lib with methods that allow for unpacking of events into maps of interfaces 2018-11-03 14:01:41 -05:00
Ian Norden
8ce75fe5ad Generic watcher that takes a contract address, grabs the contract abi and starting block number, creates custom event filters, and extracts and transforms event data into postgres. Can configure to look at only a subset of events through CLI flag. Building but needs testing. 2018-11-03 14:00:25 -05:00
Ian Norden
57820ff473 change transformer interface and watcher so that contract config is now fed into AddTransformers such that a single watcher can be loaded with transformers that use different cofigs 2018-11-03 13:49:24 -05:00
Ian Norden
44177e0772 reorganizing contract-specific modules into event_triggered contract directories 2018-11-03 13:49:23 -05:00
Ian Norden
d66e50dad6 Changes to address Rob's comments. Added generic/event_triggered transformer, tests, and repo migrations for Burn and Mint events 2018-11-03 13:49:23 -05:00
Ian Norden
4aa403d90d event_triggered transformer, transformer tests, and integrations tests for Transfer and Approval events 2018-11-03 13:49:23 -05:00
Ian Norden
aa2068bd08 adjust retriever to pull token holder addresses from Transfer and Approval events (iterating over Approvals might be redundant); edit Makefile to import new missing dependencies of go-ethereum/accounts/keystore, organizing mocks and adding event related mocks and filters 2018-11-03 13:49:23 -05:00
Ian Norden
ada872404e Transfer and Approval event repo and repo tests + db migrations for these events. 2018-11-03 13:49:23 -05:00
Ian Norden
55a73c5797 work on erc20 event_triggered functionality and generic helper for log unpacking and converting. helpers.ConvertToLog and helpers.createTopics have been adjusted to be variadic to work with event logs with any number of topics. Also uncovered issue with Dai and TrueUSD that means they doesn't really conform to the ERC20 standard. this is because they named their arguments to standard events like Approval and Trasnfer differently (e.g. Approval(src address, guy address, wad uint) and Approval(owner address, spender address, value uint) instead of the standard which is Approval(tokenOwner address, spender address, tokens uint)). This causes incompatibility with generic ERC20 entities and converters for these events. 2018-11-03 13:49:23 -05:00
Ian Norden
44e0a8d303 retriever for generating list of all token holder addresses + updated transformer to use said addresses to populate balanceOf and allowance information and added database migrations for balance and allowance tables 2018-11-03 13:49:23 -05:00
Ian Norden
1d50a0ace0 fixed tests for getting balanceOf and allowance, added balanceOf and allowance to the repo and repo tests and updated sql schema with new token_balance and token_allowance tables to accept this data 2018-11-03 13:49:23 -05:00
Ian Norden
9b41000b88 Moved fetcher to generic directory (methods have to remain public since it is in seperate package now), added FetchHash method, created ERC20 and generic getters which call the fetcher with specific contract methods (GetTotalSupply, GetBalance, GetAllowance for ERC20 getter, and GetOwner, GetStoppedStatus, GetStringName, GetHashName, GetStringSymbol, GetHashSymbol, and GetDecimals for generic getter). Getter tests cover all but GetBalance and GetAllowance, and also cover all of the Fetcher methods- but with only nil methodArgs. GetAllowance and GetBalance tests are not working against infura and these are the only contract method calls with arguments passed in so I suspect this might be where the issue lies. Have tested GetBalance using previous version of FetchContractData without the variadic input to the Pack method and it fails with the same error so I don’t think it is due to those changes. 2018-11-03 13:49:23 -05:00
Ian Norden
99e549b3df changes to repo and repo tests so that supply table is organized and accessed by token_address 2018-11-03 13:49:23 -05:00
Ian Norden
bea712ba10 fetcher tests for FetchBool and FetchAddress 2018-11-03 13:49:23 -05:00
Ian Norden
ccd21f73c9 adjusted FetchContractData to work with variable number of contract method arguments, changed fetcher's FetchSupply method to more general FetchBigInt method and added FetchBool and FetchAddress methods 2018-11-03 13:49:23 -05:00
James Christie
93960e7c4e Add several improvements to README.md (#28)
- Expand instructions around technical specifics of project
  setup, including special gotchas in Ubuntu-based
  environments
- Add statement of setup flow to give thousand-foot view to
  users setting up Vulcanize for the first time
2018-11-03 13:49:23 -05:00
Rob Mulholand
ba071ef13f Consolidate test doubles
- Migrate various mocks of core namespaces to shared version in `fakes` pkg
- Err on the side of making test doubles less sophisticated
- Don't pull over mocks of namespaces that are only used in example code
2018-11-03 13:49:23 -05:00
Rob Mulholand
5fe6394406 Add tests for pkg/geth/blockchain
- inject dependencies instead of initializing them in the constructor
2018-11-03 13:49:23 -05:00
Rob Mulholand
05186634bd Add light sync command
- Only syncs block headers (excludes block bodies, transactions, receipts, and logs)
- Modifies validation window to include the most recent block
- Isolates validation window to the variable defined in the cmd directory (blocks
  have a separate variable defined in the block_repository for determining when
  to set a block as final)
2018-11-03 13:49:23 -05:00
Elizabeth Engelman
b6f93e735f Add ERC20 token watcher example
- starting with the totalSupply function
- sets contract config on transformer by passing it into the transformer
initializer
- handles block records with the same number for different nodes for both creating token_supply records, and finding missing blocks
2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
b18d1de4fe dockerfiles/rinkeby: added named volume for pg container 2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
7b507667f8 Added user and password to db config 2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
2201efc347 Rename dev_env to rinkeby_env, added make commands description to readme 2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
0d4aa71d6d Dockerized dev environment, geth rinkeby 2018-11-03 13:49:23 -05:00
Rob Mulholand
02fc10dd3d Merge pull request #64 from chapsuk/fix_flag_description
Fixed sync command help typo
2018-06-21 12:33:05 -05:00
Maxim Krasilnikov
abb7f26124 Fixed sync command help typo 2018-06-21 19:12:09 +03:00
Rob Mulholand
b2cf8d55d8 Merge pull request #63 from chapsuk/duplicate_gouroutines
Removed useless goroutines from sync cmd
2018-06-21 10:12:49 -05:00
Elizabeth
aff2ad2b09 Merge pull request #62 from vulcanize/update-build-badge
Update build badge url
2018-06-21 10:11:06 -05:00
Maxim Krasilnikov
51f2a23b74 Removed useless goroutines from sync cmd, backFillAllBlocks always called with go func() 2018-06-21 18:05:49 +03:00
Elizabeth Engelman
8437b72ae1 Update build badge url 2018-06-21 10:02:42 -05:00
Rob Mulholand
94182b55fa Merge pull request #61 from wanderingstan/patch-1
Add "About" section to README
2018-06-21 09:49:49 -05:00
Stan James
febf412642 Add "About" section to README
Per issue https://github.com/vulcanize/vulcanizedb/issues/60 , adding description provided by @AFDudley on gitter here:
https://gitter.im/vulcanizeio/VulcanizeDB?at=5b2a54b0148056028591b323
2018-06-20 15:55:33 -06:00
Rob Mulholand
b1df139d29 Merge pull request #57 from vulcanize/fix-missing-blocks-lookup
Fix missing block numbers lookup
2018-06-13 11:28:22 -05:00
Rob Mulholand
3e39ccb9bb Fix missing block numbers lookup
- Previously, a block was treated as missing if there was a number in
the given range for which there existing a block that did not match
the node's fingerprint. This meant that in a case where we have a block
that does not match the node's fingerprint and also one that does match,
the block would be treated as missing. This led to errors being thrown
when attempting to add a block that already exists.
- These changes treat a block as missing only if we do not already have
a block that matches the number and node fingerprint.
2018-06-08 11:26:25 -05:00
Rob Mulholand
7533e6d476 Merge pull request #52 from vulcanize/cold-import
Cold import
2018-05-21 09:34:05 -05:00
Rob Mulholand
d5c2ab33fc Sync only missing blocks on cold import
= Add eth_node_fingerprint to block that can be imitated by both hot and cold imports
- Only sync missing blocks (blocks that are missing or don't share a fingerprint) on cold import
- Set block is_final status after import
2018-05-07 16:38:04 -05:00
A. F. Dudley
4baaed94ec Merge pull request #50 from gitter-badger/gitter-badge-1
Add a Gitter chat badge to README.md
2018-05-05 14:39:36 -04:00
Rob Mulholand
5a5e08bd13 Get head block number
- Allows us to refuse to sync past current head
- Creates the opportunity to add a flag for syncing all blocks
2018-05-04 11:03:50 -05:00
Rob Mulholand
5f6bf32ec1 Add cold import script 2018-05-04 10:54:00 -05:00
Rob Mulholand
462f94d84a Associate receipts with blocks instead of transactions
- Allows us to fetch receipts by block and persist even when
associated transaction is unknown. (The associated transaction
can still be derived from the tx_hash column at query time, but
is an expensive operation to require for inserts).
2018-05-04 10:52:28 -05:00
Matt K
4711708397 Update block reading test to resolve periodic fails (#53) 2018-05-04 10:41:56 -05:00
The Gitter Badger
021e252bb6 Add Gitter badge 2018-04-19 14:49:56 +00:00