Ian Norden
e390a97502
finish method polling with hash or address type argument values
...
collected from watched events; 'MissingMethodsCheckedEventsIntersection'
method to find headers which have been checked for each event of
interest but methods have not yet been polled at that header; tests for
new features; travis ci go version 1.9 -> 1.11 ; consolidate omniWatcher and lightOmniWatcher into single command with light as default
2018-12-21 10:33:31 -06:00
Ian Norden
0a59f06cac
begin work on: Add checked_headers column for methods that are polled so taht we don’t duplicate; Add batching of method polling so that we arent generating a rediculously large account address list before using it to poll methods (or persist the list in pg?); User passed ABI and other ways to get ABI; Add ability to collect []byte and hashes from events and use them in method polling same manner as addresses; Event filter addrs => only those event’s addresses/hashes are used for polling; Option to persist seen address/hash/bytes lists into pg; Only generate lists of addresses, []byte, or hashes if a method will use them later
2018-12-21 10:33:31 -06:00
Ian Norden
8c5b1b4dbe
remove accidental duplicate vendor lib that was imported by goland
2018-12-21 10:33:21 -06:00
Ian Norden
e02b33547d
finishing porting omni watcher to work with light sync; split into full, light,
...
and shared directories and refactor as much into shared; finish
lightSync omni watcher tests
2018-11-30 13:33:48 -06:00
Ian Norden
58b34548f3
vendor updates to work with go 1.11 and geth 1.8.18
2018-11-29 20:33:21 -06:00
Ian Norden
975f13b969
reorganizing omni directory and beginning light watcher work
2018-11-29 20:33:21 -06:00
Ian Norden
817bd76713
refactoring event converter and repository so that event logs are
...
persisted as they are converted, preventing build up in memory, and finishing method polling (for full sync)
2018-11-29 20:33:21 -06:00
Ian Norden
390a60f7f6
port over lightSync updates from maker repo
2018-11-29 20:33:21 -06:00
Ian Norden
26aaa8319b
Merge pull request #68 from vulcanize/omni
...
Generic Watcher -- Work Towards Epics 1/3
2018-11-15 12:51:43 -06:00
Ian Norden
417b18ec6a
Edits to address PR issues; change license from apache to AGPL; and work
...
towards generic method polling and reposito;y; config settings to
filterevents/methods by account address; refactoring some stuff out of
repo and into converter; remove fetcher and instead call
blockchain's FetchContractData directly; finishing tests
2018-11-15 12:32:52 -06:00
Ian Norden
b459cf35ed
beginning work on method polling; first need to generate list of token holder address in a completely generic/contratc agnostic fashion. Created address retriever that can iterate through any given contract's watched events, finding the inputs/arguments with address type, and generate a list from those values. Edit: Contract objects now cache every event emitted address as its event logs are transformed into the repo to grow a list of contract associated addresses as we go
2018-11-04 21:37:31 -06:00
Ian Norden
e9dbd771e5
tests and fixes for fetcher, parser, retriever, converter, and repository; update cmd and watcher
2018-11-04 01:49:11 -05:00
Ian Norden
5307de2b97
modify go-etheruem vendor lib with methods that allow for unpacking of events into maps of interfaces
2018-11-03 14:01:41 -05:00
Ian Norden
8ce75fe5ad
Generic watcher that takes a contract address, grabs the contract abi and starting block number, creates custom event filters, and extracts and transforms event data into postgres. Can configure to look at only a subset of events through CLI flag. Building but needs testing.
2018-11-03 14:00:25 -05:00
Ian Norden
57820ff473
change transformer interface and watcher so that contract config is now fed into AddTransformers such that a single watcher can be loaded with transformers that use different cofigs
2018-11-03 13:49:24 -05:00
Ian Norden
44177e0772
reorganizing contract-specific modules into event_triggered contract directories
2018-11-03 13:49:23 -05:00
Ian Norden
d66e50dad6
Changes to address Rob's comments. Added generic/event_triggered transformer, tests, and repo migrations for Burn and Mint events
2018-11-03 13:49:23 -05:00
Ian Norden
4aa403d90d
event_triggered transformer, transformer tests, and integrations tests for Transfer and Approval events
2018-11-03 13:49:23 -05:00
Ian Norden
aa2068bd08
adjust retriever to pull token holder addresses from Transfer and Approval events (iterating over Approvals might be redundant); edit Makefile to import new missing dependencies of go-ethereum/accounts/keystore, organizing mocks and adding event related mocks and filters
2018-11-03 13:49:23 -05:00
Ian Norden
ada872404e
Transfer and Approval event repo and repo tests + db migrations for these events.
2018-11-03 13:49:23 -05:00
Ian Norden
55a73c5797
work on erc20 event_triggered functionality and generic helper for log unpacking and converting. helpers.ConvertToLog and helpers.createTopics have been adjusted to be variadic to work with event logs with any number of topics. Also uncovered issue with Dai and TrueUSD that means they doesn't really conform to the ERC20 standard. this is because they named their arguments to standard events like Approval and Trasnfer differently (e.g. Approval(src address, guy address, wad uint) and Approval(owner address, spender address, value uint) instead of the standard which is Approval(tokenOwner address, spender address, tokens uint)). This causes incompatibility with generic ERC20 entities and converters for these events.
2018-11-03 13:49:23 -05:00
Ian Norden
44e0a8d303
retriever for generating list of all token holder addresses + updated transformer to use said addresses to populate balanceOf and allowance information and added database migrations for balance and allowance tables
2018-11-03 13:49:23 -05:00
Ian Norden
1d50a0ace0
fixed tests for getting balanceOf and allowance, added balanceOf and allowance to the repo and repo tests and updated sql schema with new token_balance and token_allowance tables to accept this data
2018-11-03 13:49:23 -05:00
Ian Norden
9b41000b88
Moved fetcher to generic directory (methods have to remain public since it is in seperate package now), added FetchHash method, created ERC20 and generic getters which call the fetcher with specific contract methods (GetTotalSupply, GetBalance, GetAllowance for ERC20 getter, and GetOwner, GetStoppedStatus, GetStringName, GetHashName, GetStringSymbol, GetHashSymbol, and GetDecimals for generic getter). Getter tests cover all but GetBalance and GetAllowance, and also cover all of the Fetcher methods- but with only nil methodArgs. GetAllowance and GetBalance tests are not working against infura and these are the only contract method calls with arguments passed in so I suspect this might be where the issue lies. Have tested GetBalance using previous version of FetchContractData without the variadic input to the Pack method and it fails with the same error so I don’t think it is due to those changes.
2018-11-03 13:49:23 -05:00
Ian Norden
99e549b3df
changes to repo and repo tests so that supply table is organized and accessed by token_address
2018-11-03 13:49:23 -05:00
Ian Norden
bea712ba10
fetcher tests for FetchBool and FetchAddress
2018-11-03 13:49:23 -05:00
Ian Norden
ccd21f73c9
adjusted FetchContractData to work with variable number of contract method arguments, changed fetcher's FetchSupply method to more general FetchBigInt method and added FetchBool and FetchAddress methods
2018-11-03 13:49:23 -05:00
James Christie
93960e7c4e
Add several improvements to README.md ( #28 )
...
- Expand instructions around technical specifics of project
setup, including special gotchas in Ubuntu-based
environments
- Add statement of setup flow to give thousand-foot view to
users setting up Vulcanize for the first time
2018-11-03 13:49:23 -05:00
Rob Mulholand
ba071ef13f
Consolidate test doubles
...
- Migrate various mocks of core namespaces to shared version in `fakes` pkg
- Err on the side of making test doubles less sophisticated
- Don't pull over mocks of namespaces that are only used in example code
2018-11-03 13:49:23 -05:00
Rob Mulholand
5fe6394406
Add tests for pkg/geth/blockchain
...
- inject dependencies instead of initializing them in the constructor
2018-11-03 13:49:23 -05:00
Rob Mulholand
05186634bd
Add light sync command
...
- Only syncs block headers (excludes block bodies, transactions, receipts, and logs)
- Modifies validation window to include the most recent block
- Isolates validation window to the variable defined in the cmd directory (blocks
have a separate variable defined in the block_repository for determining when
to set a block as final)
2018-11-03 13:49:23 -05:00
Elizabeth Engelman
b6f93e735f
Add ERC20 token watcher example
...
- starting with the totalSupply function
- sets contract config on transformer by passing it into the transformer
initializer
- handles block records with the same number for different nodes for both creating token_supply records, and finding missing blocks
2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
b18d1de4fe
dockerfiles/rinkeby: added named volume for pg container
2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
7b507667f8
Added user and password to db config
2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
2201efc347
Rename dev_env to rinkeby_env, added make commands description to readme
2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
0d4aa71d6d
Dockerized dev environment, geth rinkeby
2018-11-03 13:49:23 -05:00
Rob Mulholand
02fc10dd3d
Merge pull request #64 from chapsuk/fix_flag_description
...
Fixed sync command help typo
2018-06-21 12:33:05 -05:00
Maxim Krasilnikov
abb7f26124
Fixed sync command help typo
2018-06-21 19:12:09 +03:00
Rob Mulholand
b2cf8d55d8
Merge pull request #63 from chapsuk/duplicate_gouroutines
...
Removed useless goroutines from sync cmd
2018-06-21 10:12:49 -05:00
Elizabeth
aff2ad2b09
Merge pull request #62 from vulcanize/update-build-badge
...
Update build badge url
2018-06-21 10:11:06 -05:00
Maxim Krasilnikov
51f2a23b74
Removed useless goroutines from sync cmd, backFillAllBlocks always called with go func()
2018-06-21 18:05:49 +03:00
Elizabeth Engelman
8437b72ae1
Update build badge url
2018-06-21 10:02:42 -05:00
Rob Mulholand
94182b55fa
Merge pull request #61 from wanderingstan/patch-1
...
Add "About" section to README
2018-06-21 09:49:49 -05:00
Stan James
febf412642
Add "About" section to README
...
Per issue https://github.com/vulcanize/vulcanizedb/issues/60 , adding description provided by @AFDudley on gitter here:
https://gitter.im/vulcanizeio/VulcanizeDB?at=5b2a54b0148056028591b323
2018-06-20 15:55:33 -06:00
Rob Mulholand
b1df139d29
Merge pull request #57 from vulcanize/fix-missing-blocks-lookup
...
Fix missing block numbers lookup
2018-06-13 11:28:22 -05:00
Rob Mulholand
3e39ccb9bb
Fix missing block numbers lookup
...
- Previously, a block was treated as missing if there was a number in
the given range for which there existing a block that did not match
the node's fingerprint. This meant that in a case where we have a block
that does not match the node's fingerprint and also one that does match,
the block would be treated as missing. This led to errors being thrown
when attempting to add a block that already exists.
- These changes treat a block as missing only if we do not already have
a block that matches the number and node fingerprint.
2018-06-08 11:26:25 -05:00
Rob Mulholand
7533e6d476
Merge pull request #52 from vulcanize/cold-import
...
Cold import
2018-05-21 09:34:05 -05:00
Rob Mulholand
d5c2ab33fc
Sync only missing blocks on cold import
...
= Add eth_node_fingerprint to block that can be imitated by both hot and cold imports
- Only sync missing blocks (blocks that are missing or don't share a fingerprint) on cold import
- Set block is_final status after import
2018-05-07 16:38:04 -05:00
A. F. Dudley
4baaed94ec
Merge pull request #50 from gitter-badger/gitter-badge-1
...
Add a Gitter chat badge to README.md
2018-05-05 14:39:36 -04:00
Rob Mulholand
5a5e08bd13
Get head block number
...
- Allows us to refuse to sync past current head
- Creates the opportunity to add a flag for syncing all blocks
2018-05-04 11:03:50 -05:00
