Ian Norden
fcaa28d923
readme addition; fix postgraphile lodash and webpack-dev-server alerts
2019-04-09 19:03:08 -05:00
Ian Norden
1a4180d595
Merge pull request #55 from vulcanize/readme
...
Refactoring readme
2019-04-08 12:25:14 -05:00
Ian Norden
8e3cf55b7f
review fixes
2019-04-08 11:52:36 -05:00
Ian Norden
6805d3ea29
Merge pull request #52 from vulcanize/account_transformer_staging
...
Account transformer staging
2019-04-08 10:35:38 -05:00
Ian Norden
42890cdaa7
Refactoring readme
2019-04-07 13:18:26 -05:00
Ian Norden
9befc76fc6
remove LIMIT 100; method to check continuity of headers returned but doesn't require
...
the set to start at a specific block number; config for
account_transformer; review fixes; update schema
2019-04-05 17:21:59 -05:00
Ian Norden
c1940c3e58
full_sync and light_sync receipt tables and light receipt repository methods and test
2019-04-05 16:36:48 -05:00
Ian Norden
04ad7b7696
Merge pull request #47 from vulcanize/track_uncle_rewards
...
Track uncle rewards + direct balance polling
2019-04-04 15:55:40 -05:00
Ian Norden
5dcf534b2c
review fixes
2019-04-04 15:21:39 -05:00
Ian Norden
47f8d81380
Merge pull request #53 from vulcanize/geth_update
...
update vendor dep; constrain to revision of geth with UnpackIntoMap
2019-04-03 13:46:03 -05:00
Ian Norden
72c73c25b7
update vendor dep; constrain to revision of geth with UnpackIntoMap; updates to match
2019-04-03 00:14:02 -05:00
Ian Norden
197f98c93d
going ahead and indexing the entire uncle blocks (one of the issues open on public); finish tests
2019-04-02 14:32:30 -05:00
Ian Norden
fd407825c1
review fixes
2019-04-02 10:10:17 -05:00
Ian Norden
185f4c0e93
adjust block/uncle reward tests and add methods to pkg/geth blockchain and ethclient for direct fetching of eth balances
2019-04-02 10:10:17 -05:00
Ian Norden
9030cff2bd
keep track of who is receiving uncle rewards
2019-04-02 10:10:17 -05:00
Ian Norden
ce70b141e5
Merge pull request #50 from vulcanize/env
...
new travis env variable for tests
2019-03-28 09:07:23 -05:00
Ian Norden
1f46995dbd
new travis env variable for tests
2019-03-27 20:10:52 -05:00
Ian Norden
2e8f3109aa
Merge pull request #39 from vulcanize/omni_update
...
Refactoring omni pkg code + PR72 from public branch
2019-03-22 14:42:47 -05:00
Ian Norden
1aa849bcb4
PR fixes; remove all infura token references and setup travis to use encrypted env
...
variable; rest of the ethjson_rpc dependent tests extracted to integration_test
2019-03-21 18:36:51 -05:00
Ian Norden
d3e172d9ab
remove references to infura token except for in infura.toml so that tests will continue to pass on CI
2019-03-21 18:36:49 -05:00
Ian Norden
37fc038605
fixes for review comments
2019-03-21 18:36:49 -05:00
Ian Norden
37e581c7ec
PR#72 from public repo- https://github.com/vulcanize/vulcanizedb/pull/72- also needed to finish pluggin in ENS record transformer
2019-03-21 18:33:56 -05:00
Ian Norden
24879a85aa
mv pkg/omni pkg/contract_watcher
2019-03-21 18:33:56 -05:00
Ian Norden
073378de6d
update © to 2019
2019-03-21 18:33:56 -05:00
Ian Norden
e4e092f542
changes to plugin and commands to accomodate changes
2019-03-21 18:33:56 -05:00
Ian Norden
05c3b9bb48
update readme; example toml
2019-03-21 18:30:48 -05:00
Ian Norden
5101ed4369
generic transformer and watcher interfaces for plugging in transformers built from omni pkging
2019-03-21 18:30:48 -05:00
Ian Norden
8174ce4aee
refactor omni code; rename omniWatcher => contractWatcher; use config instead of excessive number of CLI flags
2019-03-21 18:30:48 -05:00
Ian Norden
84b65caedd
Merge pull request #43 from vulcanize/rename
...
rename TransformerInitializer => EventTransformerInitializer; reorganize factories
2019-03-21 15:02:54 -05:00
Ian Norden
ff55e3ba7a
TransformerInitializer => EventTransformerInitializer reorganization
2019-03-14 11:59:39 -05:00
Ian Norden
a9513c1af6
Merge pull request #32 from vulcanize/plugin_migrations
...
Set order to run plugin transformer db migrations in config
2019-03-13 17:40:54 -05:00
Ian Norden
9f8c50e3ab
tests for migration path ordering and errors
2019-03-13 11:42:30 -05:00
Ian Norden
05cb06c2d3
config example fixes
2019-03-13 11:42:29 -05:00
Ian Norden
f37c458992
the part that actually runs the migrations in order
2019-03-13 11:42:29 -05:00
Ian Norden
b3ce1ecd78
update readme and cmd usage comments
2019-03-13 11:42:29 -05:00
Ian Norden
84aa0a7eba
plugin migration order specified in config
2019-03-13 11:42:29 -05:00
Ian Norden
2ac682afa8
Merge pull request #38 from vulcanize/unpack_update
...
update parseTopicsIntoMap to handle indexed strings
2019-03-13 08:40:37 -05:00
Ian Norden
5dad19b71b
Merge branch 'staging' into unpack_update
2019-03-13 08:16:49 -05:00
Ian Norden
64b7825c71
Merge pull request #31 from vulcanize/event_docs
...
event transformers readme and remove last links to mcd_transformers in main README
2019-03-13 08:15:43 -05:00
Ian Norden
ac93139863
update parseTopicsIntoMap to handle strings
2019-03-12 13:36:23 -05:00
Ian Norden
35c5808a07
readme fixes
2019-03-12 12:54:36 -05:00
Ian Norden
1d6219cc83
event transformers readme and update main readme to remove last links to
...
mcd_transformers
2019-03-12 12:45:29 -05:00
Ian Norden
8bd397e2eb
Merge pull request #29 from vulcanize/fix
...
Fix for Travis issue at https://github.com/vulcanize/mcd_transformers/pull/6
2019-03-06 11:27:43 -06:00
Ian Norden
e1a317c3f6
different errors; readme edit
2019-03-06 00:14:37 -06:00
Ian Norden
f53a7b8b5d
Merge pull request #28 from vulcanize/geth_update
...
update geth to 1.8.23 (post can'tstantinople)
2019-03-06 00:01:53 -06:00
Ian Norden
5fd9a59027
fix for issue with Travis building plugin from plugin repo
2019-03-05 08:57:54 -06:00
Ian Norden
e2fab256c9
update geth to 1.8.23 (post can'tstantinople)
2019-03-03 03:37:27 -06:00
Ian Norden
06fab13c54
Merge pull request #2 from vulcanize/compose_and_execute
...
Compose and execute transformers
2019-03-01 06:37:57 -06:00
Ian Norden
83f7daf37d
compose and execute as separate commands; update README and add git checkout vendor/github.com/ethereum/go-ethereum/accounts/abi to make build
2019-02-28 18:15:16 -06:00
Ian Norden
2c5ddd03ba
core repo migrations in fixed form
2019-02-26 00:52:54 -06:00
Ian Norden
708425c4d6
rebase; extract factories and the mocks they are dependent on to
...
libraries/shared; adjust omni test_helpers to drop and recreate
checked_headers table to avoid reaching 1600 column limit after repeated
tests (dropping columns doesn't actually remove them from contributing
to that limit)
2019-02-25 01:34:38 -06:00
Ian Norden
17f4d3dfa5
move maker plugin generator test to mcd_transformers
2019-02-24 21:53:04 -06:00
Ian Norden
f2072561a7
removing mcd/maker references; delete test_data; update READMEs; use
...
logrus
2019-02-24 21:53:04 -06:00
Ian Norden
3d34a9e7c9
remove maker migrations and convert back to timestamps and fix bug in
...
composeAndExecute command
2019-02-24 21:52:23 -06:00
Ian Norden
b449193b16
addressing review comments; still need to reorg migrations and add helper tests
2019-02-24 21:38:48 -06:00
Ian Norden
2868cf2f73
goose changes; update plugin readme; use rel paths for transformers in config; more commenting in pkg/plugin; fix PluginWriter to work with different transformer types; reopen generator tests
2019-02-24 21:38:48 -06:00
Ian Norden
90e67d6da9
move storage mappings pkg into shared
2019-02-24 21:38:48 -06:00
Ian Norden
03a7379617
work on porting storage watcher; watcher type is defined for
...
transformers in the config, storage transformers are exported from
plugin like event transformers
2019-02-24 21:38:47 -06:00
Ian Norden
b51dcb55de
comment out plugin test til mcd_transformers is updated correctly
2019-02-24 21:38:47 -06:00
Ian Norden
a322372713
refactoring plugin generating code
2019-02-24 21:38:47 -06:00
Ian Norden
decc2a3caf
remove pkg/transformers; extract shared files needed to
...
libraries/shared; work on automated db migration management
2019-02-24 21:38:47 -06:00
Ian Norden
55fa9b8364
auto management of dependencies temporarily needed to build plugin and transformer
...
migrations; new test showing it is working with external transformers runs
properly when migrations were manually performed but still need
to test automated migration management
2019-02-24 15:23:35 -06:00
Ian Norden
6c2d895023
composeAndExecute command that loads and executes over arbitrary transformer set exported from a go plugin generated according to config file; test for plugin generation, loading, and execution; work on plugins README
2019-02-24 15:23:35 -06:00
Ian Norden
2ecc524224
Merge pull request #9 from vulcanize/VDB322-Vow-storage-diff
...
Vdb322 vow storage diff
2019-02-15 11:04:22 -06:00
Ian Norden
8749c94994
update license to AGPL-3.0
2019-01-24 14:41:30 -06:00
Ian Norden
ab371a2024
Merge remote-tracking branch 'maker/staging' into staging
2019-01-24 10:22:22 -06:00
Ian Norden
f97928d418
Merge pull request #1 from vulcanize/pub_merge
...
Merge public repo
2019-01-24 10:16:37 -06:00
Ian Norden
1f95eb6443
Merge branch 'staging' into pub_merge
2019-01-24 09:47:12 -06:00
Ian Norden
00c2052da3
for some reason an error is thrown when the ipc string is declared in integration_tests_suite_test.go but not when moved to a different file in the integration_tests package; I think this is due to switching to go1.11
2019-01-23 15:23:23 -06:00
Ian Norden
dc87f39889
change omniwatcher database test helpers so that it doesn't interfere with downstream maker tests
2019-01-23 14:55:20 -06:00
Ian Norden
46b898f690
merge public vulcanizedb/master
2019-01-23 00:37:26 -06:00
Ian Norden
aa4e698240
move interface_getter.go from ens_watcher to vDB pkging- used to construct a custom ABI based on results from calls to supportsInterface; also moved fetcher.go from examples to pkg
2019-01-15 15:49:18 -06:00
Ian Norden
59cdaa05e6
moving some memory allocation to init function rather than within
...
execution cycles and fixing test db tear down so that it drops
checked_headers table instead of individual columns (droping columns
only hides them, they remain in memory and continue to contribute to the
1600 column limit which causes us to eventually exceed it)
2019-01-07 13:07:31 -06:00
Ian Norden
c5d4e8e7af
fix conflicts and merge branch documentation
2018-12-21 11:23:22 -06:00
Ian Norden
2cbe6e7a70
method piping
2018-12-21 11:14:54 -06:00
Ian Norden
456c735087
fetch headers and logs in batches across all contracts and events
2018-12-21 10:33:31 -06:00
Ian Norden
d188329661
fetch event logs in batches- all events for a contract address at a given block height are fetched together, different contract events still fetched separately
2018-12-21 10:33:31 -06:00
Ian Norden
e390a97502
finish method polling with hash or address type argument values
...
collected from watched events; 'MissingMethodsCheckedEventsIntersection'
method to find headers which have been checked for each event of
interest but methods have not yet been polled at that header; tests for
new features; travis ci go version 1.9 -> 1.11 ; consolidate omniWatcher and lightOmniWatcher into single command with light as default
2018-12-21 10:33:31 -06:00
Ian Norden
0a59f06cac
begin work on: Add checked_headers column for methods that are polled so taht we don’t duplicate; Add batching of method polling so that we arent generating a rediculously large account address list before using it to poll methods (or persist the list in pg?); User passed ABI and other ways to get ABI; Add ability to collect []byte and hashes from events and use them in method polling same manner as addresses; Event filter addrs => only those event’s addresses/hashes are used for polling; Option to persist seen address/hash/bytes lists into pg; Only generate lists of addresses, []byte, or hashes if a method will use them later
2018-12-21 10:33:31 -06:00
Ian Norden
8c5b1b4dbe
remove accidental duplicate vendor lib that was imported by goland
2018-12-21 10:33:21 -06:00
Ian Norden
ce428199eb
finishing porting omni watcher to work with light sync; split into full, light,
...
and shared directories and refactor as much into shared; finish
lightSync omni watcher tests
2018-12-07 09:54:53 -06:00
Ian Norden
e02b33547d
finishing porting omni watcher to work with light sync; split into full, light,
...
and shared directories and refactor as much into shared; finish
lightSync omni watcher tests
2018-11-30 13:33:48 -06:00
Ian Norden
58b34548f3
vendor updates to work with go 1.11 and geth 1.8.18
2018-11-29 20:33:21 -06:00
Ian Norden
975f13b969
reorganizing omni directory and beginning light watcher work
2018-11-29 20:33:21 -06:00
Ian Norden
817bd76713
refactoring event converter and repository so that event logs are
...
persisted as they are converted, preventing build up in memory, and finishing method polling (for full sync)
2018-11-29 20:33:21 -06:00
Ian Norden
390a60f7f6
port over lightSync updates from maker repo
2018-11-29 20:33:21 -06:00
Ian Norden
26aaa8319b
Merge pull request #68 from vulcanize/omni
...
Generic Watcher -- Work Towards Epics 1/3
2018-11-15 12:51:43 -06:00
Ian Norden
417b18ec6a
Edits to address PR issues; change license from apache to AGPL; and work
...
towards generic method polling and reposito;y; config settings to
filterevents/methods by account address; refactoring some stuff out of
repo and into converter; remove fetcher and instead call
blockchain's FetchContractData directly; finishing tests
2018-11-15 12:32:52 -06:00
Ian Norden
b459cf35ed
beginning work on method polling; first need to generate list of token holder address in a completely generic/contratc agnostic fashion. Created address retriever that can iterate through any given contract's watched events, finding the inputs/arguments with address type, and generate a list from those values. Edit: Contract objects now cache every event emitted address as its event logs are transformed into the repo to grow a list of contract associated addresses as we go
2018-11-04 21:37:31 -06:00
Ian Norden
e9dbd771e5
tests and fixes for fetcher, parser, retriever, converter, and repository; update cmd and watcher
2018-11-04 01:49:11 -05:00
Ian Norden
5307de2b97
modify go-etheruem vendor lib with methods that allow for unpacking of events into maps of interfaces
2018-11-03 14:01:41 -05:00
Ian Norden
8ce75fe5ad
Generic watcher that takes a contract address, grabs the contract abi and starting block number, creates custom event filters, and extracts and transforms event data into postgres. Can configure to look at only a subset of events through CLI flag. Building but needs testing.
2018-11-03 14:00:25 -05:00
Ian Norden
57820ff473
change transformer interface and watcher so that contract config is now fed into AddTransformers such that a single watcher can be loaded with transformers that use different cofigs
2018-11-03 13:49:24 -05:00
Ian Norden
44177e0772
reorganizing contract-specific modules into event_triggered contract directories
2018-11-03 13:49:23 -05:00
Ian Norden
d66e50dad6
Changes to address Rob's comments. Added generic/event_triggered transformer, tests, and repo migrations for Burn and Mint events
2018-11-03 13:49:23 -05:00
Ian Norden
4aa403d90d
event_triggered transformer, transformer tests, and integrations tests for Transfer and Approval events
2018-11-03 13:49:23 -05:00
Ian Norden
aa2068bd08
adjust retriever to pull token holder addresses from Transfer and Approval events (iterating over Approvals might be redundant); edit Makefile to import new missing dependencies of go-ethereum/accounts/keystore, organizing mocks and adding event related mocks and filters
2018-11-03 13:49:23 -05:00
Ian Norden
ada872404e
Transfer and Approval event repo and repo tests + db migrations for these events.
2018-11-03 13:49:23 -05:00
Ian Norden
55a73c5797
work on erc20 event_triggered functionality and generic helper for log unpacking and converting. helpers.ConvertToLog and helpers.createTopics have been adjusted to be variadic to work with event logs with any number of topics. Also uncovered issue with Dai and TrueUSD that means they doesn't really conform to the ERC20 standard. this is because they named their arguments to standard events like Approval and Trasnfer differently (e.g. Approval(src address, guy address, wad uint) and Approval(owner address, spender address, value uint) instead of the standard which is Approval(tokenOwner address, spender address, tokens uint)). This causes incompatibility with generic ERC20 entities and converters for these events.
2018-11-03 13:49:23 -05:00
Ian Norden
44e0a8d303
retriever for generating list of all token holder addresses + updated transformer to use said addresses to populate balanceOf and allowance information and added database migrations for balance and allowance tables
2018-11-03 13:49:23 -05:00
Ian Norden