Commit Graph

82 Commits

Author SHA1 Message Date
Ian Norden
decc2a3caf remove pkg/transformers; extract shared files needed to
libraries/shared; work on automated db migration management
2019-02-24 21:38:47 -06:00
Ian Norden
55fa9b8364 auto management of dependencies temporarily needed to build plugin and transformer
migrations; new test showing it is working with external transformers runs
properly when migrations were manually performed but still need
to test automated migration management
2019-02-24 15:23:35 -06:00
Ian Norden
6c2d895023 composeAndExecute command that loads and executes over arbitrary transformer set exported from a go plugin generated according to config file; test for plugin generation, loading, and execution; work on plugins README 2019-02-24 15:23:35 -06:00
Edvard
0bf8bd4183 Add char replacer when searching for env variables 2019-02-11 11:27:33 +01:00
Edvard
e4968c4a75 Add back override with env variables 2019-02-11 11:26:12 +01:00
Rob Mulholand
867f92c431 (VDB-298) Consume Pit contract storage diffs
- Continuously parse storage diffs CSV data to read Pit contract state
- Convert ilks in database to raw bytes32 value for use in generating
  storage keys dynamically
- Persist storage diffs with block number and hash for validation
2019-02-06 10:31:46 -06:00
Ian Norden
1f95eb6443 Merge branch 'staging' into pub_merge 2019-01-24 09:47:12 -06:00
Ian Norden
dc87f39889 change omniwatcher database test helpers so that it doesn't interfere with downstream maker tests 2019-01-23 14:55:20 -06:00
Konstantin Zolotarev
65cb6e9092 VDB-205 Fix external configuration values (#121)
* Updated loading config from /Users/konstantinzolotarev/.vulcanizedb for addreses

* Updated contract addresses loading from env variables.

* Implement getter chain for addresses, configs and transformers

Solves the issue with declaring stuff as vars when the environment file hasn't been loaded yet. Instead, this data is fetched dynamically when the root commands are called.

* Update test data to use constant Kovan addresses

* Decouple integration tests from config file IO

* Add ABI to environment file, convert getter chains for related values

* Decouple tests suites from ABI config file IO

* Add startingBlock to environment and implement getter chain

* Exit when called without config file

* Add missing deployment blocks to staging.toml

* Remove comment
2019-01-23 19:44:09 +01:00
Ian Norden
46b898f690 merge public vulcanizedb/master 2019-01-23 00:37:26 -06:00
Ian Norden
e390a97502 finish method polling with hash or address type argument values
collected from watched events; 'MissingMethodsCheckedEventsIntersection'
method to find headers which have been checked for each event of
interest but methods have not yet been polled at that header; tests for
new features; travis ci go version 1.9 -> 1.11 ; consolidate omniWatcher and lightOmniWatcher into single command with light as default
2018-12-21 10:33:31 -06:00
Ian Norden
0a59f06cac begin work on: Add checked_headers column for methods that are polled so taht we don’t duplicate; Add batching of method polling so that we arent generating a rediculously large account address list before using it to poll methods (or persist the list in pg?); User passed ABI and other ways to get ABI; Add ability to collect []byte and hashes from events and use them in method polling same manner as addresses; Event filter addrs => only those event’s addresses/hashes are used for polling; Option to persist seen address/hash/bytes lists into pg; Only generate lists of addresses, []byte, or hashes if a method will use them later 2018-12-21 10:33:31 -06:00
Ian Norden
390a60f7f6 port over lightSync updates from maker repo 2018-11-29 20:33:21 -06:00
Taka Goto
e8be96a4e7 use logrus for logging 2018-11-21 10:14:11 -06:00
Ian Norden
417b18ec6a Edits to address PR issues; change license from apache to AGPL; and work
towards generic method polling and reposito;y; config settings to
filterevents/methods by account address; refactoring some stuff out of
repo and into converter; remove fetcher and instead call
blockchain's FetchContractData directly; finishing tests
2018-11-15 12:32:52 -06:00
Ian Norden
b459cf35ed beginning work on method polling; first need to generate list of token holder address in a completely generic/contratc agnostic fashion. Created address retriever that can iterate through any given contract's watched events, finding the inputs/arguments with address type, and generate a list from those values. Edit: Contract objects now cache every event emitted address as its event logs are transformed into the repo to grow a list of contract associated addresses as we go 2018-11-04 21:37:31 -06:00
Ian Norden
e9dbd771e5 tests and fixes for fetcher, parser, retriever, converter, and repository; update cmd and watcher 2018-11-04 01:49:11 -05:00
Ian Norden
8ce75fe5ad Generic watcher that takes a contract address, grabs the contract abi and starting block number, creates custom event filters, and extracts and transforms event data into postgres. Can configure to look at only a subset of events through CLI flag. Building but needs testing. 2018-11-03 14:00:25 -05:00
Rob Mulholand
05186634bd Add light sync command
- Only syncs block headers (excludes block bodies, transactions, receipts, and logs)
- Modifies validation window to include the most recent block
- Isolates validation window to the variable defined in the cmd directory (blocks
  have a separate variable defined in the block_repository for determining when
  to set a block as final)
2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
7b507667f8 Added user and password to db config 2018-11-03 13:49:23 -05:00
Rob Mulholand
042f1142c0 Handle headers from POA chain
- Fetching headers from Kovan requires custom type without mixHash/Nonce
2018-09-19 11:00:20 -05:00
Elizabeth
cfc8773c5d Continuous sync (#23) 2018-09-19 10:14:49 -05:00
Taka Goto
cf72eae86c add datadog monitoring for vdb 2018-09-04 15:41:24 -05:00
Rob Mulholand
634604d0b5 Combine price feed transformers
- fetches logs from all three price feeds in one query
- assumes eth/usd price feed will be updated to include LogValue event
- updates transformers to run separate from header sync
2018-08-16 11:22:16 -05:00
Rob Mulholand
9231d40369 Extract price feed contract addresses to config 2018-08-16 11:22:16 -05:00
Rob Mulholand
1355271011 Add light sync command
- Only syncs block headers (excludes block bodies, transactions, receipts, and logs)
- Modifies validation window to include the most recent block
- Isolates validation window to the variable defined in the cmd directory (blocks
  have a separate variable defined in the block_repository for determining when
  to set a block as final)
2018-07-18 16:34:12 -05:00
Maxim Krasilnikov
e9e3864ba9 Added user and password to db config 2018-06-22 18:28:34 +03:00
Rob Mulholand
d5c2ab33fc Sync only missing blocks on cold import
= Add eth_node_fingerprint to block that can be imitated by both hot and cold imports
- Only sync missing blocks (blocks that are missing or don't share a fingerprint) on cold import
- Set block is_final status after import
2018-05-07 16:38:04 -05:00
Rob Mulholand
5f6bf32ec1 Add cold import script 2018-05-04 10:54:00 -05:00
Matt K
605b0a96ae Add graphql server (#27)
* Add graphql server

* Update Makefile

* Update log_filters constraint

* Add GetLogFilter to repo

* Update travis (use Makefile, go fmt, go vet)

* Add logFilter schema and resolvers

* Add GetWatchedEvent to watched_events_repo

* Add watchedEventLog schema and resolvers
2018-02-08 10:12:08 -06:00
Matt Krump
aac2239097 Merge old private repo into vulcanize 2018-01-25 18:08:26 -06:00
Matt K
d09c2ae9bb Remove godo (#136) 2018-01-25 15:46:55 -06:00