ipld-eth-server

Author	SHA1	Message	Date
Ian Norden	fd407825c1	review fixes	2019-04-02 10:10:17 -05:00
Ian Norden	185f4c0e93	adjust block/uncle reward tests and add methods to pkg/geth blockchain and ethclient for direct fetching of eth balances	2019-04-02 10:10:17 -05:00
Ian Norden	9030cff2bd	keep track of who is receiving uncle rewards	2019-04-02 10:10:17 -05:00
Rob Mulholand	f5b32a11b0	Improve transaction syncing test coverage	2019-03-28 14:31:17 -05:00
Rob Mulholand	81dfd12665	Address PR comments	2019-03-28 14:31:17 -05:00
Rob Mulholand	54d46638a8	updates for light sync transactions	2019-03-28 14:31:17 -05:00
Rob Mulholand	79e011aad2	Add light sync transactions table - Foreign key to header instead of block - Can use same Transaction struct for both	2019-03-28 14:31:17 -05:00
Rob Mulholand	d93006321b	Rename transactions to full_sync_transactions - Table has foreign key to blocks - Add transaction RLP and transaction index to table - Enables other tables with standalone transactions or transactions associated with other data structures (e.g. headers)	2019-03-28 14:31:17 -05:00
Ian Norden	1aa849bcb4	PR fixes; remove all infura token references and setup travis to use encrypted env variable; rest of the ethjson_rpc dependent tests extracted to integration_test	2019-03-21 18:36:51 -05:00
Ian Norden	d3e172d9ab	remove references to infura token except for in infura.toml so that tests will continue to pass on CI	2019-03-21 18:36:49 -05:00
Ian Norden	37fc038605	fixes for review comments	2019-03-21 18:36:49 -05:00
Ian Norden	37e581c7ec	PR#72 from public repo- https://github.com/vulcanize/vulcanizedb/pull/72- also needed to finish pluggin in ENS record transformer	2019-03-21 18:33:56 -05:00
Ian Norden	24879a85aa	mv pkg/omni pkg/contract_watcher	2019-03-21 18:33:56 -05:00
Ian Norden	073378de6d	update © to 2019	2019-03-21 18:33:56 -05:00
Ian Norden	e4e092f542	changes to plugin and commands to accomodate changes	2019-03-21 18:33:56 -05:00
Ian Norden	8174ce4aee	refactor omni code; rename omniWatcher => contractWatcher; use config instead of excessive number of CLI flags	2019-03-21 18:30:48 -05:00
Ian Norden	ff55e3ba7a	TransformerInitializer => EventTransformerInitializer reorganization	2019-03-14 11:59:39 -05:00
Ian Norden	9f8c50e3ab	tests for migration path ordering and errors	2019-03-13 11:42:30 -05:00
Ian Norden	f37c458992	the part that actually runs the migrations in order	2019-03-13 11:42:29 -05:00
Ian Norden	84aa0a7eba	plugin migration order specified in config	2019-03-13 11:42:29 -05:00
Rob Mulholand	1414779d52	Use sqlx.Tx instead of sql.Tx - requires using db.Beginx() instead of db.Begin() - enables calling tx.Get()	2019-03-06 12:43:39 -06:00
Rob Mulholand	7ed4dcd1fa	Remove Tic logic	2019-03-06 12:43:39 -06:00
Rob Mulholand	b6bef49318	Add tests for plugin helpers - Prevents code analysis from showing functions as unused - Also address other code analysis issues	2019-03-06 12:43:39 -06:00
Rob Mulholand	a806cb2818	Extract slow tests from omni full transformer	2019-03-06 12:43:06 -06:00
Rob Mulholand	34c96654ce	Extract slow tests from omni light transformer	2019-03-06 12:43:06 -06:00
Ian Norden	8bd397e2eb	Merge pull request #29 from vulcanize/fix Fix for Travis issue at https://github.com/vulcanize/mcd_transformers/pull/6	2019-03-06 11:27:43 -06:00
Ian Norden	e1a317c3f6	different errors; readme edit	2019-03-06 00:14:37 -06:00
Ian Norden	5fd9a59027	fix for issue with Travis building plugin from plugin repo	2019-03-05 08:57:54 -06:00
Edvard Hübinette	55b6ad9381	Optimise missing block numbers query (#23 )	2019-03-05 11:52:38 +01:00
Ian Norden	83f7daf37d	compose and execute as separate commands; update README and add git checkout vendor/github.com/ethereum/go-ethereum/accounts/abi to make build	2019-02-28 18:15:16 -06:00
Ian Norden	708425c4d6	rebase; extract factories and the mocks they are dependent on to libraries/shared; adjust omni test_helpers to drop and recreate checked_headers table to avoid reaching 1600 column limit after repeated tests (dropping columns doesn't actually remove them from contributing to that limit)	2019-02-25 01:34:38 -06:00
Ian Norden	17f4d3dfa5	move maker plugin generator test to mcd_transformers	2019-02-24 21:53:04 -06:00
Ian Norden	3d34a9e7c9	remove maker migrations and convert back to timestamps and fix bug in composeAndExecute command	2019-02-24 21:52:23 -06:00
Ian Norden	b449193b16	addressing review comments; still need to reorg migrations and add helper tests	2019-02-24 21:38:48 -06:00
Ian Norden	2868cf2f73	goose changes; update plugin readme; use rel paths for transformers in config; more commenting in pkg/plugin; fix PluginWriter to work with different transformer types; reopen generator tests	2019-02-24 21:38:48 -06:00
Ian Norden	03a7379617	work on porting storage watcher; watcher type is defined for transformers in the config, storage transformers are exported from plugin like event transformers	2019-02-24 21:38:47 -06:00
Ian Norden	b51dcb55de	comment out plugin test til mcd_transformers is updated correctly	2019-02-24 21:38:47 -06:00
Ian Norden	a322372713	refactoring plugin generating code	2019-02-24 21:38:47 -06:00
Ian Norden	decc2a3caf	remove pkg/transformers; extract shared files needed to libraries/shared; work on automated db migration management	2019-02-24 21:38:47 -06:00
Ian Norden	55fa9b8364	auto management of dependencies temporarily needed to build plugin and transformer migrations; new test showing it is working with external transformers runs properly when migrations were manually performed but still need to test automated migration management	2019-02-24 15:23:35 -06:00
Ian Norden	6c2d895023	composeAndExecute command that loads and executes over arbitrary transformer set exported from a go plugin generated according to config file; test for plugin generation, loading, and execution; work on plugins README	2019-02-24 15:23:35 -06:00
Edvard Hübinette	fa4abe4e5c	Remove spammy and unnecessary error log (#21 )	2019-02-22 12:56:06 +01:00
Rob Mulholand	745d0e8eda	Use storage value metadata getter in pit repository tests	2019-02-21 15:25:16 -06:00
Rob Mulholand	a5121913f4	Error on missing keys in pit repository - If ilk missing for ilk-related value, fail fast	2019-02-21 15:03:22 -06:00
Rob Mulholand	3d9341ea1d	Reference ilks foreign key from Cat storage repository	2019-02-21 14:33:59 -06:00
Rob Mulholand	dbdd16d3a2	(VDB-380) Create separate table for ilks - reference ilk by foreign key every place it's used	2019-02-21 11:23:00 -06:00
Rob Mulholand	11dd641a84	(VDB-354) Queue unrecognized storage diffs - If we recognize a storage diff as coming from a watched contract but don't recognize the key, queue it for retrying later (after we've seen an event that might help us recognize the key) - Remove unused errs and args - Panic on unrecognized types (should not happen)	2019-02-20 15:22:39 -06:00
Edvard	fb47a747f6	Fix confusion between nflip and flip	2019-02-20 14:37:55 +01:00
Edvard	56ca45509f	DRY up storage diff repo tests	2019-02-20 14:37:55 +01:00
Edvard	5746de7c97	Add tests for Cat storage diff transformer Update schema after rebase migrations Fix small issues from review	2019-02-20 14:37:46 +01:00
Edvard	cd6611d2ec	Add Cat storage transformer	2019-02-20 14:37:46 +01:00
Edvard Hübinette	53789c82ab	Merge pull request #12 from vulcanize/VDB-392-fix-io-error-propagation VDB-392 Fix IO error propagation	2019-02-20 13:58:00 +01:00
Edvard	1a2f46b308	Fix small issues from review	2019-02-20 13:10:55 +01:00
Ian Norden	2ecc524224	Merge pull request #9 from vulcanize/VDB322-Vow-storage-diff Vdb322 vow storage diff	2019-02-15 11:04:22 -06:00
Taka Goto	64c7ae1bb1	uppercase Woe and Ash	2019-02-15 06:15:47 -06:00
Rob Mulholand	df521eea7f	Merge pull request #11 from vulcanize/storage-transformers-readme Add storage transformers readme	2019-02-14 14:24:01 -06:00
Edvard	13eacf2081	Update tests for error propagation	2019-02-14 16:04:12 +01:00
Edvard	6cd4e5ea95	Improve I/O error propagation	2019-02-14 16:03:57 +01:00
Rob Mulholand	0378068467	add example contract and instructions for watching its storage	2019-02-13 20:34:30 -06:00
Taka Goto	a8322c7d26	add woe to storage diff vow contract	2019-02-12 21:50:52 -06:00
Rob Mulholand	ee113668a5	Add first draft of storage transformers readme - Explain process of creating a new storage transformer	2019-02-12 15:58:28 -06:00
Rob Mulholand	ac9375ae47	Discard logs created during tests - Make sure output is being set to discard for logrus to reduce clutter in the console	2019-02-12 11:18:39 -06:00
Taka Goto	4fe2ff4104	repository methods:	2019-02-11 16:34:02 -06:00
Taka Goto	c2c21325b5	vow contract storage diffing excluding sin	2019-02-11 15:28:54 -06:00
Edvard Hübinette	f73623789e	Merge pull request #6 from 8thlight/VDB-337-Dockerize-vDB VDB-337 Dockerize vDB	2019-02-11 12:37:56 +01:00
Edvard	9b986c4c14	Nuke OpenRC from docker image, update readme's	2019-02-11 11:27:33 +01:00
Edvard	553fdbfa09	Default config starting block to 0	2019-02-11 11:27:33 +01:00
Edvard	e6e66d58dd	Add container service files and docker README	2019-02-11 11:26:12 +01:00
Rob Mulholand	c05d3b7836	(VDB-280) Consume vat contract storage diffs	2019-02-08 13:36:30 -06:00
Rob Mulholand	dba4e48a3e	Use map for key in storage value metadata - Enables inserting multiple keys for one storage row (e.g. ilk and urn)	2019-02-08 11:41:17 -06:00
Rob Mulholand	09abcb841a	Persist Urns in raw bytes32 format - Enables simple usage in contract storage lookups	2019-02-08 11:41:17 -06:00
Takayuki Goto	0a024d429d	* update transformer to able to recheck headers (#4 ) * update transformer to able to recheck headers * put cap on rechecking header * integration test for recheck headers * use enum for recheck headers param; make recheck cap configurable * update integration tests with new test config * update omni pkg with new recheck header column type * update migration with new migration tool and final tweaks needed to accommodate changes in omni pkg	2019-02-08 10:35:46 -06:00
Rob Mulholand	867f92c431	(VDB-298) Consume Pit contract storage diffs - Continuously parse storage diffs CSV data to read Pit contract state - Convert ilks in database to raw bytes32 value for use in generating storage keys dynamically - Persist storage diffs with block number and hash for validation	2019-02-06 10:31:46 -06:00
Edvard Hübinette	29f2537130	Merge pull request #7 from vulcanize/VDB-347-Squelch-lightSync VDB-347 Squelch lightSync in stdout	2019-02-01 11:02:48 +01:00
Edvard	a1fa21cbec	Remove test based on window buffer stuff	2019-01-31 16:39:29 +01:00
Edvard	8eea1b00ed	Redirect validation window status to logrus instead of stdout	2019-01-31 15:55:28 +01:00
Ian Norden	8749c94994	update license to AGPL-3.0	2019-01-24 14:41:30 -06:00
Ian Norden	ab371a2024	Merge remote-tracking branch 'maker/staging' into staging	2019-01-24 10:22:22 -06:00
Ian Norden	1f95eb6443	Merge branch 'staging' into pub_merge	2019-01-24 09:47:12 -06:00
Taka Goto	c5d513e96d	fix index out of range on lightSync; hopefully the new integration tests will catch them next time :)	2019-01-23 15:25:44 -06:00
Ian Norden	00c2052da3	for some reason an error is thrown when the ipc string is declared in integration_tests_suite_test.go but not when moved to a different file in the integration_tests package; I think this is due to switching to go1.11	2019-01-23 15:23:23 -06:00
Ian Norden	dc87f39889	change omniwatcher database test helpers so that it doesn't interfere with downstream maker tests	2019-01-23 14:55:20 -06:00
Konstantin Zolotarev	65cb6e9092	VDB-205 Fix external configuration values (#121 ) * Updated loading config from /Users/konstantinzolotarev/.vulcanizedb for addreses * Updated contract addresses loading from env variables. * Implement getter chain for addresses, configs and transformers Solves the issue with declaring stuff as vars when the environment file hasn't been loaded yet. Instead, this data is fetched dynamically when the root commands are called. * Update test data to use constant Kovan addresses * Decouple integration tests from config file IO * Add ABI to environment file, convert getter chains for related values * Decouple tests suites from ABI config file IO * Add startingBlock to environment and implement getter chain * Exit when called without config file * Add missing deployment blocks to staging.toml * Remove comment	2019-01-23 19:44:09 +01:00
Ian Norden	46b898f690	merge public vulcanizedb/master	2019-01-23 00:37:26 -06:00
Taka Goto	3595771825	use batchElem from rpc client	2019-01-21 14:52:37 -06:00
Takayuki Goto	54d771cb06	Merge branch 'staging' into batch-historical-headers	2019-01-16 15:56:36 -06:00
Taka Goto	24cf0661b0	lowercase address on map to get transformers	2019-01-16 14:24:00 -06:00
Taka Goto	07da9d720b	fix get headers when less than 100 fetched	2019-01-16 10:16:06 -06:00
Taka Goto	074140b9ac	iterate for loop by headers instead of blocknumbers	2019-01-15 16:34:20 -06:00
Ian Norden	aa4e698240	move interface_getter.go from ens_watcher to vDB pkging- used to construct a custom ABI based on results from calls to supportsInterface; also moved fetcher.go from examples to pkg	2019-01-15 15:49:18 -06:00
Takayuki Goto	f1ba21b505	Merge branch 'staging' into batch-historical-headers	2019-01-13 21:29:58 -06:00
yaoandrew	e2b864baa8	Add rollback if GetTicInTx fails and move GetTicInTx to shared/repository.go	2019-01-11 10:07:00 -08:00
Andrew J Yao	c8aa3dc275	Replace standard log with logrus import	2019-01-11 10:07:00 -08:00
yaoandrew	bc8b7a101c	refactor error handling in vat, cat, vow	2019-01-11 10:07:00 -08:00
yaoandrew	525fada3d7	refactor error handling in frob, pit, price, tend	2019-01-11 10:07:00 -08:00
yaoandrew	657aaccff3	refactor error handling in drip, flap, flop	2019-01-11 10:07:00 -08:00
yaoandrew	05351a8ef4	refactor error handling in dent	2019-01-11 10:07:00 -08:00
yaoandrew	08ac7f18a5	refactor error handling in flip_deal	2019-01-11 10:07:00 -08:00
yaoandrew	f655997fde	refactor error handling in pit_vow	2019-01-11 10:07:00 -08:00
yaoandrew	843d30c88e	refactor error handling in flip	2019-01-11 10:07:00 -08:00
yaoandrew	513513e93b	refactor error handling in chop_lump	2019-01-11 10:07:00 -08:00
yaoandrew	78b302eab6	refactor error handling in bite transformer	2019-01-11 10:07:00 -08:00
Edvard Hübinette	ebca338b1e	VDB-302 Sleep when no missing blocks in lightSync (#129 ) * Sleep when no missing blocks in lightSync * Fix tests and error propagation * Correct geth.log to logrus	2019-01-11 10:58:41 +01:00
Takayuki Goto	ac430ddf54	Merge branch 'staging' into batch-historical-headers	2019-01-08 19:06:53 -06:00
Taka Goto	aa99c861a8	batch headers by 100s	2019-01-08 16:53:26 -06:00
Ian Norden	59cdaa05e6	moving some memory allocation to init function rather than within execution cycles and fixing test db tear down so that it drops checked_headers table instead of individual columns (droping columns only hides them, they remain in memory and continue to contribute to the 1600 column limit which causes us to eventually exceed it)	2019-01-07 13:07:31 -06:00
Edvard	2292c94e99	Merge branch 'staging' into VDB-125-aggregate-log-fetching	2019-01-07 15:25:37 +01:00
Edvard	50cc0c2459	Update tests for watcher and repository	2019-01-07 15:20:05 +01:00
Edvard	833dde62cf	Remove injection of fetcher and repository to watcher	2019-01-07 15:19:31 +01:00
Ian Norden	2cbe6e7a70	method piping	2018-12-21 11:14:54 -06:00
Ian Norden	456c735087	fetch headers and logs in batches across all contracts and events	2018-12-21 10:33:31 -06:00
Ian Norden	d188329661	fetch event logs in batches- all events for a contract address at a given block height are fetched together, different contract events still fetched separately	2018-12-21 10:33:31 -06:00
Ian Norden	e390a97502	finish method polling with hash or address type argument values collected from watched events; 'MissingMethodsCheckedEventsIntersection' method to find headers which have been checked for each event of interest but methods have not yet been polled at that header; tests for new features; travis ci go version 1.9 -> 1.11 ; consolidate omniWatcher and lightOmniWatcher into single command with light as default	2018-12-21 10:33:31 -06:00
Ian Norden	0a59f06cac	begin work on: Add checked_headers column for methods that are polled so taht we don’t duplicate; Add batching of method polling so that we arent generating a rediculously large account address list before using it to poll methods (or persist the list in pg?); User passed ABI and other ways to get ABI; Add ability to collect []byte and hashes from events and use them in method polling same manner as addresses; Event filter addrs => only those event’s addresses/hashes are used for polling; Option to persist seen address/hash/bytes lists into pg; Only generate lists of addresses, []byte, or hashes if a method will use them later	2018-12-21 10:33:31 -06:00
Edvard	e54699c039	Remove todo on chunker non-match (it's basically a filter)	2018-12-14 15:22:46 +01:00
Edvard Hübinette	a3e8633aff	VDB-104 improve lightsync ui (#125 ) * Return PopulateMissingHeaders early if the sync is at the head of the chain * Squelch logging if no blocks to sync * Fix broken test * Refactor repository tests	2018-12-14 12:11:48 +01:00
Edvard	7cf694f5e7	Update transformer documentation	2018-12-14 10:30:08 +01:00
Edvard	35f7f719e6	Swap strings.Builder for bytes.Buffer for go 1.9 support	2018-12-13 16:08:57 +01:00
Edvard	129964f3bc	Add tests for now shared MissingHeaders	2018-12-13 13:58:16 +01:00
Edvard	e5db740ae4	Move MockTransformer to test_data	2018-12-13 13:03:51 +01:00
Edvard	d30fcfed80	Use new config getter on shared.Transformer <3	2018-12-13 12:39:57 +01:00
Edvard	882a6dd7a2	Update tests with new adding of transformers	2018-12-12 15:41:44 +01:00
Edvard	dc43547612	Improve process of adding transformers	2018-12-12 15:41:29 +01:00
Edvard	a86a2ec5c6	Improve checked header column detection	2018-12-12 12:13:23 +01:00
Edvard	4e089c363e	Fixes after merging staging	2018-12-11 15:19:27 +01:00
Edvard	ff47e2e06e	Merge branch 'staging' into VDB-125-aggregate-log-fetching	2018-12-11 15:03:21 +01:00
Edvard	9000c6e9f9	Add test for watcher log delegation	2018-12-11 15:02:32 +01:00
Edvard	5d4239de05	Finish watcher and watcher tests (+fmt)	2018-12-11 13:52:02 +01:00
Edvard	d93817f346	Interface-ify shared repository	2018-12-11 11:35:13 +01:00
Edvard	83593cb5f2	Cleanup fetcher and add test todos	2018-12-10 21:12:55 +01:00
Edvard	2d81861ae5	Add chunker tests	2018-12-10 21:11:25 +01:00
Edvard	bc9fd17793	Fix remaining integration tests	2018-12-10 17:12:51 +01:00
Edvard	3cb8fe60d4	Fix persistHeader not returning proper header ID	2018-12-10 16:58:24 +01:00
Edvard	637623b752	Fix test suites!	2018-12-10 15:40:45 +01:00
Rob Mulholand	670741616e	Use logrus for logging in populating blocks - Removes logging from test output	2018-12-07 11:24:06 -06:00
Rob Mulholand	2a518ad518	(VDB-254) Convert numerical values for Pit.file ilk - convert data to ray when what == "spot" and wad when what == "line"	2018-12-07 11:24:06 -06:00
Rob Mulholand	087af252c9	(VDB-253) Update numerical values for Pit.file debt ceiling - convert Line to wad	2018-12-07 11:24:06 -06:00
Rob Mulholand	4a5765b20d	(VDB-249) Update numerical values for Vat.fold - convert rate to ray	2018-12-07 11:24:06 -06:00
Edvard	e1ba7ac5b4	WIP update tests for aggregate fetching	2018-12-07 18:10:36 +01:00
Edvard	47c75d055b	Update repository tests for absence of MissingHeaders	2018-12-04 17:53:46 +01:00
Edvard	9789648cc2	Delegate log chunks to respective transformers	2018-12-04 17:40:39 +01:00
Edvard	8bebcdc064	Change transformer.Execute to single header	2018-12-04 17:05:34 +01:00
Edvard	38c745e8c3	Refactor fetching out from repositories to log_fetcher and watcher	2018-12-04 16:04:13 +01:00
Ian Norden	e02b33547d	finishing porting omni watcher to work with light sync; split into full, light, and shared directories and refactor as much into shared; finish lightSync omni watcher tests	2018-11-30 13:33:48 -06:00
Edvard	c26736dc9e	WIP continue on aggregate fetching architecture	2018-11-30 17:28:52 +01:00
Ian Norden	975f13b969	reorganizing omni directory and beginning light watcher work	2018-11-29 20:33:21 -06:00
Ian Norden	817bd76713	refactoring event converter and repository so that event logs are persisted as they are converted, preventing build up in memory, and finishing method polling (for full sync)	2018-11-29 20:33:21 -06:00
Ian Norden	390a60f7f6	port over lightSync updates from maker repo	2018-11-29 20:33:21 -06:00
Elizabeth	8ce901d1af	Convert drip_file_ilk tax field to ray (#117 )	2018-11-29 14:10:12 -06:00
Edvard	45a087f5d1	WIP: First steps in refactoring to aggregate fetching	2018-11-28 17:00:04 +01:00
Edvard	423fdf01b5	Add log_chunker	2018-11-28 16:59:06 +01:00
Edvard	f7ecab9dc2	Fix typo	2018-11-28 15:39:46 +01:00
Taka Goto	e8be96a4e7	use logrus for logging	2018-11-21 10:14:11 -06:00
Elizabeth	128d20c9bf	Vdb 269- fetch logs by hash (#122 ) * Upgrade geth from 1.8.15 to 1.8.18 * Update vat_tune to use shared repository methods * Query blockchain by block hash instead of block number range * Remove hash validation from repositories * Fix vow flog integration test * Update README Travis build sticker * Update constants formatting per go fmt * Update EthPublicKeyParser.ParsePublicKey to use discv5.PubkeyID method * Address PR comments	2018-11-21 09:30:00 -06:00
Elizabeth	b19e77e419	Add integration test for Vow.flog (#114 )	2018-11-15 16:44:49 -06:00
Ian Norden	417b18ec6a	Edits to address PR issues; change license from apache to AGPL; and work towards generic method polling and reposito;y; config settings to filterevents/methods by account address; refactoring some stuff out of repo and into converter; remove fetcher and instead call blockchain's FetchContractData directly; finishing tests	2018-11-15 12:32:52 -06:00
Edvard	7cd512ae91	Remove PFSF label	2018-11-15 12:14:59 -06:00
Rob Mulholand	854236fd99	(VDB-267) Remove pit file stability fee - Method is no longer present on the Pit contract	2018-11-15 12:14:59 -06:00
Edvard Hübinette	40408e8300	VDB-120 include tic in dent and tend (#118 ) * Add constants for TTL and tic for fakeHeader * Add tic to tend transformer * Update tests for tend * Fix string conversion bug in fakes * Fix tend integration tests after staging rebase * Add tic to dent transformer * Update dent tests * Change integration tests to use hardcoded block timestamp	2018-11-15 12:57:29 +01:00
Edvard	8f4648a9e3	Add support for never-ending transformer	2018-11-14 10:11:16 -06:00
Rob Mulholand	82fd73ba3f	(VDB-68) Verify log block hash matches header block hash - Delete header on conflict to prompt data refresh (cascade deletes all data associated with that block) - Derive header hash from rpc payload rather than computing it from data (prevents hash mismatch between blockchain and cache)	2018-11-13 14:51:39 -06:00
Elizabeth	6a43cd4287	Vdb 207 the rest of the repos (#111 ) * Use shared repository test examples for frob * Use shared repository methods for frob * Use shared repository test examples for flop kick * Use shared repository methods for flop kick * Use shared repository test examples for flip kick * Use shared repository methods for flip kick * Use shared repository test examples for bite * Use shared repository methods for bite * Alphabetize check headers constants	2018-11-12 10:50:51 -06:00
Edvard Hübinette	4f295b9aec	VDB-207 refactor init move split toll tune flog (#110 ) * Use shared repo tests for vat_init * Use shared methods in vat_init repository * Use shared repo tests for vat_move * Use shared methods in vat_move repository * Use shared repo tests for vat_slip * Use shared methods in vat_slip repository * Use shared repo tests for vat_toll * Use shared methods in vat_toll repository * Use shared methods in vat_tune repository * Use shared repo tests for vow_flog (+small namespace fix) * Use shared methods in vow_flog repository * Go vet + tiny oopsie	2018-11-12 14:17:58 +01:00
Edvard Hübinette	571f300392	Add rendering of available transformer names to continuousLogSync help (#109 )	2018-11-12 14:01:07 +01:00
Elizabeth	672269effa	VDB 207 refactor more repo tests (#107 ) * Use shared repo tests for vat flux * Use shared repo methods for vat flux * Use shared repo test examples for vat fold * Use shared repository methods for vat fold * Use shared repo test examples for vat grab * Use shared repo methods for vat grab * Use shared repo test examples for vat heal * Use shared repo methods for vat heal	2018-11-09 14:54:22 -06:00
Elizabeth	c1e10f09fb	Vdb 67 convert numbers to ray and wad (#105 ) * Add method to convert values to ray or wad units * Convert data to ray or wad for cat_file_chop_lump * Use shared convert functions in price feed conversion * Pull common ray/wad values into vars * Fix after rebase with staging	2018-11-09 14:53:20 -06:00
Rob Mulholand	0b58efb56a	Extract common repository methods - Inject column names to reduce duplication across common behavior - Extract checked headers column names to constants	2018-11-08 13:08:59 -06:00
Rob Mulholand	8c168ed332	Extract constants package - Reduce clutter in the shared package	2018-11-08 13:08:59 -06:00
Rob Mulholand	8c5192f82f	(VDB-210) Refactor tend repository tests	2018-11-08 13:08:59 -06:00
Rob Mulholand	1c1a9c699a	(VDB-209) Refactor price feed repository tests	2018-11-08 13:08:59 -06:00
Rob Mulholand	af8bbe5ce9	(VDB-208) Refactor pit file repository tests	2018-11-08 13:08:59 -06:00
Rob Mulholand	a1c4d70a77	(VDB-99) Transform Flap deal events - Add flapper address to deal transformer config	2018-11-07 10:52:55 -06:00
Rob Mulholand	9cc9d4d3b6	(VDB-95) Transform Flop dent events - Add flopper address to dent config	2018-11-07 10:52:55 -06:00
Rob Mulholand	52bbedd837	(VDB-98) Transform Flap tend events - Add Flapper contract address to tend transformer config	2018-11-07 10:52:55 -06:00
Rob Mulholand	b2a78f1da4	Remove unused transformer config struct	2018-11-07 10:52:55 -06:00
Rob Mulholand	cb141f7cb7	(VDB-204) Add transformer for Flap kick	2018-11-07 10:52:55 -06:00
Taka Goto	627ae288bd	use self hosted parity kovan for staging	2018-11-06 12:58:45 -06:00
Takayuki Goto	7afd978cf0	Merge pull request #100 from 8thlight/update-integration-tests integration tests hits self hosted remote parity kovan node:	2018-11-06 12:08:48 -06:00
Elizabeth	0ab5ea1b25	Refactor more repo tests (#101 ) * Use shared repository tests for deal * Small updates to repo tests * Use shared repository test examples for dent * Use shared repository test examples for drip_drip * Use shared repository test examples for drip_file_ilk * Use shared repository test examples for drip_file_repo * Use shared repository test examples for drip_file_vow * Update formatting	2018-11-06 10:51:26 -06:00
Elizabeth	a4abf3781e	Refactor repository tests (#99 ) * Add shared test examples for repositories - Create - MissingHeaders - MarkHeaderChecked * Use shared repository tests for cat_file_chop_lump * Use shared repository test for cat_file_flip * Use shared repository test for cat_file_pit_vow * Updates to shared repository tests * Use shared repository tests for deal * Small updates to repo tests	2018-11-06 10:30:31 -06:00
Elizabeth	bfe4531d7e	Common transformer tests (#97 ) * Move chop lump transformer test to a generic log_note_transformer_test * Add test cases to transformer test - cat file flip - cat file pit vow - deal - dent * Move transformer tests over to transformer factor test file - drip drip - drip file ilk/repo/vow - pit file deb ceiling/ilk/stability fee - price feeds * Move LogNoteTransformer test into factory package and remove log specific transformer tests * Move Transformer test into factory package and remove log event specific transformer tests * Respond to PR comments	2018-11-06 10:30:11 -06:00
Taka Goto	2cc9826be9	create integration toml file	2018-11-06 09:25:08 -06:00
Ian Norden	b459cf35ed	beginning work on method polling; first need to generate list of token holder address in a completely generic/contratc agnostic fashion. Created address retriever that can iterate through any given contract's watched events, finding the inputs/arguments with address type, and generate a list from those values. Edit: Contract objects now cache every event emitted address as its event logs are transformed into the repo to grow a list of contract associated addresses as we go	2018-11-04 21:37:31 -06:00
Ian Norden	e9dbd771e5	tests and fixes for fetcher, parser, retriever, converter, and repository; update cmd and watcher	2018-11-04 01:49:11 -05:00
Ian Norden	8ce75fe5ad	Generic watcher that takes a contract address, grabs the contract abi and starting block number, creates custom event filters, and extracts and transforms event data into postgres. Can configure to look at only a subset of events through CLI flag. Building but needs testing.	2018-11-03 14:00:25 -05:00
Ian Norden	9b41000b88	Moved fetcher to generic directory (methods have to remain public since it is in seperate package now), added FetchHash method, created ERC20 and generic getters which call the fetcher with specific contract methods (GetTotalSupply, GetBalance, GetAllowance for ERC20 getter, and GetOwner, GetStoppedStatus, GetStringName, GetHashName, GetStringSymbol, GetHashSymbol, and GetDecimals for generic getter). Getter tests cover all but GetBalance and GetAllowance, and also cover all of the Fetcher methods- but with only nil methodArgs. GetAllowance and GetBalance tests are not working against infura and these are the only contract method calls with arguments passed in so I suspect this might be where the issue lies. Have tested GetBalance using previous version of FetchContractData without the variadic input to the Pack method and it fails with the same error so I don’t think it is due to those changes.	2018-11-03 13:49:23 -05:00
Ian Norden	ccd21f73c9	adjusted FetchContractData to work with variable number of contract method arguments, changed fetcher's FetchSupply method to more general FetchBigInt method and added FetchBool and FetchAddress methods	2018-11-03 13:49:23 -05:00
Rob Mulholand	ba071ef13f	Consolidate test doubles - Migrate various mocks of core namespaces to shared version in `fakes` pkg - Err on the side of making test doubles less sophisticated - Don't pull over mocks of namespaces that are only used in example code	2018-11-03 13:49:23 -05:00
Rob Mulholand	5fe6394406	Add tests for pkg/geth/blockchain - inject dependencies instead of initializing them in the constructor	2018-11-03 13:49:23 -05:00
Rob Mulholand	05186634bd	Add light sync command - Only syncs block headers (excludes block bodies, transactions, receipts, and logs) - Modifies validation window to include the most recent block - Isolates validation window to the variable defined in the cmd directory (blocks have a separate variable defined in the block_repository for determining when to set a block as final)	2018-11-03 13:49:23 -05:00
Maxim Krasilnikov	7b507667f8	Added user and password to db config	2018-11-03 13:49:23 -05:00
Takayuki Goto	0d325afbfb	flop kick transformer factory (#98 ) * flop kick transformer factory * add tests for wrong model/entity	2018-11-02 09:13:10 -05:00
Elizabeth	981393d0a7	Use transformer factory for FlipKick (#96 )	2018-11-02 09:12:51 -05:00
Taka Goto	3b722c132e	integration tests hits self hosted parity kovan node:	2018-11-01 10:24:03 -05:00
Elizabeth	ab4e904713	Frob transformer factory (#95 ) * Updates to previous transformers * Use transformer factory for Frob * Move bite integration test to integrationt est pkg	2018-10-29 15:08:00 -05:00
Rob Mulholand	78b8173b37	Improve test coverage for event method signature lookups	2018-10-26 15:35:18 -05:00
Rob Mulholand	602802e5f4	Don't test order of DB results - reduce flakiness by enabling tests to pass regardless of result order	2018-10-26 15:35:18 -05:00
Rob Mulholand	ed624330f6	Rename Flog pack to Vow flog - enables differentiation if another contract implements flog - also deletes flog-specific mocks	2018-10-26 15:35:18 -05:00
Rob Mulholand	1b680d2a4b	Use transformer factory for Drip drip	2018-10-26 15:35:18 -05:00
Edvard Hübinette	f3a82e6ba1	VDB-184 Refactor Dent (#91 ) * Factorise dent transformer * Update dent transformer tests * Add additional tests to dent repository test * Update dent to use LogNoteTransformer factory	2018-10-26 13:56:49 -05:00
Rob Mulholand	d1ee501f12	Use transformer factory for Bite (#89 ) * Use transformer factory for Bite - introduces separate transformer factory for non-LogNote events - converter includes `ToEntities` for events defined via ABI * Updates after rebasing with staging	2018-10-26 13:26:10 -05:00
Rob Mulholand	0e5b23993a	Discard log output from Vat heal during tests	2018-10-26 12:02:56 -05:00
Rob Mulholand	45a52a59f9	Add repository tests for Deal	2018-10-26 12:02:56 -05:00
Edvard	0e1675354f	Update tests for deal transformer	2018-10-26 12:02:56 -05:00
Edvard	35a8009410	Refactor deal transformer to use factory	2018-10-26 12:02:56 -05:00
Elizabeth	d522d33531	Update vat heal to use transformer factory (#88 ) * Update vat heal to use transformer factory	2018-10-26 11:40:24 -05:00
Rob Mulholand	5de949f4c5	Add cat file repository tests	2018-10-26 11:36:02 -05:00
Edvard	00bb48bafb	Fix whitespace errors	2018-10-26 11:36:02 -05:00
Edvard	e6e2ed3308	Update cat file transformer tests	2018-10-26 11:36:02 -05:00
Edvard	28ab5f92ab	Factorise cat_file transformers	2018-10-26 11:36:02 -05:00
Rob Mulholand	47a3c35938	Use transformer factory for Price Feed events	2018-10-26 11:25:16 -05:00
Elizabeth	bd428e752a	Update vat grab to use transformer factory (#85 ) * Update vat grab to use transformer factory	2018-10-26 11:02:39 -05:00
Rob Mulholand	745b3a460e	Add test for updating existing checked header row	2018-10-26 10:45:49 -05:00
Rob Mulholand	e37447ae9a	Use transformer factory for Vat fold	2018-10-26 10:45:49 -05:00
Takayuki Goto	0e7f316568	flip kick integration test (#83 ) * flip kick integration test * use timestamp instead to compare date * fix after rebase staging	2018-10-26 10:26:22 -05:00
Rob Mulholand	3ec5b4afab	Remove unused param in integration test helper	2018-10-25 10:02:06 -05:00
Rob Mulholand	8831586fba	Extract shared mock converter and repository	2018-10-25 10:02:06 -05:00
Rob Mulholand	f5b9a21a8c	Use transformer factory for Vat slip	2018-10-25 10:02:06 -05:00
Rob Mulholand	1e7ee08af5	Use transformer factory for Vat toll	2018-10-25 10:02:06 -05:00
Rob Mulholand	6d662a6ce7	Add test for can't create Vat tune of wrong type	2018-10-25 10:02:06 -05:00
Rob Mulholand	a0cae99437	Use transformer factory for Vat flux	2018-10-25 10:02:06 -05:00
Edvard Hübinette	be249437c2	VDB-117: Add `Vow.flog` transformer (#80 ) * Add DB migrations for flog * Add Vow addresses and constants * Add err logging to ABI signature getter, test for flog sig * Add flog transformer * Add tests+data for flog transformer * Prepend DB tables and columns with vow_ * Panic on failure to get method sig	2018-10-25 12:19:46 +02:00
Rob Mulholand	78d746bb5b	Use transformer factory for Vat tune	2018-10-24 15:49:19 -05:00
Elizabeth	5682ee988b	Add log index to various events (#78 ) * Add log_index field to flip kick * Add log index to flop kick * Add repo tests for Cat.file * Add log_index to vat_flux * Add log_index to vat_slip * Add log_index to vat_toll * Add log_index to vat_tune * Add test to repos for updating checked headers	2018-10-23 16:33:04 -05:00
Takayuki Goto	28f8861f81	Merge pull request #71 from 8thlight/VDB-150-vat-flux-integration-test integration test for vat flux	2018-10-23 16:01:42 -05:00
Rob Mulholand	42d60c3cbd	Add log index to Vat move	2018-10-23 15:25:21 -05:00
Rob Mulholand	4ac896c246	Add log index to Vat init	2018-10-23 15:25:21 -05:00
Rob Mulholand	702f90367a	Add log index to Vat heal	2018-10-23 15:25:21 -05:00
Rob Mulholand	650bbf5874	Add log index to Vat grab	2018-10-23 15:25:21 -05:00
Rob Mulholand	1e74c73591	Add log index to Vat fold	2018-10-23 15:25:21 -05:00
Rob Mulholand	db52955669	Add log index to Tend	2018-10-23 15:25:21 -05:00
Rob Mulholand	e0253cce55	Add log index to Price Feed events	2018-10-23 15:25:21 -05:00
Rob Mulholand	a0a86871b5	Add log index to Frob	2018-10-23 15:25:21 -05:00
Rob Mulholand	b3d00fe3c5	Add log index to Pit file events	2018-10-23 15:25:21 -05:00
Edvard	a0deb5eb62	Add and use transformer labels in shared constants	2018-10-23 11:23:31 +02:00
Edvard	f55449d0e5	Replace GinkgoRandomSeed with proper randomisation	2018-10-23 10:47:01 +02:00
Edvard	794a7e5f99	Nuke some leftover converter interfaces	2018-10-23 10:47:01 +02:00
Edvard	6d9547a888	Fix invalidated test for tend repository	2018-10-23 10:47:01 +02:00
Edvard	7ee3329baf	Privatise DB repository field and add transformer test	2018-10-23 10:47:01 +02:00
Edvard	cb74cbe990	Add cast error checking to refactored transformers	2018-10-23 10:47:01 +02:00
Edvard	b7f8432e65	Fix git whitespace warnings	2018-10-23 10:47:00 +02:00
Edvard	bb204b6b14	Remove unnecessary type washing in factory transformer	2018-10-23 10:45:48 +02:00
Edvard	e7e511764e	Implement skip to next header on empty matchingLogs	2018-10-23 10:45:48 +02:00
Edvard	33f1f62c11	Implement SingleTransformerConfig	2018-10-23 10:45:48 +02:00
Edvard	7d3bf1f0f6	Fixes after rebase	2018-10-23 10:45:48 +02:00
Edvard	70722fc9e3	Add todo for changing to single topic in shared transformer config	2018-10-23 10:45:48 +02:00
Edvard	2a9fafca08	Update drip_file tests	2018-10-23 10:45:48 +02:00
Edvard	dc4eb009c1	Refactor drip_file transformers	2018-10-23 10:45:48 +02:00
Edvard	ccca6e0229	Update old pit file shared config references	2018-10-23 10:43:03 +02:00
Edvard	d9b6603109	Update pit file debt ceiling tests	2018-10-23 10:43:03 +02:00
Edvard	24199cb67e	Refactor pit file debt ceiling transformer	2018-10-23 10:43:03 +02:00
Edvard	0239c4daf6	Fix sneaky missing asterisk	2018-10-23 10:43:03 +02:00
Edvard	399b3b78a0	Update tests for pit file ilk transformer	2018-10-23 10:43:03 +02:00
Edvard	98062a7970	Refactor pit file ilk transfomer	2018-10-23 10:43:03 +02:00
Edvard	837a777c8a	Update transformer name for stability fee	2018-10-23 10:43:03 +02:00
Edvard	3412866182	Update tests for stability_fee	2018-10-23 10:43:03 +02:00
Edvard	2cc303a2f5	Refactor stability_fee transfomer	2018-10-23 10:43:03 +02:00
Edvard	57b3744085	Update tend tests	2018-10-23 10:43:03 +02:00
Edvard	a4df8f348d	Refactor Tend transformer using factory	2018-10-23 10:43:03 +02:00
Edvard	4d36f6200d	Further refactor move and init tests	2018-10-23 10:43:03 +02:00
Edvard	c4d3783f8b	Expand error logging in transformer	2018-10-23 10:43:03 +02:00
Edvard	719ffd5819	Update vat init tests for factory	2018-10-23 10:43:03 +02:00
Edvard	fedbce80f6	Refactor vat init transformer to use factory	2018-10-23 10:43:03 +02:00
Edvard	177d212cc4	Add SetDB test for vat move	2018-10-23 10:43:03 +02:00
Edvard	32654ffabc	Add transformer name to config for logging in factory transformer	2018-10-23 10:43:03 +02:00
Edvard	46d8ea8d97	Snag transformer topics from config in factory	2018-10-23 10:43:03 +02:00
Edvard	155569a1d8	Update vat_move tests	2018-10-23 10:43:03 +02:00
Edvard	af1dfbc489	Implement vat_move transformer using factory	2018-10-23 10:43:03 +02:00
Edvard	15e33db33d	Fix transformer factory	2018-10-23 10:43:03 +02:00
Edvard	1b9e6491da	Add transformer factory	2018-10-23 10:43:03 +02:00
Elizabeth	954fcf2378	integration tests (#75 ) * Deal integration test * Add LogIndex to CatFileChopLump to update unique constraint * Add LogIndex to cat_file_pit_vow records * Add integration tests for Cat.file transformers * Add log index to Cat.file flip	2018-10-22 13:28:42 -05:00
Rob Mulholand	d87fb14c6b	Add log index to Drip file events	2018-10-22 12:14:43 -05:00
Rob Mulholand	de294ad8ea	Add log index to Drip drip	2018-10-22 12:14:43 -05:00
Rob Mulholand	e5ba0496a6	Add log index to Dent	2018-10-22 12:14:43 -05:00
Rob Mulholand	1aafb7201e	Add log index to Deal	2018-10-22 12:14:43 -05:00
Rob Mulholand	495f39abb8	Add log index to Bite	2018-10-22 12:14:43 -05:00
Rob Mulholand	ea3de51792	Add Flop kick integration test	2018-10-19 14:21:57 -05:00
Rob Mulholand	fa828cf041	Add Tend integration test	2018-10-19 14:21:57 -05:00
Rob Mulholand	cbb5ffa3c7	Add Pit file stability fee integration test	2018-10-19 14:21:57 -05:00
Rob Mulholand	3449a1e246	Add Pit file debt ceiling integration test	2018-10-19 14:21:57 -05:00
Rob Mulholand	a83bf5ed0d	Add Pit file ilk integration test	2018-10-19 14:21:57 -05:00
Rob Mulholand	677e78891d	Add Drip file vow integration test	2018-10-19 14:21:57 -05:00
Rob Mulholand	0ce3e3d829	Add Bite integration test	2018-10-19 14:21:57 -05:00
Rob Mulholand	ddc7ee5143	Add Drip drip integration test	2018-10-19 10:56:28 -05:00
Rob Mulholand	b067ba565e	Add Vat fold integration test	2018-10-19 10:56:28 -05:00
Rob Mulholand	fdbe0b0b44	Add Vat heal integration test	2018-10-19 10:56:28 -05:00
Rob Mulholand	a7b9b825c8	Add Vat init integration test	2018-10-19 10:56:28 -05:00
Rob Mulholand	9b427ea1e9	Add Vat move integration test	2018-10-19 10:56:28 -05:00
Taka Goto	21088eb3a2	integration test for vat flux	2018-10-17 16:25:12 -05:00
Konstantin Zolotarev	b38c29ce43	Merge pull request #64 from 8thlight/contract_addresses Added configuration for contract addresses	2018-10-17 21:34:25 +03:00
Rob Mulholand	130d11d0aa	Add Vat slip transformer	2018-10-17 13:30:35 -05:00
Elizabeth	3baca64903	Update address formatting (#67 ) * Add getSignatures command for list of signatures in constants * Update VatTune converter's formatting of addresses and added an integration test * Move VatTune and PriceFeeds integration tests to a common suite * Update VatGrab converter for address format * Update frob converter to fix urn and ilk formatting/decoding * Update formatting * Update make test command and add integrationtest command * Add integration test to travis build * Update creating headers in integration tests to include Raw and Timestamp fields	2018-10-17 13:04:55 -05:00
Taka Goto	4721a91faa	use before each for transformer test	2018-10-17 12:15:15 -05:00
Taka Goto	a3ca319e9b	remove unnecessary byte slicing	2018-10-17 12:04:11 -05:00
Taka Goto	2f64269ab3	update from pr feedback; update tests and use shared fake header	2018-10-17 11:48:57 -05:00
Taka Goto	f32a4fa2a5	use kovan vat contract address	2018-10-17 10:55:30 -05:00
Taka Goto	2e72a7e940	vat flux transformer	2018-10-17 10:53:53 -05:00
Rob Mulholand	d0226dbce5	Convert Bite event field types to numeric - Enables arithmetic operations when constructing views - Also removes application-level references to DB ID	2018-10-16 15:27:13 -05:00
Rob Mulholand	d3df685c46	Add timestamp to block header - Enables quickly deriving timestamp from header for log events	2018-10-12 12:00:02 -05:00
Rob Mulholand	5f67161f41	Convert raw header to JSON - Enables parsing values off of header via Postgres migration	2018-10-12 12:00:02 -05:00
Edvard Hübinette	be58dd4ac8	Add `vat_move` transformer (#47 ) * Add vat_move transformer base * Add vat_move migrations * Add test data for vat_move * Add vat_move transformer to initialisers * Add numeric cast to psql insert of Rad * Add new db schema * Dependency update * Expand abbreviation in repository * Add test suite for vat_move * Add header checking to transformer and mock repository * Remove trailing zero in test data * Fix minor mishaps * Go fmt nitpicking * Refactoring in tests * Add tests covering checked headers stuff (and fix revealed bugs) * go fmt fixes * Implement batching behaviour of transformer * Small fixes after review * Go fmt	2018-10-12 16:13:13 +02:00
Konstantin Zolotarev	0b281eab47	Added default addresses to config & rename fallback var	2018-10-12 11:11:31 +03:00
Konstantin Zolotarev	245732b7f4	Added configuration for contract addresses	2018-10-11 19:10:53 +03:00
Elizabeth	6c77f369d9	Vat heal (#56 ) * VatHeal Converter * Add VatHeal repository * Add VatHeal transformer * Add VatHeal to continuousLogSync command * Mark vat_init_checked as true when creating vat init records * Update urn and v converting * Return error if Repository.MarkCheckedHeader fails * Add deleting vat heal from test cleanup method	2018-10-10 11:56:06 -05:00
David Terry	0bb3737027	vat.fold: pull common test setup into BeforeEach blocks	2018-10-10 18:13:10 +03:00
David Terry	b1745a6403	Merge branch 'staging' into VDB-101-vat-fold-transformer Conflicts: cmd/continuousLogSync.go db/schema.sql pkg/transformers/shared/constants.go pkg/transformers/transformers.go test_config/test_config.go	2018-10-10 14:09:14 +03:00
David Terry	89ec0a1cab	documentation: document config file for transformers	2018-10-10 14:06:00 +03:00
David Terry	95f1c94bf5	vat.fold: check for errors in insert before updating checked_headers	2018-10-10 14:03:37 +03:00
David Terry	1273acb733	vat.fold: add repository tests for MarkHeaderChecked	2018-10-10 13:51:44 +03:00
David Terry	3778d7ac06	vat.fold: add transformer tests for MarkHeaderChecked	2018-10-10 13:40:25 +03:00
David Terry	7e7ad13de6	vat.fold: handle errors	2018-10-10 13:39:33 +03:00
David Terry	dde57178f1	vat.fold: break out of loop in transformer if no logs found	2018-10-10 13:38:57 +03:00
David Terry	724839ce4a	documentation: document vat event types	2018-10-10 13:37:59 +03:00
Rob Mulholand	e935f2f553	Add Vat grab transformer	2018-10-09 16:48:19 -05:00
Rob Mulholand	61dca314c4	Add Vat tune transformer	2018-10-09 14:37:26 -05:00
Rob Mulholand	8ee106e14d	Add Vat toll transformer	2018-10-09 11:40:35 -05:00
David Terry	ecef1cbcdb	transformers: add step by step docs	2018-10-09 14:25:27 +03:00
David Terry	09770b2e50	Merge branch 'staging' into VDB-101-vat-fold-transformer	2018-10-09 14:15:07 +03:00
David Terry	81dec589fc	vat.fold: rollback broken test refactoring	2018-10-09 14:11:02 +03:00
David Terry	e040383fa5	vat.fold: add test for signature generation	2018-10-09 14:10:42 +03:00
David Terry	3e9901864b	vat.fold: switch to batched interface	2018-10-09 13:34:22 +03:00
Taka Goto	347a339687	update frob and bite converters and repository	2018-10-07 22:02:14 -05:00
Taka Goto	7e6302c662	update bite to take in string for urn and ilk	2018-10-04 16:35:51 -05:00
David Terry	b3e2218eb5	Merge branch 'staging' into VDB-101-vat-fold-transformer	2018-10-04 18:49:38 +03:00
David Terry	53a74c39f7	Vat.fold: correct typo	2018-10-04 17:47:03 +03:00
David Terry	dde57ea70b	Vat.fold: run transformer	2018-10-04 16:04:53 +03:00
David Terry	c8cb079da5	Vat.fold: add transformer & mocks	2018-10-04 14:34:30 +03:00
David Terry	46bba30938	Vat.fold: refactor repository tests	2018-10-03 19:38:47 +02:00
David Terry	ef0e3f9e11	Vat.fold: add repository & tests	2018-10-03 18:22:05 +02:00
David Terry	0c58e0ac5b	Vat.fold: verify that the event signature is correct	2018-10-03 18:21:27 +02:00
David Terry	96f8279421	Vat.fold: update test data so it matches a real deployed kovan event	2018-10-03 16:03:30 +02:00
David Terry	b3295e3a86	Vat.fold: add converter & tests	2018-10-03 14:08:01 +02:00
David Terry	ce4d928fb8	transformers: tweak docs	2018-10-03 14:07:32 +02:00
David Terry	22fb019d3f	docs: add links	2018-10-03 12:50:28 +02:00
David Terry	377c57ce1b	transformers: format documentation	2018-10-03 12:44:05 +02:00
Rob Mulholand	bf44ec614a	Log checked headers for MCD events (#43 ) * Log checked headers for Bite events * Log checked headers for Drip drip events * Log checked headers for Drip file events - mark headers checked for 3 implementations of file on the Drip contract * Log checked headers for Frob frob events * Log checked headers for Pit file events - mark headers checked for 3 implementations of file on the Pit contract * Log checked headers for Vat init events	2018-10-02 17:25:38 -05:00
David Terry	cf437e4007	VDB-101: Add signature for vat.fold	2018-09-28 16:31:29 +02:00
David Terry	f56c882b9d	increase timeout on price feeds integration tests	2018-09-28 14:19:40 +02:00
Rob Mulholand	02853014b2	Fix Pit file converters - Use topics 2 & 3 instead of topic 1 (call data instead of msg.sender) - Prefer extracting fields from topics rather than data	2018-09-27 10:32:08 -05:00
Rob Mulholand	c17ad1f826	Add missing Describe statements	2018-09-27 10:32:08 -05:00
Rob Mulholand	58f17a24ae	Add transformers for Cat file events - transform logs from 3 implementations of file on the Cat contract	2018-09-27 10:32:08 -05:00
Elizabeth	b0cd852aa3	Flop deal (#39 ) Refactor deal transformer to take in several contract addresses Add flop address to deal transformer	2018-09-27 10:12:29 -05:00
Elizabeth	31516ea87e	Flop kick (#28 ) * FlopKick transformer	2018-09-26 09:42:52 -05:00
Rob Mulholand	a0ba6ca6bd	Return error if RPC returns empty header - Enables graceful failure if RPC returns empty data with no error	2018-09-25 14:11:35 -05:00
Rob Mulholand	92525ca575	Make price feed log persistence atomic per block - Prevents marking header as checked if only one of several price feed logs is successfully persisted	2018-09-25 14:11:35 -05:00
Rob Mulholand	ac362650db	Log checked headers for tend events	2018-09-25 14:11:35 -05:00
Rob Mulholand	009cbb8aa0	Log checked headers for flip kick events	2018-09-25 14:11:35 -05:00
Rob Mulholand	d8b1a61ff5	Log checked headers for dent events	2018-09-25 14:11:35 -05:00
Rob Mulholand	c3f1fcf796	Log checked headers for deal events - mark header as checked if no matching logs - mark header as checked if all matching logs persisted successfully - batch convert and insert matching logs to enable transactional persist - enable skipping previously checked block headers when restarting a sync	2018-09-25 14:11:35 -05:00
Elizabeth	cee824f7ad	Add transformer documentation (#32 )	2018-09-24 15:39:00 -05:00
Rob Mulholand	e138e58120	Set timeout before assertions in integration tests - Prevent early evaluation from triggering failure	2018-09-21 11:28:17 -05:00
Rob Mulholand	56c45b6841	Add logging for every transformer - Also add drip file transformer initializer	2018-09-19 11:00:20 -05:00
Rob Mulholand	a735698f77	Update config for Kovan	2018-09-19 11:00:20 -05:00
Rob Mulholand	042f1142c0	Handle headers from POA chain - Fetching headers from Kovan requires custom type without mixHash/Nonce	2018-09-19 11:00:20 -05:00
Rob Mulholand	778517f33f	Add integration tests for price feed transformer - Verify eth/usd, mkr/usd, and rep/usd price feed logs are successfully fetched from Kovan and persisted to Postgres	2018-09-19 11:00:20 -05:00
Elizabeth	cfc8773c5d	Continuous sync (#23 )	2018-09-19 10:14:49 -05:00
Elizabeth	60901d9095	Update tend, flipkick and dent tables for consistency (#24 )	2018-09-18 09:27:14 -05:00
Rob Mulholand	06881db350	Sync unchecked headers for price feeds - Indicate that a header has been checked for logs if no logs returned - Indicate that a header has been checked for logs when persisting a log - Fetch headers that need to be checked based on absence of the above	2018-09-17 14:14:52 -05:00
Rob Mulholand	f5595bee78	Add Drip drip transformer	2018-09-13 16:35:10 -05:00
Rob Mulholand	560305f601	Update dependencies - uses newer version of go-ethereum required for go1.11	2018-09-13 16:14:35 -05:00
Elizabeth	939ead0c82	Deal transformer (#19 )	2018-09-13 09:06:26 -05:00
Rob Mulholand	d69063636e	Add raw log to price feeds	2018-09-12 11:35:32 -05:00
Rob Mulholand	b2ba7ee1e3	Add transaction index to flip kick	2018-09-12 11:35:32 -05:00
Rob Mulholand	a843de5eb7	Add drip file transformers - for all three file functions on the drip contract	2018-09-12 11:35:32 -05:00
Elizabeth	faefd620b1	Get method/event signature from ABI (#17 )	2018-09-12 08:59:43 -05:00
Rob Mulholand	4b6ef1e58b	Verify pit file logs - assure required topics + data available before parsing	2018-09-11 16:30:29 -05:00
Rob Mulholand	dc9bda7d68	Extract addresses to shared constants file	2018-09-11 16:30:29 -05:00
Rob Mulholand	ceeaa0f8d3	Update licenses and imports - Verify license added to each file - Order imports by group and separate with line breaks	2018-09-11 16:30:29 -05:00
Rob Mulholand	ff110592bd	Add Vat init transformer	2018-09-11 16:30:29 -05:00
Elizabeth	9abe3ffa68	Dent (#10 ) Convert and persist LogNote events as dent records	2018-09-11 14:01:45 -05:00
Rob Mulholand	d48cf38070	Add Pit file debt ceiling event	2018-09-05 13:51:18 -05:00
Rob Mulholand	e1f609528d	Add Pit file stability fee event - file event on the pit contract is overloaded and each implementation logs a note - existing Pit file transformer now specified as dealing with ilks	2018-09-05 11:36:13 -05:00
Rob Mulholand	a30e5036fd	generate log event signatures from functions	2018-09-04 16:19:03 -05:00
Rob Mulholand	8f5bc0a956	Add raw log to Frob record in DB - Enables parsing additional values off the log if necessary - Also update Frob to reflect DSS changes (lad => urn) - Move Frob constants to shared directory	2018-09-04 16:11:54 -05:00
Rob Mulholand	db4b891398	Backfill Pit.file log events	2018-09-04 16:07:35 -05:00
Elizabeth	53635ec362	Update flip kick and tend transformers due to contract changes * Update FlipperAddress to new local ganache address * Update flip_kick table * Update flipkick transformer to handle new signature and abi * Update tend table * Update tend converter	2018-09-04 15:50:29 -05:00
Takayuki Goto	807b7accd6	Merge pull request #12 from 8thlight/bite-event bite event transformer	2018-09-04 09:58:51 -05:00
Taka Goto	415ec3d97c	update from PR feedback	2018-08-31 10:37:02 -05:00
Taka Goto	adc7c47244	update tests and import order per PR feedback	2018-08-30 15:06:14 -05:00
Taka Goto	4ca7c10f92	fix compile error	2018-08-30 14:27:28 -05:00
Taka Goto	985fa49178	migration file to create bite; create bite repository add transaction index and raw log to bite table work on converter for bite event update bite repository, replace guy with 32byte lad; create bite converter to entity update field type for bite event; start on bite transformer finish bite event transformer	2018-08-30 08:38:43 -05:00
Rob Mulholand	1916d585fa	Incorporate DSS updates (#8 ) * Incorporate DSS updates - Lad renamed to Guy - Dink and Dart added to Frob * update test chain data * Remove Mom field from FlipKick * Update Flip ABI and sample flip kick data * Incorporate updates to Frob event - Guy renamed to Lad - Era and Gem removed, iArt added - Also turn off integration tests that read from test chain while events are actively under development	2018-08-28 14:16:39 -05:00
Elizabeth	66ad7e3021	Transform and persist Tend log events	2018-08-16 16:36:35 -05:00
Rob Mulholand	634604d0b5	Combine price feed transformers - fetches logs from all three price feeds in one query - assumes eth/usd price feed will be updated to include LogValue event - updates transformers to run separate from header sync	2018-08-16 11:22:16 -05:00
Rob Mulholand	9231d40369	Extract price feed contract addresses to config	2018-08-16 11:22:16 -05:00
Rob Mulholand	72a849a272	Update price feeds if header changed by validator - currently not validating price feeds if the underlying header already exists and is valid, since price feeds should have been added when initial header was added	2018-08-16 11:22:16 -05:00
Rob Mulholand	6e68dc4a92	Add Rep price feed	2018-08-16 11:22:16 -05:00
Rob Mulholand	2949996d22	Add Pip price feed	2018-08-16 11:22:16 -05:00
Rob Mulholand	0551147453	Add Pep price feed	2018-08-16 11:22:16 -05:00
Rob Mulholand	9f7a776d70	update chain data for frob event	2018-08-14 10:47:43 -05:00
Rob Mulholand	ac63e43543	Backfill Frob log events	2018-08-14 10:47:43 -05:00
Rob Mulholand	55229cd2eb	Refactorings to get tests passing with local Ganache setup - removes assertions against data with timestamps to facilitate running the tests against a freshly setup local Ganache instance - also applies a few `go vet` and `go fmt` changes	2018-08-09 16:55:44 -05:00
Elizabeth	b37324bf3d	Refactoring * Move flip kick files pkg/transformers/flip_kick * Consolidate test database setup * Pull ganache ipcPath from config * Update README to include info about using a Ganache chain	2018-08-07 15:17:29 -05:00
Elizabeth	c617cd9c9d	Add FlipKick log events transformer	2018-08-07 10:51:34 -05:00
Rob Mulholand	a683e45855	Consolidate test doubles - Migrate various mocks of core namespaces to shared version in `fakes` pkg - Err on the side of making test doubles less sophisticated - Don't pull over mocks of namespaces that are only used in example code	2018-07-20 11:37:46 -05:00
Rob Mulholand	63434f6bc9	Add tests for pkg/geth/blockchain - inject dependencies instead of initializing them in the constructor	2018-07-18 16:34:13 -05:00
Rob Mulholand	1355271011	Add light sync command - Only syncs block headers (excludes block bodies, transactions, receipts, and logs) - Modifies validation window to include the most recent block - Isolates validation window to the variable defined in the cmd directory (blocks have a separate variable defined in the block_repository for determining when to set a block as final)	2018-07-18 16:34:12 -05:00
Maxim Krasilnikov	e9e3864ba9	Added user and password to db config	2018-06-22 18:28:34 +03:00
Rob Mulholand	3e39ccb9bb	Fix missing block numbers lookup - Previously, a block was treated as missing if there was a number in the given range for which there existing a block that did not match the node's fingerprint. This meant that in a case where we have a block that does not match the node's fingerprint and also one that does match, the block would be treated as missing. This led to errors being thrown when attempting to add a block that already exists. - These changes treat a block as missing only if we do not already have a block that matches the number and node fingerprint.	2018-06-08 11:26:25 -05:00
Rob Mulholand	d5c2ab33fc	Sync only missing blocks on cold import = Add eth_node_fingerprint to block that can be imitated by both hot and cold imports - Only sync missing blocks (blocks that are missing or don't share a fingerprint) on cold import - Set block is_final status after import	2018-05-07 16:38:04 -05:00
Rob Mulholand	5a5e08bd13	Get head block number - Allows us to refuse to sync past current head - Creates the opportunity to add a flag for syncing all blocks	2018-05-04 11:03:50 -05:00
Rob Mulholand	5f6bf32ec1	Add cold import script	2018-05-04 10:54:00 -05:00
Rob Mulholand	462f94d84a	Associate receipts with blocks instead of transactions - Allows us to fetch receipts by block and persist even when associated transaction is unknown. (The associated transaction can still be derived from the tx_hash column at query time, but is an expensive operation to require for inserts).	2018-05-04 10:52:28 -05:00
Matt K	8a9395819c	Get transactions (#45 ) * Make transactions requests in parallel * Update transaction error handling	2018-03-27 16:06:12 -05:00
Matt Krump	88210e436a	Remove test and travis deps on private dev network	2018-03-22 10:15:18 -05:00
Matt Krump	1b4e57a5b4	Rename node table and node column	2018-03-21 13:24:13 -05:00
Matt Krump	a11fb709b2	Update watcher to use blockchain instead of contract data fetcher	2018-03-12 16:19:06 -05:00
Matt Krump	76dad443ec	Update FetchContractData to allow optional args	2018-03-12 14:21:34 -05:00
Matt Krump	5a2bb04670	Remove unused contract methods	2018-03-12 14:21:34 -05:00
Matt K	5a652190d9	Allow Parity as ingest node (#36 ) * Upgrade go-ethereum to v1.8 * Add Node Info for parity nodes * Upgrade start_private_blockchain to use v1.8	2018-03-07 15:29:21 -06:00
Matt Krump	06f78e0083	Handle events - Adds interfaces for developers to build handlers that update data in response to log events - Resolves #29	2018-03-05 10:01:50 -06:00
Matt K	ed907535e3	Separate DB access into several repos (#28 ) * Separate files for InMemory * Start using separate repos for collaborating objects * Before Updating schema * Separate various repos	2018-02-12 10:54:05 -06:00
Matt K	605b0a96ae	Add graphql server (#27 ) * Add graphql server * Update Makefile * Update log_filters constraint * Add GetLogFilter to repo * Update travis (use Makefile, go fmt, go vet) * Add logFilter schema and resolvers * Add GetWatchedEvent to watched_events_repo * Add watchedEventLog schema and resolvers	2018-02-08 10:12:08 -06:00
Matt K	d5852654bb	Update table columns (#26 ) * Update block table names * Update transaction table names	2018-02-02 16:12:14 -06:00
Matt K	aea9c7b5e2	Separate repositories (#25 ) * Separate Repository into multiple Repositories * Use struct scan for transactions * Use struct scan for blocks * Remove unused block repo methods * Update naming * Rename / Cleanup repository related fields	2018-02-02 15:53:16 -06:00
Matt K	3863bcb614	Events (#24 ) * Add watched events repo	2018-02-02 13:58:59 -06:00
Matt Krump	aac2239097	Merge old private repo into vulcanize	2018-01-25 18:08:26 -06:00
Matt K	c00b8a5a98	Add Filters (#133 ) * Add LogFilter struct * Add log_filters table * Add view for events watching * Add cmd line "add_filter" to mimic eventual endpoint * Allow multiple filters in config	2018-01-23 12:43:35 -06:00
Matt K	3f06c7374b	Update transaction value field (#132 ) * Add test for converting big.Int * Use string as internal representation of transaction value	2018-01-16 14:25:33 -06:00
Matt Krump	6583ce72b8	Remove log_uc * Logs now are attached to receipt, so removing block + index unique constraint	2018-01-16 09:44:46 -06:00
Matt Krump	82c39a2c1f	Add fk constraint on logs	2018-01-16 08:58:11 -06:00
Matt Krump	9ee13e715d	Add topics type	2018-01-15 14:49:29 -06:00
Matt Krump	50f00b80c1	Lowercase log address	2018-01-15 14:46:48 -06:00
Matt Krump	431be46005	Add store logs with receipts	2018-01-15 14:46:32 -06:00
Matt K	a9bea4f492	Allow for multiple Geth nodes (#128 )	2018-01-10 15:54:36 -06:00
Matt K	70cfa20c68	Watch contact updates (#127 ) * Downcase all arguments for contact watching * ABI retrieval from test networks	2018-01-08 15:59:47 -06:00
Matt K	14e1fc4213	Remove admin api dependency (#126 )	2018-01-08 14:19:42 -06:00
Matt K	54c4f0c2fe	Add back infura related tests (#123 )	2018-01-08 11:41:01 -06:00
A. F. Dudley	ee4f7c710a	replace '8thlight' with 'vulcanize'	2018-01-06 15:31:53 -05:00
Matt K	6decf0b54b	Remove pubsub and replace w/ polling head of chain (#122 ) * Rename geth package structs to not be prefaced with package name * No longer need to dump schema since Travis uses migrate * Rearrange history package * Removed double request for receipt from block rewards * Remove Listener + Observers and Replace w/ Polling Head * Potential Short term Issue w/ Infura (ignore these tests for now)	2018-01-05 11:55:00 -06:00
Matt K	4fabe3e917	Add receipts (#119 ) * Conversion between Geth Receipt and core.Receipt * Add receipt to DB * Insert receipts with transactions * Update Travis CI to use dep for dependencies	2018-01-03 11:23:43 -06:00
Matt Krump	b6ed4464c7	Update populate blocks to use blockchain rather than db to determine blocks to fill	2018-01-02 13:35:52 -06:00
Matt Krump	351d315a4c	Add tx fields	2017-12-28 17:23:56 -06:00
Matt Krump	8b024bade9	Add block rewards to db	2017-12-28 11:58:26 -06:00
Matt Krump	3ca4370221	Add Block + Uncle Rewards calculation	2017-12-28 09:46:14 -06:00
Matt Krump	cb4e745464	Add extra data field	2017-12-27 12:10:08 -06:00
Matt Krump	c992186846	Add Block Miner	2017-12-27 10:50:56 -06:00
Matt K	7e5e12f488	Backfill/listen for contract logs (#113 )	2017-12-22 11:42:35 -06:00
Matt K	a786241c8c	Add indices for tx_to and tx_from (#112 )	2017-12-20 16:58:37 -06:00
Matt K	24bc83a448	Block categorization (#110 ) * Add block categorization (is_final=) * Add godo task for vulcanizeDB (Example of how everything could work together) * Add unique constraint on block_number and node * Add index on block_id for transactions_table * Add node_id index on blocks table * Sort transactions returned from FindBlock by tx_hash * lowercase tx_to, tx_from like etherscan	2017-12-20 14:06:22 -06:00
Matt K	266c9587c8	Canonical blocks (#108 ) * Update Block w/ newest Block * Add cascading delete to blocks and transactions tables * Add handling for new conflicting blocks * Command line version of sliding window n behind HEAD	2017-12-19 14:14:41 -06:00
Matt Krump	84e77f259d	Use most recent blockNumber when blockNumber parame is nil	2017-12-14 10:15:36 -06:00
Matt K	0e837e2d03	Refactoring (#101 ) * Make naming consistent for watched_contracts * Update FindContract and FindBlockByNumber to return errors rather than nil	2017-12-13 10:51:11 -06:00
Matt K	a68f277066	Save Logs to DB (#100 ) * Save logs to database * Save multiple logs to db * Add block number parameter to `FindLogs`	2017-12-12 15:55:26 -06:00
Matt K	5e64283a12	Get logs for a contract (#99 ) * Add ability to fetch logs for a contract and a block * Test contract related code against Infura, so can run on Travis * Add godo task for getLogs	2017-12-11 15:08:00 -06:00
Matt K	921bde1089	Update BlockChain to record NodeInfo (#95 )	2017-12-07 13:32:16 -06:00
Matt K	18163f970e	Get ABI via etherscan API (#96 ) - Added ABI request - Add unique constraint on contract hash for watched contracts	2017-12-07 09:58:06 -06:00
Eric Meyer	0439791381	Rename WatchedContract to Contract	2017-12-05 09:39:58 -06:00
Eric Meyer	3a2e7e0cc1	Update Blockchain interface to use WatchedContract instead of Contract	2017-12-04 17:04:06 -06:00
Eric Meyer	e432219e20	Rename watched_contracts package to contract_summary	2017-12-04 17:04:06 -06:00

... 7 8 9 10 11 ...

888 Commits