7 Commits

Author	SHA1	Message	Date
Rob Mulholand	36c4da372b	(VDB-570) Handle duplicate storage diffs ... - If processing a new diff for a row that already exists in the DB, ignore the error without logging or queueing - If processing a queued diff for a row that already exists, remove it from the queue	2019-05-02 11:39:16 -05:00
Rob Mulholand	b036053937	Queue storage diffs if transformer execution fails ... - For any error, not just if key isn't recognized - Means we don't lose track of diffs on random ephemeral errors	2019-05-01 12:32:39 -05:00
Rob Mulholand	6a86de87b4	(VDB-371) Recheck queued storage ... - Iterate through queued storage at defined interval, popping rows from the queue if successfully persisted	2019-05-01 12:30:37 -05:00
Rob Mulholand	2d684c5aec	Extract storage diff fetching behind an interface ... - Replaces directly reading from a CSV - Simplifies testing - Should hopefully make it easier to plug in other sources for storage diffs (e.g. differently formatted CSVs, JSON RPC, etc)	2019-05-01 12:30:36 -05:00
Ian Norden	9f8c50e3ab	tests for migration path ordering and errors	2019-03-13 11:42:30 -05:00
Ian Norden	90e67d6da9	move storage mappings pkg into shared	2019-02-24 21:38:48 -06:00
Ian Norden	decc2a3caf	remove pkg/transformers; extract shared files needed to ... libraries/shared; work on automated db migration management	2019-02-24 21:38:47 -06:00