Rob Mulholand
56ce8bdb41
(VDB-950) Write raw diffs before transforming
...
- Raw field we can reference by FK for related data
- Enables replay for unwatched or mistransformed diffs
2019-12-03 14:51:17 -06:00
Ian Norden
8562abd180
split backfill range up into smaller bins and process them concurrently; improve tests; review fixes
2019-12-02 11:20:49 -06:00
Ian Norden
37f4a2d603
integrate backfill into storage watcher; documentation for storage backfill
2019-12-02 11:06:28 -06:00
Elizabeth Engelman
045d78be25
Rename StorageDiffRow -> StorageDiff
2019-09-25 16:32:31 -05:00
Rob Mulholand
76ab914bdc
Add license
2019-05-01 12:32:39 -05:00
Rob Mulholand
6a86de87b4
(VDB-371) Recheck queued storage
...
- Iterate through queued storage at defined interval, popping rows
from the queue if successfully persisted
2019-05-01 12:30:37 -05:00
Rob Mulholand
2d684c5aec
Extract storage diff fetching behind an interface
...
- Replaces directly reading from a CSV
- Simplifies testing
- Should hopefully make it easier to plug in other sources for storage
diffs (e.g. differently formatted CSVs, JSON RPC, etc)
2019-05-01 12:30:36 -05:00
Rob Mulholand
Ian Norden
Elizabeth Engelman