Commit Graph

223 Commits

Author SHA1 Message Date
Ian Norden
7ef132c400 new db schema 2019-12-03 14:55:53 -06:00
Rob Mulholand
56ce8bdb41 (VDB-950) Write raw diffs before transforming
- Raw field we can reference by FK for related data
- Enables replay for unwatched or mistransformed diffs
2019-12-03 14:51:17 -06:00
Rob Mulholand
6f837f5e88 Add indexes to columns used in plugin joins
- Will probably want to update these depending on what's commonly used
  in join/where statements.
2019-12-02 23:54:52 -06:00
Ian Norden
4036d9d6a0 review fixes 2019-12-02 13:49:27 -06:00
Ian Norden
c16ac026db logWithCommand; rebase fixes; config for testing super node subscription 2019-12-02 13:24:58 -06:00
Ian Norden
e912fa6c87 new db schema 2019-12-02 13:24:58 -06:00
Ian Norden
67df8dea77 header_cids.final => header_cids.uncle 2019-12-02 13:24:58 -06:00
Ian Norden
b789ab69e6 adjustments after rebase 2019-12-02 13:24:58 -06:00
Ian Norden
8b31d12716 fixes after rebase 2019-12-02 13:24:58 -06:00
Ian Norden
0bbb7a30d1 review fixes 2019-12-02 13:24:58 -06:00
Ian Norden
3fa33fb767 index receipts by the contract address 2019-12-02 13:24:58 -06:00
Ian Norden
1235ec1f97 postgres table to hold ipfs data in same database as vulcanizedb data;
shared SeedNodeTransformer interface
2019-12-02 13:24:54 -06:00
Ian Norden
4c81ca4d54 rename blocks table to eth_blocks so that we don't clash with the ipfs blocks table; dockerfile and startup_script for the seed node 2019-12-02 13:24:51 -06:00
Ian Norden
4baea2923c make db fks deferrable so that we can commit entire cid payload in single transaction; adjust buffer sizes to optimize performane and stability 2019-12-02 13:24:51 -06:00
Ian Norden
1dc46640cf request handler and response screener for seed node api service 2019-12-02 13:24:51 -06:00
Ian Norden
671a324294 update to work with updated state diffing code 2019-12-02 13:24:50 -06:00
Ian Norden
15e044403d work on mocks and unit tests 2019-12-02 13:24:50 -06:00
Ian Norden
e033eabc28 cid database table migrations 2019-12-02 13:24:46 -06:00
Gabe Laughlin
2dc8ace69b
(VDB-925) Add hashed address to address table 2019-10-30 11:56:25 -05:00
yaoandrew
deb64b8701 Rebase with staging 2019-10-28 23:30:52 -07:00
Rob Mulholand
e252229b8a Add constraint to prevent duplicate headers
- Disallow inserts of headers with the same number, hash, and node
  fingerprint, since it will enable duplicate log fetching for the
  same header
2019-10-28 14:57:13 -05:00
Rob Mulholand
3f9b034c4c Rename checked_logs => watched_logs
- We're logging that a given log has been included in any fetch calls
  for checked headers, rather than that we have already checked for
  that log
2019-09-10 21:22:14 -05:00
Rob Mulholand
3d6c973f6e Remove extraneous db constraint 2019-09-10 13:58:58 -05:00
Rob Mulholand
b9f3b9f946 Reference header sync logs address via foreign key 2019-08-28 10:41:34 -05:00
Rob Mulholand
5ac76eee74 Make check_count a column on public.headers
- Don't need to maintain it on public.checked_headers if we're not
  adding additional columns to that table
2019-08-28 09:50:17 -05:00
Rob Mulholand
222252f89a Remove extraneous migration formatting 2019-08-28 09:26:13 -05:00
Rob Mulholand
666ea1c325 Update checked headers for new transformers
- If a header was marked as checked before a transformer was added to
  the watcher, mark all headers since the new transformer's starting
  block number as unchecked.
2019-08-28 09:25:14 -05:00
Rob Mulholand
d496dad33c Decouple log extraction from transformer delegation
- limit missing headers results set to 100 so that extraction doesn't
  excessively block delegation
- wrap checked headers functions in repository struct
- move storage repository to factory, to correspond with event
  repository path
- remove unused files
- reformat sql
- remove line breaks in imports
2019-08-28 09:25:13 -05:00
Rob Mulholand
cb819fa9a6 Write event logs to database before transforming
- enables decoupling event extraction/persistence from transformation
- modifies event transformer, converter, and log chunker to accept
  payload that includes internal log database ID with log data
- remove alias for transformer pkg as shared_t
- remove unused mock watcher repository
2019-08-28 09:13:44 -05:00
Rob Mulholand
3693ed905f Rename logs to full sync logs
- Enable creating new table for logs used in event watching based on
  header sync
2019-08-28 09:11:06 -05:00
Elizabeth Engelman
4e40e892d2 Updating header_sync_receipts to have FK reference to addresses 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
bcd6d14fcd Updating full_sync_receipts to have FK reference to addresses 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
3005a1ad50 Move add address migration table up in the migration order 2019-08-23 10:11:37 -05:00
Elizabeth Engelman
750754caa7 Add an addresses table 2019-08-23 10:11:37 -05:00
Rob Mulholand
e11f2c8c59 Don't duplicate queued storage diffs
- currently, if we don't recognize the same diff several times (e.g.
  if you restart the storage diff watcher pointed at the same file),
  we'll add the same row to the queue on each run.
- these changes assure we only queue an unrecognized diff once.
2019-06-14 11:26:15 -05:00
Gabe Laughlin
b4cdf76905
(VDB-479) Upgrade Posgres to 11 2019-05-14 16:49:05 -05:00
Gabe Laughlin
a344432156
(VDB-560) Rename lightSync to headerSync 2019-05-01 12:12:55 -05:00
Rob Mulholand
fb753a1468 Update schema to remove uncles 2019-04-08 15:16:43 -05:00
Ian Norden
9befc76fc6 remove LIMIT 100; method to check continuity of headers returned but doesn't require
the set to start at a specific block number; config for
account_transformer; review fixes; update schema
2019-04-05 17:21:59 -05:00
Ian Norden
c1940c3e58 full_sync and light_sync receipt tables and light receipt repository methods and test 2019-04-05 16:36:48 -05:00
Rob Mulholand
fdd870563e Update db schema for latest changes 2019-04-05 14:51:51 -05:00
Ian Norden
5dcf534b2c review fixes 2019-04-04 15:21:39 -05:00
Ian Norden
197f98c93d going ahead and indexing the entire uncle blocks (one of the issues open on public); finish tests 2019-04-02 14:32:30 -05:00
Ian Norden
185f4c0e93 adjust block/uncle reward tests and add methods to pkg/geth blockchain and ethclient for direct fetching of eth balances 2019-04-02 10:10:17 -05:00
Ian Norden
9030cff2bd keep track of who is receiving uncle rewards 2019-04-02 10:10:17 -05:00
Rob Mulholand
54d46638a8 updates for light sync transactions 2019-03-28 14:31:17 -05:00
Rob Mulholand
79e011aad2 Add light sync transactions table
- Foreign key to header instead of block
- Can use same Transaction struct for both
2019-03-28 14:31:17 -05:00
Rob Mulholand
d93006321b Rename transactions to full_sync_transactions
- Table has foreign key to blocks
- Add transaction RLP and transaction index to table
- Enables other tables with standalone transactions or transactions
  associated with other data structures (e.g. headers)
2019-03-28 14:31:17 -05:00
Rob Mulholand
d9e2bece27 Consolidate migrations
- Facilitate modifying migrations without cascading consequences for
  later migrations updating the same tables
2019-03-23 10:57:39 -05:00
Rob Mulholand
5eff2618ed Assorted cleanups
- Remove DataDog
- Remove token_supply table
- Drop from more tables when cleaning DB for tests
2019-03-08 11:35:10 -06:00