Ian Norden
|
00031e2b83
|
resync reset validation level
|
2020-04-04 15:45:54 -05:00 |
|
Ian Norden
|
060e3430c1
|
times_validated field in header
|
2020-04-04 15:45:54 -05:00 |
|
Ian Norden
|
aa9f78a028
|
new fields in eth.receipt_cids to index contract addresses seen in
logs; handle null dst/contract addrs in trxs/rcts
|
2020-04-04 15:45:54 -05:00 |
|
Ian Norden
|
657abbb7ec
|
vacuum analyze public.blocks after cleanup
|
2020-03-26 04:25:00 -05:00 |
|
Ian Norden
|
834e2bcff4
|
additional logging to resync service; rename state/storage key to leaf key and allow insert of null values for intermediate nodes
|
2020-03-25 19:16:26 -05:00 |
|
Ian Norden
|
e570a2c9d4
|
VACUUM ANALYZE after cleaning out old data
|
2020-03-25 19:16:26 -05:00 |
|
Ian Norden
|
716cc3663f
|
don't throw error on duplicate key insert
|
2020-03-25 19:16:26 -05:00 |
|
Ian Norden
|
1aded75645
|
additional log.info for data streamed at head
|
2020-03-22 09:45:49 -05:00 |
|
Ian Norden
|
f81c440418
|
for some reason every now and then a branch node would throw a panic here; still need to figure out why; quick fix
|
2020-03-22 09:05:26 -05:00 |
|
Ian Norden
|
b2d5345c02
|
add new fields to test data to get tests passing again
|
2020-03-21 21:10:28 -05:00 |
|
Ian Norden
|
ba5d6f9b9f
|
state account support and add roots and additional metadata to header
index
|
2020-03-21 20:57:02 -05:00 |
|
Ian Norden
|
57bdcca43c
|
combine resync and supernode configs; load config params from env variables if available; allow config params to be passed in as cli flags
|
2020-03-20 13:15:50 -05:00 |
|
Ian Norden
|
c72dc273ba
|
misc fixes; config files for resyncing
|
2020-03-17 19:42:53 -05:00 |
|
Ian Norden
|
d47ba24373
|
adjust btc and eth publishers and publisher tests
|
2020-03-17 13:05:19 -05:00 |
|
Ian Norden
|
1e2b09045a
|
fix state and storage iplds/dag_putters; add support for eth rct trie and eth and btc tx tries
|
2020-03-17 11:18:20 -05:00 |
|
Ian Norden
|
e7225317d4
|
split btc and eth super node processes into serpate containers; dockerfile for Postgraphile with plugins
|
2020-03-17 08:37:00 -05:00 |
|
Ian Norden
|
36b714b6df
|
resync config initializer
|
2020-03-17 07:51:00 -05:00 |
|
Ian Norden
|
b934f6c963
|
btc and eth unit tests for db cleaning
|
2020-03-17 07:50:38 -05:00 |
|
Ian Norden
|
f814b64543
|
bug fix
|
2020-03-12 13:59:23 -05:00 |
|
Ian Norden
|
77b1c25bc1
|
resync service
|
2020-03-12 13:52:23 -05:00 |
|
Ian Norden
|
b4005eff39
|
db cleaners to eth and btc
|
2020-03-12 13:51:58 -05:00 |
|
Ian Norden
|
e5c5422edc
|
adhjustments to work with statediffing geth v1.10-alpha.2
|
2020-03-11 13:41:59 -05:00 |
|
Ian Norden
|
be875c0100
|
db comments to resolve postgraphile type conflicts
|
2020-03-10 09:38:07 -05:00 |
|
Ian Norden
|
aad318c67b
|
improved logging in super node service
|
2020-03-08 11:47:49 -05:00 |
|
Ian Norden
|
8c2a71b16f
|
finish wasm watcher engine
|
2020-02-27 15:09:20 -06:00 |
|
Ian Norden
|
330a083749
|
watcher configuration; super node versioning
|
2020-02-25 16:38:46 -06:00 |
|
Ian Norden
|
25aa4634e9
|
work on wasm watchers
|
2020-02-24 12:54:10 -06:00 |
|
Ian Norden
|
e3e8700d34
|
api test; return full ipld models to subscribers
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
fb360d8562
|
changes to super node to improve compatibility with watcher
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
f0c5ff8077
|
begin wasm watcher engine
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
94aefafd7c
|
more explicity payload err and msg
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
2fbf97da9d
|
wasm instantiator
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
5f85a7ecf6
|
more descriptive comments/method names for super node service; send msg
when backfill is finished
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
48fb5bcd27
|
separate super node instances for different chains need to be within the same process to avoid contention over ipfs repo lock
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
ef3b043f97
|
emulate btc data streamer over http; misc fixes
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
5173edf563
|
fix broken go-ipld-eth trie node dag putters
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
ca273a026d
|
major refactor part 2: remove cold import, full sync, generalize node table
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
0f765df12c
|
finish constructors
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
5d46e02509
|
tx inputs directly reference tx ouputs
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
642e08a04b
|
btc converter, publisher, and indexer unit tests
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
8643a7f3b6
|
btc ipld fetcher, paylaod fetcher, and ipld resolver
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
aa2f8bd0a5
|
btc retriever and filterer
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
f33cc3f34b
|
TxOutputs: extract and index pkscript metadata (script type, addresses, #required sigs); TxInputs: outpoint_hash => outpoint_tx_id that references transaction_cids.id
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
48f70d4ddf
|
btc mocks; reorg
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
808f1b5662
|
btc cid indexer
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
5094b975fc
|
postgres schema and tables for btc
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
9018e551ba
|
move eth tables under eth schema
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
4dde90447e
|
wrap generic top-level interfaces with loose types
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
68d8e68144
|
btc ipld publisher
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
8099f726c3
|
internalized ipld types and dag_putters; adjust converters/publishers to work with them
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
2e81f8d31a
|
btc payload converter
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
076903b174
|
btc payload streamer
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
da844b0b83
|
reorg pkg/ to prepare to support chains other than ethereumm
|
2020-02-20 16:14:16 -06:00 |
|
Ian Norden
|
33ac5978f5
|
split uncles into their own table; add tx index to the tx index (lol); continue work on geth api endpoints
|
2020-01-27 14:14:03 -06:00 |
|
Ian Norden
|
e52284b9ba
|
update documentation, bug/error fixes, work on eth api for drawbridge
|
2020-01-24 15:37:52 -06:00 |
|
Ian Norden
|
7843312815
|
major refactoring of super_node to make it easier to support other chains
|
2020-01-24 15:37:52 -06:00 |
|
Ian Norden
|
5ff05225c2
|
begin eth api recapitulation
|
2020-01-24 15:37:52 -06:00 |
|
Ian Norden
|
0785507a7d
|
support for total difficulty (needed to support some eth endpoints)
|
2020-01-24 15:37:52 -06:00 |
|
Ian Norden
|
358575335b
|
expand retriever, fetcher, and resolver interfaces in prep for eth api mimick
|
2020-01-24 15:37:52 -06:00 |
|
Rob Mulholand
|
56ce8bdb41
|
(VDB-950) Write raw diffs before transforming
- Raw field we can reference by FK for related data
- Enables replay for unwatched or mistransformed diffs
|
2019-12-03 14:51:17 -06:00 |
|
Ian Norden
|
74a90fd124
|
use Debugf instead of strconv
|
2019-12-02 23:37:01 -06:00 |
|
yaoandrew
|
5a1f599a4d
|
Clean up files after rebase
|
2019-12-02 23:36:00 -06:00 |
|
yaoandrew
|
614ea3bba1
|
Adds logging for storage and RPC
|
2019-12-02 23:10:32 -06:00 |
|
yaoandrew
|
0b9f29c300
|
Removes import alias for logrus
|
2019-12-02 23:08:48 -06:00 |
|
Ian Norden
|
4036d9d6a0
|
review fixes
|
2019-12-02 13:49:27 -06:00 |
|
Ian Norden
|
5be205ffa6
|
super node backfill breaks batch call into smaller bins; retrieve gap
test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
c16ac026db
|
logWithCommand; rebase fixes; config for testing super node subscription
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
67df8dea77
|
header_cids.final => header_cids.uncle
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
3a666df294
|
backfiller refactoring; explicity errs; golint
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
40c3aff597
|
seed => super; port 80 => port 8080; backfill process for the super_node
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
83fd76bc8a
|
fix flaky test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
b789ab69e6
|
adjustments after rebase
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
8fe273fb7b
|
getter rpc method for the seed node's geth info
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
2c997921cb
|
update dockerfile and documentation
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
8b31d12716
|
fixes after rebase
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
d2db5d0ac9
|
filterer unit test; that should be the last of them
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
03dbe5dd22
|
resolver unit test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
825b0762cd
|
retriever unit test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
9ebc785d98
|
unit test for repository/indexer
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
35c8f3561a
|
publisher unit test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
d79cc90cb2
|
unit test for ipfs fetcher
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
069646905e
|
converter unit test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
89faa85871
|
fix transaction signing, also removes need for rpc call to do it
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
e72995605d
|
possible fix for travis error
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
0bbb7a30d1
|
review fixes
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
2244d1869f
|
process receipts from payload instead of retreiving over additional rpc call
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
13d09e15c1
|
update deps
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
905585b212
|
backfill concurrently, one block at a time, while normal streaming continues
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
f2efbb5d01
|
worker pool for handling concurrent ipld putting and cid indexing without overloading Postgres connections
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
3fa33fb767
|
index receipts by the contract address
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
23a21c14f3
|
make subscription config rlp encodable, group subs of the
same type using a hash of their config, process only once for each sub
type instead of for every sub.
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
1235ec1f97
|
postgres table to hold ipfs data in same database as vulcanizedb data;
shared SeedNodeTransformer interface
|
2019-12-02 13:24:54 -06:00 |
|
Ian Norden
|
5356cd50bb
|
seed node documentation
|
2019-12-02 13:24:51 -06:00 |
|
Ian Norden
|
4c81ca4d54
|
rename blocks table to eth_blocks so that we don't clash with the ipfs blocks table; dockerfile and startup_script for the seed node
|
2019-12-02 13:24:51 -06:00 |
|
Ian Norden
|
e7cdd6247e
|
update to use ReceiptForStorage; expose rpc server over ws
|
2019-12-02 13:24:51 -06:00 |
|
Ian Norden
|
723c7c6244
|
fix backfill operations and dependency issue; hopefully travis will work now
|
2019-12-02 13:24:51 -06:00 |
|
Ian Norden
|
4baea2923c
|
make db fks deferrable so that we can commit entire cid payload in single transaction; adjust buffer sizes to optimize performane and stability
|
2019-12-02 13:24:51 -06:00 |
|
Ian Norden
|
b1bb646ad5
|
goimports + streamSubscribe command for raw access to the seed node data
|
2019-12-02 13:24:51 -06:00 |
|
Ian Norden
|
8ccdfd4835
|
fix streamFilters issue
|
2019-12-02 13:24:51 -06:00 |
|
Ian Norden
|
1dc46640cf
|
request handler and response screener for seed node api service
|
2019-12-02 13:24:51 -06:00 |
|