Ian Norden
|
cffceb53db
|
optimize GetTransactionCount, GetBalance, and GetCode to use secondary indexes instead of operating through ethdb where we have to iterate down trie from root to leaf (multiple db lookups) to access account info
|
2020-10-29 22:07:39 -05:00 |
|
Ian Norden
|
dc25ea7f87
|
new transaction endpoint unit tests (except for GetTransactionCount which is still TODO because testing its current implementation requires complete state (e.g. like eth_call test) so- ideally- will first refactor it to use state diff 2ndary indexes rather than proceeding through regular ethdb interface)
|
2020-10-29 15:20:58 -05:00 |
|
Ian Norden
|
e1026d5261
|
remaining block endpoint unit test; uncle endpoints unit tests
|
2020-10-29 14:59:09 -05:00 |
|
Ian Norden
|
1d4abcb69b
|
finish cache miss forwarding for new endpoints
|
2020-10-28 08:54:06 -05:00 |
|
Ian Norden
|
b5d57b6afc
|
the remaining, currently supportable, eth_* endpoints
|
2020-10-28 08:54:06 -05:00 |
|
Ian Norden
|
7a2ccaa8a7
|
fixes for test; update readme and gomodules
|
2020-10-28 08:40:00 -05:00 |
|
Ian Norden
|
16aa9652a5
|
integrate into serve command
|
2020-10-28 08:23:17 -05:00 |
|
Ian Norden
|
a8dd77294a
|
graphql service on top of rpc endpoints
|
2020-10-28 08:23:17 -05:00 |
|
Ian Norden
|
4f4ab1dd4f
|
update eth backend
|
2020-10-28 08:23:17 -05:00 |
|
Ian Norden
|
e561fd3178
|
minor cleanup to prom files
|
2020-10-27 14:18:59 -05:00 |
|
Ian Norden
|
2d0367fe6c
|
update env
|
2020-10-27 12:57:26 -05:00 |
|
Ian Norden
|
e1cab4fadc
|
forward cache misses (err and/or empty results) to remote node
|
2020-10-27 12:29:28 -05:00 |
|
ramil
|
909c85b547
|
monitoring documentation, prometheus config, grafana dashboard
|
2020-10-21 15:38:48 +03:00 |
|
ramil
|
c5eae5fb75
|
Merge remote-tracking branch 'remotes/vulc/master' into metrics
# Conflicts:
# go.sum
# pkg/serve/config.go
|
2020-10-21 13:53:38 +03:00 |
|
Ian Norden
|
20af343efb
|
fixes after dirty rebase; canonical hash/header finder function is in this commit now
|
2020-10-20 15:36:51 -05:00 |
|
Ian Norden
|
5b177e716c
|
test chain maker for eth_call testing
|
2020-10-20 15:29:00 -05:00 |
|
Ian Norden
|
33a0c8e0e7
|
begin eth_Call backend integration
|
2020-10-20 15:28:57 -05:00 |
|
Ian Norden
|
71bc3f8e7b
|
ipld retriever
|
2020-10-20 15:28:57 -05:00 |
|
ramil
|
bf46200c07
|
revert local changes for testing
|
2020-10-20 18:54:06 +03:00 |
|
ramil
|
797450dd2a
|
fix tests
|
2020-10-20 18:48:35 +03:00 |
|
ramil
|
6e127acbf3
|
upgrade ipld-eth-indexer from v0.2.0-alpha to v0.5.0-alpha and test fixes
|
2020-10-20 17:42:09 +03:00 |
|
Ilnur Galiev
|
4c18554fbe
|
add websocket and unixsocket counters
|
2020-10-19 23:00:09 +03:00 |
|
Ilnur Galiev
|
f627c2edfa
|
add prometheus-middlewares for http and ws endpoint
|
2020-10-19 18:00:55 +03:00 |
|
Ilnur Galiev
|
a0d70b544b
|
add prometheus metrics
|
2020-10-19 16:07:29 +03:00 |
|
ramil
|
1d57f75dcf
|
docker compose for dapptools, postgraphile, eth-indexer and eth-server
|
2020-09-06 12:32:11 +03:00 |
|
Ian Norden
|
add6afadd7
|
fix subscribe command and client (remove rlp encoding of subscription params)
|
2020-09-02 14:13:51 -05:00 |
|
Ian Norden
|
dbcb2c0cf7
|
updat dep and serve cmd
|
2020-09-02 10:19:25 -05:00 |
|
Ian Norden
|
0d28234804
|
continue refactor/purge
|
2020-09-01 16:15:31 -05:00 |
|
Ian Norden
|
d645f52e87
|
watch => serve
|
2020-08-31 10:59:41 -05:00 |
|
Ian Norden
|
5830df44a4
|
decouple from sync
|
2020-08-31 10:47:06 -05:00 |
|
Ian Norden
|
558599dd32
|
remove btc stuff
|
2020-08-31 10:42:01 -05:00 |
|
Ian Norden
|
611fc2c11e
|
eth json rpc optimizations; begin ipfs-ethdb integration
|
2020-08-14 13:14:19 -05:00 |
|
Ian Norden
|
a532e17e21
|
remove option for publishing through ipfs node interface
|
2020-08-12 09:26:18 -05:00 |
|
Ian Norden
|
39354b2114
|
write code to pg-ipfs table, for codehash lookups
|
2020-08-12 09:26:18 -05:00 |
|
Ian Norden
|
52b5c99760
|
fix support for eth testnet chains (use correct tx signer)
|
2020-08-12 09:26:18 -05:00 |
|
Ian Norden
|
7b0b613b92
|
misc adjustments
|
2020-08-07 23:17:16 -05:00 |
|
Ian Norden
|
74752d15aa
|
remove diff unique connstraint, overwrite as diff
|
2020-08-07 23:17:11 -05:00 |
|
Ian Norden
|
2b8e5ad05a
|
fix flaky test
|
2020-08-05 11:55:36 -05:00 |
|
Ian Norden
|
b9fbd2da53
|
update docs
|
2020-08-05 00:23:06 -05:00 |
|
Ian Norden
|
77b7bcc94c
|
adjust everything to work with mh fks
|
2020-08-04 22:35:05 -05:00 |
|
Ian Norden
|
0ab55ef9d8
|
distinguish between differential state/storage nodes and eventual ones
|
2020-07-01 13:44:04 -05:00 |
|
Ian Norden
|
e2bcc06f8a
|
major refactor pt 3
|
2020-06-29 19:16:52 -05:00 |
|
Ian Norden
|
449d23757e
|
pkg rename
|
2020-06-29 16:56:25 -05:00 |
|
Ian Norden
|
3dc1bf6955
|
update ipfs stack, specifically dshelp.CidToDsKey => dshelp.MultihashToDsKey
|
2020-06-22 13:15:29 -05:00 |
|
Ian Norden
|
9f3e4f1102
|
finish updating documentation after refactor
|
2020-06-20 10:59:32 -05:00 |
|
Ian Norden
|
fe083c12f9
|
trimming down to ipfs watchers
|
2020-05-29 22:02:47 -05:00 |
|
Ian Norden
|
421987ff72
|
update version in go.mod
|
2020-05-22 19:22:52 -05:00 |
|
Ian Norden
|
0101c4791a
|
pair with new statediffing geth version; travis tests will fail til release is up
|
2020-05-21 15:34:43 -05:00 |
|
Ian Norden
|
5fb1cc0696
|
refactor- focus on simplifying concurrent fetching; graceful shutdown for superNode command
|
2020-05-12 15:34:25 -05:00 |
|
Ian Norden
|
3320df264f
|
avoid decode error on storage branch nodes when element won't decode into byte array
|
2020-05-12 09:32:33 -05:00 |
|
Ian Norden
|
e2ccd3ffdb
|
update configs
|
2020-05-11 20:50:19 -05:00 |
|
Ian Norden
|
8dc31e4ca5
|
make connection pool params configurable
|
2020-05-11 10:50:05 -05:00 |
|
Ian Norden
|
fb6fdfcc13
|
separate db conneciton pools for each process
|
2020-05-06 16:16:56 -05:00 |
|
Ian Norden
|
dfb66eb67b
|
update version; minor refactoring
|
2020-05-04 12:14:57 -05:00 |
|
Ian Norden
|
9b0ded692b
|
clean up db transaction rollback/commit handling
|
2020-05-01 17:00:00 -05:00 |
|
Ian Norden
|
1541c62ddc
|
eth api tests needed to be updated too
|
2020-05-01 11:12:01 -05:00 |
|
Ian Norden
|
ff5472ccf5
|
direct fetching of iplds from pg-ipfs for btc and eth
|
2020-04-30 17:22:22 -05:00 |
|
Ian Norden
|
059664ffc9
|
combined/directp publish and indexing for btc; tests
|
2020-04-30 17:22:22 -05:00 |
|
Ian Norden
|
65da8cafe7
|
update higher level interfaces and constructs
|
2020-04-30 17:22:22 -05:00 |
|
Ian Norden
|
475ca3e6d1
|
publish and index test for eth
|
2020-04-30 17:22:22 -05:00 |
|
Ian Norden
|
680a86aee2
|
combine publishing and indexing for eth; write directly to pg and avoid need for ipfs node
|
2020-04-30 17:22:22 -05:00 |
|
Ian Norden
|
d345e6b954
|
reduce chan buffer size, make circular buffer
|
2020-04-30 17:22:22 -05:00 |
|
Ian Norden
|
9db0fb219a
|
ipfs mode enum; configuration
|
2020-04-30 17:22:18 -05:00 |
|
Ian Norden
|
ef77688145
|
additional logging at initialization
|
2020-04-28 22:08:46 -05:00 |
|
Ian Norden
|
43c254b5f6
|
fixes after tests
|
2020-04-20 11:16:08 -05:00 |
|
Ian Norden
|
eceaa0aecb
|
use configurable timeout for geth batch http requests; additional error log info in payload fetchers
|
2020-04-20 11:05:12 -05:00 |
|
Ian Norden
|
35e625695e
|
updates to documentation for super node
|
2020-04-19 18:28:36 -05:00 |
|
Ian Norden
|
79c3078c36
|
version command; log resync and super node configs; fix logfile cli flag
|
2020-04-14 12:36:07 -05:00 |
|
Ian Norden
|
f2e853e9c9
|
add back raw contract address
|
2020-04-04 15:45:55 -05:00 |
|
Ian Norden
|
8960cde4f7
|
backfill based on validation level
|
2020-04-04 15:45:54 -05:00 |
|
Ian Norden
|
649fd54a9f
|
store receipt contract hash instead of contract
|
2020-04-04 15:45:54 -05:00 |
|
Ian Norden
|
00031e2b83
|
resync reset validation level
|
2020-04-04 15:45:54 -05:00 |
|
Ian Norden
|
060e3430c1
|
times_validated field in header
|
2020-04-04 15:45:54 -05:00 |
|
Ian Norden
|
aa9f78a028
|
new fields in eth.receipt_cids to index contract addresses seen in
logs; handle null dst/contract addrs in trxs/rcts
|
2020-04-04 15:45:54 -05:00 |
|
Ian Norden
|
657abbb7ec
|
vacuum analyze public.blocks after cleanup
|
2020-03-26 04:25:00 -05:00 |
|
Ian Norden
|
834e2bcff4
|
additional logging to resync service; rename state/storage key to leaf key and allow insert of null values for intermediate nodes
|
2020-03-25 19:16:26 -05:00 |
|
Ian Norden
|
e570a2c9d4
|
VACUUM ANALYZE after cleaning out old data
|
2020-03-25 19:16:26 -05:00 |
|
Ian Norden
|
716cc3663f
|
don't throw error on duplicate key insert
|
2020-03-25 19:16:26 -05:00 |
|
Ian Norden
|
1aded75645
|
additional log.info for data streamed at head
|
2020-03-22 09:45:49 -05:00 |
|
Ian Norden
|
f81c440418
|
for some reason every now and then a branch node would throw a panic here; still need to figure out why; quick fix
|
2020-03-22 09:05:26 -05:00 |
|
Ian Norden
|
b2d5345c02
|
add new fields to test data to get tests passing again
|
2020-03-21 21:10:28 -05:00 |
|
Ian Norden
|
ba5d6f9b9f
|
state account support and add roots and additional metadata to header
index
|
2020-03-21 20:57:02 -05:00 |
|
Ian Norden
|
57bdcca43c
|
combine resync and supernode configs; load config params from env variables if available; allow config params to be passed in as cli flags
|
2020-03-20 13:15:50 -05:00 |
|
Ian Norden
|
c72dc273ba
|
misc fixes; config files for resyncing
|
2020-03-17 19:42:53 -05:00 |
|
Ian Norden
|
d47ba24373
|
adjust btc and eth publishers and publisher tests
|
2020-03-17 13:05:19 -05:00 |
|
Ian Norden
|
1e2b09045a
|
fix state and storage iplds/dag_putters; add support for eth rct trie and eth and btc tx tries
|
2020-03-17 11:18:20 -05:00 |
|
Ian Norden
|
e7225317d4
|
split btc and eth super node processes into serpate containers; dockerfile for Postgraphile with plugins
|
2020-03-17 08:37:00 -05:00 |
|
Ian Norden
|
36b714b6df
|
resync config initializer
|
2020-03-17 07:51:00 -05:00 |
|
Ian Norden
|
b934f6c963
|
btc and eth unit tests for db cleaning
|
2020-03-17 07:50:38 -05:00 |
|
Ian Norden
|
f814b64543
|
bug fix
|
2020-03-12 13:59:23 -05:00 |
|
Ian Norden
|
77b1c25bc1
|
resync service
|
2020-03-12 13:52:23 -05:00 |
|
Ian Norden
|
b4005eff39
|
db cleaners to eth and btc
|
2020-03-12 13:51:58 -05:00 |
|
Ian Norden
|
e5c5422edc
|
adhjustments to work with statediffing geth v1.10-alpha.2
|
2020-03-11 13:41:59 -05:00 |
|
Ian Norden
|
be875c0100
|
db comments to resolve postgraphile type conflicts
|
2020-03-10 09:38:07 -05:00 |
|
Ian Norden
|
aad318c67b
|
improved logging in super node service
|
2020-03-08 11:47:49 -05:00 |
|
Ian Norden
|
8c2a71b16f
|
finish wasm watcher engine
|
2020-02-27 15:09:20 -06:00 |
|
Ian Norden
|
330a083749
|
watcher configuration; super node versioning
|
2020-02-25 16:38:46 -06:00 |
|
Ian Norden
|
25aa4634e9
|
work on wasm watchers
|
2020-02-24 12:54:10 -06:00 |
|
Ian Norden
|
e3e8700d34
|
api test; return full ipld models to subscribers
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
fb360d8562
|
changes to super node to improve compatibility with watcher
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
f0c5ff8077
|
begin wasm watcher engine
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
94aefafd7c
|
more explicity payload err and msg
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
2fbf97da9d
|
wasm instantiator
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
5f85a7ecf6
|
more descriptive comments/method names for super node service; send msg
when backfill is finished
|
2020-02-23 17:15:26 -06:00 |
|
Ian Norden
|
48fb5bcd27
|
separate super node instances for different chains need to be within the same process to avoid contention over ipfs repo lock
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
ef3b043f97
|
emulate btc data streamer over http; misc fixes
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
5173edf563
|
fix broken go-ipld-eth trie node dag putters
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
ca273a026d
|
major refactor part 2: remove cold import, full sync, generalize node table
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
0f765df12c
|
finish constructors
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
5d46e02509
|
tx inputs directly reference tx ouputs
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
642e08a04b
|
btc converter, publisher, and indexer unit tests
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
8643a7f3b6
|
btc ipld fetcher, paylaod fetcher, and ipld resolver
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
aa2f8bd0a5
|
btc retriever and filterer
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
f33cc3f34b
|
TxOutputs: extract and index pkscript metadata (script type, addresses, #required sigs); TxInputs: outpoint_hash => outpoint_tx_id that references transaction_cids.id
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
48f70d4ddf
|
btc mocks; reorg
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
808f1b5662
|
btc cid indexer
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
5094b975fc
|
postgres schema and tables for btc
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
9018e551ba
|
move eth tables under eth schema
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
4dde90447e
|
wrap generic top-level interfaces with loose types
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
68d8e68144
|
btc ipld publisher
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
8099f726c3
|
internalized ipld types and dag_putters; adjust converters/publishers to work with them
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
2e81f8d31a
|
btc payload converter
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
076903b174
|
btc payload streamer
|
2020-02-20 16:14:17 -06:00 |
|
Ian Norden
|
da844b0b83
|
reorg pkg/ to prepare to support chains other than ethereumm
|
2020-02-20 16:14:16 -06:00 |
|
Ian Norden
|
33ac5978f5
|
split uncles into their own table; add tx index to the tx index (lol); continue work on geth api endpoints
|
2020-01-27 14:14:03 -06:00 |
|
Ian Norden
|
e52284b9ba
|
update documentation, bug/error fixes, work on eth api for drawbridge
|
2020-01-24 15:37:52 -06:00 |
|
Ian Norden
|
7843312815
|
major refactoring of super_node to make it easier to support other chains
|
2020-01-24 15:37:52 -06:00 |
|
Ian Norden
|
5ff05225c2
|
begin eth api recapitulation
|
2020-01-24 15:37:52 -06:00 |
|
Ian Norden
|
0785507a7d
|
support for total difficulty (needed to support some eth endpoints)
|
2020-01-24 15:37:52 -06:00 |
|
Ian Norden
|
358575335b
|
expand retriever, fetcher, and resolver interfaces in prep for eth api mimick
|
2020-01-24 15:37:52 -06:00 |
|
Rob Mulholand
|
56ce8bdb41
|
(VDB-950) Write raw diffs before transforming
- Raw field we can reference by FK for related data
- Enables replay for unwatched or mistransformed diffs
|
2019-12-03 14:51:17 -06:00 |
|
Ian Norden
|
74a90fd124
|
use Debugf instead of strconv
|
2019-12-02 23:37:01 -06:00 |
|
yaoandrew
|
5a1f599a4d
|
Clean up files after rebase
|
2019-12-02 23:36:00 -06:00 |
|
yaoandrew
|
614ea3bba1
|
Adds logging for storage and RPC
|
2019-12-02 23:10:32 -06:00 |
|
yaoandrew
|
0b9f29c300
|
Removes import alias for logrus
|
2019-12-02 23:08:48 -06:00 |
|
Ian Norden
|
4036d9d6a0
|
review fixes
|
2019-12-02 13:49:27 -06:00 |
|
Ian Norden
|
5be205ffa6
|
super node backfill breaks batch call into smaller bins; retrieve gap
test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
c16ac026db
|
logWithCommand; rebase fixes; config for testing super node subscription
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
67df8dea77
|
header_cids.final => header_cids.uncle
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
3a666df294
|
backfiller refactoring; explicity errs; golint
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
40c3aff597
|
seed => super; port 80 => port 8080; backfill process for the super_node
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
83fd76bc8a
|
fix flaky test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
b789ab69e6
|
adjustments after rebase
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
8fe273fb7b
|
getter rpc method for the seed node's geth info
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
2c997921cb
|
update dockerfile and documentation
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
8b31d12716
|
fixes after rebase
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
d2db5d0ac9
|
filterer unit test; that should be the last of them
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
03dbe5dd22
|
resolver unit test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
825b0762cd
|
retriever unit test
|
2019-12-02 13:24:58 -06:00 |
|
Ian Norden
|
9ebc785d98
|
unit test for repository/indexer
|
2019-12-02 13:24:58 -06:00 |
|