Łukasz Magiera
|
089ff08d81
|
Merge pull request #5881 from filecoin-project/fix/lint-skip
lint: don't skip builtin
|
2021-03-27 10:09:44 +01:00 |
|
Steven Allen
|
f5029a0187
|
lint: don't skip builtin
|
2021-03-25 17:21:56 -07:00 |
|
Łukasz Magiera
|
53cba157e2
|
Merge pull request #5804 from zl03jsj/fix/StateManager.Replay
fix: StateManager.Replay
|
2021-03-25 13:27:45 +01:00 |
|
Łukasz Magiera
|
fc4f60e528
|
Merge pull request #5850 from filecoin-project/feat/markets-dealid
expose NextID from nice market actor interface
|
2021-03-25 13:26:29 +01:00 |
|
Łukasz Magiera
|
c41777dcd2
|
API proxy struct codegen (#5854)
* mostly working api proxy gen
* api: Consistent api names
* fix docsgen
* regenerate api struct
* api: expand external interfaces
* Add missing gen files
* apigen: fix perm detection
* api: Move perm tags to the interface
* gofmt
* worker perms
* docsgen
* docsgen: ignore tag comments
* apigen: add codegen warning
* gofmt
* missing actor type
* docsgen
* make linter happy
* fix lint
* apigen: use directives for tags
* docsgen
* regen openrpc docs
|
2021-03-23 13:42:56 +01:00 |
|
frrist
|
0b579c1e03
|
feat(events): define Observer intreface for events
- allows tipset apply and revert to be observed
|
2021-03-22 15:37:44 -07:00 |
|
whyrusleeping
|
6bcf92f71a
|
expose NextID from nice market actor interface
|
2021-03-21 00:11:42 -07:00 |
|
Peter Rabbitson
|
c340750672
|
Nerpa reset backport/cherry-pick from 73eabc310f
|
2021-03-18 20:55:15 +01:00 |
|
Aayush Rajasekaran
|
707219b7a1
|
Disable the Calico migration on not-mainnet
|
2021-03-17 20:46:00 -04:00 |
|
Aayush Rajasekaran
|
fe473e111c
|
Fix creation of remainder account when it's not a multisig
|
2021-03-12 17:12:41 -05:00 |
|
zl
|
6f64d5db6f
|
bugfix: StateManager.Replay
|
2021-03-13 00:12:37 +08:00 |
|
whyrusleeping
|
e0b650d4ec
|
make rpcstatemanager its own package, for easy reuse
|
2021-03-11 18:21:18 -08:00 |
|
Łukasz Magiera
|
58d7627168
|
Merge pull request #5730 from filecoin-project/asr/genesis-rootkey
Customize verifreg root key and remainder account when making genesis
|
2021-03-08 22:42:09 +01:00 |
|
Aayush Rajasekaran
|
fc03a0773b
|
gen/genesis: Allow remainder to be either account or msig
|
2021-03-06 01:43:48 -05:00 |
|
Aayush Rajasekaran
|
279f9e6fec
|
gen/genesis: Allow verifreg rootkey to be either account or msig
|
2021-03-06 01:43:48 -05:00 |
|
Raúl Kripalani
|
b1b452bc0f
|
remove dependency from blockstore/splitstore => chain/store.
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
1a804fbdec
|
move splitstore into blockstore package.
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
1b51c10d78
|
split off lmdb support to a different branch.
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
0fc2f3a26f
|
fix post-rebase compilation errors
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
3282f856ec
|
fix tests
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
99c6e4f48f
|
adjust min bloom filter size
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
cae5ddce88
|
dynamically size bloom filters
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
5639261e44
|
make compaction parameters variable
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
7587ab6234
|
quiet the stupid linter
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
05fee27840
|
remove stale references to lmdb from splitstore implementation
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
f62999d2b8
|
use named constants for bloom filter parameters
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
44aadb931a
|
rehash salted keys in bloom filter
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
88849201ff
|
fix tests
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
f5ce7957f3
|
size bloom filter for 50M objects
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
f4c6bc6a86
|
comment nomenclature
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
4cc672d0c5
|
batch move objects from coldstore to hotstore
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
97abbe1eca
|
add (salted) bloom filter liveset
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
aba6530411
|
batch deletion for purging the tracking store
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
09cd1175a1
|
structured log for beginning of compaction
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
e52c709d8a
|
more accurate setting of skip params
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
2426ffb277
|
better logging plus moving some code around
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
2f26026991
|
compactSimple should walk the cold epoch at depth 1
So that it finds reachable objects that should stay in the hotstore
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
783dcda19c
|
add Sync to the tracking store
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
364076c1ea
|
set NoSync option for bolt livesets
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
73259aa350
|
add configuration for splitstore and default to a simple compaction algorithm
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
2e4d45ef07
|
test for bolt backed tracking store
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
f1c61c4753
|
implement bolt backed tracking store
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
2c1a9781cf
|
add test for bolt liveset
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
27a9b974db
|
implement bolt-backed liveset
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
68b6f913c7
|
propagate useLMDB option to splitstore through DI
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
923a3db4b0
|
abstract tracking store and live set construction
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
8f0ddac41a
|
add comment
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
e79445123f
|
handle MDB_KEY_EXIST in tracking store Puts
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
9977f5c3ec
|
rewrite sweep logic to avoid doing writes/deletes nested in a read txn
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
ee751f88cd
|
refactor lmdb specific snoop/liveset code into their own files
paves the way for different back ends
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
8e12377e69
|
handle consistency edge case
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
31268ba685
|
walk snapshot the same way snapshot exporting does; skip old msgs and receipts by default.
so that we don't panic with missing blocks in non-archival nodes
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
5068d51ac3
|
use CompactionCold epochs for delinating the cold epoch cliff
this allows us to change the thresholds for testing.
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
d44719dbd3
|
amend confusing comment
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
842ec43c2f
|
get rid of goroutine iteration in tracking store; long live ForEach
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
a586d42c3b
|
make hot store DI injectable in the split store, default to badger.
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
7044e623f9
|
flag to enable GC during compaction, disabled for now
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
f6c930d0aa
|
crank up blockstore max readers to 16K, reduce retry delays to 10us
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
95befa1e41
|
set lmdb max readers retry delay to 1ms
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
874ecd3573
|
adjust hot store options, redux.
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
ca8a673b5f
|
adjust hot store options
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
69a88d41b6
|
fix snoop test
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
cdf5bd0500
|
return annotated xerrors where appropriate
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
ea05fd9d86
|
use xerrors instead of fmt.Errorf
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
d91b60df91
|
fix potential panic with max readers retry and cursor channel
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
b9f8a3d587
|
log MDB_READERS_FULL retries
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
c89ab1a990
|
retry on MDB_READERS_FULL errors
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
2080e467ba
|
don't set MaxReaders for tracking store
|
2021-03-05 14:46:17 +02:00 |
|
Raúl Kripalani
|
877ecab960
|
update go-bs-lmdb and migrate to ledgerwatch/lmdb-go.
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
5b4e6b7b26
|
don't set max readers for livesets
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
58a8434825
|
temporary log level for splitstore to DEBUG
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
8b0087524f
|
adjust walk boundaries for marking
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
76d6edbb52
|
fix max readers for tracking store
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
1a23b1f6af
|
make CompactionThreshold a var to fix lotus-soup build
finality is not a constant there!
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
6e51e6db9a
|
better handling of MDB_KEYEXIST in Put
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
ce41e394dc
|
handle MDB_KEYEXIST in liveset marking
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
843fd09a62
|
deal with MDB_KEY_EXIST errors
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
f44cf0f2c4
|
appease linter
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
facdc555b1
|
add nil check for curTs -- some tests don't have chain state
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
e07c6c71c0
|
splitstore constructor
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
b0f48b500f
|
use CAS for compacting state
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
0af7b16ad5
|
simplify Has
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
37e391f133
|
add TODO note about map size
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
5db314f422
|
fallback to coldstore if snooping fails.
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
d20cbc0c28
|
protect against potential data races
overkill, but let's not have race detectors scream at us.
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
da478832cb
|
quiet linter
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
4763397281
|
add tracking store test
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
0d7476c5b2
|
implement LMDB-backed tracking store
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
83f8a0ab12
|
quiet linter
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
5043f31adf
|
liveset unit test
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
3f92a000c7
|
implement lmdb-backed LiveSet
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
0bf1a78b39
|
stubs for tracking store and live set
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
17bc5fcd85
|
move splitstore implementation to its own directory
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
2c9b58aaec
|
add some logging
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
c1b1a9ce2a
|
avoid race with compacting state variable
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
3083d80f5e
|
no need to import go-ipfs-blockstore, lib/blockstore will do
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
101e5c6540
|
close keys channel when dome emitting keys
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
b945747eb2
|
satisfy linter
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
2bed6c94cd
|
use dual live set marking algorithm to keep all hotly reachable objects in the hotstore
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
c2cc198316
|
fix off by 1 in marking
|
2021-03-05 14:46:17 +02:00 |
|