vyzo
|
5fb6a907cb
|
fix loop condition in batch deletion
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
98a7b884fe
|
implement DeleteMany in union blockstore
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
fdd877534f
|
walk at boundary epoch, 2 finalities from current epoch, to find live objects
objects written after that are retained anyway.
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
508fcb9d26
|
properly close snoop at shutdown
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
47d8c87486
|
fix log
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
11b2f41804
|
overestimate markSetSize a bit
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
6b680d112b
|
do tracker purge in smaller batches
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
6b8c60a659
|
don't ID wrap the hotstore
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
d2d0980532
|
don't delete in one giant batch, use smaller chunks of batchSize
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
70ebb2ad8d
|
improve startup log
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
006c55a7c9
|
add startup log
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
06d8ea10b1
|
batch delete during the cold purge
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
4c05ec28ba
|
fix FromDatastore to not do double adapting
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
ab52e34e6a
|
add comment
Co-authored-by: raulk <raul@protocol.ai>
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
86fdad2e31
|
fix typo
Co-authored-by: raulk <raul@protocol.ai>
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
2ff5aec80e
|
satisfy linter, use Prefix for common path of non inline CIDs
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
8a55b73146
|
fix the situation with WrapIDStore
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
86b73d651e
|
add DeleteMany to Blockstore interface
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
dd0c308427
|
move Blockstore config to FullNode, rename to Chainstore and add default for HotStoreType
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
6014273e69
|
storage miner doesn't need a splitstore
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
c762536dcb
|
deduplicate code
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
5184bc5c40
|
log consistency for full compaction
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
68213a92cb
|
use ioutil.TempDir for test directories
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
35d466d847
|
use sha256 for bloom key rehashing
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
f651f43c5e
|
improve comment accuracy
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
4b1e1f4b52
|
rename liveset => markset; rename snoop => tracking store; docs.
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
48f253328d
|
increase batch size to 16K
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
ce68b9b229
|
batch writes during warm up
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
8cfba5b092
|
renames and polish.
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
b1b452bc0f
|
remove dependency from blockstore/splitstore => chain/store.
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
b9400c590f
|
use crypto/rand for bloom salt
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
e612fff1fe
|
also estimate liveset size during warm up
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
748dd962d8
|
snake current tipset from head change notification
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
cb36d5b6a4
|
warm up splitstore at first head change notification
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
1a804fbdec
|
move splitstore into blockstore package.
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
1b51c10d78
|
split off lmdb support to a different branch.
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
3733456bca
|
go mod tidy
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
0fc2f3a26f
|
fix post-rebase compilation errors
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
3282f856ec
|
fix tests
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
99c6e4f48f
|
adjust min bloom filter size
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
cae5ddce88
|
dynamically size bloom filters
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
5639261e44
|
make compaction parameters variable
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
7587ab6234
|
quiet the stupid linter
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
e582f0b713
|
remove references to splitstore from lotus-shed
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
05fee27840
|
remove stale references to lmdb from splitstore implementation
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
f62999d2b8
|
use named constants for bloom filter parameters
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
44aadb931a
|
rehash salted keys in bloom filter
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
88849201ff
|
fix tests
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
f5ce7957f3
|
size bloom filter for 50M objects
|
2021-03-05 14:46:17 +02:00 |
|
vyzo
|
f4c6bc6a86
|
comment nomenclature
|
2021-03-05 14:46:17 +02:00 |
|