vyzo
|
8562a9bb82
|
garbage collect hotstore after compaction
|
2021-03-08 18:12:09 +02:00 |
|
vyzo
|
e85391b46c
|
quiet stupid linter
|
2021-03-05 20:05:32 +02:00 |
|
vyzo
|
09f5ba177a
|
add splitstore unit test
|
2021-03-05 19:55:32 +02:00 |
|
vyzo
|
0a2f2cf00d
|
use the right condition for triggering the miss metric
|
2021-03-05 14:48:59 +02:00 |
|
vyzo
|
2b32c2e597
|
add some metrics
|
2021-03-05 14:48:57 +02:00 |
|
vyzo
|
99d21573da
|
remove DEBUG log spam
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
c58df3f079
|
don't panic on compaction errors
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
9bd009d795
|
use atomics to demarkate critical section and limit close delay
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
17be7d3919
|
save markSetSize
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
aff0f1ed4c
|
deduplicate code for batch deletion
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
5fb6a907cb
|
fix loop condition in batch deletion
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
98a7b884fe
|
implement DeleteMany in union blockstore
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
fdd877534f
|
walk at boundary epoch, 2 finalities from current epoch, to find live objects
objects written after that are retained anyway.
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
508fcb9d26
|
properly close snoop at shutdown
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
47d8c87486
|
fix log
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
11b2f41804
|
overestimate markSetSize a bit
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
6b680d112b
|
do tracker purge in smaller batches
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
6b8c60a659
|
don't ID wrap the hotstore
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
d2d0980532
|
don't delete in one giant batch, use smaller chunks of batchSize
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
70ebb2ad8d
|
improve startup log
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
006c55a7c9
|
add startup log
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
06d8ea10b1
|
batch delete during the cold purge
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
4c05ec28ba
|
fix FromDatastore to not do double adapting
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
ab52e34e6a
|
add comment
Co-authored-by: raulk <raul@protocol.ai>
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
86fdad2e31
|
fix typo
Co-authored-by: raulk <raul@protocol.ai>
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
2ff5aec80e
|
satisfy linter, use Prefix for common path of non inline CIDs
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
8a55b73146
|
fix the situation with WrapIDStore
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
86b73d651e
|
add DeleteMany to Blockstore interface
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
dd0c308427
|
move Blockstore config to FullNode, rename to Chainstore and add default for HotStoreType
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
6014273e69
|
storage miner doesn't need a splitstore
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
c762536dcb
|
deduplicate code
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
5184bc5c40
|
log consistency for full compaction
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
68213a92cb
|
use ioutil.TempDir for test directories
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
35d466d847
|
use sha256 for bloom key rehashing
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
f651f43c5e
|
improve comment accuracy
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
4b1e1f4b52
|
rename liveset => markset; rename snoop => tracking store; docs.
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
48f253328d
|
increase batch size to 16K
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
ce68b9b229
|
batch writes during warm up
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
8cfba5b092
|
renames and polish.
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
b1b452bc0f
|
remove dependency from blockstore/splitstore => chain/store.
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
b9400c590f
|
use crypto/rand for bloom salt
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
e612fff1fe
|
also estimate liveset size during warm up
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
748dd962d8
|
snake current tipset from head change notification
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
cb36d5b6a4
|
warm up splitstore at first head change notification
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
1a804fbdec
|
move splitstore into blockstore package.
|
2021-03-05 14:46:18 +02:00 |
|
Raúl Kripalani
|
1b51c10d78
|
split off lmdb support to a different branch.
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
3733456bca
|
go mod tidy
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
0fc2f3a26f
|
fix post-rebase compilation errors
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
3282f856ec
|
fix tests
|
2021-03-05 14:46:18 +02:00 |
|
vyzo
|
99c6e4f48f
|
adjust min bloom filter size
|
2021-03-05 14:46:18 +02:00 |
|