Commit Graph

380 Commits

Author SHA1 Message Date
vyzo
c58df3f079 don't panic on compaction errors 2021-03-05 14:46:18 +02:00
vyzo
9bd009d795 use atomics to demarkate critical section and limit close delay 2021-03-05 14:46:18 +02:00
vyzo
17be7d3919 save markSetSize 2021-03-05 14:46:18 +02:00
vyzo
aff0f1ed4c deduplicate code for batch deletion 2021-03-05 14:46:18 +02:00
vyzo
5fb6a907cb fix loop condition in batch deletion 2021-03-05 14:46:18 +02:00
vyzo
fdd877534f walk at boundary epoch, 2 finalities from current epoch, to find live objects
objects written after that are retained anyway.
2021-03-05 14:46:18 +02:00
vyzo
508fcb9d26 properly close snoop at shutdown 2021-03-05 14:46:18 +02:00
vyzo
47d8c87486 fix log 2021-03-05 14:46:18 +02:00
vyzo
11b2f41804 overestimate markSetSize a bit 2021-03-05 14:46:18 +02:00
vyzo
6b680d112b do tracker purge in smaller batches 2021-03-05 14:46:18 +02:00
vyzo
d2d0980532 don't delete in one giant batch, use smaller chunks of batchSize 2021-03-05 14:46:18 +02:00
vyzo
70ebb2ad8d improve startup log 2021-03-05 14:46:18 +02:00
vyzo
006c55a7c9 add startup log 2021-03-05 14:46:18 +02:00
vyzo
06d8ea10b1 batch delete during the cold purge 2021-03-05 14:46:18 +02:00
vyzo
86b73d651e add DeleteMany to Blockstore interface 2021-03-05 14:46:18 +02:00
vyzo
c762536dcb deduplicate code 2021-03-05 14:46:18 +02:00
vyzo
5184bc5c40 log consistency for full compaction 2021-03-05 14:46:18 +02:00
vyzo
68213a92cb use ioutil.TempDir for test directories 2021-03-05 14:46:18 +02:00
vyzo
35d466d847 use sha256 for bloom key rehashing 2021-03-05 14:46:18 +02:00
vyzo
f651f43c5e improve comment accuracy 2021-03-05 14:46:18 +02:00
Raúl Kripalani
4b1e1f4b52 rename liveset => markset; rename snoop => tracking store; docs. 2021-03-05 14:46:18 +02:00
vyzo
48f253328d increase batch size to 16K 2021-03-05 14:46:18 +02:00
vyzo
ce68b9b229 batch writes during warm up 2021-03-05 14:46:18 +02:00
Raúl Kripalani
8cfba5b092 renames and polish. 2021-03-05 14:46:18 +02:00
Raúl Kripalani
b1b452bc0f remove dependency from blockstore/splitstore => chain/store. 2021-03-05 14:46:18 +02:00
vyzo
b9400c590f use crypto/rand for bloom salt 2021-03-05 14:46:18 +02:00
vyzo
e612fff1fe also estimate liveset size during warm up 2021-03-05 14:46:18 +02:00
vyzo
748dd962d8 snake current tipset from head change notification 2021-03-05 14:46:18 +02:00
vyzo
cb36d5b6a4 warm up splitstore at first head change notification 2021-03-05 14:46:18 +02:00
Raúl Kripalani
1a804fbdec move splitstore into blockstore package. 2021-03-05 14:46:18 +02:00