vyzo
d7ceef104e
decrease CompactionThreshold to 3 finalities
2021-07-04 18:38:28 +03:00
vyzo
e3cbeec6ee
implement chain walking
2021-07-04 18:38:28 +03:00
vyzo
04f2e102a1
kill full splitstore compaction, simplify splitstore configuration
2021-07-04 18:38:28 +03:00
Steven Allen
f353a794cb
fix: spelling
...
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
2021-04-28 22:08:37 -07:00
Steven Allen
63db9e1633
fix(splitstore): fix a panic on revert-only head changes
...
Calling, e.g., `lotus chain sethead` on an ancestor tipset won't apply
any new blocks, it'll just revert a bunch. This will lead to HeadChange
calls with no new blocks to apply.
fixes #6125
2021-04-28 20:35:30 -07:00
vyzo
1b1d3606cd
make linter happy
2021-03-11 13:10:44 +02:00
vyzo
353bb1881f
compact hotstore if it provides the method
2021-03-11 11:45:19 +02:00
vyzo
ae6410d02f
use compacting atomic to make the test deterministic
2021-03-09 09:05:36 +02:00
Steven Allen
6d2e8d721d
test: attempt to make the splitstore test deterministic
...
At a minimum, make it thread-safe.
2021-03-08 16:36:25 -08:00
vyzo
3bd77701d8
deduplicate code
2021-03-08 19:46:21 +02:00
vyzo
3d1b855f20
rename GC to CollectGarbage, ignore badger.ErrNoRewrite
2021-03-08 19:42:38 +02:00
vyzo
52de95d344
also gc in compactFull, not just compactSimple
2021-03-08 18:30:09 +02:00
vyzo
8562a9bb82
garbage collect hotstore after compaction
2021-03-08 18:12:09 +02:00
vyzo
e85391b46c
quiet stupid linter
2021-03-05 20:05:32 +02:00
vyzo
09f5ba177a
add splitstore unit test
2021-03-05 19:55:32 +02:00
vyzo
0a2f2cf00d
use the right condition for triggering the miss metric
2021-03-05 14:48:59 +02:00
vyzo
2b32c2e597
add some metrics
2021-03-05 14:48:57 +02:00
vyzo
99d21573da
remove DEBUG log spam
2021-03-05 14:46:18 +02:00
vyzo
c58df3f079
don't panic on compaction errors
2021-03-05 14:46:18 +02:00
vyzo
9bd009d795
use atomics to demarkate critical section and limit close delay
2021-03-05 14:46:18 +02:00
vyzo
17be7d3919
save markSetSize
2021-03-05 14:46:18 +02:00
vyzo
aff0f1ed4c
deduplicate code for batch deletion
2021-03-05 14:46:18 +02:00
vyzo
5fb6a907cb
fix loop condition in batch deletion
2021-03-05 14:46:18 +02:00
vyzo
fdd877534f
walk at boundary epoch, 2 finalities from current epoch, to find live objects
...
objects written after that are retained anyway.
2021-03-05 14:46:18 +02:00
vyzo
508fcb9d26
properly close snoop at shutdown
2021-03-05 14:46:18 +02:00
vyzo
47d8c87486
fix log
2021-03-05 14:46:18 +02:00
vyzo
11b2f41804
overestimate markSetSize a bit
2021-03-05 14:46:18 +02:00
vyzo
6b680d112b
do tracker purge in smaller batches
2021-03-05 14:46:18 +02:00
vyzo
d2d0980532
don't delete in one giant batch, use smaller chunks of batchSize
2021-03-05 14:46:18 +02:00
vyzo
70ebb2ad8d
improve startup log
2021-03-05 14:46:18 +02:00
vyzo
006c55a7c9
add startup log
2021-03-05 14:46:18 +02:00
vyzo
06d8ea10b1
batch delete during the cold purge
2021-03-05 14:46:18 +02:00
vyzo
86b73d651e
add DeleteMany to Blockstore interface
2021-03-05 14:46:18 +02:00
vyzo
c762536dcb
deduplicate code
2021-03-05 14:46:18 +02:00
vyzo
5184bc5c40
log consistency for full compaction
2021-03-05 14:46:18 +02:00
vyzo
68213a92cb
use ioutil.TempDir for test directories
2021-03-05 14:46:18 +02:00
vyzo
35d466d847
use sha256 for bloom key rehashing
2021-03-05 14:46:18 +02:00
vyzo
f651f43c5e
improve comment accuracy
2021-03-05 14:46:18 +02:00
Raúl Kripalani
4b1e1f4b52
rename liveset => markset; rename snoop => tracking store; docs.
2021-03-05 14:46:18 +02:00
vyzo
48f253328d
increase batch size to 16K
2021-03-05 14:46:18 +02:00
vyzo
ce68b9b229
batch writes during warm up
2021-03-05 14:46:18 +02:00
Raúl Kripalani
8cfba5b092
renames and polish.
2021-03-05 14:46:18 +02:00
Raúl Kripalani
b1b452bc0f
remove dependency from blockstore/splitstore => chain/store.
2021-03-05 14:46:18 +02:00
vyzo
b9400c590f
use crypto/rand for bloom salt
2021-03-05 14:46:18 +02:00
vyzo
e612fff1fe
also estimate liveset size during warm up
2021-03-05 14:46:18 +02:00
vyzo
748dd962d8
snake current tipset from head change notification
2021-03-05 14:46:18 +02:00
vyzo
cb36d5b6a4
warm up splitstore at first head change notification
2021-03-05 14:46:18 +02:00
Raúl Kripalani
1a804fbdec
move splitstore into blockstore package.
2021-03-05 14:46:18 +02:00
