Commit Graph

320 Commits

Author SHA1 Message Date
vyzo
fb3986226f do hotstore moving GC in splitstore with a user-specified frequency 2021-07-26 15:05:57 +03:00
vyzo
c747f2f1e2 do moving GC if the user asks for it 2021-07-26 15:05:57 +03:00
vyzo
a843665132 add options to BlockstoreGC trait 2021-07-26 15:05:57 +03:00
vyzo
94509968a0 make moveTo a private method 2021-07-26 15:05:20 +03:00
vyzo
5cf6fdf81d don't heap allocate the cond, just set L 2021-07-26 15:04:46 +03:00
vyzo
524564e2cf add some more logging around move 2021-07-26 15:04:46 +03:00
vyzo
f2c7b08be5 follow symbolic links when deleting old dbs 2021-07-26 15:04:46 +03:00
vyzo
608a9f84d2 fix copy: flush discards the transaction 2021-07-26 15:04:46 +03:00
vyzo
4b0b37a4ef fix lint
the great spellchecker strikes again
2021-07-26 15:04:46 +03:00
vyzo
4715b1f436 fix lotus-shed 2021-07-26 15:04:46 +03:00
vyzo
aec126879e add MoveTo test 2021-07-26 15:04:46 +03:00
vyzo
001c04f2dd use pooled slices for the copy 2021-07-26 15:04:46 +03:00
vyzo
b741d61b20 implement BlockstoreMover in badger 2021-07-26 15:04:46 +03:00
vyzo
2cfd73c879 manually compute size when badger is being stupid 2021-07-26 09:46:21 +03:00
vyzo
74009bd67f document lotus-shed splitstore utiilities in the README 2021-07-26 08:52:32 +03:00
vyzo
ce528a1293 implement Info in splitstore 2021-07-26 08:45:46 +03:00
vyzo
30e4b405b7 implement BlockstoreSize for badger 2021-07-26 08:43:09 +03:00
vyzo
baaa9a7738 add BlockstoreSize trait for reporting size 2021-07-26 08:42:54 +03:00
vyzo
c00b86e8a8 stop the walk on missing references 2021-07-25 13:42:20 +03:00
vyzo
5285a14d27 write check summary at the end 2021-07-25 11:42:13 +03:00
vyzo
5718da237f handle newlines consistently in check output 2021-07-25 11:30:58 +03:00
vyzo
1918ffda4e implement splitstore check 2021-07-25 10:35:37 +03:00
vyzo
d6ebead756 add comment about the necessity of FileIO 2021-07-23 16:30:55 +03:00
vyzo
1480c22447 improve README 2021-07-23 16:27:09 +03:00
vyzo
9faa3d5fd7 adjust compaction test with badger to hit the db 2021-07-23 12:47:18 +03:00
vyzo
9d5cf4177b add splitstore compaction test with badger markset 2021-07-23 12:47:18 +03:00
vyzo
dec5e13c85 update README with configuration options 2021-07-23 12:47:18 +03:00
vyzo
a8c0fd1654 tweak badger options 2021-07-23 12:47:18 +03:00
vyzo
4df6073382 some tweaks in badger markset 2021-07-23 12:47:18 +03:00
vyzo
f492f8d450 track in flight writes in badger markste, as they happen outside the lock 2021-07-23 12:47:18 +03:00
vyzo
f75d982c76 remove early occurs check from trackTxnRef
this happens inline, and it might block when using the badger markset
2021-07-23 12:47:18 +03:00
vyzo
2891a31c99 make badger markset concurrent close safe 2021-07-23 12:47:18 +03:00
vyzo
f2b7c3e6f2 reduce scope of exclusive lock in badger markset 2021-07-23 12:47:18 +03:00
vyzo
12c3432b8d document the "badger" markset type option 2021-07-23 12:47:18 +03:00
vyzo
be9530b66e finetune badger options 2021-07-23 12:47:18 +03:00
vyzo
2c26abc551 add badger markset test 2021-07-23 12:47:18 +03:00
vyzo
54a6968327 add badger-backed markset implementation 2021-07-23 12:47:18 +03:00
vyzo
5266b240b8 coalesce message and message receipt retention 2021-07-22 22:02:29 +03:00
vyzo
2a68ae8dad fix warmup by decoupling state from message receipt walk 2021-07-22 20:49:36 +03:00
Łukasz Magiera
da5aeda197
Merge branch 'master' into feat/splitstore-hot-messages 2021-07-22 12:58:06 +02:00
vyzo
c47fce8d2d test protector support 2021-07-20 09:02:45 +03:00
vyzo
ebbaf23af8 support out-of-chain reference protection 2021-07-20 09:02:40 +03:00
vyzo
006050ed27 implement hotstore message retention policy 2021-07-17 08:59:43 +03:00
vyzo
1b77361301 add option for hotstore message retention 2021-07-17 08:35:35 +03:00
vyzo
e003203bea implement exposed splitstore 2021-07-15 13:12:10 +03:00
vyzo
5a23f64b3b code reorg: break splitstore.go into smaller logical units 2021-07-14 13:11:15 -07:00
vyzo
3f3a12b75c remove BlockstoreMover interface
we decided it's premature
2021-07-14 22:59:53 +03:00
vyzo
023146803d use Broadcast for view barrier 2021-07-14 22:59:53 +03:00
vyzo
3d77ae1f4d make trackTxnRefMany consistent with trackTxnRef 2021-07-14 22:59:53 +03:00
vyzo
6f126c80bf remove redundant log, more descriptive error message for closing condition 2021-07-14 22:59:53 +03:00