vyzo
|
f2c7b08be5
|
follow symbolic links when deleting old dbs
|
2021-07-26 15:04:46 +03:00 |
|
vyzo
|
608a9f84d2
|
fix copy: flush discards the transaction
|
2021-07-26 15:04:46 +03:00 |
|
vyzo
|
4b0b37a4ef
|
fix lint
the great spellchecker strikes again
|
2021-07-26 15:04:46 +03:00 |
|
vyzo
|
4715b1f436
|
fix lotus-shed
|
2021-07-26 15:04:46 +03:00 |
|
vyzo
|
aec126879e
|
add MoveTo test
|
2021-07-26 15:04:46 +03:00 |
|
vyzo
|
001c04f2dd
|
use pooled slices for the copy
|
2021-07-26 15:04:46 +03:00 |
|
vyzo
|
b741d61b20
|
implement BlockstoreMover in badger
|
2021-07-26 15:04:46 +03:00 |
|
vyzo
|
2cfd73c879
|
manually compute size when badger is being stupid
|
2021-07-26 09:46:21 +03:00 |
|
vyzo
|
74009bd67f
|
document lotus-shed splitstore utiilities in the README
|
2021-07-26 08:52:32 +03:00 |
|
vyzo
|
ce528a1293
|
implement Info in splitstore
|
2021-07-26 08:45:46 +03:00 |
|
vyzo
|
30e4b405b7
|
implement BlockstoreSize for badger
|
2021-07-26 08:43:09 +03:00 |
|
vyzo
|
baaa9a7738
|
add BlockstoreSize trait for reporting size
|
2021-07-26 08:42:54 +03:00 |
|
vyzo
|
c00b86e8a8
|
stop the walk on missing references
|
2021-07-25 13:42:20 +03:00 |
|
vyzo
|
5285a14d27
|
write check summary at the end
|
2021-07-25 11:42:13 +03:00 |
|
vyzo
|
5718da237f
|
handle newlines consistently in check output
|
2021-07-25 11:30:58 +03:00 |
|
vyzo
|
1918ffda4e
|
implement splitstore check
|
2021-07-25 10:35:37 +03:00 |
|
vyzo
|
d6ebead756
|
add comment about the necessity of FileIO
|
2021-07-23 16:30:55 +03:00 |
|
vyzo
|
1480c22447
|
improve README
|
2021-07-23 16:27:09 +03:00 |
|
vyzo
|
9faa3d5fd7
|
adjust compaction test with badger to hit the db
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
9d5cf4177b
|
add splitstore compaction test with badger markset
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
dec5e13c85
|
update README with configuration options
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
a8c0fd1654
|
tweak badger options
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
4df6073382
|
some tweaks in badger markset
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
f492f8d450
|
track in flight writes in badger markste, as they happen outside the lock
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
f75d982c76
|
remove early occurs check from trackTxnRef
this happens inline, and it might block when using the badger markset
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
2891a31c99
|
make badger markset concurrent close safe
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
f2b7c3e6f2
|
reduce scope of exclusive lock in badger markset
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
12c3432b8d
|
document the "badger" markset type option
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
be9530b66e
|
finetune badger options
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
2c26abc551
|
add badger markset test
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
54a6968327
|
add badger-backed markset implementation
|
2021-07-23 12:47:18 +03:00 |
|
vyzo
|
5266b240b8
|
coalesce message and message receipt retention
|
2021-07-22 22:02:29 +03:00 |
|
vyzo
|
2a68ae8dad
|
fix warmup by decoupling state from message receipt walk
|
2021-07-22 20:49:36 +03:00 |
|
Łukasz Magiera
|
da5aeda197
|
Merge branch 'master' into feat/splitstore-hot-messages
|
2021-07-22 12:58:06 +02:00 |
|
vyzo
|
c47fce8d2d
|
test protector support
|
2021-07-20 09:02:45 +03:00 |
|
vyzo
|
ebbaf23af8
|
support out-of-chain reference protection
|
2021-07-20 09:02:40 +03:00 |
|
vyzo
|
006050ed27
|
implement hotstore message retention policy
|
2021-07-17 08:59:43 +03:00 |
|
vyzo
|
1b77361301
|
add option for hotstore message retention
|
2021-07-17 08:35:35 +03:00 |
|
vyzo
|
e003203bea
|
implement exposed splitstore
|
2021-07-15 13:12:10 +03:00 |
|
vyzo
|
5a23f64b3b
|
code reorg: break splitstore.go into smaller logical units
|
2021-07-14 13:11:15 -07:00 |
|
vyzo
|
3f3a12b75c
|
remove BlockstoreMover interface
we decided it's premature
|
2021-07-14 22:59:53 +03:00 |
|
vyzo
|
023146803d
|
use Broadcast for view barrier
|
2021-07-14 22:59:53 +03:00 |
|
vyzo
|
3d77ae1f4d
|
make trackTxnRefMany consistent with trackTxnRef
|
2021-07-14 22:59:53 +03:00 |
|
vyzo
|
6f126c80bf
|
remove redundant log, more descriptive error message for closing condition
|
2021-07-14 22:59:53 +03:00 |
|
vyzo
|
ff093fae00
|
use a missing compactionIndex as an indicator for warmup
so that splitstore v0 nodes upgrading will get a fresh warmup.
|
2021-07-14 22:59:53 +03:00 |
|
vyzo
|
669b47cfc9
|
do moving gc for hotstore every 20 compactions
that's about once a week
|
2021-07-14 22:59:53 +03:00 |
|
vyzo
|
818b8de182
|
keep track of the compaction serial (index)
it is useful so that:
- we only do slow (but very effective) moving gc every 10 compactions
- we can detect a splitstore v0 upgrade and re-warm up
|
2021-07-14 22:59:53 +03:00 |
|
vyzo
|
c93328b036
|
use the new traits for hotstore gc
|
2021-07-14 22:59:52 +03:00 |
|
vyzo
|
35180b4761
|
merge Compact and CollectGarbage in badger
|
2021-07-14 22:59:52 +03:00 |
|
vyzo
|
dc81c0e6a2
|
add blockstore traits related to gc
|
2021-07-14 22:59:52 +03:00 |
|