cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
12,995 Commits 24 Branches 342 Tags 314 MiB
b1f60e85e9
Commit Graph

274 Commits

Author SHA1 Message Date
vyzo
b1f60e85e9 document moving GC frequency option in README 2021-07-26 15:05:57 +03:00
vyzo
5acae50e07 add config option for splitstore moving gc frequency 2021-07-26 15:05:57 +03:00
vyzo
fb3986226f do hotstore moving GC in splitstore with a user-specified frequency 2021-07-26 15:05:57 +03:00
vyzo
74009bd67f document lotus-shed splitstore utiilities in the README 2021-07-26 08:52:32 +03:00
vyzo
ce528a1293 implement Info in splitstore 2021-07-26 08:45:46 +03:00
vyzo
c00b86e8a8 stop the walk on missing references 2021-07-25 13:42:20 +03:00
vyzo
5285a14d27 write check summary at the end 2021-07-25 11:42:13 +03:00
vyzo
5718da237f handle newlines consistently in check output 2021-07-25 11:30:58 +03:00
vyzo
1918ffda4e implement splitstore check 2021-07-25 10:35:37 +03:00
vyzo
d6ebead756 add comment about the necessity of FileIO 2021-07-23 16:30:55 +03:00
vyzo
1480c22447 improve README 2021-07-23 16:27:09 +03:00
vyzo
9faa3d5fd7 adjust compaction test with badger to hit the db 2021-07-23 12:47:18 +03:00
vyzo
9d5cf4177b add splitstore compaction test with badger markset 2021-07-23 12:47:18 +03:00
vyzo
dec5e13c85 update README with configuration options 2021-07-23 12:47:18 +03:00
vyzo
a8c0fd1654 tweak badger options 2021-07-23 12:47:18 +03:00
vyzo
4df6073382 some tweaks in badger markset 2021-07-23 12:47:18 +03:00
vyzo
f492f8d450 track in flight writes in badger markste, as they happen outside the lock 2021-07-23 12:47:18 +03:00
vyzo
f75d982c76 remove early occurs check from trackTxnRef 
this happens inline, and it might block when using the badger markset
 2021-07-23 12:47:18 +03:00
vyzo
2891a31c99 make badger markset concurrent close safe 2021-07-23 12:47:18 +03:00
vyzo
f2b7c3e6f2 reduce scope of exclusive lock in badger markset 2021-07-23 12:47:18 +03:00
vyzo
12c3432b8d document the "badger" markset type option 2021-07-23 12:47:18 +03:00
vyzo
be9530b66e finetune badger options 2021-07-23 12:47:18 +03:00
vyzo
2c26abc551 add badger markset test 2021-07-23 12:47:18 +03:00
vyzo
54a6968327 add badger-backed markset implementation 2021-07-23 12:47:18 +03:00
vyzo
5266b240b8 coalesce message and message receipt retention 2021-07-22 22:02:29 +03:00
vyzo
2a68ae8dad fix warmup by decoupling state from message receipt walk 2021-07-22 20:49:36 +03:00
Łukasz Magiera
da5aeda197
Merge branch 'master' into feat/splitstore-hot-messages 2021-07-22 12:58:06 +02:00
vyzo
c47fce8d2d test protector support 2021-07-20 09:02:45 +03:00
vyzo
ebbaf23af8 support out-of-chain reference protection 2021-07-20 09:02:40 +03:00
vyzo
006050ed27 implement hotstore message retention policy 2021-07-17 08:59:43 +03:00
vyzo
1b77361301 add option for hotstore message retention 2021-07-17 08:35:35 +03:00
vyzo
e003203bea implement exposed splitstore 2021-07-15 13:12:10 +03:00
vyzo
5a23f64b3b code reorg: break splitstore.go into smaller logical units 2021-07-14 13:11:15 -07:00
vyzo
3f3a12b75c remove BlockstoreMover interface 
we decided it's premature
 2021-07-14 22:59:53 +03:00
vyzo
023146803d use Broadcast for view barrier 2021-07-14 22:59:53 +03:00
vyzo
3d77ae1f4d make trackTxnRefMany consistent with trackTxnRef 2021-07-14 22:59:53 +03:00
vyzo
6f126c80bf remove redundant log, more descriptive error message for closing condition 2021-07-14 22:59:53 +03:00
vyzo
ff093fae00 use a missing compactionIndex as an indicator for warmup 
so that splitstore v0 nodes upgrading will get a fresh warmup.
 2021-07-14 22:59:53 +03:00
vyzo
669b47cfc9 do moving gc for hotstore every 20 compactions 
that's about once a week
 2021-07-14 22:59:53 +03:00
vyzo
818b8de182 keep track of the compaction serial (index) 
it is useful so that:
- we only do slow (but very effective) moving gc every 10 compactions
- we can detect a splitstore v0 upgrade and re-warm up
 2021-07-14 22:59:53 +03:00
vyzo
c93328b036 use the new traits for hotstore gc 2021-07-14 22:59:52 +03:00
vyzo
af399529ec finetune view waiting 2021-07-13 09:06:40 +03:00
vyzo
257423e917 fix view waiting issues with the WaitGroup 
We can add after Wait is called, which is problematic with WaitGroups.
This instead uses a mx/cond combo and waits while the count is > 0.
The only downside is that we might needlessly wait for (a bunch) of views
that started while the txn is active, but we can live with that.
 2021-07-13 09:01:50 +03:00
Steven Allen
04abd190ab nit: remove useless goto 
Because stebalien has allergies.
 2021-07-12 21:46:50 -07:00
vyzo
60212c86cb put a mutex around HeadChange 2021-07-13 03:14:13 +03:00
vyzo
759594d01c always return the waitgroup in protectView 
so that we preclude the following scenario:
    Start compaction.
    Start view.
    Finish compaction.
    Start compaction.

which would not wait for the view to complete.
 2021-07-13 03:11:40 +03:00
vyzo
df9670c58d fix lint 2021-07-10 16:38:40 +03:00
vyzo
0c5e336ff1 address review comments 2021-07-10 16:30:27 +03:00
vyzo
870a47f55d handle id cids in internal versions of view/get 2021-07-09 20:07:17 +03:00
vyzo
f5ae10e3d1 refactor debug log code to eliminate duplication 2021-07-09 19:53:51 +03:00