Commit Graph

71 Commits

Author SHA1 Message Date
Steven Allen
f353a794cb
fix: spelling
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
2021-04-28 22:08:37 -07:00
Steven Allen
63db9e1633 fix(splitstore): fix a panic on revert-only head changes
Calling, e.g., `lotus chain sethead` on an ancestor tipset won't apply
any new blocks, it'll just revert a bunch. This will lead to HeadChange
calls with no new blocks to apply.

fixes 
2021-04-28 20:35:30 -07:00
Peter Rabbitson
a5fd552a0c CachedBlockstore is not referenced as of 3795cc2bd2 2021-04-06 13:24:49 +02:00
Łukasz Magiera
5f80869fe0
Merge pull request from filecoin-project/fix/atomic-first
fix: make sure atomic 64bit fields are 64bit aligned
2021-03-25 13:29:59 +01:00
Steven Allen
25725110e7
Merge pull request from filecoin-project/fix/timed-cache-locking
fix: avoid holding a lock while calling the View callback
2021-03-12 09:25:00 -08:00
Łukasz Magiera
c69b26cfc6
Merge pull request from filecoin-project/feat/splitstore-compact-hotstore
splitstore: compact hotstore prior to garbage collection
2021-03-12 16:30:05 +01:00
Steven Allen
bba71da401 fix: return buffers after canceling badger operation
In theory, Delete/Put could fail. If it does, we'll return the buffers
to the pool before we're really done with them.

In practice, this is almost certainly not an issue as badger shouldn't
_use_ the buffer unless we flush. But I feel slightly safer this way.
2021-03-11 20:30:43 -08:00
Steven Allen
1c490f3fda fix: make sure atomic 64bit fields are 64bit aligned
Otherwise, this won't work on 32bit ARM.
2021-03-11 20:10:39 -08:00
Steven Allen
a888ea0d1f fix: avoid holding a lock while calling the View callback
Interleaved puts/views could get really slow and there's no real reason
to use view under the covers here because the underlying blockstore is
always a "memstore".
2021-03-11 20:03:38 -08:00
vyzo
1b1d3606cd make linter happy 2021-03-11 13:10:44 +02:00
vyzo
353bb1881f compact hotstore if it provides the method 2021-03-11 11:45:19 +02:00
vyzo
01ce9b5c44 add Compact to badger blockstore 2021-03-11 11:45:05 +02:00
vyzo
ae6410d02f use compacting atomic to make the test deterministic 2021-03-09 09:05:36 +02:00
Steven Allen
6d2e8d721d test: attempt to make the splitstore test deterministic
At a minimum, make it thread-safe.
2021-03-08 16:36:25 -08:00
vyzo
90741da019 tune badger gc to repeated gc the value log until there is no rewrite 2021-03-08 21:46:44 +02:00
vyzo
3bd77701d8 deduplicate code 2021-03-08 19:46:21 +02:00
vyzo
3d1b855f20 rename GC to CollectGarbage, ignore badger.ErrNoRewrite 2021-03-08 19:42:38 +02:00
vyzo
52de95d344 also gc in compactFull, not just compactSimple 2021-03-08 18:30:09 +02:00
vyzo
8562a9bb82 garbage collect hotstore after compaction 2021-03-08 18:12:09 +02:00
vyzo
e85391b46c quiet stupid linter 2021-03-05 20:05:32 +02:00
vyzo
09f5ba177a add splitstore unit test 2021-03-05 19:55:32 +02:00
vyzo
0a2f2cf00d use the right condition for triggering the miss metric 2021-03-05 14:48:59 +02:00
vyzo
2b32c2e597 add some metrics 2021-03-05 14:48:57 +02:00
vyzo
99d21573da remove DEBUG log spam 2021-03-05 14:46:18 +02:00
vyzo
c58df3f079 don't panic on compaction errors 2021-03-05 14:46:18 +02:00
vyzo
9bd009d795 use atomics to demarkate critical section and limit close delay 2021-03-05 14:46:18 +02:00
vyzo
17be7d3919 save markSetSize 2021-03-05 14:46:18 +02:00
vyzo
aff0f1ed4c deduplicate code for batch deletion 2021-03-05 14:46:18 +02:00
vyzo
5fb6a907cb fix loop condition in batch deletion 2021-03-05 14:46:18 +02:00
vyzo
98a7b884fe implement DeleteMany in union blockstore 2021-03-05 14:46:18 +02:00
vyzo
fdd877534f walk at boundary epoch, 2 finalities from current epoch, to find live objects
objects written after that are retained anyway.
2021-03-05 14:46:18 +02:00
vyzo
508fcb9d26 properly close snoop at shutdown 2021-03-05 14:46:18 +02:00
vyzo
47d8c87486 fix log 2021-03-05 14:46:18 +02:00
vyzo
11b2f41804 overestimate markSetSize a bit 2021-03-05 14:46:18 +02:00
vyzo
6b680d112b do tracker purge in smaller batches 2021-03-05 14:46:18 +02:00
vyzo
d2d0980532 don't delete in one giant batch, use smaller chunks of batchSize 2021-03-05 14:46:18 +02:00
vyzo
70ebb2ad8d improve startup log 2021-03-05 14:46:18 +02:00
vyzo
006c55a7c9 add startup log 2021-03-05 14:46:18 +02:00
vyzo
06d8ea10b1 batch delete during the cold purge 2021-03-05 14:46:18 +02:00
vyzo
4c05ec28ba fix FromDatastore to not do double adapting 2021-03-05 14:46:18 +02:00
vyzo
ab52e34e6a add comment
Co-authored-by: raulk <raul@protocol.ai>
2021-03-05 14:46:18 +02:00
vyzo
86fdad2e31 fix typo
Co-authored-by: raulk <raul@protocol.ai>
2021-03-05 14:46:18 +02:00
vyzo
2ff5aec80e satisfy linter, use Prefix for common path of non inline CIDs 2021-03-05 14:46:18 +02:00
vyzo
8a55b73146 fix the situation with WrapIDStore 2021-03-05 14:46:18 +02:00
vyzo
86b73d651e add DeleteMany to Blockstore interface 2021-03-05 14:46:18 +02:00
vyzo
c762536dcb deduplicate code 2021-03-05 14:46:18 +02:00
vyzo
5184bc5c40 log consistency for full compaction 2021-03-05 14:46:18 +02:00
vyzo
68213a92cb use ioutil.TempDir for test directories 2021-03-05 14:46:18 +02:00
vyzo
35d466d847 use sha256 for bloom key rehashing 2021-03-05 14:46:18 +02:00
vyzo
f651f43c5e improve comment accuracy 2021-03-05 14:46:18 +02:00