Łukasz Magiera
|
7efed6617c
|
Merge pull request #8029 from filecoin-project/feat/splistore-cold-object-reification-redux
splistore cold object reification redux
|
2022-02-14 19:04:02 +01:00 |
|
vyzo
|
6bcade5e6d
|
add comment about bigness of reification batch
|
2022-02-14 16:13:54 +02:00 |
|
vyzo
|
4524fbe936
|
wait for reify workers to finish when closing
|
2022-02-14 16:10:54 +02:00 |
|
vyzo
|
6c7ababd3f
|
add comment about trackTxnRefs being noops if txnActive is false
|
2022-02-14 16:06:12 +02:00 |
|
vyzo
|
a428f44793
|
don't reify objects while still warming up
|
2022-02-14 16:04:39 +02:00 |
|
vyzo
|
b576785aac
|
rename GetHotView to IsHotView
|
2022-02-14 16:03:17 +02:00 |
|
vyzo
|
e129ae3706
|
refactor nextBatch in badger markset
|
2022-02-09 18:34:50 +02:00 |
|
vyzo
|
8ddf4766ea
|
update README
Adds note about 3k IOPs requirement with badger markset, updates the memory requirement for
map to 48G based on observed behaviour of test nodes.
|
2022-02-08 10:10:51 +02:00 |
|
vyzo
|
d45e2074a6
|
update README for map as the default
|
2022-02-06 11:25:27 +02:00 |
|
vyzo
|
713edd565c
|
fix lint
|
2022-02-05 21:30:53 +02:00 |
|
vyzo
|
a32b7a32f3
|
directly mark objects in cold object reification
|
2022-02-05 20:00:15 +02:00 |
|
vyzo
|
929a05e898
|
add reification test
|
2022-02-04 16:16:34 +02:00 |
|
vyzo
|
268366e446
|
cold object reification context option
|
2022-02-04 16:07:58 +02:00 |
|
vyzo
|
4cddfd1074
|
background cold object reification
|
2022-02-04 14:29:50 +02:00 |
|
vyzo
|
049b48928f
|
add note about compaction algorithm changes in README
|
2022-02-02 14:39:21 +02:00 |
|
vyzo
|
75ad0c3c2d
|
badger markset option tweaks
|
2022-02-02 14:34:30 +02:00 |
|
vyzo
|
c1d8368fdc
|
share a concurrent visitor between workers in markLiveRefs
|
2022-02-01 21:40:09 +02:00 |
|
vyzo
|
4b4104e400
|
fix comment
|
2022-02-01 12:10:30 +02:00 |
|
vyzo
|
b13aa8f172
|
unblock waitForSync on close
|
2022-02-01 11:30:21 +02:00 |
|
vyzo
|
9c92d77b64
|
improve robustness of waitForSync
|
2022-02-01 11:25:07 +02:00 |
|
vyzo
|
cd95892bf0
|
fix test
|
2022-02-01 11:13:01 +02:00 |
|
vyzo
|
7b4ab2077b
|
wait for sync in a non racey way
|
2022-02-01 11:10:51 +02:00 |
|
vyzo
|
578b5691bc
|
check for closing after the sync wait
|
2022-02-01 10:59:08 +02:00 |
|
vyzo
|
6353fa72d8
|
decouple SyncGapTime from wait time
|
2022-02-01 10:55:15 +02:00 |
|
vyzo
|
fd07ca87ce
|
wait for the sync gap time befor starting the purge
|
2022-02-01 10:46:42 +02:00 |
|
vyzo
|
11ae85645b
|
optimize single object marking in markLiveRefs
|
2022-02-01 09:13:21 +02:00 |
|
vyzo
|
3aabb0314a
|
synchronously mark live refs on put/putmany
|
2022-02-01 09:07:14 +02:00 |
|
vyzo
|
37673c6de6
|
downgrade marking log to debug
|
2022-01-31 21:44:10 +02:00 |
|
vyzo
|
7896af731f
|
use walkObjectIncomplete for marking live refs
|
2022-01-31 21:41:42 +02:00 |
|
vyzo
|
877dfbe992
|
hold the lock in the second protect call
|
2022-01-31 19:21:03 +02:00 |
|
vyzo
|
5b9ea1b4e0
|
avoid races in beginCriticalSection
|
2022-01-31 18:48:53 +02:00 |
|
vyzo
|
710fda49f0
|
fix putmany marking
|
2022-01-31 17:35:30 +02:00 |
|
vyzo
|
2b14bda6b8
|
recursively mark puts during the critical section
|
2022-01-31 17:26:08 +02:00 |
|
vyzo
|
1abfc5b1cf
|
fix comment
|
2022-01-31 15:00:03 +02:00 |
|
vyzo
|
c9bd5ec452
|
mark tipset references to protect them during critical section
|
2022-01-31 14:54:23 +02:00 |
|
vyzo
|
ee63be26a1
|
fix race in protectView
|
2022-01-31 12:40:26 +02:00 |
|
vyzo
|
1900c907fd
|
account for missing refs in the markset in Has
|
2022-01-31 09:44:51 +02:00 |
|
vyzo
|
a9d4495d83
|
use both hot and cold when doing fetches for markset positive objects
|
2022-01-30 22:47:20 +02:00 |
|
vyzo
|
7b8447a95a
|
reinstante waitForMissingRefs
|
2022-01-30 19:00:52 +02:00 |
|
vyzo
|
7931f1f8f9
|
fix lint
|
2022-01-30 15:43:52 +02:00 |
|
vyzo
|
20b75022aa
|
fix mockStore for splitstore tests
|
2022-01-30 15:33:30 +02:00 |
|
vyzo
|
dbc8903bac
|
sortless compaction
|
2022-01-30 15:33:15 +02:00 |
|
vyzo
|
a4f720d866
|
coldset test
|
2022-01-30 15:11:25 +02:00 |
|
vyzo
|
4b8369c071
|
fix buffered reads
|
2022-01-30 15:11:18 +02:00 |
|
vyzo
|
72333147d3
|
prettier checkpoint close
|
2022-01-30 15:00:03 +02:00 |
|
vyzo
|
64cda4aeda
|
on disk coldsets
|
2022-01-30 14:19:05 +02:00 |
|
vyzo
|
6ede77b254
|
checkpoint test
|
2022-01-30 14:10:11 +02:00 |
|
vyzo
|
c94eee5fc3
|
on disk checkpoints
|
2022-01-30 12:47:30 +02:00 |
|
vyzo
|
322b85898f
|
make markSets synchronous in critical section
|
2022-01-30 12:10:08 +02:00 |
|
vyzo
|
cf09dd044a
|
moar markset tests
|
2022-01-30 11:45:12 +02:00 |
|