Eng Zer Jun
c229842f57
test: use T.TempDir to create temporary test directory
...
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-15 00:21:31 +08:00
vyzo
a7b1d86533
make cidset (in memory) visitors smarter; no need to ever visit unitary objects
2022-02-18 12:35:42 +02:00
vyzo
2795995989
add reification limit test
2022-02-18 12:19:19 +02:00
vyzo
cbd23c2b1b
add reification limit
2022-02-18 12:19:09 +02:00
vyzo
899a65ae87
fix test
2022-02-17 20:13:46 +02:00
vyzo
a20c6cb04b
temporarily disable reification
...
big reifications can use a lot of memory during sync apparently.
2022-02-17 19:56:50 +02:00
vyzo
74556edcff
don't fail reification on missing references
2022-02-17 12:52:52 +02:00
Łukasz Magiera
7efed6617c
Merge pull request #8029 from filecoin-project/feat/splistore-cold-object-reification-redux
...
splistore cold object reification redux
2022-02-14 19:04:02 +01:00
vyzo
6bcade5e6d
add comment about bigness of reification batch
2022-02-14 16:13:54 +02:00
vyzo
4524fbe936
wait for reify workers to finish when closing
2022-02-14 16:10:54 +02:00
vyzo
6c7ababd3f
add comment about trackTxnRefs being noops if txnActive is false
2022-02-14 16:06:12 +02:00
vyzo
a428f44793
don't reify objects while still warming up
2022-02-14 16:04:39 +02:00
vyzo
b576785aac
rename GetHotView to IsHotView
2022-02-14 16:03:17 +02:00
vyzo
e129ae3706
refactor nextBatch in badger markset
2022-02-09 18:34:50 +02:00
vyzo
8ddf4766ea
update README
...
Adds note about 3k IOPs requirement with badger markset, updates the memory requirement for
map to 48G based on observed behaviour of test nodes.
2022-02-08 10:10:51 +02:00
vyzo
d45e2074a6
update README for map as the default
2022-02-06 11:25:27 +02:00
vyzo
713edd565c
fix lint
2022-02-05 21:30:53 +02:00
vyzo
a32b7a32f3
directly mark objects in cold object reification
2022-02-05 20:00:15 +02:00
vyzo
929a05e898
add reification test
2022-02-04 16:16:34 +02:00
vyzo
268366e446
cold object reification context option
2022-02-04 16:07:58 +02:00
vyzo
4cddfd1074
background cold object reification
2022-02-04 14:29:50 +02:00
vyzo
049b48928f
add note about compaction algorithm changes in README
2022-02-02 14:39:21 +02:00
vyzo
75ad0c3c2d
badger markset option tweaks
2022-02-02 14:34:30 +02:00
vyzo
c1d8368fdc
share a concurrent visitor between workers in markLiveRefs
2022-02-01 21:40:09 +02:00
vyzo
4b4104e400
fix comment
2022-02-01 12:10:30 +02:00
vyzo
b13aa8f172
unblock waitForSync on close
2022-02-01 11:30:21 +02:00
vyzo
9c92d77b64
improve robustness of waitForSync
2022-02-01 11:25:07 +02:00
vyzo
cd95892bf0
fix test
2022-02-01 11:13:01 +02:00
vyzo
7b4ab2077b
wait for sync in a non racey way
2022-02-01 11:10:51 +02:00
vyzo
578b5691bc
check for closing after the sync wait
2022-02-01 10:59:08 +02:00
vyzo
6353fa72d8
decouple SyncGapTime from wait time
2022-02-01 10:55:15 +02:00
vyzo
fd07ca87ce
wait for the sync gap time befor starting the purge
2022-02-01 10:46:42 +02:00
vyzo
11ae85645b
optimize single object marking in markLiveRefs
2022-02-01 09:13:21 +02:00
vyzo
3aabb0314a
synchronously mark live refs on put/putmany
2022-02-01 09:07:14 +02:00
vyzo
37673c6de6
downgrade marking log to debug
2022-01-31 21:44:10 +02:00
vyzo
7896af731f
use walkObjectIncomplete for marking live refs
2022-01-31 21:41:42 +02:00
vyzo
877dfbe992
hold the lock in the second protect call
2022-01-31 19:21:03 +02:00
vyzo
5b9ea1b4e0
avoid races in beginCriticalSection
2022-01-31 18:48:53 +02:00
vyzo
710fda49f0
fix putmany marking
2022-01-31 17:35:30 +02:00
vyzo
2b14bda6b8
recursively mark puts during the critical section
2022-01-31 17:26:08 +02:00
vyzo
1abfc5b1cf
fix comment
2022-01-31 15:00:03 +02:00
vyzo
c9bd5ec452
mark tipset references to protect them during critical section
2022-01-31 14:54:23 +02:00
vyzo
ee63be26a1
fix race in protectView
2022-01-31 12:40:26 +02:00
vyzo
1900c907fd
account for missing refs in the markset in Has
2022-01-31 09:44:51 +02:00
vyzo
a9d4495d83
use both hot and cold when doing fetches for markset positive objects
2022-01-30 22:47:20 +02:00
vyzo
7b8447a95a
reinstante waitForMissingRefs
2022-01-30 19:00:52 +02:00
vyzo
7931f1f8f9
fix lint
2022-01-30 15:43:52 +02:00
vyzo
20b75022aa
fix mockStore for splitstore tests
2022-01-30 15:33:30 +02:00
vyzo
dbc8903bac
sortless compaction
2022-01-30 15:33:15 +02:00
vyzo
a4f720d866
coldset test
2022-01-30 15:11:25 +02:00
