Commit Graph

461 Commits

Author SHA1 Message Date
Geoff Stuart
801c670edd Remove ChainPutMany 2022-06-10 14:09:05 -04:00
Geoff Stuart
5c0f2c8ae6 Add putObj and putMany to apiBlockstore 2022-06-09 15:13:42 -04:00
Łukasz Magiera
1c055fe83b
Merge branch 'master' into test/t.TempDir 2022-03-17 12:06:52 +01:00
Darko Brdareski
79453663b3
feat: Add additional test annotations ()
* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate repub and selection tests

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

* Fix markset annotations

* doc: test annotations for the markets dagstore wrapper

* Annotate miner_api test in dagstore

* Annotate more test files

* Remove bad annotations from fsrepo

* Annotate wdpost system

* Remove bad annotations

* Renamce "conformance" to "chaos_actor" tests

* doc: stm annotations for blockheader & election proof tests

* Annotate remaining "A" tests

* annotate: stm for error_test

* memrepo_test.go

* Annotate "b" file tests

* message_test.go

* doc: stm annotate for fsrepo_test

* Annotate "c" file tests

* Annotate "D" test files

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate u,v,wl tests

* doc: stm annotations for various test files

* Annotate "T" test files

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add mempool unit tests

* Add two more memPool Add tests

* Update submodules

* Add check function tests

* Add stm annotations, refactor test helper

* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

solving merge conflicts

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

solve merge conflict

* Fix markset annotations

* Annotate miner_api test in dagstore

* Annotate more test files

* doc: test annotations for the markets dagstore wrapper

* Annotate wdpost system

* Renamce "conformance" to "chaos_actor" tests

* Annotate remaining "A" tests

* doc: stm annotations for blockheader & election proof tests

* annotate: stm for error_test

* Annotate "b" file tests

* memrepo_test.go

* Annotate "c" file tests

* message_test.go

* Annotate "D" test files

* doc: stm annotate for fsrepo_test

* Annotate u,v,wl tests

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate "T" test files

* doc: stm annotations for various test files

* Add mempool unit tests

solve merge conflict

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add two more memPool Add tests

* Update submodules

* Add check function tests

solve conflict

* Add stm annotations, refactor test helper

solve merge conflict

* Change CLI test kinds to "unit"

* Fix double merged test

* Fix ccupgrade_test merge

* Fix lint issues

* Add stm annotation to types_Test

* Test vectors submodule

* Add file annotation to burn_test

Co-authored-by: Nikola Divic <divicnikola@gmail.com>
Co-authored-by: TheMenko <themenkoprojects@gmail.com>
2022-03-16 18:37:34 +01:00
Eng Zer Jun
c229842f57
test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-15 00:21:31 +08:00
vyzo
a7b1d86533 make cidset (in memory) visitors smarter; no need to ever visit unitary objects 2022-02-18 12:35:42 +02:00
vyzo
2795995989 add reification limit test 2022-02-18 12:19:19 +02:00
vyzo
cbd23c2b1b add reification limit 2022-02-18 12:19:09 +02:00
vyzo
899a65ae87 fix test 2022-02-17 20:13:46 +02:00
vyzo
a20c6cb04b temporarily disable reification
big reifications can use a lot of memory during sync apparently.
2022-02-17 19:56:50 +02:00
vyzo
74556edcff don't fail reification on missing references 2022-02-17 12:52:52 +02:00
Łukasz Magiera
7efed6617c
Merge pull request from filecoin-project/feat/splistore-cold-object-reification-redux
splistore cold object reification redux
2022-02-14 19:04:02 +01:00
vyzo
6bcade5e6d add comment about bigness of reification batch 2022-02-14 16:13:54 +02:00
vyzo
4524fbe936 wait for reify workers to finish when closing 2022-02-14 16:10:54 +02:00
vyzo
6c7ababd3f add comment about trackTxnRefs being noops if txnActive is false 2022-02-14 16:06:12 +02:00
vyzo
a428f44793 don't reify objects while still warming up 2022-02-14 16:04:39 +02:00
vyzo
b576785aac rename GetHotView to IsHotView 2022-02-14 16:03:17 +02:00
vyzo
e129ae3706 refactor nextBatch in badger markset 2022-02-09 18:34:50 +02:00
vyzo
8ddf4766ea update README
Adds note about 3k IOPs requirement with badger markset, updates the memory requirement for
map to 48G based on observed behaviour of test nodes.
2022-02-08 10:10:51 +02:00
vyzo
d45e2074a6 update README for map as the default 2022-02-06 11:25:27 +02:00
vyzo
713edd565c fix lint 2022-02-05 21:30:53 +02:00
vyzo
a32b7a32f3 directly mark objects in cold object reification 2022-02-05 20:00:15 +02:00
vyzo
929a05e898 add reification test 2022-02-04 16:16:34 +02:00
vyzo
268366e446 cold object reification context option 2022-02-04 16:07:58 +02:00
vyzo
4cddfd1074 background cold object reification 2022-02-04 14:29:50 +02:00
vyzo
049b48928f add note about compaction algorithm changes in README 2022-02-02 14:39:21 +02:00
vyzo
75ad0c3c2d badger markset option tweaks 2022-02-02 14:34:30 +02:00
vyzo
c1d8368fdc share a concurrent visitor between workers in markLiveRefs 2022-02-01 21:40:09 +02:00
vyzo
4b4104e400 fix comment 2022-02-01 12:10:30 +02:00
vyzo
b13aa8f172 unblock waitForSync on close 2022-02-01 11:30:21 +02:00
vyzo
9c92d77b64 improve robustness of waitForSync 2022-02-01 11:25:07 +02:00
vyzo
cd95892bf0 fix test 2022-02-01 11:13:01 +02:00
vyzo
7b4ab2077b wait for sync in a non racey way 2022-02-01 11:10:51 +02:00
vyzo
578b5691bc check for closing after the sync wait 2022-02-01 10:59:08 +02:00
vyzo
6353fa72d8 decouple SyncGapTime from wait time 2022-02-01 10:55:15 +02:00
vyzo
fd07ca87ce wait for the sync gap time befor starting the purge 2022-02-01 10:46:42 +02:00
vyzo
11ae85645b optimize single object marking in markLiveRefs 2022-02-01 09:13:21 +02:00
vyzo
3aabb0314a synchronously mark live refs on put/putmany 2022-02-01 09:07:14 +02:00
vyzo
37673c6de6 downgrade marking log to debug 2022-01-31 21:44:10 +02:00
vyzo
7896af731f use walkObjectIncomplete for marking live refs 2022-01-31 21:41:42 +02:00
vyzo
877dfbe992 hold the lock in the second protect call 2022-01-31 19:21:03 +02:00
vyzo
5b9ea1b4e0 avoid races in beginCriticalSection 2022-01-31 18:48:53 +02:00
vyzo
710fda49f0 fix putmany marking 2022-01-31 17:35:30 +02:00
vyzo
2b14bda6b8 recursively mark puts during the critical section 2022-01-31 17:26:08 +02:00
vyzo
1abfc5b1cf fix comment 2022-01-31 15:00:03 +02:00
vyzo
c9bd5ec452 mark tipset references to protect them during critical section 2022-01-31 14:54:23 +02:00
vyzo
ee63be26a1 fix race in protectView 2022-01-31 12:40:26 +02:00
vyzo
1900c907fd account for missing refs in the markset in Has 2022-01-31 09:44:51 +02:00
vyzo
a9d4495d83 use both hot and cold when doing fetches for markset positive objects 2022-01-30 22:47:20 +02:00
vyzo
7b8447a95a reinstante waitForMissingRefs 2022-01-30 19:00:52 +02:00