cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
18,618 Commits 24 Branches 342 Tags 314 MiB
bd0c010be6
Commit Graph

405 Commits

Author SHA1 Message Date
zenground0
bd0c010be6 Configur buffer and threshold 2023-03-09 08:40:14 -07:00
zenground0
aac30cd840 Review Response 2023-03-09 07:57:35 -07:00
zenground0
fb7eb08624 lint 2023-03-09 06:36:35 -07:00
zenground0
140f2099bd lint 2023-03-08 22:49:30 -07:00
zenground0
0b0913f2f3 lint 2023-03-08 18:40:23 -07:00
zenground0
7931414735 Lint 2023-03-08 18:20:27 -07:00
zenground0
efbb63582e Review Response 2023-03-08 18:11:39 -07:00
zenground0
ee9ff563d2 computing szPurge leads to deadlock and unneeded, remove 2023-03-08 17:51:59 -07:00
zenground0
e5553554d1 cid key size 2023-03-08 17:30:42 -07:00
zenground0
efbc0ff12e Fix up approximation and logging 2023-03-08 09:43:37 -07:00
zenground0
e17ec37d10 fix logging 2023-03-08 09:13:00 -07:00
zenground0
bf29d41993 lint 2023-03-07 08:21:15 -07:00
zenground0
a994153e27 GC respects target for max hotstore space 2023-03-07 07:39:17 -07:00
zenground0
0fe91846cd Plan out moving GC limiting 2023-03-07 07:39:17 -07:00
zenground0
5d9739a863 Track size of dags relevant to compaction 2023-03-07 07:39:17 -07:00
zenground0
dd998d6b24 Begin account for size during walks 2023-03-07 07:39:17 -07:00
zenground0
5534755f2d Add command to trigger moving GC of hotstore manually 2023-03-06 11:59:08 -07:00
zenground0
71b21db0d9 chain prune hot -- hotstore online gc 2023-03-03 11:14:52 -05:00
ZenGround0
544af751cb
Fix the overzealous fix (#10366) 
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2023-02-28 09:22:18 -05:00
ZenGround0
0c9f697bf6
Two fixes, better logging and comments (#10332) 
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2023-02-23 10:05:18 -05:00
Peter Rabbitson
818b3b8daf fix: extend LOTUS_CHAIN_BADGERSTORE_DISABLE_FSYNC to the markset 
Without doing this walking a badger markset on a non-nvme knocks the
node hopelessly out of sync during a compaction.
 2023-02-01 21:00:39 +01:00
Jorropo
f572852d06 chore: all: bump go-libipfs to replace go-block-format 
Includes changes from:
- https://github.com/ipfs/go-block-format/pull/37
- https://github.com/ipfs/go-libipfs/pull/58
 2023-01-26 17:03:18 +01:00
raulk
cdf3812e40
NV18: Filecoin EVM runtime + Actor Events + EthAccount + EAM + f4 addressing (#9998) 
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Raul Kripalani <raulk@users.noreply.github.com>
Co-authored-by: Kevin Li <ychiaoli18@users.noreply.github.com>
Co-authored-by: vyzo <vyzo@hackzen.org>
Co-authored-by: Ian Davis <nospam@iandavis.com>
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Geoff Stuart <geoff.vball@gmail.com>
Co-authored-by: Shrenuj Bansal <shrenuj.bansal@protocol.ai>
Co-authored-by: Shrenuj Bansal <108157875+shrenujbansal@users.noreply.github.com>
Co-authored-by: Geoff Stuart <geoffrey.stuart@protocol.ai>
Co-authored-by: Aayush Rajasekaran <aayushrajasekaran@Aayushs-MacBook-Pro.local>
Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2023-01-13 19:11:13 +00:00
ZenGround0
4ffded6fef
feat:splitstore:single compaction that can handle prune aka two marksets one compaction (#9571) 
* begin

* rough draft -- this should probably actually work?

* WIP

* Start testing

* message mode

* Fix tests, make gen

* Better default

* docsgen-cli

* Review Response

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2022-11-07 16:31:12 -05:00
Łukasz Magiera
08b22edd89 fix make gen 2022-08-29 16:25:30 +02:00
ZenGround0
8b7be6d47e
feat:chain:splitstore auto prune (#9123) 
Auto Prune

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2022-08-08 16:06:32 -04:00
ZenGround0
0c91b0dc10
feat:chain:splitstore chain prune (#9056) 
* Splitstore chain prune
* Protect on reification for simpler logic and sound cold compact protect
* Recovery from checkpoint during chain prune
* Splitstore (discard and universal mode) running in itests
* Add pause and restart functions to itest block miner
* Add config options to itest full nodes
* Add FsRepo support for itest full ndoes

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2022-08-05 16:34:16 -04:00
Masih H. Derkani
6f686cae11 Upgrade to go-blockstore 1.2.0 and go-cid 0.2.0 
Upgrade required dependencies and fix breaking changes.
 2022-06-29 14:55:22 -04:00
Łukasz Magiera
05cdeb80c3 chore: remove redundant import prefixes 2022-06-15 12:06:22 +02:00
Łukasz Magiera
e65fae28de chore: fix imports 2022-06-14 17:00:51 +02:00
Łukasz Magiera
1c055fe83b
Merge branch 'master' into test/t.TempDir 2022-03-17 12:06:52 +01:00
Darko Brdareski
79453663b3
feat: Add additional test annotations (#8272) 
* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate repub and selection tests

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

* Fix markset annotations

* doc: test annotations for the markets dagstore wrapper

* Annotate miner_api test in dagstore

* Annotate more test files

* Remove bad annotations from fsrepo

* Annotate wdpost system

* Remove bad annotations

* Renamce "conformance" to "chaos_actor" tests

* doc: stm annotations for blockheader & election proof tests

* Annotate remaining "A" tests

* annotate: stm for error_test

* memrepo_test.go

* Annotate "b" file tests

* message_test.go

* doc: stm annotate for fsrepo_test

* Annotate "c" file tests

* Annotate "D" test files

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate u,v,wl tests

* doc: stm annotations for various test files

* Annotate "T" test files

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add mempool unit tests

* Add two more memPool Add tests

* Update submodules

* Add check function tests

* Add stm annotations, refactor test helper

* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

solving merge conflicts

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

solve merge conflict

* Fix markset annotations

* Annotate miner_api test in dagstore

* Annotate more test files

* doc: test annotations for the markets dagstore wrapper

* Annotate wdpost system

* Renamce "conformance" to "chaos_actor" tests

* Annotate remaining "A" tests

* doc: stm annotations for blockheader & election proof tests

* annotate: stm for error_test

* Annotate "b" file tests

* memrepo_test.go

* Annotate "c" file tests

* message_test.go

* Annotate "D" test files

* doc: stm annotate for fsrepo_test

* Annotate u,v,wl tests

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate "T" test files

* doc: stm annotations for various test files

* Add mempool unit tests

solve merge conflict

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add two more memPool Add tests

* Update submodules

* Add check function tests

solve conflict

* Add stm annotations, refactor test helper

solve merge conflict

* Change CLI test kinds to "unit"

* Fix double merged test

* Fix ccupgrade_test merge

* Fix lint issues

* Add stm annotation to types_Test

* Test vectors submodule

* Add file annotation to burn_test

Co-authored-by: Nikola Divic <divicnikola@gmail.com>
Co-authored-by: TheMenko <themenkoprojects@gmail.com>
 2022-03-16 18:37:34 +01:00
Eng Zer Jun
c229842f57
test: use T.TempDir to create temporary test directory 
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2022-03-15 00:21:31 +08:00
vyzo
a7b1d86533 make cidset (in memory) visitors smarter; no need to ever visit unitary objects 2022-02-18 12:35:42 +02:00
vyzo
2795995989 add reification limit test 2022-02-18 12:19:19 +02:00
vyzo
cbd23c2b1b add reification limit 2022-02-18 12:19:09 +02:00
vyzo
899a65ae87 fix test 2022-02-17 20:13:46 +02:00
vyzo
a20c6cb04b temporarily disable reification 
big reifications can use a lot of memory during sync apparently.
 2022-02-17 19:56:50 +02:00
vyzo
74556edcff don't fail reification on missing references 2022-02-17 12:52:52 +02:00
Łukasz Magiera
7efed6617c
Merge pull request #8029 from filecoin-project/feat/splistore-cold-object-reification-redux 
splistore cold object reification redux
 2022-02-14 19:04:02 +01:00
vyzo
6bcade5e6d add comment about bigness of reification batch 2022-02-14 16:13:54 +02:00
vyzo
4524fbe936 wait for reify workers to finish when closing 2022-02-14 16:10:54 +02:00
vyzo
6c7ababd3f add comment about trackTxnRefs being noops if txnActive is false 2022-02-14 16:06:12 +02:00
vyzo
a428f44793 don't reify objects while still warming up 2022-02-14 16:04:39 +02:00
vyzo
b576785aac rename GetHotView to IsHotView 2022-02-14 16:03:17 +02:00
vyzo
e129ae3706 refactor nextBatch in badger markset 2022-02-09 18:34:50 +02:00
vyzo
8ddf4766ea update README 
Adds note about 3k IOPs requirement with badger markset, updates the memory requirement for
map to 48G based on observed behaviour of test nodes.
 2022-02-08 10:10:51 +02:00
vyzo
d45e2074a6 update README for map as the default 2022-02-06 11:25:27 +02:00
vyzo
713edd565c fix lint 2022-02-05 21:30:53 +02:00
vyzo
a32b7a32f3 directly mark objects in cold object reification 2022-02-05 20:00:15 +02:00