Commit Graph

511 Commits

Author SHA1 Message Date
Aayush
5a061e4677 Merge branch 'releases' into asr/merge-release-into-master 2023-04-23 14:58:50 -04:00
Jorropo
fe42d974a2 chore: all: migrate from go-libipfs to boxo
github.com/ipfs/libipfs/blocks was unmigrated to github.com/ipfs/go-block-format due to compatibility issues with the rest of the IPLD stack.
2023-04-18 17:22:18 +02:00
Steven Allen
34f46d736f fix: revert chain sync mutex to a regular lock
Conditions always call "unlock" so we can't safely use the condition
with both the read and write side of lock. So we might as well revert
back to a regular lock.

fixes #10616
2023-04-10 17:16:20 +02:00
zenground0
d77ac5ddcb Stop swallowing errors 2023-04-10 17:16:13 +02:00
zenground0
0ce9ae4809 Implement yield friendly online GC 2023-04-10 17:16:01 +02:00
zenground0
fd66db680f check yield before GC 2023-04-10 17:15:52 +02:00
zenground0
b5da33c013 signal chain in and out of sync to compaction workers 2023-04-10 17:15:42 +02:00
Phi-rjan
b240031605
Limit moving gc threads (#10623)
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-04-05 11:21:32 -04:00
zenground0
c281d053d2 Limit moving gc threads 2023-04-04 14:07:36 -06:00
Aayush
66fc6dc3e5 refactor: stop using deprecated io/ioutil 2023-03-29 15:40:49 -04:00
Aayush
a724a713da refactor: stop using deprecated io/ioutil 2023-03-29 15:38:46 -04:00
ZenGround0
2b3a86eefb
Demote now common logs (#10516)
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-03-20 10:54:30 -04:00
zenground0
252387768d Don't enforce walking receipt tree during compaction 2023-03-17 09:29:18 -06:00
Peter Rabbitson
6afa99d0a1 Plumb through a proper Flush() method on all blockstores 2023-03-14 02:30:15 +01:00
zenground0
87d5a3723f lint 2023-03-09 08:57:14 -07:00
zenground0
bd0c010be6 Configur buffer and threshold 2023-03-09 08:40:14 -07:00
zenground0
aac30cd840 Review Response 2023-03-09 07:57:35 -07:00
zenground0
fb7eb08624 lint 2023-03-09 06:36:35 -07:00
zenground0
140f2099bd lint 2023-03-08 22:49:30 -07:00
zenground0
0b0913f2f3 lint 2023-03-08 18:40:23 -07:00
zenground0
7931414735 Lint 2023-03-08 18:20:27 -07:00
zenground0
efbb63582e Review Response 2023-03-08 18:11:39 -07:00
zenground0
ee9ff563d2 computing szPurge leads to deadlock and unneeded, remove 2023-03-08 17:51:59 -07:00
zenground0
e5553554d1 cid key size 2023-03-08 17:30:42 -07:00
zenground0
efbc0ff12e Fix up approximation and logging 2023-03-08 09:43:37 -07:00
zenground0
e17ec37d10 fix logging 2023-03-08 09:13:00 -07:00
zenground0
bf29d41993 lint 2023-03-07 08:21:15 -07:00
zenground0
a994153e27 GC respects target for max hotstore space 2023-03-07 07:39:17 -07:00
zenground0
0fe91846cd Plan out moving GC limiting 2023-03-07 07:39:17 -07:00
zenground0
5d9739a863 Track size of dags relevant to compaction 2023-03-07 07:39:17 -07:00
zenground0
dd998d6b24 Begin account for size during walks 2023-03-07 07:39:17 -07:00
zenground0
5534755f2d Add command to trigger moving GC of hotstore manually 2023-03-06 11:59:08 -07:00
zenground0
71b21db0d9 chain prune hot -- hotstore online gc 2023-03-03 11:14:52 -05:00
ZenGround0
544af751cb
Fix the overzealous fix (#10366)
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-02-28 09:22:18 -05:00
ZenGround0
0c9f697bf6
Two fixes, better logging and comments (#10332)
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-02-23 10:05:18 -05:00
Peter Rabbitson
818b3b8daf fix: extend LOTUS_CHAIN_BADGERSTORE_DISABLE_FSYNC to the markset
Without doing this walking a badger markset on a non-nvme knocks the
node hopelessly out of sync during a compaction.
2023-02-01 21:00:39 +01:00
Jorropo
f572852d06 chore: all: bump go-libipfs to replace go-block-format
Includes changes from:
- https://github.com/ipfs/go-block-format/pull/37
- https://github.com/ipfs/go-libipfs/pull/58
2023-01-26 17:03:18 +01:00
raulk
cdf3812e40
NV18: Filecoin EVM runtime + Actor Events + EthAccount + EAM + f4 addressing (#9998)
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Raul Kripalani <raulk@users.noreply.github.com>
Co-authored-by: Kevin Li <ychiaoli18@users.noreply.github.com>
Co-authored-by: vyzo <vyzo@hackzen.org>
Co-authored-by: Ian Davis <nospam@iandavis.com>
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Geoff Stuart <geoff.vball@gmail.com>
Co-authored-by: Shrenuj Bansal <shrenuj.bansal@protocol.ai>
Co-authored-by: Shrenuj Bansal <108157875+shrenujbansal@users.noreply.github.com>
Co-authored-by: Geoff Stuart <geoffrey.stuart@protocol.ai>
Co-authored-by: Aayush Rajasekaran <aayushrajasekaran@Aayushs-MacBook-Pro.local>
Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-01-13 19:11:13 +00:00
Łukasz Magiera
53e43a402a netbs: Drop client code for allkeys for now 2022-11-08 09:37:43 +00:00
Łukasz Magiera
401359646a netbs: Address review 2022-11-08 09:37:43 +00:00
Łukasz Magiera
1577740bc4 bstore: Network blockstore 2022-11-08 09:37:11 +00:00
ZenGround0
4ffded6fef
feat:splitstore:single compaction that can handle prune aka two marksets one compaction (#9571)
* begin

* rough draft -- this should probably actually work?

* WIP

* Start testing

* message mode

* Fix tests, make gen

* Better default

* docsgen-cli

* Review Response

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2022-11-07 16:31:12 -05:00
Aayush
2e78158099 fix imports 2022-11-07 15:10:27 -05:00
Steven Allen
385f787ffc fix: autobatch: remove potential deadlock when a block is missing
Check the _underlying_ blockstore instead of recursing. Also, drop the
lock before we do that.
2022-11-07 10:46:51 -08:00
Łukasz Magiera
516ad171d2 fix: bstore: Handle codecs correctly in membstore Get 2022-10-12 10:03:22 +02:00
Łukasz Magiera
08b22edd89 fix make gen 2022-08-29 16:25:30 +02:00
ZenGround0
8b7be6d47e
feat:chain:splitstore auto prune (#9123)
Auto Prune

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2022-08-08 16:06:32 -04:00
ZenGround0
0c91b0dc10
feat:chain:splitstore chain prune (#9056)
* Splitstore chain prune
* Protect on reification for simpler logic and sound cold compact protect
* Recovery from checkpoint during chain prune
* Splitstore (discard and universal mode) running in itests
* Add pause and restart functions to itest block miner
* Add config options to itest full nodes
* Add FsRepo support for itest full ndoes

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2022-08-05 16:34:16 -04:00
Masih H. Derkani
6f686cae11 Upgrade to go-blockstore 1.2.0 and go-cid 0.2.0
Upgrade required dependencies and fix breaking changes.
2022-06-29 14:55:22 -04:00
Łukasz Magiera
05cdeb80c3 chore: remove redundant import prefixes 2022-06-15 12:06:22 +02:00