Commit Graph

131 Commits

Author SHA1 Message Date
Aloxaf
0dd7976ffb
fix: sealing: typo in FinalizeReplicaUpdate 2023-09-08 10:15:58 +08:00
Aayush Rajasekaran
8ab6eaa6e3
Merge pull request #11230 from filecoin-project/asr/test-seal-and-verify-3-needs-to-stop
feat: skip TestSealAndVerify3 until it's fixed
2023-09-06 14:48:38 -04:00
Andrew Jackson (Ajax)
b73c016f6d unseal - comments 2023-09-01 17:15:56 -05:00
Aayush
b92cff74f2 feat: skip TestSealAndVerify3 until it's fixed 2023-09-01 10:08:55 -04:00
Łukasz Magiera
077f3f0c33 ffiwrapper: Fix sector key regen 2023-08-31 09:46:10 -04:00
Łukasz Magiera
8ea44e82d1 ffiwrapper: Correctly create unsealed file when doesn't exist 2023-08-31 09:46:10 -04:00
Łukasz Magiera
e74eb6af09 make the linter happy 2023-08-31 09:46:10 -04:00
Jennifer Wang
ad6b0575c1 make gen, mod tidy 2023-08-31 09:46:10 -04:00
Łukasz Magiera
a965fc660e fix build 2023-08-31 09:46:10 -04:00
Jennifer Wang
4b78c4ea1e ffiwrapper: Fix sector acquire logic in unsealing 2023-08-31 09:46:10 -04:00
Łukasz Magiera
6167bcc8ce ffiwrapper: Fix sector acquire logic in unsealing 2023-08-31 09:46:10 -04:00
Łukasz Magiera
fa0f117edd ffiwrapper: Fix multiple bugs in snapdeals unsealing 2023-08-31 09:46:10 -04:00
Łukasz Magiera
bcc2d7fd10 ffiwrapper: Fix RegenerateSectorKey 2023-08-31 09:46:10 -04:00
Jennifer Wang
b0b39424e4 ffiwrapper: Test lower-level GenerateSDR 2023-08-31 09:46:10 -04:00
Łukasz Magiera
f8faa85cba make gen 2023-08-28 16:12:14 +02:00
Łukasz Magiera
d83956aa6d ci: Ensure params in sealer tests 2023-08-28 13:43:50 +02:00
Łukasz Magiera
bce5535679
Merge pull request #11182 from filecoin-project/TestSealerDLBefore
Take Download out of Sealer time
2023-08-25 10:27:42 +02:00
Łukasz Magiera
366329b085
Merge pull request #11087 from filecoin-project/fix/pc1-retr-loop
fix: sealing pipeline: Fix PC1 retry loop
2023-08-24 11:08:39 +02:00
Andrew Jackson (Ajax)
c48bf864a4 Take Download out of Sealer time 2023-08-17 22:34:08 -05:00
Andrew Jackson (Ajax)
76bca6c392 mac seal test 2023-08-17 07:30:19 -05:00
nemo
c6d31c0d6d fix: downgrade harmless warning to debug 2023-08-08 10:52:35 -04:00
Łukasz Magiera
0a83896589 sealing pipeline: Better retry for soft errors 2023-07-19 09:51:59 -07:00
Łukasz Magiera
4b5a665422 Merge remote-tracking branch 'origin/master' into feat/piecereader-perf 2023-05-26 13:00:42 +02:00
Łukasz Magiera
b58daf5340 piecereader: address review 2023-05-25 16:05:36 +02:00
Łukasz Magiera
5e58f64380 piecereader: Fix double buffer free 2023-05-23 16:45:46 +02:00
Łukasz Magiera
f0fec13ec9 piecereader: Metrics fixes 2023-05-23 14:15:46 +02:00
Łukasz Magiera
326a0356a4 piecereader: Add metrics 2023-05-23 13:32:17 +02:00
Łukasz Magiera
6c62e6d8e2 fix tests/lint 2023-05-23 12:26:13 +02:00
Łukasz Magiera
587c3fde58 piecereader: Fix parallel buffer access, close readers in readInto 2023-05-23 12:26:13 +02:00
Łukasz Magiera
cd75ea0fe4 piecereader: Retune to allow parallel ReadAt calls 2023-05-23 12:26:13 +02:00
Łukasz Magiera
8b2ef40f4e storage: Weave end offset into unsealed reader code 2023-05-23 12:26:13 +02:00
Łukasz Magiera
6fd93ed170
fix: sched: Address GET_32G_MAX_CONCURRENT regression (#10850)
* Fix 1.21 regression: GET_32G_MAX_CONCURRENT + mixed prepared/executing leads to stuck scheduler

If you have 12 GET tasks and GET_32G_MAX_CONCURRENT=1, sealing jobs will only show assigned tasks for GET of the miner
and is stuck.
I believe this to be a regression of 1.21 unifying the counters, in the case of GETs where PrepType and TaskType
both being seal/v0/fetch leading to a state where tasks are blocked since already counted towards the limit.

* itests: Repro issue from PR #10633

* make counters int (non-working)

* fix: worker sched: Send taskDone notifs after tasks are done

* itests: Make TestPledgeMaxConcurrentGet actually reproduce the issue

* make the linter happy

---------

Co-authored-by: Steffen Butzer <steffen.butzer@outlook.com>
2023-05-10 15:43:42 -04:00
Łukasz Magiera
2f469c2f58 fix: tests: Use mutex-wrapped datastore in storage tests 2023-05-09 18:06:41 +02:00
Shrenuj Bansal
78800a4e77
fix: sealing: Make lotus-worker report GPU usage to miner during ReplicaUpdate task (#10806)
* Make lotus-worker report GPU usage to miner during ReplicaUpdate task

* make gen

---------

Co-authored-by: Jacob Crowther <jcrowthe@users.noreply.github.com>
2023-05-03 12:28:03 -04:00
Łukasz Magiera
9d2d53b58e fix: prover: Propagate skipped sectors in local PoSt 2023-04-27 09:26:12 +02:00
Aayush
5a061e4677 Merge branch 'releases' into asr/merge-release-into-master 2023-04-23 14:58:50 -04:00
sirius
07416e8c0c update: builtin to builtin11 in wdpost worker 2023-04-21 12:19:06 +08:00
ZenGround0
327760acff
chore: build: Merge/v22 into 21 for 23 (#10702)
* chore: update ffi to increase execution parallelism

* Don't enforce walking receipt tree during compaction

* fix: build: drop drand incentinet servers

* chore: release lotus v1.20.4

* Apply suggestions from code review

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

* feat: Introduce nv19 skeleton

Update to go-state-types v0.11.0-alpha-1

Introduce dummy v11 actor bundles

Make new actors adapters

Add upgrade to Upgrade Schedules

make jen

Update to go-state-types v0.11.0-alpha-2

* feat: vm: switch to the new exec trace format (#10372)

This is now "FVM" native. Changes include:

1. Don't treat "trace" messages like off-chain messages. E.g., don't
include CIDs, versions, etc.
2. Include IPLD codecs where applicable.
3. Remove fields that aren't filled by the FVM (timing, some errors,
code locations, etc.).

* feat: implement FIP-0061

* Address review

* Add and test the FIP-0061 migration

* Update actors bundles to fip/20230406

* Update to go-state-types master

* Update to actors v11.0.0-rc1

* - Update go state types
- Keep current expiration defaults on creation, extension some tests
- Update ffi

* ffi experiment

* Integration nv19 migration

- Open splitstore in migration shed tool
- Update state root version

* Post rebase fixup

* Fix

* gen

* nv19 invariant checking

* Try fixig blockstore so bundle is loaded

* Debug

* Fix

* Make butterfly upgrades happen

* Another ffi experiment

* Fix copy paste error

* Actually schedule migration (#10656)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Butterfly artifacts

* Set calibration net upgrade height

* Review Response

* Fix state tree version assert

* Quick butterfly upgrade to sanity check (#10660)

* Quick butterfly upgrade to sanity check

* Update butterfly artifacts

* Revert fake fix

* Give butterfly net correct genesis

* Butterfly artifacts

* Give time before upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* chore:releasepolish v1.22 release (#10666)

* Update butterfly artifacts

* register actors v11

* Update calibration upgrade time

* State inspection shed cmds

* Fix

* make gen

* Fix swallowed errors

* Lint fixup

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* v1.22.0-rc3

* bundle fix

* Feat/expedite nv19 (#10681)

* Update go-state-types

* Modify upgrade schedule and params

* Revert fip 0052

* Update gst

* docsgen

* fast butterfly migration to validate migration

* Correct epoch to match specified date

* Update actors v11

* Update changelog build version

* Update butterfly artifacts

* Fix lotus-miner init to work after upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* fix:deps:stable ffi for stable release  (#10698)

* Point to stable ffi for stable lotus release

* go mod tidy

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Update CHANGELOG.md

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

---------

Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: jennijuju <jiayingw703@gmail.com>
2023-04-19 18:40:18 -04:00
Phi
9fd69377df fix: unseal: check if sealed sector exists
Check if sealed or update sector exists when `SectorsUnsealPiece` is called.
2023-04-10 09:53:19 +02:00
Aayush
66fc6dc3e5 refactor: stop using deprecated io/ioutil 2023-03-29 15:40:49 -04:00
Aayush
a724a713da refactor: stop using deprecated io/ioutil 2023-03-29 15:38:46 -04:00
Łukasz Magiera
7e414ac390 Merge remote-tracking branch 'origin/master' into feat/go-data-transfer-v2 2023-03-15 11:37:53 +01:00
Łukasz Magiera
80ccd14447
Merge pull request #10356 from filecoin-project/feat/assigner-experiments
feat: sched: Assigner experiments
2023-03-09 01:28:31 +01:00
Łukasz Magiera
d114d8fe7e deprecate MarketListRetrievalDeals, make gen 2023-03-08 17:15:16 +01:00
Łukasz Magiera
b0ebdb6882 make gen 2023-03-06 15:07:49 +01:00
Łukasz Magiera
9a295e58b0 post worker sched: Retry on alternative worker on RPC errors 2023-03-06 14:30:58 +01:00
Łukasz Magiera
e170487faf post worker sched: Filter out disabled workers correctly 2023-03-06 13:56:23 +01:00
Łukasz Magiera
c484c38735 worker sched: Separate resource def for preparing window 2023-02-28 10:52:50 +01:00
Łukasz Magiera
2316363f7a sched: Share active/preparing task counters 2023-02-28 09:08:49 +01:00
Łukasz Magiera
745476c9ab feat: sched: Assigner experiments 2023-02-27 17:54:25 +01:00