cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
21,261 Commits 24 Branches 342 Tags 314 MiB
ecc82d4526
Commit Graph

1281 Commits

Author SHA1 Message Date
Jorropo
6c01310728
chore: migrate to boxo 
This migrates everything except the `go-car` librairy: https://github.com/ipfs/boxo/issues/218#issuecomment-1529922103

I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups.

This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649

The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions.

Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent https://github.com/golang/go/issues/58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type.

Stubbing types were reverted https://github.com/ipfs/boxo/issues/218#issuecomment-1478650351

Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
 2023-06-19 14:45:05 -07:00
Aayush Rajasekaran
ce5e242d76
Merge pull request #10915 from filecoin-project/fix/upgrade-sector-selection 
sealing: Improve upgrade sector selection
 2023-05-30 13:32:52 -04:00
Łukasz Magiera
17915dce5a sealing: Don't select sectors with no deals lined up for upgrading 2023-05-27 13:03:21 +02:00
Łukasz Magiera
4b5a665422 Merge remote-tracking branch 'origin/master' into feat/piecereader-perf 2023-05-26 13:00:42 +02:00
Łukasz Magiera
090b98ad33 sealing: Attempt to improve upgrade sector selection 2023-05-26 12:31:35 +02:00
Łukasz Magiera
b58daf5340 piecereader: address review 2023-05-25 16:05:36 +02:00
Łukasz Magiera
5e58f64380 piecereader: Fix double buffer free 2023-05-23 16:45:46 +02:00
Łukasz Magiera
f0fec13ec9 piecereader: Metrics fixes 2023-05-23 14:15:46 +02:00
Łukasz Magiera
326a0356a4 piecereader: Add metrics 2023-05-23 13:32:17 +02:00
Łukasz Magiera
6c62e6d8e2 fix tests/lint 2023-05-23 12:26:13 +02:00
Łukasz Magiera
587c3fde58 piecereader: Fix parallel buffer access, close readers in readInto 2023-05-23 12:26:13 +02:00
Łukasz Magiera
cd75ea0fe4 piecereader: Retune to allow parallel ReadAt calls 2023-05-23 12:26:13 +02:00
Łukasz Magiera
8b2ef40f4e storage: Weave end offset into unsealed reader code 2023-05-23 12:26:13 +02:00
Łukasz Magiera
6fd93ed170
fix: sched: Address GET_32G_MAX_CONCURRENT regression (#10850) 
* Fix 1.21 regression: GET_32G_MAX_CONCURRENT + mixed prepared/executing leads to stuck scheduler

If you have 12 GET tasks and GET_32G_MAX_CONCURRENT=1, sealing jobs will only show assigned tasks for GET of the miner
and is stuck.
I believe this to be a regression of 1.21 unifying the counters, in the case of GETs where PrepType and TaskType
both being seal/v0/fetch leading to a state where tasks are blocked since already counted towards the limit.

* itests: Repro issue from PR #10633

* make counters int (non-working)

* fix: worker sched: Send taskDone notifs after tasks are done

* itests: Make TestPledgeMaxConcurrentGet actually reproduce the issue

* make the linter happy

---------

Co-authored-by: Steffen Butzer <steffen.butzer@outlook.com>
 2023-05-10 15:43:42 -04:00
Łukasz Magiera
2f469c2f58 fix: tests: Use mutex-wrapped datastore in storage tests 2023-05-09 18:06:41 +02:00
Shrenuj Bansal
78800a4e77
fix: sealing: Make lotus-worker report GPU usage to miner during ReplicaUpdate task (#10806) 
* Make lotus-worker report GPU usage to miner during ReplicaUpdate task

* make gen

---------

Co-authored-by: Jacob Crowther <jcrowthe@users.noreply.github.com>
 2023-05-03 12:28:03 -04:00
Łukasz Magiera
9d2d53b58e fix: prover: Propagate skipped sectors in local PoSt 2023-04-27 09:26:12 +02:00
Łukasz Magiera
aeee542fb9
Merge pull request #10696 from filecoin-project/fix/nil-terminate-wait 
fix: sealing pipeline: Allow nil message in TerminateWait
 2023-04-25 19:28:52 +02:00
Łukasz Magiera
7e34b49b5d
Merge pull request #10661 from filecoin-project/fix/cleanup-temp-failed-fetch 
fix: storage: Remove temp fetching files after failed fetch
 2023-04-25 19:28:33 +02:00
Aayush
5a061e4677 Merge branch 'releases' into asr/merge-release-into-master 2023-04-23 14:58:50 -04:00
Łukasz Magiera
b44ae9a7d8
Merge pull request #10647 from filecoin-project/sbansal/split-pcb 
feat: sealing: Split PCA/PCB batches if gas used exceeds block limit
 2023-04-21 18:30:27 +02:00
sirius
07416e8c0c update: builtin to builtin11 in wdpost worker 2023-04-21 12:19:06 +08:00
Shrenuj Bansal
d1f3380850 change comment 2023-04-20 12:15:51 -04:00
Shrenuj Bansal
0c83781a7f Add tests for PCB/PCA batch splitting 2023-04-19 18:44:32 -04:00
ZenGround0
327760acff
chore: build: Merge/v22 into 21 for 23 (#10702) 
* chore: update ffi to increase execution parallelism

* Don't enforce walking receipt tree during compaction

* fix: build: drop drand incentinet servers

* chore: release lotus v1.20.4

* Apply suggestions from code review

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

* feat: Introduce nv19 skeleton

Update to go-state-types v0.11.0-alpha-1

Introduce dummy v11 actor bundles

Make new actors adapters

Add upgrade to Upgrade Schedules

make jen

Update to go-state-types v0.11.0-alpha-2

* feat: vm: switch to the new exec trace format (#10372)

This is now "FVM" native. Changes include:

1. Don't treat "trace" messages like off-chain messages. E.g., don't
include CIDs, versions, etc.
2. Include IPLD codecs where applicable.
3. Remove fields that aren't filled by the FVM (timing, some errors,
code locations, etc.).

* feat: implement FIP-0061

* Address review

* Add and test the FIP-0061 migration

* Update actors bundles to fip/20230406

* Update to go-state-types master

* Update to actors v11.0.0-rc1

* - Update go state types
- Keep current expiration defaults on creation, extension some tests
- Update ffi

* ffi experiment

* Integration nv19 migration

- Open splitstore in migration shed tool
- Update state root version

* Post rebase fixup

* Fix

* gen

* nv19 invariant checking

* Try fixig blockstore so bundle is loaded

* Debug

* Fix

* Make butterfly upgrades happen

* Another ffi experiment

* Fix copy paste error

* Actually schedule migration (#10656)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Butterfly artifacts

* Set calibration net upgrade height

* Review Response

* Fix state tree version assert

* Quick butterfly upgrade to sanity check (#10660)

* Quick butterfly upgrade to sanity check

* Update butterfly artifacts

* Revert fake fix

* Give butterfly net correct genesis

* Butterfly artifacts

* Give time before upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* chore:releasepolish v1.22 release (#10666)

* Update butterfly artifacts

* register actors v11

* Update calibration upgrade time

* State inspection shed cmds

* Fix

* make gen

* Fix swallowed errors

* Lint fixup

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* v1.22.0-rc3

* bundle fix

* Feat/expedite nv19 (#10681)

* Update go-state-types

* Modify upgrade schedule and params

* Revert fip 0052

* Update gst

* docsgen

* fast butterfly migration to validate migration

* Correct epoch to match specified date

* Update actors v11

* Update changelog build version

* Update butterfly artifacts

* Fix lotus-miner init to work after upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* fix:deps:stable ffi for stable release  (#10698)

* Point to stable ffi for stable lotus release

* go mod tidy

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Update CHANGELOG.md

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

---------

Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: jennijuju <jiayingw703@gmail.com>
 2023-04-19 18:40:18 -04:00
Łukasz Magiera
4bdb6b34b3 fix: sealing pipeline: Allow nil message in TerminateWait 2023-04-19 15:43:50 +02:00
Shrenuj Bansal
79826447f5 fix unit and integration test breaks 2023-04-12 21:45:43 -04:00
Łukasz Magiera
7a4f69721c fix: storage: Remove temp fetching files after failed fetch 2023-04-12 17:00:10 -07:00
Shrenuj Bansal
4eb4af639a Split PCA msg into smaller batches 2023-04-12 00:30:19 -04:00
Shrenuj Bansal
6f91dc7c5b populate result error on exit conditions 2023-04-10 16:24:13 -04:00
Shrenuj Bansal
8893c62a42 make gen 2023-04-10 15:39:42 -04:00
Shrenuj Bansal
1e25d7b453 Split precommit batches if gas used exceeds block limit 2023-04-10 14:52:39 -04:00
Phi
9fd69377df fix: unseal: check if sealed sector exists 
Check if sealed or update sector exists when `SectorsUnsealPiece` is called.
 2023-04-10 09:53:19 +02:00
Shrenuj Bansal
2278a209e2
Add feature to stagger sector prove commit submission (#10543) 
* Add feature to stagger sector prove commit submission

* make gen and docsgen as usual

* address comments and lint

* Update comment

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>

* make gen for stupid comment

* make docsgen

* address comments

---------

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
 2023-04-01 19:30:32 -04:00
Łukasz Magiera
3af5ef1167
Merge pull request #10588 from filecoin-project/fix/file-does-not-exsist-logs 
fix: log: Stop logging `file does not exists`
 2023-03-30 20:24:59 +02:00
Aayush
66fc6dc3e5 refactor: stop using deprecated io/ioutil 2023-03-29 15:40:49 -04:00
Aayush
a724a713da refactor: stop using deprecated io/ioutil 2023-03-29 15:38:46 -04:00
Phi
a99946937c fix: log: Stop logging file does not exists 
Stop logging `file does not exists` errors when retrieving disk usage information.
 2023-03-29 15:32:32 +02:00
Łukasz Magiera
7e414ac390 Merge remote-tracking branch 'origin/master' into feat/go-data-transfer-v2 2023-03-15 11:37:53 +01:00
Steven Allen
dcb49dc8ee
refactor: update cache to the new generic version (#10463) 
- Adds type safety.
- Reduces allocations.
- Fixes the drand cache (was storing by value, but retrieving by pointer)
 2023-03-13 15:29:09 -07:00
Łukasz Magiera
80ccd14447
Merge pull request #10356 from filecoin-project/feat/assigner-experiments 
feat: sched: Assigner experiments
 2023-03-09 01:28:31 +01:00
Łukasz Magiera
52ae215104
Merge pull request #10363 from filecoin-project/fix/fsm-shutdown-removed-sectors 
fix: fsm: shutdown removed sectors FSMs
 2023-03-08 19:32:47 +01:00
Łukasz Magiera
d114d8fe7e deprecate MarketListRetrievalDeals, make gen 2023-03-08 17:15:16 +01:00
Łukasz Magiera
b0ebdb6882 make gen 2023-03-06 15:07:49 +01:00
Łukasz Magiera
9a295e58b0 post worker sched: Retry on alternative worker on RPC errors 2023-03-06 14:30:58 +01:00
Łukasz Magiera
e170487faf post worker sched: Filter out disabled workers correctly 2023-03-06 13:56:23 +01:00
Łukasz Magiera
c484c38735 worker sched: Separate resource def for preparing window 2023-02-28 10:52:50 +01:00
Łukasz Magiera
04fe9062a6 fix: fsm: shutdown removed sectors FSMs 2023-02-28 09:48:17 +01:00
Łukasz Magiera
2316363f7a sched: Share active/preparing task counters 2023-02-28 09:08:49 +01:00
Łukasz Magiera
745476c9ab feat: sched: Assigner experiments 2023-02-27 17:54:25 +01:00
Łukasz Magiera
a0771b12eb
Merge pull request #9878 from filecoin-project/gstuart/check-allocation-expiry 
feat: Check for allocation expiry when waiting to seal sectors
 2023-01-27 13:05:09 +01:00
Łukasz Magiera
fc6721cff7 feat: wdpost: Emit more detailed errors 2023-01-26 11:57:16 +01:00
Adin Schmahmann
d35818293a fix: stub out the FileSize command so lotus libraries can build on Windows 2023-01-17 01:03:10 -05:00
Geoff Stuart
787470c2b4 Remove workaround that is no longer needed 2023-01-12 12:39:57 -05:00
Geoff Stuart
12a06defd1 Check sectors for allocation expiry before committing 2023-01-11 16:24:34 -05:00
Phi
bd28a4bca4 Add snap-sector types 
Add snap-sector types
 2022-12-15 14:29:58 +01:00
Phi
87e384e7fd Update svg-image 
We do not depend on specs-storage anymore, this has been moved into lotus (storiface)
 2022-12-15 10:49:58 +01:00
Phi
a8b406e7b7 Update svg-file 
Update svg-file and headings
 2022-12-15 10:15:07 +01:00
Geoff Stuart
ae4010ffb6 Drop actor keys from chain/actors/manifest.go 2022-12-13 18:02:34 -05:00
Geoff Stuart
f95e6602a0 Check for nil returns from StateSectorGetInfo 2022-12-12 10:44:08 -05:00
Łukasz Magiera
f0bf3408bc sealing: Add a guard check to getExpirationCached when sector not found 2022-12-09 09:56:11 +01:00
Łukasz Magiera
2fd0cb4aff fix: sealing: Avoid nil dereference in debug log 2022-12-09 09:54:28 +01:00
Łukasz Magiera
7b5e8d1e96
Merge pull request #9748 from filecoin-project/feat/metrics-tag-storage-use 
feat: metrics: Tag storage path types
 2022-11-29 18:06:23 +01:00
Ian Davis
0a4efd4820 Merge branch 'main' into id/update-go-version 2022-11-29 14:29:11 +00:00
Łukasz Magiera
57c31a79e7 feat: metrics: Tag storage path types 2022-11-29 13:06:56 +01:00
Łukasz Magiera
1597e85906 sched: Address review 2022-11-29 11:46:51 +01:00
Łukasz Magiera
5a458a60c2 fix sched_test 2022-11-29 11:44:57 +01:00
Łukasz Magiera
5881edb75e feat: sched: Cache worker calls 2022-11-29 11:44:57 +01:00
Łukasz Magiera
d82b2a5804
Merge pull request #9738 from filecoin-project/feat/sched-cycle-metrics 
feat: sched: Add metrics around sched cycle
 2022-11-28 23:57:58 +01:00
Łukasz Magiera
285cc66773
Merge pull request #9648 from filecoin-project/fix/snap-abort-cleanup 
fix: sealing: More complete snapdeals abort cleanup
 2022-11-28 23:26:32 +01:00
Łukasz Magiera
8ba4355cab
Merge pull request #9694 from filecoin-project/9159-allow-lotus-worker-to-unseal-sector-even-if-does-not-store-unsealed-data-long-term 
fix: sealing: Set all path types for Unseal pipeline to sealing storage
 2022-11-28 20:15:17 +01:00
Łukasz Magiera
c0fce07889 sched: Metrics around scheduldng workload 2022-11-28 19:51:18 +01:00
Łukasz Magiera
bc879c5bd8 feat: sched: Add metrics around sched cycle 2022-11-28 19:07:35 +01:00
Ian Davis
9f85d3dca7 Address simple linter issues 2022-11-24 16:32:27 +00:00
Shrenuj Bansal
fa596eee5b Move unsealed sector from sealing to long term storage after Unseal 2022-11-23 15:22:14 -05:00
Aayush
355a08cd04 fix: sealer: Tweak confidence for handleUpdateActivating 2022-11-23 14:38:57 -05:00
Łukasz Magiera
ea9a8303c8 sealing: tasks: TTReleaseUnsealed -> TTFinalizeUnsealed 2022-11-23 18:57:17 +01:00
Łukasz Magiera
d6f73c1a81 sealing: Address snap cleanup review 2022-11-23 18:57:17 +01:00
Łukasz Magiera
dc6c2b68fe sealing: Fix tests after finalize changes 2022-11-23 18:57:16 +01:00
Łukasz Magiera
87e5549af5 sealing: Split unsealed cleanup from Finalize 2022-11-23 18:57:16 +01:00
Łukasz Magiera
0bba2bd1ba sealing: Use FinalizeSector in snap abort for cleanup 2022-11-23 18:57:16 +01:00
Łukasz Magiera
211712bf6d itests: Test snapdeals abort cleanup 2022-11-23 18:57:16 +01:00
Shrenuj Bansal
e4ce056ac6 Set all path types for Unseal pipeline to sealing storage 2022-11-21 14:40:50 -05:00
Łukasz Magiera
71a84bb092 feat: wdpost: Configurable pre-check timeouts 2022-11-17 18:25:30 +01:00
Łukasz Magiera
0e07fa94d0
Merge pull request #9613 from filecoin-project/feat/wdpost-robustness 
feat: storage: Force exit GenerateSingleVanillaProof on cancelled context
 2022-11-15 10:30:25 +01:00
Łukasz Magiera
e476c1ab17
storage: paths: Fix typo in proof error message 
Co-authored-by: Piotr Bochenek <pbochenek@worldremit.com>
 2022-11-14 21:04:18 +01:00
Łukasz Magiera
51e92c97c3
Merge pull request #9642 from filecoin-project/fix/snapsector-clear-ctime 
fix: sealing pipeline: Clear CreationTime when starting sector upgrade
 2022-11-14 20:52:58 +01:00
Łukasz Magiera
c79085e5b7
Merge pull request #9598 from filecoin-project/fix/ancient-snap-deals-terrible 
fix:sealing-fsm:wait mutable fsm state for immutable sector upgrade error
 2022-11-14 19:53:37 +01:00
Łukasz Magiera
efe2d8da8a sealing pipeline: Test clearing CreationTime 2022-11-14 18:46:50 +01:00
Łukasz Magiera
d8c1b67a22 fix: sealing pipeline: Clear CreationTime when starting sector upgrade 2022-11-14 18:36:26 +01:00
zenground0
087da50ecc Lint 2022-11-14 10:16:37 -05:00
zenground0
d86a68f713 Review Response 2022-11-14 10:07:22 -05:00
Łukasz Magiera
73aabdfeff
Merge pull request #9601 from filecoin-project/fix/fsm-release-dealq-on-snap-abort 
fix: sealing pipeine: Release assigned deals on snapdeals abort
 2022-11-14 11:07:33 +01:00
Aayush
69c55d807a Sealing manager: minor refactor 2022-11-09 09:09:59 -05:00
Łukasz Magiera
cbcd0d49ae storage: Log a detailed error on PoSt challenge read timeout 2022-11-09 12:49:52 +01:00
Łukasz Magiera
42d02dd448 feat: storage: Force exit GenerateSingleVanillaProof on cancelled context 2022-11-09 12:38:23 +01:00
zenground0
3a6aae3e10 Remove another panic 2022-11-08 14:39:22 +00:00
zenground0
10b7961a99 Correctly pass abort upgrade an error 2022-11-08 14:11:28 +00:00
zenground0
7f650ad7dc Go is hard 2022-11-08 12:30:53 +00:00
Łukasz Magiera
fcad93dc10 netbs: Fix lint 2022-11-08 09:37:43 +00:00
Łukasz Magiera
ec89424c42 make repo not depend on ffi 2022-11-08 09:37:43 +00:00
Łukasz Magiera
5f5cc794f0 make config not depend on ffi 2022-11-08 09:37:43 +00:00
Łukasz Magiera
33f7746296 fix: sealing pipeine: Release assigned deals on snapdeals abort 2022-11-07 16:50:56 +00:00
zenground0
c902218baf Respect context 2022-11-07 16:03:22 +00:00
zenground0
6dee0d8ca8 Rough draft no tests 2022-11-07 14:56:53 +00:00
Aayush
c0b7343e60 Merge branch 'release/v1.18.0' into asr/merge-release-into-master 2022-11-06 14:40:13 -05:00
Łukasz Magiera
32de871243 fix: post worker sched: Don't check worker session in a busy loop 2022-10-14 15:58:25 +02:00
Aayush
65e37c29e4 address review 2022-10-12 13:54:04 -04:00
Łukasz Magiera
dc102f076a sealing pipeline: Get allocationId with StateApi 2022-10-12 13:54:03 -04:00
Łukasz Magiera
0af9888b12 sealing pipeline: Prepare deal assigning logic for FIP-45 2022-10-12 13:42:53 -04:00
Aayush
97b04b7f77 More tweaking of PreSeal for genesis 2022-10-06 11:06:21 -04:00
Aayush Rajasekaran
6510e5f129 Tweak PreSeal struct to be infra-friendly 2022-10-06 11:06:21 -04:00
Geoff Stuart
b4c04ad927 update markets 2022-10-06 11:06:21 -04:00
Shrenuj Bansal
4904411ded Address comments 2022-10-05 15:37:04 +00:00
Shrenuj Bansal
6696de6bab Add ability to only have single partition per msg for partitions with recovery sectors 2022-10-05 15:36:41 +00:00
Shrenuj Bansal
9653584d73 Address comments 2022-10-04 19:21:55 +00:00
Shrenuj Bansal
dc247cc57d Add ability to only have single partition per msg for partitions with recovery sectors 2022-10-04 18:33:18 +00:00
Aayush
32670e810c chore: refactor: rename NewestNetworkVersion 2022-09-21 15:48:51 -04:00
Łukasz Magiera
39af6e0365
Merge pull request #9328 from filecoin-project/fix/ffiwrap-close-ap 
fix: ffiwrapper: Close readers in AddPiece
 2022-09-19 17:47:15 +02:00
Łukasz Magiera
66ad2b8102 sealing: Avoid panicking in handleUpdateActivating on startup 2022-09-19 17:32:02 +02:00
Łukasz Magiera
015139d451 sealing pipeline: Clarify maybeNotifyRemoteDone 2022-09-19 12:16:11 +02:00
Łukasz Magiera
859c2606f0 sealing: Address review 2022-09-19 12:13:06 +02:00
Łukasz Magiera
a05593da5b systar: Create file after header checks 2022-09-19 12:13:06 +02:00
Łukasz Magiera
fec9c0f981 sector import: Fix lint 2022-09-19 12:13:06 +02:00
Łukasz Magiera
061a990eb8 sector import: RemoteSealingDoneEndpoint 2022-09-19 12:13:06 +02:00
Łukasz Magiera
474c60203b tarutil: Strict header checks 2022-09-19 12:13:06 +02:00
Łukasz Magiera
c17ab761c5 sector import: Remote Commit2 2022-09-19 12:13:06 +02:00
Łukasz Magiera
142894895d sector import: Implement seed/ticket/commr/commd checks 2022-09-19 12:13:06 +02:00
Łukasz Magiera
b2dfaae68c sector import: Test remote commit1 retry 2022-09-19 12:13:06 +02:00
Łukasz Magiera
9f03569cd0 sector import: Remote Commit1 2022-09-19 12:13:06 +02:00
Łukasz Magiera
830c2ffdf5 sector import: FinalizeInto, send finalized cacde in itest 2022-09-19 12:13:06 +02:00
Łukasz Magiera
9c6d531ae7 sector import: Plumb sector download task canfig 2022-09-19 12:13:06 +02:00
Łukasz Magiera
12a8ab5ac7 sector import: Add missing initial fsm state transition 2022-09-19 12:13:06 +02:00
Łukasz Magiera
fef7232c6b sector import: Initial SectorReceive itest 2022-09-19 12:13:06 +02:00
Łukasz Magiera
3caa010779 sector import: Update type-gen 2022-09-19 12:13:06 +02:00
Łukasz Magiera
cdadf5e5a9 sector import: Implement handleReceiveSector 2022-09-19 12:13:05 +02:00
Łukasz Magiera
cde23be559 sector import: Fix tests 2022-09-19 12:13:05 +02:00
Łukasz Magiera
ea99bd9763 sector import: Sector data download 2022-09-19 12:13:05 +02:00
Łukasz Magiera
fbb487ae2b sector import: Plumbing for DownloadSectorData in the sealing system 2022-09-19 12:13:05 +02:00
Łukasz Magiera
39e4845f42 sector import: Put the imported sector into the sealing pipeline 2022-09-19 12:13:05 +02:00
Łukasz Magiera
2b644525f8 sector import: More check progress 2022-09-19 12:13:05 +02:00
Łukasz Magiera
ca790b4c69 sealing: Validate piece alignment in checkPieces 2022-09-19 12:13:05 +02:00
Łukasz Magiera
332d83bd18 sector import: Check pieces in Sealing.Receive 2022-09-19 12:13:05 +02:00
Łukasz Magiera
5c485c3375 sealing: Drop redundant Piece type 2022-09-19 12:13:05 +02:00
Łukasz Magiera
29135aa77c sector import: Initial api scaffolding 2022-09-19 12:13:03 +02:00
Łukasz Magiera
d68bb937d8 fix: ffiwrapper: Close readers in AddPiece 2022-09-19 09:52:57 +02:00
Jiaying Wang
4abc38dacc
Merge pull request #9310 from filecoin-project/fix/empty-snap 
fix: sealing: Abort upgrades in sectors with no deals
 2022-09-16 21:21:39 -04:00
Łukasz Magiera
f1e4ecaaa3 fix: Drop unused PreCommitInfo from pipeline.SectorInfo 2022-09-16 17:51:09 +02:00
Łukasz Magiera
501d21aa28 fix expiration config handling in calcTargetExpiration 2022-09-16 15:37:50 +02:00
Łukasz Magiera
7a2a069a5a
Merge pull request #9282 from Juneezee/refactor/os.ReadDir 
refactor: use `os.ReadDir` for lightweight directory reading
 2022-09-15 16:25:51 +02:00
Łukasz Magiera
bf3daea124 sealing: Minimum upgrade sector expiration/target config 2022-09-15 16:19:57 +02:00
Łukasz Magiera
305cfa1f69 sealing: Pick safer minTarget in calcTargetExpiration 2022-09-15 16:09:13 +02:00
Łukasz Magiera
d2c726cd28 fix: sealing: Abort upgrades in sectors with no deals 2022-09-15 16:09:13 +02:00
Aayush
19c3232d2f more fixes 2022-09-10 12:37:03 -04:00
Aayush
c0bbaebaf4 more fixes 2022-09-09 21:40:15 -04:00
Aayush
92cc24fca2 support fvm gas tracing 2022-09-09 21:40:15 -04:00
Geoff Stuart
642555d0fb Fix discrepancies, run fiximports 2022-09-09 21:40:15 -04:00
Aayush
0efca4d266 introduce v9 actors and nv17 2022-09-09 21:40:15 -04:00
Eng Zer Jun
abef90423d
refactor: use os.ReadDir for lightweight directory reading 
`os.ReadDir` was added in Go 1.16 as part of the deprecation of `ioutil`
package. It is a more efficient implementation than `ioutil.ReadDir`.

Reference: https://pkg.go.dev/io/ioutil#ReadDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2022-09-09 19:53:40 +08:00
LexLuthr
67d4f905e6
feat: proving: Introduce manual sector fault recovery (#9144) 
* rebase

* fix lint error

* fix errors

* add itest

* make gen after rebase

* apply suggestion from review

* make gen after latest rebase

* gen after rebase

* fix waitgroup

* change as per review

* refactor Recovery Batches

* fix CLI

* gen after rebase

* close the channel
 2022-09-06 21:17:30 +05:30
Łukasz Magiera
088bf56f2a
Merge pull request #9231 from filecoin-project/fix/datacid-res-envvars 
sealing: fix: Make DataCid resource env vars make more sense
 2022-08-30 02:36:02 +02:00
Łukasz Magiera
08b22edd89 fix make gen 2022-08-29 16:25:30 +02:00
Łukasz Magiera
d00e1e78bb resources: Use 32G AP resources for DC 2022-08-29 16:12:02 +02:00
Łukasz Magiera
e4ed3f8cf8 sealing: fix: Make DataCid resource env vars make more sense 2022-08-29 15:57:58 +02:00
Dirk McCormick
964bbca792 fix: DataCid - ensure reader is closed 2022-08-29 10:31:11 +02:00
Łukasz Magiera
45d1bd61ce
Merge pull request #9183 from filecoin-project/feat/sectornum-mgmt 
feat: sealing: Use bitfields to manage sector numbers
 2022-08-26 10:59:24 -04:00
Łukasz Magiera
70f0edd041 strle, sealing: address review 2022-08-25 10:19:39 -04:00
Łukasz Magiera
b8a8648f4a numassigner: fix lint 2022-08-24 11:38:29 -04:00
Łukasz Magiera
335f5df3ec numassigner: SectorNumReserveCount 2022-08-24 11:25:37 -04:00
Aayush
5e80276631 fix: mock sealer: grab lock in ReadPiece 2022-08-23 17:24:28 -04:00
nujz
76f6eb3edf cgroupV1: memory.memsw.usage_in_bytes: no such file or directory 2022-08-23 17:37:47 +08:00
Łukasz Magiera
c18874c122 numassigner: Don't error on fresh non-genesis miners 2022-08-22 17:49:58 -04:00
Łukasz Magiera
129eba2b5d sealing: numassign: itests/bug fixes 2022-08-22 16:58:36 -04:00
Łukasz Magiera
00bef607ca sealing: numassign: take on-chain allocated numbers into account 2022-08-22 16:55:44 -04:00
Łukasz Magiera
ef2080a800 cli for managing sector reservations 2022-08-22 16:55:41 -04:00
Łukasz Magiera
ca72590e49 sealing: Use bitfields to manage sector numbers 2022-08-22 16:55:08 -04:00
Łukasz Magiera
2e29b460d2
Merge pull request #9142 from filecoin-project/chore/untangle-fsm 
sealing pipeline: Remove useless storage adapter code
 2022-08-14 15:50:31 -04:00
Łukasz Magiera
77208728ad sealing: fix lint/tests 2022-08-14 12:50:18 -04:00
Rob Quist
f5086c98e8
fix: Add information on what worker a job was assigned to in logs 
More detailed logs
 2022-08-10 21:33:19 +02:00
Łukasz Magiera
71dacb5af8 sealing pipeline: Remove useless storage adapter code 2022-08-09 13:40:06 +02:00
Łukasz Magiera
fe2a589890 sealing pipeline: Cleanup AddresSelector plumbing 2022-08-09 12:57:36 +02:00
Łukasz Magiera
4b6a9e0387 sealing pipeline: Cleanup journal plumbing 2022-08-09 12:42:35 +02:00
Łukasz Magiera
ca1ff19541 fix: storage: don't panic in getCommitCutoff when precommit is not found 2022-08-09 11:44:40 +02:00
Łukasz Magiera
cb2b4fa4aa Merge remote-tracking branch 'origin/master' into feat/storage-redeclare 2022-08-05 21:26:15 +02:00
Łukasz Magiera
819314de46 Merge remote-tracking branch 'origin/master' into feat/storage-redeclare 2022-08-05 09:01:30 +02:00
LexLuthr
a59d73d64a
Merge branch 'master' into feat/SchedRemoveRequest 2022-08-04 02:24:14 +05:30
LexLuthr
fad79f3218 respond to caller. Add itest 2022-08-04 02:00:24 +05:30
LexLuthr
cdc08e566f fix everything pointed as per review 2022-08-04 02:00:24 +05:30
LexLuthr
c736dedfa6 move requestremove inside runSched 2022-08-04 02:00:24 +05:30
LexLuthr
cf78fa99ee Rebase on master 2022-08-04 01:59:48 +05:30
Łukasz Magiera
da33d82e1b feat: sealer: Custom worker name config 2022-08-03 12:58:49 +02:00
Łukasz Magiera
4d10adae3c Merge branch 'hostname_variable' of github.com:clinta/lotus into feat/worker-name-set 2022-08-03 12:40:18 +02:00
Łukasz Magiera
4b47881950 Merge branch 'master' into sbansal/lotus-worker-stop-cmd 2022-08-03 10:42:19 +02:00
Łukasz Magiera
92d27aa6d2 Merge remote-tracking branch 'origin/master' into feat/storage-redeclare 2022-08-03 00:56:23 +02:00
Aayush
784926b2b4 add a test for Post max sectors recovery 2022-08-02 15:28:43 -04:00
Aayush
50c22aef7a fix: post: restrict recoveries per deadline 2022-08-02 12:32:44 -04:00
Łukasz Magiera
6d2990314f storage: Fix detaching shared paths 2022-08-02 15:46:35 +02:00
Aayush
650130f9c4 fix: sector counter for recovery 2022-08-01 14:30:20 -04:00
Łukasz Magiera
c215a03edd feat: wdpost: Envvar for limiting recovering sectors 2022-08-01 16:58:05 +02:00
Łukasz Magiera
3309e10b0d itests: Test storage detach/redeclare 2022-08-01 15:58:06 +02:00
Łukasz Magiera
d690b49d17 storage: Fix linter warnings 2022-08-01 15:58:06 +02:00
Łukasz Magiera
8cff52aef6 Storage detach/attach in lotus-miner, cli commands 2022-08-01 15:58:06 +02:00
Łukasz Magiera
943d2a72c6 storage: StorageRedeclareLocal on lotus-worker 2022-08-01 15:58:06 +02:00
Łukasz Magiera
f947e65f15 wip storage redeclare/detach 2022-08-01 15:58:06 +02:00
Shrenuj Bansal
dfea74cca8 Add stop cmd for lotus worker 2022-07-29 18:33:30 -04:00
Geoff Stuart
4b79cc68cb
Merge pull request #9043 from filecoin-project/gstuart/hacky-sealing-fix 
fix: sealing: Workaround for sealing bug
 2022-07-18 13:54:03 -07:00
Geoff Stuart
b22e24a95a Workaround for sealing bug 2022-07-15 11:56:08 -07:00
Aayush Rajasekaran
3353cbe23a Fixup typos 2022-07-15 08:20:37 -04:00
Łukasz Magiera
db9105e166 storage: Add one more test case to TestFileTypeAnyAllow 2022-07-15 12:47:46 +02:00
Łukasz Magiera
e1b02dd13d
storage: paths: Review suggestions 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2022-07-15 12:46:20 +02:00
Łukasz Magiera
ac2ea039b5 path index: Raise alerts on bad filter config 2022-07-12 21:45:12 +02:00
Łukasz Magiera
91e474e72e sealer: Pass correct args to SubAllowed in selectors 2022-07-12 21:45:12 +02:00
Łukasz Magiera
43db2fb8a4 miner cli: Print allow/deny types in storage list 2022-07-12 21:45:12 +02:00
Łukasz Magiera
59d069dc4d itests: Tests for storage path filters 2022-07-12 21:45:12 +02:00
Łukasz Magiera
cd682e200b sealer: Fix no-remote move selector 2022-07-12 21:45:12 +02:00
Łukasz Magiera
6ac5c16d2b storage: Path type filters 2022-07-12 21:45:12 +02:00
Jennifer Wang
8486ae0d51 Merge branch 'releases' into jen/backport 2022-07-11 14:27:26 -04:00
Aayush
c4ad7c81e2 remove mock file 2022-07-07 14:51:21 -04:00
Aayush
d68a60e8d9 feat: recovery: Config for maximum partition count per message 2022-07-07 14:48:37 -04:00
Łukasz Magiera
673f9238be feat: wdpost: Config for maximum partition count per message 2022-07-07 14:10:31 -04:00
Aayush
770c32c3cb feat: recovery: Config for maximum partition count per message 2022-07-07 12:46:33 -04:00
Łukasz Magiera
7741175b7d feat: wdpost: Config for maximum partition count per message 2022-07-07 12:38:21 +02:00
Łukasz Magiera
d5100f883d cfgdoc-gen: Handle empty lines in comments 2022-07-01 22:32:10 +02:00
Łukasz Magiera
59f3161fd6 wdpost: Config for disabling sector prechecks 2022-07-01 22:20:05 +02:00
Łukasz Magiera
84881f64ad sealer: Config for disabling builtin PoSt 2022-07-01 21:24:54 +02:00
Łukasz Magiera
0277ff4239 wdpost: move fault/recover code to a separate file 2022-07-01 21:24:22 +02:00
Geoff Stuart
56c7e7a8d6 Remove old functions for getting actors cids 2022-06-29 14:58:58 -04:00
Łukasz Magiera
db676acdf8 storage: Cleanup interfaces a bit 2022-06-17 13:52:19 +02:00
Łukasz Magiera
58416d0881 storage: Don't depend on specs-storage, move to storiface 2022-06-17 13:36:33 +02:00
Łukasz Magiera
4a4d64f528 sealing pipeline: Rename 'tok' to 'tsk' 2022-06-17 11:20:33 +02:00
Łukasz Magiera
d56241603a sealing pipeline: Remove adapter code! 2022-06-16 17:22:50 +02:00
Łukasz Magiera
59a4fe1e56 sealing pipeline: Don't adapt MpoolPushMessage 2022-06-16 15:54:15 +02:00
Łukasz Magiera
9c4d10ec73 api: handle no-precommit in StateSectorPreCommitInfo gracefully 2022-06-16 15:20:58 +02:00
Łukasz Magiera
06b3e555c5 sealing pipeline: Drop ChainBaseFee, don't wrap StateSectorPartition 2022-06-16 14:00:37 +02:00
Łukasz Magiera
db842224b8 sealing pipeline: Drop StateMarketStorageDealProposal 2022-06-16 13:33:59 +02:00
Łukasz Magiera
2dc1df878d api: Move StateComputeDataCID to full-node 2022-06-16 13:29:35 +02:00
Łukasz Magiera
a084445e6c sealing pipeline: Use non-special ChainHead 2022-06-16 13:15:49 +02:00
Łukasz Magiera
048bfe6d5b sealing pipeline: Use non-special msg wait APIs 2022-06-16 12:47:19 +02:00
Łukasz Magiera
cd4514bddf sealing pipeline: Drop StateMinerActiveSectors 2022-06-16 12:34:29 +02:00
Łukasz Magiera
9fad80ce9a sealing pipeline: Drop unused miner-info related methods 2022-06-16 12:18:24 +02:00
Łukasz Magiera
9aa5659d24 sealing pipeline: Drop TipSetToken, use TipSetKey directly 2022-06-16 12:05:58 +02:00
Łukasz Magiera
05cdeb80c3 chore: remove redundant import prefixes 2022-06-15 12:06:22 +02:00
Łukasz Magiera
39cede946c chore: Fix more imports with '2' suffix 2022-06-14 21:37:54 +02:00
Łukasz Magiera
a581d73548 chore: Fix imports with '2' suffix 2022-06-14 20:36:33 +02:00
Łukasz Magiera
28099a3905 storage: Move storage/sealer/stores to storage/paths 2022-06-14 20:25:52 +02:00
Łukasz Magiera
82857e6d5d chore: rerun gen with moved packages 2022-06-14 20:13:31 +02:00
Łukasz Magiera
a9600b8a6f storage: Move extern/sector-storage to storage/sealer 2022-06-14 20:03:38 +02:00
Łukasz Magiera
98a48a47f8 storage: Move extern/storage-sealing to storage/pipeline 2022-06-14 19:41:59 +02:00
Łukasz Magiera
cf835fcc0c fix wdpost unit tests 2022-06-14 19:33:57 +02:00
Łukasz Magiera
717bda63b1 storage: Move control address logic to storage/ctladdr 2022-06-14 19:32:29 +02:00