cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
20,582 Commits 24 Branches 342 Tags 314 MiB
f2a4891ada
Commit Graph

1214 Commits

Author SHA1 Message Date
Andrew Jackson (Ajax)
a520ee85d6 master-to-sturdy 2023-09-21 10:37:02 -05:00
Shrenuj Bansal
00b3335765 wip 2023-09-20 15:59:58 -04:00
Łukasz Magiera
3cebd99d4c pipeline: debugging C2 with synth porep 2023-09-19 15:09:46 +02:00
Aayush
45eb890917 fix: test: temporarily exempt SynthPorep constants from test 2023-09-08 11:51:30 -04:00
Phi
024dd5e08e Merge branch 'feat/nv21' into feat/implement-SyntheticPoRep 2023-09-08 16:41:39 +02:00
Aloxaf
0dd7976ffb
fix: sealing: typo in FinalizeReplicaUpdate 2023-09-08 10:15:58 +08:00
Aayush Rajasekaran
8ab6eaa6e3
Merge pull request #11230 from filecoin-project/asr/test-seal-and-verify-3-needs-to-stop 
feat: skip TestSealAndVerify3 until it's fixed
 2023-09-06 14:48:38 -04:00
Mikers
5e5a81bf23
add go linter - "unused" (#11235) 
* add go linter - "unused"

* use _ to name unused but needed padding variable

* remove unused code

* add queue test to appease unused linter

* remove unused code in test

* remove unused func

* remove unused struct identified by linter

* remove unused variable

* remove unused code

* remove unused file

* remove unused struct

* remove unused function

* remove unused observe peers function in raft

* remove unused declareFaults function

* annotate nolint:unused on needed methods
 2023-09-05 12:19:43 -10:00
Andrew Jackson (Ajax)
b73c016f6d unseal - comments 2023-09-01 17:15:56 -05:00
Aayush
b92cff74f2 feat: skip TestSealAndVerify3 until it's fixed 2023-09-01 10:08:55 -04:00
Łukasz Magiera
077f3f0c33 ffiwrapper: Fix sector key regen 2023-08-31 09:46:10 -04:00
Łukasz Magiera
8ea44e82d1 ffiwrapper: Correctly create unsealed file when doesn't exist 2023-08-31 09:46:10 -04:00
Łukasz Magiera
e74eb6af09 make the linter happy 2023-08-31 09:46:10 -04:00
Jennifer Wang
ad6b0575c1 make gen, mod tidy 2023-08-31 09:46:10 -04:00
Łukasz Magiera
a965fc660e fix build 2023-08-31 09:46:10 -04:00
Jennifer Wang
4b78c4ea1e ffiwrapper: Fix sector acquire logic in unsealing 2023-08-31 09:46:10 -04:00
Łukasz Magiera
6167bcc8ce ffiwrapper: Fix sector acquire logic in unsealing 2023-08-31 09:46:10 -04:00
Łukasz Magiera
fa0f117edd ffiwrapper: Fix multiple bugs in snapdeals unsealing 2023-08-31 09:46:10 -04:00
Łukasz Magiera
bcc2d7fd10 ffiwrapper: Fix RegenerateSectorKey 2023-08-31 09:46:10 -04:00
Jennifer Wang
b0b39424e4 ffiwrapper: Test lower-level GenerateSDR 2023-08-31 09:46:10 -04:00
Shrenuj Bansal
9cc5e84677 wip 2023-08-30 12:40:54 -04:00
Shrenuj Bansal
c0bd13eced WIP 2023-08-30 12:11:09 -04:00
Shrenuj Bansal
6b95bd24ba WIP: wdpost listener 2023-08-29 12:28:44 -04:00
Łukasz Magiera
a9396624a7
Merge pull request #11215 from filecoin-project/backport/nv21/feat/act-precommv2 
Backport: feat: sealing: Switch to calling PreCommitSectorBatch2
 2023-08-29 16:06:27 +02:00
Łukasz Magiera
4ee20965fa chore: chain/actors: Use type proxies instead of versioned GST imports 2023-08-29 15:28:16 +02:00
Łukasz Magiera
2ebd080d5e itests: Port deadline-toggling itest to nv20 2023-08-29 13:47:20 +02:00
Łukasz Magiera
3887bf97a4 sealing pipeline: Only set pc commd when deals are present 2023-08-29 13:47:20 +02:00
Łukasz Magiera
556815f844 sealing pipeline: Set commd in precommit params 2023-08-29 13:47:20 +02:00
Łukasz Magiera
1c447b265b sealing pipeline: Fix unit tests 2023-08-29 13:47:18 +02:00
Łukasz Magiera
ecb6deea09 sealing pipeline: Drop legacy single-pc1 submit path 2023-08-29 13:46:29 +02:00
Łukasz Magiera
d6aed7ba83 pc batch: drop individual precommit processing 2023-08-29 13:46:29 +02:00
Łukasz Magiera
7db6f31cca sealing: pc batch: Call PreCommitSectorBatch2 2023-08-29 13:46:29 +02:00
Phi
37f6cd3ca5 Register SynthPoRep in ResourceTable 
Register SynthPoRep in ResourceTable such that task-resource limitations are also applied to SynthPoRep-sectors (i.e PC1_32G_MAX_CONCURRENT)
 2023-08-29 13:08:00 +02:00
Łukasz Magiera
f8faa85cba make gen 2023-08-28 16:12:14 +02:00
Łukasz Magiera
d83956aa6d ci: Ensure params in sealer tests 2023-08-28 13:43:50 +02:00
Łukasz Magiera
bce5535679
Merge pull request #11182 from filecoin-project/TestSealerDLBefore 
Take Download out of Sealer time
 2023-08-25 10:27:42 +02:00
Łukasz Magiera
366329b085
Merge pull request #11087 from filecoin-project/fix/pc1-retr-loop 
fix: sealing pipeline: Fix PC1 retry loop
 2023-08-24 11:08:39 +02:00
Łukasz Magiera
882f00fbf3 sealing: retrySoftErr: Better comment on ctx handling 2023-08-23 17:29:18 +02:00
Shrenuj Bansal
15247486ae anotha one 2023-08-22 12:19:18 -04:00
Shrenuj Bansal
068416ecfc experiment 2023-08-22 11:48:38 -04:00
Shrenuj Bansal
459255d1f0 make linter happy 2023-08-22 11:06:16 -04:00
Shrenuj Bansal
a51e55390a address moar comments 2023-08-22 10:43:50 -04:00
Shrenuj Bansal
95709a0583 fix a couple of bugs and address comments 2023-08-21 11:54:51 -04:00
Andrew Jackson (Ajax)
c48bf864a4 Take Download out of Sealer time 2023-08-17 22:34:08 -05:00
Shrenuj Bansal
9262eb4fc9 change table names and address other comments 2023-08-17 13:23:03 -04:00
Andrew Jackson (Ajax)
76bca6c392 mac seal test 2023-08-17 07:30:19 -05:00
Shrenuj Bansal
169f953de4 add unlock 2023-08-16 15:36:00 -04:00
Shrenuj Bansal
cdf90b83ed address more comments 2023-08-16 10:57:08 -04:00
Shrenuj Bansal
28efd718c7 Remove commented out code 2023-08-15 12:17:23 -04:00
Shrenuj Bansal
854d3c80dd add stake lock handling 2023-08-14 21:47:27 -04:00
Shrenuj Bansal
16fb7cc0da Implement Sector Index locks APIs 2023-08-14 18:46:41 -04:00
Łukasz Magiera
33182967eb
Merge pull request #11142 from filecoin-project/feat/act-precommv2 
sealing: Switch to calling PreCommitSectorBatch2
 2023-08-14 21:31:24 +02:00
Łukasz Magiera
b11fba3561 itests: Port deadline-toggling itest to nv20 2023-08-14 20:25:36 +02:00
Shrenuj Bansal
e4c2bfc150 i hate this linter 2023-08-14 13:56:29 -04:00
Shrenuj Bansal
d746d6e6df make more lint more happy hopefully 2023-08-14 11:35:18 -04:00
Shrenuj Bansal
5161528399 make lint happy 2023-08-14 09:24:00 -04:00
Shrenuj Bansal
7759444247 address review comments 2023-08-10 18:35:35 -04:00
Shrenuj Bansal
84d9dc5dfe fix bigint overflow bug 2023-08-09 15:27:12 -04:00
Shrenuj Bansal
652ee04ce2 working changes 2023-08-08 20:59:21 -04:00
nemo
c6d31c0d6d fix: downgrade harmless warning to debug 2023-08-08 10:52:35 -04:00
Łukasz Magiera
98aff9dcc6 sealing pipeline: Only set pc commd when deals are present 2023-08-08 15:35:19 +02:00
Łukasz Magiera
40cfe626b8 sealing pipeline: Set commd in precommit params 2023-08-08 15:12:50 +02:00
Łukasz Magiera
41fac326b1 sealing pipeline: Fix unit tests 2023-08-08 15:06:43 +02:00
Łukasz Magiera
a4d22f585f sealing pipeline: Drop legacy single-pc1 submit path 2023-08-08 14:43:00 +02:00
Łukasz Magiera
1c73ac69b4 pc batch: drop individual precommit processing 2023-08-08 13:04:21 +02:00
Łukasz Magiera
ef2be0b546 sealing: pc batch: Call PreCommitSectorBatch2 2023-08-08 12:40:08 +02:00
Łukasz Magiera
884246677b
Merge pull request #11046 from filecoin-project/fix/fsm-commfail-loop 
fix: sealing: Fix RetryCommitWait loop when sector cron activation fails
 2023-08-07 12:11:57 -07:00
Shrenuj Bansal
864e8219ae WIP: sector index yugabyte impl 2023-08-05 16:15:23 -04:00
Mladen Trišić
9459c612fb sector import: fix evaluating randomness when importing a sector passing SubmitCommit state 2023-08-01 12:05:39 +02:00
Łukasz Magiera
0a83896589 sealing pipeline: Better retry for soft errors 2023-07-19 09:51:59 -07:00
Łukasz Magiera
d127208b85 sealing pipeline: Fix PC1 retry loop 2023-07-18 13:04:06 -07:00
Łukasz Magiera
85b78f9ffd update ffi to use lowercased synth challenges file name 2023-07-13 15:48:20 +02:00
Łukasz Magiera
a39694741e ffiwrapper: Test synth porep 2023-07-13 15:48:20 +02:00
Łukasz Magiera
50cf19722a Make Synthetic PoRep work 2023-07-13 15:48:20 +02:00
Łukasz Magiera
518c2cdf6e rename synth porep config to a shorter name 2023-07-13 15:48:16 +02:00
Łukasz Magiera
c7dd001c64 ffiwrapper: Synth cleanup fixes 2023-07-13 15:48:16 +02:00
Andrew Jackson (Ajax)
7f306a5b02 Synthetic PoRep Impl 2023-07-13 15:47:59 +02:00
Andrew Jackson (Ajax)
5ed8a5c0ac clearSynthProofs 2023-07-13 15:47:59 +02:00
Andrew Jackson (Ajax)
08038540e4 Impl synthPoRep 2023-07-13 15:47:59 +02:00
Andrew Jackson (Ajax)
f4fe4cae9c synthetic porep config 2023-07-13 15:47:59 +02:00
Łukasz Magiera
9b52b06ea9 fix: sealing: Fix RetryCommitWait loop when sector cron activation fails 2023-07-04 13:35:49 +02:00
Phi
8220e7f82f Tune down log-lvl 
Tune down log-lvl
 2023-07-04 09:35:49 +02:00
Łukasz Magiera
c79acf9ca4 sealing: Use only non-assigned deals when selecting snap sectors 2023-06-23 18:40:46 +02:00
Jorropo
6c01310728
chore: migrate to boxo 
This migrates everything except the `go-car` librairy: https://github.com/ipfs/boxo/issues/218#issuecomment-1529922103

I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups.

This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649

The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions.

Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent https://github.com/golang/go/issues/58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type.

Stubbing types were reverted https://github.com/ipfs/boxo/issues/218#issuecomment-1478650351

Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
 2023-06-19 14:45:05 -07:00
Aayush Rajasekaran
ce5e242d76
Merge pull request #10915 from filecoin-project/fix/upgrade-sector-selection 
sealing: Improve upgrade sector selection
 2023-05-30 13:32:52 -04:00
Łukasz Magiera
17915dce5a sealing: Don't select sectors with no deals lined up for upgrading 2023-05-27 13:03:21 +02:00
Łukasz Magiera
4b5a665422 Merge remote-tracking branch 'origin/master' into feat/piecereader-perf 2023-05-26 13:00:42 +02:00
Łukasz Magiera
090b98ad33 sealing: Attempt to improve upgrade sector selection 2023-05-26 12:31:35 +02:00
Łukasz Magiera
b58daf5340 piecereader: address review 2023-05-25 16:05:36 +02:00
Łukasz Magiera
5e58f64380 piecereader: Fix double buffer free 2023-05-23 16:45:46 +02:00
Łukasz Magiera
f0fec13ec9 piecereader: Metrics fixes 2023-05-23 14:15:46 +02:00
Łukasz Magiera
326a0356a4 piecereader: Add metrics 2023-05-23 13:32:17 +02:00
Łukasz Magiera
6c62e6d8e2 fix tests/lint 2023-05-23 12:26:13 +02:00
Łukasz Magiera
587c3fde58 piecereader: Fix parallel buffer access, close readers in readInto 2023-05-23 12:26:13 +02:00
Łukasz Magiera
cd75ea0fe4 piecereader: Retune to allow parallel ReadAt calls 2023-05-23 12:26:13 +02:00
Łukasz Magiera
8b2ef40f4e storage: Weave end offset into unsealed reader code 2023-05-23 12:26:13 +02:00
Łukasz Magiera
6fd93ed170
fix: sched: Address GET_32G_MAX_CONCURRENT regression (#10850) 
* Fix 1.21 regression: GET_32G_MAX_CONCURRENT + mixed prepared/executing leads to stuck scheduler

If you have 12 GET tasks and GET_32G_MAX_CONCURRENT=1, sealing jobs will only show assigned tasks for GET of the miner
and is stuck.
I believe this to be a regression of 1.21 unifying the counters, in the case of GETs where PrepType and TaskType
both being seal/v0/fetch leading to a state where tasks are blocked since already counted towards the limit.

* itests: Repro issue from PR #10633

* make counters int (non-working)

* fix: worker sched: Send taskDone notifs after tasks are done

* itests: Make TestPledgeMaxConcurrentGet actually reproduce the issue

* make the linter happy

---------

Co-authored-by: Steffen Butzer <steffen.butzer@outlook.com>
 2023-05-10 15:43:42 -04:00
Łukasz Magiera
2f469c2f58 fix: tests: Use mutex-wrapped datastore in storage tests 2023-05-09 18:06:41 +02:00
Shrenuj Bansal
78800a4e77
fix: sealing: Make lotus-worker report GPU usage to miner during ReplicaUpdate task (#10806) 
* Make lotus-worker report GPU usage to miner during ReplicaUpdate task

* make gen

---------

Co-authored-by: Jacob Crowther <jcrowthe@users.noreply.github.com>
 2023-05-03 12:28:03 -04:00
Łukasz Magiera
9d2d53b58e fix: prover: Propagate skipped sectors in local PoSt 2023-04-27 09:26:12 +02:00