Commit Graph

915 Commits

Author SHA1 Message Date
Łukasz Magiera
09cfad9d71 Add FinalizeReplicaUpdate into some more places 2022-02-08 17:22:41 +01:00
Łukasz Magiera
e271bae5ec try ClearCache for update cache 2022-02-08 17:22:41 +01:00
Łukasz Magiera
142ba6660a wip FinalizeReplicaUpdate 2022-02-08 17:22:41 +01:00
Aayush Rajasekaran
f476aa937e update to latest FFI 2022-02-08 10:45:58 -05:00
Darko Brdareski
e51ce5c508 Merge remote-tracking branch 'upstream/master' into bloxico/system-test-matrix 2022-01-27 10:57:56 +01:00
Aayush Rajasekaran
2d0929e305
remove a log 2022-01-25 12:55:56 -05:00
Aayush Rajasekaran
92e6f29cc8
chore: sealer: quieten a log 2022-01-24 18:28:52 -05:00
Aayush
5cafdc2f29 fix: sealer: manager should lock Unsealed for ReplicaUpdate 2022-01-21 11:12:12 -05:00
Aayush Rajasekaran
ff845aa793
Merge pull request #7977 from filecoin-project/chore/sealtasks-comment
chore: remove inaccurate comment in sealtasks
2022-01-21 10:53:04 -05:00
Aayush
752f4a3d67 Fix: sealer: ReplicaUpdate should fetch the correct files 2022-01-20 15:06:53 -05:00
Aayush Rajasekaran
3ff23ecbfa :fix: checkReplica incorrectly returns ErrBadPR 2022-01-19 12:00:27 -05:00
Aayush Rajasekaran
ab8bf393c2 create replica update paths in acquireSectors 2022-01-19 11:41:38 -05:00
Aayush Rajasekaran
d0390181ec feat: sealing: Add ReplicaUpdate work to Resource table 2022-01-19 11:41:32 -05:00
Łukasz Magiera
c41ccb6c37
chore: remove inaccurate comment in sealtasks 2022-01-19 10:46:37 +01:00
Aayush Rajasekaran
bda4e5be95 Appease the linter 2022-01-12 18:10:07 -05:00
Jennifer Wang
fd50cd128a fix lint 2022-01-11 18:34:26 -05:00
Jennifer Wang
e99b98873c Check piece before PRU2 instead of PRU1 as PRU2 is the heavy computation part 2022-01-11 18:34:20 -05:00
Jennifer Wang
6b953a03d0 just use checkPiece 2022-01-11 18:34:15 -05:00
Jennifer Wang
37a3e610b7 Add more deal expiration handling for snap deals 2022-01-11 18:34:10 -05:00
Jiaying Wang
0130b28879
Merge pull request #7923 from filecoin-project/jen/proofv11
chore: deps: Integrate proof v11.0.0
2022-01-11 17:49:43 -05:00
Aayush Rajasekaran
d645c5fbab Remove unnecessary params from VerifyWinningPost 2022-01-11 12:06:39 -05:00
Jennifer Wang
5b7da270c9 Integrate proof v11.0.0 2022-01-10 23:45:04 -05:00
zenground0
d16c5d0e93 Fix hande deal recover return value bug 2022-01-10 15:47:20 +05:30
zenground0
c309686679 Fix TooManyMarkedForUpgrade 2022-01-10 15:39:38 +05:30
zenground0
33f2d24f54 Snap Deals Integration
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
2022-01-10 15:39:38 +05:30
Darko Brdareski
dda1a42a2a Merge branch 'bloxico/system-test-matrix' of https://github.com/filecoin-project/lotus into merge_lotus 2021-12-20 15:48:16 +01:00
shotcollin
d10d0a20b1
fix typo in log warning
very minor but this warning comes up a lot so it'd be nicer if it wasn't a grammatical error too
2021-12-19 17:07:11 -07:00
Darko Brdareski
2f1f35cc71 Annotate storage miner features 2021-12-15 15:30:42 +01:00
Aayush Rajasekaran
3e288f1066 Update FFI 2021-12-13 15:47:17 -05:00
Aayush Rajasekaran
80d5e52923 Merge branch 'master' into next 2021-12-13 13:24:28 -05:00
Darko Brdareski
0169d0dafd Annotate state feature tests 2021-12-10 16:08:25 +01:00
Łukasz Magiera
e8ef39e734 stores: Reduce log spam during retrievals 2021-12-10 11:28:04 +01:00
Łukasz Magiera
c31f4de7d5 Fix mock ReadPiece 2021-12-09 16:26:59 +01:00
Łukasz Magiera
6fd1609410 fr32: Reduce MTTresh from 32M to 512k per core
This results in 64x less bytes allocated when spawning new readers
for larger pieces.

Results in about 30% speedup in 1G unpad benchmark on AMD TR 2950x
2021-12-09 16:14:47 +01:00
Łukasz Magiera
9c75a3aaa8 piecereader: Avoid allocating 1024MB slices per read 2021-12-09 15:49:43 +01:00
Łukasz Magiera
a3d8494a04 piecereader: Avoid redundant roundtrips when seeking 2021-12-09 14:52:33 +01:00
Łukasz Magiera
13b260e7f7 piecereader: Move closer to storage 2021-12-08 23:20:20 +01:00
zenground0
a5be80828a RemoveData and Decode
- Unsealing replica update with sector key works and tested
- Sector key generation added and tested
2021-12-03 15:21:06 -05:00
Łukasz Magiera
727765b248 Command to list active sector locks 2021-12-03 12:33:23 +01:00
Łukasz Magiera
71329f6c41 Address Scheduler enhancements (#7703) review 2021-11-30 20:50:40 +01:00
zenground0
40d16a8f88 Review Response 2021-11-30 13:53:37 -05:00
zenground0
f88fcdbcfc WIP 2021-11-30 12:40:14 -05:00
Łukasz Magiera
001ecbb561 fix lint 2021-11-30 02:06:58 +01:00
Łukasz Magiera
a597b072b8 fix sched tests 2021-11-30 02:06:58 +01:00
Łukasz Magiera
f25efecb74 worker: Test resource table overrides 2021-11-30 02:06:58 +01:00
Łukasz Magiera
6d52d8552b Fix docsgen 2021-11-30 02:06:58 +01:00
Łukasz Magiera
c9a2ff4007 cleanup worker resource overrides 2021-11-30 02:06:58 +01:00
Łukasz Magiera
b961e1aab5 sched resources: Separate Parallelism defaults depending on GPU presence 2021-11-30 02:06:58 +01:00
Łukasz Magiera
36868a8749 sched: C2 is not all-core load 2021-11-30 02:06:58 +01:00
Clint Armstrong
4ef8543128 Permit workers to override resource table
In an environment with heterogenious worker nodes, a universal resource
table for all workers does not allow effective scheduling of tasks. Some
workers may have different proof cache settings, changing the required
memory for different tasks. Some workers may have a different count of
CPUs per core-complex, changing the max parallelism of PC1.

This change allows workers to customize these parameters with
environment variables. A worker could set the environment variable
PC1_MIN_MEMORY for example to customize the minimum memory requirement
for PC1 tasks. If no environment variables are specified, the resource
table on the miner is used, except for PC1 parallelism.

If PC1_MAX_PARALLELISM is not specified, and
FIL_PROOFS_USE_MULTICORE_SDR is set, PC1_MAX_PARALLELSIM will
automatically be set to FIL_PROOFS_MULTICORE_SDR_PRODUCERS + 1.
2021-11-30 02:06:58 +01:00