Commit Graph

17 Commits

Author SHA1 Message Date
Clint Armstrong
4ef8543128 Permit workers to override resource table
In an environment with heterogenious worker nodes, a universal resource
table for all workers does not allow effective scheduling of tasks. Some
workers may have different proof cache settings, changing the required
memory for different tasks. Some workers may have a different count of
CPUs per core-complex, changing the max parallelism of PC1.

This change allows workers to customize these parameters with
environment variables. A worker could set the environment variable
PC1_MIN_MEMORY for example to customize the minimum memory requirement
for PC1 tasks. If no environment variables are specified, the resource
table on the miner is used, except for PC1 parallelism.

If PC1_MAX_PARALLELISM is not specified, and
FIL_PROOFS_USE_MULTICORE_SDR is set, PC1_MAX_PARALLELSIM will
automatically be set to FIL_PROOFS_MULTICORE_SDR_PRODUCERS + 1.
2021-11-30 02:06:58 +01:00
Łukasz Magiera
11d738eee0 Track prepared work 2021-10-18 18:44:56 +02:00
Łukasz Magiera
9af82f2d68 sched: Fix taskDone chan deadlock 2021-10-03 17:09:43 +02:00
Łukasz Magiera
b87142ec8e wip improve scheduling of ready work 2021-10-03 10:38:08 +02:00
Łukasz Magiera
ef03314c6d storagemgr: Cleanup workerLk around worker resources 2021-09-15 16:35:19 +02:00
Raúl Kripalani
59eab2df25 move scheduling filtering logic down. 2021-06-21 20:49:16 +01:00
Łukasz Magiera
fe230f901e Collect worker task metrics 2021-03-05 12:48:17 +01:00
liismn
e9cc697ee8
Update sched_worker.go
Avoid possible deadlock by sh.workersLk
2020-12-25 15:05:01 +08:00
yaohcn
7c0b6f41d8 fix log format 2020-11-24 19:09:48 +08:00
Dan Shao
f3838a47c7 Format workerID as string 2020-11-23 15:07:50 +08:00
Łukasz Magiera
6bea9dd178 Making sealing logic work with multiple seal proof types 2020-11-16 19:03:30 +01:00
Łukasz Magiera
5b8f555221 sched: Update after tasks finish preparing 2020-11-09 21:44:28 +01:00
Łukasz Magiera
69e44ebf07 sched: re-register worker windows after re-enabling correctly 2020-10-30 18:32:59 +01:00
Łukasz Magiera
774e2ecebf sched: Fix worker reenabling 2020-10-30 18:01:37 +01:00
Łukasz Magiera
96c5ff7e7f sched: use more letters for variables 2020-10-28 14:23:38 +01:00
Łukasz Magiera
8731fe9112 sched: split worker handling into more funcs 2020-10-28 14:14:50 +01:00
Łukasz Magiera
84b567c790 sched: move worker funcs to a separate file 2020-10-28 13:39:28 +01:00