Commit Graph

417 Commits

Author SHA1 Message Date
Łukasz Magiera
27a9dd3bbb storage: Track worker hostnames with work 2020-11-09 23:09:04 +01:00
Łukasz Magiera
5b8f555221 sched: Update after tasks finish preparing 2020-11-09 21:44:28 +01:00
Łukasz Magiera
fbc05fa5a8 stores: Report storage changes quickly 2020-11-09 21:43:52 +01:00
Łukasz Magiera
95d9084899
Merge pull request #4765 from zgfzgf/feat-sealing-garbage
optimize pledge sector
2020-11-09 21:06:33 +01:00
zgfzgf
5bcc6339b4 optimize code replace strings with constants 2020-11-09 16:21:16 +08:00
zgfzgf
3665f90795 optimize pledge sector 2020-11-08 16:24:50 +08:00
Anton Evangelatov
f7f5a27ebf remove oni submodule 2020-11-06 21:03:56 +01:00
Łukasz Magiera
cde0e58e39 Check (pre)commit receipt before other checks in failed states 2020-11-03 18:47:29 +01:00
Łukasz Magiera
761c9ee2b6
Merge pull request #4699 from blockchain-lib/master
struct name incorrect
2020-11-03 11:47:18 +01:00
dignifiedquire
5c3055219c update filecoin-ffi to master 2020-11-03 11:02:58 +01:00
stirlingx
b5d2f01993
Update selector_task.go 2020-11-03 09:20:48 +08:00
dignifiedquire
1d17d61dbc update filecoin-ffi 2020-11-02 17:05:52 +01:00
dignifiedquire
d457b7a5e3 update filecoin-ffi 2020-11-02 17:05:52 +01:00
dignifiedquire
398a831701 update blst and filecoin-ffi 2020-11-02 17:05:52 +01:00
dignifiedquire
61fa03b1ad use supranation/blst@0.2.0 directly 2020-11-02 17:05:52 +01:00
dignifiedquire
56980ffc1f remove extern/fil-blst 2020-11-02 17:05:52 +01:00
Łukasz Magiera
094ea3fe97
Merge pull request #4666 from filecoin-project/fix/sched-issues
Fix worker reenabling, handle multiple restarts in worker
2020-10-30 21:34:07 +01:00
Łukasz Magiera
69e44ebf07 sched: re-register worker windows after re-enabling correctly 2020-10-30 18:32:59 +01:00
Łukasz Magiera
f90a387f96 sched: Print worker UUIDs in shed-diag correctly 2020-10-30 18:32:16 +01:00
Łukasz Magiera
774e2ecebf sched: Fix worker reenabling 2020-10-30 18:01:37 +01:00
Łukasz Magiera
7fbb868513 Debug flag to force running sealing scheduler 2020-10-30 11:07:35 +01:00
Łukasz Magiera
ee814fb48e
Merge pull request #4643 from filecoin-project/fix/fsm-ticket-loop
storagefsm: Fix GetTicket loop when the sector is already precommitted
2020-10-29 18:25:14 +01:00
Łukasz Magiera
c6b03ce62b sectorstorage: Missing unlock in waitWork 2020-10-29 15:18:51 +01:00
Łukasz Magiera
ea5bb5cdab sectorstorage: Fix manager restart edge-case 2020-10-29 12:14:21 +01:00
Łukasz Magiera
33459f1b36 storagefsm: Fix GetTicket loop when the sector is already precommitted 2020-10-29 10:26:02 +01:00
Łukasz Magiera
f0f75e2d2c
Merge pull request #4627 from karalabe/fix-gpu-usage-tracking
extern/sector-storage: fix GPU usage overwrite bug
2020-10-29 10:13:54 +01:00
Łukasz Magiera
32ea060e99
Merge pull request #3618 from filecoin-project/feat/async-restartable-workers
Track sealing processes across lotus-miner restarts
2020-10-28 21:55:45 +01:00
Péter Szilágyi
5f657b4333
extern/sector-storage: fix GPU usage overwrite bug 2020-10-28 20:52:33 +02:00
Łukasz Magiera
da7ecc1527 Fix flaky sealing manager tests 2020-10-28 16:15:17 +01:00
Łukasz Magiera
4475b9ac05 Use filecoin-ffi master 2020-10-28 15:31:15 +01:00
Łukasz Magiera
4100f6eead fix TestWDPostDoPost 2020-10-28 15:23:21 +01:00
Łukasz Magiera
ed2f81da2f sched: Fix tests 2020-10-28 14:34:28 +01:00
Łukasz Magiera
4cf00b8b42 worker_local: address review 2020-10-28 14:29:17 +01:00
Łukasz Magiera
96c5ff7e7f sched: use more letters for variables 2020-10-28 14:23:38 +01:00
Łukasz Magiera
8731fe9112 sched: split worker handling into more funcs 2020-10-28 14:14:50 +01:00
Łukasz Magiera
84b567c790 sched: move worker funcs to a separate file 2020-10-28 13:39:28 +01:00
Łukasz Magiera
843c0719e0 Update FFI 2020-10-27 15:37:11 +01:00
Łukasz Magiera
660236b224 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-23 23:25:35 +02:00
Łukasz Magiera
29e334de54
Merge pull request #4511 from filecoin-project/steb/generalize-window-post
Manage sectors by size instead of proof type.
2020-10-22 21:27:48 +02:00
Steven Allen
bcabe7b3b5 migrate methods to abstracted methods
Method numbers never change anyways. At worst, we'll deprecate old methods and
have to explicitly import them from the correct actors version to use them.
2020-10-21 12:18:37 -07:00
Steven Allen
4e730b5ec8 port to v2 imports 2020-10-21 12:16:23 -07:00
Steven Allen
00dcb1bce9 Manage sectors by size instead of proof type.
* We may have multiple sectors with the same size and different proof types, but all these management functions stay the same.
* This simplifies PoSt logic.
2020-10-20 18:30:56 -07:00
Łukasz Magiera
8c86ea6b75 localworker: Try very hard to get ruselts to manager 2020-10-18 19:45:11 +02:00
Łukasz Magiera
dbb421c4f7 localworker: Use better context for calling returnFunc 2020-10-18 19:32:43 +02:00
Łukasz Magiera
8d06cca073 sched: Handle workers using sessions instead of connections 2020-10-18 12:36:06 +02:00
Łukasz Magiera
7ac5dc55d0 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-17 13:45:11 +02:00
Raúl Kripalani
7e6ede7563 update schema; test-vectors submodule. 2020-10-15 12:49:23 +01:00
Łukasz Magiera
b74a3229f8 fsm: process expired-ticket sectors 2020-10-13 21:35:41 +02:00
Łukasz Magiera
71b3b9075d Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-13 21:33:21 +02:00
Łukasz Magiera
65883cbb6b
Merge pull request #4290 from filecoin-project/feat/ledger-integration
adding in ledger support
2020-10-12 15:09:27 +02:00
Steven Allen
dc4e73c737 Test the tape upgrade 2020-10-12 00:01:25 -07:00
Łukasz Magiera
4ac2bee81c Update oni 2020-10-12 06:34:45 +02:00
Steven Allen
83dfc460d4 fix race in unseal
1. Remove an invalid error check.
2. Make sure to shadow the outer error type from within the goroutine instead or
reading the outer type.

This may have been causing test issues (caught in TestMinerAllInfo with the race
detector).
2020-10-09 15:39:41 -07:00
Raúl Kripalani
67b3a55e30 update test-vectors submodule. 2020-10-08 15:44:11 +01:00
Łukasz Magiera
0de3051821 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-08 13:10:41 +02:00
Steven Allen
767c346cf8 update test-vectors 2020-10-07 12:17:32 -07:00
Steven Allen
c17fa4bc35 update test-vectors for StateManager constructor change 2020-10-07 12:04:32 -07:00
Steven Allen
df4b068d0e Merge branch 'master' into next 2020-10-07 11:55:00 -07:00
Łukasz Magiera
cfd126ee9f
Merge pull request #3961 from filecoin-project/docs/miner-storage
lotus-miner: add more help text to storage / attach
2020-10-07 14:55:42 +02:00
Steven Allen
1e78cd21ec fix lotus soup build 2020-10-05 14:37:08 -07:00
Steven Allen
17846aad2f add some more policy toggles for testground 2020-10-05 14:28:47 -07:00
Łukasz Magiera
1fc23fb466 lotus-miner: Cleanup storage attach helptext a bit 2020-10-03 11:30:22 +02:00
Steven Allen
8292d60196 Merge branch 'master' into asr/spec-v1 2020-10-02 18:03:03 -07:00
Łukasz Magiera
5932f28519 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-01 02:39:48 +02:00
Łukasz Magiera
6981f776f4 Lower PC2 memory requirements 2020-10-01 00:54:53 +02:00
Łukasz Magiera
1b7cdb9341 Fix storage manager tests 2020-10-01 00:54:34 +02:00
Łukasz Magiera
5e08d56630 sched: Allow some single-thread tasks to run in parallel with PC2/C2 2020-10-01 00:28:44 +02:00
Łukasz Magiera
79d2ddf24f Review 2020-09-30 21:18:12 +02:00
Łukasz Magiera
2cfe22d4e5 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-30 20:48:16 +02:00
Łukasz Magiera
e3ee4e4718 Fix lint errors 2020-09-30 20:24:03 +02:00
Łukasz Magiera
2d16af6ee6 sectorstorage: Fix TestRedoPC1 2020-09-30 19:18:38 +02:00
Łukasz Magiera
4f97d9637e Fix storage-fsm tests 2020-09-30 19:11:58 +02:00
Steven Allen
4989b1cc90 Merge branch 'master' into asr/spec-v1 2020-09-30 09:19:12 -07:00
Łukasz Magiera
c228598098 sectorstorage: Variable scopes are really hard 2020-09-30 18:16:07 +02:00
Łukasz Magiera
a783bf9b8b storagefsm: Handle PC2 with missing replica 2020-09-30 17:53:15 +02:00
Łukasz Magiera
54fdd6ba5a sectorstorage: Variable scopes are hard 2020-09-30 17:48:48 +02:00
Łukasz Magiera
6855284d88 sectorstorage: Cancel non-running work in case of abort in sched 2020-09-30 17:26:09 +02:00
Łukasz Magiera
5bffea6f54
Merge pull request #4064 from filecoin-project/tvx
tvx: a test vector extraction and execution tool
2020-09-30 17:22:42 +02:00
Raúl Kripalani
ff8663faa0 update test-vectors submodule. 2020-09-30 14:58:22 +01:00
Łukasz Magiera
430897c9ac
Merge pull request #3445 from filecoin-project/asr/precommit-failed-fix
Fix some failed precommit handling
2020-09-30 09:35:41 +02:00
Łukasz Magiera
bc85e3ce50
Merge pull request #4107 from shaodan/worker-no-swap
Add --no-swap flag for worker
2020-09-30 09:25:57 +02:00
Dan Shao
1affd498c1 Add --no-swap flag for worker 2020-09-30 14:23:35 +08:00
Łukasz Magiera
baef3c8dd2 sectorstorage: Fix potential panic in FinalizeSector 2020-09-29 15:22:46 +02:00
Łukasz Magiera
0f2dcf28b1 fsm: Reuse tickets in PC1 on retry 2020-09-29 10:07:49 +02:00
Steven Allen
1c03862854 fix policy for specs-actors update 2020-09-28 17:28:16 -07:00
Steven Allen
233d8a9b72 update even more imports 2020-09-28 14:25:58 -07:00
Łukasz Magiera
1e6a69f8aa localworker: Don't mark calls as returned when returning fails 2020-09-28 22:10:02 +02:00
Łukasz Magiera
9bd2537971 stores: Fix error printing in http handler 2020-09-28 22:06:03 +02:00
Łukasz Magiera
810c767200 worker: Redeclare storage on reconnect 2020-09-28 21:06:49 +02:00
Łukasz Magiera
4ba7af6061 worker: Mark return methods as retry-safe 2020-09-28 20:46:44 +02:00
Łukasz Magiera
9e7d6823b1 sectorstorage: Cleanup callToWork mapping after work is done 2020-09-28 13:34:45 +02:00
Łukasz Magiera
cf71f034dc Merge remote-tracking branch 'origin/dev' into feat/async-restartable-workers 2020-09-26 12:35:32 +02:00
zgfzgf
1a7aea1906 modify error 2020-09-25 22:59:21 +08:00
zgfzgf
3207bc4704 optimize trySched 2020-09-25 22:41:29 +08:00
zgfzgf
60e950015c modify for unsafe 2020-09-25 22:13:27 +08:00
whyrusleeping
306c098d30 also update our vendored blst repo 2020-09-24 11:32:38 -07:00
whyrusleeping
6db37b72a8 update ffi to code with blst fixes 2020-09-24 11:05:21 -07:00
Łukasz Magiera
04ee53e061 sectorstorage: Show task type of ret-wait jobs 2020-09-24 11:55:11 +02:00
Steven Allen
32a699d6a3 Add some actors policy setters for testing
Addresses:
* a307e4593a (r491966115)
* a307e4593a (r491966634)

Note: This puts everything into a policy package to avoid a dependency cycle
between the build package, the miner package, and the types package. This is
also why I introduced a GetPreCommitChallengeDelay function and removed the
variable.
2020-09-23 15:00:52 -07:00
Łukasz Magiera
d817dceb05 Show lost calls in sealing jobs cli 2020-09-23 19:26:35 +02:00