Commit Graph

1326 Commits

Author SHA1 Message Date
Łukasz Magiera
f2c2993164 sched: Correct allocated resource accounting in spread assigner 2022-05-23 22:22:27 +02:00
Łukasz Magiera
fd8b91a5d4 sched: Fix resource checks 2022-05-23 22:18:41 +02:00
Łukasz Magiera
443488b096 lint, docsgen 2022-05-23 22:02:39 +02:00
Łukasz Magiera
5ba8bd3b99 sched: Configurable assigners 2022-05-23 22:02:39 +02:00
Łukasz Magiera
588b8ecbca sched: Separate WindowSelector func 2022-05-23 22:02:39 +02:00
Łukasz Magiera
9ac19cb14b feat: sealing: Put scheduler assign logic behind an interface 2022-05-23 22:02:39 +02:00
Jennifer Wang
5cfedacf9f make jen 2022-05-23 15:11:46 -04:00
unknown
c4cfb7a296 scheduling optimization 2022-05-20 17:04:51 +08:00
Raúl Kripalani
40aafbaea6 upgrade filecoin-ffi to integrate fvm 1.0.0-rc.1 and builtin-actors v7.5.0. 2022-05-20 00:45:44 +01:00
Łukasz Magiera
effee8ce1f
Merge pull request #8589 from filecoin-project/fix/sched-hang
stores: Deduplicate parallel stat requests
2022-05-18 14:35:16 +02:00
Aayush
3b23ba0520 Address review 2022-05-17 15:52:59 -04:00
Aayush
b2b20e469f Fix tests 2022-05-17 15:21:27 -04:00
Aayush
8cca9b1970 Use new go-state-types accessors 2022-05-17 15:21:27 -04:00
Łukasz Magiera
33a05d33a9
Merge pull request #8622 from RobQuistNL/patch-7
fix: Make deal making logs much less noisy
2022-05-10 20:21:29 -04:00
Aayush Rajasekaran
c69b579201 Merge branch 'releases' into asr/merge-releases 2022-05-10 20:02:01 -04:00
Łukasz Magiera
086908e535 stores: Cache disk usage results for a few seconds 2022-05-10 19:18:49 -04:00
Raúl Kripalani
b4dae335e1 reinstantiate sealer test. 2022-05-10 14:46:12 +01:00
Raúl Kripalani
7ba14f706c Merge branch 'feat/nv16' into fix/nv16-gas-outputs 2022-05-10 14:40:32 +01:00
Raúl Kripalani
830a9482e7 fix ffi version. 2022-05-10 14:23:45 +01:00
Raúl Kripalani
6050d04c7c Merge tag 'v1.15.2' into feat/nv16-merge-v1.15.2 2022-05-10 14:17:29 +01:00
Raúl Kripalani
24e68feb59 update filecoin-ffi to 57accc2a83db1c84. 2022-05-09 23:48:47 +01:00
Rob Quist
c057ce10d1
Make deal making logs much less noisy
For each upgrade-marked sector, for each deal, this "info" log comes along but its not really "info" imho
2022-05-09 22:13:09 +02:00
Łukasz Magiera
e75e864070
Merge pull request #8564 from RobQuistNL/patch-5
fix: Clean up vanilla proof fetching errors for proper display
2022-05-05 16:29:54 -04:00
Łukasz Magiera
3c9a8a42d9 more logs 2022-05-05 13:12:09 -04:00
vyzo
b5d3487ce6 update filecoin-ffi@fix/issue-523 2022-05-04 20:20:19 +03:00
vyzo
de1df6b2fe update test-vectors 2022-05-04 20:19:07 +03:00
vyzo
a8f97b5849 kill test depending on ffi generated stuffs that no longer exist 2022-05-04 20:19:07 +03:00
vyzo
b7ca80265c use filecoin-ffi@fix/issue-523 2022-05-04 20:19:07 +03:00
Steven Allen
78899be05d feat: vm: add actor error backtraces to FVM 2022-05-04 20:18:25 +03:00
Steven Allen
c7fdfb00bb ci: fix linting 2022-05-04 15:38:15 +03:00
Łukasz Magiera
1249809283 stores: Deduplicate parallel stat requests 2022-05-03 15:53:11 -04:00
Aayush Rajasekaran
662f7526c9 Update to FFI with logger bump 2022-05-03 15:12:48 -04:00
Łukasz Magiera
8863bdbfc0 sealing: Finalize snap sectors before submitting proofs 2022-05-03 15:12:30 -04:00
Łukasz Magiera
3ab625fa4c sealing: Finalize snap sectors before submitting proofs 2022-05-02 16:18:31 -04:00
Geoff Stuart
b50f4a57f0 replica unseal with work with worker fetch over rpc 2022-05-02 15:34:45 -04:00
jennijuju
4d5e850aa9 update ffi that includes the log fix 2022-05-02 10:59:21 -04:00
Geoff Stuart
94ceb0355d local worker now returns typed error instead of string 2022-05-02 09:59:41 -04:00
Geoff Stuart
00015a21d4 Fix imports 2022-05-02 09:59:41 -04:00
Geoff Stuart
f6b08af337 reorg import 2022-05-02 09:59:41 -04:00
Geoff Stuart
5030fd2589 Optimize sector key regeneration 2022-05-02 09:59:41 -04:00
Geoff Stuart
2134f165c0 Review fixes 2022-05-02 09:59:41 -04:00
Geoff Stuart
98637b357b Replica unseal working 2022-05-02 09:59:41 -04:00
Geoff Stuart
b2a6d308ba Consolidate nullreader files 2022-05-02 09:59:41 -04:00
Steven Allen
6c0c111847 feat: vm: add actor error backtraces to FVM 2022-04-28 19:58:19 +02:00
Rob Quist
562d4e0f85
Clean up vanilla fetching errors for proper display
Results;
```
2  0  2374  good
2  0  2409  good
2  0  2282  bad (generating vanilla proof: non-200 code from http://10.224.3.83:1123/remote/vanilla/single: 'fsstat: path not found
')
2  0  2579  good
2  0  2338  good
```

This PR should change that to;
```
2  0  2374  good
2  0  2409  good
2  0  2282  bad (generating vanilla proof: non-200 code from http://10.224.3.83:1123/remote/vanilla/single: 'fsstat: path not found')
2  0  2579  good
2  0  2338  good
```
2022-04-28 14:49:46 +02:00
Łukasz Magiera
779e923854 sealing: ComputeDataCid review 2022-04-27 20:44:02 +02:00
Łukasz Magiera
a2d90625d2 sealing: Fix tests 2022-04-26 21:43:24 +02:00
Łukasz Magiera
d7c52f8adf sealing: ComputeDataCid miner API 2022-04-26 21:37:48 +02:00
Łukasz Magiera
1aed631fe5 sealing: DataCid on workers 2022-04-26 20:36:23 +02:00
Steven Allen
5b1800556b ci: fix linting 2022-04-25 14:13:16 +02:00
Steven Allen
e0d2b9526e ci: fix linting 2022-04-20 12:06:28 +02:00
Aayush
b60d795a5d FVM: Support exectraces 2022-04-19 18:05:49 -04:00
Aayush Rajasekaran
2410cf3d84
Merge pull request #8514 from filecoin-project/asr/exec-trace
feat: FVM: Support exectraces
2022-04-19 18:03:25 -04:00
Aayush
36a8d9716e FVM: Support exectraces 2022-04-19 15:24:40 -04:00
vyzo
995caaaace update filecoin-ffi 2022-04-19 15:02:14 +03:00
Łukasz Magiera
eb894b0dd2
Merge pull request #8475 from filecoin-project/fix/deal-idx-oob
fix:sealing:check index out of bounds against correct param length not return length
2022-04-18 18:56:23 +02:00
zenground0
0f6da17cd1 Lint 2022-04-14 19:32:56 +03:00
zenground0
60d09579be More fixes 2022-04-14 19:32:56 +03:00
Aayush
339b7db2b7 Integrate FIP 0027 2022-04-14 19:32:56 +03:00
vyzo
10b07ecd20 update filecoin-ffi 2022-04-14 19:29:34 +03:00
zenground0
db5082552d Fix sorting bug to deflake test 2022-04-13 22:06:38 -04:00
Łukasz Magiera
0a4b49d3b3 fix: storiface: Make FSOverhead numbers more accurate 2022-04-13 11:59:48 +02:00
Aayush Rajasekaran
42a79d52a2 Fail to add expired precommits to a batch 2022-04-12 23:41:52 -04:00
zenground0
563dc8e59a Fix up tests 2022-04-12 18:31:31 -04:00
zenground0
d540f7af72 Fix typo 2022-04-12 15:41:56 -04:00
zenground0
239274cb1d Properly handle output index 2022-04-12 15:34:45 -04:00
zenground0
fe6c25353c Fix assumptions about return value 2022-04-12 14:46:01 -04:00
Jiaying Wang
1fc94e33b2 Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
fix merge error
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2022-04-12 15:29:54 +02:00
Aayush Rajasekaran
656a33863f Fixup merge 2022-04-11 20:45:13 -04:00
jennijuju
0dcad9cc18 Merge branch 'releases' into jen/mergeback 2022-04-12 01:25:02 +02:00
Łukasz Magiera
3ecb1fd5d9 sealing: Don't panic in ReleaseUnsealed with no ranges 2022-04-09 20:58:47 +02:00
Łukasz Magiera
80133aaa79 feat: sched: Improve worker assigning logic 2022-04-06 18:24:14 -04:00
Łukasz Magiera
6de4e3d4cd feat: sched: Cache worker tasks 2022-04-06 18:24:14 -04:00
Steven Allen
39dee1410b chore: ffi: update the FFI to update the FVM
This fixes the build on aarch64.
2022-04-05 21:25:00 -04:00
Steven Allen
e69cc545b3 chore: ffi: update to fix build on aarch64 2022-04-05 17:59:30 -04:00
Łukasz Magiera
68f3e4483f fix: sealing: Release unsealed sector files after snapdeals abort 2022-04-05 16:45:07 -04:00
Łukasz Magiera
e1ef1b759e
Merge pull request #8206 from firesWu/fix/optimize-worker-tracker-lock
release worker tracker lock when call cb func
2022-04-05 12:53:09 -04:00
Jennifer Wang
c4aece4ab4 pull ffi that includes the latest fvm 2022-04-01 17:02:32 -04:00
Jennifer Wang
ce0b8149e9 pull ffi that includes the latest fvm 2022-04-01 15:58:15 -04:00
Jennifer Wang
6ff99568cf pull the latest ffi 2022-03-31 23:52:08 -04:00
zl
01efb3b1a7 fix: dead loop on removing a sector 2022-03-31 23:06:54 -04:00
Łukasz Magiera
2ad3a117f1 fix: sealing: Fix PR1 worker selection 2022-03-31 23:06:17 -04:00
Łukasz Magiera
ac82366b9e fix: sealing: Fix PR1 worker selection 2022-03-31 17:01:33 -04:00
simlecode
f0944030b7 Added error checking 2022-03-30 10:28:56 +08:00
Jennifer Wang
b8e412884b deps: update ffi with actor v7.1.0 and fvm that uses the bundle that includes the new manifest 2022-03-29 20:49:40 -04:00
Łukasz Magiera
3ce467a25f
Merge pull request #8391 from filecoin-project/feat/parallel-fault-check
feat: storage: Parallel proving checks
2022-03-29 16:36:12 -04:00
Łukasz Magiera
0710342317 fix waiting in sector proving checks 2022-03-29 15:28:37 -04:00
Jennifer Wang
21a1f3a34d deps: update ffi 2022-03-29 12:02:41 -04:00
Łukasz Magiera
ebd34f1884 feat: storage: Parallel proving checks 2022-03-29 11:45:08 -04:00
Aarsh Shah
d0af0eff9f thread safe piecereader 2022-03-29 19:41:06 +05:30
zl
1646edf513 fix: dead loop on removing a sector 2022-03-28 20:51:57 +08:00
Jennifer Wang
2a988d453c chore: ffi: the latest fvm release 2022-03-26 23:25:02 -04:00
Jennifer Wang
f2fa2f5067 chore: ffi: the latest fvm release 2022-03-26 22:45:47 -04:00
Aayush Rajasekaran
b0ff3dfb90
Revert "Update params for interopnet for fvm" 2022-03-25 22:21:36 -04:00
Łukasz Magiera
7401fa234e
Merge pull request #7971 from filecoin-project/feat/post-worker
feat: PoSt workers
2022-03-25 20:31:49 -04:00
Łukasz Magiera
7a009abcdd sealing: Fix TaskType.WorkerType 2022-03-25 18:54:59 -04:00
Aayush
02a0770436 Caterpillarnet 2022-03-25 17:05:45 -04:00
Łukasz Magiera
82343a06e7 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-25 16:49:46 -04:00
Łukasz Magiera
f1ddb55252 worker: Fix challengeThrottle 2022-03-25 16:07:47 -04:00
Łukasz Magiera
69e8516d1f stores: log 404 in remote.GenerateSingleVanillaProof 2022-03-25 15:37:35 -04:00
Łukasz Magiera
7c4d3a457c
Merge pull request #8343 from filecoin-project/feat/more-snap-config
feat: sealing: More SnapDeals config knobs
2022-03-25 15:07:00 -04:00
Rjan
910d3caacf fix:snapup: Rename error message
Renaming the the error-message you get if you try to snap-up the same sector twice, to better reflect that you are snapping a sector, and not marki-for-upgrade, which now has been deprecated.
2022-03-25 16:39:53 +01:00
Łukasz Magiera
0b3144f566 workers: PoSt challenge throttle/timeout config 2022-03-24 19:42:44 -04:00
Łukasz Magiera
13701c7ce2 wdpost: Lock update files in snapped sectors 2022-03-24 18:43:08 -04:00
Łukasz Magiera
a53dc5a7c7 workers: WorkerType mehdod on SealTask 2022-03-24 18:28:45 -04:00
Łukasz Magiera
8e75a8d12d fix: sealing: always do cooldown in handleSubmitReplicaUpdateFailed 2022-03-23 14:05:43 -04:00
Aayush
7d0f5dd4f9 FFI: Bump v7 actors to v7.0.6 2022-03-22 20:44:51 -04:00
Łukasz Magiera
97a73f1c29 sealing: More SnapDeals config knobs 2022-03-21 12:04:23 +01:00
Łukasz Magiera
cca69a6c16 gen 2022-03-21 10:49:30 +01:00
Łukasz Magiera
58843857e5 fix: sealing: always do cooldown in handleSubmitReplicaUpdateFailed 2022-03-21 10:32:03 +01:00
Łukasz Magiera
5365ccfdb1 cli: Show separate worker types in miner info 2022-03-18 21:31:15 +01:00
Łukasz Magiera
36f1dd7bb3 itests: Fix flaky TestWindowPostWorkerSkipBadSector 2022-03-18 20:51:47 +01:00
Łukasz Magiera
7b81369c8c wdpost: Don't attempt to snark with skipped sectors 2022-03-18 19:45:25 +01:00
Łukasz Magiera
7a1beacd00 ffiwrapper: Test some assumptions about GeneratePoStFallbackSectorChallenges 2022-03-18 13:11:14 +01:00
Łukasz Magiera
8715745feb wdpost: Dedupe skipped sectors in generateWindowPoSt 2022-03-18 12:10:42 +01:00
Łukasz Magiera
aa124cf414 fix lint issues 2022-03-18 11:38:27 +01:00
Łukasz Magiera
6ddbe41376 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-18 10:54:44 +01:00
Łukasz Magiera
d4cbb0803b
Merge pull request #8339 from filecoin-project/backport/v1.15.1/smallfixes
backport: #8338, #8337
2022-03-17 19:50:21 +01:00
Łukasz Magiera
b26405226e
Merge pull request #8336 from filecoin-project/fix/snap-list-estimate
fix: miner cli: Estimate deal weight in sector list when upgrading
2022-03-17 17:30:41 +01:00
Łukasz Magiera
49a4c06a88 fix: sealing: FinalizeSector doesn't need sealed replica access 2022-03-17 17:08:25 +01:00
Łukasz Magiera
361dc55dfd fix: storagefsm: Fix error loop on bad event 2022-03-17 17:08:25 +01:00
Łukasz Magiera
b366e2c91f
Merge pull request #8337 from filecoin-project/fix/fin-sealed-acc
fix: sealing: FinalizeSector doesn't need sealed replica access
2022-03-17 16:58:51 +01:00
Łukasz Magiera
eb4bda2302
Merge pull request #8338 from filecoin-project/fix/fsm-badevt-loop
fix: storagefsm: Fix error loop on bad event
2022-03-17 16:58:17 +01:00
Aayush Rajasekaran
2ed2ee1da7 Merge branch 'release/v1.15.1' into feat/fvm 2022-03-17 11:31:56 -04:00
Aayush Rajasekaran
58876aefa3 Update FFI 2022-03-17 11:31:47 -04:00
Łukasz Magiera
d9a1ecd387 fix: storagefsm: Fix error loop on bad event 2022-03-17 15:40:11 +01:00
Łukasz Magiera
9debd14b6c fix: sealing: FinalizeSector doesn't need sealed replica access 2022-03-17 15:15:54 +01:00
Łukasz Magiera
ff14cd3f9d fix: Estimate deal weight in sector list when upgrading 2022-03-17 14:52:54 +01:00
Łukasz Magiera
1c055fe83b
Merge branch 'master' into test/t.TempDir 2022-03-17 12:06:52 +01:00
Łukasz Magiera
4a1b211876 miner cli: Treat Activating sectors like Proving in sectors list 2022-03-16 22:30:59 +01:00
Łukasz Magiera
d78f1e4eee sealing: Rename Sealing.creating to a less confusing name 2022-03-16 22:30:58 +01:00
Łukasz Magiera
5e14892ef8 sealing: fix lint issues 2022-03-16 22:30:57 +01:00
Aayush Rajasekaran
97c95a4e82 Update FFI 2022-03-16 17:30:47 -04:00
Łukasz Magiera
9e4c8881b6 sealing: regenerate mocks 2022-03-16 22:30:09 +01:00
Łukasz Magiera
b8ba5274e2 sealing: Enforce minimum expiration in tryGetUpgradeSector 2022-03-16 22:30:09 +01:00
Łukasz Magiera
aa4adb0dfd sealing: Fix build 2022-03-16 22:30:09 +01:00
Łukasz Magiera
62a393bf12 sealing: Address snap queue review 2022-03-16 22:30:09 +01:00
Łukasz Magiera
a440339548 sealing: Run more checks when considering Available sectors 2022-03-16 22:30:07 +01:00
Łukasz Magiera
3c15314ed5 sealing: Use bitfields in sectorActive 2022-03-16 22:29:40 +01:00
Łukasz Magiera
d3cec2f0d1 sealing: Add MakeCCSectorsAvailable config 2022-03-16 22:29:40 +01:00
Łukasz Magiera
1bd2cbbef6 sealing: Support abort uprade for Available sectors 2022-03-16 22:29:40 +01:00
Łukasz Magiera
9a854e9ab7 feat: sealing: Linter did have a good point 2022-03-16 22:29:40 +01:00
Łukasz Magiera
2cef55a4f6 feat: sealing: Sector upgrade queue 2022-03-16 22:29:37 +01:00
Łukasz Magiera
a3bdd29615
Merge pull request #8330 from filecoin-project/feat/snap-queue
feat: sealing: Sector upgrade queue
2022-03-16 22:24:45 +01:00
Łukasz Magiera
a431fdbdde miner cli: Treat Activating sectors like Proving in sectors list 2022-03-16 22:16:39 +01:00
Łukasz Magiera
60ba133fc8 sealing: Rename Sealing.creating to a less confusing name 2022-03-16 21:29:57 +01:00
Aayush Rajasekaran
90d78f924f
Merge pull request #8293 from filecoin-project/feat/fvm
Filecoin Virtual Machine integration
2022-03-16 15:51:56 -04:00
Łukasz Magiera
3b2a1ea41d sealing: fix lint issues 2022-03-16 20:28:15 +01:00
Łukasz Magiera
2961f3b864 sealing: regenerate mocks 2022-03-16 20:20:07 +01:00
Łukasz Magiera
3f23cf0c1e sealing: Enforce minimum expiration in tryGetUpgradeSector 2022-03-16 20:17:20 +01:00