cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
14,535 Commits 24 Branches 342 Tags 314 MiB
f476aa937e
Commit Graph

912 Commits

Author SHA1 Message Date
Łukasz Magiera
d9923125ff mock sealer: Simulate randomness sideeffects 2021-03-12 17:33:43 +01:00
Łukasz Magiera
e5b5bf1c1b storagefsm: Update sector stats atomically with sector creation 2021-03-12 17:25:24 +01:00
Łukasz Magiera
679f5f7661 storagefsm: Helper stat functions in updateSector 2021-03-12 17:00:20 +01:00
Łukasz Magiera
957d3f0636 storgefsm: Use locks correctly 2021-03-12 15:01:00 +01:00
Łukasz Magiera
5c605d99e2 storgefsm: Fix tests 2021-03-12 14:56:46 +01:00
Łukasz Magiera
f5ed25371b storagefsm: Trigger input processing wheen below limits 2021-03-12 14:54:10 +01:00
Łukasz Magiera
d0243904f7
Merge pull request #5779 from filecoin-project/feat/storagefsm-no-restart-log 
storagefsm: Don't log the SectorRestart event
 2021-03-11 15:18:32 +01:00
Łukasz Magiera
881c722025 storagefsm: Fix double unlock with ready WaitDeals sectors 2021-03-11 15:03:51 +01:00
Łukasz Magiera
a1b6a2f011 storagefsm: Don't log the SectorRestart event 2021-03-11 12:52:34 +01:00
Łukasz Magiera
cc490b983a
Merge pull request #5761 from filecoin-project/fix/testunpadreader-go116 
Fix TestUnpadReader on Go 1.16
 2021-03-10 13:13:27 +01:00
Łukasz Magiera
7fbb4bdd3b Fix TestUnpadReader on Go 1.16 2021-03-10 13:05:25 +01:00
Łukasz Magiera
40fdf6c180
Merge pull request #5624 from filecoin-project/feat/miner-storage-limit 
Configurable storage path storage limit
 2021-03-10 11:15:28 +01:00
Łukasz Magiera
df0a8d9507
Merge pull request #5759 from filecoin-project/asr/terminate-livecheck 
Check liveness of sectors when processing termination batches
 2021-03-10 11:14:11 +01:00
Aayush Rajasekaran
47064f987c Check liveness of sectors when processing termination batches 2021-03-09 23:36:30 -05:00
Łukasz Magiera
fe230f901e Collect worker task metrics 2021-03-05 12:48:17 +01:00
Łukasz Magiera
29c9fa3137
Merge pull request #5612 from filecoin-project/feat/cc-fsm-cleanup 
storagefsm: Cleanup CC sector creation
 2021-02-26 11:39:28 +01:00
Łukasz Magiera
e49a412f6d
Merge pull request #5375 from filecoin-project/feat/refactor-fsm-input 
storagefsm: Rewrite input handling
 2021-02-25 14:28:04 +01:00
Łukasz Magiera
2b9ed9fa3d Remove sectors with all deals expired in RecoverDealIDs 2021-02-22 21:03:51 +01:00
Łukasz Magiera
35759fa07e
Merge pull request #5635 from filecoin-project/deps/cbg-soft-map 
Update markets, cbor-gen with soft map decoding
 2021-02-22 18:27:38 +01:00
Aayush Rajasekaran
6732758d0f Tweak mock ProveCommits to respect proof sizes 2021-02-19 17:22:59 -05:00
Łukasz Magiera
26399dba70 Update markets, cbor-gen with soft map decoding 2021-02-19 20:11:43 +01:00
Łukasz Magiera
37b1fb97fb Configurable storage path storage limit 2021-02-18 16:44:34 +01:00
yaohcn
3f66e731fa fix drop sector 2021-02-18 12:00:28 +08:00
Łukasz Magiera
90b8612cf4 Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input 2021-02-16 22:39:06 +01:00
Łukasz Magiera
9ebeafb174 Update ffi 2021-02-16 22:04:08 +01:00
Łukasz Magiera
f719765069 storageminer: exit PledgeSector after sectors enter sealing pipeline 2021-02-16 17:41:58 +01:00
Łukasz Magiera
fc5e243c92 storagefsm: Cleanup CC sector creation 2021-02-16 17:14:59 +01:00
Łukasz Magiera
dd82729f60 Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input 2021-02-16 13:45:39 +01:00
Aayush Rajasekaran
bb7fabb439 Update FFI 2021-02-15 17:22:25 -05:00
Łukasz Magiera
6907e5879d Fix WaitDeals sector accounting 2021-02-11 13:52:00 +01:00
Łukasz Magiera
e27a530cbc storagefsm: cleanup openSectors better; pendingPieces by pieceCid 2021-02-09 18:44:41 +01:00
He Weidong
0e2e1125d3 fix precommit ticket expiration handling 2021-02-05 10:45:15 +08:00
Łukasz Magiera
6e7fcb7ee6 Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input 2021-02-03 14:01:56 +01:00
Łukasz Magiera
1e7a107484
Merge pull request #5433 from filecoin-project/feat/keep-unsealed-cfg 
Add miner config to always keep ensealed deal copies
 2021-02-03 12:40:02 +01:00
Łukasz Magiera
94009f247f
Merge pull request #5411 from filecoin-project/feat/sealing-handle-batch-publish 
Handle batch publish storage deals message in sealing recovery
 2021-01-28 19:33:49 +01:00
Łukasz Magiera
aebf3cd64c Update ffi with more portable blst 2021-01-27 22:36:51 +01:00
Łukasz Magiera
d4c7b63aac Add miner config to always keep ensealed deal copies 2021-01-26 17:50:31 +01:00
Łukasz Magiera
fabcbb621d Merge remote-tracking branch 'origin/master' into next 2021-01-26 00:47:22 +01:00
Łukasz Magiera
67780346e2 storagemgr: Always unseal full sectors 2021-01-25 21:39:15 +01:00
Łukasz Magiera
e4f4fa2c7d Update FFI more, fix zero aggregate sigs 2021-01-25 20:05:03 +01:00
Dirk McCormick
1b494acc9e feat: handle batch publish storage deals message in sealing recovery 2021-01-25 14:10:05 +01:00
Łukasz Magiera
9b62e224d5 Use filecoin-ffi master 2021-01-25 13:57:15 +01:00
Steven Allen
03cd3760bb correctly pick the seal type based on the network version 
Of course, we should really just run all of our tests post actors v2.
 2021-01-21 15:21:20 -08:00
Aayush Rajasekaran
ab90a3b2bc Actors update: MinerInfo.SealProofType has been removed 2021-01-21 15:21:19 -08:00
Łukasz Magiera
1070ad2289 storagefsm: Drop unused TargetWaitDealsSectors 2021-01-21 22:20:16 +01:00
Łukasz Magiera
ec4deb7e28 storagefsm: Fix unlocking in handleWaitDeals 2021-01-21 19:59:18 +01:00
Łukasz Magiera
1336d8855d storagefsm: Drop addpiece wait after fixing storageadapter 2021-01-21 17:40:50 +01:00
Łukasz Magiera
df14f156e1 storagefsm: More logging for deal test debugging 2021-01-21 17:40:50 +01:00
Łukasz Magiera
b9a9f23204 storagefsm: Add stub AddPieceFailed state 2021-01-21 17:40:50 +01:00
Łukasz Magiera
f96f12c836 storagefsm: Add rest of checks in WaitDeals 2021-01-21 17:40:50 +01:00
Łukasz Magiera
069766ecc4 storagefsm: Don't persist piece assignment queue 2021-01-21 17:40:50 +01:00
Łukasz Magiera
270f2935a9 storagefsm: Check per-sector deal limits 2021-01-21 17:40:50 +01:00
Łukasz Magiera
fd67a41c75 storagefsm: Change sector CreationTime to unix ts 2021-01-21 17:40:50 +01:00
Łukasz Magiera
9857ad8378 storagefsm: Fix some deadlock cases 2021-01-21 17:40:50 +01:00
Łukasz Magiera
e5814dac4f cbor-gen 2021-01-21 17:40:50 +01:00
Łukasz Magiera
542357a1df storagefsm: Start packing correctly 2021-01-21 17:40:50 +01:00
Łukasz Magiera
239d6f8f4d storagefsm: Rewrite input handling 2021-01-21 17:40:50 +01:00
dignifiedquire
3d9eb226fa refactor: switch to filecoin-ffi bls api for bls signatures 2021-01-20 16:19:07 +01:00
dignifiedquire
187a499cca remove extern/blst 2021-01-20 16:19:07 +01:00
Łukasz Magiera
48c6c55f68
Merge pull request #5384 from filecoin-project/fix-storage-size-estimation 
fix FileSize to return correct disk usage recursively
 2021-01-19 19:22:47 +01:00
Łukasz Magiera
560a127ce6
fsutil: Defensive check for Stat cast 2021-01-19 18:53:37 +01:00
Anton Evangelatov
834a6148b0 use stat.Blocks and not info.Size 2021-01-19 18:07:37 +01:00
Anton Evangelatov
16d07d3f18 fix FileSize to return correct disk usage recursively 2021-01-19 15:21:31 +01:00
Łukasz Magiera
0eaa6d9983 storagefsm: Send correct event on ErrExpiredTicket in CommitFailed 2021-01-17 22:29:07 +01:00
Łukasz Magiera
80b8d4b9d7 Address review 2021-01-14 20:27:15 +01:00
Łukasz Magiera
32885e1129 termination batcher: Notify based on what was sent 2021-01-14 17:14:26 +01:00
Łukasz Magiera
4015ddbb4f fsm: Fix panic in precommit check in handleTerminating 2021-01-14 16:13:32 +01:00
Łukasz Magiera
49abdd7d7d Sector termination support - address review 2021-01-14 15:46:57 +01:00
Łukasz Magiera
7ddf1d1feb Add terminating states to state lists 2021-01-14 12:44:27 +01:00
Łukasz Magiera
144b5a1350 perning termination API 2021-01-14 12:37:23 +01:00
Łukasz Magiera
1564db1fce Sector termination test 2021-01-14 00:11:41 +01:00
Łukasz Magiera
3522c8d45a SectorTerminateFlush API 2021-01-13 23:32:04 +01:00
Łukasz Magiera
174c595acc Don't declare in proving window 2021-01-13 22:19:10 +01:00
Łukasz Magiera
52cc2cd3eb Initial sector termination support 2021-01-13 00:42:01 +01:00
Łukasz Magiera
5462ecf6da gofmt 2021-01-11 16:44:11 +01:00
Łukasz Magiera
9e74c3c4e8 Multicore AddPiece CommP 2021-01-11 16:44:11 +01:00
Łukasz Magiera
299e72458e storagefsm: Fix unsealedInfoMap.lk init race 2021-01-10 14:01:29 +01:00
Łukasz Magiera
2c9f4ad5ad
Merge pull request #5251 from liismn/master 
Fix deadlock in runWorker in sched_worker.go
 2021-01-05 19:35:49 +01:00
Łukasz Magiera
44a56c7e33
Merge pull request #5208 from jsign/jsign/imprreadpiece 
sector-storage/mock: improve mocked readpiece
 2021-01-05 18:37:48 +01:00
15ho
5b235db366 fix: stores.FetchHandler: defer close file and increase buffer size 2020-12-29 10:07:28 +08:00
liismn
e9cc697ee8
Update sched_worker.go 
Avoid possible deadlock by sh.workersLk
 2020-12-25 15:05:01 +08:00
quanzhan lu
d806fafee8 update weight canSeal and canStore when attach 
update  weight canSeal and canStore when attach storage
 2020-12-22 19:07:17 +08:00
Ignacio Hagopian
f6cf27c781
improve mocked readpiece 
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
 2020-12-16 10:15:38 -03:00
zzx234234
7db5e8061d fix a coding error in truncating sector log 2020-12-11 11:31:36 +08:00
Łukasz Magiera
46a5013dc1 stores: Simplify remote copy buf size logic 2020-12-10 20:54:42 +01:00
Łukasz Magiera
53a6c4eac6 Merge branch 'iosize' of github.com:acautman/lotus into acautman-iosize 2020-12-10 20:48:04 +01:00
caoao
d4582fb32e write size according to RegisteredSealProof 2020-12-10 11:59:09 +08:00
stirlingx
3c50baaf5b
fix curSealing out of MaxSealingSectors limit 2020-12-10 11:04:42 +08:00
beck
d3fa728894
Remove extra spaces 2020-12-09 19:22:21 +08:00
Łukasz Magiera
1502ca6460 storagefsm: Fix addr selection for commits 2020-12-03 12:30:41 +01:00
Łukasz Magiera
562f5839b7 Fix lint 2020-12-02 22:01:09 +01:00
Łukasz Magiera
2fd93a55ac plumb AddressSelector to stoage fsm 2020-12-02 21:47:54 +01:00
Łukasz Magiera
289ef910a0 fix imports, docsgen 2020-12-02 00:39:55 +01:00
Łukasz Magiera
95eaf13b5a sectorstorage: Fix tests 2020-12-02 00:36:32 +01:00
Łukasz Magiera
c6f85886bf miner: Add slow mode to proving check 2020-12-02 00:32:14 +01:00
Łukasz Magiera
614f45dcfb
Merge pull request #5041 from filecoin-project/feat/5013 
Sector check command
 2020-12-01 14:59:01 +01:00
Łukasz Magiera
a4f3758f4c worker api: better grouping 2020-11-30 23:16:30 +01:00
Łukasz Magiera
e8d2560581 Fix tests 2020-11-27 16:47:48 +01:00
Łukasz Magiera
3672053ae9 worker: Support setting task types at runtime 2020-11-26 17:33:34 +01:00
Łukasz Magiera
2304ea0077
Merge pull request #5016 from filecoin-project/fix/fsm-rm-unceroverable 
storagefsm: Add missing planners
 2020-11-26 12:22:48 +01:00
Łukasz Magiera
74f040f9d5 storagefsm: Add SubmitCommit to toStatState switch 2020-11-26 12:02:35 +01:00
Łukasz Magiera
eeb38fa649 storagefsm: Test fsmPlanners map 2020-11-26 10:58:07 +01:00
Łukasz Magiera
ba87cb4fa9 storagefsm: Add missing planners 2020-11-26 10:57:47 +01:00
Łukasz Magiera
65653a1747 storagefsm: Add GetTicket to some maps 2020-11-26 10:53:31 +01:00
s1m0n21
663b3f4f21 add a sector check command 2020-11-26 15:02:43 +08:00
Łukasz Magiera
89493fe2b8 storagefsm: newDealSector review 2020-11-25 18:55:58 +01:00
Łukasz Magiera
6166204bac storagefsm: Improve new deal sector logic 2020-11-25 16:40:04 +01:00
Łukasz Magiera
214f46bc88
Merge pull request #4876 from filecoin-project/fix/miner-ticket-loop 
storagefsm: Fix expired ticket retry loop
 2020-11-24 19:54:41 +01:00
Łukasz Magiera
a63ef1dcd5
Merge pull request #4984 from yaohcn/fix-log-warn 
fix log format
 2020-11-24 18:01:56 +01:00
Łukasz Magiera
7c831101f3
Merge pull request #4727 from filecoin-project/nonsense/integrate-testplans-to-lotus 
move testground/lotus-soup testplan from oni to lotus
 2020-11-24 17:05:29 +01:00
Łukasz Magiera
fe92a6e2d4
Merge pull request #4024 from zgfzgf/sector-sched 
modify for unsafe
 2020-11-24 16:19:17 +01:00
yaohcn
7c0b6f41d8 fix log format 2020-11-24 19:09:48 +08:00
Anton Evangelatov
fd640864cb Merge branch 'master' into nonsense/integrate-testplans-to-lotus 2020-11-23 20:20:44 +01:00
Łukasz Magiera
83b1009837 storagefsm: check getticket precommit expication correctly 2020-11-23 19:12:54 +01:00
Łukasz Magiera
85e592bbe2 storagefsm: Retry pushing messages when out-of-funds 2020-11-23 19:03:31 +01:00
Łukasz Magiera
e2a6b99144 storagefsm: Fix expired ticket retry loop 2020-11-23 19:03:31 +01:00
Łukasz Magiera
0e39e760e5
Merge pull request #4929 from filecoin-project/feat/update-markets-commp-extraction 
Remove all temp file generation for deals
 2020-11-23 18:43:00 +01:00
Dirk McCormick
3634ca50ca fix: sector-storage tests 2020-11-23 16:09:57 +01:00
Dirk McCormick
c965696586 fix: ffiwrapper sealer test 2020-11-23 14:21:46 +01:00
Dan Shao
f3838a47c7 Format workerID as string 2020-11-23 15:07:50 +08:00
zgfzgf
b6893b0a3f solve merage problem 2020-11-22 16:15:30 +08:00
hannahhoward
218f7c2c5d feat(markets): update markets, extract commp 2020-11-20 13:29:14 -08:00
Łukasz Magiera
f0eac01a3c Merge remote-tracking branch 'origin/master' into nonsense/integrate-testplans-to-lotus 2020-11-20 15:22:10 +01:00
S1M0N21
2b611b3a99 Remove unused key 2020-11-19 23:48:28 +08:00
Aayush Rajasekaran
68a16afeda Merge branch 'master' into release/v1.2.0 2020-11-17 19:40:05 -05:00
Łukasz Magiera
00335878ab
Merge pull request #4712 from filecoin-project/fix/commitfailed-checks 
Check (pre)commit receipt before other checks in failed states
 2020-11-18 01:26:02 +01:00
Łukasz Magiera
b242d69805 Make storiface.CallError json-friendly 2020-11-17 16:28:41 +01:00
Łukasz Magiera
b8853aa4d5 Add error codes to worker return 2020-11-17 16:17:55 +01:00
Łukasz Magiera
e0ade92b29 Fix lint warnings 2020-11-16 19:20:18 +01:00
Łukasz Magiera
444239a643 update ffi 2020-11-16 19:17:07 +01:00
Łukasz Magiera
cd0cb1407d update ffi 2020-11-16 19:03:30 +01:00
Łukasz Magiera
915c5e0fdd Get most tests to pass 2020-11-16 19:03:30 +01:00
Łukasz Magiera
2a7e4af5ee Fix lotus-miner/lotus-worker build 2020-11-16 19:03:30 +01:00
Steven Allen
584907269a make sector-storage compile 2020-11-16 19:03:30 +01:00
Łukasz Magiera
6bea9dd178 Making sealing logic work with multiple seal proof types 2020-11-16 19:03:30 +01:00
Łukasz Magiera
6bb67f2fe9
Merge pull request #4834 from filecoin-project/feat/sector-gc 
miner: Basic storage cleanup command
 2020-11-13 00:03:17 +01:00
Łukasz Magiera
408183c7a6 docsgen: Fix CI 2020-11-12 20:15:57 +01:00
Łukasz Magiera
11b041224e fsm: Allow manually resurrecting removed sectors 2020-11-12 19:37:54 +01:00
Łukasz Magiera
8409e155d4 docsgen: Support miner/worker structs 2020-11-12 10:29:42 +01:00
Łukasz Magiera
8ac495723e gofmt 2020-11-11 17:42:23 +01:00
Łukasz Magiera
09f9f871a3 Create a command to abort sealing calls 2020-11-11 17:39:12 +01:00
Łukasz Magiera
bd71bf2379 Expand sched-diag 2020-11-11 16:47:44 +01:00
Łukasz Magiera
18aa97f962
Merge pull request #4779 from filecoin-project/feat/work-track-hostname 
storage: Track worker hostnames with work
 2020-11-10 20:38:44 +01:00
Łukasz Magiera
5caa277261 storage: Track abandoned work more correctly 2020-11-09 23:38:20 +01:00
Łukasz Magiera
f819e71d12 storage: Separate returned jobs in jobs cli 2020-11-09 23:13:29 +01:00
Łukasz Magiera
27a9dd3bbb storage: Track worker hostnames with work 2020-11-09 23:09:04 +01:00
Łukasz Magiera
5b8f555221 sched: Update after tasks finish preparing 2020-11-09 21:44:28 +01:00
Łukasz Magiera
fbc05fa5a8 stores: Report storage changes quickly 2020-11-09 21:43:52 +01:00
Łukasz Magiera
95d9084899
Merge pull request #4765 from zgfzgf/feat-sealing-garbage 
optimize pledge sector
 2020-11-09 21:06:33 +01:00
zgfzgf
5bcc6339b4 optimize code replace strings with constants 2020-11-09 16:21:16 +08:00
zgfzgf
3665f90795 optimize pledge sector 2020-11-08 16:24:50 +08:00
Anton Evangelatov
f7f5a27ebf remove oni submodule 2020-11-06 21:03:56 +01:00
Łukasz Magiera
cde0e58e39 Check (pre)commit receipt before other checks in failed states 2020-11-03 18:47:29 +01:00
Łukasz Magiera
761c9ee2b6
Merge pull request #4699 from blockchain-lib/master 
struct name incorrect
 2020-11-03 11:47:18 +01:00
dignifiedquire
5c3055219c update filecoin-ffi to master 2020-11-03 11:02:58 +01:00
stirlingx
b5d2f01993
Update selector_task.go 2020-11-03 09:20:48 +08:00
dignifiedquire
1d17d61dbc update filecoin-ffi 2020-11-02 17:05:52 +01:00
dignifiedquire
d457b7a5e3 update filecoin-ffi 2020-11-02 17:05:52 +01:00
dignifiedquire
398a831701 update blst and filecoin-ffi 2020-11-02 17:05:52 +01:00
dignifiedquire
61fa03b1ad use supranation/blst@0.2.0 directly 2020-11-02 17:05:52 +01:00
dignifiedquire
56980ffc1f remove extern/fil-blst 2020-11-02 17:05:52 +01:00
Łukasz Magiera
094ea3fe97
Merge pull request #4666 from filecoin-project/fix/sched-issues 
Fix worker reenabling, handle multiple restarts in worker
 2020-10-30 21:34:07 +01:00
Łukasz Magiera
69e44ebf07 sched: re-register worker windows after re-enabling correctly 2020-10-30 18:32:59 +01:00
Łukasz Magiera
f90a387f96 sched: Print worker UUIDs in shed-diag correctly 2020-10-30 18:32:16 +01:00
Łukasz Magiera
774e2ecebf sched: Fix worker reenabling 2020-10-30 18:01:37 +01:00
Łukasz Magiera
7fbb868513 Debug flag to force running sealing scheduler 2020-10-30 11:07:35 +01:00
Łukasz Magiera
ee814fb48e
Merge pull request #4643 from filecoin-project/fix/fsm-ticket-loop 
storagefsm: Fix GetTicket loop when the sector is already precommitted
 2020-10-29 18:25:14 +01:00
Łukasz Magiera
c6b03ce62b sectorstorage: Missing unlock in waitWork 2020-10-29 15:18:51 +01:00
Łukasz Magiera
ea5bb5cdab sectorstorage: Fix manager restart edge-case 2020-10-29 12:14:21 +01:00
Łukasz Magiera
33459f1b36 storagefsm: Fix GetTicket loop when the sector is already precommitted 2020-10-29 10:26:02 +01:00
Łukasz Magiera
f0f75e2d2c
Merge pull request #4627 from karalabe/fix-gpu-usage-tracking 
extern/sector-storage: fix GPU usage overwrite bug
 2020-10-29 10:13:54 +01:00
Łukasz Magiera
32ea060e99
Merge pull request #3618 from filecoin-project/feat/async-restartable-workers 
Track sealing processes across lotus-miner restarts
 2020-10-28 21:55:45 +01:00
Péter Szilágyi
5f657b4333
extern/sector-storage: fix GPU usage overwrite bug 2020-10-28 20:52:33 +02:00
Łukasz Magiera
da7ecc1527 Fix flaky sealing manager tests 2020-10-28 16:15:17 +01:00
Łukasz Magiera
4475b9ac05 Use filecoin-ffi master 2020-10-28 15:31:15 +01:00
Łukasz Magiera
4100f6eead fix TestWDPostDoPost 2020-10-28 15:23:21 +01:00
Łukasz Magiera
ed2f81da2f sched: Fix tests 2020-10-28 14:34:28 +01:00
Łukasz Magiera
4cf00b8b42 worker_local: address review 2020-10-28 14:29:17 +01:00
Łukasz Magiera
96c5ff7e7f sched: use more letters for variables 2020-10-28 14:23:38 +01:00
Łukasz Magiera
8731fe9112 sched: split worker handling into more funcs 2020-10-28 14:14:50 +01:00
Łukasz Magiera
84b567c790 sched: move worker funcs to a separate file 2020-10-28 13:39:28 +01:00
Łukasz Magiera
843c0719e0 Update FFI 2020-10-27 15:37:11 +01:00
Łukasz Magiera
660236b224 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-23 23:25:35 +02:00
Łukasz Magiera
29e334de54
Merge pull request #4511 from filecoin-project/steb/generalize-window-post 
Manage sectors by size instead of proof type.
 2020-10-22 21:27:48 +02:00
Steven Allen
bcabe7b3b5 migrate methods to abstracted methods 
Method numbers never change anyways. At worst, we'll deprecate old methods and
have to explicitly import them from the correct actors version to use them.
 2020-10-21 12:18:37 -07:00
Steven Allen
4e730b5ec8 port to v2 imports 2020-10-21 12:16:23 -07:00
Steven Allen
00dcb1bce9 Manage sectors by size instead of proof type. 
* We may have multiple sectors with the same size and different proof types, but all these management functions stay the same.
* This simplifies PoSt logic.
 2020-10-20 18:30:56 -07:00
Łukasz Magiera
8c86ea6b75 localworker: Try very hard to get ruselts to manager 2020-10-18 19:45:11 +02:00
Łukasz Magiera
dbb421c4f7 localworker: Use better context for calling returnFunc 2020-10-18 19:32:43 +02:00
Łukasz Magiera
8d06cca073 sched: Handle workers using sessions instead of connections 2020-10-18 12:36:06 +02:00
Łukasz Magiera
7ac5dc55d0 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-17 13:45:11 +02:00
Raúl Kripalani
7e6ede7563 update schema; test-vectors submodule. 2020-10-15 12:49:23 +01:00
Łukasz Magiera
b74a3229f8 fsm: process expired-ticket sectors 2020-10-13 21:35:41 +02:00
Łukasz Magiera
71b3b9075d Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-13 21:33:21 +02:00
Łukasz Magiera
65883cbb6b
Merge pull request #4290 from filecoin-project/feat/ledger-integration 
adding in ledger support
 2020-10-12 15:09:27 +02:00
Steven Allen
dc4e73c737 Test the tape upgrade 2020-10-12 00:01:25 -07:00
Łukasz Magiera
4ac2bee81c Update oni 2020-10-12 06:34:45 +02:00
Steven Allen
83dfc460d4 fix race in unseal 
1. Remove an invalid error check.
2. Make sure to shadow the outer error type from within the goroutine instead or
reading the outer type.

This may have been causing test issues (caught in TestMinerAllInfo with the race
detector).
 2020-10-09 15:39:41 -07:00
Raúl Kripalani
67b3a55e30 update test-vectors submodule. 2020-10-08 15:44:11 +01:00
Łukasz Magiera
0de3051821 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-08 13:10:41 +02:00
Steven Allen
767c346cf8 update test-vectors 2020-10-07 12:17:32 -07:00
Steven Allen
c17fa4bc35 update test-vectors for StateManager constructor change 2020-10-07 12:04:32 -07:00
Steven Allen
df4b068d0e Merge branch 'master' into next 2020-10-07 11:55:00 -07:00
Łukasz Magiera
cfd126ee9f
Merge pull request #3961 from filecoin-project/docs/miner-storage 
lotus-miner: add more help text to storage / attach
 2020-10-07 14:55:42 +02:00
Steven Allen
1e78cd21ec fix lotus soup build 2020-10-05 14:37:08 -07:00
Steven Allen
17846aad2f add some more policy toggles for testground 2020-10-05 14:28:47 -07:00
Łukasz Magiera
1fc23fb466 lotus-miner: Cleanup storage attach helptext a bit 2020-10-03 11:30:22 +02:00
Steven Allen
8292d60196 Merge branch 'master' into asr/spec-v1 2020-10-02 18:03:03 -07:00
Łukasz Magiera
5932f28519 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-01 02:39:48 +02:00
Łukasz Magiera
6981f776f4 Lower PC2 memory requirements 2020-10-01 00:54:53 +02:00
Łukasz Magiera
1b7cdb9341 Fix storage manager tests 2020-10-01 00:54:34 +02:00
Łukasz Magiera
5e08d56630 sched: Allow some single-thread tasks to run in parallel with PC2/C2 2020-10-01 00:28:44 +02:00
Łukasz Magiera
79d2ddf24f Review 2020-09-30 21:18:12 +02:00
Łukasz Magiera
2cfe22d4e5 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-30 20:48:16 +02:00
Łukasz Magiera
e3ee4e4718 Fix lint errors 2020-09-30 20:24:03 +02:00
Łukasz Magiera
2d16af6ee6 sectorstorage: Fix TestRedoPC1 2020-09-30 19:18:38 +02:00
Łukasz Magiera
4f97d9637e Fix storage-fsm tests 2020-09-30 19:11:58 +02:00
Steven Allen
4989b1cc90 Merge branch 'master' into asr/spec-v1 2020-09-30 09:19:12 -07:00
Łukasz Magiera
c228598098 sectorstorage: Variable scopes are really hard 2020-09-30 18:16:07 +02:00
Łukasz Magiera
a783bf9b8b storagefsm: Handle PC2 with missing replica 2020-09-30 17:53:15 +02:00
Łukasz Magiera
54fdd6ba5a sectorstorage: Variable scopes are hard 2020-09-30 17:48:48 +02:00
Łukasz Magiera
6855284d88 sectorstorage: Cancel non-running work in case of abort in sched 2020-09-30 17:26:09 +02:00
Łukasz Magiera
5bffea6f54
Merge pull request #4064 from filecoin-project/tvx 
tvx: a test vector extraction and execution tool
 2020-09-30 17:22:42 +02:00
Raúl Kripalani
ff8663faa0 update test-vectors submodule. 2020-09-30 14:58:22 +01:00
Łukasz Magiera
430897c9ac
Merge pull request #3445 from filecoin-project/asr/precommit-failed-fix 
Fix some failed precommit handling
 2020-09-30 09:35:41 +02:00
Łukasz Magiera
bc85e3ce50
Merge pull request #4107 from shaodan/worker-no-swap 
Add --no-swap flag for worker
 2020-09-30 09:25:57 +02:00
Dan Shao
1affd498c1 Add --no-swap flag for worker 2020-09-30 14:23:35 +08:00
Łukasz Magiera
baef3c8dd2 sectorstorage: Fix potential panic in FinalizeSector 2020-09-29 15:22:46 +02:00
Łukasz Magiera
0f2dcf28b1 fsm: Reuse tickets in PC1 on retry 2020-09-29 10:07:49 +02:00
Steven Allen
1c03862854 fix policy for specs-actors update 2020-09-28 17:28:16 -07:00
Steven Allen
233d8a9b72 update even more imports 2020-09-28 14:25:58 -07:00
Łukasz Magiera
1e6a69f8aa localworker: Don't mark calls as returned when returning fails 2020-09-28 22:10:02 +02:00
Łukasz Magiera
9bd2537971 stores: Fix error printing in http handler 2020-09-28 22:06:03 +02:00
Łukasz Magiera
810c767200 worker: Redeclare storage on reconnect 2020-09-28 21:06:49 +02:00
Łukasz Magiera
4ba7af6061 worker: Mark return methods as retry-safe 2020-09-28 20:46:44 +02:00
Łukasz Magiera
9e7d6823b1 sectorstorage: Cleanup callToWork mapping after work is done 2020-09-28 13:34:45 +02:00
Łukasz Magiera
cf71f034dc Merge remote-tracking branch 'origin/dev' into feat/async-restartable-workers 2020-09-26 12:35:32 +02:00
zgfzgf
1a7aea1906 modify error 2020-09-25 22:59:21 +08:00
zgfzgf
3207bc4704 optimize trySched 2020-09-25 22:41:29 +08:00
zgfzgf
60e950015c modify for unsafe 2020-09-25 22:13:27 +08:00
whyrusleeping
306c098d30 also update our vendored blst repo 2020-09-24 11:32:38 -07:00
whyrusleeping
6db37b72a8 update ffi to code with blst fixes 2020-09-24 11:05:21 -07:00
Łukasz Magiera
04ee53e061 sectorstorage: Show task type of ret-wait jobs 2020-09-24 11:55:11 +02:00
Steven Allen
32a699d6a3 Add some actors policy setters for testing 
Addresses:
* a307e4593a (r491966115)
* a307e4593a (r491966634)

Note: This puts everything into a policy package to avoid a dependency cycle
between the build package, the miner package, and the types package. This is
also why I introduced a GetPreCommitChallengeDelay function and removed the
variable.
 2020-09-23 15:00:52 -07:00
Łukasz Magiera
d817dceb05 Show lost calls in sealing jobs cli 2020-09-23 19:26:35 +02:00
Łukasz Magiera
c17f0d7e61 sectorstorage: Fix panic in returnResult 2020-09-23 17:37:05 +02:00
Łukasz Magiera
86c222ab58 sectorstorage: fix work tracking 2020-09-23 14:56:50 +02:00
Łukasz Magiera
ce6b92484f Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-23 11:31:21 +02:00
Steven Allen
773714792f update oni 2020-09-22 18:40:03 -07:00
Łukasz Magiera
6185e157e9 sectorstorage: calltracker: work around cbor-gen bytearray len limit 2020-09-23 00:29:10 +02:00
Łukasz Magiera
04ad1791b0 localworker: Fix contexts 2020-09-23 00:10:36 +02:00
Łukasz Magiera
bb5cc06677 Fix workid param hash 2020-09-22 23:33:13 +02:00
Travis Person
c66f087f4c lotus-miner: add more help text to storage / attach 2020-09-22 18:15:42 +00:00
Łukasz Magiera
706f4f2ef5 worker: Don't die with the connection 2020-09-22 18:36:44 +02:00
Aayush Rajasekaran
1dc69e397e Resolve some unnecessary actor upgrade TODOs 2020-09-22 01:34:21 -04:00
Łukasz Magiera
b8865fb182 workers: Mark on-restart-failed returned tasks as returned 2020-09-22 01:00:28 +02:00
Łukasz Magiera
03c3d8bdb3 workers: Return unfinished tasks on restart 2020-09-22 00:52:33 +02:00
Łukasz Magiera
aa5bd7bc17 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-21 22:36:16 +02:00
Łukasz Magiera
f7934b083c Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-21 19:10:18 +02:00
Alan Shaw
d9f2b909ac
chore: update test-vectors 2020-09-21 11:58:45 +01:00
Alan Shaw
a79291070e
chore: update test-vectors 2020-09-21 11:53:38 +01:00
Steven Allen
1bf3b4989d rename imports to match actors code 
`sed -i 's/\bv0\(\w\)\(\w*\)/\L\1\E\20/g' **/*.go`
 2020-09-18 14:59:27 -07:00
Łukasz Magiera
7109e95f5e
Merge pull request #3881 from filcloud/issue-3626-SectorState 
fix SectorState
 2020-09-18 21:30:18 +02:00
Łukasz Magiera
70faa36b7f Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-18 19:29:06 +02:00
Aayush Rajasekaran
fce423c743 Appease the linter and get everything building 2020-09-18 02:14:18 -04:00
Łukasz Magiera
6eda53565f Most tests passing 2020-09-17 17:30:24 +02:00
Łukasz Magiera
e632643801 api: Test return types 2020-09-17 12:24:50 +02:00
hannahhoward
691bd9f442 feat(markets): complete markets conversion 
complete markets conversion to using chain/actors types, also replacing DealProposal/DealState
interfaces with structs
 2020-09-17 00:43:14 -07:00
Aayush Rajasekaran
9e48dd211a Fixups 2020-09-17 02:34:15 -04:00
Aayush Rajasekaran
b5ba7a0fad Miner migration 2020-09-17 02:21:16 -04:00
Frank
d5af25b76c update init sector state list 2020-09-17 10:38:07 +08:00
Łukasz Magiera
17680fff55 gofmt 2020-09-17 00:35:57 +02:00
Łukasz Magiera
d9d644b27f sectorstorage: handle restarting manager, test that 2020-09-17 00:35:09 +02:00
Łukasz Magiera
5e09581256 sectorstorage: get new work tracker to run 2020-09-16 22:33:58 +02:00
Łukasz Magiera
b1361aaf8b sectorstorage: wip manager work tracker 2020-09-16 17:08:05 +02:00
Anton Evangelatov
0ff5a71114 update oni to master 2020-09-16 15:13:28 +02:00
Anton Evangelatov
b0677ab38e replace lotus and filecoin-ffi deps 2020-09-16 15:12:41 +02:00
Anton Evangelatov
3f67c4524e add filecoin-project oni as submodule and compile lotus-soup 2020-09-16 15:12:41 +02:00
Frank
ed74091c20 add exist sector state check 2020-09-16 19:49:45 +08:00
Łukasz Magiera
d4a7732d0a
Merge pull request #3800 from filecoin-project/asr/gascost 
Add an API to get detailed gas costs for a message
 2020-09-15 19:13:20 +02:00
Łukasz Magiera
03cf6cca40 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-15 17:47:03 +02:00
Steven Allen
36f920bcd7 progress 2020-09-14 21:55:49 -07:00
hannahhoward
7dc091052a feat(manager): less restrictive storage lock 
Use initial less restrictive storage lock when trying to read unsealed data before acquiring more
restrictive lock needed for unsealing
 2020-09-14 18:48:14 -07:00
Steven Allen
02dcb5e182 Merge branch 'master' into refactor/net-upgrade 2020-09-14 14:53:57 -07:00
Aayush Rajasekaran
0edcae9e04 Fixup new command 2020-09-14 14:40:38 -04:00
Łukasz Magiera
e9d25e5919 More fixes 2020-09-14 20:28:47 +02:00
Łukasz Magiera
381a6cdfac Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-14 19:11:50 +02:00
Łukasz Magiera
1ebca8f732 more working code 2020-09-14 19:09:01 +02:00
Łukasz Magiera
06ec571c20 Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-14 15:09:52 +02:00
Raúl Kripalani
6d29d75724 Merge branch 'master' into inmem-journal 2020-09-14 12:17:45 +01:00
Steven Allen
d3594835c4 [WIP] Network upgrade support 
This patch starts adding support for network upgrades.

* It adds an actors abstraction layer for loading abstract (cross-version) actors.
* It starts switching over to a shared deadline type.
* It adds an abstraction for ADTs (hamt/amt).
* It removes the callback-based API in the StateManager (difficult to abstract
across actor versions).
* It _does not_ actually add support for actors v2. We can do that in a followup
patch but that should be relatively easy.

This patch is heavily WIP and does not compile. Feel free to push changes
directly to this branch.

Notes:

* State tree access now needs a network version, because the HAMT type will change.
* I haven't figured out a nice way to abstract over changes to the _message_
types. However, many of them will be type aliased to actors v0 in actors v2 so
we can likely continue using the v0 versions (or use the v2 versions
everywhere). I've been renaming imports to `v0*` to make it clear that we're
importing types from a _specific_ actors version.

TODO:

* Consider merging incremental improvements? We'd have to get this compiling
again first but we could merge in the new abstractions, and slowly switch over.
* Finish migrating to the new abstractions.
* Remove all actor state types from the public API. See `miner.State.Info()` for
the planned approach here.
* Fix the tests. This is likely going to be a massive pain.
 2020-09-11 20:16:29 -07:00
Aarsh Shah
62b2963781 retry add piece 2020-09-11 16:58:09 +05:30
Łukasz Magiera
e996a5246b Merge remote-tracking branch 'origin/master' into feat/window-post-faulty-sectors 2020-09-10 22:18:17 +02:00
Łukasz Magiera
bbac86f745 gofmt, mod tidy 2020-09-10 22:07:20 +02:00
Łukasz Magiera
22556fb24c Add fil-blst submodule 2020-09-10 22:04:08 +02:00
Łukasz Magiera
2374cb1bc5 Update ffi 2020-09-10 21:44:01 +02:00
Łukasz Magiera
c7b0241a48 ffiwrapper: Test skipping corrupted sectors in PoSt 2020-09-10 21:19:26 +02:00
Łukasz Magiera
5f08fe7ead Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-10 17:30:54 +02:00
lanzafame
558b637b07 Increase the number of times precommit2 is attempted before moving back to precommit1 2020-09-10 13:52:21 +10:00
Łukasz Magiera
5e7737f55d wdpost: Handle skipped sectors correctly 2020-09-10 02:59:37 +02:00
whyrusleeping
7a6ceebb34 windowed post generation now returns faulty sectors 2020-09-09 14:00:15 -07:00
Raúl Kripalani
a5afd83c7b Merge branch 'master' into fix/test-vectors-import-cycle 2020-09-09 13:04:41 +01:00
Raúl Kripalani
e8d1bab914 make the runner call statediff as a binary. 2020-09-09 13:03:43 +01:00
Dirk McCormick
17c15a74a2 fix: return true from Sealer.ReadPiece() on success 2020-09-08 13:50:56 +02:00
Dirk McCormick
8bbdf2e7cb fix: storage manager - bail out on undefined unsealed cid 2020-09-08 12:54:01 +02:00
Aayush Rajasekaran
f91f665f3f Update test-vectors 2020-09-08 01:24:30 -04:00
Aayush Rajasekaran
d678fe4bfa Fix tests 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a Update to specs v0.9.6 2020-09-07 15:48:41 -04:00
Łukasz Magiera
47c59afea0
Revert "storage manager: bail out with an error if unsealed cid is undefined" 2020-09-07 20:12:29 +02:00
Łukasz Magiera
231a9e4051 Fix sealing sched tests 2020-09-07 17:55:31 +02:00
Dirk McCormick
a97f978cad fix: storage manager - dont fail on successful read piece 2020-09-07 16:14:19 +02:00
Łukasz Magiera
9e6f974f3c storage: Fix build 2020-09-07 16:12:55 +02:00
Dirk McCormick
07a4553e6e fix: storage manager - bail out with an error if unsealed cid is undefined 2020-09-07 16:04:12 +02:00
Łukasz Magiera
06e3852cef storage: Integrate async workers in sealing manager 2020-09-07 12:20:50 +02:00
Łukasz Magiera
5d73943929 storage: Fix import cycle 2020-09-06 18:54:00 +02:00
Łukasz Magiera
159ce13f5e Async worker API 2020-09-06 18:47:16 +02:00
Raúl Kripalani
1ec534d607 Merge branch 'master' into inmem-journal 2020-09-04 10:55:40 +01:00
Raúl Kripalani
d46f684bb8 fix test following merge. 2020-09-04 00:05:44 +01:00
Raúl Kripalani
c723554d2e upgrade test-vectors deps. 2020-09-03 23:44:21 +01:00
Raúl Kripalani
3206f92063 Merge branch 'master' into inmem-journal 2020-09-02 19:50:52 +01:00
Łukasz Magiera
b774563ec3
Merge pull request #3492 from filecoin-project/fix/readpiece-panic 
ffiwrapper: Fix ReadPiece panic
 2020-09-02 18:59:22 +02:00
Łukasz Magiera
ca7aa69597 ffiwrapper: More correct error check on openPartialFile 2020-09-02 18:45:07 +02:00
Łukasz Magiera
5f79ff340d
Merge pull request #3489 from filecoin-project/fix/sched-deadlocks 
sealing sched: Fix deadlock between worker.wndLk / workersLk
 2020-09-02 18:03:18 +02:00
Łukasz Magiera
5a2b439773 sched: Fix tests 2020-09-02 17:37:19 +02:00
Łukasz Magiera
7fe8580da5 sealing sched: Fix deadlock between worker.wndLk / workersLk 2020-09-02 17:06:48 +02:00
Raúl Kripalani
7cc9c83ab8 upgrade test-vectors. 2020-09-02 14:12:27 +01:00
Raúl Kripalani
0962292f92 conformance: add driver for tipset-class vectors. 2020-09-02 14:09:12 +01:00
Łukasz Magiera
fa6bae988e
Revert "conformance: add driver for tipset-class vectors." 2020-09-02 14:49:55 +02:00
Raúl Kripalani
ad5793e446 update test-vectors commit. 2020-09-01 20:01:41 +01:00
Raúl Kripalani
01737ef1cc upgrade test-vectors deps. 2020-09-01 17:00:18 +01:00
Star.LI
82f5984de1 fix crash - segment fault when partialFile.Allocated() is invoked. 
When openPartialFile is invoked, more errors than "existed error" are
returned. If only existing error is checked, the allocated field of
partialFile may be nil.

Signed-off-by: Star.LI <star@trapdoortech.com>
 2020-09-01 11:41:26 +02:00
Aayush Rajasekaran
f58e8bc9a3 Fix some failed precommit handling 2020-09-01 02:18:02 -04:00
Łukasz Magiera
e14c80360d sealing sched: Factor worker queues into utilization calc 2020-08-31 13:41:34 +02:00
Łukasz Magiera
98d51d3d80 storage: Correcty move unsealed sectors in FinalizeSector 2020-08-31 12:45:57 +02:00
Aayush Rajasekaran
27f7d3a658 Rebuild unsealed infos on miner restart 2020-08-30 18:01:07 -04:00
Łukasz Magiera
28ac2fce61 sched: Fix panic in workerCompactWindows 2020-08-29 06:41:19 +02:00
Łukasz Magiera
9d0c8ae3dd sectorstorage: update sched tests for new logic 2020-08-28 21:38:21 +02:00
Łukasz Magiera
4a75e1e4b4 sectorstorage: Don't require tasks within a window to run in order 2020-08-28 19:38:55 +02:00
Łukasz Magiera
11b11e416b sectorstorage: Compact assigned windows 2020-08-28 18:26:38 +02:00
Łukasz Magiera
20ffce5c0d Merge remote-tracking branch 'origin/master' into fix/some-fsm-issues 2020-08-28 16:47:25 +02:00
Łukasz Magiera
5ee85dc263 sectorstorage: Fix tests 2020-08-28 16:33:41 +02:00
刘林欣
93dc33687a ignore resend SectorStartPacking event, and unsealedInfoMap lock 2020-08-28 19:04:23 +08:00
Łukasz Magiera
6d1682a27e storagefsm: wire up RecoverDealIDs fully 2020-08-28 11:44:15 +02:00
Łukasz Magiera
1097d29213 sealing sched: Call trySched less when there are many tasks 2020-08-28 00:03:42 +02:00
Łukasz Magiera
a6492b1ed6 storagefsm: Auto-retry failed remove 2020-08-27 23:59:01 +02:00
Łukasz Magiera
59d2034cbb sealing sched: Wait a bit for tasks to come in on restart 2020-08-27 23:58:37 +02:00
Łukasz Magiera
7fdffc0340 sealing sched: Give more priority to tasks (re)moving data 2020-08-27 23:29:39 +02:00
Łukasz Magiera
f2bd680cc5 gofmt 2020-08-27 23:14:46 +02:00
Łukasz Magiera
59f554b658 sealing sched: Show waiting tasks assigned to workers in sealing jobs cli 2020-08-27 23:14:33 +02:00
Łukasz Magiera
d8e58e67c6 storagefsm: Treat PackingFailed sectors as expired 2020-08-27 22:41:35 +02:00
Łukasz Magiera
15204a9b90 storagefsm: fix tests 2020-08-27 21:34:45 +02:00
Łukasz Magiera
489d5239a5 storagefsm: Attempt to auto-recover from reorged DealIDs 2020-08-27 21:11:11 +02:00
Łukasz Magiera
7806a9885a storagefsm: Record publish deals message CID 2020-08-27 17:50:50 +02:00
Łukasz Magiera
29078aaae9 storagefsm: Retry sending messages on out-of-gas errors 2020-08-27 14:02:00 +02:00
Łukasz Magiera
df635579c4 storagefsm: Handle sectors with expired deals better 2020-08-27 13:51:38 +02:00
Łukasz Magiera
788c7dbf48 storagefsm: Separate satte for submitting commit message 2020-08-27 12:57:28 +02:00
Łukasz Magiera
15191ff80f storagefsm: Allow removing sectors in all states 2020-08-27 12:39:20 +02:00
Whyrusleeping
deb013cecb
Merge pull request #3081 from filecoin-project/conformance-tests 
driver, runner and CI for interoperable test-vector based testing.
 2020-08-26 14:06:39 -07:00
Raúl Kripalani
4709e231b8 drop aurora as dep; upgrade test-vectors. 2020-08-26 18:35:41 +01:00
Steven Allen
0155a31d1f Fix PoSt with bad sectors 
"skipped" sectors must be replaced with a substitute "good" sector, or the
entire partition must be skipped. They should not just be omitted.

This patch also fixes the test to verify the _entire_ proof instead of just
verifying that the proof includes the correct sectors.
 2020-08-26 09:56:51 -07:00
Raúl Kripalani
efdfd3ee3e Merge branch 'master' into inmem-journal 2020-08-26 16:38:23 +01:00
Raúl Kripalani
d3ac243cea Merge remote-tracking branch 'origin/master' into conformance-tests 2020-08-26 15:12:02 +01:00
Alan Shaw
a89faa72ea
feat(conformance): skip incorrect tests (#3310) 
Fixes https://github.com/filecoin-project/test-vectors/issues/72
 2020-08-26 11:45:34 +01:00
Łukasz Magiera
d9796cd25c sectorstorage: Make trySched less very slow 2020-08-24 19:16:16 +02:00
Łukasz Magiera
4311c96a44
Merge pull request #3225 from filecoin-project/fix/sched-missing-worker 
check that worker referenced by task is actually still there.
 2020-08-22 22:16:32 +02:00
Łukasz Magiera
a236957538
Merge pull request #3232 from filecoin-project/asr/marked-for-upgrade 
Indicate whether a sector has been marked for upgrade
 2020-08-22 21:48:45 +02:00
Łukasz Magiera
0806fd651a
Merge pull request #3183 from filecoin-project/steb/lint-all 
Lint everything
 2020-08-22 21:47:41 +02:00
Łukasz Magiera
0df6e1e2c7
Merge pull request #3220 from shaodan/fix-import 
Fix import storage-fsm
 2020-08-22 10:38:36 +02:00
Aayush Rajasekaran
984e52acbc Indicate whether a sector has been marked for upgrade 2020-08-21 15:53:39 -04:00
whyrusleeping
54862be3ff check that worker referenced by task is actually still there. 2020-08-21 10:33:36 -07:00
whyrusleeping
2e78f2c267 If sectorInfo doesnt have CommD or CommR set during commit, don't crash 2020-08-21 09:31:28 -07:00
Dan Shao
f21ce3e8ee Fix import storage-fsm -> extern/storage-sealing 2020-08-21 16:52:33 +08:00
Steven Allen
5733c71c50 Lint everything 
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.

There are three remaining XXXs marked in the code for lint.
 2020-08-20 20:46:36 -07:00
Aayush Rajasekaran
e9c526b218 Do not try to replace CC sectors with a deal-less sector 2020-08-19 22:41:23 -04:00
Raúl Kripalani
e38f4219c5 update test-vectors commit. 2020-08-19 18:19:03 +01:00
Raúl Kripalani
30b6f38704 update test-vectors commit. 2020-08-19 18:14:19 +01:00
Raúl Kripalani
04e4bab313 register chaos actor only if selector says to do so. 2020-08-19 18:10:56 +01:00
Raúl Kripalani
dddf7951ef update test-vectors commit. 2020-08-19 17:33:30 +01:00
Raúl Kripalani
a21234cd54 deploy the chaos actor. 2020-08-19 14:31:34 +01:00
Raúl Kripalani
4f4cebadc7 Merge branch 'master' into conformance-tests 2020-08-19 13:33:15 +01:00
Aayush Rajasekaran
7de7e65dbc Calculate depositMinimum for upgrading as replaced sector's Initialpledge 2020-08-18 19:22:43 -04:00
Aayush Rajasekaran
98f00b510f re-mark replaced sectors as for upgrade if precommit fails 2020-08-18 18:53:56 -04:00
Łukasz Magiera
a03192a39b Separate concurrent sealing sector limit for deals 2020-08-18 21:48:18 +02:00
Łukasz Magiera
6a7d72c030 fsm: fail early on too-many-sealing-sectors in pledge 2020-08-18 19:26:17 +02:00
Łukasz Magiera
e7d65be90a gofmt 2020-08-18 18:27:28 +02:00
Łukasz Magiera
baf91f08a1 Unbreak tests 2020-08-18 18:27:18 +02:00
Łukasz Magiera
886d9cd5eb fsm: handle already-precommitted CommitFailed sectors correctly 2020-08-18 18:02:25 +02:00
Łukasz Magiera
6a49bd6d8e fsm: Config for limiting max sealing sectors 2020-08-18 16:54:05 +02:00
Łukasz Magiera
65ffde9e4b fsm: Config for max WaitDeals sectors 2020-08-18 16:20:31 +02:00
Aayush Rajasekaran
37ff876bc6 restart WaitDeals timers if miner restarts 2020-08-18 05:07:14 -04:00
Łukasz Magiera
01df17f1e7
Merge pull request #3103 from filecoin-project/fix/fsm-recover-bork-snark 
fsm: Recover from errors from VerifySeal in checkCommit
 2020-08-17 19:00:16 +02:00
Łukasz Magiera
bb79af7721 fsm: Recover from errors from VerifySeal in checkCommit 2020-08-17 18:53:09 +02:00
Łukasz Magiera
a66e202f02 fsm: Fix panic on missing precommit info 2020-08-17 18:45:38 +02:00
Łukasz Magiera
6ef7a30b19
Merge pull request #3089 from filecoin-project/integrate/storage-fsm 
integrate extern/{storage-fsm,sector-storage} into lotus source tree
 2020-08-17 18:37:54 +02:00
Raúl Kripalani
1f016262cd readd int64 cast + nolint directive. 2020-08-17 17:26:07 +01:00
Raúl Kripalani
862bafc63a fix lint errors. 2020-08-17 14:39:33 +01:00
Raúl Kripalani
efdc428d5d keep storage-fsm (renamed to storage-sealing) and sector-storage in extern. 2020-08-17 14:26:18 +01:00
yaohcn
39bcfab37c make addpiece configurable 2020-08-17 17:39:50 +08:00
Raúl Kripalani
483e82e14b point test-vectors to master. 2020-08-16 22:05:43 +01:00
Raúl Kripalani
3c17cd655e integrate extern/sector-storage into lotus proper. 2020-08-16 11:09:58 +01:00
Raúl Kripalani
fb0ccc9260 integrate extern/storage-fsm into lotus proper. 2020-08-16 10:42:13 +01:00
Raúl Kripalani
322b33197c introduce interoperable vector-based conformance testing. 
This commit introduces a new package `conformance` containing:

 1. the test driver to exercise Lotus against interoperable
    test vectors, and
 2. the test runner, which integrates go test with the test vector
    corpus hosted at https://github.com/filecoin-project/conformance-vectors.

The corpus is mounted via a git submodule.

Right now, only message-class test vectors are supported. In the
next week, this support will be extended to tipset-class, chain-class,
and block sequence-class vectors.
 2020-08-16 00:15:52 +01:00
Łukasz Magiera
9328dc27f3 Update filecoin-ffi to fix Intel build 2020-08-15 13:19:10 +02:00
Łukasz Magiera
0c75dd3865 Support AddPiece on workers 2020-08-14 16:06:53 +02:00
Łukasz Magiera
632fd36205 sealing sched: Fix deadlock in worker watcher 2020-08-13 12:17:24 +02:00
yaohcn
1555984785 change to RLock 2020-08-13 17:31:18 +08:00
Steven Allen
3ef3f570fb Fix lint errors an broken tests 2020-08-12 23:16:44 +02:00
Łukasz Magiera
e1a1b325bc Merge remote-tracking branch 'origin/next' into feat/max-msg-fee-config 2020-08-12 21:55:03 +02:00
Łukasz Magiera
6f890a3edd Miner fee config 2020-08-12 19:47:09 +02:00
Steven Allen
41f1c74ae2 Set a minimum sector lifetime 2020-08-12 10:33:18 -07:00
Steven Allen
e18904bff5 [WIP] fix post submission 2020-08-12 10:33:18 -07:00
Steven Allen
9248e5a572 Update specs-actors, cbor-gen, and go-address 2020-08-12 10:32:39 -07:00
Steven Allen
9135a5d048 Pass bitfields by-value 
This ensures we can't end up decoding nil bitfields from clients when not
expecting them.

Part of https://github.com/filecoin-project/specs-actors/issues/895. Please see
this issue for details and leave any comments there.
 2020-08-12 10:32:39 -07:00
Łukasz Magiera
f537c48f93 v28 Proofs 2020-08-12 00:33:30 +02:00
Raúl Kripalani
d53c6f2599 storage-fsm: emit sealing state transition notifications. 2020-08-11 13:23:40 +01:00
yaohcn
6b0f607f4b add space check in StorageFindSector 2020-08-11 15:27:03 +08:00
Łukasz Magiera
624731cb80 More gomod fixing 2020-08-10 18:03:54 +02:00
Łukasz Magiera
641eb3e2cf Merge storage-fsm subtree 2020-08-10 17:40:27 +02:00
Łukasz Magiera
85d8133f4a Fix gomods 2020-08-10 17:31:14 +02:00
Łukasz Magiera
0eaf44eb31 Merge sector-storage subtree 2020-08-10 17:25:46 +02:00
Jakub Sztandera
f55ed9ea09
Update serialization vectors 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-08-07 01:21:01 +02:00
Peter Rabbitson
e266f3b955 Bump fil-commcid and filecoin-ffi deps 
Propagates correct on-chain commX CIDs
 2020-07-17 00:38:42 +02:00
Peter Rabbitson
d244749f1a Bump fil-commcid and filecoin-ffi deps 
Propagates correct on-chain commX CIDs
 2020-07-16 23:00:22 +02:00
Łukasz Magiera
17039a5672 Update FFI 2020-07-03 02:23:45 +02:00
Łukasz Magiera
c1ac986f20 Update FFI 2020-06-30 19:05:47 +02:00
Łukasz Magiera
77b3bf8bc2 Update FFI to 0.30.1 2020-06-29 20:24:27 +02:00
Łukasz Magiera
18dd1c277b Update ffi 2020-06-16 12:28:28 +02:00
Łukasz Magiera
907364ce67 Update deps, fix more tests 2020-06-15 18:30:49 +02:00
Łukasz Magiera
bd2ff4e909 specs-actors v0.6 2020-06-15 18:26:01 +02:00
Łukasz Magiera
3d1225bf34 gofmt 2020-06-15 16:28:57 +02:00
Łukasz Magiera
464f6a6b08 Update specs-actors to v0.6 2020-06-15 14:49:18 +02:00
Łukasz Magiera
f67bbbc457 specs-actors v0.6 2020-06-15 14:32:17 +02:00
Łukasz Magiera
a00df0da90 FFI v27 2020-06-10 01:44:14 +02:00
Łukasz Magiera
796af1351c v27 parameters 2020-06-09 01:02:44 +02:00
Jakub Sztandera
d6615b6286
Cleanup many lint warnings 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-05-27 22:53:20 +02:00
Jeromy
646800b8a8 update to latest ffi 2020-05-22 09:26:14 -07:00
Jeromy
a59ca7536a update to latest filecoin-ffi 2020-05-21 18:19:46 -07:00
Jeromy
fad9412c04 update to latest filecoin-ffi 2020-05-21 18:03:34 -07:00
Jeromy
e6f3b79b23 update serialization vectors 2020-05-18 11:02:34 -07:00
Łukasz Magiera
0ae0e89ddd Update FFI 2020-05-15 19:16:06 +02:00
Łukasz Magiera
f7314646ec Update FFI 2020-05-12 20:18:58 +02:00
Łukasz Magiera
6002e359ae Updates for 64G sectors 2020-05-11 22:24:42 +02:00
laser
32d092b293 v26 groth parameters 2020-05-08 14:13:37 -07:00
laser
b24920c606 update to v26 proofs 2020-05-08 13:08:02 -07:00
Łukasz Magiera
00c107548e ffi: use v26 master 2020-05-08 20:36:32 +02:00
Łukasz Magiera
75642f393d v26 2020-05-07 01:22:59 +02:00
Jeromy
6a27c65201 updating to specs actors v0.2.0 2020-04-28 13:52:07 -07:00
Frrist
0f0589bcf5 Update validation driver to match latest in chain-validation (#1532) 
* update driver to match new chain-validation interfaces

* update chain-validation

* update filecoin-ffi

* gofmt

* update chain-validation again
 2020-04-24 15:08:16 -07:00
Łukasz Magiera
ff5494af4e Cleanup gohacks, use ffi master 2020-04-13 22:21:13 +02:00
Łukasz Magiera
26b8ff722d Update Deps 2020-04-13 22:00:04 +02:00