cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
13,088 Commits 24 Branches 342 Tags 314 MiB
c31e5596e8
Commit Graph

271 Commits

Author SHA1 Message Date
Łukasz Magiera
1946d2ffd4 Wire up Precommit Batching 2021-05-18 17:37:52 +02:00
Łukasz Magiera
f66b9c5663 Maybe working precommit batching 2021-05-18 17:21:16 +02:00
Łukasz Magiera
d92c5e1001 Missing precommit batcher 2021-05-18 16:54:55 +02:00
Łukasz Magiera
c7ba083fa4 Import precommit batcher 2021-05-18 16:51:06 +02:00
vyzo
c544f4ce44 avoid extraneous rpc call for storage start epoch 2021-05-18 16:59:11 +03:00
vyzo
6b3e04b9b1 cache sector deadlines. 2021-05-18 15:28:51 +03:00
vyzo
74bad49068 correctly handle commit batch timer 2021-05-18 14:30:47 +03:00
vyzo
357c0868b7 proper config for termination batching and commit wait 2021-05-18 12:20:19 +03:00
Łukasz Magiera
ba2032c642 Fix some aggregation bugs 2021-05-17 22:51:29 +02:00
Łukasz Magiera
6278bdc69a Make things build 2021-05-17 20:47:41 +02:00
Raúl Kripalani
2d7f4b1c61 docs: add godocs to storage module. 2021-05-14 19:45:47 +01:00
Łukasz Magiera
506f39b294 WIP: Integrate FIP0013 2021-05-11 22:10:29 -04:00
Łukasz Magiera
e76935147f Merge remote-tracking branch 'origin/master' into feat/v1-api 2021-04-13 14:24:22 +02:00
Łukasz Magiera
9fb345af67 storagefsm: Test applying global events in broken state 2021-04-06 15:38:13 +02:00
Łukasz Magiera
81bd27911f Propagate StateMsg api changes 2021-04-05 19:56:53 +02:00
Łukasz Magiera
9a2ec07837 storagefsm: Apply global events even in broken states 2021-04-05 18:25:29 +02:00
Łukasz Magiera
a7122d424b
Merge pull request #5527 from filcloud/handle-ticket-expire 
fix precommit ticket expiration handling
 2021-03-29 22:55:43 +02:00
Łukasz Magiera
0974e02089 storagefsm: common createSector method 2021-03-12 18:42:17 +01:00
Łukasz Magiera
e5b5bf1c1b storagefsm: Update sector stats atomically with sector creation 2021-03-12 17:25:24 +01:00
Łukasz Magiera
679f5f7661 storagefsm: Helper stat functions in updateSector 2021-03-12 17:00:20 +01:00
Łukasz Magiera
957d3f0636 storgefsm: Use locks correctly 2021-03-12 15:01:00 +01:00
Łukasz Magiera
5c605d99e2 storgefsm: Fix tests 2021-03-12 14:56:46 +01:00
Łukasz Magiera
f5ed25371b storagefsm: Trigger input processing wheen below limits 2021-03-12 14:54:10 +01:00
Łukasz Magiera
d0243904f7
Merge pull request #5779 from filecoin-project/feat/storagefsm-no-restart-log 
storagefsm: Don't log the SectorRestart event
 2021-03-11 15:18:32 +01:00
Łukasz Magiera
881c722025 storagefsm: Fix double unlock with ready WaitDeals sectors 2021-03-11 15:03:51 +01:00
Łukasz Magiera
a1b6a2f011 storagefsm: Don't log the SectorRestart event 2021-03-11 12:52:34 +01:00
Łukasz Magiera
df0a8d9507
Merge pull request #5759 from filecoin-project/asr/terminate-livecheck 
Check liveness of sectors when processing termination batches
 2021-03-10 11:14:11 +01:00
Aayush Rajasekaran
47064f987c Check liveness of sectors when processing termination batches 2021-03-09 23:36:30 -05:00
Łukasz Magiera
29c9fa3137
Merge pull request #5612 from filecoin-project/feat/cc-fsm-cleanup 
storagefsm: Cleanup CC sector creation
 2021-02-26 11:39:28 +01:00
Łukasz Magiera
e49a412f6d
Merge pull request #5375 from filecoin-project/feat/refactor-fsm-input 
storagefsm: Rewrite input handling
 2021-02-25 14:28:04 +01:00
Łukasz Magiera
2b9ed9fa3d Remove sectors with all deals expired in RecoverDealIDs 2021-02-22 21:03:51 +01:00
Łukasz Magiera
26399dba70 Update markets, cbor-gen with soft map decoding 2021-02-19 20:11:43 +01:00
Łukasz Magiera
f719765069 storageminer: exit PledgeSector after sectors enter sealing pipeline 2021-02-16 17:41:58 +01:00
Łukasz Magiera
fc5e243c92 storagefsm: Cleanup CC sector creation 2021-02-16 17:14:59 +01:00
Łukasz Magiera
6907e5879d Fix WaitDeals sector accounting 2021-02-11 13:52:00 +01:00
Łukasz Magiera
e27a530cbc storagefsm: cleanup openSectors better; pendingPieces by pieceCid 2021-02-09 18:44:41 +01:00
He Weidong
0e2e1125d3 fix precommit ticket expiration handling 2021-02-05 10:45:15 +08:00
Łukasz Magiera
6e7fcb7ee6 Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input 2021-02-03 14:01:56 +01:00
Łukasz Magiera
1e7a107484
Merge pull request #5433 from filecoin-project/feat/keep-unsealed-cfg 
Add miner config to always keep ensealed deal copies
 2021-02-03 12:40:02 +01:00
Łukasz Magiera
94009f247f
Merge pull request #5411 from filecoin-project/feat/sealing-handle-batch-publish 
Handle batch publish storage deals message in sealing recovery
 2021-01-28 19:33:49 +01:00
Łukasz Magiera
d4c7b63aac Add miner config to always keep ensealed deal copies 2021-01-26 17:50:31 +01:00
Dirk McCormick
1b494acc9e feat: handle batch publish storage deals message in sealing recovery 2021-01-25 14:10:05 +01:00
Steven Allen
03cd3760bb correctly pick the seal type based on the network version 
Of course, we should really just run all of our tests post actors v2.
 2021-01-21 15:21:20 -08:00
Aayush Rajasekaran
ab90a3b2bc Actors update: MinerInfo.SealProofType has been removed 2021-01-21 15:21:19 -08:00
Łukasz Magiera
1070ad2289 storagefsm: Drop unused TargetWaitDealsSectors 2021-01-21 22:20:16 +01:00
Łukasz Magiera
ec4deb7e28 storagefsm: Fix unlocking in handleWaitDeals 2021-01-21 19:59:18 +01:00
Łukasz Magiera
1336d8855d storagefsm: Drop addpiece wait after fixing storageadapter 2021-01-21 17:40:50 +01:00
Łukasz Magiera
df14f156e1 storagefsm: More logging for deal test debugging 2021-01-21 17:40:50 +01:00
Łukasz Magiera
b9a9f23204 storagefsm: Add stub AddPieceFailed state 2021-01-21 17:40:50 +01:00
Łukasz Magiera
f96f12c836 storagefsm: Add rest of checks in WaitDeals 2021-01-21 17:40:50 +01:00
Łukasz Magiera
069766ecc4 storagefsm: Don't persist piece assignment queue 2021-01-21 17:40:50 +01:00
Łukasz Magiera
270f2935a9 storagefsm: Check per-sector deal limits 2021-01-21 17:40:50 +01:00
Łukasz Magiera
fd67a41c75 storagefsm: Change sector CreationTime to unix ts 2021-01-21 17:40:50 +01:00
Łukasz Magiera
9857ad8378 storagefsm: Fix some deadlock cases 2021-01-21 17:40:50 +01:00
Łukasz Magiera
e5814dac4f cbor-gen 2021-01-21 17:40:50 +01:00
Łukasz Magiera
542357a1df storagefsm: Start packing correctly 2021-01-21 17:40:50 +01:00
Łukasz Magiera
239d6f8f4d storagefsm: Rewrite input handling 2021-01-21 17:40:50 +01:00
Łukasz Magiera
0eaa6d9983 storagefsm: Send correct event on ErrExpiredTicket in CommitFailed 2021-01-17 22:29:07 +01:00
Łukasz Magiera
80b8d4b9d7 Address review 2021-01-14 20:27:15 +01:00
Łukasz Magiera
32885e1129 termination batcher: Notify based on what was sent 2021-01-14 17:14:26 +01:00
Łukasz Magiera
4015ddbb4f fsm: Fix panic in precommit check in handleTerminating 2021-01-14 16:13:32 +01:00
Łukasz Magiera
49abdd7d7d Sector termination support - address review 2021-01-14 15:46:57 +01:00
Łukasz Magiera
7ddf1d1feb Add terminating states to state lists 2021-01-14 12:44:27 +01:00
Łukasz Magiera
144b5a1350 perning termination API 2021-01-14 12:37:23 +01:00
Łukasz Magiera
1564db1fce Sector termination test 2021-01-14 00:11:41 +01:00
Łukasz Magiera
3522c8d45a SectorTerminateFlush API 2021-01-13 23:32:04 +01:00
Łukasz Magiera
174c595acc Don't declare in proving window 2021-01-13 22:19:10 +01:00
Łukasz Magiera
52cc2cd3eb Initial sector termination support 2021-01-13 00:42:01 +01:00
Łukasz Magiera
299e72458e storagefsm: Fix unsealedInfoMap.lk init race 2021-01-10 14:01:29 +01:00
zzx234234
7db5e8061d fix a coding error in truncating sector log 2020-12-11 11:31:36 +08:00
stirlingx
3c50baaf5b
fix curSealing out of MaxSealingSectors limit 2020-12-10 11:04:42 +08:00
Łukasz Magiera
1502ca6460 storagefsm: Fix addr selection for commits 2020-12-03 12:30:41 +01:00
Łukasz Magiera
562f5839b7 Fix lint 2020-12-02 22:01:09 +01:00
Łukasz Magiera
2fd93a55ac plumb AddressSelector to stoage fsm 2020-12-02 21:47:54 +01:00
Łukasz Magiera
2304ea0077
Merge pull request #5016 from filecoin-project/fix/fsm-rm-unceroverable 
storagefsm: Add missing planners
 2020-11-26 12:22:48 +01:00
Łukasz Magiera
74f040f9d5 storagefsm: Add SubmitCommit to toStatState switch 2020-11-26 12:02:35 +01:00
Łukasz Magiera
eeb38fa649 storagefsm: Test fsmPlanners map 2020-11-26 10:58:07 +01:00
Łukasz Magiera
ba87cb4fa9 storagefsm: Add missing planners 2020-11-26 10:57:47 +01:00
Łukasz Magiera
65653a1747 storagefsm: Add GetTicket to some maps 2020-11-26 10:53:31 +01:00
Łukasz Magiera
89493fe2b8 storagefsm: newDealSector review 2020-11-25 18:55:58 +01:00
Łukasz Magiera
6166204bac storagefsm: Improve new deal sector logic 2020-11-25 16:40:04 +01:00
Łukasz Magiera
214f46bc88
Merge pull request #4876 from filecoin-project/fix/miner-ticket-loop 
storagefsm: Fix expired ticket retry loop
 2020-11-24 19:54:41 +01:00
yaohcn
7c0b6f41d8 fix log format 2020-11-24 19:09:48 +08:00
Łukasz Magiera
83b1009837 storagefsm: check getticket precommit expication correctly 2020-11-23 19:12:54 +01:00
Łukasz Magiera
85e592bbe2 storagefsm: Retry pushing messages when out-of-funds 2020-11-23 19:03:31 +01:00
Łukasz Magiera
e2a6b99144 storagefsm: Fix expired ticket retry loop 2020-11-23 19:03:31 +01:00
hannahhoward
218f7c2c5d feat(markets): update markets, extract commp 2020-11-20 13:29:14 -08:00
Aayush Rajasekaran
68a16afeda Merge branch 'master' into release/v1.2.0 2020-11-17 19:40:05 -05:00
Łukasz Magiera
00335878ab
Merge pull request #4712 from filecoin-project/fix/commitfailed-checks 
Check (pre)commit receipt before other checks in failed states
 2020-11-18 01:26:02 +01:00
Łukasz Magiera
6bea9dd178 Making sealing logic work with multiple seal proof types 2020-11-16 19:03:30 +01:00
Łukasz Magiera
11b041224e fsm: Allow manually resurrecting removed sectors 2020-11-12 19:37:54 +01:00
zgfzgf
3665f90795 optimize pledge sector 2020-11-08 16:24:50 +08:00
Łukasz Magiera
cde0e58e39 Check (pre)commit receipt before other checks in failed states 2020-11-03 18:47:29 +01:00
Łukasz Magiera
33459f1b36 storagefsm: Fix GetTicket loop when the sector is already precommitted 2020-10-29 10:26:02 +01:00
Łukasz Magiera
660236b224 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-23 23:25:35 +02:00
Steven Allen
bcabe7b3b5 migrate methods to abstracted methods 
Method numbers never change anyways. At worst, we'll deprecate old methods and
have to explicitly import them from the correct actors version to use them.
 2020-10-21 12:18:37 -07:00
Steven Allen
4e730b5ec8 port to v2 imports 2020-10-21 12:16:23 -07:00
Łukasz Magiera
b74a3229f8 fsm: process expired-ticket sectors 2020-10-13 21:35:41 +02:00
Łukasz Magiera
0de3051821 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-08 13:10:41 +02:00
Łukasz Magiera
2cfe22d4e5 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-30 20:48:16 +02:00
Łukasz Magiera
e3ee4e4718 Fix lint errors 2020-09-30 20:24:03 +02:00
Łukasz Magiera
4f97d9637e Fix storage-fsm tests 2020-09-30 19:11:58 +02:00
Steven Allen
4989b1cc90 Merge branch 'master' into asr/spec-v1 2020-09-30 09:19:12 -07:00
Łukasz Magiera
a783bf9b8b storagefsm: Handle PC2 with missing replica 2020-09-30 17:53:15 +02:00
Łukasz Magiera
430897c9ac
Merge pull request #3445 from filecoin-project/asr/precommit-failed-fix 
Fix some failed precommit handling
 2020-09-30 09:35:41 +02:00
Łukasz Magiera
0f2dcf28b1 fsm: Reuse tickets in PC1 on retry 2020-09-29 10:07:49 +02:00
Steven Allen
1c03862854 fix policy for specs-actors update 2020-09-28 17:28:16 -07:00
Steven Allen
233d8a9b72 update even more imports 2020-09-28 14:25:58 -07:00
Steven Allen
32a699d6a3 Add some actors policy setters for testing 
Addresses:
* a307e4593a (r491966115)
* a307e4593a (r491966634)

Note: This puts everything into a policy package to avoid a dependency cycle
between the build package, the miner package, and the types package. This is
also why I introduced a GetPreCommitChallengeDelay function and removed the
variable.
 2020-09-23 15:00:52 -07:00
Aayush Rajasekaran
1dc69e397e Resolve some unnecessary actor upgrade TODOs 2020-09-22 01:34:21 -04:00
Łukasz Magiera
f7934b083c Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-21 19:10:18 +02:00
Steven Allen
1bf3b4989d rename imports to match actors code 
`sed -i 's/\bv0\(\w\)\(\w*\)/\L\1\E\20/g' **/*.go`
 2020-09-18 14:59:27 -07:00
Łukasz Magiera
70faa36b7f Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-18 19:29:06 +02:00
Aayush Rajasekaran
fce423c743 Appease the linter and get everything building 2020-09-18 02:14:18 -04:00
Łukasz Magiera
6eda53565f Most tests passing 2020-09-17 17:30:24 +02:00
hannahhoward
691bd9f442 feat(markets): complete markets conversion 
complete markets conversion to using chain/actors types, also replacing DealProposal/DealState
interfaces with structs
 2020-09-17 00:43:14 -07:00
Aayush Rajasekaran
9e48dd211a Fixups 2020-09-17 02:34:15 -04:00
Aayush Rajasekaran
b5ba7a0fad Miner migration 2020-09-17 02:21:16 -04:00
Frank
d5af25b76c update init sector state list 2020-09-17 10:38:07 +08:00
Frank
ed74091c20 add exist sector state check 2020-09-16 19:49:45 +08:00
Steven Allen
36f920bcd7 progress 2020-09-14 21:55:49 -07:00
Steven Allen
02dcb5e182 Merge branch 'master' into refactor/net-upgrade 2020-09-14 14:53:57 -07:00
Aayush Rajasekaran
0edcae9e04 Fixup new command 2020-09-14 14:40:38 -04:00
Łukasz Magiera
06ec571c20 Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-14 15:09:52 +02:00
Raúl Kripalani
6d29d75724 Merge branch 'master' into inmem-journal 2020-09-14 12:17:45 +01:00
Steven Allen
d3594835c4 [WIP] Network upgrade support 
This patch starts adding support for network upgrades.

* It adds an actors abstraction layer for loading abstract (cross-version) actors.
* It starts switching over to a shared deadline type.
* It adds an abstraction for ADTs (hamt/amt).
* It removes the callback-based API in the StateManager (difficult to abstract
across actor versions).
* It _does not_ actually add support for actors v2. We can do that in a followup
patch but that should be relatively easy.

This patch is heavily WIP and does not compile. Feel free to push changes
directly to this branch.

Notes:

* State tree access now needs a network version, because the HAMT type will change.
* I haven't figured out a nice way to abstract over changes to the _message_
types. However, many of them will be type aliased to actors v0 in actors v2 so
we can likely continue using the v0 versions (or use the v2 versions
everywhere). I've been renaming imports to `v0*` to make it clear that we're
importing types from a _specific_ actors version.

TODO:

* Consider merging incremental improvements? We'd have to get this compiling
again first but we could merge in the new abstractions, and slowly switch over.
* Finish migrating to the new abstractions.
* Remove all actor state types from the public API. See `miner.State.Info()` for
the planned approach here.
* Fix the tests. This is likely going to be a massive pain.
 2020-09-11 20:16:29 -07:00
Aarsh Shah
62b2963781 retry add piece 2020-09-11 16:58:09 +05:30
lanzafame
558b637b07 Increase the number of times precommit2 is attempted before moving back to precommit1 2020-09-10 13:52:21 +10:00
Aayush Rajasekaran
d678fe4bfa Fix tests 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a Update to specs v0.9.6 2020-09-07 15:48:41 -04:00
Raúl Kripalani
d46f684bb8 fix test following merge. 2020-09-04 00:05:44 +01:00
Raúl Kripalani
3206f92063 Merge branch 'master' into inmem-journal 2020-09-02 19:50:52 +01:00
Aayush Rajasekaran
f58e8bc9a3 Fix some failed precommit handling 2020-09-01 02:18:02 -04:00
Aayush Rajasekaran
27f7d3a658 Rebuild unsealed infos on miner restart 2020-08-30 18:01:07 -04:00
Łukasz Magiera
20ffce5c0d Merge remote-tracking branch 'origin/master' into fix/some-fsm-issues 2020-08-28 16:47:25 +02:00
刘林欣
93dc33687a ignore resend SectorStartPacking event, and unsealedInfoMap lock 2020-08-28 19:04:23 +08:00
Łukasz Magiera
6d1682a27e storagefsm: wire up RecoverDealIDs fully 2020-08-28 11:44:15 +02:00
Łukasz Magiera
a6492b1ed6 storagefsm: Auto-retry failed remove 2020-08-27 23:59:01 +02:00
Łukasz Magiera
f2bd680cc5 gofmt 2020-08-27 23:14:46 +02:00
Łukasz Magiera
d8e58e67c6 storagefsm: Treat PackingFailed sectors as expired 2020-08-27 22:41:35 +02:00
Łukasz Magiera
15204a9b90 storagefsm: fix tests 2020-08-27 21:34:45 +02:00
Łukasz Magiera
489d5239a5 storagefsm: Attempt to auto-recover from reorged DealIDs 2020-08-27 21:11:11 +02:00
Łukasz Magiera
7806a9885a storagefsm: Record publish deals message CID 2020-08-27 17:50:50 +02:00
Łukasz Magiera
29078aaae9 storagefsm: Retry sending messages on out-of-gas errors 2020-08-27 14:02:00 +02:00
Łukasz Magiera
df635579c4 storagefsm: Handle sectors with expired deals better 2020-08-27 13:51:38 +02:00
Łukasz Magiera
788c7dbf48 storagefsm: Separate satte for submitting commit message 2020-08-27 12:57:28 +02:00
Łukasz Magiera
15191ff80f storagefsm: Allow removing sectors in all states 2020-08-27 12:39:20 +02:00
Raúl Kripalani
efdfd3ee3e Merge branch 'master' into inmem-journal 2020-08-26 16:38:23 +01:00
Łukasz Magiera
a236957538
Merge pull request #3232 from filecoin-project/asr/marked-for-upgrade 
Indicate whether a sector has been marked for upgrade
 2020-08-22 21:48:45 +02:00
Łukasz Magiera
0806fd651a
Merge pull request #3183 from filecoin-project/steb/lint-all 
Lint everything
 2020-08-22 21:47:41 +02:00