Łukasz Magiera
e400bdf87a
Order proofs before aggregation
2021-05-18 18:58:41 +02:00
Łukasz Magiera
1946d2ffd4
Wire up Precommit Batching
2021-05-18 17:37:52 +02:00
Łukasz Magiera
f66b9c5663
Maybe working precommit batching
2021-05-18 17:21:16 +02:00
Łukasz Magiera
d92c5e1001
Missing precommit batcher
2021-05-18 16:54:55 +02:00
Łukasz Magiera
c7ba083fa4
Import precommit batcher
2021-05-18 16:51:06 +02:00
vyzo
c544f4ce44
avoid extraneous rpc call for storage start epoch
2021-05-18 16:59:11 +03:00
vyzo
6b3e04b9b1
cache sector deadlines.
2021-05-18 15:28:51 +03:00
vyzo
74bad49068
correctly handle commit batch timer
2021-05-18 14:30:47 +03:00
vyzo
357c0868b7
proper config for termination batching and commit wait
2021-05-18 12:20:19 +03:00
Łukasz Magiera
ba2032c642
Fix some aggregation bugs
2021-05-17 22:51:29 +02:00
Łukasz Magiera
6278bdc69a
Make things build
2021-05-17 20:47:41 +02:00
Raúl Kripalani
2d7f4b1c61
docs: add godocs to storage module.
2021-05-14 19:45:47 +01:00
Łukasz Magiera
506f39b294
WIP: Integrate FIP0013
2021-05-11 22:10:29 -04:00
Łukasz Magiera
e76935147f
Merge remote-tracking branch 'origin/master' into feat/v1-api
2021-04-13 14:24:22 +02:00
Łukasz Magiera
9fb345af67
storagefsm: Test applying global events in broken state
2021-04-06 15:38:13 +02:00
Łukasz Magiera
81bd27911f
Propagate StateMsg api changes
2021-04-05 19:56:53 +02:00
Łukasz Magiera
9a2ec07837
storagefsm: Apply global events even in broken states
2021-04-05 18:25:29 +02:00
Łukasz Magiera
a7122d424b
Merge pull request #5527 from filcloud/handle-ticket-expire
...
fix precommit ticket expiration handling
2021-03-29 22:55:43 +02:00
Łukasz Magiera
0974e02089
storagefsm: common createSector method
2021-03-12 18:42:17 +01:00
Łukasz Magiera
e5b5bf1c1b
storagefsm: Update sector stats atomically with sector creation
2021-03-12 17:25:24 +01:00
Łukasz Magiera
679f5f7661
storagefsm: Helper stat functions in updateSector
2021-03-12 17:00:20 +01:00
Łukasz Magiera
957d3f0636
storgefsm: Use locks correctly
2021-03-12 15:01:00 +01:00
Łukasz Magiera
5c605d99e2
storgefsm: Fix tests
2021-03-12 14:56:46 +01:00
Łukasz Magiera
f5ed25371b
storagefsm: Trigger input processing wheen below limits
2021-03-12 14:54:10 +01:00
Łukasz Magiera
d0243904f7
Merge pull request #5779 from filecoin-project/feat/storagefsm-no-restart-log
...
storagefsm: Don't log the SectorRestart event
2021-03-11 15:18:32 +01:00
Łukasz Magiera
881c722025
storagefsm: Fix double unlock with ready WaitDeals sectors
2021-03-11 15:03:51 +01:00
Łukasz Magiera
a1b6a2f011
storagefsm: Don't log the SectorRestart event
2021-03-11 12:52:34 +01:00
Łukasz Magiera
df0a8d9507
Merge pull request #5759 from filecoin-project/asr/terminate-livecheck
...
Check liveness of sectors when processing termination batches
2021-03-10 11:14:11 +01:00
Aayush Rajasekaran
47064f987c
Check liveness of sectors when processing termination batches
2021-03-09 23:36:30 -05:00
Łukasz Magiera
29c9fa3137
Merge pull request #5612 from filecoin-project/feat/cc-fsm-cleanup
...
storagefsm: Cleanup CC sector creation
2021-02-26 11:39:28 +01:00
Łukasz Magiera
e49a412f6d
Merge pull request #5375 from filecoin-project/feat/refactor-fsm-input
...
storagefsm: Rewrite input handling
2021-02-25 14:28:04 +01:00
Łukasz Magiera
2b9ed9fa3d
Remove sectors with all deals expired in RecoverDealIDs
2021-02-22 21:03:51 +01:00
Łukasz Magiera
26399dba70
Update markets, cbor-gen with soft map decoding
2021-02-19 20:11:43 +01:00
Łukasz Magiera
f719765069
storageminer: exit PledgeSector after sectors enter sealing pipeline
2021-02-16 17:41:58 +01:00
Łukasz Magiera
fc5e243c92
storagefsm: Cleanup CC sector creation
2021-02-16 17:14:59 +01:00
Łukasz Magiera
6907e5879d
Fix WaitDeals sector accounting
2021-02-11 13:52:00 +01:00
Łukasz Magiera
e27a530cbc
storagefsm: cleanup openSectors better; pendingPieces by pieceCid
2021-02-09 18:44:41 +01:00
He Weidong
0e2e1125d3
fix precommit ticket expiration handling
2021-02-05 10:45:15 +08:00
Łukasz Magiera
6e7fcb7ee6
Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input
2021-02-03 14:01:56 +01:00
Łukasz Magiera
1e7a107484
Merge pull request #5433 from filecoin-project/feat/keep-unsealed-cfg
...
Add miner config to always keep ensealed deal copies
2021-02-03 12:40:02 +01:00
Łukasz Magiera
94009f247f
Merge pull request #5411 from filecoin-project/feat/sealing-handle-batch-publish
...
Handle batch publish storage deals message in sealing recovery
2021-01-28 19:33:49 +01:00
Łukasz Magiera
d4c7b63aac
Add miner config to always keep ensealed deal copies
2021-01-26 17:50:31 +01:00
Dirk McCormick
1b494acc9e
feat: handle batch publish storage deals message in sealing recovery
2021-01-25 14:10:05 +01:00
Steven Allen
03cd3760bb
correctly pick the seal type based on the network version
...
Of course, we should really just run all of our tests post actors v2.
2021-01-21 15:21:20 -08:00
Aayush Rajasekaran
ab90a3b2bc
Actors update: MinerInfo.SealProofType has been removed
2021-01-21 15:21:19 -08:00
Łukasz Magiera
1070ad2289
storagefsm: Drop unused TargetWaitDealsSectors
2021-01-21 22:20:16 +01:00
Łukasz Magiera
ec4deb7e28
storagefsm: Fix unlocking in handleWaitDeals
2021-01-21 19:59:18 +01:00
Łukasz Magiera
1336d8855d
storagefsm: Drop addpiece wait after fixing storageadapter
2021-01-21 17:40:50 +01:00
Łukasz Magiera
df14f156e1
storagefsm: More logging for deal test debugging
2021-01-21 17:40:50 +01:00
Łukasz Magiera
b9a9f23204
storagefsm: Add stub AddPieceFailed state
2021-01-21 17:40:50 +01:00
Łukasz Magiera
f96f12c836
storagefsm: Add rest of checks in WaitDeals
2021-01-21 17:40:50 +01:00
Łukasz Magiera
069766ecc4
storagefsm: Don't persist piece assignment queue
2021-01-21 17:40:50 +01:00
Łukasz Magiera
270f2935a9
storagefsm: Check per-sector deal limits
2021-01-21 17:40:50 +01:00
Łukasz Magiera
fd67a41c75
storagefsm: Change sector CreationTime to unix ts
2021-01-21 17:40:50 +01:00
Łukasz Magiera
9857ad8378
storagefsm: Fix some deadlock cases
2021-01-21 17:40:50 +01:00
Łukasz Magiera
e5814dac4f
cbor-gen
2021-01-21 17:40:50 +01:00
Łukasz Magiera
542357a1df
storagefsm: Start packing correctly
2021-01-21 17:40:50 +01:00
Łukasz Magiera
239d6f8f4d
storagefsm: Rewrite input handling
2021-01-21 17:40:50 +01:00
Łukasz Magiera
0eaa6d9983
storagefsm: Send correct event on ErrExpiredTicket in CommitFailed
2021-01-17 22:29:07 +01:00
Łukasz Magiera
80b8d4b9d7
Address review
2021-01-14 20:27:15 +01:00
Łukasz Magiera
32885e1129
termination batcher: Notify based on what was sent
2021-01-14 17:14:26 +01:00
Łukasz Magiera
4015ddbb4f
fsm: Fix panic in precommit check in handleTerminating
2021-01-14 16:13:32 +01:00
Łukasz Magiera
49abdd7d7d
Sector termination support - address review
2021-01-14 15:46:57 +01:00
Łukasz Magiera
7ddf1d1feb
Add terminating states to state lists
2021-01-14 12:44:27 +01:00
Łukasz Magiera
144b5a1350
perning termination API
2021-01-14 12:37:23 +01:00
Łukasz Magiera
1564db1fce
Sector termination test
2021-01-14 00:11:41 +01:00
Łukasz Magiera
3522c8d45a
SectorTerminateFlush API
2021-01-13 23:32:04 +01:00
Łukasz Magiera
174c595acc
Don't declare in proving window
2021-01-13 22:19:10 +01:00
Łukasz Magiera
52cc2cd3eb
Initial sector termination support
2021-01-13 00:42:01 +01:00
Łukasz Magiera
299e72458e
storagefsm: Fix unsealedInfoMap.lk init race
2021-01-10 14:01:29 +01:00
zzx234234
7db5e8061d
fix a coding error in truncating sector log
2020-12-11 11:31:36 +08:00
stirlingx
3c50baaf5b
fix curSealing out of MaxSealingSectors limit
2020-12-10 11:04:42 +08:00
Łukasz Magiera
1502ca6460
storagefsm: Fix addr selection for commits
2020-12-03 12:30:41 +01:00
Łukasz Magiera
562f5839b7
Fix lint
2020-12-02 22:01:09 +01:00
Łukasz Magiera
2fd93a55ac
plumb AddressSelector to stoage fsm
2020-12-02 21:47:54 +01:00
Łukasz Magiera
2304ea0077
Merge pull request #5016 from filecoin-project/fix/fsm-rm-unceroverable
...
storagefsm: Add missing planners
2020-11-26 12:22:48 +01:00
Łukasz Magiera
74f040f9d5
storagefsm: Add SubmitCommit to toStatState switch
2020-11-26 12:02:35 +01:00
Łukasz Magiera
eeb38fa649
storagefsm: Test fsmPlanners map
2020-11-26 10:58:07 +01:00
Łukasz Magiera
ba87cb4fa9
storagefsm: Add missing planners
2020-11-26 10:57:47 +01:00
Łukasz Magiera
65653a1747
storagefsm: Add GetTicket to some maps
2020-11-26 10:53:31 +01:00
Łukasz Magiera
89493fe2b8
storagefsm: newDealSector review
2020-11-25 18:55:58 +01:00
Łukasz Magiera
6166204bac
storagefsm: Improve new deal sector logic
2020-11-25 16:40:04 +01:00
Łukasz Magiera
214f46bc88
Merge pull request #4876 from filecoin-project/fix/miner-ticket-loop
...
storagefsm: Fix expired ticket retry loop
2020-11-24 19:54:41 +01:00
yaohcn
7c0b6f41d8
fix log format
2020-11-24 19:09:48 +08:00
Łukasz Magiera
83b1009837
storagefsm: check getticket precommit expication correctly
2020-11-23 19:12:54 +01:00
Łukasz Magiera
85e592bbe2
storagefsm: Retry pushing messages when out-of-funds
2020-11-23 19:03:31 +01:00
Łukasz Magiera
e2a6b99144
storagefsm: Fix expired ticket retry loop
2020-11-23 19:03:31 +01:00
hannahhoward
218f7c2c5d
feat(markets): update markets, extract commp
2020-11-20 13:29:14 -08:00
Aayush Rajasekaran
68a16afeda
Merge branch 'master' into release/v1.2.0
2020-11-17 19:40:05 -05:00
Łukasz Magiera
00335878ab
Merge pull request #4712 from filecoin-project/fix/commitfailed-checks
...
Check (pre)commit receipt before other checks in failed states
2020-11-18 01:26:02 +01:00
Łukasz Magiera
6bea9dd178
Making sealing logic work with multiple seal proof types
2020-11-16 19:03:30 +01:00
Łukasz Magiera
11b041224e
fsm: Allow manually resurrecting removed sectors
2020-11-12 19:37:54 +01:00
zgfzgf
3665f90795
optimize pledge sector
2020-11-08 16:24:50 +08:00
Łukasz Magiera
cde0e58e39
Check (pre)commit receipt before other checks in failed states
2020-11-03 18:47:29 +01:00
Łukasz Magiera
33459f1b36
storagefsm: Fix GetTicket loop when the sector is already precommitted
2020-10-29 10:26:02 +01:00
Łukasz Magiera
660236b224
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-23 23:25:35 +02:00
Steven Allen
bcabe7b3b5
migrate methods to abstracted methods
...
Method numbers never change anyways. At worst, we'll deprecate old methods and
have to explicitly import them from the correct actors version to use them.
2020-10-21 12:18:37 -07:00
Steven Allen
4e730b5ec8
port to v2 imports
2020-10-21 12:16:23 -07:00
Łukasz Magiera
b74a3229f8
fsm: process expired-ticket sectors
2020-10-13 21:35:41 +02:00
Łukasz Magiera
0de3051821
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-08 13:10:41 +02:00
Łukasz Magiera
2cfe22d4e5
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-30 20:48:16 +02:00
Łukasz Magiera
e3ee4e4718
Fix lint errors
2020-09-30 20:24:03 +02:00
Łukasz Magiera
4f97d9637e
Fix storage-fsm tests
2020-09-30 19:11:58 +02:00
Steven Allen
4989b1cc90
Merge branch 'master' into asr/spec-v1
2020-09-30 09:19:12 -07:00
Łukasz Magiera
a783bf9b8b
storagefsm: Handle PC2 with missing replica
2020-09-30 17:53:15 +02:00
Łukasz Magiera
430897c9ac
Merge pull request #3445 from filecoin-project/asr/precommit-failed-fix
...
Fix some failed precommit handling
2020-09-30 09:35:41 +02:00
Łukasz Magiera
0f2dcf28b1
fsm: Reuse tickets in PC1 on retry
2020-09-29 10:07:49 +02:00
Steven Allen
1c03862854
fix policy for specs-actors update
2020-09-28 17:28:16 -07:00
Steven Allen
233d8a9b72
update even more imports
2020-09-28 14:25:58 -07:00
Steven Allen
32a699d6a3
Add some actors policy setters for testing
...
Addresses:
* a307e4593a (r491966115)a307e4593a (r491966634)
2020-09-23 15:00:52 -07:00
Aayush Rajasekaran
1dc69e397e
Resolve some unnecessary actor upgrade TODOs
2020-09-22 01:34:21 -04:00
Łukasz Magiera
f7934b083c
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
2020-09-21 19:10:18 +02:00
Steven Allen
1bf3b4989d
rename imports to match actors code
...
`sed -i 's/\bv0\(\w\)\(\w*\)/\L\1\E\20/g' **/*.go`
2020-09-18 14:59:27 -07:00
Łukasz Magiera
70faa36b7f
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
2020-09-18 19:29:06 +02:00
Aayush Rajasekaran
fce423c743
Appease the linter and get everything building
2020-09-18 02:14:18 -04:00
Łukasz Magiera
6eda53565f
Most tests passing
2020-09-17 17:30:24 +02:00
hannahhoward
691bd9f442
feat(markets): complete markets conversion
...
complete markets conversion to using chain/actors types, also replacing DealProposal/DealState
interfaces with structs
2020-09-17 00:43:14 -07:00
Aayush Rajasekaran
9e48dd211a
Fixups
2020-09-17 02:34:15 -04:00
Aayush Rajasekaran
b5ba7a0fad
Miner migration
2020-09-17 02:21:16 -04:00
Frank
d5af25b76c
update init sector state list
2020-09-17 10:38:07 +08:00
Frank
ed74091c20
add exist sector state check
2020-09-16 19:49:45 +08:00
Steven Allen
36f920bcd7
progress
2020-09-14 21:55:49 -07:00
Steven Allen
02dcb5e182
Merge branch 'master' into refactor/net-upgrade
2020-09-14 14:53:57 -07:00
Aayush Rajasekaran
0edcae9e04
Fixup new command
2020-09-14 14:40:38 -04:00
Łukasz Magiera
06ec571c20
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
2020-09-14 15:09:52 +02:00
Raúl Kripalani
6d29d75724
Merge branch 'master' into inmem-journal
2020-09-14 12:17:45 +01:00
Steven Allen
d3594835c4
[WIP] Network upgrade support
...
This patch starts adding support for network upgrades.
* It adds an actors abstraction layer for loading abstract (cross-version) actors.
* It starts switching over to a shared deadline type.
* It adds an abstraction for ADTs (hamt/amt).
* It removes the callback-based API in the StateManager (difficult to abstract
across actor versions).
* It _does not_ actually add support for actors v2. We can do that in a followup
patch but that should be relatively easy.
This patch is heavily WIP and does not compile. Feel free to push changes
directly to this branch.
Notes:
* State tree access now needs a network version, because the HAMT type will change.
* I haven't figured out a nice way to abstract over changes to the _message_
types. However, many of them will be type aliased to actors v0 in actors v2 so
we can likely continue using the v0 versions (or use the v2 versions
everywhere). I've been renaming imports to `v0*` to make it clear that we're
importing types from a _specific_ actors version.
TODO:
* Consider merging incremental improvements? We'd have to get this compiling
again first but we could merge in the new abstractions, and slowly switch over.
* Finish migrating to the new abstractions.
* Remove all actor state types from the public API. See `miner.State.Info()` for
the planned approach here.
* Fix the tests. This is likely going to be a massive pain.
2020-09-11 20:16:29 -07:00
Aarsh Shah
62b2963781
retry add piece
2020-09-11 16:58:09 +05:30
lanzafame
558b637b07
Increase the number of times precommit2 is attempted before moving back to precommit1
2020-09-10 13:52:21 +10:00
Aayush Rajasekaran
d678fe4bfa
Fix tests
2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a
Update to specs v0.9.6
2020-09-07 15:48:41 -04:00
Raúl Kripalani
d46f684bb8
fix test following merge.
2020-09-04 00:05:44 +01:00
Raúl Kripalani
3206f92063
Merge branch 'master' into inmem-journal
2020-09-02 19:50:52 +01:00
Aayush Rajasekaran
f58e8bc9a3
Fix some failed precommit handling
2020-09-01 02:18:02 -04:00
Aayush Rajasekaran
27f7d3a658
Rebuild unsealed infos on miner restart
2020-08-30 18:01:07 -04:00
Łukasz Magiera
20ffce5c0d
Merge remote-tracking branch 'origin/master' into fix/some-fsm-issues
2020-08-28 16:47:25 +02:00
刘林欣
93dc33687a
ignore resend SectorStartPacking event, and unsealedInfoMap lock
2020-08-28 19:04:23 +08:00
Łukasz Magiera
6d1682a27e
storagefsm: wire up RecoverDealIDs fully
2020-08-28 11:44:15 +02:00
Łukasz Magiera
a6492b1ed6
storagefsm: Auto-retry failed remove
2020-08-27 23:59:01 +02:00
Łukasz Magiera
f2bd680cc5
gofmt
2020-08-27 23:14:46 +02:00
Łukasz Magiera
d8e58e67c6
storagefsm: Treat PackingFailed sectors as expired
2020-08-27 22:41:35 +02:00
Łukasz Magiera
15204a9b90
storagefsm: fix tests
2020-08-27 21:34:45 +02:00
Łukasz Magiera
489d5239a5
storagefsm: Attempt to auto-recover from reorged DealIDs
2020-08-27 21:11:11 +02:00
Łukasz Magiera
7806a9885a
storagefsm: Record publish deals message CID
2020-08-27 17:50:50 +02:00
Łukasz Magiera
29078aaae9
storagefsm: Retry sending messages on out-of-gas errors
2020-08-27 14:02:00 +02:00
Łukasz Magiera
df635579c4
storagefsm: Handle sectors with expired deals better
2020-08-27 13:51:38 +02:00
Łukasz Magiera
788c7dbf48
storagefsm: Separate satte for submitting commit message
2020-08-27 12:57:28 +02:00
Łukasz Magiera
15191ff80f
storagefsm: Allow removing sectors in all states
2020-08-27 12:39:20 +02:00
Raúl Kripalani
efdfd3ee3e
Merge branch 'master' into inmem-journal
2020-08-26 16:38:23 +01:00
Łukasz Magiera
a236957538
Merge pull request #3232 from filecoin-project/asr/marked-for-upgrade
...
Indicate whether a sector has been marked for upgrade
2020-08-22 21:48:45 +02:00
Łukasz Magiera
0806fd651a
Merge pull request #3183 from filecoin-project/steb/lint-all
...
Lint everything
2020-08-22 21:47:41 +02:00
Łukasz Magiera
0df6e1e2c7
Merge pull request #3220 from shaodan/fix-import
...
Fix import storage-fsm
2020-08-22 10:38:36 +02:00
Aayush Rajasekaran
984e52acbc
Indicate whether a sector has been marked for upgrade
2020-08-21 15:53:39 -04:00
whyrusleeping
2e78f2c267
If sectorInfo doesnt have CommD or CommR set during commit, don't crash
2020-08-21 09:31:28 -07:00
Dan Shao
f21ce3e8ee
Fix import storage-fsm -> extern/storage-sealing
2020-08-21 16:52:33 +08:00
Steven Allen
5733c71c50
Lint everything
...
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.
There are three remaining XXXs marked in the code for lint.
2020-08-20 20:46:36 -07:00
Aayush Rajasekaran
e9c526b218
Do not try to replace CC sectors with a deal-less sector
2020-08-19 22:41:23 -04:00
Aayush Rajasekaran
7de7e65dbc
Calculate depositMinimum for upgrading as replaced sector's Initialpledge
2020-08-18 19:22:43 -04:00
Aayush Rajasekaran
98f00b510f
re-mark replaced sectors as for upgrade if precommit fails
2020-08-18 18:53:56 -04:00
Łukasz Magiera
a03192a39b
Separate concurrent sealing sector limit for deals
2020-08-18 21:48:18 +02:00
Łukasz Magiera
6a7d72c030
fsm: fail early on too-many-sealing-sectors in pledge
2020-08-18 19:26:17 +02:00
Łukasz Magiera
e7d65be90a
gofmt
2020-08-18 18:27:28 +02:00
Łukasz Magiera
baf91f08a1
Unbreak tests
2020-08-18 18:27:18 +02:00
Łukasz Magiera
886d9cd5eb
fsm: handle already-precommitted CommitFailed sectors correctly
2020-08-18 18:02:25 +02:00
Łukasz Magiera
6a49bd6d8e
fsm: Config for limiting max sealing sectors
2020-08-18 16:54:05 +02:00
Łukasz Magiera
65ffde9e4b
fsm: Config for max WaitDeals sectors
2020-08-18 16:20:31 +02:00
Aayush Rajasekaran
37ff876bc6
restart WaitDeals timers if miner restarts
2020-08-18 05:07:14 -04:00
Łukasz Magiera
01df17f1e7
Merge pull request #3103 from filecoin-project/fix/fsm-recover-bork-snark
...
fsm: Recover from errors from VerifySeal in checkCommit
2020-08-17 19:00:16 +02:00
Łukasz Magiera
bb79af7721
fsm: Recover from errors from VerifySeal in checkCommit
2020-08-17 18:53:09 +02:00
Łukasz Magiera
a66e202f02
fsm: Fix panic on missing precommit info
2020-08-17 18:45:38 +02:00
Raúl Kripalani
862bafc63a
fix lint errors.
2020-08-17 14:39:33 +01:00
Raúl Kripalani
efdc428d5d
keep storage-fsm (renamed to storage-sealing) and sector-storage in extern.
2020-08-17 14:26:18 +01:00
