cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
9,168 Commits 24 Branches 342 Tags 314 MiB
6d78de95a1
Commit Graph

356 Commits

Author SHA1 Message Date
Łukasz Magiera
03c3d8bdb3 workers: Return unfinished tasks on restart 2020-09-22 00:52:33 +02:00
Łukasz Magiera
aa5bd7bc17 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-21 22:36:16 +02:00
Łukasz Magiera
f7934b083c Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-21 19:10:18 +02:00
Alan Shaw
d9f2b909ac
chore: update test-vectors 2020-09-21 11:58:45 +01:00
Alan Shaw
a79291070e
chore: update test-vectors 2020-09-21 11:53:38 +01:00
Steven Allen
1bf3b4989d rename imports to match actors code 
`sed -i 's/\bv0\(\w\)\(\w*\)/\L\1\E\20/g' **/*.go`
 2020-09-18 14:59:27 -07:00
Łukasz Magiera
7109e95f5e
Merge pull request #3881 from filcloud/issue-3626-SectorState 
fix SectorState
 2020-09-18 21:30:18 +02:00
Łukasz Magiera
70faa36b7f Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-18 19:29:06 +02:00
Aayush Rajasekaran
fce423c743 Appease the linter and get everything building 2020-09-18 02:14:18 -04:00
Łukasz Magiera
6eda53565f Most tests passing 2020-09-17 17:30:24 +02:00
Łukasz Magiera
e632643801 api: Test return types 2020-09-17 12:24:50 +02:00
hannahhoward
691bd9f442 feat(markets): complete markets conversion 
complete markets conversion to using chain/actors types, also replacing DealProposal/DealState
interfaces with structs
 2020-09-17 00:43:14 -07:00
Aayush Rajasekaran
9e48dd211a Fixups 2020-09-17 02:34:15 -04:00
Aayush Rajasekaran
b5ba7a0fad Miner migration 2020-09-17 02:21:16 -04:00
Frank
d5af25b76c update init sector state list 2020-09-17 10:38:07 +08:00
Łukasz Magiera
17680fff55 gofmt 2020-09-17 00:35:57 +02:00
Łukasz Magiera
d9d644b27f sectorstorage: handle restarting manager, test that 2020-09-17 00:35:09 +02:00
Łukasz Magiera
5e09581256 sectorstorage: get new work tracker to run 2020-09-16 22:33:58 +02:00
Łukasz Magiera
b1361aaf8b sectorstorage: wip manager work tracker 2020-09-16 17:08:05 +02:00
Anton Evangelatov
0ff5a71114 update oni to master 2020-09-16 15:13:28 +02:00
Anton Evangelatov
b0677ab38e replace lotus and filecoin-ffi deps 2020-09-16 15:12:41 +02:00
Anton Evangelatov
3f67c4524e add filecoin-project oni as submodule and compile lotus-soup 2020-09-16 15:12:41 +02:00
Frank
ed74091c20 add exist sector state check 2020-09-16 19:49:45 +08:00
Łukasz Magiera
d4a7732d0a
Merge pull request #3800 from filecoin-project/asr/gascost 
Add an API to get detailed gas costs for a message
 2020-09-15 19:13:20 +02:00
Łukasz Magiera
03cf6cca40 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-15 17:47:03 +02:00
Steven Allen
36f920bcd7 progress 2020-09-14 21:55:49 -07:00
hannahhoward
7dc091052a feat(manager): less restrictive storage lock 
Use initial less restrictive storage lock when trying to read unsealed data before acquiring more
restrictive lock needed for unsealing
 2020-09-14 18:48:14 -07:00
Steven Allen
02dcb5e182 Merge branch 'master' into refactor/net-upgrade 2020-09-14 14:53:57 -07:00
Aayush Rajasekaran
0edcae9e04 Fixup new command 2020-09-14 14:40:38 -04:00
Łukasz Magiera
e9d25e5919 More fixes 2020-09-14 20:28:47 +02:00
Łukasz Magiera
381a6cdfac Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-14 19:11:50 +02:00
Łukasz Magiera
1ebca8f732 more working code 2020-09-14 19:09:01 +02:00
Łukasz Magiera
06ec571c20 Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-14 15:09:52 +02:00
Raúl Kripalani
6d29d75724 Merge branch 'master' into inmem-journal 2020-09-14 12:17:45 +01:00
Steven Allen
d3594835c4 [WIP] Network upgrade support 
This patch starts adding support for network upgrades.

* It adds an actors abstraction layer for loading abstract (cross-version) actors.
* It starts switching over to a shared deadline type.
* It adds an abstraction for ADTs (hamt/amt).
* It removes the callback-based API in the StateManager (difficult to abstract
across actor versions).
* It _does not_ actually add support for actors v2. We can do that in a followup
patch but that should be relatively easy.

This patch is heavily WIP and does not compile. Feel free to push changes
directly to this branch.

Notes:

* State tree access now needs a network version, because the HAMT type will change.
* I haven't figured out a nice way to abstract over changes to the _message_
types. However, many of them will be type aliased to actors v0 in actors v2 so
we can likely continue using the v0 versions (or use the v2 versions
everywhere). I've been renaming imports to `v0*` to make it clear that we're
importing types from a _specific_ actors version.

TODO:

* Consider merging incremental improvements? We'd have to get this compiling
again first but we could merge in the new abstractions, and slowly switch over.
* Finish migrating to the new abstractions.
* Remove all actor state types from the public API. See `miner.State.Info()` for
the planned approach here.
* Fix the tests. This is likely going to be a massive pain.
 2020-09-11 20:16:29 -07:00
Aarsh Shah
62b2963781 retry add piece 2020-09-11 16:58:09 +05:30
Łukasz Magiera
e996a5246b Merge remote-tracking branch 'origin/master' into feat/window-post-faulty-sectors 2020-09-10 22:18:17 +02:00
Łukasz Magiera
bbac86f745 gofmt, mod tidy 2020-09-10 22:07:20 +02:00
Łukasz Magiera
22556fb24c Add fil-blst submodule 2020-09-10 22:04:08 +02:00
Łukasz Magiera
2374cb1bc5 Update ffi 2020-09-10 21:44:01 +02:00
Łukasz Magiera
c7b0241a48 ffiwrapper: Test skipping corrupted sectors in PoSt 2020-09-10 21:19:26 +02:00
Łukasz Magiera
5f08fe7ead Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-10 17:30:54 +02:00
lanzafame
558b637b07 Increase the number of times precommit2 is attempted before moving back to precommit1 2020-09-10 13:52:21 +10:00
Łukasz Magiera
5e7737f55d wdpost: Handle skipped sectors correctly 2020-09-10 02:59:37 +02:00
whyrusleeping
7a6ceebb34 windowed post generation now returns faulty sectors 2020-09-09 14:00:15 -07:00
Raúl Kripalani
a5afd83c7b Merge branch 'master' into fix/test-vectors-import-cycle 2020-09-09 13:04:41 +01:00
Raúl Kripalani
e8d1bab914 make the runner call statediff as a binary. 2020-09-09 13:03:43 +01:00
Dirk McCormick
17c15a74a2 fix: return true from Sealer.ReadPiece() on success 2020-09-08 13:50:56 +02:00
Dirk McCormick
8bbdf2e7cb fix: storage manager - bail out on undefined unsealed cid 2020-09-08 12:54:01 +02:00
Aayush Rajasekaran
f91f665f3f Update test-vectors 2020-09-08 01:24:30 -04:00
Aayush Rajasekaran
d678fe4bfa Fix tests 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a Update to specs v0.9.6 2020-09-07 15:48:41 -04:00
Łukasz Magiera
47c59afea0
Revert "storage manager: bail out with an error if unsealed cid is undefined" 2020-09-07 20:12:29 +02:00
Łukasz Magiera
231a9e4051 Fix sealing sched tests 2020-09-07 17:55:31 +02:00
Dirk McCormick
a97f978cad fix: storage manager - dont fail on successful read piece 2020-09-07 16:14:19 +02:00
Łukasz Magiera
9e6f974f3c storage: Fix build 2020-09-07 16:12:55 +02:00
Dirk McCormick
07a4553e6e fix: storage manager - bail out with an error if unsealed cid is undefined 2020-09-07 16:04:12 +02:00
Łukasz Magiera
06e3852cef storage: Integrate async workers in sealing manager 2020-09-07 12:20:50 +02:00
Łukasz Magiera
5d73943929 storage: Fix import cycle 2020-09-06 18:54:00 +02:00
Łukasz Magiera
159ce13f5e Async worker API 2020-09-06 18:47:16 +02:00
Raúl Kripalani
1ec534d607 Merge branch 'master' into inmem-journal 2020-09-04 10:55:40 +01:00
Raúl Kripalani
d46f684bb8 fix test following merge. 2020-09-04 00:05:44 +01:00
Raúl Kripalani
c723554d2e upgrade test-vectors deps. 2020-09-03 23:44:21 +01:00
Raúl Kripalani
3206f92063 Merge branch 'master' into inmem-journal 2020-09-02 19:50:52 +01:00
Łukasz Magiera
b774563ec3
Merge pull request #3492 from filecoin-project/fix/readpiece-panic 
ffiwrapper: Fix ReadPiece panic
 2020-09-02 18:59:22 +02:00
Łukasz Magiera
ca7aa69597 ffiwrapper: More correct error check on openPartialFile 2020-09-02 18:45:07 +02:00
Łukasz Magiera
5f79ff340d
Merge pull request #3489 from filecoin-project/fix/sched-deadlocks 
sealing sched: Fix deadlock between worker.wndLk / workersLk
 2020-09-02 18:03:18 +02:00
Łukasz Magiera
5a2b439773 sched: Fix tests 2020-09-02 17:37:19 +02:00
Łukasz Magiera
7fe8580da5 sealing sched: Fix deadlock between worker.wndLk / workersLk 2020-09-02 17:06:48 +02:00
Raúl Kripalani
7cc9c83ab8 upgrade test-vectors. 2020-09-02 14:12:27 +01:00
Raúl Kripalani
0962292f92 conformance: add driver for tipset-class vectors. 2020-09-02 14:09:12 +01:00
Łukasz Magiera
fa6bae988e
Revert "conformance: add driver for tipset-class vectors." 2020-09-02 14:49:55 +02:00
Raúl Kripalani
ad5793e446 update test-vectors commit. 2020-09-01 20:01:41 +01:00
Raúl Kripalani
01737ef1cc upgrade test-vectors deps. 2020-09-01 17:00:18 +01:00
Star.LI
82f5984de1 fix crash - segment fault when partialFile.Allocated() is invoked. 
When openPartialFile is invoked, more errors than "existed error" are
returned. If only existing error is checked, the allocated field of
partialFile may be nil.

Signed-off-by: Star.LI <star@trapdoortech.com>
 2020-09-01 11:41:26 +02:00
Aayush Rajasekaran
f58e8bc9a3 Fix some failed precommit handling 2020-09-01 02:18:02 -04:00
Łukasz Magiera
e14c80360d sealing sched: Factor worker queues into utilization calc 2020-08-31 13:41:34 +02:00
Łukasz Magiera
98d51d3d80 storage: Correcty move unsealed sectors in FinalizeSector 2020-08-31 12:45:57 +02:00
Aayush Rajasekaran
27f7d3a658 Rebuild unsealed infos on miner restart 2020-08-30 18:01:07 -04:00
Łukasz Magiera
28ac2fce61 sched: Fix panic in workerCompactWindows 2020-08-29 06:41:19 +02:00
Łukasz Magiera
9d0c8ae3dd sectorstorage: update sched tests for new logic 2020-08-28 21:38:21 +02:00
Łukasz Magiera
4a75e1e4b4 sectorstorage: Don't require tasks within a window to run in order 2020-08-28 19:38:55 +02:00
Łukasz Magiera
11b11e416b sectorstorage: Compact assigned windows 2020-08-28 18:26:38 +02:00
Łukasz Magiera
20ffce5c0d Merge remote-tracking branch 'origin/master' into fix/some-fsm-issues 2020-08-28 16:47:25 +02:00
Łukasz Magiera
5ee85dc263 sectorstorage: Fix tests 2020-08-28 16:33:41 +02:00
刘林欣
93dc33687a ignore resend SectorStartPacking event, and unsealedInfoMap lock 2020-08-28 19:04:23 +08:00
Łukasz Magiera
6d1682a27e storagefsm: wire up RecoverDealIDs fully 2020-08-28 11:44:15 +02:00
Łukasz Magiera
1097d29213 sealing sched: Call trySched less when there are many tasks 2020-08-28 00:03:42 +02:00
Łukasz Magiera
a6492b1ed6 storagefsm: Auto-retry failed remove 2020-08-27 23:59:01 +02:00
Łukasz Magiera
59d2034cbb sealing sched: Wait a bit for tasks to come in on restart 2020-08-27 23:58:37 +02:00
Łukasz Magiera
7fdffc0340 sealing sched: Give more priority to tasks (re)moving data 2020-08-27 23:29:39 +02:00
Łukasz Magiera
f2bd680cc5 gofmt 2020-08-27 23:14:46 +02:00
Łukasz Magiera
59f554b658 sealing sched: Show waiting tasks assigned to workers in sealing jobs cli 2020-08-27 23:14:33 +02:00
Łukasz Magiera
d8e58e67c6 storagefsm: Treat PackingFailed sectors as expired 2020-08-27 22:41:35 +02:00
Łukasz Magiera
15204a9b90 storagefsm: fix tests 2020-08-27 21:34:45 +02:00
Łukasz Magiera
489d5239a5 storagefsm: Attempt to auto-recover from reorged DealIDs 2020-08-27 21:11:11 +02:00
Łukasz Magiera
7806a9885a storagefsm: Record publish deals message CID 2020-08-27 17:50:50 +02:00
Łukasz Magiera
29078aaae9 storagefsm: Retry sending messages on out-of-gas errors 2020-08-27 14:02:00 +02:00
Łukasz Magiera
df635579c4 storagefsm: Handle sectors with expired deals better 2020-08-27 13:51:38 +02:00
Łukasz Magiera
788c7dbf48 storagefsm: Separate satte for submitting commit message 2020-08-27 12:57:28 +02:00
Łukasz Magiera
15191ff80f storagefsm: Allow removing sectors in all states 2020-08-27 12:39:20 +02:00
Whyrusleeping
deb013cecb
Merge pull request #3081 from filecoin-project/conformance-tests 
driver, runner and CI for interoperable test-vector based testing.
 2020-08-26 14:06:39 -07:00
Raúl Kripalani
4709e231b8 drop aurora as dep; upgrade test-vectors. 2020-08-26 18:35:41 +01:00
Steven Allen
0155a31d1f Fix PoSt with bad sectors 
"skipped" sectors must be replaced with a substitute "good" sector, or the
entire partition must be skipped. They should not just be omitted.

This patch also fixes the test to verify the _entire_ proof instead of just
verifying that the proof includes the correct sectors.
 2020-08-26 09:56:51 -07:00
Raúl Kripalani
efdfd3ee3e Merge branch 'master' into inmem-journal 2020-08-26 16:38:23 +01:00
Raúl Kripalani
d3ac243cea Merge remote-tracking branch 'origin/master' into conformance-tests 2020-08-26 15:12:02 +01:00
Alan Shaw
a89faa72ea
feat(conformance): skip incorrect tests (#3310) 
Fixes https://github.com/filecoin-project/test-vectors/issues/72
 2020-08-26 11:45:34 +01:00
Łukasz Magiera
d9796cd25c sectorstorage: Make trySched less very slow 2020-08-24 19:16:16 +02:00
Łukasz Magiera
4311c96a44
Merge pull request #3225 from filecoin-project/fix/sched-missing-worker 
check that worker referenced by task is actually still there.
 2020-08-22 22:16:32 +02:00
Łukasz Magiera
a236957538
Merge pull request #3232 from filecoin-project/asr/marked-for-upgrade 
Indicate whether a sector has been marked for upgrade
 2020-08-22 21:48:45 +02:00
Łukasz Magiera
0806fd651a
Merge pull request #3183 from filecoin-project/steb/lint-all 
Lint everything
 2020-08-22 21:47:41 +02:00
Łukasz Magiera
0df6e1e2c7
Merge pull request #3220 from shaodan/fix-import 
Fix import storage-fsm
 2020-08-22 10:38:36 +02:00
Aayush Rajasekaran
984e52acbc Indicate whether a sector has been marked for upgrade 2020-08-21 15:53:39 -04:00
whyrusleeping
54862be3ff check that worker referenced by task is actually still there. 2020-08-21 10:33:36 -07:00
whyrusleeping
2e78f2c267 If sectorInfo doesnt have CommD or CommR set during commit, don't crash 2020-08-21 09:31:28 -07:00
Dan Shao
f21ce3e8ee Fix import storage-fsm -> extern/storage-sealing 2020-08-21 16:52:33 +08:00
Steven Allen
5733c71c50 Lint everything 
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.

There are three remaining XXXs marked in the code for lint.
 2020-08-20 20:46:36 -07:00
Aayush Rajasekaran
e9c526b218 Do not try to replace CC sectors with a deal-less sector 2020-08-19 22:41:23 -04:00
Raúl Kripalani
e38f4219c5 update test-vectors commit. 2020-08-19 18:19:03 +01:00
Raúl Kripalani
30b6f38704 update test-vectors commit. 2020-08-19 18:14:19 +01:00
Raúl Kripalani
04e4bab313 register chaos actor only if selector says to do so. 2020-08-19 18:10:56 +01:00
Raúl Kripalani
dddf7951ef update test-vectors commit. 2020-08-19 17:33:30 +01:00
Raúl Kripalani
a21234cd54 deploy the chaos actor. 2020-08-19 14:31:34 +01:00
Raúl Kripalani
4f4cebadc7 Merge branch 'master' into conformance-tests 2020-08-19 13:33:15 +01:00
Aayush Rajasekaran
7de7e65dbc Calculate depositMinimum for upgrading as replaced sector's Initialpledge 2020-08-18 19:22:43 -04:00
Aayush Rajasekaran
98f00b510f re-mark replaced sectors as for upgrade if precommit fails 2020-08-18 18:53:56 -04:00
Łukasz Magiera
a03192a39b Separate concurrent sealing sector limit for deals 2020-08-18 21:48:18 +02:00
Łukasz Magiera
6a7d72c030 fsm: fail early on too-many-sealing-sectors in pledge 2020-08-18 19:26:17 +02:00
Łukasz Magiera
e7d65be90a gofmt 2020-08-18 18:27:28 +02:00
Łukasz Magiera
baf91f08a1 Unbreak tests 2020-08-18 18:27:18 +02:00
Łukasz Magiera
886d9cd5eb fsm: handle already-precommitted CommitFailed sectors correctly 2020-08-18 18:02:25 +02:00
Łukasz Magiera
6a49bd6d8e fsm: Config for limiting max sealing sectors 2020-08-18 16:54:05 +02:00
Łukasz Magiera
65ffde9e4b fsm: Config for max WaitDeals sectors 2020-08-18 16:20:31 +02:00
Aayush Rajasekaran
37ff876bc6 restart WaitDeals timers if miner restarts 2020-08-18 05:07:14 -04:00
Łukasz Magiera
01df17f1e7
Merge pull request #3103 from filecoin-project/fix/fsm-recover-bork-snark 
fsm: Recover from errors from VerifySeal in checkCommit
 2020-08-17 19:00:16 +02:00
Łukasz Magiera
bb79af7721 fsm: Recover from errors from VerifySeal in checkCommit 2020-08-17 18:53:09 +02:00
Łukasz Magiera
a66e202f02 fsm: Fix panic on missing precommit info 2020-08-17 18:45:38 +02:00
Łukasz Magiera
6ef7a30b19
Merge pull request #3089 from filecoin-project/integrate/storage-fsm 
integrate extern/{storage-fsm,sector-storage} into lotus source tree
 2020-08-17 18:37:54 +02:00
Raúl Kripalani
1f016262cd readd int64 cast + nolint directive. 2020-08-17 17:26:07 +01:00
Raúl Kripalani
862bafc63a fix lint errors. 2020-08-17 14:39:33 +01:00
Raúl Kripalani
efdc428d5d keep storage-fsm (renamed to storage-sealing) and sector-storage in extern. 2020-08-17 14:26:18 +01:00
yaohcn
39bcfab37c make addpiece configurable 2020-08-17 17:39:50 +08:00
Raúl Kripalani
483e82e14b point test-vectors to master. 2020-08-16 22:05:43 +01:00
Raúl Kripalani
3c17cd655e integrate extern/sector-storage into lotus proper. 2020-08-16 11:09:58 +01:00
Raúl Kripalani
fb0ccc9260 integrate extern/storage-fsm into lotus proper. 2020-08-16 10:42:13 +01:00
Raúl Kripalani
322b33197c introduce interoperable vector-based conformance testing. 
This commit introduces a new package `conformance` containing:

 1. the test driver to exercise Lotus against interoperable
    test vectors, and
 2. the test runner, which integrates go test with the test vector
    corpus hosted at https://github.com/filecoin-project/conformance-vectors.

The corpus is mounted via a git submodule.

Right now, only message-class test vectors are supported. In the
next week, this support will be extended to tipset-class, chain-class,
and block sequence-class vectors.
 2020-08-16 00:15:52 +01:00
Łukasz Magiera
9328dc27f3 Update filecoin-ffi to fix Intel build 2020-08-15 13:19:10 +02:00
Łukasz Magiera
0c75dd3865 Support AddPiece on workers 2020-08-14 16:06:53 +02:00
Łukasz Magiera
632fd36205 sealing sched: Fix deadlock in worker watcher 2020-08-13 12:17:24 +02:00
yaohcn
1555984785 change to RLock 2020-08-13 17:31:18 +08:00