Steven Allen
767c346cf8
update test-vectors
2020-10-07 12:17:32 -07:00
Steven Allen
c17fa4bc35
update test-vectors for StateManager constructor change
2020-10-07 12:04:32 -07:00
Steven Allen
df4b068d0e
Merge branch 'master' into next
2020-10-07 11:55:00 -07:00
Łukasz Magiera
cfd126ee9f
Merge pull request #3961 from filecoin-project/docs/miner-storage
...
lotus-miner: add more help text to storage / attach
2020-10-07 14:55:42 +02:00
Steven Allen
1e78cd21ec
fix lotus soup build
2020-10-05 14:37:08 -07:00
Steven Allen
17846aad2f
add some more policy toggles for testground
2020-10-05 14:28:47 -07:00
Łukasz Magiera
1fc23fb466
lotus-miner: Cleanup storage attach helptext a bit
2020-10-03 11:30:22 +02:00
Steven Allen
8292d60196
Merge branch 'master' into asr/spec-v1
2020-10-02 18:03:03 -07:00
Łukasz Magiera
5932f28519
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-01 02:39:48 +02:00
Łukasz Magiera
6981f776f4
Lower PC2 memory requirements
2020-10-01 00:54:53 +02:00
Łukasz Magiera
1b7cdb9341
Fix storage manager tests
2020-10-01 00:54:34 +02:00
Łukasz Magiera
5e08d56630
sched: Allow some single-thread tasks to run in parallel with PC2/C2
2020-10-01 00:28:44 +02:00
Łukasz Magiera
79d2ddf24f
Review
2020-09-30 21:18:12 +02:00
Łukasz Magiera
2cfe22d4e5
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-30 20:48:16 +02:00
Łukasz Magiera
e3ee4e4718
Fix lint errors
2020-09-30 20:24:03 +02:00
Łukasz Magiera
2d16af6ee6
sectorstorage: Fix TestRedoPC1
2020-09-30 19:18:38 +02:00
Łukasz Magiera
4f97d9637e
Fix storage-fsm tests
2020-09-30 19:11:58 +02:00
Steven Allen
4989b1cc90
Merge branch 'master' into asr/spec-v1
2020-09-30 09:19:12 -07:00
Łukasz Magiera
c228598098
sectorstorage: Variable scopes are really hard
2020-09-30 18:16:07 +02:00
Łukasz Magiera
a783bf9b8b
storagefsm: Handle PC2 with missing replica
2020-09-30 17:53:15 +02:00
Łukasz Magiera
54fdd6ba5a
sectorstorage: Variable scopes are hard
2020-09-30 17:48:48 +02:00
Łukasz Magiera
6855284d88
sectorstorage: Cancel non-running work in case of abort in sched
2020-09-30 17:26:09 +02:00
Łukasz Magiera
5bffea6f54
Merge pull request #4064 from filecoin-project/tvx
...
tvx: a test vector extraction and execution tool
2020-09-30 17:22:42 +02:00
Raúl Kripalani
ff8663faa0
update test-vectors submodule.
2020-09-30 14:58:22 +01:00
Łukasz Magiera
430897c9ac
Merge pull request #3445 from filecoin-project/asr/precommit-failed-fix
...
Fix some failed precommit handling
2020-09-30 09:35:41 +02:00
Łukasz Magiera
bc85e3ce50
Merge pull request #4107 from shaodan/worker-no-swap
...
Add --no-swap flag for worker
2020-09-30 09:25:57 +02:00
Dan Shao
1affd498c1
Add --no-swap flag for worker
2020-09-30 14:23:35 +08:00
Łukasz Magiera
baef3c8dd2
sectorstorage: Fix potential panic in FinalizeSector
2020-09-29 15:22:46 +02:00
Łukasz Magiera
0f2dcf28b1
fsm: Reuse tickets in PC1 on retry
2020-09-29 10:07:49 +02:00
Steven Allen
1c03862854
fix policy for specs-actors update
2020-09-28 17:28:16 -07:00
Steven Allen
233d8a9b72
update even more imports
2020-09-28 14:25:58 -07:00
Łukasz Magiera
1e6a69f8aa
localworker: Don't mark calls as returned when returning fails
2020-09-28 22:10:02 +02:00
Łukasz Magiera
9bd2537971
stores: Fix error printing in http handler
2020-09-28 22:06:03 +02:00
Łukasz Magiera
810c767200
worker: Redeclare storage on reconnect
2020-09-28 21:06:49 +02:00
Łukasz Magiera
4ba7af6061
worker: Mark return methods as retry-safe
2020-09-28 20:46:44 +02:00
Łukasz Magiera
9e7d6823b1
sectorstorage: Cleanup callToWork mapping after work is done
2020-09-28 13:34:45 +02:00
Łukasz Magiera
cf71f034dc
Merge remote-tracking branch 'origin/dev' into feat/async-restartable-workers
2020-09-26 12:35:32 +02:00
zgfzgf
1a7aea1906
modify error
2020-09-25 22:59:21 +08:00
zgfzgf
3207bc4704
optimize trySched
2020-09-25 22:41:29 +08:00
zgfzgf
60e950015c
modify for unsafe
2020-09-25 22:13:27 +08:00
whyrusleeping
306c098d30
also update our vendored blst repo
2020-09-24 11:32:38 -07:00
whyrusleeping
6db37b72a8
update ffi to code with blst fixes
2020-09-24 11:05:21 -07:00
Łukasz Magiera
04ee53e061
sectorstorage: Show task type of ret-wait jobs
2020-09-24 11:55:11 +02:00
Steven Allen
32a699d6a3
Add some actors policy setters for testing
...
Addresses:
* a307e4593a (r491966115)a307e4593a (r491966634)
2020-09-23 15:00:52 -07:00
Łukasz Magiera
d817dceb05
Show lost calls in sealing jobs cli
2020-09-23 19:26:35 +02:00
Łukasz Magiera
c17f0d7e61
sectorstorage: Fix panic in returnResult
2020-09-23 17:37:05 +02:00
Łukasz Magiera
86c222ab58
sectorstorage: fix work tracking
2020-09-23 14:56:50 +02:00
Łukasz Magiera
ce6b92484f
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-23 11:31:21 +02:00
Steven Allen
773714792f
update oni
2020-09-22 18:40:03 -07:00
Łukasz Magiera
6185e157e9
sectorstorage: calltracker: work around cbor-gen bytearray len limit
2020-09-23 00:29:10 +02:00
Łukasz Magiera
04ad1791b0
localworker: Fix contexts
2020-09-23 00:10:36 +02:00
Łukasz Magiera
bb5cc06677
Fix workid param hash
2020-09-22 23:33:13 +02:00
Travis Person
c66f087f4c
lotus-miner: add more help text to storage / attach
2020-09-22 18:15:42 +00:00
Łukasz Magiera
706f4f2ef5
worker: Don't die with the connection
2020-09-22 18:36:44 +02:00
Aayush Rajasekaran
1dc69e397e
Resolve some unnecessary actor upgrade TODOs
2020-09-22 01:34:21 -04:00
Łukasz Magiera
b8865fb182
workers: Mark on-restart-failed returned tasks as returned
2020-09-22 01:00:28 +02:00
Łukasz Magiera
03c3d8bdb3
workers: Return unfinished tasks on restart
2020-09-22 00:52:33 +02:00
Łukasz Magiera
aa5bd7bc17
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-21 22:36:16 +02:00
Łukasz Magiera
f7934b083c
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
2020-09-21 19:10:18 +02:00
Alan Shaw
d9f2b909ac
chore: update test-vectors
2020-09-21 11:58:45 +01:00
Alan Shaw
a79291070e
chore: update test-vectors
2020-09-21 11:53:38 +01:00
Steven Allen
1bf3b4989d
rename imports to match actors code
...
`sed -i 's/\bv0\(\w\)\(\w*\)/\L\1\E\20/g' **/*.go`
2020-09-18 14:59:27 -07:00
Łukasz Magiera
7109e95f5e
Merge pull request #3881 from filcloud/issue-3626-SectorState
...
fix SectorState
2020-09-18 21:30:18 +02:00
Łukasz Magiera
70faa36b7f
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
2020-09-18 19:29:06 +02:00
Aayush Rajasekaran
fce423c743
Appease the linter and get everything building
2020-09-18 02:14:18 -04:00
Łukasz Magiera
6eda53565f
Most tests passing
2020-09-17 17:30:24 +02:00
Łukasz Magiera
e632643801
api: Test return types
2020-09-17 12:24:50 +02:00
hannahhoward
691bd9f442
feat(markets): complete markets conversion
...
complete markets conversion to using chain/actors types, also replacing DealProposal/DealState
interfaces with structs
2020-09-17 00:43:14 -07:00
Aayush Rajasekaran
9e48dd211a
Fixups
2020-09-17 02:34:15 -04:00
Aayush Rajasekaran
b5ba7a0fad
Miner migration
2020-09-17 02:21:16 -04:00
Frank
d5af25b76c
update init sector state list
2020-09-17 10:38:07 +08:00
Łukasz Magiera
17680fff55
gofmt
2020-09-17 00:35:57 +02:00
Łukasz Magiera
d9d644b27f
sectorstorage: handle restarting manager, test that
2020-09-17 00:35:09 +02:00
Łukasz Magiera
5e09581256
sectorstorage: get new work tracker to run
2020-09-16 22:33:58 +02:00
Łukasz Magiera
b1361aaf8b
sectorstorage: wip manager work tracker
2020-09-16 17:08:05 +02:00
Anton Evangelatov
0ff5a71114
update oni to master
2020-09-16 15:13:28 +02:00
Anton Evangelatov
b0677ab38e
replace lotus and filecoin-ffi deps
2020-09-16 15:12:41 +02:00
Anton Evangelatov
3f67c4524e
add filecoin-project oni as submodule and compile lotus-soup
2020-09-16 15:12:41 +02:00
Frank
ed74091c20
add exist sector state check
2020-09-16 19:49:45 +08:00
Łukasz Magiera
d4a7732d0a
Merge pull request #3800 from filecoin-project/asr/gascost
...
Add an API to get detailed gas costs for a message
2020-09-15 19:13:20 +02:00
Łukasz Magiera
03cf6cca40
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-15 17:47:03 +02:00
Steven Allen
36f920bcd7
progress
2020-09-14 21:55:49 -07:00
hannahhoward
7dc091052a
feat(manager): less restrictive storage lock
...
Use initial less restrictive storage lock when trying to read unsealed data before acquiring more
restrictive lock needed for unsealing
2020-09-14 18:48:14 -07:00
Steven Allen
02dcb5e182
Merge branch 'master' into refactor/net-upgrade
2020-09-14 14:53:57 -07:00
Aayush Rajasekaran
0edcae9e04
Fixup new command
2020-09-14 14:40:38 -04:00
Łukasz Magiera
e9d25e5919
More fixes
2020-09-14 20:28:47 +02:00
Łukasz Magiera
381a6cdfac
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-14 19:11:50 +02:00
Łukasz Magiera
1ebca8f732
more working code
2020-09-14 19:09:01 +02:00
Łukasz Magiera
06ec571c20
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
2020-09-14 15:09:52 +02:00
Raúl Kripalani
6d29d75724
Merge branch 'master' into inmem-journal
2020-09-14 12:17:45 +01:00
Steven Allen
d3594835c4
[WIP] Network upgrade support
...
This patch starts adding support for network upgrades.
* It adds an actors abstraction layer for loading abstract (cross-version) actors.
* It starts switching over to a shared deadline type.
* It adds an abstraction for ADTs (hamt/amt).
* It removes the callback-based API in the StateManager (difficult to abstract
across actor versions).
* It _does not_ actually add support for actors v2. We can do that in a followup
patch but that should be relatively easy.
This patch is heavily WIP and does not compile. Feel free to push changes
directly to this branch.
Notes:
* State tree access now needs a network version, because the HAMT type will change.
* I haven't figured out a nice way to abstract over changes to the _message_
types. However, many of them will be type aliased to actors v0 in actors v2 so
we can likely continue using the v0 versions (or use the v2 versions
everywhere). I've been renaming imports to `v0*` to make it clear that we're
importing types from a _specific_ actors version.
TODO:
* Consider merging incremental improvements? We'd have to get this compiling
again first but we could merge in the new abstractions, and slowly switch over.
* Finish migrating to the new abstractions.
* Remove all actor state types from the public API. See `miner.State.Info()` for
the planned approach here.
* Fix the tests. This is likely going to be a massive pain.
2020-09-11 20:16:29 -07:00
Aarsh Shah
62b2963781
retry add piece
2020-09-11 16:58:09 +05:30
Łukasz Magiera
e996a5246b
Merge remote-tracking branch 'origin/master' into feat/window-post-faulty-sectors
2020-09-10 22:18:17 +02:00
Łukasz Magiera
bbac86f745
gofmt, mod tidy
2020-09-10 22:07:20 +02:00
Łukasz Magiera
22556fb24c
Add fil-blst submodule
2020-09-10 22:04:08 +02:00
Łukasz Magiera
2374cb1bc5
Update ffi
2020-09-10 21:44:01 +02:00
Łukasz Magiera
c7b0241a48
ffiwrapper: Test skipping corrupted sectors in PoSt
2020-09-10 21:19:26 +02:00
Łukasz Magiera
5f08fe7ead
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-10 17:30:54 +02:00
lanzafame
558b637b07
Increase the number of times precommit2 is attempted before moving back to precommit1
2020-09-10 13:52:21 +10:00
Łukasz Magiera
5e7737f55d
wdpost: Handle skipped sectors correctly
2020-09-10 02:59:37 +02:00
whyrusleeping
7a6ceebb34
windowed post generation now returns faulty sectors
2020-09-09 14:00:15 -07:00
Raúl Kripalani
a5afd83c7b
Merge branch 'master' into fix/test-vectors-import-cycle
2020-09-09 13:04:41 +01:00
Raúl Kripalani
e8d1bab914
make the runner call statediff as a binary.
2020-09-09 13:03:43 +01:00
Dirk McCormick
17c15a74a2
fix: return true from Sealer.ReadPiece() on success
2020-09-08 13:50:56 +02:00
Dirk McCormick
8bbdf2e7cb
fix: storage manager - bail out on undefined unsealed cid
2020-09-08 12:54:01 +02:00
Aayush Rajasekaran
f91f665f3f
Update test-vectors
2020-09-08 01:24:30 -04:00
Aayush Rajasekaran
d678fe4bfa
Fix tests
2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a
Update to specs v0.9.6
2020-09-07 15:48:41 -04:00
Łukasz Magiera
47c59afea0
Revert "storage manager: bail out with an error if unsealed cid is undefined"
2020-09-07 20:12:29 +02:00
Łukasz Magiera
231a9e4051
Fix sealing sched tests
2020-09-07 17:55:31 +02:00
Dirk McCormick
a97f978cad
fix: storage manager - dont fail on successful read piece
2020-09-07 16:14:19 +02:00
Łukasz Magiera
9e6f974f3c
storage: Fix build
2020-09-07 16:12:55 +02:00
Dirk McCormick
07a4553e6e
fix: storage manager - bail out with an error if unsealed cid is undefined
2020-09-07 16:04:12 +02:00
Łukasz Magiera
06e3852cef
storage: Integrate async workers in sealing manager
2020-09-07 12:20:50 +02:00
Łukasz Magiera
5d73943929
storage: Fix import cycle
2020-09-06 18:54:00 +02:00
Łukasz Magiera
159ce13f5e
Async worker API
2020-09-06 18:47:16 +02:00
Raúl Kripalani
1ec534d607
Merge branch 'master' into inmem-journal
2020-09-04 10:55:40 +01:00
Raúl Kripalani
d46f684bb8
fix test following merge.
2020-09-04 00:05:44 +01:00
Raúl Kripalani
c723554d2e
upgrade test-vectors deps.
2020-09-03 23:44:21 +01:00
Raúl Kripalani
3206f92063
Merge branch 'master' into inmem-journal
2020-09-02 19:50:52 +01:00
Łukasz Magiera
b774563ec3
Merge pull request #3492 from filecoin-project/fix/readpiece-panic
...
ffiwrapper: Fix ReadPiece panic
2020-09-02 18:59:22 +02:00
Łukasz Magiera
ca7aa69597
ffiwrapper: More correct error check on openPartialFile
2020-09-02 18:45:07 +02:00
Łukasz Magiera
5f79ff340d
Merge pull request #3489 from filecoin-project/fix/sched-deadlocks
...
sealing sched: Fix deadlock between worker.wndLk / workersLk
2020-09-02 18:03:18 +02:00
Łukasz Magiera
5a2b439773
sched: Fix tests
2020-09-02 17:37:19 +02:00
Łukasz Magiera
7fe8580da5
sealing sched: Fix deadlock between worker.wndLk / workersLk
2020-09-02 17:06:48 +02:00
Raúl Kripalani
7cc9c83ab8
upgrade test-vectors.
2020-09-02 14:12:27 +01:00
Raúl Kripalani
0962292f92
conformance: add driver for tipset-class vectors.
2020-09-02 14:09:12 +01:00
Łukasz Magiera
fa6bae988e
Revert "conformance: add driver for tipset-class vectors."
2020-09-02 14:49:55 +02:00
Raúl Kripalani
ad5793e446
update test-vectors commit.
2020-09-01 20:01:41 +01:00
Raúl Kripalani
01737ef1cc
upgrade test-vectors deps.
2020-09-01 17:00:18 +01:00
Star.LI
82f5984de1
fix crash - segment fault when partialFile.Allocated() is invoked.
...
When openPartialFile is invoked, more errors than "existed error" are
returned. If only existing error is checked, the allocated field of
partialFile may be nil.
Signed-off-by: Star.LI <star@trapdoortech.com>
2020-09-01 11:41:26 +02:00
Aayush Rajasekaran
f58e8bc9a3
Fix some failed precommit handling
2020-09-01 02:18:02 -04:00
Łukasz Magiera
e14c80360d
sealing sched: Factor worker queues into utilization calc
2020-08-31 13:41:34 +02:00
Łukasz Magiera
98d51d3d80
storage: Correcty move unsealed sectors in FinalizeSector
2020-08-31 12:45:57 +02:00
Aayush Rajasekaran
27f7d3a658
Rebuild unsealed infos on miner restart
2020-08-30 18:01:07 -04:00
Łukasz Magiera
28ac2fce61
sched: Fix panic in workerCompactWindows
2020-08-29 06:41:19 +02:00
Łukasz Magiera
9d0c8ae3dd
sectorstorage: update sched tests for new logic
2020-08-28 21:38:21 +02:00
Łukasz Magiera
4a75e1e4b4
sectorstorage: Don't require tasks within a window to run in order
2020-08-28 19:38:55 +02:00
Łukasz Magiera
11b11e416b
sectorstorage: Compact assigned windows
2020-08-28 18:26:38 +02:00
Łukasz Magiera
20ffce5c0d
Merge remote-tracking branch 'origin/master' into fix/some-fsm-issues
2020-08-28 16:47:25 +02:00
Łukasz Magiera
5ee85dc263
sectorstorage: Fix tests
2020-08-28 16:33:41 +02:00
刘林欣
93dc33687a
ignore resend SectorStartPacking event, and unsealedInfoMap lock
2020-08-28 19:04:23 +08:00
Łukasz Magiera
6d1682a27e
storagefsm: wire up RecoverDealIDs fully
2020-08-28 11:44:15 +02:00
Łukasz Magiera
1097d29213
sealing sched: Call trySched less when there are many tasks
2020-08-28 00:03:42 +02:00
Łukasz Magiera
a6492b1ed6
storagefsm: Auto-retry failed remove
2020-08-27 23:59:01 +02:00
Łukasz Magiera
59d2034cbb
sealing sched: Wait a bit for tasks to come in on restart
2020-08-27 23:58:37 +02:00
Łukasz Magiera
7fdffc0340
sealing sched: Give more priority to tasks (re)moving data
2020-08-27 23:29:39 +02:00
Łukasz Magiera
f2bd680cc5
gofmt
2020-08-27 23:14:46 +02:00
Łukasz Magiera
59f554b658
sealing sched: Show waiting tasks assigned to workers in sealing jobs cli
2020-08-27 23:14:33 +02:00
Łukasz Magiera
d8e58e67c6
storagefsm: Treat PackingFailed sectors as expired
2020-08-27 22:41:35 +02:00
