Łukasz Magiera
f96f12c836
storagefsm: Add rest of checks in WaitDeals
2021-01-21 17:40:50 +01:00
Łukasz Magiera
069766ecc4
storagefsm: Don't persist piece assignment queue
2021-01-21 17:40:50 +01:00
Łukasz Magiera
270f2935a9
storagefsm: Check per-sector deal limits
2021-01-21 17:40:50 +01:00
Łukasz Magiera
fd67a41c75
storagefsm: Change sector CreationTime to unix ts
2021-01-21 17:40:50 +01:00
Łukasz Magiera
9857ad8378
storagefsm: Fix some deadlock cases
2021-01-21 17:40:50 +01:00
Łukasz Magiera
e5814dac4f
cbor-gen
2021-01-21 17:40:50 +01:00
Łukasz Magiera
542357a1df
storagefsm: Start packing correctly
2021-01-21 17:40:50 +01:00
Łukasz Magiera
239d6f8f4d
storagefsm: Rewrite input handling
2021-01-21 17:40:50 +01:00
dignifiedquire
3d9eb226fa
refactor: switch to filecoin-ffi bls api for bls signatures
2021-01-20 16:19:07 +01:00
dignifiedquire
187a499cca
remove extern/blst
2021-01-20 16:19:07 +01:00
Łukasz Magiera
48c6c55f68
Merge pull request #5384 from filecoin-project/fix-storage-size-estimation
...
fix FileSize to return correct disk usage recursively
2021-01-19 19:22:47 +01:00
Łukasz Magiera
560a127ce6
fsutil: Defensive check for Stat cast
2021-01-19 18:53:37 +01:00
Anton Evangelatov
834a6148b0
use stat.Blocks and not info.Size
2021-01-19 18:07:37 +01:00
Anton Evangelatov
16d07d3f18
fix FileSize to return correct disk usage recursively
2021-01-19 15:21:31 +01:00
Łukasz Magiera
0eaa6d9983
storagefsm: Send correct event on ErrExpiredTicket in CommitFailed
2021-01-17 22:29:07 +01:00
Łukasz Magiera
80b8d4b9d7
Address review
2021-01-14 20:27:15 +01:00
Łukasz Magiera
32885e1129
termination batcher: Notify based on what was sent
2021-01-14 17:14:26 +01:00
Łukasz Magiera
4015ddbb4f
fsm: Fix panic in precommit check in handleTerminating
2021-01-14 16:13:32 +01:00
Łukasz Magiera
49abdd7d7d
Sector termination support - address review
2021-01-14 15:46:57 +01:00
Łukasz Magiera
7ddf1d1feb
Add terminating states to state lists
2021-01-14 12:44:27 +01:00
Łukasz Magiera
144b5a1350
perning termination API
2021-01-14 12:37:23 +01:00
Łukasz Magiera
1564db1fce
Sector termination test
2021-01-14 00:11:41 +01:00
Łukasz Magiera
3522c8d45a
SectorTerminateFlush API
2021-01-13 23:32:04 +01:00
Łukasz Magiera
174c595acc
Don't declare in proving window
2021-01-13 22:19:10 +01:00
Łukasz Magiera
52cc2cd3eb
Initial sector termination support
2021-01-13 00:42:01 +01:00
Łukasz Magiera
5462ecf6da
gofmt
2021-01-11 16:44:11 +01:00
Łukasz Magiera
9e74c3c4e8
Multicore AddPiece CommP
2021-01-11 16:44:11 +01:00
Łukasz Magiera
299e72458e
storagefsm: Fix unsealedInfoMap.lk init race
2021-01-10 14:01:29 +01:00
Łukasz Magiera
2c9f4ad5ad
Merge pull request #5251 from liismn/master
...
Fix deadlock in runWorker in sched_worker.go
2021-01-05 19:35:49 +01:00
Łukasz Magiera
44a56c7e33
Merge pull request #5208 from jsign/jsign/imprreadpiece
...
sector-storage/mock: improve mocked readpiece
2021-01-05 18:37:48 +01:00
15ho
5b235db366
fix: stores.FetchHandler: defer close file and increase buffer size
2020-12-29 10:07:28 +08:00
liismn
e9cc697ee8
Update sched_worker.go
...
Avoid possible deadlock by sh.workersLk
2020-12-25 15:05:01 +08:00
quanzhan lu
d806fafee8
update weight canSeal and canStore when attach
...
update weight canSeal and canStore when attach storage
2020-12-22 19:07:17 +08:00
Ignacio Hagopian
f6cf27c781
improve mocked readpiece
...
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
2020-12-16 10:15:38 -03:00
zzx234234
7db5e8061d
fix a coding error in truncating sector log
2020-12-11 11:31:36 +08:00
Łukasz Magiera
46a5013dc1
stores: Simplify remote copy buf size logic
2020-12-10 20:54:42 +01:00
Łukasz Magiera
53a6c4eac6
Merge branch 'iosize' of github.com:acautman/lotus into acautman-iosize
2020-12-10 20:48:04 +01:00
caoao
d4582fb32e
write size according to RegisteredSealProof
2020-12-10 11:59:09 +08:00
stirlingx
3c50baaf5b
fix curSealing out of MaxSealingSectors limit
2020-12-10 11:04:42 +08:00
beck
d3fa728894
Remove extra spaces
2020-12-09 19:22:21 +08:00
Łukasz Magiera
1502ca6460
storagefsm: Fix addr selection for commits
2020-12-03 12:30:41 +01:00
Łukasz Magiera
562f5839b7
Fix lint
2020-12-02 22:01:09 +01:00
Łukasz Magiera
2fd93a55ac
plumb AddressSelector to stoage fsm
2020-12-02 21:47:54 +01:00
Łukasz Magiera
289ef910a0
fix imports, docsgen
2020-12-02 00:39:55 +01:00
Łukasz Magiera
95eaf13b5a
sectorstorage: Fix tests
2020-12-02 00:36:32 +01:00
Łukasz Magiera
c6f85886bf
miner: Add slow mode to proving check
2020-12-02 00:32:14 +01:00
Łukasz Magiera
614f45dcfb
Merge pull request #5041 from filecoin-project/feat/5013
...
Sector check command
2020-12-01 14:59:01 +01:00
Łukasz Magiera
a4f3758f4c
worker api: better grouping
2020-11-30 23:16:30 +01:00
Łukasz Magiera
e8d2560581
Fix tests
2020-11-27 16:47:48 +01:00
Łukasz Magiera
3672053ae9
worker: Support setting task types at runtime
2020-11-26 17:33:34 +01:00
Łukasz Magiera
2304ea0077
Merge pull request #5016 from filecoin-project/fix/fsm-rm-unceroverable
...
storagefsm: Add missing planners
2020-11-26 12:22:48 +01:00
Łukasz Magiera
74f040f9d5
storagefsm: Add SubmitCommit to toStatState switch
2020-11-26 12:02:35 +01:00
Łukasz Magiera
eeb38fa649
storagefsm: Test fsmPlanners map
2020-11-26 10:58:07 +01:00
Łukasz Magiera
ba87cb4fa9
storagefsm: Add missing planners
2020-11-26 10:57:47 +01:00
Łukasz Magiera
65653a1747
storagefsm: Add GetTicket to some maps
2020-11-26 10:53:31 +01:00
s1m0n21
663b3f4f21
add a sector check command
2020-11-26 15:02:43 +08:00
Łukasz Magiera
89493fe2b8
storagefsm: newDealSector review
2020-11-25 18:55:58 +01:00
Łukasz Magiera
6166204bac
storagefsm: Improve new deal sector logic
2020-11-25 16:40:04 +01:00
Łukasz Magiera
214f46bc88
Merge pull request #4876 from filecoin-project/fix/miner-ticket-loop
...
storagefsm: Fix expired ticket retry loop
2020-11-24 19:54:41 +01:00
Łukasz Magiera
a63ef1dcd5
Merge pull request #4984 from yaohcn/fix-log-warn
...
fix log format
2020-11-24 18:01:56 +01:00
Łukasz Magiera
7c831101f3
Merge pull request #4727 from filecoin-project/nonsense/integrate-testplans-to-lotus
...
move testground/lotus-soup testplan from oni to lotus
2020-11-24 17:05:29 +01:00
Łukasz Magiera
fe92a6e2d4
Merge pull request #4024 from zgfzgf/sector-sched
...
modify for unsafe
2020-11-24 16:19:17 +01:00
yaohcn
7c0b6f41d8
fix log format
2020-11-24 19:09:48 +08:00
Anton Evangelatov
fd640864cb
Merge branch 'master' into nonsense/integrate-testplans-to-lotus
2020-11-23 20:20:44 +01:00
Łukasz Magiera
83b1009837
storagefsm: check getticket precommit expication correctly
2020-11-23 19:12:54 +01:00
Łukasz Magiera
85e592bbe2
storagefsm: Retry pushing messages when out-of-funds
2020-11-23 19:03:31 +01:00
Łukasz Magiera
e2a6b99144
storagefsm: Fix expired ticket retry loop
2020-11-23 19:03:31 +01:00
Łukasz Magiera
0e39e760e5
Merge pull request #4929 from filecoin-project/feat/update-markets-commp-extraction
...
Remove all temp file generation for deals
2020-11-23 18:43:00 +01:00
Dirk McCormick
3634ca50ca
fix: sector-storage tests
2020-11-23 16:09:57 +01:00
Dirk McCormick
c965696586
fix: ffiwrapper sealer test
2020-11-23 14:21:46 +01:00
Dan Shao
f3838a47c7
Format workerID as string
2020-11-23 15:07:50 +08:00
zgfzgf
b6893b0a3f
solve merage problem
2020-11-22 16:15:30 +08:00
hannahhoward
218f7c2c5d
feat(markets): update markets, extract commp
2020-11-20 13:29:14 -08:00
Łukasz Magiera
f0eac01a3c
Merge remote-tracking branch 'origin/master' into nonsense/integrate-testplans-to-lotus
2020-11-20 15:22:10 +01:00
S1M0N21
2b611b3a99
Remove unused key
2020-11-19 23:48:28 +08:00
Aayush Rajasekaran
68a16afeda
Merge branch 'master' into release/v1.2.0
2020-11-17 19:40:05 -05:00
Łukasz Magiera
00335878ab
Merge pull request #4712 from filecoin-project/fix/commitfailed-checks
...
Check (pre)commit receipt before other checks in failed states
2020-11-18 01:26:02 +01:00
Łukasz Magiera
b242d69805
Make storiface.CallError json-friendly
2020-11-17 16:28:41 +01:00
Łukasz Magiera
b8853aa4d5
Add error codes to worker return
2020-11-17 16:17:55 +01:00
Łukasz Magiera
e0ade92b29
Fix lint warnings
2020-11-16 19:20:18 +01:00
Łukasz Magiera
444239a643
update ffi
2020-11-16 19:17:07 +01:00
Łukasz Magiera
cd0cb1407d
update ffi
2020-11-16 19:03:30 +01:00
Łukasz Magiera
915c5e0fdd
Get most tests to pass
2020-11-16 19:03:30 +01:00
Łukasz Magiera
2a7e4af5ee
Fix lotus-miner/lotus-worker build
2020-11-16 19:03:30 +01:00
Steven Allen
584907269a
make sector-storage compile
2020-11-16 19:03:30 +01:00
Łukasz Magiera
6bea9dd178
Making sealing logic work with multiple seal proof types
2020-11-16 19:03:30 +01:00
Łukasz Magiera
6bb67f2fe9
Merge pull request #4834 from filecoin-project/feat/sector-gc
...
miner: Basic storage cleanup command
2020-11-13 00:03:17 +01:00
Łukasz Magiera
408183c7a6
docsgen: Fix CI
2020-11-12 20:15:57 +01:00
Łukasz Magiera
11b041224e
fsm: Allow manually resurrecting removed sectors
2020-11-12 19:37:54 +01:00
Łukasz Magiera
8409e155d4
docsgen: Support miner/worker structs
2020-11-12 10:29:42 +01:00
Łukasz Magiera
8ac495723e
gofmt
2020-11-11 17:42:23 +01:00
Łukasz Magiera
09f9f871a3
Create a command to abort sealing calls
2020-11-11 17:39:12 +01:00
Łukasz Magiera
bd71bf2379
Expand sched-diag
2020-11-11 16:47:44 +01:00
Łukasz Magiera
18aa97f962
Merge pull request #4779 from filecoin-project/feat/work-track-hostname
...
storage: Track worker hostnames with work
2020-11-10 20:38:44 +01:00
Łukasz Magiera
5caa277261
storage: Track abandoned work more correctly
2020-11-09 23:38:20 +01:00
Łukasz Magiera
f819e71d12
storage: Separate returned jobs in jobs cli
2020-11-09 23:13:29 +01:00
Łukasz Magiera
27a9dd3bbb
storage: Track worker hostnames with work
2020-11-09 23:09:04 +01:00
Łukasz Magiera
5b8f555221
sched: Update after tasks finish preparing
2020-11-09 21:44:28 +01:00
Łukasz Magiera
fbc05fa5a8
stores: Report storage changes quickly
2020-11-09 21:43:52 +01:00
Łukasz Magiera
95d9084899
Merge pull request #4765 from zgfzgf/feat-sealing-garbage
...
optimize pledge sector
2020-11-09 21:06:33 +01:00
zgfzgf
5bcc6339b4
optimize code replace strings with constants
2020-11-09 16:21:16 +08:00
zgfzgf
3665f90795
optimize pledge sector
2020-11-08 16:24:50 +08:00
Anton Evangelatov
f7f5a27ebf
remove oni submodule
2020-11-06 21:03:56 +01:00
Łukasz Magiera
cde0e58e39
Check (pre)commit receipt before other checks in failed states
2020-11-03 18:47:29 +01:00
Łukasz Magiera
761c9ee2b6
Merge pull request #4699 from blockchain-lib/master
...
struct name incorrect
2020-11-03 11:47:18 +01:00
dignifiedquire
5c3055219c
update filecoin-ffi to master
2020-11-03 11:02:58 +01:00
stirlingx
b5d2f01993
Update selector_task.go
2020-11-03 09:20:48 +08:00
dignifiedquire
1d17d61dbc
update filecoin-ffi
2020-11-02 17:05:52 +01:00
dignifiedquire
d457b7a5e3
update filecoin-ffi
2020-11-02 17:05:52 +01:00
dignifiedquire
398a831701
update blst and filecoin-ffi
2020-11-02 17:05:52 +01:00
dignifiedquire
61fa03b1ad
use supranation/blst@0.2.0 directly
2020-11-02 17:05:52 +01:00
dignifiedquire
56980ffc1f
remove extern/fil-blst
2020-11-02 17:05:52 +01:00
Łukasz Magiera
094ea3fe97
Merge pull request #4666 from filecoin-project/fix/sched-issues
...
Fix worker reenabling, handle multiple restarts in worker
2020-10-30 21:34:07 +01:00
Łukasz Magiera
69e44ebf07
sched: re-register worker windows after re-enabling correctly
2020-10-30 18:32:59 +01:00
Łukasz Magiera
f90a387f96
sched: Print worker UUIDs in shed-diag correctly
2020-10-30 18:32:16 +01:00
Łukasz Magiera
774e2ecebf
sched: Fix worker reenabling
2020-10-30 18:01:37 +01:00
Łukasz Magiera
7fbb868513
Debug flag to force running sealing scheduler
2020-10-30 11:07:35 +01:00
Łukasz Magiera
ee814fb48e
Merge pull request #4643 from filecoin-project/fix/fsm-ticket-loop
...
storagefsm: Fix GetTicket loop when the sector is already precommitted
2020-10-29 18:25:14 +01:00
Łukasz Magiera
c6b03ce62b
sectorstorage: Missing unlock in waitWork
2020-10-29 15:18:51 +01:00
Łukasz Magiera
ea5bb5cdab
sectorstorage: Fix manager restart edge-case
2020-10-29 12:14:21 +01:00
Łukasz Magiera
33459f1b36
storagefsm: Fix GetTicket loop when the sector is already precommitted
2020-10-29 10:26:02 +01:00
Łukasz Magiera
f0f75e2d2c
Merge pull request #4627 from karalabe/fix-gpu-usage-tracking
...
extern/sector-storage: fix GPU usage overwrite bug
2020-10-29 10:13:54 +01:00
Łukasz Magiera
32ea060e99
Merge pull request #3618 from filecoin-project/feat/async-restartable-workers
...
Track sealing processes across lotus-miner restarts
2020-10-28 21:55:45 +01:00
Péter Szilágyi
5f657b4333
extern/sector-storage: fix GPU usage overwrite bug
2020-10-28 20:52:33 +02:00
Łukasz Magiera
da7ecc1527
Fix flaky sealing manager tests
2020-10-28 16:15:17 +01:00
Łukasz Magiera
4475b9ac05
Use filecoin-ffi master
2020-10-28 15:31:15 +01:00
Łukasz Magiera
4100f6eead
fix TestWDPostDoPost
2020-10-28 15:23:21 +01:00
Łukasz Magiera
ed2f81da2f
sched: Fix tests
2020-10-28 14:34:28 +01:00
Łukasz Magiera
4cf00b8b42
worker_local: address review
2020-10-28 14:29:17 +01:00
Łukasz Magiera
96c5ff7e7f
sched: use more letters for variables
2020-10-28 14:23:38 +01:00
Łukasz Magiera
8731fe9112
sched: split worker handling into more funcs
2020-10-28 14:14:50 +01:00
Łukasz Magiera
84b567c790
sched: move worker funcs to a separate file
2020-10-28 13:39:28 +01:00
Łukasz Magiera
843c0719e0
Update FFI
2020-10-27 15:37:11 +01:00
Łukasz Magiera
660236b224
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-23 23:25:35 +02:00
Łukasz Magiera
29e334de54
Merge pull request #4511 from filecoin-project/steb/generalize-window-post
...
Manage sectors by size instead of proof type.
2020-10-22 21:27:48 +02:00
Steven Allen
bcabe7b3b5
migrate methods to abstracted methods
...
Method numbers never change anyways. At worst, we'll deprecate old methods and
have to explicitly import them from the correct actors version to use them.
2020-10-21 12:18:37 -07:00
Steven Allen
4e730b5ec8
port to v2 imports
2020-10-21 12:16:23 -07:00
Steven Allen
00dcb1bce9
Manage sectors by size instead of proof type.
...
* We may have multiple sectors with the same size and different proof types, but all these management functions stay the same.
* This simplifies PoSt logic.
2020-10-20 18:30:56 -07:00
Łukasz Magiera
8c86ea6b75
localworker: Try very hard to get ruselts to manager
2020-10-18 19:45:11 +02:00
Łukasz Magiera
dbb421c4f7
localworker: Use better context for calling returnFunc
2020-10-18 19:32:43 +02:00
Łukasz Magiera
8d06cca073
sched: Handle workers using sessions instead of connections
2020-10-18 12:36:06 +02:00
Łukasz Magiera
7ac5dc55d0
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-17 13:45:11 +02:00
Raúl Kripalani
7e6ede7563
update schema; test-vectors submodule.
2020-10-15 12:49:23 +01:00
Łukasz Magiera
b74a3229f8
fsm: process expired-ticket sectors
2020-10-13 21:35:41 +02:00
Łukasz Magiera
71b3b9075d
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-13 21:33:21 +02:00
Łukasz Magiera
65883cbb6b
Merge pull request #4290 from filecoin-project/feat/ledger-integration
...
adding in ledger support
2020-10-12 15:09:27 +02:00
Steven Allen
dc4e73c737
Test the tape upgrade
2020-10-12 00:01:25 -07:00
Łukasz Magiera
4ac2bee81c
Update oni
2020-10-12 06:34:45 +02:00
Steven Allen
83dfc460d4
fix race in unseal
...
1. Remove an invalid error check.
2. Make sure to shadow the outer error type from within the goroutine instead or
reading the outer type.
This may have been causing test issues (caught in TestMinerAllInfo with the race
detector).
2020-10-09 15:39:41 -07:00
Raúl Kripalani
67b3a55e30
update test-vectors submodule.
2020-10-08 15:44:11 +01:00
Łukasz Magiera
0de3051821
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-08 13:10:41 +02:00
Steven Allen
767c346cf8
update test-vectors
2020-10-07 12:17:32 -07:00
Steven Allen
c17fa4bc35
update test-vectors for StateManager constructor change
2020-10-07 12:04:32 -07:00
Steven Allen
df4b068d0e
Merge branch 'master' into next
2020-10-07 11:55:00 -07:00
Łukasz Magiera
cfd126ee9f
Merge pull request #3961 from filecoin-project/docs/miner-storage
...
lotus-miner: add more help text to storage / attach
2020-10-07 14:55:42 +02:00
Steven Allen
1e78cd21ec
fix lotus soup build
2020-10-05 14:37:08 -07:00
Steven Allen
17846aad2f
add some more policy toggles for testground
2020-10-05 14:28:47 -07:00
Łukasz Magiera
1fc23fb466
lotus-miner: Cleanup storage attach helptext a bit
2020-10-03 11:30:22 +02:00
Steven Allen
8292d60196
Merge branch 'master' into asr/spec-v1
2020-10-02 18:03:03 -07:00
Łukasz Magiera
5932f28519
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-01 02:39:48 +02:00
Łukasz Magiera
6981f776f4
Lower PC2 memory requirements
2020-10-01 00:54:53 +02:00
Łukasz Magiera
1b7cdb9341
Fix storage manager tests
2020-10-01 00:54:34 +02:00
Łukasz Magiera
5e08d56630
sched: Allow some single-thread tasks to run in parallel with PC2/C2
2020-10-01 00:28:44 +02:00
Łukasz Magiera
79d2ddf24f
Review
2020-09-30 21:18:12 +02:00
Łukasz Magiera
2cfe22d4e5
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-30 20:48:16 +02:00
Łukasz Magiera
e3ee4e4718
Fix lint errors
2020-09-30 20:24:03 +02:00
Łukasz Magiera
2d16af6ee6
sectorstorage: Fix TestRedoPC1
2020-09-30 19:18:38 +02:00
Łukasz Magiera
4f97d9637e
Fix storage-fsm tests
2020-09-30 19:11:58 +02:00
Steven Allen
4989b1cc90
Merge branch 'master' into asr/spec-v1
2020-09-30 09:19:12 -07:00
Łukasz Magiera
c228598098
sectorstorage: Variable scopes are really hard
2020-09-30 18:16:07 +02:00
Łukasz Magiera
a783bf9b8b
storagefsm: Handle PC2 with missing replica
2020-09-30 17:53:15 +02:00
Łukasz Magiera
54fdd6ba5a
sectorstorage: Variable scopes are hard
2020-09-30 17:48:48 +02:00
Łukasz Magiera
6855284d88
sectorstorage: Cancel non-running work in case of abort in sched
2020-09-30 17:26:09 +02:00
Łukasz Magiera
5bffea6f54
Merge pull request #4064 from filecoin-project/tvx
...
tvx: a test vector extraction and execution tool
2020-09-30 17:22:42 +02:00
Raúl Kripalani
ff8663faa0
update test-vectors submodule.
2020-09-30 14:58:22 +01:00
Łukasz Magiera
430897c9ac
Merge pull request #3445 from filecoin-project/asr/precommit-failed-fix
...
Fix some failed precommit handling
2020-09-30 09:35:41 +02:00
Łukasz Magiera
bc85e3ce50
Merge pull request #4107 from shaodan/worker-no-swap
...
Add --no-swap flag for worker
2020-09-30 09:25:57 +02:00
Dan Shao
1affd498c1
Add --no-swap flag for worker
2020-09-30 14:23:35 +08:00
Łukasz Magiera
baef3c8dd2
sectorstorage: Fix potential panic in FinalizeSector
2020-09-29 15:22:46 +02:00
Łukasz Magiera
0f2dcf28b1
fsm: Reuse tickets in PC1 on retry
2020-09-29 10:07:49 +02:00
Steven Allen
1c03862854
fix policy for specs-actors update
2020-09-28 17:28:16 -07:00
Steven Allen
233d8a9b72
update even more imports
2020-09-28 14:25:58 -07:00
Łukasz Magiera
1e6a69f8aa
localworker: Don't mark calls as returned when returning fails
2020-09-28 22:10:02 +02:00
Łukasz Magiera
9bd2537971
stores: Fix error printing in http handler
2020-09-28 22:06:03 +02:00
Łukasz Magiera
810c767200
worker: Redeclare storage on reconnect
2020-09-28 21:06:49 +02:00
Łukasz Magiera
4ba7af6061
worker: Mark return methods as retry-safe
2020-09-28 20:46:44 +02:00
Łukasz Magiera
9e7d6823b1
sectorstorage: Cleanup callToWork mapping after work is done
2020-09-28 13:34:45 +02:00
Łukasz Magiera
cf71f034dc
Merge remote-tracking branch 'origin/dev' into feat/async-restartable-workers
2020-09-26 12:35:32 +02:00
zgfzgf
1a7aea1906
modify error
2020-09-25 22:59:21 +08:00
zgfzgf
3207bc4704
optimize trySched
2020-09-25 22:41:29 +08:00
zgfzgf
60e950015c
modify for unsafe
2020-09-25 22:13:27 +08:00
whyrusleeping
306c098d30
also update our vendored blst repo
2020-09-24 11:32:38 -07:00
whyrusleeping
6db37b72a8
update ffi to code with blst fixes
2020-09-24 11:05:21 -07:00
Łukasz Magiera
04ee53e061
sectorstorage: Show task type of ret-wait jobs
2020-09-24 11:55:11 +02:00
Steven Allen
32a699d6a3
Add some actors policy setters for testing
...
Addresses:
* a307e4593a (r491966115)a307e4593a (r491966634)
2020-09-23 15:00:52 -07:00
Łukasz Magiera
d817dceb05
Show lost calls in sealing jobs cli
2020-09-23 19:26:35 +02:00
Łukasz Magiera
c17f0d7e61
sectorstorage: Fix panic in returnResult
2020-09-23 17:37:05 +02:00
Łukasz Magiera
86c222ab58
sectorstorage: fix work tracking
2020-09-23 14:56:50 +02:00
Łukasz Magiera
ce6b92484f
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-23 11:31:21 +02:00
Steven Allen
773714792f
update oni
2020-09-22 18:40:03 -07:00
Łukasz Magiera
6185e157e9
sectorstorage: calltracker: work around cbor-gen bytearray len limit
2020-09-23 00:29:10 +02:00
Łukasz Magiera
04ad1791b0
localworker: Fix contexts
2020-09-23 00:10:36 +02:00
Łukasz Magiera
bb5cc06677
Fix workid param hash
2020-09-22 23:33:13 +02:00
Travis Person
c66f087f4c
lotus-miner: add more help text to storage / attach
2020-09-22 18:15:42 +00:00
Łukasz Magiera
706f4f2ef5
worker: Don't die with the connection
2020-09-22 18:36:44 +02:00
Aayush Rajasekaran
1dc69e397e
Resolve some unnecessary actor upgrade TODOs
2020-09-22 01:34:21 -04:00
Łukasz Magiera
b8865fb182
workers: Mark on-restart-failed returned tasks as returned
2020-09-22 01:00:28 +02:00
Łukasz Magiera
03c3d8bdb3
workers: Return unfinished tasks on restart
2020-09-22 00:52:33 +02:00
Łukasz Magiera
aa5bd7bc17
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-21 22:36:16 +02:00
Łukasz Magiera
f7934b083c
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
2020-09-21 19:10:18 +02:00
Alan Shaw
d9f2b909ac
chore: update test-vectors
2020-09-21 11:58:45 +01:00
Alan Shaw
a79291070e
chore: update test-vectors
2020-09-21 11:53:38 +01:00
Steven Allen
1bf3b4989d
rename imports to match actors code
...
`sed -i 's/\bv0\(\w\)\(\w*\)/\L\1\E\20/g' **/*.go`
2020-09-18 14:59:27 -07:00
Łukasz Magiera
7109e95f5e
Merge pull request #3881 from filcloud/issue-3626-SectorState
...
fix SectorState
2020-09-18 21:30:18 +02:00
Łukasz Magiera
70faa36b7f
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
2020-09-18 19:29:06 +02:00
Aayush Rajasekaran
fce423c743
Appease the linter and get everything building
2020-09-18 02:14:18 -04:00
Łukasz Magiera
6eda53565f
Most tests passing
2020-09-17 17:30:24 +02:00
Łukasz Magiera
e632643801
api: Test return types
2020-09-17 12:24:50 +02:00
hannahhoward
691bd9f442
feat(markets): complete markets conversion
...
complete markets conversion to using chain/actors types, also replacing DealProposal/DealState
interfaces with structs
2020-09-17 00:43:14 -07:00
Aayush Rajasekaran
9e48dd211a
Fixups
2020-09-17 02:34:15 -04:00
Aayush Rajasekaran
b5ba7a0fad
Miner migration
2020-09-17 02:21:16 -04:00
Frank
d5af25b76c
update init sector state list
2020-09-17 10:38:07 +08:00
Łukasz Magiera
17680fff55
gofmt
2020-09-17 00:35:57 +02:00
Łukasz Magiera
d9d644b27f
sectorstorage: handle restarting manager, test that
2020-09-17 00:35:09 +02:00
Łukasz Magiera
5e09581256
sectorstorage: get new work tracker to run
2020-09-16 22:33:58 +02:00
Łukasz Magiera
b1361aaf8b
sectorstorage: wip manager work tracker
2020-09-16 17:08:05 +02:00
Anton Evangelatov
0ff5a71114
update oni to master
2020-09-16 15:13:28 +02:00
Anton Evangelatov
b0677ab38e
replace lotus and filecoin-ffi deps
2020-09-16 15:12:41 +02:00
Anton Evangelatov
3f67c4524e
add filecoin-project oni as submodule and compile lotus-soup
2020-09-16 15:12:41 +02:00
Frank
ed74091c20
add exist sector state check
2020-09-16 19:49:45 +08:00
Łukasz Magiera
d4a7732d0a
Merge pull request #3800 from filecoin-project/asr/gascost
...
Add an API to get detailed gas costs for a message
2020-09-15 19:13:20 +02:00
Łukasz Magiera
03cf6cca40
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-15 17:47:03 +02:00
Steven Allen
36f920bcd7
progress
2020-09-14 21:55:49 -07:00
hannahhoward
7dc091052a
feat(manager): less restrictive storage lock
...
Use initial less restrictive storage lock when trying to read unsealed data before acquiring more
restrictive lock needed for unsealing
2020-09-14 18:48:14 -07:00
Steven Allen
02dcb5e182
Merge branch 'master' into refactor/net-upgrade
2020-09-14 14:53:57 -07:00
Aayush Rajasekaran
0edcae9e04
Fixup new command
2020-09-14 14:40:38 -04:00
Łukasz Magiera
e9d25e5919
More fixes
2020-09-14 20:28:47 +02:00
Łukasz Magiera
381a6cdfac
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-14 19:11:50 +02:00
Łukasz Magiera
1ebca8f732
more working code
2020-09-14 19:09:01 +02:00
Łukasz Magiera
06ec571c20
Merge remote-tracking branch 'origin/master' into refactor/net-upgrade
2020-09-14 15:09:52 +02:00
Raúl Kripalani
6d29d75724
Merge branch 'master' into inmem-journal
2020-09-14 12:17:45 +01:00
Steven Allen
d3594835c4
[WIP] Network upgrade support
...
This patch starts adding support for network upgrades.
* It adds an actors abstraction layer for loading abstract (cross-version) actors.
* It starts switching over to a shared deadline type.
* It adds an abstraction for ADTs (hamt/amt).
* It removes the callback-based API in the StateManager (difficult to abstract
across actor versions).
* It _does not_ actually add support for actors v2. We can do that in a followup
patch but that should be relatively easy.
This patch is heavily WIP and does not compile. Feel free to push changes
directly to this branch.
Notes:
* State tree access now needs a network version, because the HAMT type will change.
* I haven't figured out a nice way to abstract over changes to the _message_
types. However, many of them will be type aliased to actors v0 in actors v2 so
we can likely continue using the v0 versions (or use the v2 versions
everywhere). I've been renaming imports to `v0*` to make it clear that we're
importing types from a _specific_ actors version.
TODO:
* Consider merging incremental improvements? We'd have to get this compiling
again first but we could merge in the new abstractions, and slowly switch over.
* Finish migrating to the new abstractions.
* Remove all actor state types from the public API. See `miner.State.Info()` for
the planned approach here.
* Fix the tests. This is likely going to be a massive pain.
2020-09-11 20:16:29 -07:00
Aarsh Shah
62b2963781
retry add piece
2020-09-11 16:58:09 +05:30
Łukasz Magiera
e996a5246b
Merge remote-tracking branch 'origin/master' into feat/window-post-faulty-sectors
2020-09-10 22:18:17 +02:00
Łukasz Magiera
bbac86f745
gofmt, mod tidy
2020-09-10 22:07:20 +02:00
Łukasz Magiera
22556fb24c
Add fil-blst submodule
2020-09-10 22:04:08 +02:00
Łukasz Magiera
2374cb1bc5
Update ffi
2020-09-10 21:44:01 +02:00
Łukasz Magiera
c7b0241a48
ffiwrapper: Test skipping corrupted sectors in PoSt
2020-09-10 21:19:26 +02:00
Łukasz Magiera
5f08fe7ead
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-10 17:30:54 +02:00
lanzafame
558b637b07
Increase the number of times precommit2 is attempted before moving back to precommit1
2020-09-10 13:52:21 +10:00
Łukasz Magiera
5e7737f55d
wdpost: Handle skipped sectors correctly
2020-09-10 02:59:37 +02:00
whyrusleeping
7a6ceebb34
windowed post generation now returns faulty sectors
2020-09-09 14:00:15 -07:00
Raúl Kripalani
a5afd83c7b
Merge branch 'master' into fix/test-vectors-import-cycle
2020-09-09 13:04:41 +01:00
Raúl Kripalani
e8d1bab914
make the runner call statediff as a binary.
2020-09-09 13:03:43 +01:00
Dirk McCormick
17c15a74a2
fix: return true from Sealer.ReadPiece() on success
2020-09-08 13:50:56 +02:00
Dirk McCormick
8bbdf2e7cb
fix: storage manager - bail out on undefined unsealed cid
2020-09-08 12:54:01 +02:00
Aayush Rajasekaran
f91f665f3f
Update test-vectors
2020-09-08 01:24:30 -04:00
Aayush Rajasekaran
d678fe4bfa
Fix tests
2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a
Update to specs v0.9.6
2020-09-07 15:48:41 -04:00
Łukasz Magiera
47c59afea0
Revert "storage manager: bail out with an error if unsealed cid is undefined"
2020-09-07 20:12:29 +02:00
Łukasz Magiera
231a9e4051
Fix sealing sched tests
2020-09-07 17:55:31 +02:00
Dirk McCormick
a97f978cad
fix: storage manager - dont fail on successful read piece
2020-09-07 16:14:19 +02:00
Łukasz Magiera
9e6f974f3c
storage: Fix build
2020-09-07 16:12:55 +02:00
Dirk McCormick
07a4553e6e
fix: storage manager - bail out with an error if unsealed cid is undefined
2020-09-07 16:04:12 +02:00
Łukasz Magiera
06e3852cef
storage: Integrate async workers in sealing manager
2020-09-07 12:20:50 +02:00
Łukasz Magiera
5d73943929
storage: Fix import cycle
2020-09-06 18:54:00 +02:00
Łukasz Magiera
159ce13f5e
Async worker API
2020-09-06 18:47:16 +02:00
Raúl Kripalani
1ec534d607
Merge branch 'master' into inmem-journal
2020-09-04 10:55:40 +01:00
Raúl Kripalani
d46f684bb8
fix test following merge.
2020-09-04 00:05:44 +01:00
Raúl Kripalani
c723554d2e
upgrade test-vectors deps.
2020-09-03 23:44:21 +01:00
Raúl Kripalani
3206f92063
Merge branch 'master' into inmem-journal
2020-09-02 19:50:52 +01:00
Łukasz Magiera
b774563ec3
Merge pull request #3492 from filecoin-project/fix/readpiece-panic
...
ffiwrapper: Fix ReadPiece panic
2020-09-02 18:59:22 +02:00
Łukasz Magiera
ca7aa69597
ffiwrapper: More correct error check on openPartialFile
2020-09-02 18:45:07 +02:00
Łukasz Magiera
5f79ff340d
Merge pull request #3489 from filecoin-project/fix/sched-deadlocks
...
sealing sched: Fix deadlock between worker.wndLk / workersLk
2020-09-02 18:03:18 +02:00
Łukasz Magiera
5a2b439773
sched: Fix tests
2020-09-02 17:37:19 +02:00
Łukasz Magiera
7fe8580da5
sealing sched: Fix deadlock between worker.wndLk / workersLk
2020-09-02 17:06:48 +02:00
Raúl Kripalani
7cc9c83ab8
upgrade test-vectors.
2020-09-02 14:12:27 +01:00
Raúl Kripalani
0962292f92
conformance: add driver for tipset-class vectors.
2020-09-02 14:09:12 +01:00
Łukasz Magiera
fa6bae988e
Revert "conformance: add driver for tipset-class vectors."
2020-09-02 14:49:55 +02:00
Raúl Kripalani
ad5793e446
update test-vectors commit.
2020-09-01 20:01:41 +01:00
Raúl Kripalani
01737ef1cc
upgrade test-vectors deps.
2020-09-01 17:00:18 +01:00
Star.LI
82f5984de1
fix crash - segment fault when partialFile.Allocated() is invoked.
...
When openPartialFile is invoked, more errors than "existed error" are
returned. If only existing error is checked, the allocated field of
partialFile may be nil.
Signed-off-by: Star.LI <star@trapdoortech.com>
2020-09-01 11:41:26 +02:00
Aayush Rajasekaran
f58e8bc9a3
Fix some failed precommit handling
2020-09-01 02:18:02 -04:00
Łukasz Magiera
e14c80360d
sealing sched: Factor worker queues into utilization calc
2020-08-31 13:41:34 +02:00
Łukasz Magiera
98d51d3d80
storage: Correcty move unsealed sectors in FinalizeSector
2020-08-31 12:45:57 +02:00
Aayush Rajasekaran
27f7d3a658
Rebuild unsealed infos on miner restart
2020-08-30 18:01:07 -04:00
Łukasz Magiera
28ac2fce61
sched: Fix panic in workerCompactWindows
2020-08-29 06:41:19 +02:00
Łukasz Magiera
9d0c8ae3dd
sectorstorage: update sched tests for new logic
2020-08-28 21:38:21 +02:00
Łukasz Magiera
4a75e1e4b4
sectorstorage: Don't require tasks within a window to run in order
2020-08-28 19:38:55 +02:00
Łukasz Magiera
11b11e416b
sectorstorage: Compact assigned windows
2020-08-28 18:26:38 +02:00
Łukasz Magiera
20ffce5c0d
Merge remote-tracking branch 'origin/master' into fix/some-fsm-issues
2020-08-28 16:47:25 +02:00
Łukasz Magiera
5ee85dc263
sectorstorage: Fix tests
2020-08-28 16:33:41 +02:00
刘林欣
93dc33687a
ignore resend SectorStartPacking event, and unsealedInfoMap lock
2020-08-28 19:04:23 +08:00
Łukasz Magiera
6d1682a27e
storagefsm: wire up RecoverDealIDs fully
2020-08-28 11:44:15 +02:00
Łukasz Magiera
1097d29213
sealing sched: Call trySched less when there are many tasks
2020-08-28 00:03:42 +02:00
Łukasz Magiera
a6492b1ed6
storagefsm: Auto-retry failed remove
2020-08-27 23:59:01 +02:00
Łukasz Magiera
59d2034cbb
sealing sched: Wait a bit for tasks to come in on restart
2020-08-27 23:58:37 +02:00
Łukasz Magiera
7fdffc0340
sealing sched: Give more priority to tasks (re)moving data
2020-08-27 23:29:39 +02:00
Łukasz Magiera
f2bd680cc5
gofmt
2020-08-27 23:14:46 +02:00
Łukasz Magiera
59f554b658
sealing sched: Show waiting tasks assigned to workers in sealing jobs cli
2020-08-27 23:14:33 +02:00
Łukasz Magiera
d8e58e67c6
storagefsm: Treat PackingFailed sectors as expired
2020-08-27 22:41:35 +02:00
Łukasz Magiera
15204a9b90
storagefsm: fix tests
2020-08-27 21:34:45 +02:00
Łukasz Magiera
489d5239a5
storagefsm: Attempt to auto-recover from reorged DealIDs
2020-08-27 21:11:11 +02:00
Łukasz Magiera
7806a9885a
storagefsm: Record publish deals message CID
2020-08-27 17:50:50 +02:00
Łukasz Magiera
29078aaae9
storagefsm: Retry sending messages on out-of-gas errors
2020-08-27 14:02:00 +02:00
Łukasz Magiera
df635579c4
storagefsm: Handle sectors with expired deals better
2020-08-27 13:51:38 +02:00
Łukasz Magiera
788c7dbf48
storagefsm: Separate satte for submitting commit message
2020-08-27 12:57:28 +02:00
Łukasz Magiera
15191ff80f
storagefsm: Allow removing sectors in all states
2020-08-27 12:39:20 +02:00
Whyrusleeping
deb013cecb
Merge pull request #3081 from filecoin-project/conformance-tests
...
driver, runner and CI for interoperable test-vector based testing.
2020-08-26 14:06:39 -07:00
Raúl Kripalani
4709e231b8
drop aurora as dep; upgrade test-vectors.
2020-08-26 18:35:41 +01:00
Steven Allen
0155a31d1f
Fix PoSt with bad sectors
...
"skipped" sectors must be replaced with a substitute "good" sector, or the
entire partition must be skipped. They should not just be omitted.
This patch also fixes the test to verify the _entire_ proof instead of just
verifying that the proof includes the correct sectors.
2020-08-26 09:56:51 -07:00
Raúl Kripalani
efdfd3ee3e
Merge branch 'master' into inmem-journal
2020-08-26 16:38:23 +01:00
Raúl Kripalani
d3ac243cea
Merge remote-tracking branch 'origin/master' into conformance-tests
2020-08-26 15:12:02 +01:00
Alan Shaw
a89faa72ea
feat(conformance): skip incorrect tests ( #3310 )
...
Fixes https://github.com/filecoin-project/test-vectors/issues/72
2020-08-26 11:45:34 +01:00
Łukasz Magiera
d9796cd25c
sectorstorage: Make trySched less very slow
2020-08-24 19:16:16 +02:00
Łukasz Magiera
4311c96a44
Merge pull request #3225 from filecoin-project/fix/sched-missing-worker
...
check that worker referenced by task is actually still there.
2020-08-22 22:16:32 +02:00
Łukasz Magiera
a236957538
Merge pull request #3232 from filecoin-project/asr/marked-for-upgrade
...
Indicate whether a sector has been marked for upgrade
2020-08-22 21:48:45 +02:00
Łukasz Magiera
0806fd651a
Merge pull request #3183 from filecoin-project/steb/lint-all
...
Lint everything
2020-08-22 21:47:41 +02:00
Łukasz Magiera
0df6e1e2c7
Merge pull request #3220 from shaodan/fix-import
...
Fix import storage-fsm
2020-08-22 10:38:36 +02:00
Aayush Rajasekaran
984e52acbc
Indicate whether a sector has been marked for upgrade
2020-08-21 15:53:39 -04:00
whyrusleeping
54862be3ff
check that worker referenced by task is actually still there.
2020-08-21 10:33:36 -07:00
whyrusleeping
2e78f2c267
If sectorInfo doesnt have CommD or CommR set during commit, don't crash
2020-08-21 09:31:28 -07:00
Dan Shao
f21ce3e8ee
Fix import storage-fsm -> extern/storage-sealing
2020-08-21 16:52:33 +08:00
Steven Allen
5733c71c50
Lint everything
...
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.
There are three remaining XXXs marked in the code for lint.
2020-08-20 20:46:36 -07:00
Aayush Rajasekaran
e9c526b218
Do not try to replace CC sectors with a deal-less sector
2020-08-19 22:41:23 -04:00
Raúl Kripalani
e38f4219c5
update test-vectors commit.
2020-08-19 18:19:03 +01:00
Raúl Kripalani
30b6f38704
update test-vectors commit.
2020-08-19 18:14:19 +01:00
Raúl Kripalani
04e4bab313
register chaos actor only if selector says to do so.
2020-08-19 18:10:56 +01:00
Raúl Kripalani
dddf7951ef
update test-vectors commit.
2020-08-19 17:33:30 +01:00
Raúl Kripalani
a21234cd54
deploy the chaos actor.
2020-08-19 14:31:34 +01:00
Raúl Kripalani
4f4cebadc7
Merge branch 'master' into conformance-tests
2020-08-19 13:33:15 +01:00
Aayush Rajasekaran
7de7e65dbc
Calculate depositMinimum for upgrading as replaced sector's Initialpledge
2020-08-18 19:22:43 -04:00
Aayush Rajasekaran
98f00b510f
re-mark replaced sectors as for upgrade if precommit fails
2020-08-18 18:53:56 -04:00
Łukasz Magiera
a03192a39b
Separate concurrent sealing sector limit for deals
2020-08-18 21:48:18 +02:00
Łukasz Magiera
6a7d72c030
fsm: fail early on too-many-sealing-sectors in pledge
2020-08-18 19:26:17 +02:00
Łukasz Magiera
e7d65be90a
gofmt
2020-08-18 18:27:28 +02:00
Łukasz Magiera
baf91f08a1
Unbreak tests
2020-08-18 18:27:18 +02:00
Łukasz Magiera
886d9cd5eb
fsm: handle already-precommitted CommitFailed sectors correctly
2020-08-18 18:02:25 +02:00
Łukasz Magiera
6a49bd6d8e
fsm: Config for limiting max sealing sectors
2020-08-18 16:54:05 +02:00
Łukasz Magiera
65ffde9e4b
fsm: Config for max WaitDeals sectors
2020-08-18 16:20:31 +02:00
Aayush Rajasekaran
37ff876bc6
restart WaitDeals timers if miner restarts
2020-08-18 05:07:14 -04:00
Łukasz Magiera
01df17f1e7
Merge pull request #3103 from filecoin-project/fix/fsm-recover-bork-snark
...
fsm: Recover from errors from VerifySeal in checkCommit
2020-08-17 19:00:16 +02:00
Łukasz Magiera
bb79af7721
fsm: Recover from errors from VerifySeal in checkCommit
2020-08-17 18:53:09 +02:00
Łukasz Magiera
a66e202f02
fsm: Fix panic on missing precommit info
2020-08-17 18:45:38 +02:00
Łukasz Magiera
6ef7a30b19
Merge pull request #3089 from filecoin-project/integrate/storage-fsm
...
integrate extern/{storage-fsm,sector-storage} into lotus source tree
2020-08-17 18:37:54 +02:00
Raúl Kripalani
1f016262cd
readd int64 cast + nolint directive.
2020-08-17 17:26:07 +01:00
Raúl Kripalani
862bafc63a
fix lint errors.
2020-08-17 14:39:33 +01:00
Raúl Kripalani
efdc428d5d
keep storage-fsm (renamed to storage-sealing) and sector-storage in extern.
2020-08-17 14:26:18 +01:00
yaohcn
39bcfab37c
make addpiece configurable
2020-08-17 17:39:50 +08:00
Raúl Kripalani
483e82e14b
point test-vectors to master.
2020-08-16 22:05:43 +01:00
