Łukasz Magiera
1e4456138e
Merge master into feat/nv13
2021-05-27 12:28:20 +02:00
Łukasz Magiera
fc76a09b19
mock: Use real aggregate lengths
2021-05-26 10:06:41 +02:00
Łukasz Magiera
04658e1cae
fix lint
2021-05-25 16:26:59 +02:00
Łukasz Magiera
f5409845b5
Some review addressing
2021-05-25 16:07:45 +02:00
aarshkshah1992
0f3ff9e06a
fix typo
2021-05-24 15:22:11 +05:30
aarshkshah1992
dc6dbc9a11
dpr changes and test based on new unsealing PR
2021-05-22 22:40:21 +05:30
aarshkshah1992
50e023edd3
changes as per review
2021-05-21 19:15:05 +05:30
aarshkshah1992
2c9f5922b5
logs to debug read & unseal
2021-05-21 19:02:37 +05:30
aarshkshah1992
8ff5bce7a3
logs to debug read and unseal
2021-05-21 19:01:17 +05:30
aarshkshah1992
fb29f782df
integration test should remove unsealed files
2021-05-21 16:15:08 +05:30
aarshkshah1992
536d7c4275
clean up logging
2021-05-21 15:50:25 +05:30
aarshkshah1992
40642b2cad
better logging
2021-05-21 15:16:20 +05:30
aarshkshah1992
85f2ac8789
more logging
2021-05-21 14:56:37 +05:30
aarshkshah1992
78a0458ada
finish integration tests
2021-05-21 11:00:17 +05:30
aarshkshah1992
c17300dc1f
remove read task type and run gen and docsgen
2021-05-20 15:25:46 -06:00
aarshkshah1992
50d7acfa0c
fix go mod
2021-05-20 15:25:46 -06:00
aarshkshah1992
bd9959070e
unit tests for the remote store Reader
2021-05-20 15:25:46 -06:00
aarshkshah1992
9b34494501
use mockgen
2021-05-20 15:25:46 -06:00
aarshkshah1992
31a5f68dfa
use an actual worker in the integration tests
2021-05-20 15:25:46 -06:00
Dirk McCormick
ec85a973b1
feat: TestPieceProviderReadPiece
2021-05-20 15:25:46 -06:00
aarshkshah1992
8c4c26ca4c
fix linting problems
2021-05-20 15:25:46 -06:00
aarshkshah1992
dd9c9fd4fd
test http handler
2021-05-20 15:25:46 -06:00
aarshkshah1992
c58048d16a
address review comments
2021-05-20 15:25:46 -06:00
Aarsh Shah
d33d426692
Apply suggestions from code review
...
Co-authored-by: dirkmc <dirkmdev@gmail.com>
2021-05-20 15:25:46 -06:00
aarshkshah1992
73613ee883
docs, logs and green ci
2021-05-20 15:25:46 -06:00
aarshkshah1992
2a40c802ea
bypass task scheduler for reading unsealed pieces
2021-05-20 15:25:46 -06:00
Anton Evangelatov
9a7b0b657e
comment
2021-05-20 13:18:56 +02:00
Anton Evangelatov
19bd5beb96
update remoteGetSector
2021-05-20 13:15:57 +02:00
Anton Evangelatov
1a9b5760a7
add remoteGetAllocated http handler
2021-05-20 13:14:25 +02:00
Anton Evangelatov
76bb424de0
update StorageMinerAPI
2021-05-20 13:01:14 +02:00
Anton Evangelatov
cb603c62d9
update retrievaladapter ; add piece_provider
2021-05-20 12:49:53 +02:00
Anton Evangelatov
c12d802811
update reader to use ReadAtLeast
2021-05-20 12:38:38 +02:00
Anton Evangelatov
2aad7b6979
update sectorstorage.New (Manager) interface
2021-05-20 12:32:29 +02:00
Łukasz Magiera
e088c71b9a
marketadapter: Handle batch sealing messages
2021-05-19 20:07:20 +02:00
Łukasz Magiera
a5677d1b7a
ffiwrapper: Separate Prover interface
2021-05-19 15:20:23 +02:00
Anton Evangelatov
55401116bb
add UnpaddedByteIndex.Valid()
2021-05-19 13:16:29 +02:00
Anton Evangelatov
e275b54f55
export PartialFile and OpenPartialFile
2021-05-19 13:14:16 +02:00
Anton Evangelatov
a989f60e27
add SectorAddPieceToAny and SectorUnsealPiece to StorageMiner iface; model moved to api package - PieceDealInfo, DealSchedule
2021-05-19 13:05:07 +02:00
Łukasz Magiera
dd393b470f
Fix aggregation inputs
2021-05-18 20:36:47 +02:00
Łukasz Magiera
e400bdf87a
Order proofs before aggregation
2021-05-18 18:58:41 +02:00
Łukasz Magiera
9690bc882c
Test to trigger batching logic
2021-05-18 18:41:42 +02:00
Łukasz Magiera
fe9311e435
update ffi
2021-05-18 13:24:10 +02:00
Łukasz Magiera
6278bdc69a
Make things build
2021-05-17 20:47:41 +02:00
Łukasz Magiera
506f39b294
WIP: Integrate FIP0013
2021-05-11 22:10:29 -04:00
Anton Evangelatov
eb13c74dce
panic on unknown pathType
2021-05-11 18:14:01 +02:00
Anton Evangelatov
e07438417c
consider storiface.PathStorage when calculating storage requirements
2021-05-11 13:19:26 +02:00
whyrusleeping
ac77c51d5e
address nit
2021-04-29 10:24:16 -07:00
whyrusleeping
48feb52cbf
add a test for adding padded pieces
2021-04-27 16:28:25 -07:00
whyrusleeping
6daaf6ac2e
attempt to do better padding on pieces being written into sectors
2021-04-27 11:39:06 -07:00
Łukasz Magiera
a24a73bb52
Merge tag 'v1.8.0' into release/v1.9.0
2021-04-27 10:30:33 +02:00
Łukasz Magiera
c4137a438d
Merge commit 'f1ded63d608e1b191d79c780b8065f8cff367c98' into feat/nv12-1.11
2021-04-27 08:22:55 +02:00
Aayush Rajasekaran
f1ded63d60
v4 specs-actors integration, nv12 migration
2021-04-27 02:02:44 -04:00
Dirk McCormick
61344644a4
feat: add more debug logging for unsealing
2021-04-22 11:33:28 +02:00
yaohcn
844d1d364d
fix health report
2021-04-11 14:47:04 +08:00
Łukasz Magiera
9d4935a736
Merge pull request #5976 from filecoin-project/fix/storage-demote-stat-err
...
localstorage: Demote reservation stat error to debug
2021-04-09 21:08:08 +02:00
Łukasz Magiera
bed5ec81bd
localstorage: Demote reservation stat error to debug
2021-04-06 16:06:47 +02:00
Peter Rabbitson
25a77d905d
Centralize everything on ipfs/go-log/v2
...
I am not entirely sure this is right, but everything seems to build...
2021-04-06 15:04:32 +02:00
Łukasz Magiera
f372964a60
fix lint
2021-04-01 16:40:11 +02:00
Łukasz Magiera
c7de7882a1
Merge branch 'master' of github.com:ppiaas/lotus-ppiaas into fix/stg-fetch-close
2021-04-01 16:38:17 +02:00
yaohcn
3895b2ccca
fix reservations
2021-03-31 14:30:24 +08:00
Steven Allen
b74b1658b2
test: mock proof side effects on verify as well
2021-03-12 09:06:41 -08:00
Łukasz Magiera
d9923125ff
mock sealer: Simulate randomness sideeffects
2021-03-12 17:33:43 +01:00
Łukasz Magiera
cc490b983a
Merge pull request #5761 from filecoin-project/fix/testunpadreader-go116
...
Fix TestUnpadReader on Go 1.16
2021-03-10 13:13:27 +01:00
Łukasz Magiera
7fbb4bdd3b
Fix TestUnpadReader on Go 1.16
2021-03-10 13:05:25 +01:00
Łukasz Magiera
40fdf6c180
Merge pull request #5624 from filecoin-project/feat/miner-storage-limit
...
Configurable storage path storage limit
2021-03-10 11:15:28 +01:00
Łukasz Magiera
fe230f901e
Collect worker task metrics
2021-03-05 12:48:17 +01:00
Łukasz Magiera
35759fa07e
Merge pull request #5635 from filecoin-project/deps/cbg-soft-map
...
Update markets, cbor-gen with soft map decoding
2021-02-22 18:27:38 +01:00
Aayush Rajasekaran
6732758d0f
Tweak mock ProveCommits to respect proof sizes
2021-02-19 17:22:59 -05:00
Łukasz Magiera
26399dba70
Update markets, cbor-gen with soft map decoding
2021-02-19 20:11:43 +01:00
Łukasz Magiera
37b1fb97fb
Configurable storage path storage limit
2021-02-18 16:44:34 +01:00
yaohcn
3f66e731fa
fix drop sector
2021-02-18 12:00:28 +08:00
Łukasz Magiera
67780346e2
storagemgr: Always unseal full sectors
2021-01-25 21:39:15 +01:00
Łukasz Magiera
48c6c55f68
Merge pull request #5384 from filecoin-project/fix-storage-size-estimation
...
fix FileSize to return correct disk usage recursively
2021-01-19 19:22:47 +01:00
Łukasz Magiera
560a127ce6
fsutil: Defensive check for Stat cast
2021-01-19 18:53:37 +01:00
Anton Evangelatov
834a6148b0
use stat.Blocks and not info.Size
2021-01-19 18:07:37 +01:00
Anton Evangelatov
16d07d3f18
fix FileSize to return correct disk usage recursively
2021-01-19 15:21:31 +01:00
Łukasz Magiera
5462ecf6da
gofmt
2021-01-11 16:44:11 +01:00
Łukasz Magiera
9e74c3c4e8
Multicore AddPiece CommP
2021-01-11 16:44:11 +01:00
Łukasz Magiera
2c9f4ad5ad
Merge pull request #5251 from liismn/master
...
Fix deadlock in runWorker in sched_worker.go
2021-01-05 19:35:49 +01:00
Łukasz Magiera
44a56c7e33
Merge pull request #5208 from jsign/jsign/imprreadpiece
...
sector-storage/mock: improve mocked readpiece
2021-01-05 18:37:48 +01:00
15ho
5b235db366
fix: stores.FetchHandler: defer close file and increase buffer size
2020-12-29 10:07:28 +08:00
liismn
e9cc697ee8
Update sched_worker.go
...
Avoid possible deadlock by sh.workersLk
2020-12-25 15:05:01 +08:00
quanzhan lu
d806fafee8
update weight canSeal and canStore when attach
...
update weight canSeal and canStore when attach storage
2020-12-22 19:07:17 +08:00
Ignacio Hagopian
f6cf27c781
improve mocked readpiece
...
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
2020-12-16 10:15:38 -03:00
Łukasz Magiera
46a5013dc1
stores: Simplify remote copy buf size logic
2020-12-10 20:54:42 +01:00
caoao
d4582fb32e
write size according to RegisteredSealProof
2020-12-10 11:59:09 +08:00
beck
d3fa728894
Remove extra spaces
2020-12-09 19:22:21 +08:00
Łukasz Magiera
289ef910a0
fix imports, docsgen
2020-12-02 00:39:55 +01:00
Łukasz Magiera
95eaf13b5a
sectorstorage: Fix tests
2020-12-02 00:36:32 +01:00
Łukasz Magiera
c6f85886bf
miner: Add slow mode to proving check
2020-12-02 00:32:14 +01:00
Łukasz Magiera
614f45dcfb
Merge pull request #5041 from filecoin-project/feat/5013
...
Sector check command
2020-12-01 14:59:01 +01:00
Łukasz Magiera
a4f3758f4c
worker api: better grouping
2020-11-30 23:16:30 +01:00
Łukasz Magiera
e8d2560581
Fix tests
2020-11-27 16:47:48 +01:00
Łukasz Magiera
3672053ae9
worker: Support setting task types at runtime
2020-11-26 17:33:34 +01:00
s1m0n21
663b3f4f21
add a sector check command
2020-11-26 15:02:43 +08:00
Łukasz Magiera
a63ef1dcd5
Merge pull request #4984 from yaohcn/fix-log-warn
...
fix log format
2020-11-24 18:01:56 +01:00
Łukasz Magiera
fe92a6e2d4
Merge pull request #4024 from zgfzgf/sector-sched
...
modify for unsafe
2020-11-24 16:19:17 +01:00
yaohcn
7c0b6f41d8
fix log format
2020-11-24 19:09:48 +08:00
Łukasz Magiera
0e39e760e5
Merge pull request #4929 from filecoin-project/feat/update-markets-commp-extraction
...
Remove all temp file generation for deals
2020-11-23 18:43:00 +01:00
Dirk McCormick
3634ca50ca
fix: sector-storage tests
2020-11-23 16:09:57 +01:00
Dirk McCormick
c965696586
fix: ffiwrapper sealer test
2020-11-23 14:21:46 +01:00
Dan Shao
f3838a47c7
Format workerID as string
2020-11-23 15:07:50 +08:00
zgfzgf
b6893b0a3f
solve merage problem
2020-11-22 16:15:30 +08:00
hannahhoward
218f7c2c5d
feat(markets): update markets, extract commp
2020-11-20 13:29:14 -08:00
S1M0N21
2b611b3a99
Remove unused key
2020-11-19 23:48:28 +08:00
Łukasz Magiera
b242d69805
Make storiface.CallError json-friendly
2020-11-17 16:28:41 +01:00
Łukasz Magiera
b8853aa4d5
Add error codes to worker return
2020-11-17 16:17:55 +01:00
Łukasz Magiera
e0ade92b29
Fix lint warnings
2020-11-16 19:20:18 +01:00
Łukasz Magiera
915c5e0fdd
Get most tests to pass
2020-11-16 19:03:30 +01:00
Łukasz Magiera
2a7e4af5ee
Fix lotus-miner/lotus-worker build
2020-11-16 19:03:30 +01:00
Steven Allen
584907269a
make sector-storage compile
2020-11-16 19:03:30 +01:00
Łukasz Magiera
6bea9dd178
Making sealing logic work with multiple seal proof types
2020-11-16 19:03:30 +01:00
Łukasz Magiera
408183c7a6
docsgen: Fix CI
2020-11-12 20:15:57 +01:00
Łukasz Magiera
8409e155d4
docsgen: Support miner/worker structs
2020-11-12 10:29:42 +01:00
Łukasz Magiera
8ac495723e
gofmt
2020-11-11 17:42:23 +01:00
Łukasz Magiera
09f9f871a3
Create a command to abort sealing calls
2020-11-11 17:39:12 +01:00
Łukasz Magiera
bd71bf2379
Expand sched-diag
2020-11-11 16:47:44 +01:00
Łukasz Magiera
18aa97f962
Merge pull request #4779 from filecoin-project/feat/work-track-hostname
...
storage: Track worker hostnames with work
2020-11-10 20:38:44 +01:00
Łukasz Magiera
5caa277261
storage: Track abandoned work more correctly
2020-11-09 23:38:20 +01:00
Łukasz Magiera
f819e71d12
storage: Separate returned jobs in jobs cli
2020-11-09 23:13:29 +01:00
Łukasz Magiera
27a9dd3bbb
storage: Track worker hostnames with work
2020-11-09 23:09:04 +01:00
Łukasz Magiera
5b8f555221
sched: Update after tasks finish preparing
2020-11-09 21:44:28 +01:00
Łukasz Magiera
fbc05fa5a8
stores: Report storage changes quickly
2020-11-09 21:43:52 +01:00
zgfzgf
5bcc6339b4
optimize code replace strings with constants
2020-11-09 16:21:16 +08:00
stirlingx
b5d2f01993
Update selector_task.go
2020-11-03 09:20:48 +08:00
Łukasz Magiera
094ea3fe97
Merge pull request #4666 from filecoin-project/fix/sched-issues
...
Fix worker reenabling, handle multiple restarts in worker
2020-10-30 21:34:07 +01:00
Łukasz Magiera
69e44ebf07
sched: re-register worker windows after re-enabling correctly
2020-10-30 18:32:59 +01:00
Łukasz Magiera
f90a387f96
sched: Print worker UUIDs in shed-diag correctly
2020-10-30 18:32:16 +01:00
Łukasz Magiera
774e2ecebf
sched: Fix worker reenabling
2020-10-30 18:01:37 +01:00
Łukasz Magiera
7fbb868513
Debug flag to force running sealing scheduler
2020-10-30 11:07:35 +01:00
Łukasz Magiera
c6b03ce62b
sectorstorage: Missing unlock in waitWork
2020-10-29 15:18:51 +01:00
Łukasz Magiera
ea5bb5cdab
sectorstorage: Fix manager restart edge-case
2020-10-29 12:14:21 +01:00
Łukasz Magiera
f0f75e2d2c
Merge pull request #4627 from karalabe/fix-gpu-usage-tracking
...
extern/sector-storage: fix GPU usage overwrite bug
2020-10-29 10:13:54 +01:00
Péter Szilágyi
5f657b4333
extern/sector-storage: fix GPU usage overwrite bug
2020-10-28 20:52:33 +02:00
Łukasz Magiera
da7ecc1527
Fix flaky sealing manager tests
2020-10-28 16:15:17 +01:00
Łukasz Magiera
4100f6eead
fix TestWDPostDoPost
2020-10-28 15:23:21 +01:00
Łukasz Magiera
ed2f81da2f
sched: Fix tests
2020-10-28 14:34:28 +01:00
Łukasz Magiera
4cf00b8b42
worker_local: address review
2020-10-28 14:29:17 +01:00
Łukasz Magiera
96c5ff7e7f
sched: use more letters for variables
2020-10-28 14:23:38 +01:00
Łukasz Magiera
8731fe9112
sched: split worker handling into more funcs
2020-10-28 14:14:50 +01:00
Łukasz Magiera
84b567c790
sched: move worker funcs to a separate file
2020-10-28 13:39:28 +01:00
Łukasz Magiera
660236b224
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-23 23:25:35 +02:00
Łukasz Magiera
29e334de54
Merge pull request #4511 from filecoin-project/steb/generalize-window-post
...
Manage sectors by size instead of proof type.
2020-10-22 21:27:48 +02:00
Steven Allen
4e730b5ec8
port to v2 imports
2020-10-21 12:16:23 -07:00
Steven Allen
00dcb1bce9
Manage sectors by size instead of proof type.
...
* We may have multiple sectors with the same size and different proof types, but all these management functions stay the same.
* This simplifies PoSt logic.
2020-10-20 18:30:56 -07:00
Łukasz Magiera
8c86ea6b75
localworker: Try very hard to get ruselts to manager
2020-10-18 19:45:11 +02:00
Łukasz Magiera
dbb421c4f7
localworker: Use better context for calling returnFunc
2020-10-18 19:32:43 +02:00
Łukasz Magiera
8d06cca073
sched: Handle workers using sessions instead of connections
2020-10-18 12:36:06 +02:00
Łukasz Magiera
71b3b9075d
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-13 21:33:21 +02:00
Steven Allen
dc4e73c737
Test the tape upgrade
2020-10-12 00:01:25 -07:00
