Łukasz Magiera
6fd1609410
fr32: Reduce MTTresh from 32M to 512k per core
...
This results in 64x less bytes allocated when spawning new readers
for larger pieces.
Results in about 30% speedup in 1G unpad benchmark on AMD TR 2950x
2021-12-09 16:14:47 +01:00
Łukasz Magiera
9c75a3aaa8
piecereader: Avoid allocating 1024MB slices per read
2021-12-09 15:49:43 +01:00
Łukasz Magiera
a3d8494a04
piecereader: Avoid redundant roundtrips when seeking
2021-12-09 14:52:33 +01:00
Łukasz Magiera
13b260e7f7
piecereader: Move closer to storage
2021-12-08 23:20:20 +01:00
zenground0
a5be80828a
RemoveData and Decode
...
- Unsealing replica update with sector key works and tested
- Sector key generation added and tested
2021-12-03 15:21:06 -05:00
Łukasz Magiera
727765b248
Command to list active sector locks
2021-12-03 12:33:23 +01:00
Łukasz Magiera
71329f6c41
Address Scheduler enhancements ( #7703 ) review
2021-11-30 20:50:40 +01:00
zenground0
40d16a8f88
Review Response
2021-11-30 13:53:37 -05:00
zenground0
f88fcdbcfc
WIP
2021-11-30 12:40:14 -05:00
Łukasz Magiera
001ecbb561
fix lint
2021-11-30 02:06:58 +01:00
Łukasz Magiera
a597b072b8
fix sched tests
2021-11-30 02:06:58 +01:00
Łukasz Magiera
f25efecb74
worker: Test resource table overrides
2021-11-30 02:06:58 +01:00
Łukasz Magiera
6d52d8552b
Fix docsgen
2021-11-30 02:06:58 +01:00
Łukasz Magiera
c9a2ff4007
cleanup worker resource overrides
2021-11-30 02:06:58 +01:00
Łukasz Magiera
b961e1aab5
sched resources: Separate Parallelism defaults depending on GPU presence
2021-11-30 02:06:58 +01:00
Łukasz Magiera
36868a8749
sched: C2 is not all-core load
2021-11-30 02:06:58 +01:00
Clint Armstrong
4ef8543128
Permit workers to override resource table
...
In an environment with heterogenious worker nodes, a universal resource
table for all workers does not allow effective scheduling of tasks. Some
workers may have different proof cache settings, changing the required
memory for different tasks. Some workers may have a different count of
CPUs per core-complex, changing the max parallelism of PC1.
This change allows workers to customize these parameters with
environment variables. A worker could set the environment variable
PC1_MIN_MEMORY for example to customize the minimum memory requirement
for PC1 tasks. If no environment variables are specified, the resource
table on the miner is used, except for PC1 parallelism.
If PC1_MAX_PARALLELISM is not specified, and
FIL_PROOFS_USE_MULTICORE_SDR is set, PC1_MAX_PARALLELSIM will
automatically be set to FIL_PROOFS_MULTICORE_SDR_PRODUCERS + 1.
2021-11-30 02:06:58 +01:00
Clint Armstrong
93e4656a27
Use a float to represent GPU utilization
...
Before this change workers can only be allocated one GPU task,
regardless of how much of the GPU resources that task uses, or how many
GPUs are in the system.
This makes GPUUtilization a float which can represent that a task needs
a portion, or multiple GPUs. GPUs are accounted for like RAM and CPUs so
that workers with more GPUs can be allocated more tasks.
A known issue is that PC2 cannot use multiple GPUs. And even if the
worker has multiple GPUs and is allocated multiple PC2 tasks, those
tasks will only run on the first GPU.
This could result in unexpected behavior when a worker with multiple
GPUs is assigned multiple PC2 tasks. But this should not suprise any
existing users who upgrade, as any existing users who run workers with
multiple GPUs should already know this and be running a worker per GPU
for PC2. But now those users have the freedom to customize the GPU
utilization of PC2 to be less than one and effectively run multiple PC2
processes in a single worker.
C2 is capable of utilizing multiple GPUs, and now workers can be
customized for C2 accordingly.
2021-11-30 02:06:58 +01:00
Clint Armstrong
c4f46171ae
Report memory used and swap used in worker res
...
Attempting to report "memory used by other processes" in the MemReserved
field fails to take into account the fact that the system's memory used
includes memory used by ongoing tasks.
To properly account for this, worker should report the memory and swap
used, then the scheduler that is aware of the memory requirements for a
task can determine if there is sufficient memory available for a task.
2021-11-30 02:06:58 +01:00
Clint Armstrong
e2a1ca7caa
Use cgroup limits in worker memory calculations
...
Worker processes may have memory limitations imposed by Systemd. But
/proc/meminfo shows the entire system memory regardless of these limits.
This results in the scheduler believing the worker has the entire system
memory avaliable and the worker being allocated too many tasks.
This change attempts to read cgroup memory limits for the worker
process. It supports cgroups v1 and v2, and compares cgroup limits
against the system memory and returns the most conservative values to
prevent the worker from being allocated too many tasks and potentially
triggering an OOM event.
2021-11-30 02:06:58 +01:00
Łukasz Magiera
d21c44e266
ffiwrapper: Validate PC2 by calling C1 with random seeds
2021-11-30 01:33:05 +01:00
zenground0
7d2b3f05db
WIP sector storage and integration test
2021-11-29 10:24:00 -05:00
Łukasz Magiera
05aa860459
Request correct read size with startOffset in pieceProvider
2021-11-27 00:05:45 +01:00
Łukasz Magiera
743ce5a40f
Add startOffset support to mock SectorMgr.ReadPiece
2021-11-26 18:48:52 +01:00
Łukasz Magiera
f6de16e95a
Fix sector-storage tests
2021-11-26 18:16:53 +01:00
Łukasz Magiera
8d955d5f30
dagstore mount: Add random access support
2021-11-26 17:40:53 +01:00
Łukasz Magiera
8454abcf45
storage: Use 1M buffers for Tar transfers
2021-11-24 20:08:37 +01:00
Łukasz Magiera
2a1505b364
storage: Test StorageFindSector with groups
2021-11-23 16:11:04 +01:00
Łukasz Magiera
8b548ac02f
storage: Check allowlists in StorageFindSector
2021-11-23 16:11:04 +01:00
Łukasz Magiera
5c77c25747
storage: Add Group tags to StorageInfo
2021-11-23 16:11:04 +01:00
Łukasz Magiera
d1a63e4173
remote store: Remove debug printf
2021-11-22 17:50:12 +01:00
Aayush Rajasekaran
8665e32221
Update deps
2021-11-17 15:55:14 -05:00
Aayush Rajasekaran
07f1be0ff6
Plug in the FFI call
2021-11-17 15:55:14 -05:00
Aayush Rajasekaran
1449644c34
Integrate v7 actors
2021-11-17 15:55:14 -05:00
Łukasz Magiera
e508055dc1
make gen
2021-10-19 11:13:23 +02:00
Łukasz Magiera
080aa3356a
Fix locks in worker-tracked
2021-10-18 20:19:21 +02:00
Łukasz Magiera
70589e4406
Block work in tracked worker before it is started
2021-10-18 18:44:56 +02:00
Łukasz Magiera
261238e157
Show prepared tasks in sealing jobs
2021-10-18 18:44:56 +02:00
Łukasz Magiera
11d738eee0
Track prepared work
2021-10-18 18:44:56 +02:00
Marten Seemann
03806f7063
add missing build constraint to statfs_unix.go
2021-10-18 16:29:01 +02:00
Łukasz Magiera
f352c18290
Don't remove sector data when moving data into a shared path
2021-10-11 21:11:38 +02:00
Łukasz Magiera
9af82f2d68
sched: Fix taskDone chan deadlock
2021-10-03 17:09:43 +02:00
Łukasz Magiera
b87142ec8e
wip improve scheduling of ready work
2021-10-03 10:38:08 +02:00
Łukasz Magiera
d7fbd8b67d
Update proofs to v10.0.0
2021-10-01 18:38:27 +02:00
Łukasz Magiera
a8a9818043
Expose storage states on the metrics endpoint
2021-10-01 14:45:01 +02:00
Łukasz Magiera
ef03314c6d
storagemgr: Cleanup workerLk around worker resources
2021-09-15 16:35:19 +02:00
Łukasz Magiera
3118bd1039
stores: Fix reserved disk usage log spam
2021-08-31 13:36:09 +02:00
Steven Allen
1cf556c3a2
feat: expose ChainGetPath on the gateway
2021-08-30 16:43:21 -07:00
Łukasz Magiera
2293ecd8e8
Reduce lotus-miner startup spam
2021-08-27 19:41:54 +02:00
Aarsh Shah
d7076778e2
integrate DAG store and CARv2 in deal-making ( #6671 )
...
This commit removes badger from the deal-making processes, and
moves to a new architecture with the dagstore as the cental
component on the miner-side, and CARv2s on the client-side.
Every deal that has been handed off to the sealing subsystem becomes
a shard in the dagstore. Shards are mounted via the LotusMount, which
teaches the dagstore how to load the related piece when serving
retrievals.
When the miner starts the Lotus for the first time with this patch,
we will perform a one-time migration of all active deals into the
dagstore. This is a lightweight process, and it consists simply
of registering the shards in the dagstore.
Shards are backed by the unsealed copy of the piece. This is currently
a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so
when it's time to acquire a shard to serve a retrieval, the unsealed
CARv1 is joined with its index (safeguarded by the dagstore), to form
a read-only blockstore, thus taking the place of the monolithic
badger.
Data transfers have been adjusted to interface directly with CARv2 files.
On inbound transfers (client retrievals, miner storage deals), we stream
the received data into a CARv2 ReadWrite blockstore. On outbound transfers
(client storage deals, miner retrievals), we serve the data off a CARv2
ReadOnly blockstore.
Client-side imports are managed by the refactored *imports.Manager
component (when not using IPFS integration). Just like it before, we use
the go-filestore library to avoid duplicating the data from the original
file in the resulting UnixFS DAG (concretely the leaves). However, the
target of those imports are what we call "ref-CARv2s": CARv2 files placed
under the `$LOTUS_PATH/imports` directory, containing the intermediate
nodes in full, and the leaves as positional references to the original file
on disk.
Client-side retrievals are placed into CARv2 files in the location:
`$LOTUS_PATH/retrievals`.
A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore`
subcommands have been introduced on the miner-side to inspect and manage
the dagstore.
Despite moving to a CARv2-backed system, the IPFS integration has been
respected, and it continues to be possible to make storage deals with data
held in an IPFS node, and to perform retrievals directly into an IPFS node.
NOTE: because the "staging" and "client" Badger blockstores are no longer
used, existing imports on the client will be rendered useless. On startup,
Lotus will enumerate all imports and print WARN statements on the log for
each import that needs to be reimported. These log lines contain these
messages:
- import lacks carv2 path; import will not work; please reimport
- import has missing/broken carv2; please reimport
At the end, we will print a "sanity check completed" message indicating
the count of imports found, and how many were deemed broken.
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
2021-08-16 23:34:32 +01:00
Steven Allen
18f39be3ba
fix: don't check for t_aux when proving
...
We don't need it.
2021-08-09 11:07:35 -07:00
Anton Evangelatov
16784aa2cc
remove pieceProvider from DI; small refactors
2021-07-12 11:30:26 +02:00
Anton Evangelatov
566a9d5541
fix signature
2021-07-06 18:12:30 +02:00
Anton Evangelatov
7a78527fde
remove not used sa; cleanup
2021-07-06 17:46:21 +02:00
Anton Evangelatov
604be5fc82
Update extern/sector-storage/stores/remote.go
...
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2021-07-06 17:24:56 +02:00
Anton Evangelatov
4be0a7a215
resolve conflicts with master
2021-07-05 13:13:32 +02:00
Łukasz Magiera
8a94ab676e
storage: Fix FinalizeSector with sectors in stoage paths
2021-07-02 19:54:45 +02:00
Anton Evangelatov
21b51328f9
adding TestDealWithMarketAndMinerNode
2021-06-23 12:44:52 +02:00
Anton Evangelatov
6720463799
resolve merge conflicts
2021-06-22 11:28:23 +02:00
Raúl Kripalani
684cce198f
add a unit test.
2021-06-21 20:49:24 +01:00
Raúl Kripalani
59eab2df25
move scheduling filtering logic down.
2021-06-21 20:49:16 +01:00
Raúl Kripalani
83839362c5
fix boolean condition.
2021-06-21 20:35:51 +01:00
Raúl Kripalani
b6147fb27f
extern/storage: retype resource filtering strategy to enum.
2021-06-21 20:28:15 +01:00
Raúl Kripalani
f3b6f8de1a
add ability to ignore worker resources when scheduling.
2021-06-21 20:08:18 +01:00
Łukasz Magiera
b37a66c7c9
Merge remote-tracking branch 'origin/release/v1.10.0' into chore/merge-1.10
2021-06-21 17:03:03 +02:00
Łukasz Magiera
5516f3492e
ffiwrapper: Assert aggregate length
2021-06-18 12:02:42 +02:00
Łukasz Magiera
44de67cf7d
Merge pull request #6175 from filecoin-project/feat/dynamic-retreival-pricing
...
Dynamic Retrieval pricing
2021-06-17 10:25:48 +02:00
aarshkshah1992
9002fcbe47
local unsealed file wont have the unsealed piece
2021-06-17 09:46:29 +05:30
Anton Evangelatov
d45bb14015
Merge branch 'master' into nonsense/split-market-miner-processes
2021-06-16 18:49:15 +02:00
Aayush Rajasekaran
520a0091f5
Merge branch 'releases' into release/v1.10.0
2021-06-15 17:40:36 -04:00
Łukasz Magiera
c7c593c74e
TestMulticoreSDR: Setup rust logger after envvar check
2021-06-15 17:59:46 +02:00
Łukasz Magiera
6b0aed9317
Setup rust logger in the test
2021-06-15 17:53:26 +02:00
Łukasz Magiera
265afd696e
Run TestMulticoreSDR on Circle
2021-06-15 17:46:06 +02:00
Łukasz Magiera
9ae780902a
Test multicore SDR support
2021-06-15 17:36:10 +02:00
Anton Evangelatov
beb5bffaf6
fix log.Error; rename RemoteStore
2021-06-09 13:05:54 +02:00
Anton Evangelatov
30ba9a2751
remove double http handler decl
2021-06-08 15:36:54 +02:00
Anton Evangelatov
4dd093b160
fix piece provider test
2021-06-08 14:57:45 +02:00
Anton Evangelatov
9f3ec82cc1
initial resolution of conflicts
2021-06-08 13:01:22 +02:00
aarshkshah1992
4e9bb16532
changes as per review
2021-06-08 10:09:27 +05:30
aarshkshah1992
212d0fc264
fix remote store diff
2021-06-08 09:54:45 +05:30
aarshkshah1992
e4e60d7af4
fix mock
2021-06-08 09:37:12 +05:30
aarshkshah1992
16166504f4
merged master
2021-06-08 09:17:40 +05:30
aarshkshah1992
ddd9bf610e
fix CI
2021-06-07 16:45:52 +05:30
Anton Evangelatov
b0cb0c1a4a
do not depend on filecoin-ffi in api package
2021-06-07 11:51:25 +02:00
aarshkshah1992
8d9cef17af
changes as per review
2021-06-07 15:03:09 +05:30
aarshkshah1992
2a134887c3
logs to debug read & unseal
2021-06-07 15:03:09 +05:30
aarshkshah1992
ec6a49693f
logs to debug read and unseal
2021-06-07 15:03:09 +05:30
aarshkshah1992
207f0d901a
integration test should remove unsealed files
2021-06-07 15:03:09 +05:30
aarshkshah1992
73f7825fbb
clean up logging
2021-06-07 15:03:09 +05:30
aarshkshah1992
3b792a32c3
better logging
2021-06-07 15:03:09 +05:30
aarshkshah1992
22f36483cb
more logging
2021-06-07 15:03:09 +05:30
aarshkshah1992
acfa3d7370
finish integration tests
2021-06-07 15:03:09 +05:30
aarshkshah1992
ad4b182bfe
remove read task type and run gen and docsgen
2021-06-07 15:03:06 +05:30
aarshkshah1992
35a0dbfa8c
fix go mod
2021-06-07 15:02:52 +05:30
aarshkshah1992
4efaa8d688
unit tests for the remote store Reader
2021-06-07 15:02:52 +05:30
aarshkshah1992
0d88800eb9
use mockgen
2021-06-07 15:02:52 +05:30
aarshkshah1992
77b5e8d045
use an actual worker in the integration tests
2021-06-07 15:02:52 +05:30
Dirk McCormick
6879ae9e6a
feat: TestPieceProviderReadPiece
2021-06-07 15:02:52 +05:30
aarshkshah1992
74372d3e81
fix linting problems
2021-06-07 15:02:52 +05:30
aarshkshah1992
759d8f090b
test http handler
2021-06-07 15:02:52 +05:30
aarshkshah1992
db5c88196d
address review comments
2021-06-07 15:02:52 +05:30
Aarsh Shah
c853350bdf
Apply suggestions from code review
...
Co-authored-by: dirkmc <dirkmdev@gmail.com>
2021-06-07 15:02:52 +05:30
aarshkshah1992
65eb610ec3
docs, logs and green ci
2021-06-07 15:02:49 +05:30
aarshkshah1992
670835fca0
bypass task scheduler for reading unsealed pieces
2021-06-07 15:02:04 +05:30
Anton Evangelatov
8bd09e39ca
resolve merge conflicts
2021-06-04 16:17:00 +02:00
Anton Evangelatov
88756f3ebf
fix TestAPIDeal tests
2021-06-04 15:41:38 +02:00
aarshkshah1992
05ba3de5cc
changes as per review
2021-06-04 10:27:43 +05:30
Łukasz Magiera
565bb4f589
mock: Log debug info on bad aggregates
2021-06-03 17:35:10 +02:00
aarshkshah1992
43453bb633
fix broken test
2021-06-03 10:15:23 +05:30
Łukasz Magiera
40cc29d723
Skip FD check in TestDownloadParams
2021-06-02 19:50:17 +02:00
Anton Evangelatov
d195a12565
Merge branch 'master' into nonsense/split-market-miner-processes
2021-05-31 11:26:11 +02:00
Łukasz Magiera
1e4456138e
Merge master into feat/nv13
2021-05-27 12:28:20 +02:00
Łukasz Magiera
fc76a09b19
mock: Use real aggregate lengths
2021-05-26 10:06:41 +02:00
Łukasz Magiera
04658e1cae
fix lint
2021-05-25 16:26:59 +02:00
Łukasz Magiera
f5409845b5
Some review addressing
2021-05-25 16:07:45 +02:00
aarshkshah1992
0f3ff9e06a
fix typo
2021-05-24 15:22:11 +05:30
aarshkshah1992
dc6dbc9a11
dpr changes and test based on new unsealing PR
2021-05-22 22:40:21 +05:30
aarshkshah1992
50e023edd3
changes as per review
2021-05-21 19:15:05 +05:30
aarshkshah1992
2c9f5922b5
logs to debug read & unseal
2021-05-21 19:02:37 +05:30
aarshkshah1992
8ff5bce7a3
logs to debug read and unseal
2021-05-21 19:01:17 +05:30
aarshkshah1992
fb29f782df
integration test should remove unsealed files
2021-05-21 16:15:08 +05:30
aarshkshah1992
536d7c4275
clean up logging
2021-05-21 15:50:25 +05:30
aarshkshah1992
40642b2cad
better logging
2021-05-21 15:16:20 +05:30
aarshkshah1992
85f2ac8789
more logging
2021-05-21 14:56:37 +05:30
aarshkshah1992
78a0458ada
finish integration tests
2021-05-21 11:00:17 +05:30
aarshkshah1992
c17300dc1f
remove read task type and run gen and docsgen
2021-05-20 15:25:46 -06:00
aarshkshah1992
50d7acfa0c
fix go mod
2021-05-20 15:25:46 -06:00
aarshkshah1992
bd9959070e
unit tests for the remote store Reader
2021-05-20 15:25:46 -06:00
aarshkshah1992
9b34494501
use mockgen
2021-05-20 15:25:46 -06:00
aarshkshah1992
31a5f68dfa
use an actual worker in the integration tests
2021-05-20 15:25:46 -06:00
Dirk McCormick
ec85a973b1
feat: TestPieceProviderReadPiece
2021-05-20 15:25:46 -06:00
aarshkshah1992
8c4c26ca4c
fix linting problems
2021-05-20 15:25:46 -06:00
aarshkshah1992
dd9c9fd4fd
test http handler
2021-05-20 15:25:46 -06:00
aarshkshah1992
c58048d16a
address review comments
2021-05-20 15:25:46 -06:00
Aarsh Shah
d33d426692
Apply suggestions from code review
...
Co-authored-by: dirkmc <dirkmdev@gmail.com>
2021-05-20 15:25:46 -06:00
aarshkshah1992
73613ee883
docs, logs and green ci
2021-05-20 15:25:46 -06:00
aarshkshah1992
2a40c802ea
bypass task scheduler for reading unsealed pieces
2021-05-20 15:25:46 -06:00
Anton Evangelatov
9a7b0b657e
comment
2021-05-20 13:18:56 +02:00
Anton Evangelatov
19bd5beb96
update remoteGetSector
2021-05-20 13:15:57 +02:00
Anton Evangelatov
1a9b5760a7
add remoteGetAllocated http handler
2021-05-20 13:14:25 +02:00
Anton Evangelatov
76bb424de0
update StorageMinerAPI
2021-05-20 13:01:14 +02:00
Anton Evangelatov
cb603c62d9
update retrievaladapter ; add piece_provider
2021-05-20 12:49:53 +02:00
Anton Evangelatov
c12d802811
update reader to use ReadAtLeast
2021-05-20 12:38:38 +02:00
Anton Evangelatov
2aad7b6979
update sectorstorage.New (Manager) interface
2021-05-20 12:32:29 +02:00
Łukasz Magiera
e088c71b9a
marketadapter: Handle batch sealing messages
2021-05-19 20:07:20 +02:00
Łukasz Magiera
a5677d1b7a
ffiwrapper: Separate Prover interface
2021-05-19 15:20:23 +02:00
Anton Evangelatov
55401116bb
add UnpaddedByteIndex.Valid()
2021-05-19 13:16:29 +02:00
Anton Evangelatov
e275b54f55
export PartialFile and OpenPartialFile
2021-05-19 13:14:16 +02:00
Anton Evangelatov
a989f60e27
add SectorAddPieceToAny and SectorUnsealPiece to StorageMiner iface; model moved to api package - PieceDealInfo, DealSchedule
2021-05-19 13:05:07 +02:00
Łukasz Magiera
dd393b470f
Fix aggregation inputs
2021-05-18 20:36:47 +02:00
Łukasz Magiera
e400bdf87a
Order proofs before aggregation
2021-05-18 18:58:41 +02:00
Łukasz Magiera
9690bc882c
Test to trigger batching logic
2021-05-18 18:41:42 +02:00
Łukasz Magiera
fe9311e435
update ffi
2021-05-18 13:24:10 +02:00
Łukasz Magiera
6278bdc69a
Make things build
2021-05-17 20:47:41 +02:00
Łukasz Magiera
506f39b294
WIP: Integrate FIP0013
2021-05-11 22:10:29 -04:00
Anton Evangelatov
eb13c74dce
panic on unknown pathType
2021-05-11 18:14:01 +02:00
Anton Evangelatov
e07438417c
consider storiface.PathStorage when calculating storage requirements
2021-05-11 13:19:26 +02:00
whyrusleeping
ac77c51d5e
address nit
2021-04-29 10:24:16 -07:00
whyrusleeping
48feb52cbf
add a test for adding padded pieces
2021-04-27 16:28:25 -07:00
whyrusleeping
6daaf6ac2e
attempt to do better padding on pieces being written into sectors
2021-04-27 11:39:06 -07:00
Łukasz Magiera
a24a73bb52
Merge tag 'v1.8.0' into release/v1.9.0
2021-04-27 10:30:33 +02:00
Łukasz Magiera
c4137a438d
Merge commit 'f1ded63d608e1b191d79c780b8065f8cff367c98' into feat/nv12-1.11
2021-04-27 08:22:55 +02:00
Aayush Rajasekaran
f1ded63d60
v4 specs-actors integration, nv12 migration
2021-04-27 02:02:44 -04:00
Dirk McCormick
61344644a4
feat: add more debug logging for unsealing
2021-04-22 11:33:28 +02:00
yaohcn
844d1d364d
fix health report
2021-04-11 14:47:04 +08:00
Łukasz Magiera
9d4935a736
Merge pull request #5976 from filecoin-project/fix/storage-demote-stat-err
...
localstorage: Demote reservation stat error to debug
2021-04-09 21:08:08 +02:00
Łukasz Magiera
bed5ec81bd
localstorage: Demote reservation stat error to debug
2021-04-06 16:06:47 +02:00
Peter Rabbitson
25a77d905d
Centralize everything on ipfs/go-log/v2
...
I am not entirely sure this is right, but everything seems to build...
2021-04-06 15:04:32 +02:00
Łukasz Magiera
f372964a60
fix lint
2021-04-01 16:40:11 +02:00
Łukasz Magiera
c7de7882a1
Merge branch 'master' of github.com:ppiaas/lotus-ppiaas into fix/stg-fetch-close
2021-04-01 16:38:17 +02:00
yaohcn
3895b2ccca
fix reservations
2021-03-31 14:30:24 +08:00
Steven Allen
b74b1658b2
test: mock proof side effects on verify as well
2021-03-12 09:06:41 -08:00
Łukasz Magiera
d9923125ff
mock sealer: Simulate randomness sideeffects
2021-03-12 17:33:43 +01:00
Łukasz Magiera
cc490b983a
Merge pull request #5761 from filecoin-project/fix/testunpadreader-go116
...
Fix TestUnpadReader on Go 1.16
2021-03-10 13:13:27 +01:00
Łukasz Magiera
7fbb4bdd3b
Fix TestUnpadReader on Go 1.16
2021-03-10 13:05:25 +01:00
Łukasz Magiera
40fdf6c180
Merge pull request #5624 from filecoin-project/feat/miner-storage-limit
...
Configurable storage path storage limit
2021-03-10 11:15:28 +01:00
Łukasz Magiera
fe230f901e
Collect worker task metrics
2021-03-05 12:48:17 +01:00
Łukasz Magiera
35759fa07e
Merge pull request #5635 from filecoin-project/deps/cbg-soft-map
...
Update markets, cbor-gen with soft map decoding
2021-02-22 18:27:38 +01:00
Aayush Rajasekaran
6732758d0f
Tweak mock ProveCommits to respect proof sizes
2021-02-19 17:22:59 -05:00
Łukasz Magiera
26399dba70
Update markets, cbor-gen with soft map decoding
2021-02-19 20:11:43 +01:00
Łukasz Magiera
37b1fb97fb
Configurable storage path storage limit
2021-02-18 16:44:34 +01:00
yaohcn
3f66e731fa
fix drop sector
2021-02-18 12:00:28 +08:00
Łukasz Magiera
67780346e2
storagemgr: Always unseal full sectors
2021-01-25 21:39:15 +01:00
Łukasz Magiera
48c6c55f68
Merge pull request #5384 from filecoin-project/fix-storage-size-estimation
...
fix FileSize to return correct disk usage recursively
2021-01-19 19:22:47 +01:00
Łukasz Magiera
560a127ce6
fsutil: Defensive check for Stat cast
2021-01-19 18:53:37 +01:00
Anton Evangelatov
834a6148b0
use stat.Blocks and not info.Size
2021-01-19 18:07:37 +01:00
Anton Evangelatov
16d07d3f18
fix FileSize to return correct disk usage recursively
2021-01-19 15:21:31 +01:00
Łukasz Magiera
5462ecf6da
gofmt
2021-01-11 16:44:11 +01:00
Łukasz Magiera
9e74c3c4e8
Multicore AddPiece CommP
2021-01-11 16:44:11 +01:00
Łukasz Magiera
2c9f4ad5ad
Merge pull request #5251 from liismn/master
...
Fix deadlock in runWorker in sched_worker.go
2021-01-05 19:35:49 +01:00
Łukasz Magiera
44a56c7e33
Merge pull request #5208 from jsign/jsign/imprreadpiece
...
sector-storage/mock: improve mocked readpiece
2021-01-05 18:37:48 +01:00
15ho
5b235db366
fix: stores.FetchHandler: defer close file and increase buffer size
2020-12-29 10:07:28 +08:00
liismn
e9cc697ee8
Update sched_worker.go
...
Avoid possible deadlock by sh.workersLk
2020-12-25 15:05:01 +08:00
quanzhan lu
d806fafee8
update weight canSeal and canStore when attach
...
update weight canSeal and canStore when attach storage
2020-12-22 19:07:17 +08:00
Ignacio Hagopian
f6cf27c781
improve mocked readpiece
...
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
2020-12-16 10:15:38 -03:00
Łukasz Magiera
46a5013dc1
stores: Simplify remote copy buf size logic
2020-12-10 20:54:42 +01:00
caoao
d4582fb32e
write size according to RegisteredSealProof
2020-12-10 11:59:09 +08:00
beck
d3fa728894
Remove extra spaces
2020-12-09 19:22:21 +08:00
Łukasz Magiera
289ef910a0
fix imports, docsgen
2020-12-02 00:39:55 +01:00
Łukasz Magiera
95eaf13b5a
sectorstorage: Fix tests
2020-12-02 00:36:32 +01:00