cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
14,325 Commits 24 Branches 342 Tags 314 MiB
33134e617f
Commit Graph

14325 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Clint Armstrong
c4f46171ae Report memory used and swap used in worker res 
Attempting to report "memory used by other processes" in the MemReserved
field fails to take into account the fact that the system's memory used
includes memory used by ongoing tasks.

To properly account for this, worker should report the memory and swap
used, then the scheduler that is aware of the memory requirements for a
task can determine if there is sufficient memory available for a task.
 2021-11-30 02:06:58 +01:00
Clint Armstrong
e2a1ca7caa Use cgroup limits in worker memory calculations 
Worker processes may have memory limitations imposed by Systemd. But
/proc/meminfo shows the entire system memory regardless of these limits.
This results in the scheduler believing the worker has the entire system
memory avaliable and the worker being allocated too many tasks.

This change attempts to read cgroup memory limits for the worker
process. It supports cgroups v1 and v2, and compares cgroup limits
against the system memory and returns the most conservative values to
prevent the worker from being allocated too many tasks and potentially
triggering an OOM event.
 2021-11-30 02:06:58 +01:00
Łukasz Magiera
19e808fffb
Merge pull request #7710 from filecoin-project/feat/pc2-validation 
ffiwrapper: Validate PC2 by calling C1 with random seeds
 2021-11-30 02:05:32 +01:00
Łukasz Magiera
d21c44e266 ffiwrapper: Validate PC2 by calling C1 with random seeds 2021-11-30 01:33:05 +01:00
Jiaying Wang
01215b7d2c
Merge pull request #7709 from filecoin-project/logic-error-miner-entrypoint 
fix logic error
 2021-11-29 18:16:58 -05:00
Łukasz Magiera
da9b0c6735
Merge pull request #7708 from filecoin-project/feat/update-graphsync-v0.10.6 
Update go-graphsync v0.10.6
 2021-11-29 23:51:32 +01:00
Jiaying Wang
4d2f3375e8
Merge pull request #7699 from filecoin-project/feat/listcids-verbose 
Add verbose mode to lotus-miner pieces list-cids
 2021-11-29 17:43:09 -05:00
Cory Schwartz
27efbc3485 fix logic error 2021-11-29 13:52:06 -08:00
hannahhoward
d4074e45cf feat(deps): update go-graphsync v0.10.6 2021-11-29 13:29:12 -08:00
Łukasz Magiera
f39942283b
Merge pull request #7706 from filecoin-project/feat/ret-matchding-cars 
retrieval: Only output matching nodes, MatchPath dagspec
 2021-11-29 22:14:25 +01:00
Łukasz Magiera
468b90a4a4 cli docsgen 2021-11-29 21:50:23 +01:00
Łukasz Magiera
092e12d1be cli: boolean logic is hard 2021-11-29 21:41:38 +01:00
Łukasz Magiera
320b36495d gofmt 2021-11-29 21:39:27 +01:00
Łukasz Magiera
e4f47de6ef retrieval: Check required flags for --car-export-merkle-proof 2021-11-29 21:39:18 +01:00
Łukasz Magiera
410ecb4bbc retrieval: --car-export-merkle-proof flag for client retrieve 2021-11-29 21:37:28 +01:00
Łukasz Magiera
d019853687
review: Cleanup some comments 
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
 2021-11-29 21:29:00 +01:00
Łukasz Magiera
4d51980cb5 deps: Use tagged go-ipld-selector-text-lite 2021-11-29 21:22:30 +01:00
Łukasz Magiera
5b5e6b9e44 retrieval: DagSpec.MatchPath -> ExportMerkleProof 2021-11-29 21:14:00 +01:00
Łukasz Magiera
58a084049d retrieval: Fix traversal in ls 2021-11-29 21:08:53 +01:00
Łukasz Magiera
9538fc9723 mod tidy, docsgen 2021-11-29 20:56:40 +01:00
Łukasz Magiera
227188e908 retrieval: Test non-matching path traversal 2021-11-29 20:52:55 +01:00
Łukasz Magiera
61791b90ea retrieval: Only output matching nodes, MatchPath dagspec 2021-11-29 20:40:55 +01:00
Steven Allen
797147097c
Merge pull request #7689 from filecoin-project/disable-mplex 
disable mplex stream muxer
 2021-11-29 09:46:24 -08:00
Łukasz Magiera
f8b132890c Add verbose mode to lotus-miner pieces list-cids 2021-11-29 17:19:47 +01:00
Łukasz Magiera
134aee4582 mod tidy 2021-11-29 16:44:58 +01:00
Łukasz Magiera
77d75b7739
Merge pull request #7688 from filecoin-project/chore/partret_cleanup_comment_and_flow 
Cleanup partial retrieval codepaths ( zero functional changes )
 2021-11-29 16:39:55 +01:00
Łukasz Magiera
26c3752f48
Merge pull request #7693 from filecoin-project/feat/randacc-dagstore-mount 
Make small retrieval 200x faster
 2021-11-29 16:10:18 +01:00
Łukasz Magiera
4bcde2f0ff dagstore pieceReader: Cleanup reader nil check 2021-11-29 15:32:27 +01:00
Jiaying Wang
160bb0b050
Merge pull request #7698 from filecoin-project/jen/releasetomaster 
Releases back to master
 2021-11-26 18:44:18 -05:00
Łukasz Magiera
05aa860459 Request correct read size with startOffset in pieceProvider 2021-11-27 00:05:45 +01:00
Jiaying Wang
a4c2a20851
Merge pull request #7691 from filecoin-project/misc/rle-dump 
Add RLE dump code
 2021-11-26 17:15:52 -05:00
Jennifer Wang
4df9179d34 Merge branch 'releases' into jen/releasetomaster 2021-11-26 17:09:22 -05:00
Jiaying Wang
8943de27e4
Merge pull request #7695 from filecoin-project/release/v1.13.1 
Release/v1.13.1
 2021-11-26 16:59:54 -05:00
Jiaying Wang
fc728d9e11
Merge pull request #7694 from filecoin-project/jen/v1.13.1-prep 
v1.13.1 prep
 2021-11-26 16:44:46 -05:00
Jennifer Wang
f6131ce3b5 v1.13.1 prep 2021-11-26 16:33:39 -05:00
Łukasz Magiera
9110e6f632 dagstore pieceReader: add debug log on stream restart 2021-11-26 20:24:51 +01:00
Łukasz Magiera
331702cd95 Tweak MaxPieceReaderBurnBytes 2021-11-26 18:49:41 +01:00
Łukasz Magiera
743ce5a40f Add startOffset support to mock SectorMgr.ReadPiece 2021-11-26 18:48:52 +01:00
Łukasz Magiera
abec7dd3bc Fix imports 2021-11-26 18:40:27 +01:00
Łukasz Magiera
f6de16e95a Fix sector-storage tests 2021-11-26 18:16:53 +01:00
Łukasz Magiera
a9ee263682 Fix dagstore pieceReader burn logic 2021-11-26 18:01:09 +01:00
Łukasz Magiera
8d955d5f30 dagstore mount: Add random access support 2021-11-26 17:40:53 +01:00
Jakub Sztandera
4d8be81a8f
Add usage 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-11-26 15:01:05 +01:00
Jakub Sztandera
e3c7b8d006
Fix typo 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2021-11-26 14:42:18 +01:00
Łukasz Magiera
3b1d86b750
Merge pull request #7690 from filecoin-project/misc/updage-archive-script 
Update archive script
 2021-11-26 13:47:40 +01:00
Marten Seemann
cf0faf58dc remove muxer config via LIBP2P_MUX_PREFS env 2021-11-26 11:01:54 +04:00
Jakub Sztandera
af113f867d
Add dump code 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-11-25 20:53:49 +01:00
Jakub Sztandera
ca230bba09
Add releases to protected 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-11-25 20:19:20 +01:00
Jakub Sztandera
a26a2160ae
Fix the script 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-11-25 20:14:19 +01:00
Jakub Sztandera
aafdf3d093
Update archive script 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-11-25 19:52:04 +01:00