Clint Armstrong
e2a1ca7caa
Use cgroup limits in worker memory calculations
...
Worker processes may have memory limitations imposed by Systemd. But
/proc/meminfo shows the entire system memory regardless of these limits.
This results in the scheduler believing the worker has the entire system
memory avaliable and the worker being allocated too many tasks.
This change attempts to read cgroup memory limits for the worker
process. It supports cgroups v1 and v2, and compares cgroup limits
against the system memory and returns the most conservative values to
prevent the worker from being allocated too many tasks and potentially
triggering an OOM event.
2021-11-30 02:06:58 +01:00
Łukasz Magiera
19e808fffb
Merge pull request #7710 from filecoin-project/feat/pc2-validation
...
ffiwrapper: Validate PC2 by calling C1 with random seeds
2021-11-30 02:05:32 +01:00
Łukasz Magiera
d21c44e266
ffiwrapper: Validate PC2 by calling C1 with random seeds
2021-11-30 01:33:05 +01:00
Jiaying Wang
01215b7d2c
Merge pull request #7709 from filecoin-project/logic-error-miner-entrypoint
...
fix logic error
2021-11-29 18:16:58 -05:00
Łukasz Magiera
da9b0c6735
Merge pull request #7708 from filecoin-project/feat/update-graphsync-v0.10.6
...
Update go-graphsync v0.10.6
2021-11-29 23:51:32 +01:00
Jiaying Wang
4d2f3375e8
Merge pull request #7699 from filecoin-project/feat/listcids-verbose
...
Add verbose mode to lotus-miner pieces list-cids
2021-11-29 17:43:09 -05:00
Cory Schwartz
27efbc3485
fix logic error
2021-11-29 13:52:06 -08:00
hannahhoward
d4074e45cf
feat(deps): update go-graphsync v0.10.6
2021-11-29 13:29:12 -08:00
Łukasz Magiera
f39942283b
Merge pull request #7706 from filecoin-project/feat/ret-matchding-cars
...
retrieval: Only output matching nodes, MatchPath dagspec
2021-11-29 22:14:25 +01:00
Łukasz Magiera
468b90a4a4
cli docsgen
2021-11-29 21:50:23 +01:00
Łukasz Magiera
092e12d1be
cli: boolean logic is hard
2021-11-29 21:41:38 +01:00
Łukasz Magiera
320b36495d
gofmt
2021-11-29 21:39:27 +01:00
Łukasz Magiera
e4f47de6ef
retrieval: Check required flags for --car-export-merkle-proof
2021-11-29 21:39:18 +01:00
Łukasz Magiera
410ecb4bbc
retrieval: --car-export-merkle-proof flag for client retrieve
2021-11-29 21:37:28 +01:00
Łukasz Magiera
d019853687
review: Cleanup some comments
...
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
2021-11-29 21:29:00 +01:00
Łukasz Magiera
4d51980cb5
deps: Use tagged go-ipld-selector-text-lite
2021-11-29 21:22:30 +01:00
Łukasz Magiera
5b5e6b9e44
retrieval: DagSpec.MatchPath -> ExportMerkleProof
2021-11-29 21:14:00 +01:00
Łukasz Magiera
58a084049d
retrieval: Fix traversal in ls
2021-11-29 21:08:53 +01:00
Łukasz Magiera
9538fc9723
mod tidy, docsgen
2021-11-29 20:56:40 +01:00
Łukasz Magiera
227188e908
retrieval: Test non-matching path traversal
2021-11-29 20:52:55 +01:00
Łukasz Magiera
61791b90ea
retrieval: Only output matching nodes, MatchPath dagspec
2021-11-29 20:40:55 +01:00
Steven Allen
797147097c
Merge pull request #7689 from filecoin-project/disable-mplex
...
disable mplex stream muxer
2021-11-29 09:46:24 -08:00
Łukasz Magiera
f8b132890c
Add verbose mode to lotus-miner pieces list-cids
2021-11-29 17:19:47 +01:00
Łukasz Magiera
134aee4582
mod tidy
2021-11-29 16:44:58 +01:00
Łukasz Magiera
77d75b7739
Merge pull request #7688 from filecoin-project/chore/partret_cleanup_comment_and_flow
...
Cleanup partial retrieval codepaths ( zero functional changes )
2021-11-29 16:39:55 +01:00
Łukasz Magiera
26c3752f48
Merge pull request #7693 from filecoin-project/feat/randacc-dagstore-mount
...
Make small retrieval 200x faster
2021-11-29 16:10:18 +01:00
Łukasz Magiera
4bcde2f0ff
dagstore pieceReader: Cleanup reader nil check
2021-11-29 15:32:27 +01:00
Jiaying Wang
160bb0b050
Merge pull request #7698 from filecoin-project/jen/releasetomaster
...
Releases back to master
2021-11-26 18:44:18 -05:00
Łukasz Magiera
05aa860459
Request correct read size with startOffset in pieceProvider
2021-11-27 00:05:45 +01:00
Jiaying Wang
a4c2a20851
Merge pull request #7691 from filecoin-project/misc/rle-dump
...
Add RLE dump code
2021-11-26 17:15:52 -05:00
Jennifer Wang
4df9179d34
Merge branch 'releases' into jen/releasetomaster
2021-11-26 17:09:22 -05:00
Jiaying Wang
8943de27e4
Merge pull request #7695 from filecoin-project/release/v1.13.1
...
Release/v1.13.1
2021-11-26 16:59:54 -05:00
Jiaying Wang
fc728d9e11
Merge pull request #7694 from filecoin-project/jen/v1.13.1-prep
...
v1.13.1 prep
2021-11-26 16:44:46 -05:00
Jennifer Wang
f6131ce3b5
v1.13.1 prep
2021-11-26 16:33:39 -05:00
Łukasz Magiera
9110e6f632
dagstore pieceReader: add debug log on stream restart
2021-11-26 20:24:51 +01:00
Łukasz Magiera
331702cd95
Tweak MaxPieceReaderBurnBytes
2021-11-26 18:49:41 +01:00
Łukasz Magiera
743ce5a40f
Add startOffset support to mock SectorMgr.ReadPiece
2021-11-26 18:48:52 +01:00
Łukasz Magiera
abec7dd3bc
Fix imports
2021-11-26 18:40:27 +01:00
Łukasz Magiera
f6de16e95a
Fix sector-storage tests
2021-11-26 18:16:53 +01:00
Łukasz Magiera
a9ee263682
Fix dagstore pieceReader burn logic
2021-11-26 18:01:09 +01:00
Łukasz Magiera
8d955d5f30
dagstore mount: Add random access support
2021-11-26 17:40:53 +01:00
Jakub Sztandera
4d8be81a8f
Add usage
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-11-26 15:01:05 +01:00
Jakub Sztandera
e3c7b8d006
Fix typo
...
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
2021-11-26 14:42:18 +01:00
Łukasz Magiera
3b1d86b750
Merge pull request #7690 from filecoin-project/misc/updage-archive-script
...
Update archive script
2021-11-26 13:47:40 +01:00
Marten Seemann
cf0faf58dc
remove muxer config via LIBP2P_MUX_PREFS env
2021-11-26 11:01:54 +04:00
Jakub Sztandera
af113f867d
Add dump code
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-11-25 20:53:49 +01:00
Jakub Sztandera
ca230bba09
Add releases to protected
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-11-25 20:19:20 +01:00
Jakub Sztandera
a26a2160ae
Fix the script
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-11-25 20:14:19 +01:00
Jakub Sztandera
aafdf3d093
Update archive script
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-11-25 19:52:04 +01:00
Marten Seemann
12dc3baff9
disable mplex stream muxer
2021-11-25 09:38:57 -08:00
