专栏首页云计算与大数据k8s Pod unable to create new native thread

k8s Pod unable to create new native thread

java.lang.OutOfMemoryError: unable to create new native thread

Exception in thread "ActiveMQ Task-551" java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949) at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1017) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1163) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

ulimit -a

core file size (blocks, -c) 0

data seg size (kbytes, -d) unlimited

scheduling priority (-e) 0

file size (blocks, -f) unlimited

pending signals (-i) 1029315

max locked memory (kbytes, -l) 64

max memory size (kbytes, -m) unlimited

open files (-n) 65535

pipe size (512 bytes, -p) 8

POSIX message queues (bytes, -q) 819200

real-time priority (-r) 0

stack size (kbytes, -s) 8192

cpu time (seconds, -t) unlimited

max user processes (-u) 40960

virtual memory (kbytes, -v) unlimited

file locks (-x) unlimited

Understanding /etc/security/limits.conf file

https://www.thegeekdiary.com/understanding-etc-security-limits-conf-file-to-set-ulimit/

[domain] can be:

  • an user name
  • a group name, with @group syntax
  • the wildcard *, for default entry
  • the wildcard %, can be also used with %group syntax, for maxlogin limit

[type] can have the two values:

  • “soft” for enforcing the soft limits
  • “hard” for enforcing hard limits

[item] can be one of the following:

  • core – limits the core file size (KB)
  • data – max data size (KB)
  • fsize – maximum filesize (KB)
  • memlock – max locked-in-memory address space (KB)
  • nofile – max number of open files
  • rss – max resident set size (KB)
  • stack – max stack size (KB)
  • cpu – max CPU time (MIN)
  • nproc – max number of processes
  • as – address space limit (KB)
  • maxlogins – max number of logins for this user
  • maxsyslogins – max number of logins on the system
  • priority – the priority to run user process with
  • locks – max number of file locks the user can hold
  • sigpending – max number of pending signals
  • msgqueue – max memory used by POSIX message queues (bytes)
  • nice – max nice priority allowed to raise to values: [-20, 19]
  • rtprio – max realtime priority
man limits.conf

修改docker 容器pids-limit限制

/etc/sysconfig/docker

# Modify these options if you want to change the way the docker daemon runs

OPTIONS=' --selinux-enabled --selinux-enabled --insecure-registry 10.161.0.0/16 --log-driver json-file --log-opt max-size=50M --log-opt max-file=3 --bip=10.41.199.1/24 --insecure-registry harbor.cloud.bz --add-registry harbor.cloud.bz --signature-verification=False --default-pids-limit=40960'

if [ -z "${DOCKER_CERT_PATH}" ]; then

DOCKER_CERT_PATH=/etc/docker

fi

# Do not add registries in this file anymore. Use /etc/containers/registries.conf

# instead. For more information reference the registries.conf(5) man page.

# Location used for temporary files, such as those created by

# docker load and build operations. Default is /var/lib/docker/tmp

# Can be overriden by setting the following environment variable.

# DOCKER_TMPDIR=/var/tmp

# Controls the /etc/cron.daily/docker-logrotate cron job status.

# To disable, uncomment the line below.

# LOGROTATE=false

# docker-latest daemon can be used by starting the docker-latest unitfile.

# To use docker-latest client, uncomment below lines

#DOCKERBINARY=/usr/bin/docker-latest

#DOCKERDBINARY=/usr/bin/dockerd-latest

#DOCKER_CONTAINERD_BINARY=/usr/bin/docker-containerd-latest

#DOCKER_CONTAINERD_SHIM_BINARY=/usr/bin/docker-containerd-shim-latest

ADD_REGISTRY='--add-registry registry.redhat.io'

How to set nproc (Hard and Soft) Values in CentOS / RHEL 5,6,7

https://www.thegeekdiary.com/how-to-set-nproc-hard-and-soft-values-in-centos-rhel-567/

How to Increase Number of Open Files Limit in Linux

https://www.tecmint.com/increase-set-open-file-limits-in-linux/

Find Linux Open File Limit

The value is stored in:

# cat /proc/sys/fs/file-max

818354

Check Hard Limit in Linux

# ulimit -Hn

4096

Check Soft Limits in Linux

# ulimit -Sn

1024

To see the hard and soft values for different users, you can simply switch user with “su” to the user which limits you want to check.

For example:

# su marin
$ ulimit -Sn

1024
$ ulimit -Hn

4096

How to Check System wide File Descriptors Limits in Linux

If you are running a server, some of your applications may require higher limits for opened file descriptors. A good example for such are MySQL/MariaDB services or Apache web server.

You can increase the limit of opened files in Linux by editing the kernel directive fs.file-max. For that purpose, you can use the sysctl utility.

Sysctl is used to configure kernel parameters at runtime.

For example, to increase open file limit to 500000, you can use the following command as root:

# sysctl -w fs.file-max=500000

You can check the current value for opened files with the following command:

$ cat /proc/sys/fs/file-max

With the above command the changes you have made will only remain active until the next reboot. If you wish to apply them permanently, you will have to edit the following file:

# vi /etc/sysctl.conf

Add the following line:

fs.file-max=500000

Of course, you can change the number per your needs. To verify the changes again use:

# cat /proc/sys/fs/file-max

Users will need to logout and login again for the changes to take effect. If you want to apply the limit immediately, you can use the following command:

# sysctl -p

Set User Level Open File limits in Linux

The above examples, showed how to set global limits, but you may want to apply limits per user basis. For that purpose, as user root, you will need to edit the following file:

# vi /etc/security/limits.conf

If you are a Linux administrator, I suggest you that you become very familiar with that file and what you can do to it. Read all of the comments in it as it provides great flexibility in terms of managing system resources by limiting users/groups on different levels.

The lines that you should add take the following parameters:

<domain>        <type>  <item>  <value>

Here is an example of setting a soft and hard limits for user marin:

## Example hard limit for max opened files
marin        hard nofile 4096
## Example soft limit for max opened files
marin        soft nofile 1024

本文分享自微信公众号 - 黑洞日志(heidcloud)

原文出处及转载信息见文内详细说明,如有侵权,请联系 yunjia_community@tencent.com 删除。

原始发表时间:2019-05-14

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • 1.25 VR扫描:GDC 2019调查报告公布;《Titanic VR》开发商宣布去年营收达70万欧元

    GDC 2019大会将在3月举行,举办方UBM于近日公布了其最新版《游戏行业调查报告》。在被调查的4000多名游戏开发者中,17%的开发者表示其正在为VR头显开...

    VRPinea
  • yum的工作原理及使用

    每次yum install或者search的时候,都会去解析/etc/yum.repos.d/下面所有以.repo结尾的配置文件,获取yum服务器地址。yum会...

    十毛
  • 2.15 VR扫描:Dirac Research获1320万英镑投资;乐高推“AR闹鬼”建筑积木Hidden Side

    近日,瑞典数字声音解决方案专家Dirac Research宣布完成1320万英镑的融资。Dirac Research是一家不断致力于改进音频技术的公司,其业务范...

    VRPinea
  • 「docker实战篇」python的docker-docker系统管理-基本命令(29)

    PS:docker的基本命令很重要,但是如果会使用help这个功能官方介绍的更清楚,我这里也是把英文做了翻译。主要还是理解原理。

    IT故事会
  • 0579-5.15.1-Java 应用程序中修改Kerberos ticket_lifetime参数无效异常分析

    在Kerberos环境中,我们的应用程序通过Java代码来提交任务需要先进行Kerberos凭证的初始化然后进行应用程序的提交,本文档主要讲述Java应用程序中...

    Fayson
  • 看国外媒体对NVIDIA Jetson Nano的评测报告

    由于目前Jetson Nano还在路上,没有拿到实物,所以我们一起来看看他都说了些啥,来解解渴吧!

    GPUS Lady
  • 以最简单方式学习Linux

    有很多关于Linux的书籍,博客。大多数都会比较“粗暴“的将一大堆的命令塞给读者,从而使很多.NET程序员望而却步。未入其门就路过了。

    马哥linux运维
  • 手把手教你在NVIDIA Jetson Xavier上安装Deepstream 3.0!

    今天栏主向大家介绍了如何在NVIDIA Jetson Xavier上安装Deepstream 3.0.

    GPUS Lady
  • 网络通信基础重难点解析 12 :Linux epoll 模型

    综合 select 和 poll 的一些优缺点,Linux 从内核 2.6 版本开始引入了更高效的 epoll 模型,本节我们来详细介绍 epoll 模型。

    范蠡
  • 配置vlmcsd,让你的Linux服务器瞬间变成KMS服务器

    注意:请尽可能避免在海外服务器上部署KMS服务器(尤其是美国本土的服务器)!这是一种盗版行为,可能会因为微软的律师函or服务器被封停!这不是开玩笑!

    傲云

扫码关注云+社区

领取腾讯云代金券