Unable to create more than 1024 Threads in the container.

Latest response

Recently we were getting the following exception in one of our Containers which was running a Java application. This container used to run perfectly on Openshift 3.11.

Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:717

As seen below, when you execute the aforementioned code in Openshift 3.11, it used to create 4000+ Threads. But in Openshift 4.2 it just created around 1000 Threads.

Openshift 3.11
ocp3.11

Openshift 4.2
ocp4.2

I also have made sure there is enough memory for the containers and ulimits are properly set within the container. Please refer below,
limits

Note: I used Openshift Playground for the above Tests.

What has changed in OCP 4.2 and how can I increase this limit? Thanks in advance.

Responses

I believe it's because this cgroup is in play:

$ cat /sys/fs/cgroup/pids/pids.max 
1024

while running your Java app, you can watch /sys/fs/cgroup/pids/pids.current until it approaches 1024 at which point your app bails out.

Now, I too wonder where in OpenShift this can be configured.