OutOfMemoryError: Unable to create new native thread on EAP. Half of available heap occupied by org.hornetq.core.client.impl.ClientSessionFactoryImpl instances

Solution Unverified - Updated -

Issue

We received OutOfMemoryError: Unable to create new native thread on EAP 6.0.1

Memory Analyzer Tool (MAT) leak suspect report shows:

264,219 instances of "org.hornetq.core.client.impl.ClientSessionFactoryImpl", loaded by "org.jboss.modules.ModuleClassLoader @ 0xe196b970" occupy 272,770,344 (51.75%) bytes. These instances are referenced from one instance of "java.util.HashMap$Entry[]", loaded by "<system class loader>"

These all seem to be held in memory by a single thread:

Thread-27 (HornetQ-server-HornetQServerImpl::serverUUID=3a863084-f2f8-11e2-a4a4-d1e05e73dcf8-887675906)
  at org.hornetq.utils.UTF8Util$StringUtilBuffer.<init>(II)V (UTF8Util.java:267)
  at org.hornetq.utils.UTF8Util$StringUtilBuffer.<init>()V (UTF8Util.java:261)
  at org.hornetq.utils.UTF8Util.getThreadLocalBuffer()Lorg/hornetq/utils/UTF8Util$StringUtilBuffer; (UTF8Util.java:186)
  at org.hornetq.utils.UTF8Util.saveUTF(Lorg/hornetq/api/core/HornetQBuffer;Ljava/lang/String;)V (UTF8Util.java:45)
  at org.hornetq.core.buffers.impl.ChannelBufferWrapper.writeUTF(Ljava/lang/String;)V (ChannelBufferWrapper.java:197)
  at org.hornetq.core.buffers.impl.ChannelBufferWrapper.writeStringInternal(Ljava/lang/String;)V (ChannelBufferWrapper.java:186)
  at org.hornetq.core.buffers.impl.ChannelBufferWrapper.writeString(Ljava/lang/String;)V (ChannelBufferWrapper.java:166)
  at org.hornetq.core.protocol.core.impl.wireformat.NodeAnnounceMessage.encodeRest(Lorg/hornetq/api/core/HornetQBuffer;)V (NodeAnnounceMessage.java:101)
  at org.hornetq.core.protocol.core.impl.PacketImpl.encode(Lorg/hornetq/spi/core/protocol/RemotingConnection;)Lorg/hornetq/api/core/HornetQBuffer; (PacketImpl.java:242)
  at org.hornetq.core.protocol.core.impl.ChannelImpl.send(Lorg/hornetq/core/protocol/core/Packet;ZZ)V (ChannelImpl.java:183)
  at org.hornetq.core.protocol.core.impl.ChannelImpl.send(Lorg/hornetq/core/protocol/core/Packet;)V (ChannelImpl.java:157)
  at org.hornetq.core.client.impl.ClientSessionFactoryImpl.sendNodeAnnounce(JLjava/lang/String;ZLorg/hornetq/api/core/TransportConfiguration;Lorg/hornetq/api/core/TransportConfiguration;)V (ClientSessionFactoryImpl.java:1342)
  at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl.onConnection(Lorg/hornetq/core/client/impl/ClientSessionFactoryInternal;)V (ClusterConnectionImpl.java:567)
  at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnection()Lorg/hornetq/core/protocol/core/CoreRemotingConnection; (ClientSessionFactoryImpl.java:1317)
  at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(I)V (ClientSessionFactoryImpl.java:994)
  at org.hornetq.core.client.impl.ClientSessionFactoryImpl.connect(IZ)V (ClientSessionFactoryImpl.java:225)
  at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(Lorg/hornetq/api/core/TransportConfiguration;)Lorg/hornetq/api/core/client/ClientSessionFactory; (ServerLocatorImpl.java:669)
  at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(Ljava/lang/String;)Lorg/hornetq/api/core/client/ClientSessionFactory; (ServerLocatorImpl.java:625)
  at org.hornetq.core.server.cluster.impl.ClusterConnectionBridge.createSessionFactory()Lorg/hornetq/core/client/impl/ClientSessionFactoryInternal; (ClusterConnectionBridge.java:152)
  at org.hornetq.core.server.cluster.impl.BridgeImpl.connect()V (BridgeImpl.java:792)
  at org.hornetq.core.server.cluster.impl.BridgeImpl$ConnectRunnable.run()V (BridgeImpl.java:1091)
  at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run()V (OrderedExecutorFactory.java:100)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:615)
  at java.lang.Thread.run()V (Thread.java:724)

Environment

Red Hat JBoss Enterprise Application Platform (EAP) 6.0.1

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content