EJB standalone client invocation timeout hangs server threads in JBoss EAP 6

Solution Verified - Updated -

Issue

Steps to reproduce:

  • Deploy an EJB on EAP 6.1 (same behavior is in EAP 6.2 as well)
  • Execute the client
  • The client makes a remote EJB call to the server, which doesn't respond within $invocation_timeout and causes the client to receive the following exception:
    SearchInputData trackId=ef413ca5-5c67-4e85-8270-919be90df29f
java.util.concurrent.TimeoutException: No invocation response received in 1000 milliseconds
    at org.jboss.ejb.client.EJBClientInvocationContext$InvocationTimeoutResultProducer.getResult(EJBClientInvocationContext.java:698)
    at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:272)
    at org.jboss.ejb.client.EJBHomeCreateInterceptor.handleInvocationResult(EJBHomeCreateInterceptor.java:84)
    at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:274)
  • After that, the client waits 60 seconds and makes the same call again, resulting in another error. These exceptions are purposeful to show the issue on the server.
  • While this is happening, the server is receiving the request and doing some things on the backend.
  • After the client exception occurs, the client thread goes away, but the server side thread is seemingly stuck in the following stack indefinitely (thought I only waited like 20 minutes):
    "EJB default - 7" prio=10 tid=0x00007fcaa4131800 nid=0x7296 in Object.wait() [0x00007fcae59d3000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00000000f8098040> (a org.xnio.streams.BufferPipeOutputStream)
    at java.lang.Object.wait(Object.java:503)
    at org.jboss.remoting3.remote.OutboundMessage$1.accept(OutboundMessage.java:106)
    at org.xnio.streams.BufferPipeOutputStream.send(BufferPipeOutputStream.java:126)
    at org.xnio.streams.BufferPipeOutputStream.send(BufferPipeOutputStream.java:114)
    at org.xnio.streams.BufferPipeOutputStream.getBuffer(BufferPipeOutputStream.java:78)
    at org.xnio.streams.BufferPipeOutputStream.write(BufferPipeOutputStream.java:96)
    - locked <0x00000000f8098040> (a org.xnio.streams.BufferPipeOutputStream)
    at org.jboss.remoting3.remote.OutboundMessage.write(OutboundMessage.java:234)
    at java.io.DataOutputStream.write(DataOutputStream.java:107)
    - locked <0x00000000f809a0c8> (a java.io.DataOutputStream)
    at org.jboss.as.ejb3.remote.protocol.AbstractMessageHandler$1.write(AbstractMessageHandler.java:217)
    at org.jboss.marshalling.OutputStreamByteOutput.write(OutputStreamByteOutput.java:56)
    at org.jboss.marshalling.UTFUtils.writeUTFBytes(UTFUtils.java:135)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:278)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:682)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:598)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1066)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1022)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:888)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1066)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1022)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:888)
    at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62)
    at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:115)
    at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.writeMethodInvocationResponse(MethodInvocationMessageHandler.java:372)
    at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.access$600(MethodInvocationMessageHandler.java:70)
    at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler$1.run(MethodInvocationMessageHandler.java:246)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)
  • That stack trace occurs on every EJB default thread that gets created to service the client.

Environment

  • Red Hat JBoss Enterprise Application Platform (EAP)
    • 6.1.0

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content