EJB standalone client invocation timeout hangs server threads in JBoss EAP 6
Issue
Steps to reproduce:
- Deploy an EJB on EAP 6.1 (same behavior is in EAP 6.2 as well)
- Execute the client
- The client makes a remote EJB call to the server, which doesn't respond within $invocation_timeout and causes the client to receive the following exception:
SearchInputData trackId=ef413ca5-5c67-4e85-8270-919be90df29f
java.util.concurrent.TimeoutException: No invocation response received in 1000 milliseconds
at org.jboss.ejb.client.EJBClientInvocationContext$InvocationTimeoutResultProducer.getResult(EJBClientInvocationContext.java:698)
at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:272)
at org.jboss.ejb.client.EJBHomeCreateInterceptor.handleInvocationResult(EJBHomeCreateInterceptor.java:84)
at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:274)
- After that, the client waits 60 seconds and makes the same call again, resulting in another error. These exceptions are purposeful to show the issue on the server.
- While this is happening, the server is receiving the request and doing some things on the backend.
- After the client exception occurs, the client thread goes away, but the server side thread is seemingly stuck in the following stack indefinitely (thought I only waited like 20 minutes):
"EJB default - 7" prio=10 tid=0x00007fcaa4131800 nid=0x7296 in Object.wait() [0x00007fcae59d3000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000f8098040> (a org.xnio.streams.BufferPipeOutputStream)
at java.lang.Object.wait(Object.java:503)
at org.jboss.remoting3.remote.OutboundMessage$1.accept(OutboundMessage.java:106)
at org.xnio.streams.BufferPipeOutputStream.send(BufferPipeOutputStream.java:126)
at org.xnio.streams.BufferPipeOutputStream.send(BufferPipeOutputStream.java:114)
at org.xnio.streams.BufferPipeOutputStream.getBuffer(BufferPipeOutputStream.java:78)
at org.xnio.streams.BufferPipeOutputStream.write(BufferPipeOutputStream.java:96)
- locked <0x00000000f8098040> (a org.xnio.streams.BufferPipeOutputStream)
at org.jboss.remoting3.remote.OutboundMessage.write(OutboundMessage.java:234)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
- locked <0x00000000f809a0c8> (a java.io.DataOutputStream)
at org.jboss.as.ejb3.remote.protocol.AbstractMessageHandler$1.write(AbstractMessageHandler.java:217)
at org.jboss.marshalling.OutputStreamByteOutput.write(OutputStreamByteOutput.java:56)
at org.jboss.marshalling.UTFUtils.writeUTFBytes(UTFUtils.java:135)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:278)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:682)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:598)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1066)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1022)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:888)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1066)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1022)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:888)
at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62)
at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:115)
at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.writeMethodInvocationResponse(MethodInvocationMessageHandler.java:372)
at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.access$600(MethodInvocationMessageHandler.java:70)
at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler$1.run(MethodInvocationMessageHandler.java:246)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at org.jboss.threads.JBossThread.run(JBossThread.java:122)
- That stack trace occurs on every EJB default thread that gets created to service the client.
Environment
- Red Hat JBoss Enterprise Application Platform (EAP)
- 6.1.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.