JBoss shutdown hangs to be caused by deadlock between "JBoss Shutdown Hook" and JBoss Messaging thread
Issue
-
In EAP 4.3, JBoss shutdown hangs to be caused by deadlock between "JBoss Shutdown Hook" and JBoss Messaging thread "WorkerThread":
"JBoss Shutdown Hook" daemon prio=1 tid=0x0a023258 nid=0x2a38 in Object.wait() [0x826f8000..0x826fa130] at java.lang.Object.wait(Native Method) - waiting on <0x91593088> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock) at java.lang.Object.wait(Object.java:474) at EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240) - locked <0x91593088> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock) at org.jboss.jms.server.remoting.JMSServerInvocationHandler.setClosed(JMSServerInvocationHandler.java:68) at org.jboss.jms.server.ServerPeer.stopService(ServerPeer.java:349) - locked <0x914cd0a0> (a org.jboss.jms.server.ServerPeer) at org.jboss.system.ServiceMBeanSupport.jbossInternalStop(ServiceMBeanSupport.java:315) at org.jboss.system.ServiceMBeanSupport.stop(ServiceMBeanSupport.java:206) at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155) at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94) at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133) at org.jboss.mx.server.Invocation.invoke(Invocation.java:88) at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:142) at org.jboss.mx.server.Invocation.invoke(Invocation.java:88) at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264) at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659) at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:995) at $Proxy0.stop(Unknown Source) at org.jboss.system.ServiceController.stop(ServiceController.java:508) at org.jboss.system.ServiceController.stop(ServiceController.java:499) ...(snip)... ...(snip)... "WorkerThread#12[10.133.46.128:50365]" prio=1 tid=0x0badb610 nid=0x1077 waiting for monitor entry [0x85657000..0x85657eb0] at org.jboss.jms.server.ServerPeer.getDefaultDLQInstance(ServerPeer.java:1187) - waiting to lock <0x914cd0a0> (a org.jboss.jms.server.ServerPeer) at org.jboss.jms.server.endpoint.ServerSessionEndpoint.<init>(ServerSessionEndpoint.java:220) at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.createSessionDelegate(ServerConnectionEndpoint.java:249) at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$createSessionDelegate$aop(ConnectionAdvised.java:75) at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$createSessionDelegate_6052335267724906805.invokeNext(ConnectionAdvised$createSessionDelegate_6052335267724906805.java) at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105) at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$createSessionDelegate_6052335267724906805.invokeNext(ConnectionAdvised$createSessionDelegate_6052335267724906805.java) at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.createSessionDelegate(ConnectionAdvised.java) at org.jboss.jms.wireformat.ConnectionCreateSessionDelegateRequest.serverInvoke(ConnectionCreateSessionDelegateRequest.java:87) at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143) at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:866) at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:608) at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420) at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
-
In EAP 5, JBoss shutdown hangs to be caused by deadlock between "JBoss Shutdown Hook" and JBoss Messaging thread "Thread-N":
"JBoss Shutdown Hook" daemon prio=11 tid=0x00002aaac0143800 nid=0x7ef4 in Object.wait() [0x0000000044aa1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x0000000795991950> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock) at java.lang.Object.wait(Object.java:485) at EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240) - locked <0x0000000795991950> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock) at org.jboss.jms.server.remoting.JMSServerInvocationHandler.setClosed(JMSServerInvocationHandler.java:68) at org.jboss.jms.server.ServerPeer.stopService(ServerPeer.java:388) - locked <0x0000000794c09488> (a org.jboss.jms.server.ServerPeer) at org.jboss.system.ServiceMBeanSupport.jbossInternalStop(ServiceMBeanSupport.java:405) at org.jboss.system.ServiceMBeanSupport.stop(ServiceMBeanSupport.java:281) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:157) at org.jboss.mx.server.Invocation.dispatch(Invocation.java:96) at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:138) at org.jboss.mx.server.Invocation.invoke(Invocation.java:90) at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:140) at org.jboss.mx.server.Invocation.invoke(Invocation.java:90) at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264) at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:668) at org.jboss.system.microcontainer.ServiceProxy.invoke(ServiceProxy.java:206) at $Proxy38.stop(Unknown Source) at org.jboss.system.microcontainer.StartStopLifecycleAction.uninstallAction(StartStopLifecycleAction.java:56) at org.jboss.system.microcontainer.StartStopLifecycleAction.uninstallAction(StartStopLifecycleAction.java:37) at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleUninstallAction(SimpleControllerContextAction.java:79) at org.jboss.dependency.plugins.action.AccessControllerContextAction.uninstall(AccessControllerContextAction.java:131) at org.jboss.dependency.plugins.AbstractControllerContextActions.uninstall(AbstractControllerContextActions.java:58) at org.jboss.dependency.plugins.AbstractControllerContext.uninstall(AbstractControllerContext.java:354) at org.jboss.dependency.plugins.AbstractController.uninstall(AbstractController.java:1685) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:1295) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:1198) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:1254) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:1198) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:1254) at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:831) at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:556) at org.jboss.system.ServiceController.doChange(ServiceController.java:688) at org.jboss.system.ServiceController.stop(ServiceController.java:510) ...(snip)... ...(snip)... "Thread-20" prio=10 tid=0x00002aaabe4ca800 nid=0x49b9 waiting for monitor entry [0x0000000040f66000] java.lang.Thread.State: BLOCKED (on object monitor) at org.jboss.jms.server.ServerPeer.getDefaultExpiryQueueInstance(ServerPeer.java:1572) - waiting to lock <0x0000000794c09488> (a org.jboss.jms.server.ServerPeer) at org.jboss.jms.server.endpoint.ServerSessionEndpoint.<init>(ServerSessionEndpoint.java:230) at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.createSessionDelegate(ServerConnectionEndpoint.java:249) at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$createSessionDelegate$aop(ConnectionAdvised.java:75) at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$createSessionDelegate_6052335267724906805.invokeTarget(ConnectionAdvised$createSessionDelegate_6052335267724906805.java) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111) at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.createSessionDelegate(ConnectionAdvised.java) at org.jboss.jms.wireformat.ConnectionCreateSessionDelegateRequest.serverInvoke(ConnectionCreateSessionDelegateRequest.java:87) at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:157) at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:930) at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:106) at org.jboss.remoting.Client.invoke(Client.java:2034) at org.jboss.remoting.Client.invoke(Client.java:877) at org.jboss.remoting.Client.invoke(Client.java:865) at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189) at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160) at org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$createSessionDelegate$aop(ClientConnectionDelegate.java:190) at org.jboss.jms.client.delegate.ClientConnectionDelegate$createSessionDelegate_6052335267724906805.invokeTarget(ClientConnectionDelegate$createSessionDelegate_6052335267724906805.java) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111) at org.jboss.jms.client.container.StateCreationAspect.handleCreateSessionDelegate(StateCreationAspect.java:115) at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect_z_handleCreateSessionDelegate_237427554.invoke(StateCreationAspect_z_handleCreateSessionDelegate_237427554.java) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) at org.jboss.jms.client.container.ConnectionAspect.handleCreateSessionDelegate(ConnectionAspect.java:162) at sun.reflect.GeneratedMethodAccessor352.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92) at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:172) at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) at org.jboss.jms.client.delegate.ClientConnectionDelegate.createSessionDelegate(ClientConnectionDelegate.java) at org.jboss.jms.client.JBossConnection.createSessionInternal(JBossConnection.java:269) at org.jboss.jms.client.JBossConnection.createSession(JBossConnection.java:91) at org.jboss.jms.server.recovery.MessagingXAResourceWrapper.connect(MessagingXAResourceWrapper.java:300) - locked <0x0000000795408ed0> (a java.lang.Object) at org.jboss.jms.server.recovery.MessagingXAResourceWrapper.getDelegate(MessagingXAResourceWrapper.java:260) at org.jboss.jms.server.recovery.MessagingXAResourceWrapper.recover(MessagingXAResourceWrapper.java:108) at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.xaRecovery(XARecoveryModule.java:1167) at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.resourceInitiatedRecovery(XARecoveryModule.java:665) at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkSecondPass(XARecoveryModule.java:200) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:799) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:412)
Environment
- Red Hat JBoss Enterprise Application Platform (EAP)
- 4.3
- 5.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.