Hi all,

I am having some issues where from time to time my JBoss 5.2 instance(s) won't fully start up.

I can see in the logs, that the JBoss instance(s) gets until the ** INFO [Version] HV000001: Hibernate Validator 4.3.1.Final ** point, and after that it just hangs and never completes. I have found so far that this happens due to deadlocks that occur while the instance(s) is starting. Running "jstack ", would show me the deadlock which i provide below. The instances are running an ATG10.2 application. I don't see any resource issues on my servers(plenty of memory, plenty of space)

Here is where the deadlock happens:

Found one Java-level deadlock:

"Thread-12":
waiting to lock monitor 0x000000004956e470 (object 0x0000000400d9e870, a java.lang.Object),
which is held by "main"
"main":
waiting for ownable synchronizer 0x00000004013f1bd8, (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync),
which is held by "Thread-12"

And here are the top line of the stack for "main" and "Thread-12":

Java stack information for the threads listed above:

"Thread-12":
at java.beans.Introspector.getBeanInfo(Introspector.java:157)
- waiting to lock 0x0000000400d9e870> (a java.lang.Object)
at org.jboss.aop.advice.GenericAspectFactory.configureInstance(GenericAspectFactory.java:223)
at org.jboss.aop.advice.GenericAspectFactory.createPerVM(GenericAspectFactory.java:147)
at org.jboss.aop.AspectManager.createPerVmAspect(AspectManager.java:2052)
- locked 0x00000006d7b9c0b0> (a org.jboss.aop.advice.AspectDefinition)
at org.jboss.aop.AspectManager.getPerVMAspect(AspectManager.java:2028)
- locked 0x00000006d7b9c0b0> (a org.jboss.aop.advice.AspectDefinition)
at org.jboss.aop.AspectManager.getPerVMAspect(AspectManager.java:2011)
at org.jboss.aop.advice.PerVmAdvice.generateOptimized(PerVmAdvice.java:51)
- locked 0x00000003f4949dc0> (a java.lang.Class for org.jboss.aop.advice.PerVmAdvice)
at org.jboss.aop.advice.AdviceFactory.create(AdviceFactory.java:106)
at org.jboss.aop.Advisor.createInterceptorChain(Advisor.java:1328)
at org.jboss.aop.Advisor.pointcutResolved(Advisor.java:1942)
at org.jboss.aop.MethodMatchInfo.applyBinding(MethodMatchInfo.java:152)
at org.jboss.aop.MethodMatchInfo.simplePopulateBindings(MethodMatchInfo.java:113)
at org.jboss.aop.MethodMatchInfo.populateBindings(MethodMatchInfo.java:96)
at org.jboss.aop.Advisor.finalizeMethodChain(Advisor.java:1553)
at org.jboss.aop.ClassAdvisor.finalizeChains(ClassAdvisor.java:741)
at org.jboss.aop.ClassAdvisor.doRebuildInterceptorsForAddedBinding(ClassAdvisor.java:1163)
at org.jboss.aop.ClassAdvisor$RebuildInterceptorsAction$2.rebuildInterceptorsForAddedBinding(ClassAdvisor.java:2367)
at org.jboss.aop.ClassAdvisor.rebuildInterceptorsForAddedBinding(ClassAdvisor.java:1141)
at org.jboss.aop.Advisor.newBindingAdded(Advisor.java:1134)
- locked 0x00000007487bcde0> (a org.jboss.aop.ClassAdvisor)
at org.jboss.aop.AspectManager.updateAdvisorsForAddedBinding(AspectManager.java:1530)
- locked 0x00000004012d09b0> (a java.util.WeakHashMap)
at org.jboss.aop.AspectManager.addBinding(AspectManager.java:1491)
- locked 0x00000004012d09b0> (a java.util.WeakHashMap)
at org.jboss.aop.AspectXmlLoader.deployBinding(AspectXmlLoader.java:292)
at org.jboss.aop.AspectXmlLoader.deployTopElements(AspectXmlLoader.java:1145)
at org.jboss.aop.AspectXmlLoader.deployXML(AspectXmlLoader.java:990)
at org.jboss.aop.AspectXmlLoader.deployXML(AspectXmlLoader.java:981)
at org.jboss.jms.client.container.JmsClientAspectXMLLoader.deployXML(JmsClientAspectXMLLoader.java:93)
- locked 0x00000004012d0908> (a org.jboss.aop.AspectManager)

"main":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for 0x00000004013f1bd8> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:941)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1261)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:677)
at org.jboss.aop.util.AOPLock.lockRead(AOPLock.java:40)
at org.jboss.aop.AspectManager.translate(AspectManager.java:1060)
at org.jboss.aop.AspectManager.transform(AspectManager.java:1023)
at org.jboss.aop.standalone.AOPTransformer.aspectTransform(AOPTransformer.java:87)
at org.jboss.aop.standalone.AOPTransformer.transform(AOPTransformer.java:75)
at sun.instrument.TransformerManager.transform(TransformerManager.java:169)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:365)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:637)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at org.jboss.classloader.spi.base.BaseClassLoader.access$200(BaseClassLoader.java:48)
at org.jboss.classloader.spi.base.BaseClassLoader$2.run(BaseClassLoader.java:637)
at org.jboss.classloader.spi.base.BaseClassLoader$2.run(BaseClassLoader.java:596)
at java.security.AccessController.doPrivileged(Native Method)
at org.jboss.classloader.spi.base.BaseClassLoader.loadClassLocally(BaseClassLoader.java:595)
- locked 0x00000007406c4210> (a org.jboss.classloader.spi.base.BaseClassLoader)
at org.jboss.classloader.spi.base.BaseClassLoader.loadClassLocally(BaseClassLoader.java:572)
at org.jboss.classloader.spi.base.BaseDelegateLoader.loadClass(BaseDelegateLoader.java:135)
at org.jboss.classloader.spi.filter.FilteredDelegateLoader.loadClass(FilteredDelegateLoader.java:131)
at org.jboss.classloader.spi.base.ClassLoadingTask$ThreadTask.run(ClassLoadingTask.java:455)
at org.jboss.classloader.spi.base.ClassLoaderManager.nextTask(ClassLoaderManager.java:267)
at org.jboss.classloader.spi.base.ClassLoaderManager.process(ClassLoaderManager.java:166)
at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:304)
at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:1261)
at org.jboss.classloader.spi.base.BaseClassLoader.loadClassFromDomain(BaseClassLoader.java:914)
at org.jboss.classloader.spi.base.BaseClassLoader.doLoadClass(BaseClassLoader.java:506)
- locked 0x00000007406c4210> (a org.jboss.classloader.spi.base.BaseClassLoader)

Anyone has ran into this deadlock before? Any suggestion is highly appreciated. If you need any other information, please let me know.
If it helps to see the full stack of the two deadlocks, i can also provide that.

Thank you in advance !