Tomcat comes to a halt with threads waiting on GenericObjectPool

Issue

Tomcat became unresponsive and thread dumps showed many threads waiting to lock a org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.  The waiting threads look like:

"ajp-8009-1" daemon prio=10 tid=0x0896e400 nid=0x380e waiting for monitor entry [0x612b9000]
   java.lang.Thread.State: BLOCKED (on object monitor)
     at org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:781)
     - waiting to lock <0x75df1678> (a org.apache.tomcat.dbcp.pool.impl.GenericObjectPool)
     at org.apache.tomcat.dbcp.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
     at org.apache.tomcat.dbcp.dbcp.BasicDataSource.getConnection(BasicDataSource.java:540)

The thread holding the lock looks like:

"ajp-8009-2" daemon prio=10 tid=0x08dadc00 nid=0x3673 runnable [0x62bab000]
   java.lang.Thread.State: RUNNABLE
     at java.net.SocketInputStream.socketRead0(Native Method)
     at java.net.SocketInputStream.read(SocketInputStream.java:146)
     at oracle.net.ns.Packet.receive(Packet.java:283)
     at oracle.net.ns.DataPacket.receive(DataPacket.java:103)
     at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:230)
     at oracle.net.ns.NetInputStream.read(NetInputStream.java:175)
     at oracle.net.ns.NetInputStream.read(NetInputStream.java:100)
     at oracle.net.ns.NetInputStream.read(NetInputStream.java:85)
     at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:123)
     at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:79)
     at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1122)
     at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1099)
     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:288)
     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:191)
     at oracle.jdbc.driver.T4C7Ocommoncall.doOLOGOFF(T4C7Ocommoncall.java:61)
     at oracle.jdbc.driver.T4CConnection.logoff(T4CConnection.java:498)
     - locked <0x7c45fc88> (a oracle.jdbc.driver.T4CConnection)
     at oracle.jdbc.driver.PhysicalConnection.close(PhysicalConnection.java:3931)
     - locked <0x7c45fc88> (a oracle.jdbc.driver.T4CConnection)
     at org.apache.tomcat.dbcp.dbcp.DelegatingConnection.close(DelegatingConnection.java:159)
     at org.apache.tomcat.dbcp.dbcp.PoolableConnection.reallyClose(PoolableConnection.java:95)
     at org.apache.tomcat.dbcp.dbcp.PoolableConnectionFactory.destroyObject(PoolableConnectionFactory.java:301)
     at org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool(GenericObjectPool.java:946)
     at org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.returnObject(GenericObjectPool.java:917)
     - locked <0x75df1678> (a org.apache.tomcat.dbcp.pool.impl.GenericObjectPool)
     at org.apache.tomcat.dbcp.dbcp.PoolableConnection.close(PoolableConnection.java:80)
     - locked <0x7c45fc00> (a org.apache.tomcat.dbcp.dbcp.PoolableConnection)
     at org.apache.tomcat.dbcp.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.close(PoolingDataSource.java:180)

• Issue about connection pool configuration. We have sometime issues on our JBoss web server based on tomcat 6 and JVM 1.6.0_45.Particularly we noticed lost of connection with the db (Oracle 11.2.04) with subsequent hang of Tomcat (JbossWS).