EAP 6 deadlocks on a logging ConsoleHandler and java.io.PrintStream
Issue
- JBoss EAP 6 became unresponsive. Thread dumps showed the following deadlock:
"http-/0.0.0.0:80-2":
at java.util.logging.StreamHandler.publish(StreamHandler.java:174)
- waiting to lock <0x00000000f93c4498> (a java.util.logging.ConsoleHandler)
at java.util.logging.ConsoleHandler.publish(ConsoleHandler.java:88)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:290)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.Logger.logRaw(Logger.java:721)
at org.jboss.logmanager.Logger.log(Logger.java:506)
at org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71)
- locked <0x00000000fa687ef8> (a java.lang.StringBuilder)
at org.jboss.stdio.WriterOutputStream.finish(WriterOutputStream.java:143)
at org.jboss.stdio.WriterOutputStream.flush(WriterOutputStream.java:164)
- locked <0x00000000fa68c960> (a sun.nio.cs.UTF_8$Decoder)
at java.io.PrintStream.write(PrintStream.java:432)
- locked <0x00000000fa687e90> (a java.io.PrintStream)
at org.jboss.stdio.StdioContext$DelegatingPrintStream.write(StdioContext.java:225)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:276)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)
- locked <0x00000000f93deff0> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)
at java.util.logging.StreamHandler.flush(StreamHandler.java:225)
- locked <0x00000000f93de058> (a java.util.logging.ConsoleHandler)
at java.util.logging.ConsoleHandler.publish(ConsoleHandler.java:89)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:290)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.Logger.logRaw(Logger.java:721)
at org.jboss.logmanager.Logger.log(Logger.java:672)
at org.jboss.logmanager.Logger.log(Logger.java:684)
at org.jboss.logging.JBossLogManagerLogger.doLog(JBossLogManagerLogger.java:40)
at org.jboss.logging.Logger.debug(Logger.java:382)
...
"http-/0.0.0.0:80-1":
at java.io.PrintStream.flush(PrintStream.java:285)
- waiting to lock <0x00000000fa687e90> (a java.io.PrintStream)
at org.jboss.stdio.StdioContext$DelegatingPrintStream.flush(StdioContext.java:209)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:278)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)
- locked <0x00000000f93c6680> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)
at java.util.logging.StreamHandler.flush(StreamHandler.java:225)
- locked <0x00000000f93c4498> (a java.util.logging.ConsoleHandler)
at java.util.logging.ConsoleHandler.publish(ConsoleHandler.java:89)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:290)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:298)
at org.jboss.logmanager.Logger.logRaw(Logger.java:721)
at org.jboss.logmanager.Logger.log(Logger.java:672)
at org.jboss.logmanager.Logger.log(Logger.java:684)
at org.jboss.logging.JBossLogManagerLogger.doLog(JBossLogManagerLogger.java:40)
at org.jboss.logging.Logger.debug(Logger.java:382)
...
- We see a deadlock between multiple ConsoleAppenders, for example:
"ServerService Thread Pool -- 58":
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:231)
- waiting to lock <0x0000000702fa6888> (a org.apache.log4j.ConsoleAppender)
at org.apache.log4j.JBossAppenderHandler.doPublish(JBossAppenderHandler.java:34)
at org.jboss.logmanager.ExtHandler.publish(ExtHandler.java:76)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:302)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:310)
at org.jboss.logmanager.Logger.logRaw(Logger.java:719)
at org.jboss.logmanager.Logger.log(Logger.java:504)
at org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71)
- locked <0x0000000702fa9458> (a java.lang.StringBuilder)
at org.jboss.stdio.WriterOutputStream.finish(WriterOutputStream.java:143)
at org.jboss.stdio.WriterOutputStream.flush(WriterOutputStream.java:164)
- locked <0x0000000702fa9510> (a sun.nio.cs.UTF_8$Decoder)
at java.io.PrintStream.write(PrintStream.java:482)
- locked <0x0000000702fa93f0> (a java.io.PrintStream)
at org.jboss.stdio.StdioContext$DelegatingPrintStream.write(StdioContext.java:264)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
- locked <0x0000000700121e10> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
- locked <0x00000007001213b8> (a org.apache.log4j.ConsoleAppender)
at org.apache.log4j.JBossAppenderHandler.doPublish(JBossAppenderHandler.java:34)
"ServerService Thread Pool -- 75":
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:231)
- waiting to lock <0x00000007001213b8> (a org.apache.log4j.ConsoleAppender)
at org.apache.log4j.JBossAppenderHandler.doPublish(JBossAppenderHandler.java:34)
at org.jboss.logmanager.ExtHandler.publish(ExtHandler.java:76)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:302)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:310)
at org.jboss.logmanager.Logger.logRaw(Logger.java:719)
at org.jboss.logmanager.Logger.log(Logger.java:504)
at org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71)
- locked <0x0000000700123fe0> (a java.lang.StringBuilder)
at org.jboss.stdio.WriterOutputStream.finish(WriterOutputStream.java:143)
at org.jboss.stdio.WriterOutputStream.flush(WriterOutputStream.java:164)
- locked <0x0000000700124098> (a sun.nio.cs.UTF_8$Decoder)
at java.io.PrintStream.write(PrintStream.java:482)
- locked <0x0000000700123f78> (a java.io.PrintStream)
at org.jboss.stdio.StdioContext$DelegatingPrintStream.write(StdioContext.java:264)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
- locked <0x0000000702fa72c8> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
- locked <0x0000000702fa6888> (a org.apache.log4j.ConsoleAppender)
at org.apache.log4j.JBossAppenderHandler.doPublish(JBossAppenderHandler.java:34)
Environment
- JBoss Enterprise Application Platform (EAP) 6.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.