- Linux and most Unix-like environments
- How do I generate a thread dump in JBoss while running on Linux?
- How do I generate a JBoss stack trace on Linux?
- How do I redirect output of kill -3 to a file?
- JBoss has high cpu usage, freezes, hangs, or doesn't release idle threads, how can I get a thread dump to troubleshoot?
- JMS messages are getting lost, how can I generate a thread dump to investigate?
- Unable to take thread dump using Kill -3 in server.log or console.log
ResolutionFollowing are methods for generating a Java thread dump on Unix: 1) Note the process ID number of the Java process (e.g. using
ps -axw, etc.) and send a QUIT signal to the process with the
kill -3command. For example:
2) Download either threaddump_linux.sh.tar.gz or threaddump_solaris.sh.tar.gz , and extract the script. Make the script executeable with
kill -3 JAVA_PID
It will capture a series of 6 thread dumps spaced 20 seconds apart (modify as needed), passing in the Java process ID as an argument. For example:
Be sure to test the script before the issue happens to make sure it runs properly in your environment. 3) Download either threaddump_linux-continuous.sh.tar.gz or threaddump_solaris-continuous.sh.tar.gz , and extract the script. Make the script executeable with
Linux: sh ./threaddump_linux.sh JAVA_PID Solaris: bash ./threaddump_solaris.sh JAVA_PID
It will capture thread dumps spaced 20 seconds apart (modify as needed), passing in the Java process ID as an argument. For example:
Be sure to test the script before the issue happens to make sure it runs properly in your environment. 4) Use the below command to start your JBoss instance and then use kill -3 to generate the threaddumps.
Linux: sh ./threaddump_linux-continuous.sh JAVA_PID Solaris: bash ./threaddump_solaris-continuous.sh JAVA_PID
If the Java application is started with a service script that logs console output, the thread dumps will be in the console log. Otherwise, redirect
stdoutto a file on startup.
This will redirect your output/threadump to the file console specified in the above command. 4) OpenJDK / Sun JDK
nohup $JBOSS_HOME/bin/run.sh -c yourinstancename $JBOSS_OPTS >>; console-$(date +%Y%m%d).out 2>&1 < /dev/null & kill -3 JAVA_PID
jps -lvto find the Java process ID for issuing
Be sure the
-XrsJVM option is not being used, as it causes SIGQUIT and SIGWAITING signals to be ignored. Running
kill -3sends a SIGQUIT signal to the JVM, so using this option will cause
kill -3to be ignored. See Java Application launcher.
If using OpenJDK or Sun JDK 1.6 or later, using
jstackis an option. This is useful when redirecting standard out to a file is problematic for some reason (e.g. it is not desirable to restart the JVM just to redirect standard out). Execute the following, passing in the Java process ID:
5) Download 'linux_jstack-continuous.tar.gz', and extract the script. Make the script executeable with
jstack -l JAVA_PID > jstack.out
It will use jstack to capture a series of 6 thread dumps spaced 20 seconds apart (modify as needed), passing in the Java process ID as an argument. Make sure you set "JAVA_HOME" in this script. It generates a file called "jstack_threaddump.out" in the directory where this script is executed. For example:
- Verify that the Java process is still running with the
ps auxcommand (
STATcolumn) . For example,
jstack -F <pid>puts the target Java process in a "trace stop" (T) state. Threads in the (T) state will receive the signal for a thread dump; however, output will be delayed until the process continues.
- You need to make sure to execute
jstackcommand from the same user as the java process. Please see this article for more details.
- There are known bugs related to using other
-m, etc.) and/or thread dump tools are not able to parse the output, so if any other options besides
-lare proposed, be sure to test capturing and parsing the output.
- In some circumstances running jstack might cause a performance impact. It is reported that a slowness in response time can be observed when running jstack in some scenarios.
- If you want to take thread dumps to identify which Java threads consume high cpu, please see How do I identify high CPU utilization by Java threads on Linux/Solaris and use the attached sample script in the article.
JBoss EAP 5.xTo redirect
stdoutto a file on startup:
If having trouble redirecting
sh run.sh > console.log 2>&1
$JBOSS_HOME/bin/run.shand change the line:
Save and restart JBoss normally. Now
org.jboss.Main "$@" > console.log
kill -QUITshould create a thread dump in the file
JBoss EAP 6.xThe recommendation is to use jstack.
JBoss Fuse 6If you are capturing thread dumps from a child container, note that the process is different to the root container. Be sure to check the process ID (PID) using 'ps' or similar before selecting the PID to run the script against. eg:
ps -ef | grep child1 testusr 27803 1 0 09:41 pts/1 00:00:12 /usr/java/jdk1.7.0_21/jre/bin/java -server ... -Dkaraf.home=/home/testusr/apps/product-distributions -Dkaraf.base=/home/testusr/apps/product-distributions/instances/child1 org.apache.karaf.main.Main
How do I identify high CPU utilization by Java threads on Linux/Solaris
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.