How to use JDK Flight Recorder (JFR)?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL)
  • Windows
  • Red Hat build of OpenJDK
    • OpenJDK 8 u262+
    • OpenJDK 11

Issue

  • How to use JDK Flight Recorder ?
  • Heap dump can not be provided because it contains sensitive information.
  • Heap dump from a production system can not be provided due to application performance impact.

Resolution

JDK Flight Recorder (JFR) is a diagnostic and profiling tool for a running Java application.

Use one of the three approaches below, either at the start of the process with it, or using java diagnostics (jcmd), or via JMX client (Cryostat):

First option: Enable recording and specify recording options with the following JVM options when starting the application:

-XX:StartFlightRecording=duration=200s,filename=flight.jfr

To specify a full path to the jfr recording (e.g. filename=/path/to/flight.jfr).

Second option: Start/Stop the recording with Java Diagnostic tool ( jcmd <pid|MainClass> [parameters] ):

 $ jcmd PID JFR.start filename=flight.jfr

This will start a recording called flight.jfr without limit of duration so use jcmd JFR.stop to stop the recording

Third option: For instance on OCP, use JMX on JMC or Cryostat Operator (via cryostat CRD) to attach.

jcmd options

jcmd Command Action
JFR.check Check the status of all recordings running for the specified process, including the recording identification number, file name, duration, and so on.
JFR.stop Stop a recording with a specific identification number (by default, recording 1 is stopped).
JFR.dump Dump the data collected so far by the recording with a specific identification number (by default, data from recording 1 is dumped)

Use jcmd PID help for a list of commands. For example:

$ jcmd 12345 help | grep JFR
JFR.check
JFR.configure
JFR.dump
JFR.start
JFR.stop

jcmd help on command

Use jcmd PID help to get more information on a command. For example:

$ jcmd 123456 help JFR.start
123456:
JFR.start
Starts a new JFR recording

Impact: Medium: Depending on the settings for a recording, the impact can range from low to high.

Permission: java.lang.management.ManagementPermission(monitor)

Syntax : JFR.start [options]

Options: (options must be specified using the <key> or <key>=<value> syntax)
    name : [optional] Name that can be used to identify recording, e.g. \"My Recording\" (STRING, no default value)
    settings : [optional] Settings file(s), e.g. profile or default. See JRE_HOME/lib/jfr (STRING SET, no default value)
    delay : [optional] Delay recording start with (s)econds, (m)inutes), (h)ours), or (d)ays, e.g. 5h. (NANOTIME, 0)
    duration : [optional] Duration of recording in (s)econds, (m)inutes, (h)ours, or (d)ays, e.g. 300s. (NANOTIME, 0)
    disk : [optional] Recording should be persisted to disk (BOOLEAN, no default value)
    filename : [optional] Resulting recording filename, e.g. \"/home/user/My Recording.jfr\" (STRING, no default value)
...

Limit the duration

To limit the duration of the recording, to for 4 hours for example, use run:

$JAVA_HOME/bin/jcmd $JBOSS_PID JFR.start duration=4h

Deprecated flags

  • The flag FlightRecorder is unnecessary after 8u240 JDK-8225312 and the record can be done without it.
  • The flag UnlockCommercialFeatures is also unnecessary when recording flight recording data.
  • The flag FlightRecorderOptions=maxage was deprecated. Use StartFlightRecording=maxage instead. Be careful with : instead of = after StartFlightRecording depending on the JDK version.

Increasing fidelity of JFR

The flag DebugNonSafepoints can be used to increase/improve the profiler data, however it must be combined with UnlockDiagnosticVMOptions flag.

Example usage for JDK 11:

For JDK 11 (or JDK 1.8.240+) it is not required to use the flags FlightRecorder or UnlockCommercialFeatures one can just start a process and record, full example below:

#start java process
$ export JAVA_HOME=/path/java-11-openjdk-11.0.12.0.7-0.portable.jdk.el.x86_64
$ ./bin/standalone.sh  <--------------- eap for example
#jcmd to get PID
$ ./jcmd
28416 /PATH/jboss-eap-7.4/jboss-modules.jar -mp /path/jboss-eap-7.4/modules org.jboss.as.standalone -Djboss.home.dir=/jboss-eap-7.4 -Djboss.server.base.dir=/jboss-eap-7.4/standalone
28608 jdk.jcmd/sun.tools.jcmd.JCmd
#start
$ ./jcmd PID JFR.start filename=/path/flight.jfr
PID:
Started recording 1. No limit specified, using maxsize=250MB as default.
Use jcmd PID JFR.dump name=1 to copy recording data to file. <----- use this name to stop it
#check
$ ./jcmd PID JFR.check
23735:
Recording 1: name=1 maxsize=250.0MB (running)
#dump
$ ./jcmd PID JFR.dump filename=/path/flight.jfr
23735:
Dumped recording, 1.1 MB written to: /path/flight.jfr
# stop
$ ./jcmd 23735 JFR.stop name=1 <------------------ name 1 == default
23735:
Stopped recording "1".

Notes:

  • JFR has very low overhead (normally < 1%) because it's built into the runtime (c/c++ code). The default settings are suitable for both development and production environments.

  • jcmd should be run as the same user as the Java process. Running jcmd as the LOCAL SYSTEM account and JFR as the currently logged in user will result in an "Insufficient Privileges" error. Use the PsExec Tool from Microsoft to escalate to the proper user.

  • Depending on the option required, the usage of StartFlightRecording is preferred than FlightRecorderOptions JDK-8221632.

Visualizing Data

Use Java Mission Control (jmc) to analyze the data from the jfr file. jmc is included in the Windows distro (JAVA_HOME/missioncontrol portable build, C:\jdk\missioncontrol msi install) and is available as a separate rpm on RHEL.

alt text

Attachments

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments