Chapter 3. Configuring Java applications
To enable Cryostat to gather, store, and analyze Java Flight Recorder (JFR) data about target applications that run on Java Virtual Machine (JVM)s, you must configure the applications so that Cryostat can detect and connect to them.
You can configure the applications in any of the following ways:
- By using the Cryostat agent component for detection and connectivity, which is implemented as a Java Instrumentation Agent and acts as a plug-in for applications that run on the JVM
- By configuring the applications to allow Java Management Extensions (JMX) connections and using an OpenShift Service for detection and JMX for connectivity
- By using the Cryostat agent for detection and JMX for connectivity
Cryostat agent
The Cryostat agent provides a read-only HTTP API for Cryostat functions and removes the requirement for target applications to expose a JMX port.
The agent performs the following actions:
- Fetches data from the JVM and sends it back to Cryostat over HTTP by looking for data within itself and the application to which it is plugged in
- Locates Cryostat applications that run on the network and initiates a connection to those applications
- Communicates back to Cryostat about itself and provides information on how to reach it
- Pushes its own JFR data to Cryostat by initiating a network connection with Cryostat, which then analyzes and saves the data
The agent supports a limited set of JFR operations and you must configure JFR recordings in advance before the attached application starts. You can configure the JFR data you want the agent to collect, a schedule for pushing that data to Cryostat, and the Cryostat server URL to which you want to push.
You can use the command line to install the agent on your target JVM. You must add the agent to your application builds, then rebuild and deploy the application.
After the agent is installed to the running JVM instance, the agent starts collecting data and sending it to Cryostat for analysis.
Remote Java Management Extensions (JMX) connections
JMX is a standard feature on a JVM with which you can monitor and manage target applications that run on the JVM. For Cryostat to use JMX, you must enable and configure JMX when you start the JVM because Cryostat requires the target applications to expose a JMX port.
Cryostat communicates with the target applications over this JMX port to start and stop JFR recordings and to pull JFR data over the network, enabling Cryostat to store and analyze this JFR data. Remote monitoring requires security to ensure that unauthorized persons cannot access application. Cryostat prompts you to enter your credentials before Cryostat can access any of the application’s JFR recordings.
Cryostat agent and JMX hybrid
You can configure your target applications to use a hybrid approach where you use both the Cryostat agent and JMX. With this approach, you use the Cryostat agent to detect the target applications and JMX to expose the JFR data to Cryostat, which allows for more flexibility.
For example, you can use the agent to detect the applications without needing to depend on specific port numbers and also use the JMX connections to start and stop JFR flight recordings on demand.
3.1. Configuring applications by using the Cryostat agent
You can use the Cryostat agent, implemented as a Java Instrumentation Agent, to configure target applications so that Cryostat can detect the applications, collect data, and send the data to Cryostat for analysis.
Prerequisites
- Logged in to your Cryostat web console.
- Installed JDK version 11 or later.
Procedure
Install the Cryostat agent. Choose one of the following options, depending on your application build:
Using Maven:
Update the application
pom.xml
file with the Cryostat agent JAR file information.Example pom.xml
<project> ... <repositories> <repository> <id>redhat-maven-repository</id> <url>https://maven.repository.redhat.com/earlyaccess/all/</url> </repository> </repositories> ... <build> <plugins> <plugin> <artifactId>maven-dependency-plugin</artifactId> <version>3.3.0</version> <executions> <execution> <phase>prepare-package</phase> <goals> <goal>copy</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>io.cryostat</groupId> <artifactId>cryostat-agent</artifactId> <version>0.2.2.redhat-00001</version> </artifactItem> </artifactItems> <stripVersion>true</stripVersion> </configuration> </execution> </executions> </plugin> </plugins> ... </build> ... </project>
The next time you build your application, the Cryostat agent JAR file is available at
target/dependency/cryostat-agent.jar
.Using Gradle:
Update the
build.gradle
file.Example
build.gradle
filerepositories { … maven { url "https://maven.repository.redhat.com/earlyaccess/all/" credentials { username "myusername" password "mytoken" } } }
How you package the agent JAR file into the application depends on the Gradle plug-ins that you use for the build. For example, if you are using the Jib plug-in, update the
build.gradle
file as follows:Example
build.gradle
fileplugins { id 'java' id 'application' id 'com.google.cloud.tools.jib' version '3.3.1' id 'com.ryandens.javaagent-jib' version '0.5.0' } … dependencies { … javaagent 'io.cryostat:cryostat-agent:0.2.2.redhat-00001'
Update the Docker file. The following example uses the
JAVA_OPTS
environment variable to pass the relevant JVM information.Example
... COPY target/dependency/cryostat-agent.jar /deployments/app/ ... ENV JAVA_OPTS="-javaagent:/deployments/app/cryostat-agent.jar"
Rebuild the container image that is specific to your application.
docker build -t docker.io/myorg/myapp:latest -f src/main/docker/Dockerfile
To supply the JVM system properties or environment variables that you need to configure the Cryostat agent, push the updated image and then modify your application deployment.
Example
apiVersion: apps/v1 kind: Deployment ... spec: ... template: ... spec: containers: - name: sample-app image: docker.io/myorg/myapp:latest env: - name: CRYOSTAT_AGENT_APP_NAME value: "myapp" # Replace this with the Kubernetes DNS record # for the Cryostat Service - name: CRYOSTAT_AGENT_BASEURI value: "http://cryostat.mynamespace.mycluster.svc:8181" - name: POD_IP valueFrom: fieldRef: fieldPath: status.PodIP - name: CRYOSTAT_AGENT_CALLBACK value: "http://$(POD_IP):9977" 1 # Replace "abcd1234" with a base64-encoded authentication token - name: CRYOSTAT_AGENT_AUTHORIZATION 2 value: "Bearer abcd1234" ports: - containerPort: 9977 protocol: TCP resources: {} restartPolicy: Always status: {}
-
<1>: Port number
9977
is the default HTTP port that the agent exposes for the internal web server that services Cryostat requests. You can change this port number if it conflicts with your target application into which the agent is installed. -
<2>: The
CRYOSTAT_AGENT_AUTHORIZATION
value shows the credentials that the agent includes in the API requests to Cryostat to advertise its own presence or to push JFR data. You can also create a KubernetesService Account
for this purpose and replaceabcd1234
with the base64-encoded authentication token associated with the service account.
-
<1>: Port number
3.2. Configuring applications by using JMX connections
For Cryostat to detect and communicate with your target Java applications, you can configure the applications to allow remote Java Management Extensions (JMX) connections.
Prerequisites
- Logged in to your Cryostat web console.
- Created a Cryostat instance in your project.
Procedure
To enable remote JMX connections, complete the following steps:
On your application, define the following Java system property:
-Dcom.sun.management.jmxremote.port=<port_num>
NoteTo add the
-Dcom.sun.management.jmxremote.port=<port_num>
property without having to rebuild the target application, you can set theJAVA_OPTS_APPEND
environment variable on the application.JAVA_OPTS_APPEND
is an environment variable that is used by Red Hat Universal Base Images (UBI) only.If you use Red Hat UBI to build application images, set the
JAVA_OPTS_APPEND
variable at build time in the application Docker file, or at runtime by running the following command:oc set env deployment <name> JAVA_OPTS_APPEND="..."
If you do not use Red Hat UBI to build application images, refer to the documentation for your base image for information about how to add the Java system properties at build time or runtime.
Specify that the application listens for remote JMX connections by allowing traffic to the application. Use an Red Hat OpenShift Service and specify the following values for its remote JMX port:
Example
service.yaml
apiVersion: v1 kind: Service ... spec: ports: - name: "jfr-jmx" port: 9091 targetPort: 9091 ...
Secure the remote JMX connections:
Enable and configure authentication and SSL/TLS for the remote JMX connections in your application:
-Dcom.sun.management.jmxremote.port=<port_num> # enable JMX authentication -Dcom.sun.management.jmxremote.authenticate=true # define users for JMX auth -Dcom.sun.management.jmxremote.password.file=</path/to/jmxremote.password> # set permissions for JMX users -Dcom.sun.management.jmxremote.access.file=</path/to/jmxremote.access> # enable JMX SSL -Dcom.sun.management.jmxremote.ssl=true # enable JMX registry SSL -Dcom.sun.management.jmxremote.registry.ssl=true # set your SSL keystore -Djavax.net.ssl.keyStore=</path/to/keystore> # set your SSL keystore password -Djavax.net.ssl.keyStorePassword=<password>
Configure Cryostat to trust the application TLS certificate. Create a secret for the application in the same namespace as your Cryostat application and configure Cryostat to refer to the secret. To create a secret for the certificate, run the following command:
oc create secret generic myapp-cert --from-file=tls.crt=/path/to/cert.pem
NoteThe certificate must be in a
.pem
file format.- When you create your Cryostat instance, add the secret to the list of trusted TLS certificates. For more information, see Configuring TLS certificates.
To allow your applications to verify that Cryostat is connecting to them by a means other than password authentication, enable TLS client authentication:
-Dcom.sun.management.jmxremote.ssl.need.client.auth=true -Djavax.net.ssl.trustStore=</path/to/truststore> -Djavax.net.ssl.trustStorePassword=<password>
NoteTLS client authentication requires the cert-manager operator for Red Hat OpenShift.
If you use TLS client authentication for remote JMX connections, the application truststore must contain a Cryostat certificate. The Cryostat operator cert-manager integration creates a self-signed certificate for the Cryostat deployment. This certificate is located in the
<cryostat>-tls
secret, where <cryostat> is the name of the Cryostat instance you created.NoteThe cert-manager Operator also places a Java keystore truststore in the secret.
To mount this truststore in your application deployment, run the following command, replacing "<myapp>" with the name of your application deployment and "<cryostat>" with the name of your Cryostat instance:
oc set volumes deploy <myapp> --add --name=truststore \ --secret-name=<cryostat>-tls --sub-path=truststore.p12 \ --mount-path=/var/run/secrets/<myapp>/truststore.p12
The Cryostat operator generates the truststore password, which you can find in the
<cryostat>-keystore
secret. To mount this as an environment variable in your application deployment, run the following command:oc set env deploy <myapp> --from='secret/<cryostat>-keystore'
Configure the Java arguments for the container. Run the following commands:
-Dcom.sun.management.jmxremote.ssl.need.client.auth=true -Djavax.net.ssl.trustStore=/var/run/secrets/<myapp>/truststore.p12 -Djavax.net.ssl.trustStorePassword="$(KEYSTORE_PASS)"
WarningIf you deployed Cryostat and your applications in a testing environment, you might want to configure the target applications without any JMX or TLS authentication. You can do so by using the following set of Java system properties, however, this configuration is not secure and not recommended.
-Dcom.sun.management.jmxremote.port=<port_num> -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false
Additional resources
3.3. Configuring applications by using the Cryostat agent and JMX connections
You can configure target applications that run on Java Virtual Machines (JVM) to use a combination of the Cryostat agent and Java Management Extensions (JMX) connections to detect and communicate with the target applications.
You use the Cryostat agent to detect and communicate with the target application and use JMX to expose the Java Flight Recorder (JFR) data.
You must configure the Cryostat agent to communicate with Cryostat about itself and that the agent is reachable through JMX rather than through HTTP.
Prerequisites
- Logged in to your Cryostat web console.
- Created a Cryostat instance in your project.
Procedure
Install the Cryostat agent. For application builds that use Maven, update the application
pom.xml
file with the Cryostat agent JAR file information.Example
pom.xml
file<project> ... <repositories> <repository> <id>redhat-maven-repository</id> <url>https://maven.repository.redhat.com/earlyaccess/all/</url> </repository> </repositories> ... <build> <plugins> <plugin> <artifactId>maven-dependency-plugin</artifactId> <version>3.3.0</version> <executions> <execution> <phase>prepare-package</phase> <goals> <goal>copy</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>io.cryostat</groupId> <artifactId>cryostat-agent</artifactId> <version>0.2.2.redhat-00001</version> </artifactItem> </artifactItems> <stripVersion>true</stripVersion> </configuration> </execution> </executions> </plugin> </plugins> ... </build> ... </project>
Modify your application deployment:
Example
apiVersion: apps/v1 kind: Deployment ... spec: ... template: ... spec: containers: - name: sample-app image: docker.io/myorg/myapp:latest env: - name: CRYOSTAT_AGENT_APP_NAME value: "myapp" # Replace this with the Kubernetes DNS record # for the Cryostat Service - name: CRYOSTAT_AGENT_BASEURI value: "http://cryostat.mynamespace.mycluster.svc:8181" - name: POD_IP valueFrom: fieldRef: fieldPath: status.PodIP - name: CRYOSTAT_AGENT_CALLBACK value: "http://$(POD_IP):9977" - name: CRYOSTAT_AGENT_AUTHORIZATION # Replace "abcd1234" with a base64-encoded authentication token value: "Bearer abcd1234" # This environment variable is key to the Cryostat agent # and JMX "hybrid" setup. # Set the Cryostat agent to register itself with Cryostat # as reachable through JMX, rather than reachable through HTTP. - name: CRYOSTAT_AGENT_REGISTRATION_PREFER_JMX value: "true" # Configure the application to load the agent JAR file and # to enable JMX, so that the Cryostat agent can register # itself as reachable through JMX. # To configure authentication and SSL/TLS for the JMX # connections, see <1>. - name: JAVA_OPTS value: >- -javaagent:/deployments/app/cryostat-agent.jar -Dcom.sun.management.jmxremote.port=9091 1 ports: - containerPort: 9977 protocol: TCP resources: {} restartPolicy: Always status: {}
<1>: To configure authentication and SSL/TLS for the JMX connections and view further configuration options, see Configuring applications by using JMX connections.
To enable Cryostat to detect the target application and connect to the Cryostat agent, configure an application
Service
:Example
apiVersion: v1 kind: Service ... spec: ports: - name: "jfr-jmx" port: 9091 targetPort: 9091 - name: "cryostat-agent" port: 9977 targetPort: 9977 ...