Chapter 1. Monitoring and managing Red Hat Fuse applications on OpenShift

1.1. About the Fuse Console

The Red Hat Fuse Console is a web console based on HawtIO open source software. For a list of supported browsers, go to Supported Configurations.

The Fuse Console provides a central interface to examine and manage the details of one or more deployed Fuse containers. You can also monitor Red Hat Fuse and system resources, perform updates, and start or stop services.

The Fuse Console is available when you install Red Hat Fuse standalone or use Fuse on OpenShift. The integrations that you can view and manage in the Fuse Console depend on the plugins that are running. Possible plugins include:

  • Camel
  • JMX
  • Spring Boot
  • OSGI
  • Runtime
  • Logs

1.2. Setting up the Fuse Console on OpenShift 4.x

On OpenShift 4.x, setting up the Fuse Console involves securing, installing, and deploying it.

First, you must generate a client certificate so that you can secure the Fuse Console as described in Section 1.2.1, “Generating a certificate to secure the Fuse Console on OpenShift 4.x”.

After you generate the client certificate, you have these options for installing and deploying the Fuse Console:

Note
  • User management for the Fuse Console is handled by OpenShift.
  • Role-based access control (for users accessing the Fuse Console after it is deployed) is not yet available for Fuse on OpenShift.

1.2.1. Generating a certificate to secure the Fuse Console on OpenShift 4.x

On OpenShift 4.x, to keep the connection between the Fuse Console proxy and the Jolokia agent secure, you must generate a client certificate before you deploy the Fuse Console. You must use the service signing certificate authority private key to sign the client certificate.

Important

You must generate and sign a separate client certificate for each OpenShift cluster. Do not use the same certificate for more than one cluster.

Prerequisites

  • You have cluster admin access to the OpenShift cluster.
  • If you are generating certificates for more than one OpenShift cluster and you previously generated a certificate for a different cluster in the current directory, do one of the following to ensure that you generate a different certificate for the current cluster:

    • Delete the existing certificate files (for example, ca.crt, ca.key, and ca.srl) from the current directory.
    • Change to a different working directory. For example, if your current working directory is named cluster1, create a new cluster2 directory and change your working directory to it:

      mkdir ../cluster2

      cd ../cluster2

Procedure

  1. Login to OpenShift as a user with cluster admin access:

    oc login -u <user_with_cluster_admin_role>
  2. Retrieve the service signing certificate authority keys, by executing the following commands:

    • To retrieve the certificate:

      oc get secrets/signing-key -n openshift-service-ca -o "jsonpath={.data['tls\.crt']}" | base64 --decode > ca.crt
    • To retrieve the private key:

      oc get secrets/signing-key -n openshift-service-ca -o "jsonpath={.data['tls\.key']}" | base64 --decode > ca.key
  3. Generate the client certificate, as documented in Kubernetes certificates administration, using either easyrsa, openssl, or cfssl.

    Here are the example commands using openssl:

    1. Generate the private key:

      openssl genrsa -out server.key 2048
    2. Write the CSR config file.

      cat <<EOT >> csr.conf
        [ req ]
        default_bits = 2048
        prompt = no
        default_md = sha256
        distinguished_name = dn
      
        [ dn ]
        CN = fuse-console.fuse.svc
      
        [ v3_ext ]
        authorityKeyIdentifier=keyid,issuer:always
        keyUsage=keyEncipherment,dataEncipherment,digitalSignature
        extendedKeyUsage=serverAuth,clientAuth
      EOT
    3. Generate the CSR:

      openssl req -new -key server.key -out server.csr -config csr.conf
    4. Issue the signed certificate:

      openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 10000 -extensions v3_ext -extfile csr.conf

Next steps

You need this certificate to create the secret for the Fuse Console as described in the following sections, depending on how you want to install the Fuse Console:

1.2.2. Installing and deploying the Fuse Console on OpenShift 4.x by using the OperatorHub

To install the Fuse Console on OpenShift 4.x, you can use the Fuse Console Operator provided in the OpenShift OperatorHub. To deploy the Fuse Console, you create an instance of the installed operator.

Prerequisite

Procedure

To install and deploy the Fuse Console:

  1. Log in to the OpenShift console in your web browser as a user with cluster admin access.
  2. Select Home > Projects, and then select the project to which you want to add the Fuse Console.
  3. Click Operators and then click OperatorHub.
  4. In the search field window, type Fuse Console to filter the list of operators.
  5. Click Fuse Console Operator.
  6. In the Fuse Console Operator install window, click Install.

    The Create Operator Subscription form opens.

    • For Installation mode, you install the Fuse Console Operator to a specific namespace (the current OpenShift project).

      Note that after you install the operator, you can then choose to deploy the Fuse Console to monitor applications in all namespaces on the cluster or to monitor applications only in the namespace in which the Fuse Console operator is installed.

    • For the Approval Strategy, you can select Automatic or Manual to configure how OpenShift handles updates to the Fuse Console Operator.

      • If you select Automatic updates, when a new version of the Fuse Console Operator is available, the OpenShift Operator Lifecycle Manager (OLM) automatically upgrades the running instance of the Fuse Console without human intervention.
      • If you select Manual updates, when a newer version of an Operator is available, the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to have the Fuse Console Operator updated to the new version.
  7. Click Subscribe.

    OpenShift installs the Fuse Console Operator in the current namespace.

  8. To verify the installation, click Operators and then click Installed Operators. You can see the Fuse Console in the list of operators.
  9. In a Terminal window, use the certificate that you generated in Securing the Fuse Console on OpenShift 4.x to create the secret and mount it in the Fuse Console by using the following command where APP_NAME is the name of the Fuse Console Deployment (for example, fuse-console).

    oc create secret tls APP_NAME-tls-proxying --cert server.crt --key server.key

    For example, if the name of the Fuse Console application is fuse-console, type this command:

    oc create secret tls fuse-console-tls-proxying --cert server.crt --key server.key

    If successful, this command returns a response that confirms the secret was created, for example:

    secret/fuse-console-operator-tls-proxying created
  10. To deploy the Fuse Console by using the OpenShift web console:

    1. In the list of Installed Operators, under the Name column, click Fuse Console.
    2. On the Overview page under Provided APIs, click Create Instance. A new Custom Resource Definition (CRD) file opens.

      By default, the Fuse Console is deployed to the current namespace.

    3. If you want to deploy the Fuse Console to discover and manage applications within the current namespace, skip to the next step.

      Optionally, if you want to deploy the Fuse Console to discover and manage applications across all namespaces that are on the cluster (and for which you are an authenticated user), edit the CRD file by changing the value of the spec.type field from namespace to cluster.

    4. Click Create.

      The Fuse Console CRD page opens showing the new Fuse Console service.

  11. To open the Fuse Console:

    1. Obtain the Fuse Console URL:

      • From within the OpenShift web console, select Networking > Routes.
      • From the command line, type the oc get routes command.
    2. In your web browser, open the Fuse Console URL and then log in.

      An Authorize Access page opens in the browser listing the required permissions.

    3. Click Allow selected permissions.

      The Fuse Console opens in the browser and shows the Fuse application pods that you have authorization to access.

  12. Click Connect for the application that you want to view.

    A new browser window opens showing the application in the Fuse Console.

1.2.3. Installing and deploying the Fuse Console on OpenShift 4.x by using the command line

On OpenShift 4.x, you can choose one of these deployment options to install and deploy the Fuse Console from the command line:

  • cluster - The Fuse Console can discover and connect to Fuse applications deployed across multiple namespaces (projects) on the OpenShift cluster. To deploy this template, you must have the administrator role for the OpenShift cluster.
  • namespace - The Fuse Console has access to a specific OpenShift project (namespace). To deploy this template, you must have the administrator role for the OpenShift project.

To view a list of the parameters for the Fuse Console templates, run the following OpenShift command:

oc process --parameters -f https://raw.githubusercontent.com/jboss-fuse/application-templates/application-templates-2.1.0.fuse-760043-redhat-00003/fuse-console-namespace-os4.json

Prerequisites

Procedure

  1. Verify that the Fuse Console image stream is installed by using the following command to retrieve a list of all templates:

    oc get template -n openshift
  2. Optionally, if you want to update the already installed image stream with new release tags, use the following command to import the Fuse Console image to the openshift namespace:

    oc import-image fuse7/fuse7-console:1.6 --from=registry.redhat.io/fuse7/fuse-console:1.6 --confirm -n openshift
  3. Obtain the Fuse Console APP_NAME value by running the following command:

    • For the cluster template:

      oc process --parameters -f https://raw.githubusercontent.com/jboss-fuse/application-templates/application-templates-2.1.0.fuse-760043-redhat-00003/fuse-console-cluster-os4.json
    • For the namespace template:

      oc process --parameters -f https://raw.githubusercontent.com/jboss-fuse/application-templates/application-templates-2.1.0.fuse-760043-redhat-00003/fuse-console-namespace-os4.json
  4. From the certificate that you generated in Securing the Fuse Console on OpenShift 4.x, create the secret and mount it in the Fuse Console by using the following command (where APP_NAME is the name of the Fuse Console application).

    oc create secret tls APP_NAME-tls-proxying --cert server.crt --key server.key
  5. Create a new application by running the following command (where $project is the name of your OpenShift project, $APP_NAME is name of the application obtained in the above step, and $DOMAIN_NAME is the hostname to access the Fuse Console:

    • For the cluster template:

      oc new-app -n $project -f {templates-base-url}/fuse-console-cluster-os4.json -p ROUTE_HOSTNAME=$APP_NAME.$DOMAIN_NAME -p APP_NAME=$APP_NAME
    • For the namespace template:

      oc new-app -n myproject -f {templates-base-url}/fuse-console-namespace-os4.json
  6. Obtain the status and the URL of your Fuse Console deployment by running this command:

    oc status
  7. To access the Fuse Console from a browser, use the URL that is returned in Step 6 (for example, https://fuse-console.192.168.64.12.nip.io).

1.2.4. Upgrading the Fuse Console on OpenShift 4.x

Red Hat OpenShift 4.x handles updates to operators, including the Red Hat Fuse operators. For more information see the Operators OpenShift documentation.

In turn, operator updates can trigger application upgrades, depending on how the application is configured.

For Fuse Console applications, you can also trigger an upgrade to an application by editing the .spec.version field of the application custom resource definition.

Prerequisite

  • You have OpenShift cluster admin permissions.

Procedure

To upgrade a Fuse Console application:

  1. In a terminal window, use the following command to change the .spec.version field of the application custom resource definition:

    oc patch <project-name> <custom-resource-name> --type='merge' -p '{"spec":{"version":"1.7.1"}}'

    For example:

    oc patch myproject example-fuseconsole --type='merge' -p '{"spec":{"version":"1.7.1"}}'
  2. Check that the application’s status has updated:

     oc get myproject

    The response shows information about the application, including the version number:

    NAME                  AGE   URL                                        IMAGE
    example-fuseconsole   1m    https://fuseconsole.192.168.64.38.nip.io   docker.io/fuseconsole/online:1.7.1

    When you change the value of the .spec.version field, OpenShift automatically redeploys the application.

  3. To check the status of the redeployment that is triggered by the version change:

    oc rollout status deployment.v1.apps/example-fuseconsole

    A successful deployment shows this response:

    deployment "example-fuseconsole" successfully rolled out

1.3. Setting up the Fuse Console on OpenShift 3.11

On OpenShift 3.11, you can set up the Fuse Console in two ways:

  • By adding the centralized Fuse Console catalog item to a project so that you can monitor all the running Fuse containers in the project.
  • From a specific pod so that you can monitor that single running Fuse container.

You can deploy the Fuse Console either from the OpenShift Console or from the command line.

Note

To install Fuse Console on Minishift or CDK based enviroments, follow the steps explained in the KCS article below.

  • To install Fuse Console on Minishift or CDK based enviroments, see KCS 4998441.
  • If it is necessary to disable Jolokia authentication see the workaround described in KCS 3988671.

Prerequisites

  • Install the Fuse on OpenShift image streams and the templates for the Fuse Console as described in Fuse on OpenShift Guide.
  • For cluster mode on OpenShift 3.11, you need the cluster admin role and the cluster mode template. Run the following command:

    oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:openshift-infra:template-instance-controller
Note
  • The cluster mode template is only available, by default, on the latest version of the OpenShift Container Platform. It is not provided with the OpenShift Online default catalog.
  • The Fuse Console templates configure end-to-end encryption by default so that your Fuse Console requests are secured end-to-end, from the browser to the in-cluster services.
  • User management for the Fuse Console is handled by OpenShift.
  • Role-based access control (for users accessing the Fuse Console after it is deployed) is not yet available for Fuse on OpenShift.

Section 1.3.1, “Deploying the Fuse Console from the OpenShift 3.11 Console”

Section 1.3.2, “Monitoring a single Fuse pod from the Fuse Console on OpenShift 3.11”

Section 1.3.3, “Deploying the Fuse Console from the command line”

1.3.1. Deploying the Fuse Console from the OpenShift 3.11 Console

To deploy the Fuse Console on your OpenShift cluster from the OpenShift 3.11 Console, follow these steps.

Procedure

  1. In the OpenShift console, open an existing project or create a new project.
  2. Add the Fuse Console to your OpenShift project:

    1. Select Add to ProjectBrowse Catalog.

      The Select an item to add to the current project page opens.

    2. In the Search field, type Fuse Console.

      The Red Hat Fuse 7.x Console and Red Hat Fuse 7.x Console (cluster) items should appear as the search result.

Note

If the Red Hat Fuse Console items do not appear as the search result, or if the items that appear are not the latest version, you can install the Fuse Console templates manually as described in the "Prepare the OpenShift server" section of the Fuse on OpenShift Guide.

  1. Click one of the Red Hat Fuse Console items:

    • Red Hat Fuse 7.x Console - This version of the Fuse Console discovers and connects to Fuse applications deployed in the current OpenShift project.
    • Red Hat Fuse 7.x Console (cluster) - This version of the Fuse Console can discover and connect to Fuse applications deployed across multiple projects on the OpenShift cluster.
  2. In the Red Hat Fuse Console wizard, click Next. The Configuration page of the wizard opens.

    Optionally, you can change the default values of the configuration parameters.

    1. Click Create.

      The Results page of the wizard indicates that the Red Hat Fuse Console has been created.

    2. Click the Continue to the project overview link to verify that the Fuse Console application is added to the project.
    3. To open the Fuse Console, click the provided URL link and then log in.

      An Authorize Access page opens in the browser listing the required permissions.

    4. Click Allow selected permissions.

      The Fuse Console opens in the browser and shows the Fuse pods running in the project.

    5. Click Connect for the application that you want to view.

      A new browser window opens showing the application in the Fuse Console.

1.3.2. Monitoring a single Fuse pod from the Fuse Console on OpenShift 3.11

You can open the Fuse Console for a Fuse pod running on OpenShift 3.11:

  1. From the Applications → Pods view in your OpenShift project, click on the pod name to view the details of the running Fuse pod. On the right-hand side of this page, you see a summary of the container template:

    Container Template View

  2. From this view, click on the Open Java Console link to open the Fuse Console.

    Fuse Console view

    Note

    In order to configure OpenShift to display a link to Fuse Console in the pod view, the pod running a Fuse on OpenShift image must declare a TCP port within a name attribute set to jolokia:

    {
      "kind": "Pod",
      [...]
      "spec": {
        "containers": [
          {
            [...]
            "ports": [
              {
                "name": "jolokia",
                "containerPort": 8778,
                "protocol": "TCP"
              }

1.3.3. Deploying the Fuse Console from the command line

Table 1.1, “Fuse Console templates” describes the OpenShift 3.1 templates that you can use to deploy the Fuse Console from the command line, depending on the type of Fuse application deployment.

Table 1.1. Fuse Console templates

TypeDescription

fis-console-cluster-template.json

The Fuse Console can discover and connect to Fuse applications deployed across multiple namespaces or projects. To deploy this template, you must have the OpenShift cluster-admin role.

fis-console-namespace-template.json

This template restricts the Fuse Console access to the current OpenShift project (namespace), and as such acts as a single tenant deployment. To deploy this template, you must have the admin role for the current OpenShift project.

Optionally, you can view a list of the parameters for all of the templates by running this command:

oc process --parameters -f https://raw.githubusercontent.com/jboss-fuse/application-templates/application-templates-2.1.0.fuse-760043-redhat-00003/fis-console-namespace-template.json

Procedure

To deploy the Fuse Console from the command line:

  1. Create a new application based on a Fuse Console template by running one of the following commands (where myproject is the name of your project):

    • For the Fuse Console cluster template, where myhost is the hostname to access the Fuse Console:

      oc new-app -n myproject -f https://raw.githubusercontent.com/jboss-fuse/application-templates/application-templates-2.1.0.fuse-760043-redhat-00003/fis-console-cluster-template.json -p ROUTE_HOSTNAME=myhost
    • For the Fuse Console namespace template:

      oc new-app -n myproject -f https://raw.githubusercontent.com/jboss-fuse/application-templates/application-templates-2.1.0.fuse-760043-redhat-00003/fis-console-namespace-template.json
      Note

      You can omit the route_hostname parameter for the namespace template because OpenShift automatically generates one.

  2. Obtain the status and the URL of your Fuse Console deployment by running this command:

    oc status
  3. To access the Fuse Console from a browser, use the provided URL (for example, https://fuse-console.192.168.64.12.nip.io).

1.4. Viewing containers and applications

When you login to the Fuse Console for OpenShift, the Fuse Console home page shows the available containers.

Procedure

  • To manage (create, edit, or delete) containers, use the OpenShift console.
  • To view Fuse applications on the OpenShift cluster, click the Online tab.

1.5. Viewing and managing Apache Camel applications

In the Fuse Console’s Camel tab, you view and manage Apache Camel contexts, routes, and dependencies.

You can view the following details:

  • A list of all running Camel contexts
  • Detailed information of each Camel context such as Camel version number and runtime statics
  • Lists of all routes in each Camel application and their runtime statistics
  • Graphical representation of the running routes along with real time metrics

You can also interact with a Camel application by:

  • Starting and suspending contexts
  • Managing the lifecycle of all Camel applications and their routes, so you can restart, stop, pause, resume, etc.
  • Live tracing and debugging of running routes
  • Browsing and sending messages to Camel endpoints

Prerequisite

The Camel tab is only available when you connect to a container that uses one or more Camel routes.

1.5.1. Starting, suspending, or deleting a context

  1. In the Camel tab’s tree view, click Camel Contexts.
  2. Check the box next to one or more contexts in the list.
  3. Click Start or Suspend.
  4. To delete a context:

    1. Stop the context.
    2. Click the ellipse icon and then select Delete from the dropdown menu.
Note

When you delete a context, you remove it from the deployed application.

1.5.2. Viewing Camel application details

  1. In the Camel tab’s tree view, click a Camel application.
  2. To view a list of application attributes and values, click Attributes.
  3. To view a graphical representation of the application attributes, click Chart and then click Edit to select the attributes that you want to see in the chart.
  4. To view inflight and blocked exchanges, click Exchanges.
  5. To view application endpoints, click Endpoints. You can filter the list by URL, Route ID, and direction.
  6. To view, enable, and disable statistics related to the Camel built-in type conversion mechanism that is used to convert message bodies and message headers to different types, click Type Converters.
  7. To view and execute JMX operations, such as adding or updating routes from XML or finding all Camel components available in the classpath, click Operations.

1.5.3. Viewing a list of the Camel routes and interacting with them

  1. To view a list of routes:

    1. Click the Camel tab.
    2. In the tree view, click the application’s routes folder:

      mf list of routes

  2. To start, stop, or delete one or more routes:

    1. Check the box next to one or more routes in the list.
    2. Click Start or Stop.
    3. To delete a route, you must first stop it. Then click the ellipse icon and select Delete from the dropdown menu.

      mf delete route

      Note
      • When you delete a route, you remove it from the deployed application.
      • You can also select a specific route in the tree view and then click the upper-right menu to start, stop, or delete it.
  3. To view a graphical diagram of the routes, click Route Diagram.
  4. To view inflight and blocked exchanges, click Exchanges.
  5. To view endpoints, click Endpoints. You can filter the list by URL, Route ID, and direction.
  6. Click Type Converters to view, enable, and disable statistics related to the Camel built-in type conversion mechanism, which is used to convert message bodies and message headers to different types.
  7. To interact with a specific route:

    1. In the Camel tab’s tree view, select a route.
    2. To view a list of route attributes and values, click Attributes.
    3. To view a graphical representation of the route attributes, click Chart. You can click Edit to select the attributes that you want to see in the chart.
    4. To view inflight and blocked exchanges, click Exchanges.
    5. Click Operations to view and execute JMX operations on the route, such as dumping the route as XML or getting the route’s Camel ID value.
  8. To trace messages through a route:

    1. In the Camel tab’s tree view, select a route.
    2. Select Trace, and then click Start tracing.
  9. To send messages to a route:

    1. In the Camel tab’s tree view, open the context’s endpoints folder and then select an endpoint.
    2. Click the Send subtab.
    3. Configure the message in JSON or XML format.
    4. Click Send.
    5. Return to the route’s Trace tab to view the flow of messages through the route.

1.5.4. Debugging a route

  1. In the Camel tab’s tree view, select a route.
  2. Select Debug, and then click Start debugging.
  3. To add a breakpoint, select a node in the diagram and then click Add breakpoint. A red dot appears in the node:

    mf breakpoint

    The node is added to the list of breakpoints:

    mf breakpointlist

  4. Click the down arrow to step to the next node or the Play button to resume running the route.
  5. Click the Pause button to suspend all threads for the route.
  6. Click Stop debugging when you are done. All breakpoints are cleared.

1.6. Viewing and managing JMX domains and MBeans

Java Management Extensions (JMX) is a Java technology that allows you to manage resources (services, devices, and applications) dynamically at runtime. The resources are represented by objects called MBeans (for Managed Bean). You can manage and monitor resources as soon as they are created, implemented, or installed.

With the JMX plugin on the Fuse Console, you can view and manage JMX domains and MBeans. You can view MBean attributes, run commands, and create charts that show statistics for the MBeans.

The JMX tab provides a tree view of the active JMX domains and MBeans organized in folders. You can view details and execute commands on the MBeans.

Procedure

  1. To view and edit MBean attributes:

    1. In the tree view, select an MBean.
    2. Click the Attributes tab.
    3. Click an attribute to see its details.
  2. To perform operations:

    1. In the tree view, select an MBean.
    2. Click the Operations tab, expand one of the listed operations.
    3. Click Execute to run the operation.
  3. To view charts:

    1. In the tree view, select an item.
    2. Click the Chart tab.

1.7. Viewing diagnostics

Use the Diagnostics tab to view diagnostic information about the JVM via the JVM DiagnosticCommand and HotspotDiangostic interfaces.

Note

The functionality is similar to the Diagnostic Commands view in Java Mission Control (jmc) or the command line tool jcmd. The plugin will provide corresponding jcmd commands in some scenarios.

Procedure

  1. To retrieve the number of instances of loaded classes and the amount of bytes they take up, click Class Histogram. If the operation is repeated, the tab shows the difference since last run.
  2. To view the JVM diagnostic flag setting, click the JVM flags.
  3. For a running JVM, you can also modify the flag settings.

Additional resources

The supported JVM depends on the platform, for more information go to one of the following sources:

1.8. Viewing threads

You can view and monitor the state of threads.

Procedure

  1. Click the Runtime tab and then the Threads subtab. The Threads page lists active threads and stack trace details for each thread. By default, the thread list shows all threads in descending ID order.
  2. To sort the list by increasing ID, click the ID column label.
  3. Optionally, filter the list by thread state (for example, Blocked) or by thread name.
  4. To drill down to detailed information for a specific thread, such as the lock class name and full stack trace for that thread, in the Actions column, click More.

1.9. Ensuring that data displays correctly in the Fuse Console

If the display of the queues and connections in the Fuse Console is missing queues, missing connections, or displaying inconsistent icons, adjust the Jolokia collection size parameter that specifies the maximum number of elements in an array that Jolokia marshals in a response.

Procedure

  1. In the upper right corner of the Fuse Console, click the user icon and then click Preferences.

    mf preferences

  2. Increase the value of the Maximum collection size option (the default is 50,000).
  3. Click Close.