Configuring notifications and integrations on the Red Hat Hybrid Cloud Console

Red Hat Hybrid Cloud Console 2023

Configuring Hybrid Cloud Console settings so that account users receive event-triggered notifications about RHEL systems

Red Hat Customer Content Services

Abstract

Using notifications to learn of identified events that have occurred and could impact your organization.
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright's message.

Chapter 1. Introduction to Red Hat Hybrid Cloud Console notifications and integrations

The notifications service on Red Hat Hybrid Cloud Console frees users from having to check their user interface routinely, looking for event-triggered notifications. Instead, when events occur, the service automatically takes the action to send notifications about the events to users.

Important

Groups and roles, including the Notifications administrator role, must be configured by an Organization Administrator in order for events to be reported through email and integrations.

To learn more about User Access on the Red Hat Hybrid Cloud Console platform, see the User Access Configuration Guide for Role-based Access Control (RBAC)^.

1.1. What the notifications service does

Through the notifications service, Red Hat Hybrid Cloud Console applications and services have a standardized way of notifying users of events. By setting up behavior groups, a Notifications administrator specifies the notification delivery method, and whether event notifications are sent to all users on an account or only to Organization Administrators.

For example, the Notifications administrator can configure the service to send an email notification for (advisor service) new-recommendation hits on a system. Similarly, the administrator might decide to trigger a notification that sends a message to a third-party application using the webhook integration type.

An Organization Administrator designates Notifications administrators by creating a User Access group with the Notifications administrator role, then adding account members to the group. A Notifications administrator then configures notification behavior groups that define actions taken when service-specific events occur.

The notifications service transmits event-triggered notifications to users’ email accounts, or to third-party applications using webhooks. Users on the Hybrid Cloud Console account set their own preferences for receiving email notifications. In User preferences > Notifications > application bundle, each user configures their personal settings to receive event notification emails as an instant notification or daily digest.

Important

Selecting Instant notification for any service can potentially result in receiving a very large number of emails.

1.2. Notification and integration concepts

The following table defines terms that are important for understanding how the notifications service works:

Table 1.1. Notifications concepts

ConceptDescription

Actions

Operations performed in response to an event. Examples include sending an email or forwarding a notification to an integration endpoint, such as a webhook. Actions are defined in behavior groups configured by a Notifications administrator.

Application bundle

Application bundle refers to an application group within the Hybrid Cloud Console, such as Red Hat Enterprise Linux (Insights for RHEL) or OpenShift.

Behavior groups

Behavior groups determine what actions to take when an event occurs, and whether to notify all account users or only designated administrators. Once a Notifications administrator creates a behavior group, they associate it with event types, enabling Notifications administrators to apply the same actions to all application-specific events.

NOTE: Notifications administrators configure notification behavior groups separately, for each application bundle.

Email preferences

Individual users with access to applications on the Hybrid Cloud Console set their personal email preferences. Users can configure personal email notifications to arrive either instantly, as the event occurs, or consolidated into a daily digest that arrives at midnight, 00:00 Coordinated Universal Time (UTC), for all accounts.

IMPORTANT: Selecting Instant notification for any service can potentially result in receiving a very large number of emails.

Event type

Event types are application-specific system changes that trigger the application or service to initiate notification actions. Event types are created by application developers at Red Hat and are unique for each application bundle. Examples from the Insights for RHEL (Red Hat Enterprise Linux) application bundle include:

* Policies service: Policy triggered

* Drift service: Drift from baseline detected

* Advisor service: New recommendation; Resolved recommendation

Integrations

Integrations define the method of delivery of notifications to third-party applications configured by the Notifications administrator. Once configured, the notifications service sends the HTTP POST messages to endpoints.

User access roles

The following User Access roles interact with notifications:

* Organization Administrator

* Notifications administrator

* Notifications viewer

1.3. Insights events and notifications options

There are three main ways to integrate Insights into your organization’s workflows:

  • Using the Insights APIs
  • Using webhooks and/or emails directly to users
  • Using integrations with a third-party application, such as Splunk

1.3.1. Using the Insights APIs

Insights APIs are publicly available and can be queried from any authenticated client (RBAC controlled).

For more information about the available endpoints for applications and services, refer to the Red Hat Insights API documentation. For an example of CSV-formatted responses, see the System Comparison API Documentation.

The Red Hat Insights API cheat sheet covers the use of Insights APIs. It provides examples to help you to get started quickly with authentication and with querying the endpoints. Most of the examples use the curl command, but this document also includes sample code for clients that are written in Python, and an Ansible playbook that performs similar operations.

1.3.2. Using Integrations webhooks

Webhooks work in a similar way to APIs, except that they enable one-way data sharing when events trigger them. APIs share data in both directions. Applications that allow inbound data requests are said to have exposed webhooks.

You can configure Insights to send POST messages to specific endpoints in exposed webhooks within applications. This capability works in concert with the Notifications service. For example, you can configure Insights to automatically email new Advisor recommendations to selected administrator accounts as soon as Insights identifies them.

Once you configure the endpoints in Notifications, you can subscribe to a stream of Insights events and automatically forward that stream to the webhook(s) of your choice. Each event contains additional metadata, which you can use to process the event (for example, perform specific actions and/or trigger responses) as part of your operational workflow. You configure the implementation and data handling within your application.

For more information about how to configure and use webhooks, refer to Configure integrations.

1.3.3. Using a third-party application

Insights third-party application integrations work in one of two ways, depending on the use case:

  • Using Insights APIs to collect data and perform tasks
  • Subscribe to streams of Insights events

You can use Red Hat Insights integrations to forward events to specific third-party applications. In this version of Red Hat Insights, the Red Hat Insights application for Splunk forwards selected Insights events to Splunk. This allows you to view, and use Insights data, in your existing workflows from the Red Hat Insights application for Splunk dashboard.

For more information about the Red Hat Insights application for Splunk, see Installing and Configuring the Red Hat Insights application for Splunk.

Chapter 2. Configure User Access

Before account users can configure notifications and integration settings, a group with the Notification administrator role must be configured in User Access by an Organization Administrator. In Red Hat Hybrid Cloud Console > Settings menu (gear icon) > Identity & Access Management > User Access > Groups, an Organization Administrator performs the following high-level steps:

  • Create a User Access group for Notifications administrators.

    • Add the Notifications administrator role to the group.
    • Add members (users with account access) to the group.

Organization Administrator

The Organization Administrator configures the User Access group for Notifications administrators, then adds the Notifications administrator role and users to the group.

Notifications administrator role

Notifications administrators configure how applications interact with notifications. Notifications administrators configure behavior groups to define how applications notify users about events. Administrators can configure additional integrations as they become available, as well as edit, disable, and remove existing integrations.

Notifications viewer role

The Notifications viewer role is automatically conferred to everyone on the account and limits how a user can interact with notifications service views and configurations. A viewer can view notification configurations, but cannot modify or remove them. A viewer also cannot configure.r modify, or remove integrations.

For learn more about User Access on the Red Hat Hybrid Cloud Console platform, see the User Access Configuration Guide for Role-based Access Control (RBAC)^.

2.1. Creating and configuring a notifications group in User Access

The following procedure shows how an Organization Administrator on the account creates a group with the Notifications administrator role and adds members to the group.

Prerequisites

  • You must be logged into your Red Hat Hybrid Cloud Console account as an Organization Administrator.

Procedure

  1. Click the gear icon in the upper right quadrant of the application window and select Settings.

    img hcc toolbar settings

  2. From the Settings menu on the left, click User Access and select Groups.
  3. Click Create group.
  4. Enter a group name, for example, Notifications Administrators, and a description, then click Next.
  5. Select the role to add to this group, in this case Notifications administrator. Click the checkbox for that role, then click Next.
  6. Add members to the group. Search for individual users or filter by username, email, or status. Check the box next to each intended member’s name, then click Next.
  7. Review the details to make sure everything is correct. Click Back if you need to go back and change something.
  8. Click Submit to finish creating the group.

2.2. Editing or removing a User Access group

If you need to edit or remove a User Access group, perform the following steps:

Prerequisites

  • You must be logged into your Red Hat Hybrid Cloud Console account as an Organization Administrator.

Procedure

  1. Locate the group name in Red Hat Hybrid Cloud Console > Settings menu (gear icon) > Identity & Access Management > User Access > Groups.
  2. Click the menu options icon on the far right of the group name row ( img hcc icons options ), and click Edit or Delete.
  3. Make and save changes or delete the group.

Chapter 3. Configure integrations

The Red Hat Hybrid Cloud Console notifications and integrations services work together to transmit messages to third-party application endpoints, such as instant messaging platforms and external ticketing systems, when triggering events occur.

This enables Notifications administrators to integrate Hybrid Cloud Console functionality into the operational workflow used in their organization. Integrations are configured by a Notifications administrator as endpoints in Red Hat https://console.redhat.com[Red Hat Hybrid Cloud Console > Settings > Integrations].

Note

Webhook is the current integration type supported in the Red Hat Hybrid Cloud Console platform. When configured, the service sends an HTTP POST message to the specified third-party applications endpoint.

3.1. HTTP POST messages

The following screenshot is an example of an HTTP POST message sent to a third-party application endpoint. Event types are specific to a service or application. For example, the Insights for RHEL application bundle currently notifies configured users of events from the policies, advisor, and drift services. The following example notification from the Insights for RHEL advisor service was triggered by a new recommendation on a host system.

img hcc notif schema

In the example, the blocks contain the following information:

  1. Information about the bundle and application sending the notification

    • bundle: Name of the application bundle
    • application: Name of the individual application or service sending the event-triggered notification
    • event_type: The event type that triggered the notification
    • account_id: The Red Hat account from which the notification was sent
    • timestamp: ISO-8601 formatted date showing when the notification was sent
  2. Information about the application or service-specific event

    • payload: The application payload, a JSON string containing all the data sent by the application
  3. Information about the system on which the event occurred. For example:

    • inventory_id: System ID
    • hostname: System name
    • rhel_version: RHEL version running on the system

The metadata field is not currently being used.

3.2. Setting up integrations

The Notifications administrator sets up integrations for the organization. In addition to adding new integrations, the Notifications administrator can edit, remove, or disable any listed integration by clicking the More options menu icon img hcc icons options , located to the right of the integration name, and then clicking the appropriate option.

Prerequisites

  • To perform the following procedure, a user must be logged into the Red Hat Hybrid Cloud Console platform with Notifications administrator privileges configured in User Access.

Procedure

  1. Navigate to Red Hat Hybrid Cloud Console > Settings > Integrations.
  2. Click Add integration.

    1. Enter an Integration name.
    2. Select an integration Type, such as webhook.
    3. Provide the Endpoint URL.
    4. The checkbox to Enable SSL verification is checked by default.

      Important

      SSL is essential for protecting the data sent to the integration endpoint. SSL should always be used when integrating Red Hat Hybrid Cloud Console to third party applications.

    5. Provide a Secret token, if required.

      Note

      If defined, the Secret token is used as an ‘X-Insight-Token’ header on the POST HTTP request.

    6. Click Save.

The new integration is enabled by default and available as an integration option when a Notifications administrator configures behavior groups in the notifications service. In order to disable the integration, use the toggle button on the Integrations list, Enabled column.

Chapter 4. Configure notification behavior groups

The Notifications administrator configures notifications for the account through behavior groups. After creating a behavior group, the Notifications administrator associates it with triggering events, which are unique to each application bundle.

When an event occurs, all users on the account who selected in their user preferences to receive notifications will receive them, as well as all third-party applications specified as integration actions in the behavior group.

4.1. Creating a behavior group

Use the Create behavior group wizard to create a new behavior group. The wizard enables you to select notifications, assign notifications to users or groups of users, and associate Insights events with behavior groups.

Note

Events can be associated with multiple behavior groups. If a behavior group is not associated with any event, no notifications are sent when an event occurs.

You may create a maximum of 64 behavior groups within an organization.

Prerequisites

  • An Organization Administrator has configured notification groups, roles, and members in User Access.
  • You must be logged into the Red Hat Hybrid Cloud Console platform.
  • You have Notifications administrator privileges configured in User Access.

Procedure

  1. Navigate to Insights for Red Hat Enterprise Linux > Settings.
  2. In the Settings menu, expand Notifications and select an application bundle, such as Red Hat Enterprise Linux.
  3. Click Create new group. The Create behavior group wizard opens.

    img hcc notif create behavior group wizard1
  4. Enter a group name, and then click Next. The Actions and recipients page appears.
  5. From the Actions drop-down list, select Send an email, or select an integration from the list.

    img hcc notif create behavior group wizard2
    • If you selected Send an email, select a recipient from the Recipient drop-down list. You may specify that the email notification should go to all users in the organization, or you can limit the recipients to administrators in the organization. If you have User Access groups configured, you may select a User Access group to receive email notifications.
    • If you selected an integration (such as Integration: Webhook or Integration: Splunk), select an integration endpoint from the Recipient drop-down list (for example, SPLUNK_AUTOMATION).

      Note

      You configure endpoints for each integration in Settings > Integrations. If an action is disabled, no integrations exist for that event type.

    • To add additional actions to notify additional integration endpoints, click Add action.
  6. When you have finished adding actions, click Next. The Associate event types page appears.

    img hcc notif create behavior wizard4
  7. Select the types of events that you want to include in your notifications. When you have finished selecting event types, click Next. The Review page appears.

    img hcc notif create behavior wizard5
  8. Review the settings you selected for the behavior group. To modify the settings, click Back. To save the settings and create the behavior group, click Finish.

Additional resources

4.2. Modifying a behavior group

Use the Edit behavior group wizard to modify settings for an existing behavior group. The wizard enables you to select notifications, assign notifications to users or groups of users, and associate Red Hat Insights events with behavior groups.

Note

Events can be associated with multiple behavior groups. If a behavior group is not associated with any event, nothing happens when an event occurs.

Prerequisites

  • An Organization Administrator has configured notification groups, roles, and members in User Access.
  • You must be logged into the Red Hat Hybrid Cloud Console platform with Notifications administrator privileges.

Procedure

  1. Navigate to Insights for Red Hat Enterprise Linux > Settings.
  2. In the Settings menu, expand Notifications and select an application bundle, such as Red Hat Enterprise Linux.
  3. Click Behavior Groups to display the available behavior groups.
  4. Click the More options menu (image::hcc-icons-options.png[]) in the top right corner of the behavior group you want to modify. The Edit behavior group wizard opens.

    img hcc notif edit behavior group wizard1
  5. Enter a group name, and then click Next. The Actions and recipients page appears.
  6. From the Actions drop-down list, select Send an email, or select an integration from the list.

    img hcc notif edit behavior group wizard2
    • If you select Send an email, select a recipient from the Recipient drop-down list. You may specify that the email notification should go to all users in the organization, or you can limit the recipients to administrators in the organization. If you have User Access groups configured, you may select a User Access group to receive email notifications.
    • If you select an integration (such as Integration: Webhook or Integration: Splunk), select an integration endpoint from the Recipient drop-down list (for example, SPLUNK_AUTOMATION).

      Note

      You configure endpoints for each integration in Settings > Integrations.

    • To add additional actions to notify additional integration endpoints, click Add action.
  7. When you have finished adding actions, click Next. The Associate event types page appears.

    img hcc notif edit behavior group wizard3
  8. Select the types of events that you want to include in your notifications. When you have finished selecting event types, click Next. The Review page appears.

    img hcc notif edit behavior group wizard4
  9. Review the settings you selected for the behavior group. To continue to modify the settings, click Back. To save the settings for the behavior group, click Finish.

Chapter 5. Configure user preferences

Each user on the Red Hat Hybrid Cloud Console account must opt in to receive email notifications. If you don’t set your user preferences, you will not receive emails about events.

Select the services from which to receive the notifications, and the frequency: instantly (after each triggered event) or as a daily digest.

Important

Selecting Instant notification for any service can potentially result in receiving a very large number of emails.

5.1. Configuring user preferences for email notifications

Each user configures their own preferences for receiving emails about event-driven system changes.

Prerequisites

  • You must be a registered user and logged into the Red Hat Hybrid Cloud Console platform.

Procedure

  1. Locate your user name in the upper-right part of the application window.
  2. Click the arrow to the right of your username and select User Preferences.
  3. In the left navigation panel, click Notifications and select the appropriate application bundle, for example Red Hat Enterprise Linux or OpenShift.
  4. Select an email preference for each service.

    Important

    Selecting Instant notification for any service can potentially result in receiving a very large number of emails.

  5. Click Save.

Email notifications are delivered in the format and frequency you select.

Note

If you decide to stop receiving notifications, select Unsubscribe from all, and click Save. You will no longer receive any email notifications unless you return to this screen and subscribe to them once again.

Chapter 6. Installing and configuring Red Hat Insights application for Splunk

6.1. About the Red Hat Insights application for Splunk

The Red Hat Insights application for Splunk forwards selected Insights events to Splunk. The application seamlessly integrates with Red Hat Insights, so that you can focus on handling the data on the Splunk application side, in the same way you manage other sources of data.

6.1.1. Contacting support

If you have any issues with the Red Hat Insights application for Splunk, contact Red Hat for support at https://access.redhat.com. Splunk will not provide warm transfers or basic troubleshooting. The Red Hat Insights application for Splunk is fully supported by Red Hat.

6.1.2. Prerequisites

  • sc_admin (Splunk Cloud Administrator) role to install apps on Splunk Cloud Platform.
  • admin-level access to install apps in Splunk Enterprise.
  • Org Admin-level login access to Red Hat Insights.

6.1.3. Additional resources

6.2. Installing the Red Hat Insights application for Splunk

Prerequisites

Procedure

  1. Navigate to the Splunk home page.
  2. Click Settings (gear icon) in the left panel to manage apps. The Apps page opens.
  3. Use the Search field to search for Red Hat Insights Application for Splunk. The application appears in the search results.
  4. Select the application.
  5. Click Install. When the installation process completes, the message Install successful displays.
  6. Click Set up now. The Set up integration with Red Hat page displays. The page includes the HTTP Event Collector (HEC) name and Default index fields.

    img evnt splunk start2

  7. Open Splunk in a second browser window or tab.
  8. In the second Splunk page, click the Settings drop-down menu in the upper right of the page and select Indexes.

    Note

    Indexes is located in the Data section of the Settings drop-down menu.

  9. Click New Index.
  10. Type a name for the index in the Name field (for example, redhatinsights). Leave the rest of the fields blank.
  11. Click Save. The index you created appears in the Indexes list.
  12. Click Enable next to the name of the new index to enable it.
  13. Navigate back to the first Splunk screen with the Set up integration with Red Hat page.
  14. Type the name for the HEC in the HEC name field (for example, redhatinsights).
  15. Type the name of the index you just created in the Default index field (for example, redhatinsights).
  16. Click Next.
  17. Click Review.
  18. Click Submit. The HEC name you created appears in the HEC Name field.
  19. Click Next to create the HEC URL and HEC Token.

    img evnt create hec

  20. Click Next: Configure Splunk integration in Insights. This button is disabled until you click on a Copy button for either the HEC URL or HEC token. This opens console.redhat.com in a new browser tab.

    Note

    If the new tab for console.redhat.com does not open, disable the popup blocker in your browser.

  21. Click Copy to copy the HEC URL value in Splunk Enterprise, and then paste it into the Splunk HEC URL field on the Integrations page in console.redhat.com.
  22. Add the port, if needed. The default port for Splunk Cloud Platform is 443. The default port for Splunk Enterprise and Splunk Cloud free trial is 8088.

    img evnt paste hec in hcc2

Note

If you are using Splunk Cloud, configure the HEC URL to follow Splunk Cloud format. If you are using Splunk Enterprise, skip the following section.

6.2.1. Configuring the HEC URL in Splunk Cloud

The HEC URL in Splunk Cloud Platform takes the following standard form:

<protocol>://http-inputs-<host>.splunkcloud.com:<port>/<endpoint>

For Splunk Cloud Platform on Google Cloud Platform (GCP), the HEC URL standard form is slightly different:

<protocol>://http-inputs.<host>.splunkcloud.com:<port>/<endpoint>
  • Edit the HEC URL you just pasted into the Splunk HEC URL field on the Integrations page.

    Substitute your values for the following:

    • protocol: either http or https
    • host: name of the Splunk Cloud Platform instance that runs the HEC, followed by the domain .splunkcloud.com.
    • port: the HEC port number (443 by default on Splunk Cloud Platform instances).
    • endpoint: the HEC endpoint you want to use. In many cases, you use the /services/collector/event endpoint for JavaScript Object Notation (JSON)-formatted events, or the services/collector/raw endpoint for raw events.
Note

The HEC URL for AWS has a slightly different format than the HEC URL on GCP.

  • Add http-inputs- before the host on AWS.
  • Add http-inputs. before the host on GCP.
  • In both cases, add the domain .splunkcloud.com after the host value.

Examples

  • Splunk Cloud Platform on GCP using JSON:

    https://http-inputs.myhost.splunkcloud.com:443/services/collector/event
  • Splunk Cloud free trial on AWS using raw events:

    https://http-inputs-otherhost.splunkcloud.com:443/services/collector/raw

6.2.2. Completing the setup process

  1. Copy the HEC Token value in Splunk. Paste it into the Splunk HEC Token field in console.redhat.com.
  2. In console.redhat.com, click Run configuration. Red Hat Insights sets up the integration, creates the behavior group, and associates Insights events to the behavior group. The status message section on the right side of the page shows the status of each of these actions.

    img evnt run config in hcc

  3. When the setup completes successfully, click Next: Review. The application returns the message Splunk integration in Insights completed.

    img evnt hcc config complete

  4. Click Go back to the Splunk application. This redirects you to the Set up integration with Red Hat screen in Splunk.
  5. Click Finish set up to complete the setup in Splunk.

    img evnt splunk setup complete

  6. Click Go to dashboard to be redirected to your Splunk dashboard.

    img evnt splunk dashboard

Note

If the integration configuration fails during the Insights setup process, contact Red Hat support.

The setup automation performs the following tasks:

  • Creates a user group with Notifications Administrator role using the Organization Administrator permissions. You can also create the user group manually. For more information about manual configuration, see Manually configuring a Notifications Administrator group in your Insights account.
  • Uses the Splunk HEC URL and HEC token to create a new integration called SPLUNK_AUTOMATION, with the integration type Splunk.
  • Creates a new Behavior Group called SPLUNK_AUTOMATION_GROUP on the RHEL bundle. The group includes an action to send notifications to the SPLUNK_AUTOMATION Splunk integration.
  • Assigns the new Behavior Group SPLUNK_AUTOMATION_GROUP to all Insights services. This forwards the events from all services to Splunk. Currently, the Behavior Group forwards events from the Advisor, Policies, and Drift services.

When Splunk begins to receive notifications from Insights, the Red Hat Insights application for Splunk dashboard shows event activity. Each number contains a hyperlink to Insights.

img evnt splunk dashboard2

To view a list of Insights events on the Splunk dashboard, click the Events tab. Each event is hyperlinked to Insights.

splunk events log

6.3. Enabling the HEC token

Before Splunk can receive Insights events, you must enable the HEC token.

Prerequisites

  • An Organization Administrator-level login to Red Hat Insights.
  • You must have the admin role in Splunk Enterprise, or the sc_admin role in Splunk Cloud.

Procedure

  1. From the Splunk main page, navigate to Settings.
  2. Select Data Inputs, and then select HTTP Event Collector. The HTTP Event Collector page shows the HEC, its Token value, the corresponding index that you selected during setup, and the status of the HEC.

    img evnt splunk hec screen

  3. Click Global Settings in the upper right corner of the page. The Edit Global Settings dialog box displays.

    img evnt splunk global settings

  4. Select Enabled. This enables the HEC token that was automatically created during the setup process.

The HEC token uses a default HTTP port number of 8088. If you are using a different port, you must update your Insights Splunk Integration to match.

Additional Resources

6.4. Manually configuring a Notifications Administrator group in your Insights account

Important

The Red Hat Insights application for Splunk automated installation/setup process automatically configures a Notifications Administrator role and group in your Insights account. Use this procedure only if you want to manually create the Notifications Administrator role and group.

Procedure

  1. Navigate to Settings, and then select My User Access from the drop-down menu.
  2. Select Groups. The Groups page appears.
  3. Click Create Group. The Name and Description page appears.
  4. Create a name for the group (for example, splunknotifgroup) and click Next. The Add Roles page appears.
  5. To create the Notifications Administrator role, click in the Search field and type notif.
  6. Select Notifications Administrator from the search results, and then click Next. The Add Members page appears.
  7. Select the group members from the list who should have Notifications Administrator role.
  8. Click Next. The Review Details page appears.
  9. Review the details: Group Name, Role, and Members, and then click Submit.

Insights verifies the details, and then the new group appears on the Groups page. The Success adding group message displays. Group members (for example, members of splunknotifgroup) can now configure notifications and integrations.

6.5. Manually configuring a Splunk Integration

Important

The Hybrid Cloud Console application for Splunk automated installation/setup process automatically configures Splunk integration to your Insights account. Use this procedure only if you want to configure the integration manually.

Prerequisites

  • HEC URL from Splunk Cloud or Splunk Enterprise
  • HEC token value from Splunk Cloud or Splunk Enterprise
  • Notifications Administrator access to Red Hat Insights

Procedure

  1. Navigate to Settings, and select Integrations from the drop-down menu. The Integrations page appears.
  2. Click Add Integration. The Add Integration dialog box appears.

    img evnt add integration

  3. Click Type from the drop-down menu and select Splunk.
  4. Type a name for your new integration into the Integration Name field (for example, redhat_splunk).
  5. In the Endpoint URL field, add your Splunk HEC endpoint URL.

    1. For Splunk Enterprise, Splunk uses port 8088 by default. For example: https://<splunk-endpoint>:8088
    2. For Splunk Cloud, Splunk uses port 443. For more information about Splunk Cloud on AWS or GCP, see Send data to HTTP Event Collector.

      Important

      The service automatically adds <endpoint> (the http path). You do not need to include it in the form input for the Endpoint URL.

  6. In the Secret token field, add the Splunk HEC token value.
  7. Optional. Add any notes or other information about this integration to the Extras field.
  8. Click Save.

These examples show endpoint URLs with the correct port numbers for Splunk platforms.

Additional resources

6.6. Troubleshooting integration with Splunk

Here are some common configuration errors in the Splunk environment that could result in Splunk not receiving events from Red Hat Insights:

  • Make sure the HEC is enabled (under Global Settings). See Enabling the HEC.
  • Make sure that the default index has not changed for the HEC (it should be redhatinsights).
  • Make sure the firewall allows for incoming requests on the configured Splunk HEC port (default is 8088). If you are using AWS for your instance, allow any of the ports Splunk may need. For more information, refer to Splunk Phantom ports and endpoints.

6.6.1. Events show as sent within Insights but do not appear in Splunk

  • Check your firewall for where your Splunk setup resides.
  • Ensure that the Splunk port is allowed (port 8088 by default).

Additional resources

Chapter 7. Installing and configuring the ServiceNow Flow Templates for Red Hat Insights

7.1. About the ServiceNow flow templates for Red Hat Insights

The Flow Templates for Red Hat Insights application integrates with Insights for Red Hat Enterprise Linux services. The templates provide ServiceNow Flows for creating Incidents out of found vulnerabilities, performance, system configuration recommendation, and other risks. The application includes a sample flow that you can edit to customize it for your organization.

This Insights for RHEL application forwards selected Insights events to ServiceNow. The application seamlessly integrates with Insights for Red Hat Enterprise Linux, so that you can focus on handling the data on the ServiceNow application side, in the same way you manage other sources of data.

Before data flow can take place, you must install the ServiceNow IntegrationHub Enterprise Pack Installer plugin. Once the plugin and the application are configured, event data flows from the Insights Hybrid Cloud Console to your ServiceNow instance. A ServiceNow REST API - Asynchronous trigger receiver is used within the application flow.

This version of the application supports handling events from the following Insights services:

  • Advisor
  • Vulnerability
  • Any additional events that you might have configured

Install the Flow Templates for Red Hat Insights application from the ServiceNow Store.

Insights is included as part of your Red Hat subscription, and is accessible through Red Hat Hybrid Cloud Console.

7.2. Contacting support

If you have any issues with the Red Hat Insights application for ServiceNow, contact Red Hat for support at access.redhat.com. ServiceNow will not provide troubleshooting. The Red Hat Insights application for ServiceNow is fully supported by Red Hat.

Additional resources

7.3. Installing and configuring the ServiceNow Flow Templates for Red Hat Insights

Prerequisites

  • You have an Org Admin-level login to Insights for Red Hat Enterprise Linux.
  • Notifications Admin-level permissions are configured in User Access.
  • Ensure that you have a Red Hat Subscription, and that you can access Red Hat Hybrid Cloud Console.
  • Popup blockers are disabled in your browser.
  • ServiceNow IntegrationHub Enterprise Pack Installer plugin is installed. If it is not installed, request the plugin from ServiceNow.
  • ServiceNow Roles required for installation: admin, x_rhtpp_rh_webhook.rest, sn_appclient.app_client_company_installer (can only install applications that match the instance company), or sn_appclient.app_client_user.
  • ServiceNow: Incident (write) access for table permissions.
  • You are using the San Diego or later release of ServiceNow.

Procedure

  1. Log in to your ServiceNow instance.
  2. Navigate to the ServiceNow Store home page.

    1. Install the Flow Templates for Red Hat Insights from the ServiceNow Store (or as an Update Set).
    2. Create a new user with User ID rh_insights_integration.
    3. Check Internal Integration User for the user you just created.
    4. Ensure that the user is Active.
    5. Assign role x_rhtpp_rh_webhook.rest to the user.
    6. Generate a password for the user. Copy this information for use during the setup process.
  3. Open console.redhat.com in a new browser window or tab.
  4. Navigate to Red Hat Hybrid Cloud Console > Settings > Integrations. For more information about how to set up integrations, see Configuring integrations.

    1. Click Add integration.
    2. Create a name for the integration, such as ServiceNow integration.
    3. Select the ServiceNow integration type.
    4. Provide this Endpoint URL and replace <instance.servicenow.com> with your ServiceNow instance: https://<instance.servicenow.com>/api/x_rhtpp_rh_webhook/flow_templates_for_red_hat_insights.
    5. The checkbox to Enable SSL is checked by default.
    6. In the Secret token field, paste the generated password of the rh_insights_integration user that you created in ServiceNow.
  5. Navigate to Red Hat Hybrid Cloud Console > Settings > Notifications.
  6. Select Notifications, and then select Red Hat Enterprise Linux.
  7. Click Create new group. The Create new behavior group dialog box appears.
  8. Type the name of the new behavior group in the Name field and click Next.
  9. For Actions, select Integration: ServiceNow from the drop-down list.
  10. For Recipient, select the integration for ServiceNow that you created earlier (for example, ServiceNow integration).
  11. Click Next. The Associate event types screen appears.
  12. Select the following event types from the list:

    • Advisor new recommendation
    • New vulnerability with CVSS >= 7.0
    • New vulnerability with Critical Severity
    • New vulnerability containing Security rule
    • Any vulnerability with known exploit
  13. Click Next, and then click Finish to complete the setup process.

Verification

To confirm that the application has been configured successfully, view the Event Log on the Red Hat Hybrid Cloud Console, and view Flow Executions on your ServiceNow Instance (Process Automation > Flow Administration > Today’s Executions).

Note that events are generated on certain conditions (for example, when a system configured with Red Hat Insights checks in). If no events appear in ServiceNow, check the Event log to see whether any events matching the conditions have occurred.

Note

If the integration configuration fails during the Insights setup process, contact Red Hat support.

7.4. Troubleshooting integration with ServiceNow

If ServiceNow is not receiving events from Insights, check these configuration steps.

  • Ensure that the integration in the Red Hat Hybrid Cloud Console is enabled and has type ServiceNow.
  • Ensure that the integration in the Red Hat Hybrid Console has a correct URL. The URL should start with https://<instance.servicenow.com>/api/x_rhtpp_rh_webhook/flow_templates_for_red_hat_insights.
  • Ensure that the x_rhtpp_rh_webhook.rest` user role is defined in ServiceNow. Otherwise, notifications from Insights will not work even if the application has been installed correctly.
  • Ensure that the rh_insights_integration ServiceNow user exists, is active, and has the x_rhtpp_rh_webhook.rest role assigned.
  • If necessary, reset the password for the rh_insights_integration ServiceNow user, and reset the user password in the integration on the Red Hat Hybrid Cloud Console.

Chapter 8. Configuring Red Hat Insights integration with Slack

8.1. About the Red Hat Insights integration with Slack

You can configure Red Hat Insights to send event notifications to a new or existing Slack channel. This enables you to send notifications to a selected user, or to all users on a channel. The Slack integration supports events from all Insights services.

Note

The Slack integration in this example is configured for Red Hat Enterprise Linux. The integration also works with other Red Hat OpenShift, Application Services, and Hybrid Cloud Console events.

The Slack integration uses Incoming Webhooks to receive event data. For more information about webhooks, see https://api.slack.com/messaging/webhooks.

8.1.1. Contacting Support

If you have any issues with the Red Hat Insights integration with Slack, contact Red Hat for support at access.redhat.com. Slack will not provide troubleshooting. The Red Hat Insights integration with Slack is fully supported by Red Hat.

8.2. Configuring Slack integration

Note

Sending messages using Incoming Webhooks is a legacy application. The preferred method to integrate Slack is to build your own custom workflow as shown in the Slack help center at https://slack.com/help. However, integrating Red Hat Insights using this method is not yet available because custom workflows in Slack do not yet support nested JSON structures as workflow variables.

Prerequisites

  • Owner or admin access to the Slack instance where you want to add Incoming Webhooks.
  • App Manager permissions to add Slack apps to a channel.
  • A Slack channel or user to receive the notifications.

Procedure

  1. In the Slack application, navigate to the channel or user name to receive the notifications.
  2. Click the channel name at the top of the screen. The configuration window appears.
  3. Select Integrations, and then click Manage Apps. The Add apps to channel window appears.
  4. Search for Incoming Webhooks, and then select it from the search results.
  5. Click Add to Slack to add the Insights integration to the channel. The Post to Channel box appears.
  6. Click the Choose a Channel drop-down menu to select an existing channel, or type the user or channel name in the field. To create a new channel, click or create a new channel, and then type the name of the new channel and an optional description. Click Create.

    Note

    The field requires a channel name. If you do not specify a channel, Slack integration uses #general.

    Note

    If you do not have App Manager permissions to add the app to the channel, click the Request Configuration button to send an add request to the App Manager for the channel.

  7. Click Add Incoming Webhooks Integration. The Configuration page for the app appears in the Slack app directory.

    img notif incoming webhooks

    img notif integration settings modified

  8. Scroll down to Integration Settings. The generated Webhook URL appears in the Webhook URL field.
  9. Copy the URL displayed in the field. You will use it to set up the integration in the Red Hat Hybrid Cloud Console.
  10. Optional. Add a name for the integration in the Customize Name field.
  11. Optional. Upload an icon in the Customize Icon field.
  12. Click Save Settings.

8.3. Configuring the Slack integration in the Console

Prerequisites

  • Organization administrator or Notifications administrator access to the {HCC}.

Procedure

  1. Navigate to the Red Hat Hybrid Cloud Console.
  2. Click the gear icon to select Settings, and then select Integrations from the menu on the left side of the screen.
  3. Click Add Integration.
  4. Create a name for the integration (for example, SLACK_INTEGRATION).
  5. Click the Type drop-down menu and select Slack.

    img notif add integration

  6. Paste the URL that you copied from the Webhook URL field during Slack side setup into the Endpoint URL field.
  7. Add the name of the Slack channel configured during Slack side setup to the Channel field.
  8. Click Save. The Last Connection Attempt state takes a few minutes to process the change.
  9. Navigate to Notifications > Red Hat Enterprise Linux.
  10. Navigate to Behavior Groups to add the new endpoint to an existing behavior group, or to create a new behavior group. For more information about creating behavior groups, see Configure notification behavior groups.
  11. Click Edit Behavior Group.
  12. Select Integration: Slack in the Actions column.

    img notif actions recipients

  13. The list of configured Slack integrations shows in the Recipient drop-down list. Select the integration name that you previously created.

    img notif add slack channel

  14. Click Next. The Associate event types screen appears.
  15. Select the event types for which you want notifications.
  16. When you have finished selecting event types, click Next. The Review screen appears.
  17. Review the settings for the behavior group and then click Finish.

    Note

    You can create and edit multiple behavior groups to include any additional platforms that the Notifications service supports.

  18. Return to Settings > Integrations. When the Slack integration is ready to send events to Slack, the Last connection attempt column shows Ready. If the notification reached Slack successfully, the Last connection attempt column shows Success.

    img notif ready success

When an event is triggered, Insights sends the notification to the Slack channel. The notification includes the name of the associated service, as well as that of the system that triggered the notification. To view more details, click on the hyperlinks.

Additional resources

Verification

To verify that events are being sent to the Slack channel, look in the Event Log. To view the Event Log, navigate to Settings > Notifications > Red Hat Enterprise Linux and click View event log.

Integration: Slack appears in the Actions column for the event. If the notification was successful, the integration shows green. Roll over the integration to verify that an event was sent.

+ img notif slack message output2

Chapter 9. Troubleshoot notification failures with the event log and integration settings

Troubleshoot notification failures with the event log and integration settings The notifications service event log enables Notifications administrators to easily see when notifications are not working properly. The event log provides a list of all triggered events on the Red Hat Hybrid Cloud Console account, and actions taken (as configured in the associated behavior group) for the past 14 days.

img hcc notif event log

In the Actions column, each event shows the integration type highlighted in green or red. These visual color codes indicate the status of the message transmission: success or failure, respectively.

The following use cases illustrate the troubleshootings capabilities of the event log:

Check an endpoint configuration to troubleshoot a degrading connection.

The filterable event log is a useful troubleshooting tool to see a failed notification event and identify potential issues with endpoints. After seeing a failed action in the event log, the Notifications administrator can check the endpoint and the status of the last five connection attempts on the Integrations screen.

In the integrations service, the following connection statuses are reflected by color:

  • Green: Five previous transmissions were successful
  • Red: Five previous transmissions were unsuccessful (timeout, 404 error, etc)
  • Yellow: Connection is degraded; at least two of the five previous transmissions were unsuccessful
  • Unknown: The integration has not yet been called, or is not associated with a behavior group

Determine whether a user’s non-receipt of emails is a configuration or user error.

The event log can answer questions related to receipt of emails. By showing the email action for an event as green, the event log enables a Notifications administrator to confirm emails were sent successfully. An issue with the receipt of notification emails may be with individual user preferences and not with notification configuration.

Important

Even with notifications and integrations configured properly, individual users on the Red Hat Hybrid Cloud Console account must configure their User Preferences to receive emails.

9.1. Checking for connection failures in the event log

Use the following procedure to check for notification action failures.

Prerequisites

  • You must be logged into Red Hat Hybrid Cloud Console with the Notifications administrator privileges configured in User Access.
  • Before users receive notifications using the webhook integration type, endpoints for your organization’s preferred webhook application must be configured by a Notifications administrator in the integrations service.
  • Before users receive email notifications, they must configure their personal email notification preferences for each Hybrid Cloud Console application bundle.

Procedure

  1. Navigate to Red Hat Hybrid Cloud Console > Settings > Notifications > Event log.
  2. Filter the events list as needed by event, application, or application bundle and select to show events from today, yesterday, the last seven days, the last 14 days (default), or set a custom range within the last 14 days.
  3. Sort the Date and time column as needed in ascending or descending order.
  4. If users of the application bundle are not receiving the notifications as intended, you can click View notification settings and change settings and/or have users check their user preferences for receiving email notifications.

Providing feedback on Red Hat documentation

We appreciate your feedback on our documentation. To provide feedback, highlight text in a document and add comments.

Prerequisites

  • You are logged in to the Red Hat Customer Portal.
  • In the Red Hat Customer Portal, the document is in the Multi-page HTML viewing format.

Procedure

To provide your feedback, perform the following steps:

  1. Click the Feedback button in the top-right corner of the document to see existing feedback.

    Note

    The feedback feature is enabled only in the Multi-page HTML format.

  2. Highlight the section of the document where you want to provide feedback.
  3. Click the Add Feedback pop-up that appears near the highlighted text.

    A text box appears in the feedback section on the right side of the page.

  4. Enter your feedback in the text box and click Submit.

    A documentation issue is created.

  5. To view the issue, click the issue link in the feedback view.

Legal Notice

Copyright © 2023 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.