Menu Close

Configuring notifications and integrations on the Red Hat Hybrid Cloud Console

Red Hat Hybrid Cloud Console 2022

Configuring Hybrid Cloud Console settings so that account users receive event-triggered notifications about RHEL systems

Red Hat Customer Content Services

Abstract

Using notifications to learn of identified events that have occurred and could impact your organization.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.

Providing feedback on Red Hat Hybrid Cloud Console documentation

We appreciate your input on our documentation. Please let us know how we could make it better. To do so, create a Bugzilla ticket:

  1. Go to the Bugzilla website.
  2. As the Component, use Documentation.
  3. Fill in the Description field with your suggestion for improvement. Include a link to the relevant part(s) of documentation.
  4. Click Submit Bug.

Chapter 1. Introduction to Red Hat Hybrid Cloud Console notifications and integrations

The notifications service on Red Hat Hybrid Cloud Console frees users from having to check their user interface routinely, looking for event-triggered notifications. Instead, when events occur, the service automatically takes the action to send notifications about the events to users.

Important

Groups and roles, including the Notifications administrator role, must be configured by an Organization Administrator in order for events to be reported through email and integrations.

To learn more about User Access on the Red Hat Hybrid Cloud Console platform, see the User Access Configuration Guide for Role-based Access Control (RBAC).

1.1. What the notifications service does

Through the notifications service, Red Hat Hybrid Cloud Console applications and services have a standardized way of notifying users of events. By setting up behavior groups, a Notifications administrator specifies the notification delivery method, and whether event notifications are sent to all users on an account or only to Organization Administrators.

For example, the Notifications administrator can configure the service to send an email notification for (advisor service) new-recommendation hits on a system. Similarly, the administrator might decide to trigger a notification that sends a message to a third-party application using the webhook integration type.

An Organization Administrator designates Notifications administrators by creating a User Access group with the Notifications administrator role, then adding account members to the group. A Notifications administrator then configures notification behavior groups that define actions taken when service-specific events occur.

The notifications service transmits event-triggered notifications to users’ email accounts, or to third-party applications using webhooks. Users on the Hybrid Cloud Console account set their own preferences for receiving email notifications. In User preferences > Notifications > application bundle, each user configures their personal settings to receive event notification emails as an instant notification or daily digest.

Important

Selecting Instant notification for any service can potentially result in receiving a very large number of emails.

1.2. Notification and integration concepts

The following table defines terms that are important for understanding how the notifications service works:

Table 1.1. Notifications concepts

ConceptDescription

Actions

Operations performed in response to an event. Examples include sending an email or forwarding a notification to an integration endpoint, such as a webhook. Actions are defined in behavior groups configured by a Notifications administrator.

Application bundle

Application bundle refers to an application group within the Hybrid Cloud Console, such as Red Hat Enterprise Linux (Insights for RHEL) or OpenShift.

Behavior groups

Behavior groups determine what actions to take when an event occurs, and whether to notify all account users or only designated administrators. Once a Notifications administrator creates a behavior group, they associate it with event types, enabling Notifications administrators to apply the same actions to all application-specific events.

NOTE: Notifications administrators configure notification behavior groups separately, for each application bundle.

Email preferences

Individual users with access to applications on the Hybrid Cloud Console set their personal email preferences. Users can configure personal email notifications to arrive either instantly, as the event occurs, or consolidated into a daily digest that arrives at midnight, 00:00 Coordinated Universal Time (UTC), for all accounts.

IMPORTANT: Selecting Instant notification for any service can potentially result in receiving a very large number of emails.

Event type

Event types are application-specific system changes that trigger the application or service to initiate notification actions. Event types are created by application developers at Red Hat and are unique for each application bundle. Examples from the Insights for RHEL (Red Hat Enterprise Linux) application bundle include:

* Policies service: Policy triggered

* Drift service: Drift from baseline detected

* Advisor service: New recommendation; Resolved recommendation

Integrations

Integrations define the method of delivery of notifications to third-party applications configured by the Notifications administrator. Once configured, the notifications service sends the HTTP POST messages to endpoints.

User access roles

The following User Access roles interact with notifications:

* Organization Administrator

* Notifications administrator

* Notifications viewer

1.3. Insights events and notifications options

There are three main ways to integrate Insights into your organization’s workflows:

  • Using the Insights APIs
  • Using webhooks and/or emails directly to users
  • Using integrations with a third-party application, such as Splunk

1.3.1. Using the Insights APIs

Insights APIs are publicly available and can be queried from any authenticated client (RBAC controlled).

For more information about the available endpoints for applications and services, refer to the Red Hat Insights API documentation. For an example of CSV-formatted responses, see the System Comparison API Documentation.

The Red Hat Insights API cheat sheet covers the use of Insights APIs. It provides examples to help you to get started quickly with authentication and with querying the endpoints. Most of the examples use the curl command, but this document also includes sample code for clients that are written in Python, and an Ansible playbook that performs similar operations.

1.3.2. Using Integrations webhooks

Webhooks work in a similar way to APIs, except that they enable one-way data sharing when events trigger them. APIs share data in both directions. Applications that allow inbound data requests are said to have exposed webhooks.

You can configure Insights to send POST messages to specific endpoints in exposed webhooks within applications. This capability works in concert with the Notifications service. For example, you can configure Insights to automatically email new Advisor recommendations to selected administrator accounts as soon as Insights identifies them.

Once you configure the endpoints in Notifications, you can subscribe to a stream of Insights events and automatically forward that stream to the webhook(s) of your choice. Each event contains additional metadata, which you can use to process the event (for example, perform specific actions and/or trigger responses) as part of your operational workflow. You configure the implementation and data handling within your application.

For more information about how to configure and use webhooks, refer to Configure integrations.

1.3.3. Using a third-party application

Insights third-party application integrations work in one of two ways, depending on the use case:

  • Using Insights APIs to collect data and perform tasks
  • Subscribe to streams of Insights events

You can use Red Hat Insights integrations to forward events to specific third-party applications. In this version of Red Hat Insights, the Red Hat Insights application for Splunk forwards selected Insights events to Splunk. This allows you to view, and use Insights data, in your existing workflows from the Red Hat Insights application for Splunk dashboard.

For more information about the Red Hat Insights application for Splunk, see Installing and Configuring the Red Hat Insights application for Splunk.

Chapter 2. Configure User Access

Before account users can configure notifications and integration settings, a group with the Notification administrator role must be configured in User Access by an Organization Administrator. In User Access > Groups, an Organization Administrator performs the following high-level steps:

  • Create a User Access group for Notifications administrators.

    • Add the Notifications administrator role to the group.
    • Add members (users with account access) to the group.

Organization Administrator

The Organization Administrator configures the User Access group for Notifications administrators, then adds the Notifications administrator role and users to the group.

Notifications administrator role

Notifications administrators configure how applications interact with notifications. Notifications administrators configure behavior groups to define how applications notify users about events. Administrators can configure additional integrations as they become available, as well as edit, disable, and remove existing integrations.

Notifications viewer role

The Notifications viewer role is automatically conferred to everyone on the account and limits how a user can interact with notifications service views and configurations. A viewer can view notification configurations, but cannot modify or remove them. A viewer also cannot configure.r modify, or remove integrations.

For learn more about User Access on the Red Hat Hybrid Cloud Console platform, see the User Access Configuration Guide for Role-based Access Control (RBAC).

2.1. Creating and configuring a notifications group in User Access

The following procedure shows how an Organization Administrator on the account creates a group with the Notifications administrator role and adds members to the group.

Prerequisites

  • You must be logged into your Red Hat Hybrid Cloud Console account as an Organization Administrator.

Procedure

  1. Click the gear icon in the upper right quadrant of the application window and select Settings.

    img hcc toolbar settings

  2. From the Settings menu on the left, click User Access and select Groups.
  3. Click Create group.
  4. Enter a group name, for example, Notifications Administrators, and a description, then click Next.
  5. Select the role to add to this group, in this case Notifications administrator. Click the checkbox for that role, then click Next.
  6. Add members to the group. Search for individual users or filter by username, email, or status. Check the box next to each intended member’s name, then click Next.
  7. Review the details to make sure everything is correct. Click Back if you need to go back and change something.
  8. Click Submit to finish creating the group.

2.2. Editing or removing a User Access group

If you need to edit or remove a User Access group, perform the following steps:

Prerequisites

  • You must be logged into your Red Hat Hybrid Cloud Console account as an Organization Administrator.

Procedure

  1. Locate the group name in User Access > Groups.
  2. Click the menu options icon on the far right of the group name row ( img hcc icons options ), and click Edit or Delete.
  3. Make and save changes or delete the group.

Chapter 3. Configure integrations

The Red Hat Hybrid Cloud Console notifications and integrations services work together to transmit messages to third-party application endpoints, such as instant messaging platforms and external ticketing systems, when triggering events occur.

This enables Notifications administrators to integrate Hybrid Cloud Console functionality into the operational workflow used in their organization. Integrations are configured by a Notifications administrator as endpoints in Red Hat https://console.redhat.com[Red Hat Hybrid Cloud Console > Settings > Integrations].

Note

Webhook is the current integration type supported in the Red Hat Hybrid Cloud Console platform. When configured, the service sends an HTTP POST message to the specified third-party applications endpoint.

3.1. HTTP POST messages

The following screenshot is an example of an HTTP POST message sent to a third-party application endpoint. Event types are specific to a service or application. For example, the Insights for RHEL application bundle currently notifies configured users of events from the policies, advisor, and drift services. The following example notification from the Insights for RHEL advisor service was triggered by a new recommendation on a host system.

img hcc notif schema

In the example, the blocks contain the following information:

  1. Information about the bundle and application sending the notification

    • bundle: Name of the application bundle
    • application: Name of the individual application or service sending the event-triggered notification
    • event_type: The event type that triggered the notification
    • account_id: The Red Hat account from which the notification was sent
    • timestamp: ISO-8601 formatted date showing when the notification was sent
  2. Information about the application or service-specific event

    • payload: The application payload, a JSON string containing all the data sent by the application
  3. Information about the system on which the event occurred. For example:

    • inventory_id: System ID
    • hostname: System name
    • rhel_version: RHEL version running on the system

The metadata field is not currently being used.

3.2. Setting up integrations

The Notifications administrator sets up integrations for the organization. In addition to adding new integrations, the Notifications administrator can edit, remove, or disable any listed integration by clicking the More options menu icon img hcc icons options , located to the right of the integration name, and then clicking the appropriate option.

Prerequisites

  • To perform the following procedure, a user must be logged into the Red Hat Hybrid Cloud Console platform with Notifications administrator privileges configured in User Access.

Procedure

  1. Navigate to Red Hat Hybrid Cloud Console > Settings > Integrations.
  2. Click Add integration.

    1. Enter an Integration name.
    2. Select an integration Type, such as webhook.
    3. Provide the Endpoint URL.
    4. The checkbox to Enable SSL verification is checked by default.

      Important

      SSL is essential for protecting the data sent to the integration endpoint. SSL should always be used when integrating Red Hat Hybrid Cloud Console to third party applications.

    5. Provide a Secret token, if required.

      Note

      If defined, the Secret token is used as an ‘X-Insight-Token’ header on the POST HTTP request.

    6. Click Save.

The new integration is enabled by default and available as an integration option when a Notifications administrator configures behavior groups in the notifications service. In order to disable the integration, use the toggle button on the Integrations list, Enabled column.

Chapter 4. Configure notification behavior groups

The Notifications administrator configures notifications for the account through behavior groups. After creating a behavior group, the Notifications administrator associates it with triggering events, which are unique to each application bundle.

When an event occurs, all users on the account who selected in their user preferences to receive notifications will receive them, as well as all third-party applications specified as integration actions in the behavior group.

4.1. Creating a behavior group

Use the following procedure to create a new notifications behavior group:

Prerequisites

  • An Organization Administrator has configured notification groups, roles, and members in User Access.
  • You must be logged into the Red Hat Hybrid Cloud Console platform with Notifications administrator privileges.

Procedure

  1. Navigate to Red Hat Hybrid Cloud Console > Settings.
  2. In the settings menu, expand Notifications and select an application bundle, such as Red Hat Enterprise Linux or OpenShift.
  3. Click Create new group.
  4. Enter a group name. From the Actions dropdown list, select from the following options:

    1. Select Send an email, then select whether the email notification should go to all users in the organization, or to limit the recipients to administrators in the organization.
    2. Select Integration: Webhook and select an option from the dropdown list, which is populated by endpoints configured in Settings > Integrations.
    3. You can add additional actions as needed to notify all of your organization’s integration endpoints.
  5. Edit or delete the behavior group as needed by locating it in the Behavior groups list on the notifications page for the application bundle, clicking the options menu, img hcc icons options , and clicking Edit or Delete.

    Note

    Clicking Delete prompts a warning to the user of the consequences of deleting a behavior group assigned to event types.

4.2. Associating a behavior group with events

Use the following procedure to associate a notifications service behavior group with triggering events.

Prerequisites

  • To perform the following procedure, a user must be logged into the Red Hat Hybrid Cloud Console platform with Notifications administrator privileges configured in User Access.
  • A Notifications administrator has already created the behavior group.

Procedure

  1. Navigate to Red Hat Hybrid Cloud Console > Settings.
  2. In the settings menu, expand Notifications and select an application bundle where the behavior group was created, such as Red Hat Enterprise Linux or OpenShift.
  3. Select a behavior group with which to associate events. You can enter the name of the group in the search box, or scroll through the behavior group cards.
  4. In the list of events for the application bundle, click the pencil icon on the far right of the event name row. This will enable a drop-down list in the behavior column for the event.
  5. Click the behavior drop-down list and select the behavior group(s) you want to associate with the event.
  6. Click the checkmark in the event row to accept your selection for that event.
  7. Repeat the previous step for each event.
Note

Events can have multiple behavior groups associated with them. It is also possible that a behavior group is not associated with any event. In that case, when an event occurs, no action is taken.

4.3. Sending notifications to a group using User Access

In addition to sending email notifications to all subscribed users in the groups Users:Admins and Users:All, you can use Role-Based Access Control (RBAC) to send email notifications to a specific list of users in a custom User Access Group (RBAC Group).

First, an Organization Administrator creates User Access Groups and adds group members. A Notifications Administrator then creates a new Behavior Group and assigns one or more custom User Access Groups as the recipients of email notifications from the Behavior Group.

For more information about how to create User Access (RBAC) Groups, see Managing access with roles and members.

For more information about how to create Behavior Groups, see Configuring notification behavior groups.

Prerequisites

  • One or more User Access groups
  • Notifications Administrator-level access to Red Hat Insights

Procedure

  1. Navigate to Red Hat Hybrid Cloud Console > Settings.
  2. In the Settings menu, click Notifications.
  3. Select Red Hat Enterprise Linux. The Notifications behavior groups display.
  4. Click Create new group. The Create new behavior group dialog box appears.

    img notif user access behavior group
  5. Type the name of the new behavior group in the Group Name field.
  6. Click Select Action.

    img notif notifications selection
  7. Select Send an email to send email notifications to the members of the behavior group.
  8. Click Recipient. A drop-down list of existing groups displays. The User Access (RBAC) groups appear under the User Access Group heading.

    img notif select rbac group
  9. Select the group(s) that you want to add to your behavior group and click Save. The new behavior group appears under Behavior Groups. The User Access group(s) you selected appear as recipients in the new behavior group, preceded by User Access Groups:.
  10. After you create the behavior group, associate it with the events from which the group should receive notifications. For more information about how to associate events with recipients, see Associating a behavior group with events.

Additional resources

Chapter 5. Configure user preferences

Each user on the Red Hat Hybrid Cloud Console account must opt in to receive email notifications. If you don’t set your user preferences, you will not receive emails about events.

Select the services from which to receive the notifications, and the frequency: instantly (after each triggered event) or as a daily digest.

Important

Selecting Instant notification for any service can potentially result in receiving a very large number of emails.

5.1. Configuring user preferences for email notifications

Each user configures their own preferences for receiving emails about event-driven system changes.

Prerequisites

  • You must be a registered user and logged into the Red Hat Hybrid Cloud Console platform.

Procedure

  1. Locate your user name in the upper-right part of the application window.
  2. Click the arrow to the right of your username and select User Preferences.
  3. In the left navigation panel, click Notifications and select the appropriate application bundle, for example Red Hat Enterprise Linux or OpenShift.
  4. Select an email preference for each service.

    Important

    Selecting Instant notification for any service can potentially result in receiving a very large number of emails.

  5. Click Save.

Email notifications are delivered in the format and frequency you select.

Note

If you decide to stop receiving notifications, select Unsubscribe from all, and click Save. You will no longer receive any email notifications unless you return to this screen and subscribe to them once again.

Chapter 6. Installing and configuring Red Hat Insights application for Splunk

6.1. About the Red Hat Insights application for Splunk

The Red Hat Insights application for Splunk forwards selected Insights events to Splunk. The application seamlessly integrates with Red Hat Insights, so that you can focus on handling the data on the Splunk application side, in the same way you manage other sources of data.

This version of the application supports events from the following Insights services:

  • Advisor
  • Compliance
  • Drift
  • Policies

6.1.1. Contacting support

If you have any issues with the Red Hat Insights application for Splunk, contact Red Hat for support at https://access.redhat.com. Splunk will not provide warm transfers or basic troubleshooting. The Red Hat Insights application for Splunk is fully supported by Red Hat.

6.1.2. Prerequisites

  • sc_admin (Splunk Cloud Administrator) role to install apps on Splunk Cloud Platform.
  • admin-level access to install apps in Splunk Enterprise.
  • Org Admin-level login access to Red Hat Insights.

6.1.3. Additional resources

6.2. Installing the Red Hat Insights application for Splunk

Prerequisites

Procedure

  1. Navigate to the Splunk home page.
  2. Click Settings (gear icon) in the left panel to manage apps. The Apps page opens.
  3. Use the Search field to search for Red Hat Insights Application for Splunk. The application appears in the search results.
  4. Select the application.
  5. Click Install. When the installation process completes, the message Install successful displays.
  6. Click Set up now. The Set up integration with Red Hat page displays. The page includes the HTTP Event Collector (HEC) name and Default index fields.

    img evnt splunk start2

  7. Open Splunk in a second browser window or tab.
  8. In the second Splunk page, click the Settings drop-down menu in the upper right of the page and select Indexes.

    Note

    Indexes is located in the Data section of the Settings drop-down menu.

  9. Click New Index.
  10. Type a name for the index in the Name field (for example, redhatinsights). Leave the rest of the fields blank.
  11. Click Save. The index you created appears in the Indexes list.
  12. Click Enable next to the name of the new index to enable it.
  13. Navigate back to the first Splunk screen with the Set up integration with Red Hat page.
  14. Type the name for the HEC in the HEC name field (for example, redhatinsights).
  15. Type the name of the index you just created in the Default index field (for example, redhatinsights).
  16. Click Next.
  17. Click Review.
  18. Click Submit. The HEC name you created appears in the HEC Name field.
  19. Click Next to create the HEC URL and HEC Token.

    img evnt create hec

  20. Click Next: Configure Splunk integration in Insights. This button is disabled until you click on a Copy button for either the HEC URL or HEC token. This opens console.redhat.com in a new browser tab.

    Note

    If the new tab for console.redhat.com does not open, disable the popup blocker in your browser.

  21. Click Copy to copy the HEC URL value in Splunk Enterprise, and then paste it into the Splunk HEC URL field on the Integrations page in console.redhat.com.
  22. Add the port, if needed. The default port for Splunk is 8088.

    img evnt paste hec in hcc2

  23. Copy the HEC Token value in Splunk Enterprise. Paste it into the Splunk HEC Token field in console.redhat.com.
  24. In console.redhat.com, click Run configuration. Red Hat Insights sets up the integration, creates the behavior group, and associates Insights events to the behavior group. The status message section on the right side of the page shows the status of each of these actions.

    img evnt run config in hcc

  25. When the setup completes successfully, click Next: Review. The application returns the message Splunk integration in Insights completed.

    img evnt hcc config complete

  26. Click Go back to the Splunk application. This redirects you to the Set up integration with Red Hat screen in Splunk.
  27. Click Finish set up to complete the setup in Splunk.

    img evnt splunk setup complete

  28. Click Go to dashboard to be redirected to your Splunk dashboard.

    img evnt splunk dashboard

Note

If the integration configuration fails during the Insights setup process, contact Red Hat support.

The setup automation performs the following tasks:

  • Creates a user group with Notifications Administrator role using the Organization Administrator permissions. You can also create the user group manually. For more information about manual configuration, see Manually configuring a Notifications Administrator group in your Insights account.
  • Uses the Splunk HEC URL and HEC token to create a new integration called SPLUNK_AUTOMATION, with the integration type Splunk.
  • Creates a new Behavior Group called SPLUNK_AUTOMATION_GROUP on the RHEL bundle. The group includes an action to send notifications to the SPLUNK_AUTOMATION Splunk integration.
  • Assigns the new Behavior Group SPLUNK_AUTOMATION_GROUP to all Insights services. This forwards the events from all services to Splunk. Currently, the Behavior Group forwards events from the Advisor, Policies, and Drift services.

When Splunk begins to receive notifications from Insights, the Red Hat Insights application for Splunk dashboard shows event activity. Each number contains a hyperlink to Insights.

img evnt splunk dashboard2

To view a list of Insights events on the Splunk dashboard, click the Events tab. Each event is hyperlinked to Insights.

splunk events log

6.3. Enabling the HEC token

Before Splunk can receive Insights events, you must enable the HEC token.

Prerequisites

  • An Organization Administrator-level login to Red Hat Insights.
  • You must have the admin role in Splunk Enterprise, or the sc_admin role in Splunk Cloud.

Procedure

  1. From the Splunk main page, navigate to Settings.
  2. Select Data Inputs, and then select HTTP Event Collector. The HTTP Event Collector page shows the HEC, its Token value, the corresponding index that you selected during setup, and the status of the HEC.

    img evnt splunk hec screen

  3. Click Global Settings in the upper right corner of the page. The Edit Global Settings dialog box displays.

    img evnt splunk global settings

  4. Select Enabled. This enables the HEC token that was automatically created during the setup process.

The HEC token uses a default HTTP port number of 8088. If you are using a different port, you must update your Insights Splunk Integration to match.

Additional Resources

6.4. Manually configuring a Notifications Administrator group in your Insights account

Important

The Red Hat Insights application for Splunk automated installation/setup process automatically configures a Notifications Administrator role and group in your Insights account. Use this procedure only if you want to manually create the Notifications Administrator role and group.

Procedure

  1. Navigate to Settings, and then select My User Access from the drop-down menu.
  2. Select Groups. The Groups page appears.
  3. Click Create Group. The Name and Description page appears.
  4. Create a name for the group (for example, splunknotifgroup) and click Next. The Add Roles page appears.
  5. To create the Notifications Administrator role, click in the Search field and type notif.
  6. Select Notifications Administrator from the search results, and then click Next. The Add Members page appears.
  7. Select the group members from the list who should have Notifications Administrator role.
  8. Click Next. The Review Details page appears.
  9. Review the details: Group Name, Role, and Members, and then click Submit.

Insights verifies the details, and then the new group appears on the Groups page. The Success adding group message displays. Group members (for example, members of splunknotifgroup) can now configure notifications and integrations.

6.5. Manually configuring a Splunk Integration

Important

The Red Hat Insights application for Splunk automated installation/setup process automatically configures Splunk integration to your Insights account. Use this procedure only if you want to configure the integration manually.

Prerequisites

  • HEC URL from Splunk Cloud or Splunk Enterprise.
  • HEC token value from Splunk Cloud or Splunk Enterprise.
  • Notifications Administrator access to Red Hat Insights.

Procedure

  1. Navigate to Settings, and select Integrations from the drop-down menu. The Integrations page appears.
  2. Click Add Integration. The Add Integration dialog box appears.

    img evnt add integration

  3. Click Type from the drop-down menu and select Splunk.
  4. Type a name for your new integration into the Integration Name field (for example, redhat_splunk).
  5. In the Endpoint URL field, add the URL for your Splunk HEC endpoint URL. Splunk uses port 8088 by default. For example: http://<splunk-endpoint>:8088
  6. In the Secret token field, add the Splunk HEC token value.
  7. Optional. Add any notes or other information about this integration to the Extras field.
  8. Click Save.

Additional Resources

6.6. Troubleshooting integration with Splunk

Here are some common configuration errors in the Splunk environment that could result in Splunk not receiving events from Red Hat Insights:

  • Make sure the HEC is enabled (under Global Settings). See Enabling the HEC.
  • Make sure that the default index has not changed for the HEC (it should be redhatinsights).
  • Make sure the firewall allows for incoming requests on the configured Splunk HEC port (default is 8088). If you are using AWS for your instance, allow any of the ports Splunk may need. For more information, refer to Splunk Phantom ports and endpoints.

6.6.1. Events show as sent within Insights but do not appear in Splunk

  • Check your firewall for where your Splunk setup resides.
  • Ensure that the Splunk port is allowed (port 8088 by default).

Additional resources

Chapter 7. Troubleshoot notification failures with the event log and integration settings

Troubleshoot notification failures with the event log and integration settings The notifications service event log enables Notifications administrators to easily see when notifications are not working properly. The event log provides a list of all triggered events on the Red Hat Hybrid Cloud Console account, and actions taken (as configured in the associated behavior group) for the past 14 days.

img hcc notif event log

In the Actions column, each event shows the integration type highlighted in green or red. These visual color codes indicate the status of the message transmission: success or failure, respectively.

The following use cases illustrate the troubleshootings capabilities of the event log:

Check an endpoint configuration to troubleshoot a degrading connection.

The filterable event log is a useful troubleshooting tool to see a failed notification event and identify potential issues with endpoints. After seeing a failed action in the event log, the Notifications administrator can check the endpoint and the status of the last five connection attempts on the Integrations screen.

In the integrations service, the following connection statuses are reflected by color:

  • Green: Five previous transmissions were successful
  • Red: Five previous transmissions were unsuccessful (timeout, 404 error, etc)
  • Yellow: Connection is degraded; at least two of the five previous transmissions were unsuccessful
  • Unknown: The integration has not yet been called, or is not associated with a behavior group

Determine whether a user’s non-receipt of emails is a configuration or user error.

The event log can answer questions related to receipt of emails. By showing the email action for an event as green, the event log enables a Notifications administrator to confirm emails were sent successfully. An issue with the receipt of notification emails may be with individual user preferences and not with notification configuration.

Important

Even with notifications and integrations configured properly, individual users on the Red Hat Hybrid Cloud Console account must configure their User Preferences to receive emails.

7.1. Checking for connection failures in the event log

Use the following procedure to check for notification action failures.

Prerequisites

  • You must be logged into Red Hat Hybrid Cloud Console with the Notifications administrator privileges configured in User Access.
  • Before users receive notifications using the webhook integration type, endpoints for your organization’s preferred webhook application must be configured by a Notifications administrator in the integrations service.
  • Before users receive email notifications, they must configure their personal email notification preferences for each Hybrid Cloud Console application bundle.

Procedure

  1. Navigate to Red Hat Hybrid Cloud Console > Settings > Notifications > Event log.
  2. Filter the events list as needed by event, application, or application bundle and select to show events from today, yesterday, the last seven days, the last 14 days (default), or set a custom range within the last 14 days.
  3. Sort the Date and time column as needed in ascending or descending order.
  4. If users of the application bundle are not receiving the notifications as intended, you can click View notification settings and change settings and/or have users check their user preferences for receiving email notifications.

Legal Notice

Copyright © 2022 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.