Red Hat Training

A Red Hat training course is available for Red Hat Satellite

8.4. Notifications

In addition to viewing probe status within the RHN interface, you may be notified whenever a probe changes state. This is especially important when monitoring mission-critical production systems. For this reason, Red Hat recommends taking advantage of this feature.
To enable probe notifications within RHN, you must have identified a mail exchange server and mail domain during installation of your RHN Satellite and configured sendmail to properly handle incoming mail. Refer to the Installation chapter of the RHN Satellite Installation Guide for details.

8.4.1. Creating Notification Methods

Notifications are sent via a notification method, an email or pager address associated with a specific RHN user. Although the address is tied to a particular user account, it may serve multiple administrators through an alias or mailing list. Each user account can contain multiple notification methods. To create a notification method:
  1. Log into the RHN website as either an Satellite Administrator or Monitoring Administrator.
  2. Navigate to the User Details ⇒ Notification Methods tab and click create new method.
  3. Enter an intuitive, descriptive label for the method name, such as DBA day email, and provide the correct email or pager address. Remember, the labels for all notification methods are available in a single list during probe creation, so they should be unique to your organization.
  4. Select the checkbox if you desire abbreviated messages to be sent to the pager. This shorter format contains only the probe state, system hostname, probe name, time of message, and Send ID. The standard, longer format displays additional message headers, system and probe details, and instructions for response.
  5. When finished, click Create Method. The new method shows up in the User Details ⇒ Notification Methods tab and the Notification page under the top Monitoring category. Click its name to edit or delete it.
  6. While adding probes, select the Probe Notifications checkbox and select the new notification method from the resulting dropdown menu. Notification methods assigned to probes cannot be deleted until they are dis-associated from the probe.

8.4.2. Receiving Notifications

If you create notification methods and associate them with probes, you must be prepared to receive them. These notifications come in the form of brief text messages sent to either email or pager addresses. Here is an example of an email notification: 
Subject: CRITICAL: [hostname]: Satellite: Users at 1
From: "Monitoring Satellite Notification" (rogerthat01@redhat.com)
Date: Mon, 6 Dec 2004 13:42:28 -0800  
To: user@organization.com

This is RHN Monitoring Satellite notification 01dc8hqw. 

Time: Mon Dec 06, 21:42:25 PST 
State: CRITICAL
System: [hostname] ([IP address]) 
Probe: Satellite: Users 
Message: Users 6 (above critical threshold of 2)
Notification #116 for Users
 
Run from: RHN Monitoring Satellite
As you can see, the longer email notifications contain virtually everything you would need to know about the associated probe. In addition to the probe command, run time, system monitored, and state, the message contains the Send ID, which is a unique character string representing the precise message and probe. In the above message, the Send ID is 01dc8hqw.
Pager notifications, by necessity, contain only the most important details, namely the subject of the email message (containing state, system, probe, and time) and the Send ID. Here is an example pager notification: 
 CRITICAL: [hostname]: Satellite: Users at 21:42 PST, notification 01dc8hqw

8.4.3. Redirecting Notifications

Upon receiving a notification, you may redirect it by including advanced notification rules within an acknowledgment email. Just reply to the notification and include the desired option. These are the possible redirect options, or filter types:
  • ACK METOO — Sends the notification to the redirect destination(s) in addition to the default destination.
  • ACK SUSPEND — Suspends the notification method for a specified time period.
  • ACK AUTOACK — Does not change the destination of the notification, but automatically acknowledges matching alerts as soon as they are sent.
  • ACK REDIR — Sends the notification to the redirect destination(s) instead of the default destination.
The format of the rule should be filter_type probe_type duration email_address where filter_type indicates one of the previous advanced commands, probe_type indicates probe or system, duration indicates the length of time for the redirect, and email_address indicates the intended recipient. For example: 
 ACK METOO system 1h boss@domain.com
Capitalization is not required. Duration can be listed in minutes (m), hours (h), or days (d). Email addresses are needed only for redirects (REDIR) and supplemental (METOO) notifications.
The description of the action contained in the resulting email defaults to the command entered by the user. The reason listed is a summary of the action, such as email ack redirect by user@domain.com where user equals the sender of the email.

Note

You can halt or redirect almost all probe notifications by replying to a notification emails with a variation of the command ack suspend host. However, you cannot halt Satellite probe notifications by responding to a probe with ack suspend host or other redirect responses. These probes require you to change the notifications within the web interface of the Satellite.

8.4.4. Filtering Notifications

Since notifications can be generated whenever a probe changes state, simple changes in your network can result in a flood of notifications. The creation, cancellation, and application of Notification filters is discussed in detail in Section 7.10.2.1, “Notification ⇒ Filters”.

8.4.5. Deleting Notification Methods

Theoretically, removing notification methods should be as easy as creating them. After all, you must populate no fields to conduct the deletion and a button exists for this explicit purpose. However, existing relationships between methods and probes can complicate this process. Follow these steps to remove a notification method:
  1. Log into the RHN website as an Satellite Administrator or Monitoring Administrator.
  2. Navigate to the Monitoring ⇒ Notifications page and click the name of the method to be removed.
  3. On the User Details ⇒ Notification Methods tab, click delete method. If the method is not associated with any probes, you are presented with a confirmation page. Click Confirm Deletion. The notification method is removed.

    Note

    Since both the notification method name and address can be edited, consider updating the method rather than deleting it. This redirects notifications from all probes using the method without having to edit each probe and create a new notification method.
  4. If the method is associated with one or more probes, you are presented with a list of the probes using the method and the systems to which the probes are attached instead of a confirmation page. Click the probe name to go directly to the System Details ⇒ Probes tab.
  5. On the System Details ⇒ Probes tab, select another notification method and click Update Probe.
  6. You may now return to the Monitoring ⇒ Notifications page and delete the notification method.