Red Hat Enterprise Linux 6

Cluster Administration

Configuring and Managing the High Availability Add-On

Edition 0

Logo

Legal Notice

Copyright © 2013 Red Hat, Inc. and others.
This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack Logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.

Abstract

Configuring and Managing the High Availability Add-On describes the configuration and management of the High Availability Add-On for Red Hat Enterprise Linux 6.
Introduction
1. Document Conventions
1.1. Typographic Conventions
1.2. Pull-quote Conventions
1.3. Notes and Warnings
2. Feedback
1. Red Hat High Availability Add-On Configuration and Management Overview
1.1. New and Changed Features
1.1.1. New and Changed Features for Red Hat Enterprise Linux 6.1
1.1.2. New and Changed Features for Red Hat Enterprise Linux 6.2
1.1.3. New and Changed Features for Red Hat Enterprise Linux 6.3
1.1.4. New and Changed Features for Red Hat Enterprise Linux 6.4
1.1.5. New and Changed Features for Red Hat Enterprise Linux 6.5
1.2. Configuration Basics
1.3. Setting Up Hardware
1.4. Installing Red Hat High Availability Add-On software
1.5. Configuring Red Hat High Availability Add-On Software
2. Before Configuring the Red Hat High Availability Add-On
2.1. General Configuration Considerations
2.2. Compatible Hardware
2.3. Enabling IP Ports
2.3.1. Enabling IP Ports on Cluster Nodes
2.3.2. Enabling the IP Port for luci
2.3.3. Configuring the iptables Firewall to Allow Cluster Components
2.4. Configuring luci with /etc/sysconfig/luci
2.5. Configuring ACPI For Use with Integrated Fence Devices
2.5.1. Disabling ACPI Soft-Off with chkconfig Management
2.5.2. Disabling ACPI Soft-Off with the BIOS
2.5.3. Disabling ACPI Completely in the grub.conf File
2.6. Considerations for Configuring HA Services
2.7. Configuration Validation
2.8. Considerations for NetworkManager
2.9. Considerations for Using Quorum Disk
2.10. Red Hat High Availability Add-On and SELinux
2.11. Multicast Addresses
2.12. UDP Unicast Traffic
2.13. Considerations for ricci
2.14. Configuring Virtual Machines in a Clustered Environment
3. Configuring Red Hat High Availability Add-On With Conga
3.1. Configuration Tasks
3.2. Starting luci
3.3. Controlling Access to luci
3.4. Creating a Cluster
3.5. Global Cluster Properties
3.5.1. Configuring General Properties
3.5.2. Configuring Fence Daemon Properties
3.5.3. Network Configuration
3.5.4. Configuring Redundant Ring Protocol
3.5.5. Quorum Disk Configuration
3.5.6. Logging Configuration
3.6. Configuring Fence Devices
3.6.1. Creating a Fence Device
3.6.2. Modifying a Fence Device
3.6.3. Deleting a Fence Device
3.7. Configuring Fencing for Cluster Members
3.7.1. Configuring a Single Fence Device for a Node
3.7.2. Configuring a Backup Fence Device
3.7.3. Configuring a Node with Redundant Power
3.8. Configuring a Failover Domain
3.8.1. Adding a Failover Domain
3.8.2. Modifying a Failover Domain
3.8.3. Deleting a Failover Domain
3.9. Configuring Global Cluster Resources
3.10. Adding a Cluster Service to the Cluster
4. Managing Red Hat High Availability Add-On With Conga
4.1. Adding an Existing Cluster to the luci Interface
4.2. Removing a Cluster from the luci Interface
4.3. Managing Cluster Nodes
4.3.1. Rebooting a Cluster Node
4.3.2. Causing a Node to Leave or Join a Cluster
4.3.3. Adding a Member to a Running Cluster
4.3.4. Deleting a Member from a Cluster
4.4. Starting, Stopping, Restarting, and Deleting Clusters
4.5. Managing High-Availability Services
4.6. Backing Up and Restoring the luci Configuration
5. Configuring Red Hat High Availability Add-On With the ccs Command
5.1. Operational Overview
5.1.1. Creating the Cluster Configuration File on a Local System
5.1.2. Viewing the Current Cluster Configuration
5.1.3. Specifying ricci Passwords with the ccs Command
5.1.4. Modifying Cluster Configuration Components
5.1.5. Commands that Overwrite Previous Settings
5.1.6. Configuration Validation
5.2. Configuration Tasks
5.3. Starting ricci
5.4. Creating A Cluster
5.5. Configuring Fence Devices
5.6. Listing Fence Devices and Fence Device Options
5.7. Configuring Fencing for Cluster Members
5.7.1. Configuring a Single Power-Based Fence Device for a Node
5.7.2. Configuring a Single Storage-Based Fence Device for a Node
5.7.3. Configuring a Backup Fence Device
5.7.4. Configuring a Node with Redundant Power
5.7.5. Removing Fence Methods and Fence Instances
5.8. Configuring a Failover Domain
5.9. Configuring Global Cluster Resources
5.10. Adding a Cluster Service to the Cluster
5.11. Listing Available Cluster Services and Resources
5.12. Virtual Machine Resources
5.13. Configuring a Quorum Disk
5.14. Miscellaneous Cluster Configuration
5.14.1. Cluster Configuration Version
5.14.2. Multicast Configuration
5.14.3. Configuring a Two-Node Cluster
5.14.4. Logging
5.14.5. Configuring Redundant Ring Protocol
5.15. Propagating the Configuration File to the Cluster Nodes
6. Managing Red Hat High Availability Add-On With ccs
6.1. Managing Cluster Nodes
6.1.1. Causing a Node to Leave or Join a Cluster
6.1.2. Adding a Member to a Running Cluster
6.2. Starting and Stopping a Cluster
6.3. Diagnosing and Correcting Problems in a Cluster
7. Configuring Red Hat High Availability Manually
7.1. Configuration Tasks
7.2. Creating a Basic Cluster Configuration File
7.3. Configuring Fencing
7.4. Configuring Failover Domains
7.5. Configuring HA Services
7.5.1. Adding Cluster Resources
7.5.2. Adding a Cluster Service to the Cluster
7.6. Configuring Redundant Ring Protocol
7.7. Configuring Debug Options
7.8. Configuring nfsexport and nfsserver Resources
7.9. Verifying a Configuration
8. Managing Red Hat High Availability Add-On With Command Line Tools
8.1. Starting and Stopping the Cluster Software
8.1.1. Starting Cluster Software
8.1.2. Stopping Cluster Software
8.2. Deleting or Adding a Node
8.2.1. Deleting a Node from a Cluster
8.2.2. Adding a Node to a Cluster
8.2.3. Examples of Three-Node and Two-Node Configurations
8.3. Managing High-Availability Services
8.3.1. Displaying HA Service Status with clustat
8.3.2. Managing HA Services with clusvcadm
8.4. Updating a Configuration
8.4.1. Updating a Configuration Using cman_tool version -r
8.4.2. Updating a Configuration Using scp
9. Diagnosing and Correcting Problems in a Cluster
9.1. Configuration Changes Do Not Take Effect
9.2. Cluster Does Not Form
9.3. Nodes Unable to Rejoin Cluster after Fence or Reboot
9.4. Cluster Daemon crashes
9.4.1. Capturing the rgmanager Core at Runtime
9.4.2. Capturing the Core When the Daemon Crashes
9.4.3. Recording a gdb Backtrace Session
9.5. Cluster Services Hang
9.6. Cluster Service Will Not Start
9.7. Cluster-Controlled Services Fails to Migrate
9.8. Each Node in a Two-Node Cluster Reports Second Node Down
9.9. Nodes are Fenced on LUN Path Failure
9.10. Quorum Disk Does Not Appear as Cluster Member
9.11. Unusual Failover Behavior
9.12. Fencing Occurs at Random
9.13. Debug Logging for Distributed Lock Manager (DLM) Needs to be Enabled
10. SNMP Configuration with the Red Hat High Availability Add-On
10.1. SNMP and the Red Hat High Availability Add-On
10.2. Configuring SNMP with the Red Hat High Availability Add-On
10.3. Forwarding SNMP traps
10.4. SNMP Traps Produced by Red Hat High Availability Add-On
11. Clustered Samba Configuration
11.1. CTDB Overview
11.2. Required Packages
11.3. GFS2 Configuration
11.4. CTDB Configuration
11.5. Samba Configuration
11.6. Starting CTDB and Samba Services
11.7. Using the Clustered Samba Server
A. Fence Device Parameters
B. HA Resource Parameters
C. HA Resource Behavior
C.1. Parent, Child, and Sibling Relationships Among Resources
C.2. Sibling Start Ordering and Resource Child Ordering
C.2.1. Typed Child Resource Start and Stop Ordering
C.2.2. Non-typed Child Resource Start and Stop Ordering
C.3. Inheritance, the <resources> Block, and Reusing Resources
C.4. Failure Recovery and Independent Subtrees
C.5. Debugging and Testing Services and Resource Ordering
D. Cluster Service Resource Check and Failover Timeout
D.1. Modifying the Resource Status Check Interval
D.2. Enforcing Resource Timeouts
E. Command Line Tools Summary
F. High Availability LVM (HA-LVM)
F.1. Configuring HA-LVM Failover with CLVM (preferred)
F.2. Configuring HA-LVM Failover with Tagging
G. Revision History
Index