Rgmanager service doesn't detect storage failure with multipath devices on Red Hat Enterprise Linux
Issue
- When both the fibre cables of HBA are removed, the failover does not happens. In fact, the log messages does not have any log about rgmanager for the same.... When we stop rgmanager on active node (node1 and node2) the service failover to passive node (node3). But if the storage resource is turned down on active node, the service failover does not happen. Cluster is unable to sense the failure on active node and does not transfer the service on passive node...
- We tried to write to filesystem after disconnection of fibre cables, but unable to do so...the command stucks. No ext3 errors after disconnection of cables.
- Cluster don't switch resources when a disk failure occurs
Environment
- Red Hat Enterprise Linux (RHEL), including:
- Red Hat Enterprise Linux 4 (RHEL4) with Cluster Suite Add-on
- Red Hat Enterprise Linux 5 (RHEL5) Advanced Platform
- Red Hat Enterprise Linux 6 (RHEL6) with High Availability or Resilient Storage Add-ons.
- Red Hat Cluster Suite Cluster consisting of at least 2 nodes.
- Services managed by "Resource Group Manager" (rgmanager)
- Services consist of non-cluster filesystem mounts (example ext3, xfs, etc) using fs.sh resource script.
- Cluster storage is accessible by all cluster nodes from SAN
- Storage is managed by device-mapper-multipath
- Either Defaults for SAN are configured using one of the following settings, or /etc/multipath.conf is configured to use these settings:
- features = "1 queue_if_no_path"; or
- no_path_retry = queue (or an integer larger than 5)
- The issue is triggered by intentionally (or unintentionally) removing the fibre from the HBA's, causing the link between server and storage to be severed.
- Any other method of causing paths to fail in device-mapper-multipath can also trigger this issue.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.
