Cman binds to wrong ip address when 2 NICs on subnet with Red Hat High Availability Addon
Issue
- After rebooting both cluster nodes, CMAN failed to start and throw "TOTEM" error. I can ping cluster IP successfully but cluster is showing some network error in "UNICAST" mode. When I changed it to "MULTICAST" mode, CMAN successfully starts but created a fencing loop where each server reboots one after another comes live.
Environment
- Red Hat Enterprise Linux 6 (With High Availability or Resilient Storage Add-ons)
- High Availability Cluster with 2 or more nodes
-
2 or more network interfaces active on cluster nodes
-
At least 2 network interfaces are on the same subnet. For example, in the following example, eth1 and bond0 are both on 10.0.0.0/24:
$ grep -B2 -e 10.0.0.XX -e 10.0.0.YY sos_commands/networking/ip_address 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff inet 10.0.0.YY/24 brd 10.0.0.255 scope global eth1 -- 6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff inet 10.0.0.XX/24 brd 10.0.0.255 scope global bond0
-
One of the interfaces on the same subnet is configured to be used for cluster communication:
# cman_tool status | grep "Node addresses" Node addresses: 10.0.0.YY
-
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.