One cluster node gets fenced after running firewall-cmd --reload

Solution Verified - Updated -

Issue

  • Reloading firewalld in a cluster with knet transport causes a node to be fenced.
[root@rhel-8-0-2 ~]# date && time firewall-cmd --reload && date
Mon Jan  7 01:10:13 PST 2019
success

real    0m1.757s
user    0m0.266s
sys 0m0.036s
Mon Jan  7 01:10:15 PST 2019

[root@rhel-8-0-2 ~]# cat /var/log/messages
...
Jan  7 01:10:14 rhel-8-0-2 corosync[822]:  [KNET  ] link: host: 1 link: 0 is down
Jan  7 01:10:14 rhel-8-0-2 corosync[822]:  [TOTEM ] Token has not been received in 750 ms
Jan  7 01:10:14 rhel-8-0-2 corosync[822]:  [TOTEM ] A processor failed, forming new configuration.
Jan  7 01:10:16 rhel-8-0-2 corosync[822]:  [TOTEM ] A new membership (2:6004) was formed. Members left: 1
Jan  7 01:10:16 rhel-8-0-2 corosync[822]:  [TOTEM ] Failed to receive the leave message. failed: 1
Jan  7 01:10:16 rhel-8-0-2 corosync[822]:  [CPG   ] downlist left_list: 1 received
Jan  7 01:10:16 rhel-8-0-2 corosync[822]:  [QUORUM] Members[1]: 2
Jan  7 01:10:16 rhel-8-0-2 corosync[822]:  [MAIN  ] Completed service synchronization, ready to provide service.
Jan  7 01:10:16 rhel-8-0-2 pacemaker-controld[842]: notice: Node rhel-8-0-1 state is now lost
Jan  7 01:10:16 rhel-8-0-2 pacemaker-fenced[838]: notice: Node rhel-8-0-1 state is now lost
Jan  7 01:10:16 rhel-8-0-2 pacemaker-based[837]: notice: Node rhel-8-0-1 state is now lost
Jan  7 01:10:16 rhel-8-0-2 pacemaker-controld[842]: warning: Our DC node (rhel-8-0-1) left the cluster
Jan  7 01:10:16 rhel-8-0-2 pacemaker-based[837]: notice: Purged 1 peer with id=1 and/or uname=rhel-8-0-1 from the membership cache
Jan  7 01:10:16 rhel-8-0-2 pacemakerd[836]: notice: Node rhel-8-0-1 state is now lost
Jan  7 01:10:16 rhel-8-0-2 pacemaker-fenced[838]: notice: Purged 1 peer with id=1 and/or uname=rhel-8-0-1 from the membership cache
Jan  7 01:10:16 rhel-8-0-2 pacemaker-controld[842]: notice: State transition S_NOT_DC -> S_ELECTION
Jan  7 01:10:16 rhel-8-0-2 pacemaker-controld[842]: notice: State transition S_ELECTION -> S_INTEGRATION
Jan  7 01:10:16 rhel-8-0-2 pacemaker-attrd[840]: notice: Node rhel-8-0-1 state is now lost
Jan  7 01:10:16 rhel-8-0-2 pacemaker-attrd[840]: notice: Removing all rhel-8-0-1 attributes for peer loss
Jan  7 01:10:16 rhel-8-0-2 pacemaker-attrd[840]: notice: Purged 1 peer with id=1 and/or uname=rhel-8-0-1 from the membership cache
Jan  7 01:10:16 rhel-8-0-2 pacemaker-schedulerd[841]: warning: Cluster node rhel-8-0-1 will be fenced: peer is no longer part of the cluster
Jan  7 01:10:16 rhel-8-0-2 pacemaker-schedulerd[841]: warning: Node rhel-8-0-1 is unclean

Environment

  • Red Hat Enterprise Linux (RHEL) 8 with the High Availability Add-on
  • knet transport

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content