corosync is repeatedly reporting "A processor joined or left the membership" every few seconds and corosync is using a lot of memory in a RHEL 6 or 7 High Availability cluster

Solution Verified - Updated -

Issue

  • I see corosync reporting over and over every few seconds that "A processor joined or left the membership"
Jan 15 12:42:53 node1 corosync[1804]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Jan 15 12:42:53 node1 corosync[1804]:   [CPG   ] chosen downlist: sender r(0) ip(10.10.10.1) ; members(old:2 left:0)
Jan 15 12:42:53 node1 corosync[1804]:   [MAIN  ] Completed service synchronization, ready to provide service.
Jan 15 12:42:56 node1 corosync[1804]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Jan 15 12:42:56 node1 corosync[1804]:   [CPG   ] chosen downlist: sender r(0) ip(10.10.10.1) ; members(old:2 left:0)
Jan 15 12:42:56 node1 corosync[1804]:   [MAIN  ] Completed service synchronization, ready to provide service.
Jan 15 12:43:00 node1 corosync[1804]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Jan 15 12:43:00 node1 corosync[1804]:   [CPG   ] chosen downlist: sender r(0) ip(10.10.10.1) ; members(old:2 left:0)
Jan 15 12:43:00 node1 corosync[1804]:   [MAIN  ] Completed service synchronization, ready to provide service.
  • corosync seems to be using a lot of memory and there are repeated "A processor joined or left the membership" messages in the logs
  • corosync caused an oom-killer kill after the node had been in the cluster for some time.

Environment

  • Red Hat Enterprise Linux (RHEL) 6 or 7 with the High Availability Add On

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content