"[TOTEM] Retransmit List" messages repeatedly seen in RHEL High Availability cluster node logs

Issue

• There are lots of repeating messages written to /var/log/messages similar to the following below on Red hat Enterprise Linux 5 cluster nodes:

Jul 25 04:20:16 node1 openais[18311]: [TOTEM] Retransmit List: 6053c6  
Jul 25 04:20:16 node1 openais[18311]: [TOTEM] Retransmit List: 6053d0  

• There are lots of repeating messages written to /var/log/messages similar to the following below on Red Hat Enterprise Linux 6 cluster nodes:

Sep 22 12:37:45 node1 corosync [TOTEM] Retransmit List: 31 
Sep 22 12:37:46 node1 corosync [TOTEM] Retransmit List 1

• RHEL nodes fails to join the cluster with Retransmit List errors in logs.
• clvmd hangs and lvm commands like lvs, lvdisplay, etc do not respond after seeing repeating "Retransmit List" messages in the logs.
• rgmanager or clusvcadm operations do not complete when there are retransmits in /var/log/messages.
• GFS2 I/O hangs for short periods and the logs show "Retransmit List" repeating over and over during that time.
• Cluster node hangs at ''Joining fence domain'' while booting or while starting cman service after normal boot.
• Getting openais retransmission errors in logs on cluster nodes.