Clustered LVM mirror creation or I/O hangs in RHEL with devices larger than 1.5Tb

Issue

• Creation of new mirrored logical volume in the Red Hat cluster times out:

Error locking on node <nodename>: Command timed out
Failed to activate new LV.
Unable to deactivate failed new LV. Manual intervention required.

• The mirror sync process or I/O to clustered mirror hangs when the device is larger than 1.5Tb

device-mapper: dm-log-clustered: [U9LjlBOE] Request timed out: [DM_CLOG_CTR/22] - retrying

• pvmove of clustered volumes greater than 1.5Tb hangs or causes I/O to that device to hang

• While migrating LUNs from one array to another all of a sudden the cluster went in a state of confusion, and I continuously keep getting the dm-log-clustered retry and "request timed out" messages, mixed with clogd "sync checkpoint retry messages

Aug  8 09:38:36 node2 device-mapper: dm-log-clustered: [knvU0IKd] Request timed out: [DM_CLOG_RESUME/XXX] - retrying

Aug  8 09:38:36 node1 clogd[10366]: Sync checkpoint section create retry
Aug  8 09:38:36 node1 last message repeated 103 times
Aug  8 09:38:36 node1 kernel: device-mapper: dm-log-clustered: [knvU0IKd] Request timed out: [DM_CLOG_CLEAR_REGION/581] - retrying
Aug  8 09:38:36 node1 clogd[10366]: Sync checkpoint section create retry
Aug  8 09:38:51 node1 last message repeated 7395 times