VM migrations between all hosts in a RHEV cluster fail with "Migration failed due to Error: Could not connect to peer host".

Issue

• VM migration is failing with the error Migration failed due to Error: Could not connect to peer host.
• VM migrations between all hosts in a newly-configured cluster fail.
• VM migrations between hosts in existing clusters are successful.
• All of the hosts are running the same version of RHEV-H.
• The vdsm logs on the source host contain;

Thread-4156119::ERROR::2014-02-21 00:19:01,885::vm::148::vm.Vm::(_setupVdsConnection) vmId=`aa0a19b5-fbb0-4805-9a5b-48630c49597d`::Error initiating connection
Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 143, in _setupVdsConnection
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1235, in request
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1349, in send_content
  File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
  File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
  File "/usr/lib64/python2.6/httplib.py", line 739, in send
  File "/usr/share/vdsm/kaxmlrpclib.py", line 151, in connect
  File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 172, in connect
  File "/usr/lib64/python2.6/ssl.py", line 338, in wrap_socket
  File "/usr/lib64/python2.6/ssl.py", line 120, in __init__
  File "/usr/lib64/python2.6/ssl.py", line 279, in do_handshake
SSLError: _ssl.c:475: The handshake operation timed out

• The vdsm logs on the destination host contain;

BindingXMLRPC::ERROR::2014-02-21 00:27:04,973::BindingXMLRPC::72::vds::(threaded_start) xml-rpc handler exception
Traceback (most recent call last):
  File "/usr/share/vdsm/BindingXMLRPC.py", line 68, in threaded_start
  File "/usr/lib64/python2.6/SocketServer.py", line 268, in handle_request
  File "/usr/lib64/python2.6/SocketServer.py", line 278, in _handle_request_noblock
  File "/usr/lib64/python2.6/SocketServer.py", line 446, in get_request
  File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 116, in accept
  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 167, in accept
  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 156, in accept_ssl
SSLError: unexpected eof

• Running vdsClient to communicate with vdsm on the destination host hangs;

# vdsClient -s <destination host ip> list

• Running openssl s_client to communicate with vdsm's port 54321 on the destination host hangs;

# openssl s_client -connect 172.19.28.16:54321 -CAfile /etc/pki/vdsm/certs/cacert.pem
CONNECTED(00000003)