VM migrations between all hosts in a RHEV cluster fail with "Migration failed due to Error: Could not connect to peer host".

Solution Verified - Updated -

Issue

  • VM migration is failing with the error Migration failed due to Error: Could not connect to peer host.
  • VM migrations between all hosts in a newly-configured cluster fail.
  • VM migrations between hosts in existing clusters are successful.
  • All of the hosts are running the same version of RHEV-H.
  • The vdsm logs on the source host contain;
Thread-4156119::ERROR::2014-02-21 00:19:01,885::vm::148::vm.Vm::(_setupVdsConnection) vmId=`aa0a19b5-fbb0-4805-9a5b-48630c49597d`::Error initiating connection
Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 143, in _setupVdsConnection
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1235, in request
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1349, in send_content
  File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
  File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
  File "/usr/lib64/python2.6/httplib.py", line 739, in send
  File "/usr/share/vdsm/kaxmlrpclib.py", line 151, in connect
  File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 172, in connect
  File "/usr/lib64/python2.6/ssl.py", line 338, in wrap_socket
  File "/usr/lib64/python2.6/ssl.py", line 120, in __init__
  File "/usr/lib64/python2.6/ssl.py", line 279, in do_handshake
SSLError: _ssl.c:475: The handshake operation timed out
  • The vdsm logs on the destination host contain;
BindingXMLRPC::ERROR::2014-02-21 00:27:04,973::BindingXMLRPC::72::vds::(threaded_start) xml-rpc handler exception
Traceback (most recent call last):
  File "/usr/share/vdsm/BindingXMLRPC.py", line 68, in threaded_start
  File "/usr/lib64/python2.6/SocketServer.py", line 268, in handle_request
  File "/usr/lib64/python2.6/SocketServer.py", line 278, in _handle_request_noblock
  File "/usr/lib64/python2.6/SocketServer.py", line 446, in get_request
  File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 116, in accept
  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 167, in accept
  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 156, in accept_ssl
SSLError: unexpected eof
  • Running vdsClient to communicate with vdsm on the destination host hangs;
# vdsClient -s <destination host ip> list
  • Running openssl s_client to communicate with vdsm's port 54321 on the destination host hangs;
# openssl s_client -connect 172.19.28.16:54321 -CAfile /etc/pki/vdsm/certs/cacert.pem
CONNECTED(00000003)

Environment

  • Red Hat Enterprise Virtualization Hypervisor (RHEV-H) 6.4 20130912.1
  • Migration network or rhevm network configured for Jumbo Frames (MTU 9000)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In