RHEV: Hosted-Engine fails to start after RHEV-H rebooted

Issue

• Hosted-Engine unable to start after RHEV-H rebooted
• The following errors can be seen:

ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'set-storage-domain FilesystemBackend dom_type=iscsi sd_uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'#012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 299, in _dispatch#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 65, in set_storage_domain#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 450, in connect#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 107, in get_domain_path#012BackendFailureException: path to storage domain XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX not found in /rhev/data-center/mnt

vdsm vds ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012  File "/usr/share/vdsm/API.py", line 1707, in _getHaInfo#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 100, in get_all_stats#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 177, in _configure_broker_conn#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 176, in set_storage_domain#012RequestError: Failed to set storage domain FilesystemBackend, options {'dom_type': 'iscsi', 'sd_uuid': 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'}: Request failed: <class 'ovirt_hosted_engine_ha.lib.storage_backends.BackendFailureException'>

ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Error: 'path to storage domain XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX not found in /rhev/data-center/mnt' - trying to restart agent

• When checking hosted-engine --vm-status, the following is displayed:

# hosted-engine --vm-status
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_setup/vm_status.py", line 116, in <module>
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_setup/vm_status.py", line 59, in print_status
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 157, in get_all_host_stats
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 100, in get_all_stats
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 177, in _configure_broker_conn
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 176, in set_storage_domain
ovirt_hosted_engine_ha.lib.exceptions.RequestError: Failed to set storage domain FilesystemBackend, options {'dom_type': 'iscsi', 'sd_uuid': 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'}: Request failed: <class 'ovirt_hosted_engine_ha.lib.storage_backends.BackendFailureException'>