VDSM logs continuously report "Storage domain does not exist" message on RHEV host.

Solution Unverified - Updated -

Environment

  • Red Hat Enterprise Virtualization 3

Issue

  • Every few seconds VDSM tries to find storage domain 'SD UUID'. It cannot find it, so it tries again endlessly. Storage domain is already removed/destroyed and there is no reference to the SD UUID in hypervisor.

  • Tried solution outlined in knowlege base article 42102 but still seeing same messages in the vdsm.log.

Resolution

  • This problem mostly occurs when metadata contains deleted/orphan storage domain reference. To check if there is any orphan storage domain entry present in the master storage domain metadata multiple logs and storage domain metadata need to be checked.

  • Please Open a service request with Red Hat Technical Support and provide rhevm log-collector along with current SPM host sosreport . Use below command to collect log-collector.

rhevm-log-collector --host='FQDN or IP of Current SPM HOST'

Root Cause

  • Storage Domain did not deleted properly, From rhevm database its entry got deleted however, Master storage domain metadata still contain deleted Storage Domain UUID as active. This causes metadata mismatch/corruption.

  • Due to the mismatch vdsm continuously look for the storage domain but as its physically not available" storage domain does not exist" error message reported.

Diagnostic Steps

  • On host, below messages is reported in vdsm.log
StorageDomainDoesNotExist: Storage domain does not exist: ('SD-UUID',)
Thread-10406::DEBUG::2012-07-26 02:55:45,952::task::495::TaskManager.Task::(_debug) Task 6063a778-4828-4541-95aa-a613865f3cf7: finished: {'info': {'spm_id': 1, 'master_uuid': 'ece11e05-862c-4cb7-b258-0c7c30c9c634', 'name':
  • Metadata shows that the SD is as active
POOL_DESCRIPTION=PoolName
POOL_DOMAINS=bbc050b2-38c2-44de-95b1-ce638fa7e910:Active,ece11e05-862c-4cb7-b258-0c7c30c9c634:Active,SD-UUID:Active,<< 6b55d766-06ec-410c-a660-5ee525ba29a5:Active

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments