Error when trying to bring up Storage Domain (KeyError: 'VERSION')

Solution Verified - Updated -

Issue

  • RHEV environment has gone down due to power failure.
  • When trying bring up hosts, none will take SPM role.
Thread-1157::ERROR::2016-04-23 16:46:11,240::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain <domainID> monitoring information
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/domainMonitor.py", line 221, in _monitorDomain
  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
  File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain
  File "/usr/share/vdsm/storage/fileSD.py", line 164, in __init__
  File "/usr/share/vdsm/storage/sd.py", line 313, in __init__
  File "/usr/share/vdsm/storage/sd.py", line 344, in _makeClusterLock
  File "/usr/share/vdsm/storage/sd.py", line 450, in getVersion
  File "/usr/share/vdsm/storage/sd.py", line 750, in getMetaParam
  File "/usr/share/vdsm/storage/persistentDict.py", line 89, in __getitem__
  File "/usr/share/vdsm/storage/persistentDict.py", line 201, in __getitem__
KeyError: 'VERSION'

Thread-1175::ERROR::2016-04-23 16:46:29,006::task::866::Storage.TaskManager.Task::(_setError) Task=`<taskID`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
  File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
  File "/usr/share/vdsm/storage/hsm.py", line 2189, in getAllTasksStatuses
  File "/usr/share/vdsm/storage/securable.py", line 75, in wrapper
SecureError: Secured object is not in safe state
  • Hosts have been rebooted, and manually fenced

Environment

  • Red Hat Enterprise Virtualization 3.5, 3.1 compatibility mode
  • Red Hat Enterprise Virtualization Hypervisor 6.7 (20160219.0.el6ev)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.