Error when trying to bring up Storage Domain (KeyError: 'VERSION')
Issue
- RHEV environment has gone down due to power failure.
- When trying bring up hosts, none will take SPM role.
Thread-1157::ERROR::2016-04-23 16:46:11,240::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain <domainID> monitoring information
Traceback (most recent call last):
File "/usr/share/vdsm/storage/domainMonitor.py", line 221, in _monitorDomain
File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain
File "/usr/share/vdsm/storage/fileSD.py", line 164, in __init__
File "/usr/share/vdsm/storage/sd.py", line 313, in __init__
File "/usr/share/vdsm/storage/sd.py", line 344, in _makeClusterLock
File "/usr/share/vdsm/storage/sd.py", line 450, in getVersion
File "/usr/share/vdsm/storage/sd.py", line 750, in getMetaParam
File "/usr/share/vdsm/storage/persistentDict.py", line 89, in __getitem__
File "/usr/share/vdsm/storage/persistentDict.py", line 201, in __getitem__
KeyError: 'VERSION'
Thread-1175::ERROR::2016-04-23 16:46:29,006::task::866::Storage.TaskManager.Task::(_setError) Task=`<taskID`::Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 873, in _run
File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
File "/usr/share/vdsm/storage/hsm.py", line 2189, in getAllTasksStatuses
File "/usr/share/vdsm/storage/securable.py", line 75, in wrapper
SecureError: Secured object is not in safe state
- Hosts have been rebooted, and manually fenced
Environment
- Red Hat Enterprise Virtualization 3.5, 3.1 compatibility mode
- Red Hat Enterprise Virtualization Hypervisor 6.7 (20160219.0.el6ev)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.