Why is RHEV 3 storage domain frequently marked as "in problem" and then "recovered from problem"?
Issue
- In
/var/log/ovirt-engine/engine.log
there are frequent logs as:
2014-03-27 19:03:54,991 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-4-thread-13) domain $Domain_ID in problem. vds: host1.example.com
2014-03-27 19:04:34,220 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-4-thread-37) Domain c$Domain_ID recovered from problem. vds: host1.example.com
2014-03-27 19:04:34,220 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-4-thread-37) Domain $Domain_ID has recovered from problem. No active host in the DC is reporting it as problematic, so clearing the domain recovery timer.
2016-03-26 16:24:23,288 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-15) Domain $Domain_ID check timeot 51.3 is too big
2016-03-26 16:24:23,292 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (org.ovirt.thread.pool-7-thread-15) Storage Domain $Storage_Domain_Name of pool $Storage_Pool_Name is in problem in host $host_name
2016-03-26 16:24:23,318 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-15) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host $host_name reports about one of the Active Storage Domains as Problematic.
2016-03-26 16:24:23,343 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-79) [27d897cc] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: $ID Type: VDS
2016-03-26 16:24:23,362 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) [27d897cc] START, SetVdsStatusVDSCommand(HostName = $host_name, HostId = $host_ID, status=NonOperational, nonOperationalReason=STORAGE_DOMAIN_UNREACHABLE, stopSpmFailureLogged=false), log id: 5d95cfd
Environment
- Red Hat Enterprise Virtualization 3.x
- Red Hat Enterprise Virtualization Hypervisor (RHEV-H) 6.x and 7.x
- Red Hat Enterprise Linux 6 and 7 Hosts
- Hosts/Hypervisors with vdsm version 4.17 or lower.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.