VDSM periodically logs Worker blocked warning messages in vdsm.virt.sampling.HostMonitor
Issue
- No apparent problem is seen, but VDSM logs are full of warnings regarding blocked workers doing HostMonitor
- VDSM logs are filled with the following warning:
WARN Worker blocked: <Worker name=periodic/82 running <Task <Operation action=<vdsm.virt.sampling.HostMonitor object at 0x7f94e443fe50> at 0x7f94e443fe90> timeout=15,
File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap
self.__bootstrap_inner()
File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner
self.run()
File: "/usr/lib64/python2.7/threading.py", line 765, in run
self.__target(*self.__args, **self.__kwargs)
File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 194, in run
ret = func(*args, **kwargs)
File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run
self._execute_task()
File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task
task()
File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__
self._callable()
File: "/usr/lib/python2.7/site-packages/vdsm/virt/periodic.py", line 220, in __call__
self._func()
File: "/usr/lib/python2.7/site-packages/vdsm/virt/sampling.py", line 576, in __call__
sample = HostSample(self._pid)
File: "/usr/lib/python2.7/site-packages/vdsm/virt/sampling.py", line 240, in __init__
self.interfaces = _get_interfaces_and_samples()
File: "/usr/lib/python2.7/site-packages/vdsm/virt/sampling.py", line 200, in _get_interfaces_and_samples
for link in ipwrapper.getLinks():
File: "/usr/lib/python2.7/site-packages/vdsm/network/ipwrapper.py", line 267, in getLinks
in six.viewitems(dpdk.get_dpdk_devices()))
File: "/usr/lib/python2.7/site-packages/vdsm/network/link/dpdk.py", line 47, in get_dpdk_devices
dpdk_devices = _get_dpdk_devices()
File: "/usr/lib/python2.7/site-packages/vdsm/common/cache.py", line 41, in __call__
value = self.func(*args)
File: "/usr/lib/python2.7/site-packages/vdsm/network/link/dpdk.py", line 111, in _get_dpdk_devices
devices = _lshw_command()
File: "/usr/lib/python2.7/site-packages/vdsm/network/link/dpdk.py", line 123, in _lshw_command
rc, out, err = cmd.exec_sync(['lshw', '-json'] + filterout_cmd)
File: "/usr/lib/python2.7/site-packages/vdsm/network/cmd.py", line 38, in exec_sync
retcode, out, err = exec_sync_bytes(cmds)
File: "/usr/lib/python2.7/site-packages/vdsm/common/cmdutils.py", line 156, in exec_cmd
out, err = p.communicate()
File: "/usr/lib64/python2.7/site-packages/subprocess32.py", line 924, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File: "/usr/lib64/python2.7/site-packages/subprocess32.py", line 1706, in _communicate
orig_timeout)
File: "/usr/lib64/python2.7/site-packages/subprocess32.py", line 1779, in _communicate_with_poll
ready = poller.poll(self._remaining_time(endtime))
Environment
- Red Hat Virtualization 4.x
- vdsm-4.20.43-1.el7ev.x86_64
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.