"File lock acquisition" error causes an instance to fail to boot when NFSv3 share is mounted on /var/lib/nova/instances
Issue
-
"File lock acquisition" error causes an instance to fail to boot when NFSv3 share is mounted on /var/lib/nova/instances.
Sample log
INFO nova.virt.libvirt.driver [req-YYY - - - - -] [instance: XXX] Creating image ERROR nova.compute.manager [req-YYY - - - - -] [instance: XXX] Instance failed to spawn TRACE nova.compute.manager [instance: XXX] Traceback (most recent call last): TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2455, in _build_resources TRACE nova.compute.manager [instance: XXX] yield resources TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2327, in _build_and_run_instance TRACE nova.compute.manager [instance: XXX] block_device_info=block_device_info) TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2424, in spawn TRACE nova.compute.manager [instance: XXX] admin_pass=admin_password) TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2835, in _create_image TRACE nova.compute.manager [instance: XXX] instance, size, fallback_from_host) TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6015, in _try_fetch_image_cache TRACE nova.compute.manager [instance: XXX] size=size) TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 231, in cache TRACE nova.compute.manager [instance: XXX] *args, **kwargs) TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 537, in create_image TRACE nova.compute.manager [instance: XXX] copy_qcow2_image(base, self.path, size) TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 439, in inner TRACE nova.compute.manager [instance: XXX] do_log=False, semaphores=semaphores, delay=delay): TRACE nova.compute.manager [instance: XXX] File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ TRACE nova.compute.manager [instance: XXX] return self.gen.next() TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 395, in lock TRACE nova.compute.manager [instance: XXX] ext_lock.acquire(delay=delay) TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 209, in acquire TRACE nova.compute.manager [instance: XXX] do_acquire() TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 158, in wrapper TRACE nova.compute.manager [instance: XXX] return r.call(func, *args, **kwargs) TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/retrying.py", line 222, in call TRACE nova.compute.manager [instance: XXX] if not self.should_reject(attempt): TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/retrying.py", line 206, in should_reject TRACE nova.compute.manager [instance: XXX] reject |= self._retry_on_exception(attempt.value[1]) TRACE nova.compute.manager [instance: XXX] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 130, in retry_on_exception TRACE nova.compute.manager [instance: XXX] 'exception': e, TRACE nova.compute.manager [instance: XXX] error: Unable to acquire lock on `/var/lib/nova/instances/locks/nova-ZZZ` due to [Errno 5] Input/output error TRACE nova.compute.manager [instance: XXX]
Environment
- Red Hat OpenStack Platform
- openstack-nova
- NFSv3 storage without NLMv4 support is mounted on /var/lib/nova/instances
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
