Intropection is not starting/finishing correctly and logs say node is locked
Issue
The introspection is not starting/finishing correctly and the follwoing logs can seen in mistral-executor.logs:
017-05-02 08:14:45.990 1464 ERROR mistral.engine.default_executor [-] Failed to run action [action_ex_id=8fff1f31-45bd-4c82-bf42-60c0a4f80e2f, action_cls='<class 'mistral.actions.action_factory.IronicAction'>', attributes='{u'client_method_name': u'node.set_provision_state'}', params='{u'state': u'provide', u'node_uuid': u'ea73efd1-a4db-46f8-a843-469ac52ef55e', u'configdrive': None, u'cleansteps': None}']
IronicAction.node.set_provision_state failed: <class 'ironicclient.common.apiclient.exceptions.Conflict'>: Node ea73efd1-a4db-46f8-a843-469ac52ef55e is locked by host nManager.testdev.env, please retry after the current operation is completed.
2017-05-02 08:14:45.836 2514 DEBUG wsme.api [req-764a9d87-2360-4475-a93a-d68c99c02838 admin admin - - -] Client-side error: Node 07cfae86-e8b3-4dc8-bf92-22ac77b509da is locked by host nManager.testdev.env, please retry after the current operation is completed.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 199, in inner
return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/ironic/conductor/manager.py", line 1032, in do_provisioning_action
% action) as task:
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 158, in acquire
driver_name=driver_name, purpose=purpose)
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 223, in __init__
self.release_resources()
File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
self.force_reraise()
File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
six.reraise(self.type_, self.value, self.tb)
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 210, in __init__
self._lock()
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 255, in _lock
reserve_node()
File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
raise attempt.get()
File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
six.reraise(self.value[0], self.value[1], self.value[2])
File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 248, in reserve_node
self.node_id)
File "/usr/lib/python2.7/site-packages/ironic/objects/node.py", line 270, in reserve
db_node = cls.dbapi.reserve_node(tag, node_id)
File "/usr/lib/python2.7/site-packages/ironic/db/sqlalchemy/api.py", line 264, in reserve_node
host=node['reservation'])
NodeLocked: Node 07cfae86-e8b3-4dc8-bf92-22ac77b509da is locked by host nManager.testdev.env, please retry after the current operation is completed.
Environment
- Red Hat Openstack Platform 10
- ssh_pxe ironic driver
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
