Intropection is not starting/finishing correctly and logs say node is locked
Issue
The introspection is not starting/finishing correctly and the follwoing logs can seen in mistral-executor.logs:
017-05-02 08:14:45.990 1464 ERROR mistral.engine.default_executor [-] Failed to run action [action_ex_id=8fff1f31-45bd-4c82-bf42-60c0a4f80e2f, action_cls='<class 'mistral.actions.action_factory.IronicAction'>', attributes='{u'client_method_name': u'node.set_provision_state'}', params='{u'state': u'provide', u'node_uuid': u'ea73efd1-a4db-46f8-a843-469ac52ef55e', u'configdrive': None, u'cleansteps': None}']
IronicAction.node.set_provision_state failed: <class 'ironicclient.common.apiclient.exceptions.Conflict'>: Node ea73efd1-a4db-46f8-a843-469ac52ef55e is locked by host nManager.testdev.env, please retry after the current operation is completed.
2017-05-02 08:14:45.836 2514 DEBUG wsme.api [req-764a9d87-2360-4475-a93a-d68c99c02838 admin admin - - -] Client-side error: Node 07cfae86-e8b3-4dc8-bf92-22ac77b509da is locked by host nManager.testdev.env, please retry after the current operation is completed.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 199, in inner
return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/ironic/conductor/manager.py", line 1032, in do_provisioning_action
% action) as task:
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 158, in acquire
driver_name=driver_name, purpose=purpose)
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 223, in __init__
self.release_resources()
File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
self.force_reraise()
File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
six.reraise(self.type_, self.value, self.tb)
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 210, in __init__
self._lock()
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 255, in _lock
reserve_node()
File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
raise attempt.get()
File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
six.reraise(self.value[0], self.value[1], self.value[2])
File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 248, in reserve_node
self.node_id)
File "/usr/lib/python2.7/site-packages/ironic/objects/node.py", line 270, in reserve
db_node = cls.dbapi.reserve_node(tag, node_id)
File "/usr/lib/python2.7/site-packages/ironic/db/sqlalchemy/api.py", line 264, in reserve_node
host=node['reservation'])
NodeLocked: Node 07cfae86-e8b3-4dc8-bf92-22ac77b509da is locked by host nManager.testdev.env, please retry after the current operation is completed.
Environment
- Red Hat Openstack Platform 10
- ssh_pxe ironic driver
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.