Hardware introspection failed in OpenStack 7

Issue

Hardware introspection fails when adding nodes to a running overcloud and dracut falls into emergency mode.

Ironic-discoverd logs contains the following:

Nov 26 07:49:12 <hostname> ironic-discoverd[831]: ERROR:ironic_discoverd.utils:Unexpected exception during processing
Nov 26 07:49:12 <hostname> ironic-discoverd[831]: INFO:werkzeug:171.21.212.148 - - [26/Nov/2015 07:49:12] "POST /v1/continue HTTP/1.1" 400 -
Nov 26 11:26:26 <hostname> ironic-discoverd[831]: INFO:ironic_discoverd.utils:Node 057031e4-2d9f-48c9-8e68-53a6b1dfe395 is in mainten...s check
Nov 26 11:26:26 <hostname> ironic-discoverd[831]: INFO:werkzeug:172.21.41.4 - - [26/Nov/2015 11:26:26] "POST /v1/introspection/057031..." 202 -
Nov 26 11:26:26 <hostname> ironic-discoverd[831]: INFO:ironic_discoverd.introspect:Whitelisting MAC's [u'a0:36:9f:75:a8:22'] for node...irewall
Nov 26 11:30:45 <hostname> ironic-discoverd[831]: INFO:ironic_discoverd.plugins.standard:Discovered data: CPUs: 48 x86_64, memory 393...277 GiB
Nov 26 11:30:45 <hostname> ironic-discoverd[831]: INFO:ironic_discoverd.plugins.standard:PXE boot interface was a0:36:9f:75:a8:22
Nov 26 11:30:45 <hostname> ironic-discoverd[831]: WARNING:ironic_discoverd.plugins.standard:The following interfaces were invalid or ...u'ip': 
Nov 26 11:30:46 <hostname> ironic-discoverd[831]: INFO:ironic_discoverd.plugins.standard:Eligible interfaces are {u'p2p2': {u'ip': u'...8:22'}}
Nov 26 11:30:46 <hostname> ironic-discoverd[831]: INFO:ironic_discoverd.utils:Node 057031e4-2d9f-48c9-8e68-53a6b1dfe395 is in mainten...s check
Nov 26 11:32:48 <hostname> ironic-discoverd[831]: ERROR:ironicclient.common.http:Error contacting Ironic server: A port with MAC addr...1 of 61
Nov 26 11:32:48 <hostname> ironic-discoverd[831]: WARNING:ironic_discoverd.process:MAC a0:36:9f:75:a8:22 appeared in introspection da...kipping
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: ERROR:swiftclient:('Connection aborted.', error(111, 'ECONNREFUSED'))
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: Traceback (most recent call last):
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1261, in _retry
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: rv = func(self.url, self.token, *args, **kwargs)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 731, in put_container
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: conn.request(method, path, '', headers)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 212, in request
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: files=files, **self.requests_args)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 195, in _request
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: return self.request_session.request(*arg, **kwarg)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: resp = self.send(prep, **send_kwargs)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: r = adapter.send(request, **kwargs)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: raise ConnectionError(err, request=request)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: ConnectionError: ('Connection aborted.', error(111, 'ECONNREFUSED'))
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: ERROR:ironic_discoverd.process:Unexpected exception during processing
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: Traceback (most recent call last):
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/ironic_discoverd/process.py", line 95, in process
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: return _process_node(ironic, node, node_info, cached_node)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/ironic_discoverd/process.py", line 142, in _...ss_node
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: node_patches, port_patches = _run_post_hooks(node, ports, node_info)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/ironic_discoverd/process.py", line 114, in _...t_hooks
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: node_info)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/ironic_discoverd/plugins/edeploy.py", line 5..._update
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: json.dumps(node_info['data']))
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/ironic_discoverd/plugins/edeploy.py", line 6...ardware
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: swift_api.create_object(name, data)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/ironic_discoverd/common/swift.py", line 110,..._object
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: self.connection.put_container(container)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1337, in put_container
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: response_dict=response_dict)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1261, in _retry
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: rv = func(self.url, self.token, *args, **kwargs)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 731, in put_container
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: conn.request(method, path, '', headers)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 212, in request
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: files=files, **self.requests_args)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 195, in _request
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: return self.request_session.request(*arg, **kwarg)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: resp = self.send(prep, **send_kwargs)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: r = adapter.send(request, **kwargs)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: raise ConnectionError(err, request=request)
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: ConnectionError: ('Connection aborted.', error(111, 'ECONNREFUSED'))
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: ERROR:ironic_discoverd.utils:Unexpected exception during processing
Nov 26 11:32:51 <hostname> ironic-discoverd[831]: INFO:werkzeug:171.21.212.149 - - [26/Nov/2015 11:32:51] "POST /v1/continue HTTP/1.1" 400 -

I am going to look after the ECONNREFUSED.

Regards