Why ceilometer services cannot communicate with keystone and glance over https?
Issue
-
Ceilometer services fail to work with keystone and glance services over https (with ssl). Problems have been detected at least in:
- ceilometer-central-agent can not access glance over https.
- ceilometer-alarm-evaluator can not access keystone over https.
-
ceilometer-central-agent fails to get info from glance via ssh with below error message.:
2014-02-06 13:29:03.418 21957 WARNING ceilometer.central.manager [-] Continue after error from image.size: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
2014-02-06 13:29:03.418 21957 ERROR ceilometer.central.manager [-] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager Traceback (most recent call last):
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/ceilometer/central/manager.py", line 46, in poll_and_publish
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager cache,
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/ceilometer/image/glance.py", line 122, in get_samples
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager for image in self._iter_images(manager.keystone, cache):
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/ceilometer/image/glance.py", line 76, in _iter_images
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager cache['images'] = list(self._get_images(ksclient))
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/ceilometer/image/glance.py", line 52, in _get_images
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager client.images.list(filters={"is_public": False})))
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/glanceclient/v1/images.py", line 174, in paginate
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager images = self._list(url, "images")
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/glanceclient/common/base.py", line 53, in _list
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager resp, body = self.api.json_request('GET', url)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 266, in json_request
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager resp, body_iter = self._http_request(url, method, **kwargs)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 225, in _http_request
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager conn.request(method, conn_url, **kwargs)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 914, in request
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self._send_request(method, url, body, headers)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self.endheaders()
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self._send_output()
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self.send(msg)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 759, in send
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self.sock.sendall(str)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/eventlet/green/OpenSSL/SSL.py", line 99, in sendall
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager tail = self.send(data)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/eventlet/green/OpenSSL/SSL.py", line 79, in write
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager return self.fd.write(data)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager Error: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager
- ceilometer-agent-evaluator fails:
2014-02-04 19:09:47.498 20593 ERROR ceilometer.openstack.common.threadgroup [-] Authorization Failed: Unable to establish connection tohttps://jtechslbvip.hi.inet:35357/v2.0/tokens
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup Traceback (most recent call last):
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/openstack/common/threadgroup.py", line 117, in wait
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup x.wait()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/openstack/common/threadgroup.py", line 49, in wait
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup return self.thread.wait()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 166, in wait
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup return self._exit_event.wait()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/event.py", line 116, in wait
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup return hubs.get_hub().switch()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup return self.greenlet.switch()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 192, in main
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup result = function(*args, **kwargs)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/openstack/common/service.py", line 448, in run_service
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup service.start()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/alarm/service.py", line 182, in start
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup *[eval_interval, self._client])
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/alarm/service.py", line 90, in _client
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup self.api_client = ceiloclient.get_client(2, **creds)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometerclient/client.py", line 81, in get_client
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup _ksclient = _get_ksclient(**ks_kwargs)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometerclient/client.py", line 35, in _get_ksclient
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup insecure=kwargs.get('insecure'))
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/keystoneclient/v2_0/client.py", line 139, in __init__
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup self.authenticate()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/keystoneclient/httpclient.py", line 468, in authenticate
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup resp, body = self.get_raw_token_from_identity_service(**kwargs)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/keystoneclient/v2_0/client.py", line 168, in get_raw_token_from_identity_service
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup "%s" % e)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup AuthorizationFailure: Authorization Failed: Unable to establish connection tohttps://jtechslbvip.hi.inet:35357/v2.0/tokens
Environment
- Red Hat Enterprise Linux Openstack Platform v4.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.