Why ceilometer services cannot communicate with keystone and glance over https?
Issue
-
Ceilometer services fail to work with keystone and glance services over https (with ssl). Problems have been detected at least in:
- ceilometer-central-agent can not access glance over https.
- ceilometer-alarm-evaluator can not access keystone over https.
-
ceilometer-central-agent fails to get info from glance via ssh with below error message.:
2014-02-06 13:29:03.418 21957 WARNING ceilometer.central.manager [-] Continue after error from image.size: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
2014-02-06 13:29:03.418 21957 ERROR ceilometer.central.manager [-] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager Traceback (most recent call last):
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/ceilometer/central/manager.py", line 46, in poll_and_publish
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager cache,
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/ceilometer/image/glance.py", line 122, in get_samples
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager for image in self._iter_images(manager.keystone, cache):
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/ceilometer/image/glance.py", line 76, in _iter_images
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager cache['images'] = list(self._get_images(ksclient))
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/ceilometer/image/glance.py", line 52, in _get_images
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager client.images.list(filters={"is_public": False})))
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/glanceclient/v1/images.py", line 174, in paginate
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager images = self._list(url, "images")
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/glanceclient/common/base.py", line 53, in _list
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager resp, body = self.api.json_request('GET', url)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 266, in json_request
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager resp, body_iter = self._http_request(url, method, **kwargs)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 225, in _http_request
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager conn.request(method, conn_url, **kwargs)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 914, in request
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self._send_request(method, url, body, headers)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self.endheaders()
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self._send_output()
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self.send(msg)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib64/python2.6/httplib.py", line 759, in send
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager self.sock.sendall(str)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/eventlet/green/OpenSSL/SSL.py", line 99, in sendall
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager tail = self.send(data)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager File "/usr/lib/python2.6/site-packages/eventlet/green/OpenSSL/SSL.py", line 79, in write
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager return self.fd.write(data)
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager Error: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
2014-02-06 13:29:03.418 21957 TRACE ceilometer.central.manager
- ceilometer-agent-evaluator fails:
2014-02-04 19:09:47.498 20593 ERROR ceilometer.openstack.common.threadgroup [-] Authorization Failed: Unable to establish connection tohttps://jtechslbvip.hi.inet:35357/v2.0/tokens
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup Traceback (most recent call last):
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/openstack/common/threadgroup.py", line 117, in wait
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup x.wait()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/openstack/common/threadgroup.py", line 49, in wait
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup return self.thread.wait()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 166, in wait
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup return self._exit_event.wait()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/event.py", line 116, in wait
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup return hubs.get_hub().switch()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup return self.greenlet.switch()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 192, in main
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup result = function(*args, **kwargs)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/openstack/common/service.py", line 448, in run_service
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup service.start()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/alarm/service.py", line 182, in start
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup *[eval_interval, self._client])
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometer/alarm/service.py", line 90, in _client
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup self.api_client = ceiloclient.get_client(2, **creds)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometerclient/client.py", line 81, in get_client
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup _ksclient = _get_ksclient(**ks_kwargs)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/ceilometerclient/client.py", line 35, in _get_ksclient
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup insecure=kwargs.get('insecure'))
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/keystoneclient/v2_0/client.py", line 139, in __init__
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup self.authenticate()
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/keystoneclient/httpclient.py", line 468, in authenticate
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup resp, body = self.get_raw_token_from_identity_service(**kwargs)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/keystoneclient/v2_0/client.py", line 168, in get_raw_token_from_identity_service
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup "%s" % e)
2014-02-04 19:09:47.498 20593 TRACE ceilometer.openstack.common.threadgroup AuthorizationFailure: Authorization Failed: Unable to establish connection tohttps://jtechslbvip.hi.inet:35357/v2.0/tokens
Environment
- Red Hat Enterprise Linux Openstack Platform v4.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
