'metrics' pool is missing on major upgrade from Red Hat OpenStack Platform 9 to 10

Solution In Progress - Updated -

Issue

Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

After a major upgrade from OSP 9 to 10, the metrics pool is not configured in Ceph.

From /var/log/gnocchi/metricd.log

2017-09-13 15:17:45.479 603330 INFO gnocchi.storage.ceph [-] Ceph storage backend use 'cradox' python library
2017-09-13 15:17:45.487 603330 ERROR cotyledon [-] Unhandled exception
2017-09-13 15:17:45.487 603330 ERROR cotyledon Traceback (most recent call last):
2017-09-13 15:17:45.487 603330 ERROR cotyledon   File "/usr/lib/python2.7/site-packages/cotyledon/__init__.py", line 52, in _exit_on_exception
2017-09-13 15:17:45.487 603330 ERROR cotyledon     yield
2017-09-13 15:17:45.487 603330 ERROR cotyledon   File "/usr/lib/python2.7/site-packages/cotyledon/__init__.py", line 130, in _run
2017-09-13 15:17:45.487 603330 ERROR cotyledon     self.run()
2017-09-13 15:17:45.487 603330 ERROR cotyledon   File "/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 92, in run
2017-09-13 15:17:45.487 603330 ERROR cotyledon     self._configure()
2017-09-13 15:17:45.487 603330 ERROR cotyledon   File "/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 87, in _configure
2017-09-13 15:17:45.487 603330 ERROR cotyledon     self.store = storage.get_driver(self.conf)
2017-09-13 15:17:45.487 603330 ERROR cotyledon   File "/usr/lib/python2.7/site-packages/gnocchi/storage/__init__.py", line 159, in get_driver
2017-09-13 15:17:45.487 603330 ERROR cotyledon     return get_driver_class(conf)(conf.storage)
2017-09-13 15:17:45.487 603330 ERROR cotyledon   File "/usr/lib/python2.7/site-packages/gnocchi/storage/ceph.py", line 99, in __init__
2017-09-13 15:17:45.487 603330 ERROR cotyledon     self.ioctx = self.rados.open_ioctx(self.pool)
2017-09-13 15:17:45.487 603330 ERROR cotyledon   File "cradox.pyx", line 413, in cradox.requires.wrapper.validate_func (cradox.c:4188)
2017-09-13 15:17:45.487 603330 ERROR cotyledon   File "cradox.pyx", line 1047, in cradox.Rados.open_ioctx (cradox.c:12325)
2017-09-13 15:17:45.487 603330 ERROR cotyledon ObjectNotFound: error opening pool 'metrics'
2017-09-13 15:17:45.487 603330 ERROR cotyledon

This causes several issues.

First, this causes https://access.redhat.com/solutions/3066751
Secondly, the absence of the metrics pool in RHOSP after an upgrade of OSP 9 to OSP 10 causes gnocchi to create too many connections to redis. These connections exhaust the number of sockets on the system due to too many sockets in the TIME_WAIT state. This leads to a rabbitmq failure, because it cannot open new ports.

Environment

Red Hat OpenStack Platform 10

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content