Cinder-volume service down. "cinder.volume.drivers.rbd ObjectNotFound: error connecting to the cluster"

Posted on

Hi,
Running OSP10 version TripleO on the Cluster.

[root@as-vnfv-director1 templates]# cinder service-list
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| cinder-scheduler | hostgroup | nova | enabled | up | 2020-05-21T15:48:14.000000 | - |
| cinder-volume | hostgroup@tripleo_ceph | nova | enabled | down | 2020-05-19T11:50:05.000000 | - |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+

On attempting to restart cinder-volume on controller, throws the below error:
2020-05-19 07:51:52.924 1030906 ERROR cinder.volume.drivers.rbd [req-32be05f0-b39b-4609-84f4-e049728cd85a - - - - -] Error connecting to ceph cluster.
2020-05-19 07:51:52.924 1030906 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
2020-05-19 07:51:52.924 1030906 ERROR cinder.volume.drivers.rbd File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 360, in _connect_to_rados
2020-05-19 07:51:52.924 1030906 ERROR cinder.volume.drivers.rbd client.connect()
2020-05-19 07:51:52.924 1030906 ERROR cinder.volume.drivers.rbd File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
2020-05-19 07:51:52.924 1030906 ERROR cinder.volume.drivers.rbd ObjectNotFound: error connecting to the cluster
2020-05-19 07:51:52.924 1030906 ERROR cinder.volume.drivers.rbd

Attached full logs:

I attempted manual connect to the cluster and i do NOT encounter any error

cluster = rados.Rados(rados_id='openstack', clustername='ceph', conffile='/etc/ceph/ceph.conf')
cluster.connect()

Any help in this regard is highly appreciated.

Thanks

SA

Responses