cinder-volume service is down

Issue

• We did expansion of RHOSP13 overcloud and after successfull deployment.

• Cinder-volume service is down.

• Even restarted the pcs cluster, docker cinder-volume and rebooted all the controllers

2020-11-19 10:35:52.935 9442 ERROR oslo_db.sqlalchemy.engines [req-95ede41b-e99c-45b2-8bc6-0d5500b4354f - - - - -] Database connection was found disconnected; reconnecting: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8)
[...]
2020-11-19 13:12:08.449 42 ERROR cinder.utils [req-ac8fecda-5257-4ab4-9ac5-52fcf8c0da77 - - - - -] Volume driver LVMVolumeDriver not initialized
2020-11-19 13:12:08.450 42 ERROR cinder.volume.manager [req-ac8fecda-5257-4ab4-9ac5-52fcf8c0da77 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready.
[...]
2020-11-19 13:18:08.470 42 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_iscsi is reporting problems, not sending heartbeat. Service will appear "down".
2020-11-19 13:18:18.463 42 WARNING cinder.volume.manager [req-8f7bd81e-4c3b-4cdf-b1dc-cf092095f0b4 - - - - -] Update driver status failed: (config name tripleo_iscsi) is uninitialized.
2020-11-19 13:18:18.471 42 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_iscsi is reporting problems, not sending heartbeat. Service will appear "down".
2020-11-19 13:18:28.471 42 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_iscsi is reporting problems, not sending heartbeat. Service will appear "down".

• No changes in the template file as it's just compute count has been increased.

• We see the service is down here:

(overcloud) [stack@undercloud ~]$ cinder service-list
+------------------+--------------------------------------+------+---------+-------+----------------------------+-----------------+
| Binary           | Host                                 | Zone | Status  | State | Updated_at                 | Disabled Reason |
+------------------+--------------------------------------+------+---------+-------+----------------------------+-----------------+
| cinder-scheduler | overcloud-controller-0 | nova | enabled | up    | 2020-11-19T14:28:59.000000 | -               |
| cinder-scheduler | overcloud-controller-1 | nova | enabled | up    | 2020-11-19T14:28:59.000000 | -               |
| cinder-scheduler | overcloud-controller-2 | nova | enabled | up    | 2020-11-19T14:28:55.000000 | -               |
| cinder-volume    | hostgroup@tripleo_iscsi              | nova | enabled | down  | 2020-11-19T13:12:06.000000 | -               |
+------------------+--------------------------------------+------+---------+-------+----------------------------+-----------------+

• No specific Cinder service we using it's local storage for vm's.