No bundles are starting on 2 out of 3 controllers

Solution In Progress - Updated -

Issue

  • No bundles are starting on 2 out of 3 controllers:
[root@overcloud-controller-1 rabbitmq-bundle-2]# pcs status
Cluster name: tripleo_cluster
Stack: corosync
Current DC: overcloud-controller-2 (version 1.1.20-5.el7_7.2-3c4c782f70) - partition with quorum
Last updated: Tue May 18 16:00:04 2021
Last change: Tue May 18 14:58:59 2021 by root via cibadmin on overcloud-controller-2

12 nodes configured
40 resources configured

Online: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]
GuestOnline: [ galera-bundle-0@overcloud-controller-0 rabbitmq-bundle-0@overcloud-controller-0 redis-bundle-0@overcloud-controller-0 ]

Full list of resources:

 Docker container set: rabbitmq-bundle [10.10.10.10:8787/rhosp13/openstack-rabbitmq:pcmklatest]
   rabbitmq-bundle-0    (ocf::heartbeat:rabbitmq-cluster):      Started overcloud-controller-0
   rabbitmq-bundle-1    (ocf::heartbeat:rabbitmq-cluster):      Stopped
   rabbitmq-bundle-2    (ocf::heartbeat:rabbitmq-cluster):      Stopped
 Docker container set: galera-bundle [10.10.10.10:8787/rhosp13/openstack-mariadb:pcmklatest]
   galera-bundle-0      (ocf::heartbeat:galera):        Slave overcloud-controller-0
   galera-bundle-1      (ocf::heartbeat:galera):        Stopped
   galera-bundle-2      (ocf::heartbeat:galera):        Stopped
 Docker container set: redis-bundle [10.10.10.10:8787/rhosp13/openstack-redis:pcmklatest]
   redis-bundle-0       (ocf::heartbeat:redis): Master overcloud-controller-0
   redis-bundle-1       (ocf::heartbeat:redis): Stopped
   redis-bundle-2       (ocf::heartbeat:redis): Stopped
 ip-10.244.9.4  (ocf::heartbeat:IPaddr2):       Started overcloud-controller-0
 ip-10.57.73.37 (ocf::heartbeat:IPaddr2):       Started overcloud-controller-0
 ip-10.57.73.102        (ocf::heartbeat:IPaddr2):       Started overcloud-controller-0
 ip-10.244.44.222       (ocf::heartbeat:IPaddr2):       Started overcloud-controller-0
 ip-10.57.73.101        (ocf::heartbeat:IPaddr2):       Started overcloud-controller-0
 ip-10.57.73.52 (ocf::heartbeat:IPaddr2):       Started overcloud-controller-0
 Docker container set: haproxy-bundle [10.10.10.10:8787/rhosp13/openstack-haproxy:pcmklatest]
   haproxy-bundle-docker-0      (ocf::heartbeat:docker):        Started overcloud-controller-0
   haproxy-bundle-docker-1      (ocf::heartbeat:docker):        Stopped
   haproxy-bundle-docker-2      (ocf::heartbeat:docker):        Stopped
 stonith-fence_ipmilan-5254002c193c     (stonith:fence_ipmilan):        Started overcloud-controller-1
 stonith-fence_ipmilan-5254005519f7     (stonith:fence_ipmilan):        Started overcloud-controller-2
 stonith-fence_ipmilan-5254006cad25     (stonith:fence_ipmilan):        Started overcloud-controller-1
 Docker container: openstack-cinder-volume [10.10.10.10:8787/rhosp13/openstack-cinder-volume:pcmklatest]
   openstack-cinder-volume-docker-0     (ocf::heartbeat:docker):        Started overcloud-controller-0

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

Environment

  • Red Hat OpenStack Platform 13.0 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content