OpenStack Galera will not start with a failed Controller node and galera-bundle resources in Slave

Solution Verified - Updated -

Issue

  • One controller is down due to H/W failure or File Corruption
  • This causes Galera not to start on the other two controller nodes and remains in Slave state

    12 nodes configured
    40 resource instances configured
    
    Online: [ controller-1 controller-2 ]
    OFFLINE: [ controller-0 ]                   <<------
    GuestOnline: [ galera-bundle-0@controller-2 galera-bundle-1@controller-1 rabbitmq-bundle-1@controller-1 rabbitmq-bundle-2@controller-2 redis-bundle-1@controller-1 redis-bundle-2@controller-2 ]
    
    Full list of resources:
    
    Docker container set: rabbitmq-bundle [192.168.24.1:8787/rhosp13/openstack-rabbitmq:pcmklatest]
    rabbitmq-bundle-0    (ocf::heartbeat:rabbitmq-cluster):      Stopped
    rabbitmq-bundle-1    (ocf::heartbeat:rabbitmq-cluster):      Started controller-1
    rabbitmq-bundle-2    (ocf::heartbeat:rabbitmq-cluster):      Started controller-2
    Docker container set: galera-bundle [192.168.24.1:8787/rhosp13/openstack-mariadb:pcmklatest]
    galera-bundle-0      (ocf::heartbeat:galera):        Slave controller-2
    galera-bundle-1      (ocf::heartbeat:galera):        Slave controller-1
    galera-bundle-2      (ocf::heartbeat:galera):        Stopped
    Docker container set: redis-bundle [192.168.24.1:8787/rhosp13/openstack-redis:pcmklatest]
    redis-bundle-0       (ocf::heartbeat:redis): Stopped
    redis-bundle-1       (ocf::heartbeat:redis): Master controller-1
    redis-bundle-2       (ocf::heartbeat:redis): Slave controller-2
    ip-192.x.x.x       (ocf::heartbeat:IPaddr2):       Started controller-2
    ip-172.x.x.x       (ocf::heartbeat:IPaddr2):       Started controller-1
    ip-172.x.x.x       (ocf::heartbeat:IPaddr2):       Started controller-2
    ip-172.x.x.x       (ocf::heartbeat:IPaddr2):       Started controller-1
    ip-172.x.x.x       (ocf::heartbeat:IPaddr2):       Started controller-1
    ip-172.x.x.x       (ocf::heartbeat:IPaddr2):       Started controller-2
    Docker container set: haproxy-bundle [192.168.24.1:8787/rhosp13/openstack-haproxy:pcmklatest]
    haproxy-bundle-docker-0      (ocf::heartbeat:docker):        Stopped
    haproxy-bundle-docker-1      (ocf::heartbeat:docker):        Started controller-1
    haproxy-bundle-docker-2      (ocf::heartbeat:docker):        Started controller-2
    Docker container: openstack-cinder-volume [192.168.24.1:8787/rhosp13/openstack-cinder-volume:pcmklatest]
    openstack-cinder-volume-docker-0     (ocf::heartbeat:docker):        Started controller-2
    stonith-fence_rhevm-566f32200003       (stonith:fence_rhevm):  Started controller-1
    stonith-fence_rhevm-566f3220001a       (stonith:fence_rhevm):  Started controller-2
    stonith-fence_rhevm-566f3220001d       (stonith:fence_rhevm):  Started controller-1
    

Environment

Red Hat OpenStack Platform 13

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In