rabbitmq-bundle service down in one of the controller node [Sonipat/Production]

Solution In Progress - Updated -

Issue

  • rabbitmq-bundle service down in one of the controller node
Failed Resource Actions:
* rabbitmq-bundle-docker-2_stop_0 on overcloud-control-1 'unknown error' (1): call=241, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:09:41 2020', queued=0ms, exec=20294ms
* rabbitmq-bundle-2_start_0 on overcloud-control-1 'unknown error' (1): call=51, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:09:23 2020', queued=0ms, exec=0ms
* rabbitmq-bundle-docker-1_stop_0 on overcloud-control-0 'unknown error' (1): call=110, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:57 2020', queued=0ms, exec=20001ms
* openstack-cinder-volume-docker-0_monitor_60000 on overcloud-control-0 'unknown error' (1): call=102, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:45 2020', queued=0ms, exec=0ms
* galera-bundle-docker-0_stop_0 on overcloud-control-0 'unknown error' (1): call=104, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:36 2020', queued=0ms, exec=20003ms
* redis-bundle-docker-0_monitor_60000 on overcloud-control-0 'unknown error' (1): call=84, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:45 2020', queued=0ms, exec=0ms
* haproxy-bundle-docker-0_monitor_60000 on overcloud-control-0 'unknown error' (1): call=100, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:45 2020', queued=0ms, exec=0msxec=0ms
  • pcs resource cleanup doesn't bring it up

  • docker ps | rabbitmq shows it is running:

f909f8f1dac7        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:pcmklatest                  "/bin/bash /usr/lo..."   2 days ago          Up 2 days                                         rabbitmq-bundle-docker-2
84521ddaa0f6        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:13.0-78                     "/docker_puppet_ap..."   4 months ago        Exited (0) 4 months ago                           rabbitmq_init_bundle
69e2dcf9fe76        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:13.0-78                     "/usr/bin/bootstra..."   4 months ago        Exited (0) 4 months ago                           rabbitmq_restart_bundle
eede3dd4a448        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:13.0-78                     "/bin/bash -c '/us..."   4 months ago        Exited (0) 4 months ago                           rabbitmq_image_tag
6051609748bc        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:13.0-78                     "kolla_start"            4 months ago        Exited (0) 4 months ago                           rabbitmq_bootstrap

Environment

  • Red Hat OpenStack Platform 13.0 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In