rabbitmq-bundle service down in one of the controller node [Sonipat/Production]

Solution In Progress - Updated -

Issue

  • rabbitmq-bundle service down in one of the controller node
Failed Resource Actions:
* rabbitmq-bundle-docker-2_stop_0 on overcloud-control-1 'unknown error' (1): call=241, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:09:41 2020', queued=0ms, exec=20294ms
* rabbitmq-bundle-2_start_0 on overcloud-control-1 'unknown error' (1): call=51, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:09:23 2020', queued=0ms, exec=0ms
* rabbitmq-bundle-docker-1_stop_0 on overcloud-control-0 'unknown error' (1): call=110, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:57 2020', queued=0ms, exec=20001ms
* openstack-cinder-volume-docker-0_monitor_60000 on overcloud-control-0 'unknown error' (1): call=102, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:45 2020', queued=0ms, exec=0ms
* galera-bundle-docker-0_stop_0 on overcloud-control-0 'unknown error' (1): call=104, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:36 2020', queued=0ms, exec=20003ms
* redis-bundle-docker-0_monitor_60000 on overcloud-control-0 'unknown error' (1): call=84, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:45 2020', queued=0ms, exec=0ms
* haproxy-bundle-docker-0_monitor_60000 on overcloud-control-0 'unknown error' (1): call=100, status=Timed Out, exitreason='',
    last-rc-change='Sat Jan 25 00:15:45 2020', queued=0ms, exec=0msxec=0ms
  • pcs resource cleanup doesn't bring it up

  • docker ps | rabbitmq shows it is running:

f909f8f1dac7        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:pcmklatest                  "/bin/bash /usr/lo..."   2 days ago          Up 2 days                                         rabbitmq-bundle-docker-2
84521ddaa0f6        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:13.0-78                     "/docker_puppet_ap..."   4 months ago        Exited (0) 4 months ago                           rabbitmq_init_bundle
69e2dcf9fe76        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:13.0-78                     "/usr/bin/bootstra..."   4 months ago        Exited (0) 4 months ago                           rabbitmq_restart_bundle
eede3dd4a448        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:13.0-78                     "/bin/bash -c '/us..."   4 months ago        Exited (0) 4 months ago                           rabbitmq_image_tag
6051609748bc        rhosp13-asset.localdomain:8787/rhosp13/openstack-rabbitmq:13.0-78                     "kolla_start"            4 months ago        Exited (0) 4 months ago                           rabbitmq_bootstrap

Environment

  • Red Hat OpenStack Platform 13.0 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content