service openstack-cinder-volume-docker-0 is missing after minor upgrade
Issue
- Following a minor update,
openstack-cinder-volume-docker-0
stopped and never restarted:
[root@overcloud-controller-0 cinder]# pcs status Cluster name: tripleo_cluster
Stack: corosync
Current DC: overcloud-controller-0 (version 1.1.19-8.el7_6.4-c3c624ea3d) - partition with quorum Last updated: Tue May 7 14:59:24 2019 Last change: Tue May 7 14:00:55 2019 by hacluster via crmd on overcloud-controller-1
15 nodes configured
67 resources configured
Online: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]
GuestOnline: [ galera-bundle-0@overcloud-controller-0 galera-bundle-1@overcloud-controller-1 galera-bundle-2@overcloud-controller-2 ovn-dbs-bundle-0@overcloud-controller-0 ovn-dbs-bundle-1@overcloud-controller-1 ovn-dbs-bundle-2@overcloud-controller-2 rabbitmq-bundle-0@overcloud-controller-0 rabbitmq-bundle-1@overcloud-controller-1 rabbitmq-bundle-2@overcloud-controller-2 redis-bundle-0@overcloud-controller-0 redis-bundle-1@overcloud-controller-1 redis-bundle-2@overcloud-controller-2 ]
Full list of resources:
Docker container set: rabbitmq-bundle [10.10.10.10::8787/rhosp13/openstack-rabbitmq:pcmklatest]
rabbitmq-bundle-0 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-controller-0
rabbitmq-bundle-1 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-controller-1
rabbitmq-bundle-2 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-controller-2
Docker container set: galera-bundle [10.10.10.10::8787/rhosp13/openstack-mariadb:pcmklatest]
galera-bundle-0 (ocf::heartbeat:galera): Master overcloud-controller-0
galera-bundle-1 (ocf::heartbeat:galera): Master overcloud-controller-1
galera-bundle-2 (ocf::heartbeat:galera): Master overcloud-controller-2
Docker container set: redis-bundle [10.10.10.10::8787/rhosp13/openstack-redis:pcmklatest]
redis-bundle-0 (ocf::heartbeat:redis): Slave overcloud-controller-0
redis-bundle-1 (ocf::heartbeat:redis): Master overcloud-controller-1
redis-bundle-2 (ocf::heartbeat:redis): Slave overcloud-controller-2
ip-10.151.111.160 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0
ip-10.32.133.151 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1
ip-10.151.111.69 (ocf::heartbeat:IPaddr2): Started overcloud-controller-2
ip-10.151.111.70 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0
ip-10.151.111.204 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0
ip-10.151.109.12 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1
Docker container set: haproxy-bundle [10.10.10.10::8787/rhosp13/openstack-haproxy:pcmklatest]
haproxy-bundle-docker-0 (ocf::heartbeat:docker): Started overcloud-controller-0
haproxy-bundle-docker-1 (ocf::heartbeat:docker): Started overcloud-controller-1
haproxy-bundle-docker-2 (ocf::heartbeat:docker): Started overcloud-controller-2
Docker container set: ovn-dbs-bundle [10.10.10.10::8787/rhosp13/openstack-ovn-northd:pcmklatest]
ovn-dbs-bundle-0 (ocf::ovn:ovndb-servers): Master overcloud-controller-0
ovn-dbs-bundle-1 (ocf::ovn:ovndb-servers): Slave overcloud-controller-1
ovn-dbs-bundle-2 (ocf::ovn:ovndb-servers): Slave overcloud-controller-2
stonith-fence_compute-fence-nova (stonith:fence_compute): Started overcloud-controller-2
Clone Set: compute-unfence-trigger-clone [compute-unfence-trigger]
Stopped: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] nova-evacuate (ocf::openstack:NovaEvacuate): Started overcloud-controller-1
stonith-fence_ipmilan-e4434b5adcc0 (stonith:fence_ipmilan): Started overcloud-controller-1
stonith-fence_ipmilan-e4434b5aca88 (stonith:fence_ipmilan): Started overcloud-controller-2
stonith-fence_ipmilan-e4434b5aca08 (stonith:fence_ipmilan): Started overcloud-controller-2
Docker container: openstack-cinder-volume [10.10.10.10:8787/custom/openstack-cinder-volume-custom:pcmklatest]
openstack-cinder-volume-docker-0 (ocf::heartbeat:docker): Stopped
Docker container: openstack-cinder-backup [10.10.10.10::8787/rhosp13/openstack-cinder-backup:pcmklatest]
openstack-cinder-backup-docker-0 (ocf::heartbeat:docker): Started overcloud-controller-2
Failed Actions:
* openstack-cinder-volume-docker-0_start_0 on overcloud-controller-2 'unknown error' (1): call=142, status=complete, exitreason='failed to pull image 10.10.10.10:8787/custom/openstack-cinder-volume-custom:pcmklatest',
last-rc-change='Tue May 7 14:09:35 2019', queued=0ms, exec=263ms
* openstack-cinder-volume-docker-0_start_0 on overcloud-controller-0 'unknown error' (1): call=109, status=complete, exitreason='failed to pull image 10.10.10.10:8787/custom/openstack-cinder-volume-custom:pcmklatest',
last-rc-change='Tue May 7 14:08:54 2019', queued=0ms, exec=334ms
* openstack-cinder-volume-docker-0_start_0 on overcloud-controller-1 'unknown error' (1): call=144, status=complete, exitreason='failed to pull image 10.10.10.10:8787/custom/openstack-cinder-volume-custom:pcmklatest',
last-rc-change='Tue May 7 14:09:48 2019', queued=0ms, exec=258ms
Daemon Status:
corosync: active/enabled
pacemaker: active/enabled
pcsd: active/enabled
- Containers using the tag
latest
were not updated
Environment
- Red Hat OpenStack Platform 13.0 (RHOSP)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.