Menu Close
Settings Close

Language and Page Formatting Options

Red Hat Training

A Red Hat training course is available for Red Hat OpenStack Platform

Chapter 4. Non-Director Environments: Upgrading OpenStack Services Simultaneously

This scenario upgrades from Red Hat OpenStack Platform 9 to Red Hat OpenStack Platform 10 in environments that do not use the director. This procedure upgrades all services on all nodes. This involves the following workflow:

  1. Disabling all OpenStack services
  2. Performing a package upgrade
  3. Performing synchronization of all databases
  4. Enabling all OpenStack services
Note

The procedures in this chapter follow the architectural naming convention followed by all Red Hat OpenStack Platform documentation. If you are unfamiliar with this convention, refer to Architecture Guide available at Red Hat OpenStack Platform Documentation Suite before proceeding.

4.1. Disabling all OpenStack Services

The first step to performing a complete upgrade of Red Hat OpenStack Platform on a node involves shutting down all OpenStack services. This step differs based on whether the node OpenStack uses high availability tools for management (e.g. using Pacemaker on Controller nodes). This step contains instructions for both node types.

Standard Nodes

Before updating, take a systemd snapshot of the OpenStack Platform services.

# systemctl snapshot openstack-services

Disable the main OpenStack Platform services:

# systemctl stop 'openstack-*'
# systemctl stop 'neutron-*'
# systemctl stop 'openvswitch'

High Availability Nodes

You need to disable all OpenStack services but leave the database and load balancing services active. For example, switch the HAProxy, Galera, and MongoDB services to unmanaged in Pacemaker:

# pcs resource unmanage haproxy
# pcs resource unmanage galera
# pcs resource unmanage mongod

Disable the remaining Pacemaker-managed resources by setting the stop-all-resources property on the cluster. Run the following on a single member of your Pacemaker cluster:

# pcs property set stop-all-resources=true

Wait until all Pacemaker-managed resources have stopped. Run the pcs status command to see the status of each resource.

# pcs status
Important

HAProxy might show a broadcast message for unavailable services. This is normal behavior.

4.2. Performing a Package Upgrade

The next step upgrades all packages on a node. Perform this step on each node with OpenStack services.

Change to the Red Hat OpenStack Platform 10 repository using the subscription-manager command:

# subscription-manager repos --disable=rhel-7-server-openstack-9-rpms
# subscription-manager repos --enable=rhel-7-server-openstack-10-rpms

Run the yum update command on the node:

# yum update

Wait until the package upgrade completes.

Review the resulting configuration files. The upgraded packages will have installed .rpmnew files appropriate to the Red Hat OpenStack Platform 10 version of the service. New versions of OpenStack services may deprecate certain configuration options. You should also review your OpenStack logs for any deprecation warnings, because these may cause problems during future upgrades. For more information on the new, updated and deprecated configuration options for each service, see Configuration Reference available from Red Hat OpenStack Platform Documentation Suite.

Perform the package upgrade on each node in your environment.

4.3. Performing Synchronization of all Databases

The next step upgrades the database for each service.

Note

Flush expired tokens in the Identity service to decrease the time required to synchronize the database.

# keystone-manage token_flush

Upgrade the database schema for each service that uses the database. Run the following commands on the node hosting the service’s database.

Table 4.1. Commands to Synchronize OpenStack Service Databases

ServiceProject nameCommand

Identity

keystone

# su -s /bin/sh -c "keystone-manage db_sync" keystone

Image Service

glance

# su -s /bin/sh -c "glance-manage db_sync" glance

Block Storage

cinder

# su -s /bin/sh -c "cinder-manage db sync" cinder

Orchestration

heat

# su -s /bin/sh -c "heat-manage db_sync" heat

Compute

nova

# su -s /bin/sh -c "nova-manage api_db sync" nova

# su -s /bin/sh -c "nova-manage db sync" nova

Telemetry

ceilometer

# ceilometer-dbsync

Telemetry Alarming

aodh

# aodh-dbsync

Telemetry Metrics

gnocchi

# gnocchi-upgrade

Clustering

sahara

# su -s /bin/sh -c "sahara-db-manage upgrade heads" sahara

Networking

neutron

# su -s /bin/sh -c "neutron-db-manage upgrade heads" neutron

4.4. Enabling all OpenStack Services

The final step enables the OpenStack services on the node. This step differs based on whether the node OpenStack uses high availability tools for management. For example, using Pacemaker on Controller nodes. This step contains instructions for both node types.

Standard Nodes

Restart all OpenStack services:

# systemctl isolate openstack-services.snapshot

High Availability Nodes

Restart your resources through Pacemaker. Reset the stop-all-resources property on a single member of your Pacemaker cluster. For example:

# pcs property set stop-all-resources=false

Wait until all resources have started. Run the pcs status command to see the status of each resource:

# pcs status

Enable Pacemaker management for any unmanaged resources, such as the databases and load balancer:

# pcs resource manage haproxy
# pcs resource manage galera
# pcs resource manage mongod
Note

This upgrade scenario has some additional upgrade procedures detailed in Chapter 7, Additional Procedures for Non-Director Environments. These additional procedures are optional for manual environments but help align with the current OpenStack Platform recommendations.

4.5. Post-Upgrade Notes

New versions of OpenStack services may deprecate certain configuration options. You should also review your OpenStack logs for any deprecation warnings, because these may cause problems during a future upgrade. For more information on the new, updated and deprecated configuration options for each service, see Configuration Reference available from Red Hat OpenStack Platform Documentation Suite.