Chapter 3. Director-Based Environments: Performing Upgrades to Major Versions

To update your modified template collection, you need to:

This helps identify customizations from the old template collection that you can incorporate into the new template collection. Incorporate customization into the new custom template collection.

This step replaces the old OpenStack Telemetry Alarming (ceilometer-alarm) service with a new component (aodh). This process involves running openstack overcloud deploy with a new environment file (major-upgrade-aodh.yaml), which:

Run the openstack overcloud deploy from your Undercloud and include the major-upgrade-aodh.yaml environment file. Make sure you also include all custom environment files relevant to your environment, such as network isolation and storage.

This following is an example of an openstack overcloud deploy command with the added file:

$ openstack overcloud deploy --force-postconfig --templates \
  --control-scale 3 \
  --compute-scale 3 \
  -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml  \
  -e /home/stack/templates/network_env.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-aodh.yaml

Wait until the Overcloud updates with the new environment file’s configuration.

This step upgrades the OpenStack Identity (keystone) service to run as a Web Server Gateway Interface (WSGI) applet under httpd instead of a standalone service on Controller nodes. This process automatically disables the standalone openstack-keystone service and installs the necessary configuration to enable the WSGI applet.

Run the openstack overcloud deploy from your Undercloud and include the major-upgrade-keystone-liberty-mitaka.yaml environment file. Make sure you also include all custom environment files relevant to your environment, such as network isolation and storage.

This following is an example of an openstack overcloud deploy command with the added file:

$ openstack overcloud deploy --templates \
  --control-scale 3 \
  --compute-scale 3 \
  -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml  \
  -e /home/stack/templates/network_env.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-keystone-liberty-mitaka.yaml

Wait until the Overcloud updates with the new environment file’s configuration.

This step installs scripts on each non-Controller node. These script perform the major version package upgrades and configuration. Each script differs depending on the node type. For example, Compute nodes receive different upgrade scripts to Ceph Storage nodes.

This initialization step also updates enabled repositories on all overcloud nodes. This means you do not need to disable old repositories and enable new repositories manually.

Run the openstack overcloud deploy from your Undercloud and include the major-upgrade-pacemaker-init.yaml environment file. Make sure you also include all custom environment files relevant to your environment, such as network isolation and storage.

This following is an example of an openstack overcloud deploy command with the added file:

$ openstack overcloud deploy --templates \
  --control-scale 3 \
  --compute-scale 3 \
  -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml  \
  -e /home/stack/templates/network_env.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-pacemaker-init.yaml

Wait until the Overcloud updates with the new environment file’s configuration.

The director uses the upgrade-non-controller.sh command to run the upgrade script passed to each non-Controller node from the major-upgrade-pacemaker-init.yaml environment file. For this step, upgrade each Object Storage node using the following command:

$ for NODE in `nova list | grep swift | awk -F "|" '{ print $2 }'` ; do upgrade-non-controller.sh --upgrade $NODE ; done

Wait until each Object Storage node completes its upgrade.

Check the /var/log/yum.log file on each node to see if either the kernel or openvswitch packages have updated their major or minor versions. If so, perform a reboot of each node:

Upgrading the Controller nodes involves including another environment file (major-upgrade-pacemaker.yaml) that provides a full upgrade to Controller nodes running high availability tools.

Run the openstack overcloud deploy from your Undercloud and include the major-upgrade-pacemaker.yaml environment file. Remember to include all custom environment files relevant to your environment, such as network isolation and storage.

This following is an example of an openstack overcloud deploy command with the added file:

$ openstack overcloud deploy --templates \
  --control-scale 3 \
  --compute-scale 3 \
  -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml \
  -e network_env.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-pacemaker.yaml

Wait until the Overcloud updates with the new environment file’s configuration.

Check the /var/log/yum.log file on each node to see if either the kernel or openvswitch packages have updated their major or minor versions. If so, perform a reboot of each node:

The director uses the upgrade-non-controller.sh command to run the upgrade script passed to each non-Controller node from the major-upgrade-pacemaker-init.yaml environment file. For this step, upgrade each Ceph Storage node with the following command:

Upgrade each Ceph Storage nodes:

$ for NODE in `nova list | grep ceph | awk -F "|" '{ print $2 }'` ; do upgrade-non-controller.sh --upgrade $NODE ; done

Check the /var/log/yum.log file on each node to see if either the kernel or openvswitch packages have updated their major or minor versions. If so, perform a reboot of each node:

Upgrade each Compute node individually and ensure zero downtime of instances in your OpenStack Platform environment. This involves the following workflow:

List all Compute nodes and their UUIDs:

$ nova list | grep "compute"

Select a Compute node to upgrade and first migrate its instances using the following process:

The director uses the upgrade-non-controller.sh command to run the upgrade script passed to each non-Controller node from the major-upgrade-pacemaker-init.yaml environment file. Upgrade each Compute node with the following command:

$ source ~/stackrc
$ upgrade-non-controller.sh --upgrade NODE_UUID

Replace NODE_UUID with the UUID of the chosen Compute node. Wait until the Compute node completes its upgrade.

Check the /var/log/yum.log file on the Compute node you have upgraded to see if either the kernel or openvswitch packages have updated their major or minor versions. If so, perform a reboot of each node:

Repeat the migration and reboot process for each node individually until you have rebooted all nodes.

The director needs to run through the upgrade finalization to ensure the Overcloud stack is synchronized with the current Heat template collection. This involves an environment file (major-upgrade-pacemaker-converge.yaml), which you include using the openstack overcloud deploy command.

Run the openstack overcloud deploy from your Undercloud and include the major-upgrade-pacemaker-converge.yaml environment file. Make sure you also include all custom environment files relevant to your environment, such as network isolation and storage.

This following is an example of an openstack overcloud deploy command with the added file:

$ openstack overcloud deploy --force-postconfig --templates \
  --control-scale 3 \
  --compute-scale 3 \
  -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml \
  -e network_env.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-pacemaker-converge.yaml

Wait until the Overcloud updates with the new environment file’s configuration.

This completes the Overcloud upgrade procedure.

Be aware of the following notes after upgrading the Overcloud to Red Hat OpenStack Platform 9: