Chapter 4. Updating the Overcloud

This process updates the overcloud.

Prerequisites

  • You have updated the undercloud to the latest version.

4.1. Running the overcloud update preparation

To prepare the overcloud for the update process, run the openstack overcloud update prepare command, which performs the following tasks:

  • Updates the overcloud plan to OpenStack Platform 16.1
  • Prepares the nodes for the update

Prerequisites

  • If you use a Ceph subscription and have configured director to use the overcloud-minimal image for Ceph storage nodes, you must ensure that in the roles_data.yaml role definition file, the rhsm_enforce parameter is set to False.

Procedure

  1. Source the stackrc file:

    $ source ~/stackrc
  2. Run the update preparation command:

    $ openstack overcloud update prepare \
        --templates \
        --stack <stack_name> \
        -r <roles_data_file> \
        -n <network_data_file> \
        -e <environment_file> \
        -e <environment_file> \
        …​

    Include the following options relevant to your environment:

    • If the name of your overcloud stack is different to the default name overcloud, include the --stack option in the update preparation command and replace <stack_name> with the name of your stack.
    • If using your own custom roles, include your custom roles (<roles_data>) file (-r)
    • If using custom networks, include your composable network (_<network_data>) file _ (-n)
    • Any custom configuration environment files (-e)
  3. Wait until the update preparation completes.

4.2. Running the container image preparation

The overcloud requires the latest OpenStack Platform 16.1 container images before performing the update. This involves executing the container_image_prepare external update process. To execute this process, you must run the openstack overcloud external-update run command against tasks tagged with the container_image_prepare tag. These tasks perform the following actions:

  • Automatically prepare all container image configuration relevant to your environment.
  • Pull the relevant container images to your undercloud, unless you have previously disabled this option.
Note

If you are not using the default stack name (overcloud), set your stack name with the --stack <stack_name> option replacing <stack_name> with the name of your stack.

Procedure

  1. Source the stackrc file:

    $ source ~/stackrc
  2. Run the openstack overcloud external-update run command against tasks tagged with the container_image_prepare tag:

    $ openstack overcloud external-update run --stack <stack_name> --tags container_image_prepare

4.3. Updating all Controller nodes

This process updates all the Controller nodes to the latest OpenStack Platform 16.1 version. The process involves running the openstack overcloud update run command and including the --limit Controller option to restrict operations to the Controller nodes only.

Important

Until BZ#1872404 is resolved, for nodes based on composable roles, you must update the Database role first, before you can update Controller, Messaging, Compute, Ceph, and other roles.

Note

If you are not using the default stack name (overcloud), set your stack name with the --stack <stack_name> option replacing <stack_name> with the name of your stack.

Procedure

  1. Source the stackrc file:

    $ source ~/stackrc
  2. Run the update command:

    $ openstack overcloud update run --stack <stack_name> --limit Controller --playbook all
  3. Wait until the Controller node update completes.

4.4. Updating all Compute nodes

This process updates all Compute nodes to the latest OpenStack Platform 16.1 version. The process involves running the openstack overcloud update run command and including the --limit Compute option to restrict operations to the Compute nodes only.

Parallelization considerations

When you update a large number of Compute nodes, to improve performance, you can run the openstack overcloud update run command with the --limit Compute option in parallel on batches of 20 nodes. For example, if you have 80 Compute nodes in your deployment, you can run the following commands to update the Compute nodes in parallel:

$ openstack overcloud update run --limit 'Compute[0:19]' > update-compute-0-19.log 2>&1 &
$ openstack overcloud update run --limit 'Compute[20:39]' > update-compute-20-39.log 2>&1 &
$ openstack overcloud update run --limit 'Compute[40:59]' > update-compute-40-59.log 2>&1 &
$ openstack overcloud update run --limit 'Compute[60:79]' > update-compute-60-79.log 2>&1 &

The 'Compute[0:19]', 'Compute[20:39]', 'Compute[40:59]', and 'Compute[60:79]' way of partitioning the nodes space is random and you don’t have control over which nodes are updated.

To update specific Compute nodes, list the nodes that you want to update in a batch separated by a comma:

$ openstack overcloud update run --limit <Compute0>,<Compute1>,<Compute2>,<Compute3>
Note

If you are not using the default stack name (overcloud), set your stack name with the --stack <stack_name> option replacing <stack_name> with the name of your stack.

Procedure

  1. Source the stackrc file:

    $ source ~/stackrc
  2. Run the update command:

    $ openstack overcloud update run --stack <stack_name> --limit Compute --playbook all
  3. Wait until the Compute node update completes.

4.5. Updating all HCI Compute nodes

This process updates the Hyperconverged Infrastructure (HCI) Compute nodes. The process involves:

  • Running the openstack overcloud update run command and including the --limit ComputeHCI option to restrict operations to the HCI nodes only.
  • Running the openstack overcloud external-update run --tags ceph command to perform an update to a containerized Red Hat Ceph Storage 4 cluster.
Note

If you are not using the default stack name (overcloud), set your stack name with the --stack <stack_name> option replacing <stack_name> with the name of your stack.

Procedure

  1. Source the stackrc file:

    $ source ~/stackrc
  2. Run the update command:

    $ openstack overcloud update run --stack <stack_name> --limit ComputeHCI --playbook all
  3. Wait until the node update completes.
  4. Run the Ceph Storage update command. For example:

    $ openstack overcloud external-update run --stack <stack_name> --tags ceph
  5. Wait until the Compute HCI node update completes.

4.6. Updating all Ceph Storage nodes

This process updates the Ceph Storage nodes. The process involves:

  • Running the openstack overcloud update run command and including the --limit CephStorage option to restrict operations to the Ceph Storage nodes only.
  • Running the openstack overcloud external-update run command to run ceph-ansible as an external process and update the Red Hat Ceph Storage 3 containers.
Note

If you are not using the default stack name (overcloud), set your stack name with the --stack <stack_name> option replacing <stack_name> with the name of your stack.

Procedure

  1. Source the stackrc file:

    $ source ~/stackrc
  2. Run the update command:

    $ openstack overcloud update run --stack  <stack_name> --limit CephStorage --playbook all
  3. Wait until the node update completes.
  4. Run the Ceph Storage container update command:

    $ openstack overcloud external-update run --tags ceph
  5. Wait until the Ceph Storage container update completes.

4.7. Performing online database updates

Some overcloud components require an online upgrade (or migration) of their databases tables. This involves executing the online_upgrade external update process. To execute this process, run the openstack overcloud external-update run command against tasks tagged with the online_upgrade tag. This performs online database updates to the following components:

  • OpenStack Block Storage (cinder)
  • OpenStack Compute (nova)

Procedure

  1. Source the stackrc file:

    $ source ~/stackrc
  2. Run the openstack overcloud external-update run command against tasks that use the online_upgrade tag:

    $ openstack overcloud external-update run --tags online_upgrade

4.8. Finalizing the update

The update requires a final step to update the overcloud stack. This ensures the stack’s resource structure aligns with a regular deployment of OpenStack Platform 16.1 and allows you to perform standard openstack overcloud deploy functions in the future.

Procedure

  1. Source the stackrc file:

    $ source ~/stackrc
  2. Run the update finalization command:

    $ openstack overcloud update converge \
        --templates \
        --stack <stack_name> \
        -r <roles_data_file> \
        -n <network_data_file> \
        -e <environment_file> \
        -e <environment_file> \
        ...
        ...

    Include the following options relevant to your environment:

    • If the name of your overcloud stack is different to the default name overcloud, include the --stack option in the update preparation command and replace <stack_name> with the name of your stack.
    • If using your own custom roles, include your custom roles (<roles_data>) file (-r)
    • If using custom networks, include your composable network (<network_data>) file (-n)
    • Any custom configuration environment files (-e).
  3. Wait until the update finalization completes.