Chapter 4. Upgrade OpenStack by Updating Each Service Individually, with Live Compute

With this procedure, all services are upgraded individually. Each service upgrade involves a package update and a database schema update.
A live Compute upgrade minimizes interruptions to your Compute service, with only a few minutes for the smaller services, and a longer migration interval for the workloads moving to newly-upgraded Compute hosts. Existing workloads can run indefinitely, and you do not need to wait for a database migration.

Note

This method may require additional hardware resources to bring up the Red Hat Enterprise Linux OpenStack Platform 7 Compute nodes.

Note

The procedures in this chapter follow the architectural naming convention followed by all Red Hat Enterprise Linux OpenStack Platform documentation. If you are unfamiliar with this convention, refer to Architecture Guide available at: Red Hat Enterprise Linux OpenStack Platform Documentation Suite before proceeding.
Before you begin upgrading your OpenStack deployment, subscribe to the proper channels. For more information, refer Chapter 2, Prerequisites

4.1. Upgrading OpenStack by Updating Service Individually, With Live Compute in a Non-HA Environment

This section describes the steps you should follow to upgrade your cloud deployment by updating one service at a time with live compute in a non High Availability (HA) environment.
  1. Pre-upgrade tasks:
    On all of your hosts:
    1. Install the yum repository for Red Hat Enterprise Linux OpenStack Platform 7 (Kilo).
    2. Upgrade the openstack-selinux package, if available: 
      # yum upgrade openstack-selinux
      This is necessary to ensure that the upgraded services will run correctly on a system with SELinux enabled.
  2. Upgrade each of your services:
    The following steps provide specific instructions for each service, and the order in which they should be upgraded.
    1. Identity (keystone)
      Earlier versions of the installer may not have configured your system to automatically purge expired Keystone token, it is possible that your token table has a large number of expired entries. This can dramatically increase the time it takes to complete the database schema upgrade.
      To flush expired tokens from the database and alleviate the problem, the keystone-manage command can be used before running the Identity database upgrade.
      This will flush expired tokens from the database. You can arrange to run this command periodically (e.g., daily) using cron.
      On your Identity host, run: 
      # openstack-service stop keystone
# yum -d1 -y upgrade \*keystone\*
# keystone-manage token_flush
# openstack-db --service keystone --update
# openstack-service start keystone
    2. Object Storage (swift)
      On your Object Storage hosts, run: 
      # openstack-service stop swift
# yum -d1 -y upgrade \*swift\*
# openstack-service start swift
    3. Image Service (glance)
      On your Image Service host, run: 
      # openstack-service stop glance
# yum -d1 -y upgrade \*glance\*
# openstack-db --service glance --update
# openstack-service start glance
    4. Block Storage (cinder)
      On your Block Storage host, run: 
      # openstack-service stop cinder
# yum -d1 -y upgrade \*cinder\*
# openstack-db --service cinder --update
# openstack-service start cinder
    5. Orchestration (heat)
      On your Orchestration host, run: 
      # openstack-service stop heat
# yum -d1 -y upgrade \*heat\*
# openstack-db --service heat --update
# openstack-service start heat
    6. Telemetry (ceilometer)
      1. On all nodes hosting Telemetry component services, run: 
        # openstack-service stop ceilometer
# yum -d1 -y upgrade \*ceilometer\*
      2. On the controller node, where database is installed, run: 
        # ceilometer-dbsync
        This command allows you to configure MySQL as a back-end for Telemetry service.
        For a list of Telemetry component services, refer to Launch the Telemetry API and Agents.
      3. After completing the package upgrade, restart the Telemetry service by running the following command on all nodes hosting Telemetry component services: 
        # openstack-service start ceilometer
    7. Compute (nova)
      1. If you are performing a rolling upgrade of your compute hosts you need to set explicit API version limits to ensure compatibility between your Juno and Kilo environments.
        Before starting Kilo controller or compute services, you need to set the compute option in the [upgrade_levels] section of nova.conf to juno: 
        # crudini --set /etc/nova/nova.conf upgrade_levels compute juno
        You need to make this change on your controllers and on your compute hosts.
        You should undo this operation after upgrading all of your compute hosts to OpenStack Kilo.
      2. On your Compute host, run: 
        # openstack-service stop nova
# yum -d1 -y upgrade \*nova\*
# openstack-db --service nova --update
      3. After fully upgrading to Kilo (i.e. all nodes are running Kilo), you should start a background migration of flavor information. Kilo conductor nodes will do this on the fly when necessary, but the rest of the idle data needs to be migrated in the the background. Run the following command as a nova user: 
        # runuser -u nova -- nova-manage db migrate_flavor_data
      4. After you have upgraded all of your hosts to Kilo, you will want to remove the API limits configured in the previous step. On all of your hosts: 
        # crudini --del /etc/nova/nova.conf upgrade_levels compute
      5. Restart the Compute service on all the compute hosts and controllers: 
        # openstack-service start nova
    8. OpenStack Networking (neutron)
      1. On your OpenStack Networking host, run: 
        # openstack-service stop neutron
# yum -d1 -y upgrade \*neutron\*
# openstack-db --service neutron --update
      2. Once you have completed upgrading the OpenStack Networking service, you need to edit the rootwrap dhcp.filter configuration file.
        To do so, in the /usr/share/neutron/rootwrap/dhcp.filters file, replace the value of dnsmasq. For example, replace: 
        dnsmasq: EnvFilter, env, root, CONFIG_FILE=, NETWORK_ID=, dnsmasq
        with: 
        dnsmasq: CommandFilter, dnsmasq, root
      3. Restart the OpenStack Networking service: 
        # openstack-service start neutron
    9. Dashboard (horizon)
      On your Dashboard host, run: 
      # yum -y upgrade \*horizon\* \*openstack-dashboard\*
# yum -d1 -y upgrade \*horizon\* \*python-django\*
# systemctl restart httpd
  3. Post-upgrade tasks:
    1. After completing all of your individual service upgrades, you should perform a complete package upgrade on all of your systems: 
      # yum upgrade
      This will ensure that all packages are up-to-date. You may want to schedule a restart of your OpenStack hosts at a future date in order to ensure that all running processes are using updated versions of the underlying binaries.
    2. Review the resulting configuration files. The upgraded packages will have installed .rpmnew files appropriate to the Red Hat Enterprise Linux OpenStack Platform 7 version of the service.
      New versions of OpenStack services may deprecate certain configuration options. You should also review your OpenStack logs for any deprecation warnings, because these may cause problems during a future upgrade. For more information on the new, updated and deprecated configuration options for each service , see Configuration Reference available from: Red Hat Enterprise Linux OpenStack Platform Documentation Suite.