Upgrading OpenStack by Updating All Services Simultaneously

Procedure 1. Upgrading OpenStack components on a host

1. Ensure the openstack-utils package is installed:

   # yum install openstack-utils

2. Take down all OpenStack services on all the nodes. This step depends on how your services are distributed among your nodes.
   To stop all the OpenStack services running on a node, log in to the node and run:

   # openstack-service stop

3. Perform a complete upgrade of all packages, and then flush expired tokens in the Identity service (might decrease the time required to synchronize the database):

   # yum upgrade
   # keystone-manage token_flush

4. Upgrade the database schema for each service that uses the database. To do so, log in to the node hosting the service and run:

   # openstack-db --service SERVICENAME --update

   Use the service's project name as the SERVICENAME. For example, to upgrade the database schema of the Identity service:

   # openstack-db --service keystone --update

   
   ⁠
   

   Table 7.  Project name of each OpenStack service that uses the database

   Service	Project name
   Identity	keystone
   Block Storage	cinder
   Image Service	glance
   Compute	nova
   Networking	neutron
   Orchestration	heat
   Dashboard	horizon

5. Review the resulting configuration files. The upgraded packages will have installed .rpmnew files appropriate to the Red Hat Enterprise Linux OpenStack Platform 5 version of the service.

Procedure 2. Configuring neutron and nova interactions

1. Configure neutron to notify nova when the port status is active:

   # openstack-config --set /etc/neutron/neutron.conf DEFAULT notify_nova_on_port_status_changes true

2. Configure neutron to use the new address of your Compute API service:

   # openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_url http://COMPUTEAPIADDRESS:8774/v2

   Replace COMPUTEAPIADDRESS with the host name or IP address of the Compute API service (openstack-nova-api). In most deployments, this service is also hosted in the Compute controller node.
3. Provide neutron with the required credentials:

   # openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_admin_username NOVAADMIN

   # openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_admin_password NOVAPASSWD

   # openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_admin_tenant_id NOVATENANT

   # openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_admin_auth_url KEYSTONEENDPOINT

   Where:
   • NOVAADMIN is a user with administrative rights within the OpenStack environment.

     Note

     The Deploying OpenStack: Learning Environments (Manual Set Up) contains instructions on how to configure Compute service authentication. Here, a user named compute with the role admin and tenant services is created specifically for the Compute service. This user is appropriate as NOVAADMIN.
   • NOVAPASSWD is the corresponding password of NOVAADMIN.

     Note

     If you cannot find or determine the password of an existing NOVAADMIN, you can create a new administrative user instead. For instructions, see https://access.redhat.com/articles/1247413.
   • NOVATENANT is the UUID of the primary tenant associated with NOVAADMIN.
   • KEYSTONEENDPOINT is the administrative endpoint URL of the keystone service.

Procedure 3. Setting up initial ML2 configuration

1. Install the openstack-neutron-ml2 package:

   # yum install openstack-neutron-ml2

2. Create a symbolic link to direct Networking to the ML2 config file ml2_conf.ini:

   # ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini

3. Add the required plug-in settings to the [ml2] section of the ml2_conf.ini file:
   
   ⁠
   

   Table 8.  Required plug-in settings for ML2

   Configuration option	Description
   mechanism_drivers	An ordered, comma-delimited list of networking mechanism driver entrypoints to be loaded from the neutron.ml2.mechanism_drivers namespace. Supported values are local, flat, vlan, gre, and vxlan.
   tenant_network_types	An ordered, comma-delimited list of network_types to allocate as tenant networks. Supported values are openvswitch, linuxbridge, and l2population.
   type_drivers	Comma-delimited list of network type driver entry points to be loaded from the neutron.ml2.type_drivers namespace. Supported values are the same as mechanism_drivers.

4. Add the required settings for your chosen mechanism driver. Refer to the following table for the required setting/s per network type driver (type_drivers):
   
   ⁠
   

   Table 9.  Required settings per driver type

   type_driver	Section	Configuration option	Description
   flat	[ml2_type_flat]	flat_networks	List of physical_network names with which flat networks can be created. Use * to allow flat networks with arbitrary physical_network names.
   gre	[ml2_type_gre]	tunnel_id_ranges	Comma-separated list of MIN:MAX tunneling tuples enumerating ranges of GRE tunnel IDs that are available for tenant network allocation
   vlan	[ml2_type_vlan]	network_vlan_ranges	List of physical network names usable for VLAN provider and tenant networks; each name in the list can include ranges of VLAN tags available for allocation to tenant networks. The syntax for each name in the list can be in the form physical_network:vlan_min:vlan_max or physical_network.
   vxlan (1 of 2)	[ml2_type_vxlan]	vni_ranges	Comma-separated list of vni_min:vni_max tuples enumerating ranges of VXLAN VNI IDs that are available for tenant network allocation
   vxlan (2 of 2)	[ml2_type_vxlan]	vxlan_group	Multicast group for VXLAN. If unset, disables VXLAN multicast mode.

Procedure 4. Migrating to ML2

1. To migrate the plug-in database using neutron.db.migration.migrate_to_ml2, run:

   # python -m neutron.db.migration.migrate_to_ml2 PLUGIN mysql://DBADMIN:DBPASS@DBHOST/DBNAME

   Where:
   • PLUGIN is the original plug-in used. Supported values are linuxbridge and openvswitch.
   • DBADMIN is a valid admin username to the OpenStack Networking database (for example, neutron).
   • DBPASS is the corresponding password of DBADMIN.
   • DBHOST is the IP address of the OpenStack Networking database host.
   • DBNAME is the name of the OpenStack Networking database. By default, PackStack assigns the database name ovs_neutron or neutron_linux_bridge, depending on the plug-in selected.

   Note

   The actual values used by the Networking service for DBADMIN, DBPASS, DBHOST, and DBNAME are set in the [database] section of the /etc/neutron/neutron.conf. Specifically, these values are configured in the connection setting with the following syntax:

   connection = mysql://DBADMIN:DBPASS@DBHOST/DBNAME

   Optional parameters include:
   • --tunnel-type TUNNEL: if you configured your plug-in to use tunneling, then use this parameter to also migrate tunneling data across. Replace TUNNEL with the type of tunneling you configured; supported values are GRE and VXLAN.
   • --vxlan-udp-port VXLANPORT: if you are migrating VXLAN tunneling data, then you can use this parameter to supply the UDP port that VXLAN tunnels should use (replacing VXLANPORT accordingly). If you do not use this parameter with --tunnel-type VXLAN, then VXLAN tunnels will use port 4789 by default.
2. After migrating the plug-in database, manually enable the ML2 plug-in. To do so, first open the /etc/neutron/neutron.conf configuration file and set the core_plugin parameter to the ML2 plug-in:

   core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

3. Then, add the L3 router plugin (neutron.services.l3_router.l3_router_plugin.L3RouterPlugin) to the service_plugins parameter:

   service_plugins = neutron.services.firewall.fwaas_plugin.FirewallPlugin,neutron.services.l3_router.l3_router_plugin.L3RouterPlugin

   Both parameters are in the DEFAULT section of /etc/neutron/neutron.conf. When manually setting configuration files, ensure that each line has no whitespace at the beginning; these could cause parsing errors and prevent services from functioning properly.

Procedure 5. Configuring the OpenStack Networking service to complete its upgrade

1. Set a longer heartbeat for all Networking service agents. The following commands set the recommended values for this release.
   On the Networking controller node, run:

   # openstack-config --set /etc/neutron/neutron.conf DEFAULT agent_down_time 75

   Then, on each node hosting a Networking service agent, run:

   # openstack-config --set /etc/neutron/neutron.conf agent report_interval 30

   With these settings, the Networking service will check every 30 seconds whether an agent is down; the agent is only declared down if it is unreachable for 75 seconds.
2. Kill the dnsmasq process on the Networking controller node (this is required for the DHCP agent to work):

   # killall dnsmasq

3. Restart the OpenStack Networking service to apply all the settings. To do so, run the following command on all nodes hosting Networking services:

   # openstack-service start neutron

Procedure 6. Configuring the firewall to allow Compute Service traffic

1. Log in as the root user to the Compute node.
2. Open the /etc/sysconfig/iptables file in a text editor.
3. Add an INPUT rule allowing TCP traffic on ports used for console traffic by adding the following line to the file:

   -A INPUT -p tcp --dport 5900:6900 -j ACCEPT

   The new rule must appear before any INPUT rules that REJECT traffic.
4. Save the changes to the /etc/sysconfig/iptables file.
5. Restart the iptables service to ensure that the change takes effect.

   # service iptables restart

Procedure 7. Manually configuring dashboard local settings for upgrade

1. Back up your existing /etc/openstack-dashboard/local_settings file.
2. Replace the contents of /etc/openstack-dashboard/local_settings with the contents of local_settings.rpmnew.
3. Update your new /etc/openstack-dashboard/local_settings file with the OPENSTACK_HOST and SECRET_KEY values from your backup /etc/openstack-dashboard/local_settings.
   If you have any other required, non-default settings from the backup /etc/openstack-dashboard/local_settings file, copy them to the new /etc/openstack-dashboard/local_settings file as well.
4. If you are running Django 1.5 (or later), you must ensure that there is a correctly configured ALLOWED_HOSTS setting in your local_settings file. ALLOWED_HOSTS contains a list of host names that can be used to contact your Dashboard service:
   • If people will be accessing the Dashboard service using "http://dashboard.example.com", you would set:

     ALLOWED_HOSTS=['dashboard.example.com']

   • If you are running the Dashboard service on your local system, you can use:

     ALLOWED_HOSTS=['localhost']

   • If people will be using IP addresses instead of, or in addition to, hostnames, an example might be:

     ALLOWED_HOSTS=['dashboard.example.com', '192.168.122.200']

   Note

   For more information about the ALLOWED_HOSTS setting, see the Django Documentation.
5. Restart the web server to apply all changes:

   # service httpd restart

Procedure 8. Upgrading the Database Server

1. From each node in your Red Hat Enterprise Linux OpenStack Platform environment, stop all Red Hat Enterprise Linux OpenStack Platform services that access the database server.
2. On the system on which the database server is installed, stop the database server:

   # service mysqld stop

3. Remove the MySQL server package:

   # yum -y remove mysql-server

4. Ensure all packages are up to date:

   # yum -y update

5. Install the MariaDB server packages:

   # yum -y install mariadb-galera-server

6. Configure the database server to start at boot time:

   # chkconfig mysqld on

7. Start the database server:

   # service mysqld start

8. Upgrade the internal mysql database schema, entering the password for the administrative user when prompted:

   # mysql_upgrade -u root -p

9. From each node in your Red Hat Enterprise Linux OpenStack Platform environment, start all Red Hat Enterprise Linux OpenStack Platform services that access the database server.