Chapter 3. Release Information

These release notes highlight technology preview items, recommended practices, known issues, and deprecated functionality that you should consider when you deploy this release of Red Hat OpenStack Platform. Notes for updates released during the support lifecycle of this Red Hat OpenStack Platform release appear in the advisory text associated with each update.

3.1. Red Hat OpenStack Platform 16.2 GA

These release notes highlight technology preview items, recommended practices, known issues, and deprecated functionality to be taken into consideration when deploying this release of Red Hat OpenStack Platform.

3.1.1. Advisory List

This release includes the following advisories:

Release of components for OSP 16.2
Red Hat OpenStack Platform 16.2 deployment images
Release of containers for OSP 16.2 director operator tech preview
Moderate: Red Hat OpenStack Platform 16.2 (etcd) security update
Important: Red Hat OpenStack Platform 16.2 (openstack-neutron) security update
Release of components for OSP 16.2 - Containers
Moderate: Red Hat OpenStack Platform 16.2 (python-django20) security update

3.1.2. Bug Fix

These bugs were fixed in this release of Red Hat OpenStack Platform:

Before this update, writing an image to RBD could be very slow. This update improves the process for writing an image to RBD, which improves the time it takes for images to be written to RBD.

Typically, when you create an encrypted volume from a snapshot of an encrypted volume, the source volume is the same size or smaller than the destination volume.

In previous releases, if you created an encrypted volume from a snapshot of an encrypted volume, and the destination volume was close to or equal to the size of the source volume, the Block Storage service (cinder) silently truncated the data in the new destination volume.

With this release, the Block Storage service calculates the size of the destination volume to include the current size of the encryption header, which eliminates the data truncation.

Before this update, when you resized or migrated an instance that had a vGPU flavor you needed to rebuild the instance manually to re-allocate the vGPU resources. With this update, instances with a vGPU flavor are automatically re-allocated the vGPU resources after resize and cold migration operations.

Previously, the Shared File Systems service (manila) API that brings external shares into service management did not check for duplicated export locations. An existing share brought into the service multiple times results in an inconsistent state.

With this release, the API evaluates the export locations of known or existing shares before allowing external shares to be managed, and prevents existing shares from being erroneously brought into the Shared File Systems service again.

Before this update, RBD performance was degraded when multiple instances were launched simultaneously. This was due to the Image service starting multiple threads to perform the same copying operation. This update resolves the issue.
This update fixes an Image service (glance) configuration error that prevented users from creating a virtual machine with watchdog by setting flavor metadata.
In prior releases, you could not delete snapshots that have snapshot dependencies. With this release, you can delete snapshots that have snapshot dependencies.
When multiple storage backends are configured on the Shared File Systems service (manila), each storage back end might support a different storage protocol. Before this update, the Shared File Systems service scheduler did not consider the storage protocol and capability of the shared storage back ends when deciding where to place them, which caused share provisioning to fail. With this update, the Shared File Systems service scheduler now automatically considers the share type extra specs with the storage protocol, which makes it possible use the CapabilitiesFilter to compare storage back end capabilities and provision shares successfully.
Before this update, validation results were not being logged and validation artifacts were not being collected as the permissions required to access the requested logging directory were not granted. This update resolves the issue, and validation results are successfully logged and validation artifacts are collected.
The Unisphere for PowerMax REST endpoints have changed from 91 to 92. This update changes how URIs are created, to allow for full coverage of all possible Unisphere REST API endpoints.

When an instance is created, the Compute service (nova) sanitizes the instance display name to generate a valid hostname when DNS integration is enabled in the Networking service (neutron).

Before this update, the sanitization did not replace periods ('.') in instance names, for example, 'rhel-8.4'. This could result in display names being recognized as Fully Qualified Domain Names (FQDNs) which produced invalid hostnames. When instance names contained periods and DNS integration was enabled in the Networking service, the Networking service would reject the invalid hostname resulting in a failure to create the instance and a HTTP 500 server error from the Compute service.

With this update, periods are now replaced by hyphens in instance names to prevent hostnames being parsed as FQDNs. You can continue to use free-form strings for instance display names.

Before this update, some exceptions were not being caught during connections to iSCSI portals, such as failures in iscsiadm -m session. This caused _connect_vol threads to abort unexpectedly in some failure patterns, which caused subsequent steps to hang while waiting for results from _connect_vol threads. This update ensures that any exceptions during connections to iSCSI portals are handled correctly in the _connect_vol method, to avoid unhandled exceptions during connecting to iSCSI portals, and unexpected aborts that have no updated thread results.
This update adds Challenge Handshake Authentication Protocol (CHAP) support to the Dell EMC PowerStore driver. PowerStore can now be used with enabled CHAP as a storage back end.

Before this update, the NetApp SolidFire driver would create a duplicate volume when the API response is lost due to a connection error and the driver retries the API request. This occurred when the SolidFire back end successfully received and processed a create volume operation, but failed to deliver the response back to the driver. This update resolves the issue by:

  1. Checking if the volume name already exists in the backend before trying to create it. If a volume is found, an exception is raised and the process is aborted.
  2. Checking for volume creation right after a read timeout is detected, to prevent invalid API calls.
  3. Adding the ´sf_volume_create_timeout´ option to the SolidFire driver, to allow users to set the appropriate timeout value for their environment.
Before this update, execution of the validation package check-latest-packages-version was slow. This update resolves the issue.
This release supports port filtering for the Dell EMC XtremeIO driver for the Block Storage service (cinder).
Before this update, if PowerStore ports were configured for multiple purposes, such as for iSCSI or Replication, the driver reported that it could find no accessible iSCSI targets. This was because the REST filter was wrong. This update fixes the PowerStore iSCSI targets filtering.
Before this update, when iSCSI or FC targets, such as ESXi, were not connected to the RHOSP host, the attach volume operation waited until it timed out. With this update, a new option to support ports filtering has been added to the Dell EMC XtremIO driver for the Block Storage service (cinder).
Before this update, the validation variable in one code path was referenced but never assigned, which resulted in an unhandled exception during validation. This has been fixed.
Before this update, Ansible redirected output to all registered non-stdout callback plug-ins by default, which resulted in VF callbacks processing information from other processes using ansible runtime. This issue has been resolved and the output of other processes is no longer stored in the validations logging directory.
This update fixes an issue that caused Neutron agents, such as Neutron DHCP, to fail when they tried to create resources in OVN because ML2/OVN prevented RPC workers from connecting the OVN southbound database.

Before this update, the Shared File Systems service (manila) dashboard had dynamic form elements whose names could potentially cause the forms to become unresponsive. This meant that the creation of share groups, share networks, and shares within share networks did not work.

With this update, dynamic elements whose names could be problematic are encoded, which means that creation of share groups, share networks, and shares within share networks functions normally.


The Shared File Systems service (manila) uses the CephFS volume client to communicate with Ceph Storage clusters. Previously, the CephFS volume client package would abort while creating or deleting file systems.

The aborted operations caused the manila-share process within the Shared File Systems service to restart, which caused shares that were being provisioned or deleted to be stuck in creating or deleting states, respectively.

With this release, the CephFS volume client package no longer aborts provisioning or deletion requests, and the manila-share process does not restart during these operations.


This update fixes an issue that caused Neutron agents, such as Neutron DHCP, to fail when they tried to create resources in OVN. This was caused by residual data left in the OVN databases when QoS rules were created for floating IPs.

This update eliminates the residual data and fixes the problem.


This update fixes a known issue where the Open Virtual Network (OVN) Metadata service was not available to VM instances bound to an SR-IOV virtual function. The issue did not affect network function but these instances did not receive their SSH keys in the absence of a Metadata service connection.

The metadata service connectivity for SR-IOV ports now functions as expected.


Before this fix, grub2 tooling wrote kernel argument changes to /boot/grub2/grubenv. This file was not available to UEFI boot systems, and caused kernel argument changes not to persist across reboots on UEFI boot nodes.

This fix changes both the /boot/grub2/grubenv file and the /boot/efi/EFI/redhat/grubenv files when you make kernel argument changes. As a result, RHOSP director now applies persistent Kernel argument changes for UEFI boot nodes.

3.1.3. Enhancements

This release of Red Hat OpenStack Platform features the following enhancements:

This enhancement adds support for Entrust nShield HSM deployment in high availability mode with OpenStack Key Manager (barbican).
Images with null bytes take up a lot of space. With this release, you can enable sparse image upload and save space when you upload images. Sparse image upload is supported only with Ceph RBD.
This enhancement adds the type HostDomain. HostDomain is the same as HostAddress with the added support of the underscore character - RFC1033. Systems such as DomainKeys and service records use the underscore. The Compute service can use the HostDomain type to define live_migration_inbound_addr.
Red Hat OpenStack Platform 16.2 includes support for Single Root Input/Output Virtualization (SR-IOV) and Data Plane Development Kit (DPDK) workloads on AMD hosts.
This enhancement improves the efficiency, performance, and execution time of deployment and update tasks for environments with a large number of roles. The logging output of the deployment process has been improved to include task IDs for better tracking of specific task executions, which can occur at different times. The task IDs can now be used to correlate timing and execution when troubleshooting executions.

During stack update the KernelArgs could be modified or appended. A reboot of the affected nodes needs to be performed manually.

For example, if the current deployment has the following configuration, it is possible to change hugepages=64, or add or remove arguments during the stack update:

`KernelArgs: "default_hugepagesz=1GB hugepagesz=1G hugepages=32 intel_iommu=on iommu=pt isolcpus=1-11,13-23"

For example:

KernelArgs: "default_hugepagesz=1GB hugepagesz=1G hugepages=64 intel_iommu=on iommu=pt isolcpus=1-24"
KernelArgs: "isolcpus=1-11,13-23"

Complete removal of KernelArgs during the update is not supported. Also KernelArgs could be newly added as well to an existing overcloud node, however the reboot would be triggered in this case.


With this enhancement, you can improve the performance of live migrations by using the following new parameters:

  • NovaLiveMigrationPermitPostCopy - When enabled, the instance is activated on the destination node before migration is complete, and an upper bound is set on the memory that needs to be transferred, which improves the live migration of larger instances. This parameter is enabled by default.
  • NovaLiveMigrationPermitAutoConverge - When enabled, if an on-going live migration is progressing slowly the instance CPU is throttled until the memory copy process is faster than the instance’s memory writes. This parameter is disabled by default. To enable NovaLiveMigrationPermitAutoConverge, add the following configuration to an environment file:

        NovaLiveMigrationPermitAutoConverge: true
This enhancement improves the performance and application of the check-latest-packages-version validation.
This enhancement adds new validation for tripleo-latest-packages-version. This validation checks if the listed tripleo packages are up to date with repositories.
Before this update, the PluginInstanceFormat parameter for collectd could specify only one of the following values: 'none', 'name', 'uuid', or 'metadata'. After this update, the PluginInstanceFormat parameter for collectd can now specify more than one value, which results in more information being sent in the plugin_instance label of collectd metrics.
With this update, you can use the openstack undercloud backup command with the --db-only option to create a backup of the database that runs on the undercloud node. You can use that backup to recover the state of the database in the event that it becomes corrupted during the upgrade process.

3.1.4. Technology Preview

The items listed in this section are provided as Technology Previews. For further information on the scope of Technology Preview status, and the associated support implications, refer to

In Red Hat OpenStack Platform 16.2, a technology preview is available that supports Precision Time Protocol (PTP) with Timemaster.

The Red Hat OpenStack Platform (RHOSP) director Operator creates a set of custom resource definitions (CRDs) on top of Red Hat OpenShift Container Platform (OCP) to manage resources normally created by the RHOSP undercloud. CRDs are split into two types for hardware provisioning and software configuration. The operator includes CRDs to create and manage overcloud networks, manage IP addresses, create VM sets for RHOSP Controllers, and create bare metal sets for RHOSP Computes.

For Technology Preview, the software configuration is accomplished with an OpenStackClient pod, which uses traditional RHOSP/TripleO interfaces and CLI commands. Work is ongoing to produce a more scalable Heat-to-Ansible playbook deployment workflow within the RHOSP director Operator.

3.1.5. Release Notes

This section outlines important details about the release, including recommended practices and notable changes to Red Hat OpenStack Platform. You must take this information into account to ensure the best possible outcomes for your deployment.


For glance image conversion, the glance-direct method is not enabled by default. To enable this feature, set enabled_import_methods to [glance-direct,web-download] or [glance-direct] in the DEFAULT section of the glance-api.conf file.

The Image Service (glance) must have a staging area when you use the glance-direct import method. Set the node_staging_uri option int he DEFAULT section of the glance-api.conf file to file://<absolute-directory-path>. This path must be on a shared file system that is available to all Image Service API nodes.

The python-networking-fujitsu package is not included with RHOSP 16.2.
In this release, EFI bootloader assets for whole-disk images are preserved during deployment, so the shim bootloader is no longer overwritten. This ensures that Secure Boot is switched on after deployment.

Starting with Red Hat Ceph Storage 4, you can enable encryption for all traffic generated by the Ceph daemons over the network.

The secure mode setting for messenger v2 encrypts the communication between Ceph daemons and Ceph clients, effecting an end-to-end encryption.

A new tripleo-heat-templates parameter can enable the on-wire encryption between daemons and clients. To configure Ceph to enable the on-wire encryption between daemons and clients, add the following lines to the overcloud deployment environment files:

  CephMsgrSecureMode: true
When configuring bandwidth-aware scheduling of SR-IOV workloads use the heat parameter, resource_provider_hypervisors. This parameter defines a paired list <network_device>:<hypervisor>. Ensure that you define each hypervisor using a fully qualified domain name (FQDN).

In previous releases, administrators had to add the ceph plugin to the CollectdExtraPlugins parameter in their custom environment files.

With this release, the ceph plugin loads automatically on Ceph Storage nodes. Therefore, before you upgrade from Red Hat OpenStack Platform 13 to 16.2, you must remove the ceph plugin from the CollectdExtraPlugins parameter in your custom environment files.

3.1.6. Known Issues

These known issues exist in Red Hat OpenStack Platform at this time:


Starting with Red Hat Enterprise Linux (RHEL) version 8.3, support for the Intel Transactional Synchronization Extensions (TSX) feature is disabled by default. Currently, this causes instance live migration to fail when migrating from hosts where the TSX kernel argument is enabled to hosts where the TSX kernel argument is disabled.

This impact applies only to Intel hosts that support the TSX feature. For more information about the CPUs that are affected by this issue, see Affected Configurations.

For more information, review the following Red Hat Knowledgebase solution Guidance on Intel TSX impact on OpenStack guests.


In Red Hat OpenStack Platform (RHOSP) deployments that use the Modular Layer 2 plug-in with the Open vSwitch (ML2/OVS) mechanism driver, there is currently a known issue where the RHOSP Orchestration service (heat) parameter, NeutronL3AgentAvailabilityZone does not set the relevant Neutron L3 agent parameter correctly.

Workaround: use a custom hieradata statement to set this value. In the example that follows, replace [ROLE] with the composable role name that is appropriate for your site.


    neutron::agents::l3::availability_zone: role_availability_zone

For more information, see Puppet: Customizing hieradata for roles in the Advanced Overcloud Customization guide.


Rebooting a node with a virtual function (VF) attached to OVS-DPDK (vfio-pci driver) results in VF uninitialized on that physical function (PF). As a result, virtual machines are unable to use the VFs from that PF. If a second VF is used for another OSP network, it will not function as expected after reboot.

As a workaround, perform the following steps on the compute node before rebooting the node:

  1. Delete the file "/etc/udev/rules.d/70-os-net-config-sriov.rules"
  2. Modify the "Before" criteria of "/etc/systemd/system/sriov_config.service" file to add "" The modified "Before" should look like: openvswitch.service

    The workaround fixes the issue and all the VFs initialize as expected.


RHOSP does not yet support ML2/OVN with VXLAN networks. The migration process includes steps to convert VXLAN networks to Geneve. When the migration target version is RHOSP 16.2.0, a bug prevents the expected VXLAN to Geneve conversion, and the networks remain configured as VXLAN.

This bug affects only migrations to ML2/OVN on RHOSP 16.2. It does not affect migrations to ML2/OVN on RHOSP 16.1.

3.1.7. Deprecated Functionality

The items in this section are either no longer supported, or will no longer be supported in a future release.

For Distributed Compute Node deployments which use storage, dcn-hci.yaml has been renamed to dcn-storage.yaml because DCN sites with storage have the option of not using HCI (Hyper-Converged Infrastructure). dcn-hci.yaml is deprecated but will remain in the environments directory for backwards compatibility. dcn-hci.yaml will be removed in Red Hat OpenStack platform 17. dcn-storage.yaml should be used in place of dcn-hci.yaml.
Block Storage service (cinder) backup with Google Cloud Storage is being deprecated. Support will be removed in the next major release.
In this release, Block Storage service (cinder) backup support for Google Cloud Services (GCS) has been deprecated. Support will be removed in Red Hat OpenStack Platform (RHOSP) 17.0.

In Red Hat Openstack Platform (RHOSP) 16.2, support for the QXL video model is deprecated, due to the removal of support for the Spice graphics software in RHEL 9. This will cause issues for instances that use QXL when migrating from RHEL-8 to RHEL-9. Red Hat recommends using the virtio video model for both UEFI and BIOS instances, instead of qxl. When creating a new instance from an image, set the video model before launching the new instance:

$ openstack image set --property hw_video_model=virtio <image>

To update the video model for existing instances that use the QXL video model:

  1. Stop the instance.
  2. Snapshot the instance.
  3. Update the image metadata on the instance snapshot image to include the property hw_video_model=virtio.
  4. Create a new instance using the instance snapshot.

    For more information on supported video models, see