6.7. RHBA-2016:0264 — Red Hat Enterprise Linux OSP 7 director Bug Fix Advisory

This firewall configuration for the Undercloud lacked certain ports, which resulted in dropped packets for Internal API messages. This fix adds the missing ports (13000, 13774, 13696, 13385, 13292, 13696, 13004, 13080, 13385) to the Undercloud's firewall rules. The Internal API now accepts messages on these ports.

The Undercloud's firewall lacked a port for Ceilometer's Public API over SSL. This fix adds the port (13777) to the Undercloud's installation script. Now Ceilometer accepts Public API requests over SSL.

The Undercloud installation script recreated users on subsequent runs. This causes the service user IDs to change, which causes trust issues for running certain services. This fix stops the installation script from recreating users. Now service user IDs remain consistent with their respective services.

This enhancement provides new iPXE images for the Red Hat OpenStack Platform director. The new iPXE images add extra network boot support for certain NICs

The 'openstack overcloud update' command sought a list of events for each resource. When listing events for a resource, the Heat API returned a HTTP 404 error (Not Found) for resources with no events. The resource was considered as non-existent (due to the 404 error) and the client would fail. This occurred in situations where a previous update would end after adding a resource to the stack but before any events occur, such as a resource waiting at a breakpoint at the time the update ended. This fix adds error handling to the client, which resolves issues with the 404 error.

Previous Overcloud images used EDT for the timezone, which caused problems for users outside of the EDT timezone. This fix adds a customizable 'TimeZone' parameter to the Heat template collection. Users can set the 'TimeZone' parameter to their own timezone. If blank, it defaults to UTC.

The iSCSI initiator name was the same for all Compute nodes in an Overcloud, which causes live migration of instances to fail. This fix modifies the iSCSI initiator name during Overcloud deployment. Now live migration succeeds over iSCSI.

This enhancement adds support for the Nuage on highly available Overcloud environments. This includes Nuage-specific parameters in the director's Heat template collection, and environment files to enable the Nuage backend on Controller and Compute nodes.

This enhancement adds support for the Nuage metadata agent on the Overcloud. This includes parameters in the director's Heat template collection for the Nuage metadata agent.

The *ExtraConfig hiera data parameters did not work for non-Controller nodes. This is due to missing parameter definitions for non-Controller node types. This fix implements these parameters into the director's Heat template collection. Now the director writes the *ExtraConfig hiera data the the appropriate node types.

In Red Hat OpenStack Platform 7.0, Overclouds using a flat network had an additional Public IP created. This is no longer required but needs preservation for backwards compatibility. If upgrading from 7.0 and using a flat network topology, include the following environment file:

/usr/share/openstack-tripleo-heat-templates/environments/updates/update-from-publicvip-on-ctlplane.yaml

This environment file preserves the additional IP address from 7.0 and behaving as 'public_virtual_ip'.

High traffic Red Hat OpenStack Platform networks caused timeouts (specifically DNS timeouts) due to low maximum for netfilter connections tracking. This update increases the 'nf_conntrack_max' kernel parameter to 500000. This resolves the timeout issues.

This enhancement adds support to register Overcloud nodes to a Red Hat Satellite 5 server. Previous versions allowed registration only to a Red Hat Satellite 6 server. Now the director determines whether to register to a Red Hat Satellite 5 or Red Hat Satellite 6 server when using the '--reg-method satellite' option during Overcloud creation.

Pacemaker used a 100s timeout for service resources. However, a systemd timeout requires an additional timeout period after the initial timeout to accommodate for a SIGTERM and then a SIGKILL. This fix increases the Pacemaker timeout to 200s to accommodate two full systemd timeout periods. Now the timeout period is enough for systemd to perform a SIGTERM and then a SIGKILL.

Swift caused deployment errors for an IPv6-based Overcloud due to problems with processing Swift's IPv6 addresses. This fix corrects how the IPv6 addresses are processed. Swift now deploys successfully.

Corosync failed to start in an IPv6-based Overcloud. This is due to a missing '--ipv6' option when the director tries to start Corosync. This fix adds this option to the Controller's Puppet manifest and also adds related parameters to the Heat template collection. Corosync now starts successfully in IPv6-based Overclouds.

This enhancement adds SSL support to the Overcloud's Public API. Users can now configure SSL on the Overcloud using the 'environments/enable-tls.yaml' from the director's Heat template collection. Copy and modify this environment file to suit your SSL requirements. For more information, see " ⁠6.2.7. Enabling SSL/TLS on the Overcloud" in the Director Installation and Usage guide for Red Hat OpenStack Platform 7.3.

The validation script for deployment testing node availability only supported IPv4. This caused connectivity checks for an IPv6-based Overcloud to fail. This mix modifies the validation script to detect whether the IP address is v4 or v6 and run the respective connectivity check commands. Now connectivity checks succeed for IPv6-based Overclouds.

The IPv6 network interface templates contained an error that set 'ExternalInterfaceDefaultRoute' to an IPv4 value. This fix corrects the error and sets the default route to an IPv6 value. The 'ExternalInterfaceDefaultRoute' now configures correctly.

Pacemaker failed to start in an IPv6-based Overcloud deployment due to using IPv4-based settings (/32) for the VIP netmask. This fix determines if the Overcloud uses IPv6 and sets the VIP netmasks to the appropriate values (/64 in most cases). Pacemaker now starts successfully in the Overcloud.

In an IPv6-based Overcloud, Galera failed to start due to issues with using an IPv6 address in configuration. This fix copnfigures the 'bind-address' parameter to use the hostname, which all nodes should have in their ''/etc/hosts' file. Galera now starts successfully in IPv6 Overclouds.

Non-Controller nodes reported package dependency issues due to delegating Puppet as the mechanism to update certain packages and excluding them from YUM updates. This fix sets all Non-Controller nodes to use Puppet as the update mechanism. Now packages on non-Controller nodes update without package dependency issues.

In an IPv6-based Overcloud, the director incorrectly parsed MariaDB DSN strings containing IPv6 addresses. This caused Puppet to report duplicate 'Mysql_database' resources due to all databases using the first bit grouping of the IPv6 address as the database name (e.g. 'fd00'). This fix adds logic to check if the string uses an IPv4 or IPv6 address and parse the string accordingly. Puppet no longer reports duplicate 'Mysql_database' resources.

'nova-consoleauth' failed to start due to how it parsed IPv6 addresses for 'memcached_servers' in 'nova.conf'. This fix corrects for the director's Heat template collection parses 'memcached_servers'. The 'nova-consoleauth' service now starts successfully.

In an IPv6-based Overcloud, RabbitMQ lacked some IPv6-specific options when starting. This caused RabbitMQ to fail on some nodes, which cause other services to fail due to Pacemaker contraints. This fix adds the IPv6-specific options. Now RabbitMQ and other services start successfully.

Overcloud deployments with IPv6 endpoints caused Glance to report a HTTP 500 error. This is due to how the director parsed IPv6 addresses. This fix corrects how the director parses the IPv6 address for Glance. Glance now works in IPv6-based Overclouds.

The VNC servers on Compute nodes would bind to IPv4 addresses only. Users could not access VNC consoles in an IPv6 environment since the Internal API network used IPv6 endpoints. The fix allows the VNC servers to bind on IPv6 addresses when deploying an IPv6 overcloud. Now users can access instances through VNC consoles in an IPv6 deployment.

Setting a fixed IPv6 address for Overcloud networks failed due to Neutron not allowing fixed IP addresses in SLAAC mode. This fix changes the default IPv6 address mechanism to 'dhcpv6-stateful'. Now the director can configure the Overcloud using fixed IPv6 addresses.

The Puppet manifest in the Heat template collection started the HTTP service through systemd. This caused Pacemaker to fail starting the service. This fix modifies the manifest to only configure but not start the HTTP service so that Pacemaker can assume control over HTTP. Pacemaker now starts the service successfully.

In mixed environments where some networks use IPv4 addressing and others IPv6 addressing, the Overcloud used IPv6 CIDR for IPv4 VIPs too. The Overcloud deployment failed because Pacemaker refused to start the IPv4 VIPs. This fix adds functionality to identify the VIP type (IPv4 or IPv6) during deployment and adapt the appropriate CIDR. Each IPv4 and IPv6 VIP now uses the appropriate CIDR.

The Ceilometer Compute Agent in IPv6-based Overclouds could not reach the public endpoint and reported errors such as:

ConnectionError: ('Connection aborted.', gaierror(-2, 'Name or service not known'))

This fix switches the endpoint to 'internalURL' instead of 'publicURL'.

The 'router_delete_namespaces' (L3 agent) and 'dhcp_delete_namespaces' (DHCP agent) configuration settings defaulted to 'false' in the Red Hat OpenStack Platform 7 Puppet modules. This disabled cleanup for unused network namespaces, which was only required for older versions of Linux. This fix sets the defaults for these parameters to 'true' in the Puppet modules so Red Hat OpenStack Platform takes advantage of network namespace cleanup.

The director inappropriately configured Ceph Storage IPv6, which caused deployment timeouts and Overcloud deployment failure. This fix adds an input parameter (CephIPv6) in Heat templates, which sets the relevant Ceph config option (ms_bind_ipv6). Ceph Storage is now functional and the Overcloud deployment completes when using IPv6 for the Storage network.

Compute nodes detected IPv6 router announcements (RA) on the Overcloud's IPv6-based Tenant network. This caused the Compute nodes to receive IPv6 addresses and default routes on the 'qbr' interface. This fix sets the 'net.ipv6.conf.default.disable_ipv6' kernel parameter to 1 on all nodes. This disables automatic configuration from RAs and allows the director to define the IPv6 addresses and default route.

In an Overcloud with HA Controller nodes, the 'cinder-volume' service might move to a new node. This causes problems modifying and deleting volumes due to a different hostname for the volume service. This fix sets a consistent hostname for the 'cinder-volume' service on all Controller nodes. Users can now modify and delete volumes on a HA Overcloud without issue.

Heat used a 1MB payload size for returned output. YUM's output exceeded this limit if updating a high number of packages on the Overcloud, which caused 'openstack overcloud update' to fail. This fix adds the '-q' option to YUM during an Overcloud update. This option sets the output to quiet mode, which reduced the output. YUM's output no longer exceeds Heat's limit and 'openstack overcloud update' succeeds.

Compute nodes detected IPv6 router announcements (RA) on the Overcloud's IPv6-based Tenant network. This caused problems with the IPv6 routing table, such as setting the default route to the Neutron router. This fix sets the 'net.ipv6.conf.default.accept_ra' kernel parameter to 0 on all nodes. Now the Compute node no longer accepts router announcements from the Tenant networks.

A bug in Heat caused validation of old parameters against the new template. Upgrades of Overclouds from Red Hat OpenStack Platform 7.2 to Red Hat OpenStack Platform 7.3 failed with the error: "resources.SwiftDevicesAndProxyConfig: Property controller_swift_proxy_memcaches_v6 not assigned". This fix add defaults to Swift parameters in the director's Heat template collection, which resolves the error.

Ceilometer services failed to start in the Overcloud due to a incorrectly parsed IPv6 address from the director's Heat template collection. This fix correctly parses the IPv6 address. Ceilometer now starts correctly in an IPv6-based Overcloud.

When deploying an Overcloud with an external load balancer, the 'RedisVipPort' parameter resolved to the 'from_service.yaml' template in the director's Heat template collection. However, an issue with the 'ip_address_uri' output parameter in 'from_service.yaml' template provided the wrong value. This fix corrects the 'ip_address_uri' output parameter in 'from_service.yaml'. Now the 'from_service.yaml' template returns the correct value to 'RedisVipPort'.

Keystone endpoint creation failed due to incorrectly parsed IPv6 addresses. This fix modifies the Keystone client creation mechanism to correctly parse IPv6 addresses. 'os-cloud-config' now creates Keystone endpoints successfully.

Keystone client creation failed due to incorrectly parsed IPv6 addresses. This fix modifies the Keystone client creation mechanism to correctly parse IPv6 addresses. 'os-cloud-config' now creates Keystone clients successfully.

'os-net-config' only wrote IPv4 routes to /etc/sysconfig/network-scripts/route-*. However, IPv6 routes used /etc/sysconfig/network-scripts/route6-* files. This fix modifies 'os-net-config' to detect whether a route is IPv4 or IPv6 and write to the appropriate file. This allows 'os-net-config' to define IPv6 in the Linux configuration.

Multiple services attempted NTP configuration on the Overcloud and the last service configured it incorrectly. This caused time synchronization issues across all Overcloud nodes. As a workaround, delete /usr/libexec/os-apply-config/templates/etc/ntp.conf from all Overcloud nodes and re-run the deployment command to re-apply the puppet configuration. This is required for users updating from an older version of Red Hat OpenStack Platform to 7.3. This fix is not necessary on new 7.3 deployments. NTP now configures correctly.

An issue with the OpenStack Platform director 7.2 ramdisk and kernel image caused provisioning failure with the following error:

mount: you must specify the filesystem type
Failed to mount root partition /dev/sda on /mnt/rootfs

This update reverts the ramdisk and kernel image to the OpenStack Platform director 7.1 images. Using these images, the director now provisions Overcloud nodes without failure.

NOTE: An alternative workaround is to disable localboot option for the different node types. For example, to disable localboot with Controller nodes, run:

$ nova flavor-key control unset capabilities:boot_option

'ceilometer-dbsync' fails in a highly available IPv6 Overcloud. This is due to how the director parsed IPv6 addresses for MongoDB. This fix corrects how the director parses IPv6 addresses. Now 'ceilometer-dbsync'  runs successfully.

Horizon failed to load in IPv6 Overclouds due to issues with how the director detected and parsed IPv6 addresses for Memcached. This fix changes how the director's Heat template collection enables IPv6 addresses for Memcached. This includes a new parameter 'MemcachedIPv6' that defines if Memcached uses IPv4 or IPv6 addresses.

An SELinux issue stopped RabbitMQ from starting on IPv6-based Overclouds. This fix corrects the SELinux issue and RabbitMQ now starts successfully.