Chapter 3. RHSA-2015:2666 - OpenShift Enterprise 2.2.8 Security, Bug Fix, and Enhancement Update

Command Line Interface

BZ#1155003

The `rhc server add` command was previously missing the `[--ssl-client-key-file FILE]` option in its summary and `--help` responses. The command also requires all three arguments (`[--ssl-ca-file FILE]`, `[--ssl-client-cert-file FILE]`, and `[--ssl-client-key-file FILE]`) in order to properly allow `rhc server add` to use the certificate information. However, there was no error message to inform the user when only one or two of the options were present. This was causing confusion on the proper way to use SSL certificate information with this command. This bug fix updates the usage statements for the `rhc server add` command to now include `[--ssl-client-key-file FILE]` in their argument list. There is also a check for when users included only one or two files, which will properly inform the user that all one commands are required. This should reduce confusion and provide much better feedback on the usage of the `rhc server add` command.

BZ#1139608

Due to lack of file name checking, the `rhc snapshot` command would previously overwrite the snapshot file of an application without warning the user. If the snapshot file already existed, it was written if `rhc snapshot` was executed on the same application. This bug fix adds proper file name checking to notify users if the same file name already exists. Then, it allows users to input their option to overwrite the file or not.

BZ#1177753

Security policies within an organization may dictate how SSH must be used, requiring a custom SSH executable to be created and used exclusively. This enhancement enables all `rhc` subcommands that must use an SSH implementation to use an SSH executable specified by the user. Otherwise, these subcommands continue as they did previously, usually using Ruby's Net::SSH implementation. The option `--ssh` is now added as a global option to all commands and is also available for use in the rhc `express.conf` configuration file.

BZ#1140766

The usage information for the `-i` flag to the `oo-admin-ctl-district` command did not explain the phrase "node identity". This bug fix updates the usage information to tell the reader that a node's server identity is its host name. In addition, the usage information was changed to mention that a comma-separated list or a regex enclosed in slashes may be specified with the `-i` flag.

Cartridge

BZ#1270660

Previously, the EWS cartridge started an application's gear and marked the application as "up" in HAProxy. If the application took some time to deploy, it caused an outage window as traffic was routed to the application. This was corrected either when HAProxy executed its next health check and disabled the application or when the deployment finished. As a result, it was possible for EWS cartridge deployments to be marked "up" and routable when in fact they were not yet ready to service requests. This bug fix introduces the OPENSHIFT_JBOSSEWS_START_DELAY EWS environment variable to allow application owners to delay the registration of there deployment with haproxy. Set this variable to include a delay (in seconds) which causes deployments to halt after a gear start in a similar manner to the EAP cartridge. The difference between the EWS and EAP cartridges is that Tomcat implements a sleep (or hang) because it does not have a management interface (https://access.redhat.com/solutions/901043) to interact with like EAP to check that deployments have finished. Using the OPENSHIFT_JBOSSEWS_START_DELAY variable can cause application deployments to take longer but it can be used to avoid outages with new deployments.

BZ#1265609

When using the Python cartridge, the pandas package had several dependencies that were not installed successfully using the setup.py method. This bug fix updates the cartridge to use the `pip install` method, which resolves the dependency issue and allows the pandas package to be installed properly. However, to avoid a regression issue, a marker `pip_install` is required to use `pip install`. Otherwise, the standard setup.py installation method is used instead.

BZ#1211526

In some cases, it was possible for a restart of a gear with an HAProxy cartridge to result in more than one HAProxy process running. This resulted in the HAProxy cartridge's process not being killed after a restart, and an HAProxy would be running without a proper pid file. Instead of determining the existence of an HAProxy process from the pid file, this bug fix updates the stop function to now check the process list. As a result, the HAProxy process is now properly killed if it still exists after the stop during a restart.

BZ#1277695

Particular locales trigger different definitions for regular expression character classes [a-z] and [A-Z] in Bash. This caused regex-based host name validation to fail when, for example, the Estonian locale "et_EE" was set in $LC_ALL, because the Estonian letter "u" falls outside of the [a-z] character class, and "rhcloud.com" has a "u" in it. This bug fix updates the `update-cluster` script to enforce use of the POSIX locale just for the locale-sensitive regex, and to restore the user's locale setting immediately afterwards. As a result, host name validation for the `update-cluster` script now works regardless of what locale may be specified by the user.

BZ#1280438

When a gear was almost at its quota limit, a warning message was returned from commands used by haproxy_ctld. This caused the haproxy_ctld log to incorrectly fill up with the quota warning message, which was not intended to be logged by haproxy_ctld. This bug fix filters the warning messages when logging for haproxy_ctld, and as a result quota limit warning messages are no longer logged to the haproxy_ctld log file.

Broker

BZ#1268080

Previously, pending operations failed to run and were added back to the job queue if the parent operation did not exist. When a parent operation was missing, child operations would never complete. This bug fix ensures that if a parent operation does not exist, the discrepancy is logged and the child operation moves on. As a result, if a parent operation is deleted or is otherwise missing, the remaining child operations are still be able to be completed.

BZ#1218872

If the MINIMUM_SSH_KEY_SIZE parameter was not set in the /etc/openshift/broker.conf file, a nil value was passed into a condition statement which failed due to lack of nil checking. This caused the `rhc` command to fail to upload the SSH key while creating a new domain. This bug fix updates the condition statement to handle a nil value and return the correct DEFAULT_MINIMUM_SSH_KEY_SIZE value. As a result, the `rhc` command can now successfully upload the SSH key in this scenario.

BZ#1111598

When an error was discovered from running the `oo-admin-check` command, the error output told the user to run the `oo-admin-repair tool` to fix them. However, a number of errors were possible that could not be resolved with `oo-admin-repair` tool, which caused misdirection on how to correctly address these errors. This bug fix updates individual error messages with relevant solutions or links to a Red Hat solutions page if available. The generic error message now directs the user to the `oo-admin-repair` man page to see if their problem is something that it may be able to resolve.

BZ#1110415

Prior to OpenShift Enterprise 2.1, the command to clear both the broker's and the console's cache was `oo-admin-broker-cache --clear --console`. As of OSE 2.1, `oo-admin-broker-cache` only clears the broker's cache, and it is necessary to use `oo-admin-console-cache` to clear the console's cache. However, running `oo-admin-broker-cache --clear --console` did not warn that the `--console` flag did nothing; instead, `oo-admin-broker-cache` would silently ignore the `--console` flag. This bug fix updates the `oo-admin-broker-cache` command to no longer ignore the `--console` flag; instead, specifying the flag now causes the command to print an error with usage information.

BZ#1101973

The `test_for_nonrpm_rubygems` test in the `oo-diagnostics` tool previously used shell globbing to search for gemspec files in several paths. However, shell globbing returns back the glob pattern (rather than an empty list) if the pattern does not match anything. The test failed to take this shell feature into account, instead interpreting a returned glob pattern as a file name. If one of the directories that the test searched was empty, the test would output spurious error messages such as "ls: cannot access /opt/rh/ruby193/root/usr/local/share/gems/specifications/*.gemspec: No such file or directory". This bug fix updates the `oo-diagnostics` tool to handle empty gemdirs. Specifically, the glob search in the `test_for_nonrpm_rubygems` was rewritten to perform the search using Ruby code rather than shell. Globbing in Ruby returns an empty list if the glob pattern does not match anything. As a result, the `oo-diagnostics` tool no longer prints spurious error messages for empty directories.

BZ#1264722

Previously, there was a conflict in the `oo-register-dns` command between the two `-h` options which were associated with both the `--help` and `--with-node-hostname` options. This caused the command to fail with getoptlong object errors. This bug fix updates the command, and the `--help` and `-?` options now show usage information while the `-h` options is only associated with the `--with- node-hostname` option. As a result, these errors no longer occur.

BZ#1272195

Previously, the `oo-admin-ctl-app` command did not check the min_limit on the group_overrides while it was removing a gear from an application. This made it possible for a gear to be removed from a scalable application beyond the min_limit. This bug fix updates the `oo-admin-ctl-app` command to check the min_limit before removing a gear from a scalable application. As a result, the `remove-gear` command now fails if the number of gears have already reached the min_limit.

BZ#1064039

The `oo-diagnostics` tool did not warn users if there were "Broker key authentication failed" errors in log files, which indicated potential key/salt inconsistencies. This is problematic because if gears are created with a mismatched key/salt, future gears become inaccessible due to "401 Unauthorized" errors. This bug fix adds a proper check to the `oo-diagnostics` tool to issue warnings if these errors are listed in log files. Also, a suggested fix is included in the warning message to help users rectify the issue.

Node

BZ#1271338

When restoring SELinux labels, the action was performed on a directory, which does not include hidden files. This caused SELinux labels to not be properly restored on hidden files within a gear directory if they were incorrect. This bug fix ensures that the SELinux label change is performed on all files within a directory, rather than the directory. As a result, hidden files in a gear now have the proper SELinux labels set when they are incorrect.

BZ#1045226

The `oo-auto-idler` man page incorrectly stated that the command replaced the `oo-last-access` command instead of the `oo-list-stale` command. This bug fix updates the man page to correctly state that the `oo-auto-idler` command replaces the `oo-list-stale` command.

BZ#1111501

By default, the REPORT_BUILD_ANALYTICS parameter in the /etc/openshift/node.conf file was set to `true`. With this default configuration, the OpenShift node attempted to notify the broker about application builds via a special REST API on the broker. However, the broker did not have this REST API in its default configuration which caused the node's attempt to result in the following spurious log message and a backtrace in the /var/log/openshift/broker/httpd/error_log file every time a user performed a `git push` to an application: 'Exception ActionController::RoutingError in Rack application object (No route matches [POST] "/broker/analytics")'. This bug fix changes the default value of the REPORT_BUILD_ANALYTICS parameter to `false.` As a result, the spurious log messages should no longer appear by default.

BZ#1054441

When the BROKER_HOST parameter in the /etc/openshift/node.conf file and OPENSHIFT_BROKER_HOST environment variables are inconsistent, some cartridges, such as Jenkins, fail to work as they use those variables. The same is true for CLOUD_DOMAIN variables. This bug fix adds a warning to the `oo-accept-node` command to notify administrators to fix such inconsistencies. As a result, a warning is now issued if these variables do not match.

REST API

BZ#1255426

Previously, setting a minimum scale of at least 2 on an application and then making the application highly-available (HA) caused configuration issues for the application. This caused the application to be unable to unset HA and set the minimum scale back down to 1. With this bug fix, when an application has HA disabled, the configuration now allows the application to be scaled back down to a single gear.

RFE

BZ#1238305

The author of a gear-placement plug-in may want to use the domain_id to determine where gears are placed. For example, applications in a particular domain may need to be isolated on a separate set of nodes in order to comply with requirements specified in external security policies. This enhancement expands the input parameters to gear-placement plug-ins to include the domain ID of the application, and as a result gear-placement plug-ins can now make placement decisions using applications' respective domain IDs.

BZ#1241675

Normally, when the routing daemon is configured with F5 BIG-IP®, the routing daemon checks for the `openshift_application_aliases` local traffic policy on startup and creates the policy if it does not exist. However, the routing daemon previously did not validate that any existing policy was configured properly. As a result, if the F5 administrator set up the policy manually and configured it incorrectly, or changed the policy's configuration later, the policy may no longer function properly. This enhancement changes the routing daemon's startup check to include validation of the policy. In particular, the routing daemon now checks that the policy is configured with the `forwarding` control and the `http` aspect, and it updates the policy's configuration if required.

Routing

BZ#1248439

The Nginx load balancer previously passed on the host name configured when balancing requests. This configured host name is the name of the pool in the configuration, which does not resolve to anything in DNS. Anything that uses the host header on the request may fail to resolve the proper host. This bug fix ensures that Nginx passes the gear's correct host name in the host header, and as a result any application that utilizes the value of the host header in a request can now resolve the application host name properly.

BZ#1282520

When the routing daemon is configured with F5 BIG-IP®, the routing daemon is supposed to check for the `openshift_application_aliases` local traffic policy on startup and create the policy if it does not exist. However, a logic error introduced with the release of routing-daemon-0.23.2.0-1 in OpenShift Enterprise 2.2.5 prevented this check from functioning properly. If the `openshift_application_aliases` policy did not already exist, this caused the routing daemon to fail to configure policy rules. This bug fix addresses the logic error in the routing daemon's initialization, and as a result the routing daemon now creates the `openshift_application_aliases` local traffic policy on startup if required.

Vulnerability

BZ#1239072

An implementation error related to the memory management of request and responses was found within HAProxy's buffer_slow_realign() function. An unauthenticated remote attacker could possibly use this flaw to leak certain memory buffer contents from a past request or session.