8.6. Using an External Routing Layer for High-Availability Applications

Using an external routing layer, you can enable scalable applications to become highly available. OpenShift Enterprise exposes the following gear events from scalable applications, which are significant because they expose and conceal ports on gears where web traffic is received:
  • Adding and deleting applications
  • Scaling applications up or down
  • Adding or removing aliases and custom certificates
You can implement a routing plug-in in OpenShift Enterprise that allows an external routing solution in your existing infrastructure, such as nginx, Nginx Plus®, or F5 BIG-IP LTM® (Local Traffic Manager™), to balance web traffic loads using these exposed ports. Introducing a router that dynamically controls this traffic to gears allows an OpenShift Enterprise deployment to achieve high availability. Without this feature, the DNS-based routing has a single point of failure on the scalable application's HAProxy gear. See the OpenShift Enterprise User Guide [5] for more information on basic routing for scalable applications when a routing layer is not being used.
You can either develop your own routing plug-in to handle these events or use the included sample routing plug-in. The sample plug-in takes the gear events and publishes corresponding messages on ActiveMQ as topic notifications. You must then configure a routing listener, which can be a script or daemon, that listens for these notifications and programs your chosen routing solution dynamically to route the traffic to the gears.
If you are using the sample routing plug-in, starting in OpenShift Enterprise 2.2 you can configure a sample routing daemon to be the listener, which currently is supported for use with an nginx or Nginx Plus® routing back end. Starting in OpenShift Enterprise 2.2.4, the routing daemon also supports an external LTM® version 11.6.0 routing back end. Otherwise, you must develop your own routing listener based on your chosen routing solution.

8.6.1. Selecting an External Routing Solution

If you plan to configure a routing plug-in and listener to enable scalable applications to become highly available, you must first have an external routing solution, such as nginx, Nginx Plus®, or F5 BIG-IP LTM®, installed and configured in your environment to serve as the routing back end.
The following sections highlight and provide basic installation information for a few of these routing solutions. Note that currently only nginx, Nginx Plus®, and LTM® are supported for integration with the sample routing daemon described later in Section 8.6.3, “Configuring a Routing Daemon or Listener”, but you can develop a custom routing listener based on your chosen routing solution.
Using nginx or Nginx Plus®

nginx is a web and proxy server with a focus on high concurrency, performance, and low memory usage. It can be installed on a Red Hat Enterprise Linux 6 host and is currently included in Red Hat Software Collections 1.2. The Red Hat Software Collections version does not include the Nginx Plus® commercial features. If you want to use the Nginx Plus® commercial features, install Nginx Plus® using the subscription model offered directly from http://nginx.com.

The sample routing daemon, available starting in OpenShift Enterprise 2.2, supports integration with nginx or Nginx Plus® and automatically creates and manage server.conf and pool_*.conf files under the configured directory. After each update, the routing daemon reloads the configured nginx or Nginx Plus® service.
The routing daemon also includes configuration options that enable the use of an Nginx Plus® feature that enabled proactive health checks for upstream group members. It is not necessary to use this feature unless you want to ensure a seamless transition if an upstream group member is not responding successfully.

Important

Normally HTTPS can be used to resolve application requests. However, to do so when the routing daemon is configured to use nginx or Nginx Plus® external routing, users must first add a custom domain alias and SSL certificate to their application. For example, a user can do so by running the following:
# rhc alias add App_Name Custom_Domain_Alias
# rhc alias update-cert App_Name Custom_Domain_Alias --certificate Cert_File --private-key Key_File
If you configure the current version of the routing daemon to use nginx or Nginx Plus®, you must inform users of your deployment about the above requirement. See the OpenShift Enterprise User Guide [6] for more information on user management of custom domains and SSL certificates.
Detailed configuration instructions for the routing daemon itself are provided later in Section 8.6.3, “Configuring a Routing Daemon or Listener”.

Procedure 8.16. To Install nginx from Red Hat Software Collections:

  1. Register a Red Hat Enterprise Linux 6 host to Red Hat Network and ensure the Red Hat Enterprise Linux 6 Server and Red Hat Software Collections 1 channels are enabled. For example, after registering the host with Red Hat Subscription Management (RHSM), enable the channels with the following command:
    # subscription-manager repos --enable=rhel-6-server-rpms --enable=rhel-server-rhscl-6-rpms
  2. Install nginx:
    # yum install nginx16
  3. Enable the following SELinux Boolean:
    # setsebool -P httpd_can_network_connect=true
  4. Start the nginx service:
    # chkconfig nginx16-nginx on
    # service nginx16-nginx start
See http://nginx.org/en/docs for the official nginx documentation.
Using F5 BIG-IP LTM®

Starting in OpenShift Enterprise 2.2.4, the sample routing daemon supports integration with F5 BIG-IP LTM® (Local Traffic Manager™) version 11.6.0. See the official LTM® documentation for installation instructions.

Important

Custom alias HTTPS termination at LTM® requires SNI configuration in LTM®. See the F5® Knowledge Base for configuration details.
To integrate with OpenShift Enterprise, you must configure your LTM® instance with two virtual servers: one for HTTP traffic and one for HTTPS traffic. Each virtual server must be assigned at least one VIP. A default client-ssl profile must also be configured as the default SNI client-ssl profile. Although the naming of the default client-ssl profile is unimportant, it must be added to the HTTPS virtual server.
OpenShift Enterprise integration requires an LTM® user with the Administrator role, for example, the default admin account. Without this role, the user that the routing daemon authenticates will not have the correct privileges or configuration to use the advanced shell. Also, the LTM® admin user's Terminal Access must be set to Advanced Shell so that remote bash commands can be executed.

Procedure 8.17. To Grant a User Advanced Shell Execution:

  1. On the F5® console, navigate to System->Users->User List->Username.
  2. In the dropdown box labeled Terminal Access, choose the Advanced Shell option.
  3. Click on the Update button.

Note

See the LTM® documentation for more information on assigning a user the Administrator role, and the different options for the Terminal Access dropdown box.
Additionally, for the remote key management commands to execute, the BIGIP_SSHKEY public key must be added to the LTM® admin user's .ssh/authorized_keys file.
When configured for LTM®, the routing daemon automatically:
  • Creates pools and associated local-traffic policy rules.
  • Adds profiles to the virtual servers.
  • Adds members to the pools.
  • Deletes members from the pools.
  • Deletes empty pools and unused policy rules when appropriate.
The routing daemon names the pools after applications following the template /Common/ose-#{app_name}-#{namespace} and creates policy rules to forward requests to pools comprising the gears of the named application. Detailed configuration instructions for the routing daemon itself are provided later in Section 8.6.3, “Configuring a Routing Daemon or Listener”.