-
Language:
English
-
Language:
English
8.6.3. Configuring a Routing Daemon or Listener
Prerequisites:
After configuring a routing plug-in on the broker host, you must configure a routing listener, which can be a script or daemon, that listens for notifications of application life-cycle events. The listener must program your chosen external routing solution using the notifications to dynamically route traffic to the gears.
If you are using the sample routing plug-in described in Section 8.6.2, “Configuring the Sample Routing Plug-In”, starting in OpenShift Enterprise 2.2 you can install and configure a sample routing daemon to listen to the event notifications published on ActiveMQ by the sample routing plug-in. The routing daemon supports integration with an external nginx or Nginx Plus® routing back end. Starting in OpenShift Enterprise 2.2.4, the routing daemon also supports integration with an external F5 BIG-IP LTM® (Local Traffic Manager™) version 11.6.0 routing back end. Otherwise, you must develop your own routing daemon or listener for your chosen routing solution.
Both options, configuring the sample daemon or developing your own listener, are detailed below.
Configuring the Sample Routing Daemon
The following procedure assumes that you have already set up nginx, Nginx Plus®, or LTM® as a routing back end as described in Section 8.6.1, “Selecting an External Routing Solution”.
Procedure 8.19. To Install and Configure the Sample Routing Daemon:
- The sample routing daemon is provided by the rubygem-openshift-origin-routing-daemon package. The host you are installing the routing daemon on must have the
Red Hat OpenShift Enterprise 2.2 Infrastructure
channel enabled to access the package. See Section 7.1, “Configuring Broker Host Entitlements” for more information.For nginx or Nginx Plus® usage, because the routing daemon directly manages the nginx configuration files, you must install the package on the same host where nginx or Nginx Plus® is running. Nginx Plus® offers features such as a REST API and clustering, but the current version of the routing daemon must still be run on the same host.For LTM® usage, you must install the package on a Red Hat Enterprise Linux 6 host that is separate from the host where LTM® is running. This is because the daemon manages LTM® using a SOAP or REST interface.Install the rubygem-openshift-origin-routing-daemon package on the appropriate host:#
yum install rubygem-openshift-origin-routing-daemon
- Edit the
/etc/openshift/routing-daemon.conf
file and set theACTIVEMQ_*
parameters to the appropriate host address, credentials, and ActiveMQ topic or queue destination:ACTIVEMQ_HOST=broker.example.com ACTIVEMQ_USER=routinginfo ACTIVEMQ_PASSWORD=routinginfopasswd ACTIVEMQ_PORT=61613 ACTIVEMQ_DESTINATION=/topic/routinginfo
In OpenShift Enterprise 2.1.2 and later, you can set theACTIVEMQ_HOST
parameter as a comma-separated list of host:port pairs if you are using multiple ActiveMQ brokers:ACTIVEMQ_HOST='192.168.59.163:61613,192.168.59.147:61613'
- If you optionally enabled SSL connections per ActiveMQ host in the routing plug-in, set the
plugin.activemq*
parameters in this file to the same values used in the/opt/rh/ruby193/root/etc/mcollective/client.cfg
file on the broker:plugin.activemq.pool.size = 1 plugin.activemq.pool.1.host = activemq.example.com plugin.activemq.pool.1.port = 61614 plugin.activemq.pool.1.ssl = true plugin.activemq.pool.1.ssl.ca = /etc/keys/activemq.example.com.crt plugin.activemq.pool.1.ssl.key = /etc/keys/activemq.example.com.key plugin.activemq.pool.1.ssl.cert = /etc/keys/activemq.example.com.crt
If you have multiple pools, ensure thatplugin.activemq.pool.size
is set appropriately and create unique blocks for each pool:plugin.activemq.pool.size = 2 plugin.activemq.pool.1.host = activemq1.example.com plugin.activemq.pool.1.port = 61614 plugin.activemq.pool.1.ssl = true plugin.activemq.pool.1.ssl.ca = /etc/keys/activemq1.example.com.crt plugin.activemq.pool.1.ssl.key = /etc/keys/activemq1.example.com.key plugin.activemq.pool.1.ssl.cert = /etc/keys/activemq1.example.com.crt plugin.activemq.pool.2.host = activemq2.example.com plugin.activemq.pool.2.port = 61614 plugin.activemq.pool.2.ssl = true plugin.activemq.pool.2.ssl.ca = /etc/keys/activemq2.example.com.crt plugin.activemq.pool.2.ssl.key = /etc/keys/activemq2.example.com.key plugin.activemq.pool.2.ssl.cert = /etc/keys/activemq2.example.com.crt
The files set in the*ssl.ca
,*ssl.key
, and*ssl.cert
parameters must be copied from the ActiveMQ broker or brokers and placed locally for the routing daemon to use.Note that while setting theplugin.activemq*
parameters overrides theACTIVEMQ_HOST
andACTIVEMQ_PORT
parameters in this file, theACTIVEMQ_USERNAME
andACTIVEMQ_PASSWORD
parameters in this file are still used by the routing daemon and must be set. - Set the
CLOUD_DOMAIN
parameter to the domain you are using:CLOUD_DOMAIN=example.com
- To use a different prefix in URLs for high-availability applications, you can modify the
HA_DNS_PREFIX
parameter:HA_DNS_PREFIX="ha-"
This parameter and theHA_DNS_PREFIX
parameter in the/etc/openshift/broker.conf
file, covered in Section 8.6.4, “Enabling Support for High-Availability Applications” , must be set to the same value. - If you are using nginx or Nginx Plus®, set the
LOAD_BALANCER
parameter to thenginx
module:LOAD_BALANCER=nginx
If you are using LTM®, set theLOAD_BALANCER
parameter to thef5
module:LOAD_BALANCER=f5
Ensure that only oneLOAD_BALANCER
line is uncommented and enabled in the file. - If you are using nginx or Nginx Plus®, set the appropriate values for the following
nginx
module parameters if they differ from the defaults:NGINX_CONFDIR=/opt/rh/nginx16/root/etc/nginx/conf.d NGINX_SERVICE=nginx16-nginx
If you are using Nginx Plus®, you can uncomment and set the following parameters to enable health checking. This enables active health checking and takes servers out of the upstream pool without having a client request initiate the check.NGINX_PLUS=true NGINX_PLUS_HEALTH_CHECK_INTERVAL=2s NGINX_PLUS_HEALTH_CHECK_FAILS=1 NGINX_PLUS_HEALTH_CHECK_PASSES=5 NGINX_PLUS_HEALTH_CHECK_URI=/ NGINX_PLUS_HEALTH_CHECK_MATCH_STATUS=200 NGINX_PLUS_HEALTH_CHECK_SHARED_MEMORY=64k
- If you are using LTM®, set the appropriate values for the following parameters to match your LTM® configuration:
BIGIP_HOST=127.0.0.1 BIGIP_USERNAME=admin BIGIP_PASSWORD=passwd BIGIP_SSHKEY=/etc/openshift/bigip.key
Set the following parameters to match the LTM® virtual server names you created:VIRTUAL_SERVER=ose-vserver VIRTUAL_HTTPS_SERVER=https-ose-vserver
Also set theMONITOR_NAME
parameter to match your LTM® configuration:MONITOR_NAME=monitor_name
For thelbaas
module, set the appropriate values for the following parameters to match your LBaaS configuration:LBAAS_HOST=127.0.0.1 LBAAS_TENANT=openshift LBAAS_TIMEOUT=300 LBAAS_OPEN_TIMEOUT=300 LBAAS_KEYSTONE_HOST=10.0.0.1 LBAAS_KEYSTONE_USERNAME=user LBAAS_KEYSTONE_PASSWORD=passwd LBAAS_KEYSTONE_TENANT=lbms
- By default, new pools are created and named with the form
pool_ose_{appname}_{namespace}_80
. You can optionally override this defaults by setting appropriate value for thePOOL_NAME
parameter:POOL_NAME=pool_ose_%a_%n_80
If you change this value, set it to contain the following format so each application gets its own uniquely named pool:%a
is expanded to the name of the application.%n
is expanded to the application's namespace (domain).
- The BIG-IP LTM back end can add an existing monitor to newly created pools. The following settings control how these monitors are created:
#MONITOR_NAME=monitor_ose_%a_%n #MONITOR_PATH=/health_check.php #MONITOR_UP_CODE=1 MONITOR_TYPE=http-ecv #MONITOR_TYPE=https-ecv #MONITOR_INTERVAL=10 #MONITOR_TIMEOUT=5
Set theMONITOR_NAME
parameter to the name of the monitor to use, and set theMONITOR_PATH
parameter to the path name to use for the monitor. Alternatively, leave either parameter unspecified to disable the monitor functionality.As with thePOOL_NAME
andROUTE_NAME
parameters, theMONITOR_NAME
andMONITOR_PATH
parameters both can contain%a
and%n
formats, which are expanded the same way. Unlike thePOOL_NAME
andROUTE_NAME
parameters, however, you may or may not want to reuse the same monitor for different applications. The routing daemon automatically creates a new monitor when the format used from theMONITOR_NAME
parameter expands a string that does not match the name of any existing monitor.Set theMONITOR_UP_CODE
parameter to the code that indicates that a pool member is up, or leave it unspecified to use the default value of1
.MONITOR_TYPE
specifies the type of probe that the external load-balancer should use to check the health status of applications. The only other recognized value forMONITOR_TYPE
ishttps-ecv
, which defines the protocol to be HTTPS. All other values forMONITOR_TYPE
translate to HTTP.Note that ECV stands for “extended content verification", referring to the fact that the monitor makes an HTTP request and looks at the reply to verify that it is the expected response (meaning the application server is responding), as opposed to merely pinging the server to ensure it is returning an ICMP ping reply (meaning the operating system is responding).Set theMONITOR_INTERVAL
parameter to the interval at which the monitor sends requests, or leave it unspecified to use the default value of10
.Set theMONITOR_TIMEOUT
parameter to the monitor's timeout for its requests, or leave it unset to use the default value of5
.It is expected that for each pool member, the routing solution sends aGET
request to the resource identified on that host by the value of theMONITOR_PATH
parameter for the associated monitor, and that the host responds with the value of theMONITOR_UP_CODE
parameter if the host is up or some other response if the host is not up. - You can change the port that nginx or Nginx Plus® listens on for HTTP or HTTPS, if required, by setting the following parameters:
SSL_PORT=443 HTTP_PORT=80
For Nginx Plus®, setting the above parameters is all that is required. For nginx 1.6 (from Red Hat Software Collections), however, you must also modify the/opt/rh/nginx16/root/etc/nginx/nginx.conf
file to listen on different ports. For example for HTTP, change80
on the following line to another port:listen 80;
In both cases (nginx 1.6 and Nginx Plus®), ensure theSSL_PORT
andHTTP_PORT
parameters are set to the ports you intend nginx or Nginx Plus® to listen to, and ensure your host firewall configuration allows ingress traffic on these ports. - Start the routing daemon:
#
chkconfig openshift-routing-daemon on
#service openshift-routing-daemon start
Developing a Custom Routing Listener
If you are not using the sample routing daemon, you can develop your own listener to listen to the event notifications published on ActiveMQ by the sample routing plug-in. The plug-in creates notification messages for the following events:
Table 8.3. Sample Routing Plug-in Notifications
Event | Message Format | Additional Details |
---|---|---|
Application created |
:action => :create_application,
:app_name => app.name,
:namespace => app.domain.namespace,
:scalable => app.scalable,
:ha => app.ha,
| |
Application deleted |
:action => :delete_application,
:app_name => app.name,
:namespace => app.domain.namespace
:scalable => app.scalable,
:ha => app.ha,
| |
Public endpoint created |
:action => :add_public_endpoint,
:app_name => app.name,
:namespace => app.domain.namespace,
:gear_id => gear._id.to_s,
:public_port_name => endpoint_name,
:public_address => public_ip,
:public_port => public_port.to_i,
:protocols => protocols,
:types => types,
:mappings => mappings
|
Values for the
protocols variable include:
Values for the
types variable include:
These variables depend on values set in the cartridge manifest.
|
Public endpoint deleted |
:action => :remove_public_endpoint,
:app_name => app.name,
:namespace => app.domain.namespace,
:gear_id => gear._id.to_s,
:public_address => public_ip,
:public_port => public_port.to_i
| |
SSL certificate added |
:action => :add_ssl,
:app_name => app.name,
:namespace => app.domain.namespace,
:alias => fqdn,
:ssl => ssl_cert,
:private_key => pvt_key,
:pass_phrase => passphrase
| |
SSL certificate removed |
:action => :remove_ssl,
:app_name => app.name,
:namespace => app.domain.namespace,
:alias => fqdn
| |
Alias added |
:action => :add_alias,
:app_name => app.name,
:namespace => app.domain.namespace,
:alias => alias_str
| |
Alias removed |
:action => :remove_alias,
:app_name => app.name,
:namespace => app.domain.namespace,
:alias => alias_str
|
Note
The
add_gear
and delete_gear
actions have been deprecated. Use add_public_endpoint
for add_gear
and remove_public_endpoint
for delete_gear
instead.
Follow these guidelines when developing a routing listener for the sample routing plug-in:
Routing Listener Guidelines
- Listen to the ActiveMQ topic
routinginfo
. Verify that the user credentials match those configured in the/etc/openshift/plugins.d/openshift-origin-routing-activemq.conf
file of the sample routing plug-in. - For each gear event, reload the routing table of the router.
- Use the
protocols
value provided with theadd_public_endpoint
action to tailor your routing methods. - Use the
types
value to identify the type of endpoint. - Use the
mappings
value to identify URL routes. Routes that are not root may require source IP or SSL certificate verifications. A common use case involves administrative consoles such as phpMyAdmin.
- Look for actions involving SSL certificates, such as
add_ssl
andremove_ssl
, and decide whether to configure the router accordingly for incoming requests. - Look for actions involving aliases, such as
add_alias
andremove_alias
. Aliases must always be accommodated for during the application's life cycle.
Note
The
add_public_endpoint
and remove_public_endpoint
actions do not correspond to the actual addition and removal of gears, but rather to the exposure and concealment of ports. One gear added to an application may result in several exposed ports, which will all result in respective add_public_endpoint
notifications at the router level.
Example 8.18. Simple Routing Listener
The following
listener.rb
script file is an example model for a simple routing listener. This Ruby script uses Nginx as the external routing solution, and the pseudo code provided is an example only. The example handles the following tasks:
- Look for messages with an
add_public_endpoint
action and aload_balancer
type, then edit the router configuration file for the application. - Look for messages with a
remove_public_endpoint
action and aload_balancer
type, then edit the router configuration file for the application. - Look for messages with a
delete_application
action and remove the router configuration file for the application.
#!/usr/bin/ruby require 'rubygems' require 'stomp' require 'yaml' CONF_DIR='/etc/nginx/conf.d/' def add_haproxy(appname, namespace, ip, port) scope = "#{appname}-#{namespace}" file = File.join(CONF_DIR, "#{scope}.conf") if File.exist?(file) `sed -i 's/upstream #{scope} {/&\\n server #{ip}:#{port};/' #{file}` else # write a new one template = <<-EOF upstream #{scope} { server #{ip}:#{port}; } server { listen 8000; server_name ha-#{scope}.dev.rhcloud.com; location / { proxy_pass http://#{scope}; } } EOF File.open(file, 'w') { |f| f.write(template) } end `nginx -s reload` end c = Stomp::Client.new("routinginfo", "routinginfopasswd", "localhost", 61613, true) c.subscribe('/topic/routinginfo') { |msg| h = YAML.load(msg.body) if h[:action] == :add_public_endpoint if h[:types].include? "load_balancer" add_haproxy(h[:app_name], h[:namespace], h[:public_address], h[:public_port]) puts "Added routing endpoint for #{h[:app_name]}-#{h[:namespace]}" end elsif h[:action] == :remove_public_endpoint # script does not actually act upon the remove_public_endpoint as written elsif h[:action] == :delete_application scope = '#{h[:app_name]}-#{h[:namespace]}' file = File.join(CONF_DIR, "#{scope}.conf") if File.exist?(file) `rm -f #{file}` `nginx -s reload` puts "Removed configuration for #{scope}" end end } c.join