17.6. Web, HTTP Connectors, and HTTP Clustering

17.6.1. About the mod_cluster HTTP Connector

The mod_cluster module enables load balancing and is referred to as a connector. To learn about other connectors, see one of the following:
The mod_cluster connector has several advantages over other connectors.
  • The mod_cluster Management Protocol (MCMP) is an additional connection between the JBoss Enterprise Application Platform 6 servers and the Apache HTTP Server with the mod_cluster module enabled. It is used by the JBoss Enterprise Application Platform servers to transmit server-side load balance factors and lifecycle events back to the Apache HTTP Server via a custom set of HTTP methods.
  • Dynamic configuration of Apache HTTP Server with mod_cluster allows JBoss EAP 6 servers to join the load balancing arrangement without manual configuration.
  • JBoss EAP 6 performs the load-balancing factor calculations, rather than relying on the Apache HTTP Server with mod_cluster. This makes load balancing metrics more accurate than other connectors.
  • The mod_cluster connector gives fine-grained application lifecycle control. Each JBoss EAP 6 server forwards web application context lifecycle events to the Apache HTTP Server, informing it to start or stop routing requests for a given context. This prevents end users from seeing HTTP errors due to unavailable resources.
  • AJP, HTTP or HTTPS transports can be used.

17.6.2. Configure the mod_cluster Subsystem

The mod_cluster subsystem can be configured via the Management Console and Management CLI. In this topic the various configuration options are described, grouped as they appear in the Management Console. Example Management CLI commands are provided for each option.

Note

The mod_cluster configuration page is only visible for ha and full-ha profiles. For a managed domain these profiles are ha and full-ha, and for a standalone server they are standalone-ha and standalone-full-ha.
Management Console

Click the Configuration tab. If you are configuring a managed domain, select the correct profile from the Profile drop-down list. Expand the Subsystems menu, then expand the Web menu and select mod_cluster.

Table 17.8. mod_cluster Advertising Configuration Options

Option Description CLI Command
Load Balancing Group
If this is not null, requests are sent to a specific load balancing group on the load balancer. Leave this blank if you do not want to use load balancing groups. This is unset by default.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=load-balancing-group,value=myGroup)
Balancer
This attribute specifies what mod_proxy balancer is to be automatically configured by mod_cluster on the Apache HTTP Server. The default is none, in which case the default of mycluster is used (balancer://mycluster/ when expressed in mod_proxy terms). This default value is configured on the Apache HTTP Server side with the ManagerBalancerName directive.
If you use two different balancer attribute values on the JBoss EAP 6 worker instances, there will be two different mod_proxy balancers created by mod_cluster automatically on the Apache HTTP Server.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=balancer,value=myBalancer)
Advertise Socket
The name of the socket binding to use for cluster advertising.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=advertise-socket,value=modcluster)
Advertise Security Key
A string containing the security key for advertising.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=advertise-security-key,value=myKey)
Advertise
Whether or not advertising is enabled. Defaults to true.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=advertise,value=true)

Table 17.9. mod_cluster Session Configuration Options

Option Description CLI Command
Sticky Session
Whether to use sticky sessions for requests. This means that after the client makes a connection to a specific node, further communication is routed to that same node unless it becomes unavailable. This defaults to true, which is the recommended setting.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=sticky-session,value=true)
Sticky Session Force
If true, a request is not redirected to a new node if its initial node becomes unavailable but instead it fails. This defaults to false.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=sticky-session-force,value=false)
Sticky Session Remove
Remove session information on failover. This defaults to false.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=sticky-session-remove,value=false)

Table 17.10. mod_cluster Web Context Configuration Options

Option Description CLI Command
Auto Enable Contexts
Whether to add new contexts to mod_cluster by default or not. This defaults to true. If you change the default and need to enable context manually, the Web Application can enable its context using the enable() MBean method, or via the mod_cluster manager, which is a web application which runs on the httpd proxy on a named virtual host or port which is specified in that httpd's configuration.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=auto-enable-contexts,value=true)
Excluded Contexts
A comma-separated list of contexts that mod_cluster should ignore. If no host is indicated, the host is assumed to be localhost. ROOT indicates the root context of the Web Application. The default value is ROOT,invoker,jbossws,juddi,console.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=excluded-contexts,value="ROOT,invoker,jbossws,juddi,console")

Table 17.11. mod_cluster Proxy Configuration Options

Option Description CLI Command
Proxy URL
If defined, this value will be prepended to the URL of MCMP commands.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=proxy-url,value=myhost)
Proxy List
A comma-separated list of httpd proxy addresses, in the format hostname:port. This indicates the list of proxies that the mod_cluster process will attempt to communicate with initially.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=proxy-list,value="127.0.0.1,127.0.0.2")
Configure SSL Communication for mod_cluster

By default, mod_cluster communication happens over an unencrypted HTTP link. If you set the connector scheme to HTTPS (refer to Table 17.9, “mod_cluster Session Configuration Options”), the settings below tell mod_cluster where to find the information to encrypt the connection.

Table 17.12. mod_cluster SSL Configuration Options

Option Description CLI Command
Key Alias
The key alias, which was chosen when the certificate was created.
/subsystem=modcluster/mod-cluster-config=configuration/ssl=configuration/:write-attribute(name=key-alias,value=jboss)
Password
This password is the keystore password for both keystores: certificate-key-file (Key File) and ca-certificate-file (Cert File) and the key/certificate entry specified with Key Alias inside Cert File.

Note

@ca-certificate-password is the truststore password and value remains undefined if you have not specified it.
/subsystem=modcluster/mod-cluster-config=configuration/ssl=configuration/:write-attribute(name=password,value=changeit)
CA Cert File/Trust Store
Trust store used to validate the web server certificate.
/subsystem=modcluster/mod-cluster-config=configuration/ssl=configuration/:write-attribute(name=ca-certificate-file,value=${user.home}/jboss.crt)
Key Store
Key store that holds the certificate and private key that identifies this instance.
/subsystem=modcluster/mod-cluster-config=configuration/ssl=configuration/:write-attribute(name=certificate-key-file,value=${user.home}/.keystore)
Cipher Suite
The allowed encryption cipher suite.
/subsystem=modcluster/mod-cluster-config=configuration/ssl=configuration/:write-attribute(name=cipher-suite,value=ALL)
Revocation URL
The URL of the Certificate Authority revocation list.
/subsystem=modcluster/mod-cluster-config=configuration/ssl=configuration/:write-attribute(name=ca-revocation-url,value=jboss.crl)
Protocol
The SSL protocols, which are enabled.
You can also specify a combination of protocols, which is comma separated. For example, TLSv1, TLSv1.1,TLSv1.2.

Warning

Red Hat recommends that you explicitly disable SSL in favor of TLSv1.1 or TLSv1.2 in all affected packages.
/subsystem=modcluster/mod-cluster-config=configuration/ssl=configuration/:write-attribute(name=protocol,value="TLSv1, TLSv1.1,TLSv1.2")
Configure mod_cluster Networking Options

The available mod_cluster networking options control several different timeout behaviors for different types of services with which the mod_cluster service communicates.

Table 17.13. mod_cluster Networking Configuration Options

Option Description CLI Command
Node Timeout
Timeout, in seconds, for proxy connections to a worker. This is the time that mod_cluster will wait for the back-end response before returning an error. If the node-timeout attribute is undefined, the httpd ProxyTimeout directive is used. If ProxyTimeout is undefined, the httpd Timeout directive is used, which defaults to 300 seconds.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=node-timeout,value=-1)
Socket Timeout
Number of seconds to wait for a response from an httpd proxy to MCMP commands before timing out, and flagging the proxy as in error.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=socket-timeout,value=20)
Stop Context Timeout
The amount of time, measure in units specified by stopContextTimeoutUnit, for which to wait for clean shutdown of a context (completion of pending requests for a distributable context; or destruction/expiration of active sessions for a non-distributable context).
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=stop-context-timeout,value=10)
Session Draining Strategy
Whether to drain sessions before undeploying a web application.
DEFAULT
Drain sessions before web application undeploy only if the web application is non-distributable.
ALWAYS
Always drain sessions before web application undeploy, even for distributable web applications.
NEVER
Do not drain sessions before web application undeploy, even for non-distributable web application.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=session-draining-strategy,value=DEFAULT)
Max Attempts
Number of times an httpd proxy will attempt to send a given request to a node before giving up. The minimum value is 1, meaning try only once. The mod_proxy default is also 1, which means that no retry occurs.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=max-attempts,value=1)
Flush Packets
Whether or not to enable packet flushing to the Web server. Defaults to false.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=flush-packets,value=false)
Flush Wait
How long, in seconds, to wait before flushing packets to the Web server. Defaults to -1. A value of -1 means to wait forever before flushing packets.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=flush-wait,value=-1)
Ping
How long, in seconds, to wait for a response to a ping from a worker. Defaults to 10 seconds.
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=ping,value=10)
SMAX
Soft maximum idle connection count (the same as smax in mod_proxy documentation). The maximum value depends on the httpd thread configuration, and can be either ThreadsPerChild or 1.
profile=full-ha/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=smax,value=ThreadsPerChild)
TTL
Time to live (in seconds) for idle connections above smax, default is 60
/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=ttl,value=-1)
mod_cluster Load Provider Configuration Options

The following mod_cluster configuration options are not available in the management console, but can only be set using the Management CLI.

A simple load provider is used if no dynamic load provider is present. It assigns each cluster member a load factor of 1, and distributes work evenly without applying a load balancing algorithm. To add it, use the following Management CLI command.
[standalone@localhost:9990 /] /subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=simple-load-provider, value=1)
A dynamic load provider can be configured to use a variety of algorithms in combination, in order to determine which worker receives the next request. You can create a load provider and configure it to suit your environment, and you can have more than one load metric active simultaneously by adding them via the CLI. The default dynamic load provider uses busyness as the determining load metric. The dynamic load provider options and possible load metrics are shown below.

Table 17.14. mod_cluster Dynamic Load Provider Options

Option Description CLI Command
Decay
The factor by which historical metrics should decay in significance.
/subsystem=modcluster/mod-cluster-config=configuration/dynamic-load-provider=configuration/:write-attribute(name=decay,value=2)
History
The number of historic load metric records to consider when determining the load.
/subsystem=modcluster/mod-cluster-config=configuration/dynamic-load-provider=configuration/:write-attribute(name=history,value=9)
Load Metric
The default load metric included with the dynamic load provider in JBoss EAP 6 is busyness, which calculates the load of the worker from the amount of threads in the thread pool being busy serving requests. You can set the capacity of this metric by which the actual load is divided: calculated_load / capacity. You can set multiple load metrics within the dynamic load provider.
/subsystem=modcluster/mod-cluster-config=configuration/dynamic-load-provider=configuration/load-metric=busyness/:write-attribute(name=capacity,value=1.0)
/subsystem=modcluster/mod-cluster-config=configuration/dynamic-load-provider=configuration/load-metric=busyness/:write-attribute(name=type,value=busyness)
/subsystem=modcluster/mod-cluster-config=configuration/dynamic-load-provider=configuration/load-metric=busyness/:write-attribute(name=weight,value=1)

Load Metric Algorithms

cpu
The cpu load metric uses average CPU load to determine which node receives the next work load.
mem
The mem load metric uses free native memory as a load metric. Usage of this metric is discouraged because it provides a value that includes buffers and cache, so it is always a very low figure on every decent system with good memory management.
heap
The heap load metric uses the heap usage to determine which worker receives the next work load.
sessions
The session load metric uses the number of active sessions as a metric.
requests
The requests load metric uses the number of client requests to determine which worker receives the next work load. For instance, capacity 1000 means that 1000 requests/sec is considered to be a full load.
send-traffic
The send-traffic load metric uses the amount of traffic sent from the worker to the clients. E.g. the default capacity of 512 indicates that the node should be considered under full load if the average outbound traffic is 512 KB/s or higher.
receive-traffic
The receive-traffic load metric uses the amount of traffic sent to the worker from the clients. E.g. the default capacity of 1024 indicates that the worker should be considered under full load if the average inbound traffic is 1024 KB/s or higher.
busyness
This metric represents the amount of threads from the thread pool being busy serving requests.

Example 17.2. Add a Load Metric

To add a load metric, use the add-metric command.
/subsystem=modcluster/mod-cluster-config=configuration/:add-metric(type=sessions)

Example 17.3. Set a Value for an Existing Metric

To set a value for an existing metric, use the write-attribute command.
/subsystem=modcluster/mod-cluster-config=configuration/dynamic-load-provider=configuration/load-metric=cpu/:write-attribute(name="weight",value="3")

Example 17.4. Change the Value of an Existing Metric

To change the value of an existing metric, use the write-attribute command.
/subsystem=modcluster/mod-cluster-config=configuration/dynamic-load-provider=configuration/load-metric=cpu/:write-attribute(name="type",value="busyness")

Example 17.5. Remove an Existing Metric

To remove an existing metric, use the remove-metric command.
/subsystem=modcluster/mod-cluster-config=configuration/:remove-metric(type=sessions)

17.6.3. Install the mod_cluster Module Into Apache HTTP Server or JBoss Enterprise Web Server (Zip)

Prerequisites

  • To perform this task, you must be using Apache HTTP Server installed in Red Hat Enterprise Linux 6, or JBoss Enterprise Web Server, or the standalone Apache HTTP Server included as a separate downloadable component of JBoss EAP 6.
  • If you need to install Apache HTTP Server in Red Hat Enterprise Linux 6, use the instructions from the Red Hat Enterprise Linux 6 Deployment Guide.
  • If you need to install the standalone Apache HTTP Server included as a separate downloadable component of JBoss EAP 6, refer to Section 17.4.3, “Install Apache HTTP Server in Red Hat Enterprise Linux 5, 6, and 7 (Zip)”.
  • If you need to install JBoss Enterprise Web Server, use the instructions from the JBoss Enterprise Web Server Installation Guide.
  • Download the Webserver Connecter Natives package for your operating system and architecture from the Red Hat Customer Portal at https://access.redhat.com. This package contains the mod_cluster binary web server modules precompiled for your operating system. After you extract the archive, the modules are located in the EAP_HOME/modules/system/layers/base/native/lib/httpd/modules directory.
    The etc/ directory contains some example configuration files, and the share/ directory contains some supplemental documentation.
  • You must be logged in with administrative (root) privileges.

Note

If you use a 64 bit system the mod_cluster binary web server modules will be located here: EAP_HOME/modules/system/layers/base/native/lib64/httpd/modules. You must use this path whenever you need access to the modules.

Procedure 17.12. Install the mod_cluster Module

  1. Determine your Apache HTTP Server configuration location.

    Your Apache HTTP Server configuration location will be different depending on whether you are using Red Hat Enterprise Linux's Apache HTTP Server, the standalone Apache HTTP Server included as a separate downloadable component with JBoss EAP 6, or the Apache HTTP Server available in JBoss Enterprise Web Server. It is one of the following three options, and is referred to in the rest of this task as HTTPD_HOME.
    • Apache HTTP Server - /etc/httpd/
    • JBoss EAP 6 Apache HTTP Server - This location is chosen by you, based on the requirements of your infrastructure.
    • JBoss Enterprise Web Server Apache HTTP Server - EWS_HOME/httpd/
  2. Copy the modules to the Apache HTTP Server modules directory.

    Copy the four modules (the files ending in .so) from the EAP_HOME/modules/system/layers/base/native/lib/httpd/modules directory of the extracted Webserver Natives archive to the HTTPD_MODULES/ directory.
  3. For JBoss Enterprise Web Server, disable the mod_proxy_balancer module.

    If you use JBoss Enterprise Web Server, the mod_proxy_balancer module is enabled by default. It is incompatible with mod_cluster. To disable it, edit the HTTPD_CONF/httpd.conf and comment out the following line by placing a # (hash) symbol before the line which loads the module. The line is shown without the comment and then with it, below.
    LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
    # LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
    Save and close the file.
  4. Configure the mod_cluster module.

    The Webserver Natives archive contains a sample mod_cluster.conf file (EAP_HOME/modules/system/layers/base/native/etc/httpd/conf). This file can be used as a guide or copied and edited to create a HTTPD_CONF.D/JBoss_HTTP.conf file.

    Note

    Using the name JBoss_HTTP.conf is an arbitrary convention in this document. The configuration file will be loaded, regardless of its name, if it is saved in the conf.d/ directory with the .conf extension.
    Add the following to your configuration file:
    LoadModule slotmem_module modules/mod_slotmem.so
    LoadModule manager_module modules/mod_manager.so
    LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
    LoadModule advertise_module modules/mod_advertise.so
    
    This causes Apache HTTP Server to automatically load the modules that mod_cluster needs in order to function.
  5. Create a proxy server listener.

    Continue editing HTTPD_CONF.D/JBoss_HTTP.conf and add the following minimal configuration, replacing the values in capital letters with suitable values for your environment.
    Listen IP_ADDRESS:PORT
    <VirtualHost IP_ADDRESS:PORT>  
    	  <Location />
              Order deny,allow
              Deny from all
              Allow from *.MYDOMAIN.COM
    	  </Location>
    	  
    	  KeepAliveTimeout 60
    	  MaxKeepAliveRequests 0
    	  EnableMCPMReceive
    	  
    	  ManagerBalancerName mycluster
    	  ServerAdvertise On
    	  
    </VirtualHost>
    These directives create a new virtual server which listens on IP_ADDRESS:PORT, allows connections from MYDOMAIN.COM, and advertises itself as a balancer called mycluster. These directives are covered in detail in the documentation for Apache Web Server. To learn more about the ServerAdvertise and EnableMCPMReceive directives, and the implications of server advertisement, see Section 17.6.5, “Configure Server Advertisement Properties for Your mod_cluster-enabled Web Server”.
    Save the file and exit.
  6. Restart the Apache HTTP Server.

    The way to restart the Apache HTTP Server depends on whether you are using Red Hat Enterprise Linux's Apache HTTP Server or the Apache HTTP Server included in JBoss Enterprise Web Server. Choose one of the two methods below.
    • Red Hat Enterprise Linux 6 Apache HTTP Server

      Issue the following command:
      [root@host]# service httpd restart
    • JBoss Enterprise Web Server HTTP Server

      JBoss Enterprise Web Server runs on both Red Hat Enterprise Linux and Microsoft Windows Server. The method for restarting the Apache HTTP Server is different for each.
      • Red Hat Enterprise Linux

        In Red Hat Enterprise Linux, JBoss Enterprise Web Server installs its Apache HTTP Server as a service. To restart the Apache HTTP Server, issue the following two commands:
        [root@host ~]# service httpd stop
        [root@host ~]# service httpd start
      • Microsoft Windows Server

        Issue the following commands in a command prompt with administrative privileges:
        C:\> net stop httpd
        C:\> net start httpd
Result

The Apache HTTP Server is now configured as a load balancer, and can work with the mod_cluster subsystem running JBoss EAP 6. To configure JBoss EAP 6 to be aware of mod_cluster, see Section 17.6.6, “Configure a mod_cluster Worker Node”.

17.6.4. Install the mod_cluster Module Into Apache HTTP Server or JBoss Enterprise Web Server (RPM)

Prerequisites

  • To perform this task, you must be using the Apache HTTP Server installed in Red Hat Enterprise Linux 6, JBoss Enterprise Web Server, or the standalone Apache HTTP Server included as a separate downloadable component of JBoss EAP 6.
  • If you need to install Apache HTTP Server in Red Hat Enterprise Linux 6, use the instructions from the Red Hat Enterprise Linux 6 Deployment Guide.
  • If you need to install the standalone Apache HTTP Server included as a separate downloadable component of JBoss EAP 6, refer to Section 17.4.3, “Install Apache HTTP Server in Red Hat Enterprise Linux 5, 6, and 7 (Zip)” .
  • If you need to install JBoss Enterprise Web Server, use the instructions from the JBoss Enterprise Web Server Installation Guide.
  • You must be logged in with administrative (root) privileges.
  • You must have an active subscription to the jbappplatform-6-ARCH-server-VERS-rpm RHN channel.
The RPM installation method is similar between Red Hat Enterprise Linux 5 and 6, only requiring minor variations for Red Hat Enterprise Linux 6 users who have Apache HTTP Server 2.2.15 installed.
  1. Install the mod_cluster-native package using YUM:
    yum install mod_cluster-native
    
  2. Apache HTTP Server:
    • If you choose to stay on Apache HTTP Server 2.2.15, you must disable the mod_proxy_balancer module loaded by default by commmenting the LoadModule proxy_balancer_module line in the httpd.conf file.
      Either edit the file manually or use the following command:
      sed -i 's/^LoadModule proxy_balancer_module/#LoadModule proxy_balancer_module/;s/$//' /etc/httpd/conf/httpd.conf
      
    • If you choose to upgrade to Apache HTTP Server 2.2.26, install the latest version using the following command.
      yum install httpd
  3. To have the Apache HTTP Server service start at boot, enter the following command:
    • For Red Hat Enterprise Linux 5 and 6:
      service httpd add
    • For Red Hat Enterprise Linux 7:
      systemctl enable httpd22.service
  4. Start the mod_cluster balancer with the following command:
    • For Red Hat Enterprise Linux 5 and 6:
      service httpd start
    • For Red Hat Enterprise Linux 7:
      systemctl start httpd22.service

17.6.5. Configure Server Advertisement Properties for Your mod_cluster-enabled Web Server

Summary

For instructions on configuring your web server to interact with the mod_cluster load balancer, see Section 17.6.3, “Install the mod_cluster Module Into Apache HTTP Server or JBoss Enterprise Web Server (Zip)”. One aspect of the configuration which needs more explanation is server advertisement.

When server advertisement is active, the web server broadcasts messages containing the IP address and port number specified in the mod_cluster virtual host. To configure these values, see Section 17.6.3, “Install the mod_cluster Module Into Apache HTTP Server or JBoss Enterprise Web Server (Zip)”. If UDP multicast is not available on your network, or you prefer to configure workers with a static list of proxy servers, you can disable server advertisement and manually configure the worker nodes. See Section 17.6.6, “Configure a mod_cluster Worker Node” for information on configuring a worker.
The changes in this procedure need to be made to the httpd.conf associated with your Apache HTTP Server instance. This is often /etc/httpd/conf/httpd.conf in Red Hat Enterprise Linux, or may be in the etc/ directory of your standalone Apache HTTP Server instance.

Procedure 17.13. Edit the httpd.conf file and implement the changes

  1. Disable the AdvertiseFrequency parameter, if it exists.

    If you have a line like the following in your <VirtualHost> statement, comment it out by putting a # (hash) character before the first character. The value may be different from 5.
    AdvertiseFrequency 5
  2. Add the directive to disable server advertisement.

    Add the following directive inside the <VirtualHost> statement, to disable server advertisement.
    ServerAdvertise Off
  3. Enable the ability to receive MCPM messages.

    Add the following directive to allow the Web server to receive MCPM messages from the worker nodes.
    EnableMCPMReceive
  4. Restart the Web server.

    Restart the Web server by issuing one of the following, depending on whether you use Red Hat Enterprise Linux or Microsoft Windows Server.
    • Red Hat Enterprise Linux

      [root@host ]# service httpd restart
    • Microsoft Windows Server

      net stop Apache2.2
      net start Apache2.2
      
Result

The web server no longer advertises the IP address and port of your mod_cluster proxy. To reiterate, you need to configure your worker nodes to use a static address and port to communicate with the proxy. See Section 17.6.6, “Configure a mod_cluster Worker Node” for more details.

17.6.6. Configure a mod_cluster Worker Node

Summary

A mod_cluster worker node consists of a JBoss EAP 6 server. This server can be part of a server group in a Managed Domain, or a standalone server. A separate process runs within JBoss EAP 6, which manages all of the worker nodes of the cluster. This is called the master. For more conceptual information about nodes, see Section 17.2.4, “Node types”. For an overview of web server load balancing, see to Section 17.2.3, “Overview of HTTP Connectors”.

Worker nodes in a managed domain share an identical configuration across a server group. Worker nodes running as standalone servers are configured individually. The configuration steps are otherwise identical.

Worker Node Configuration

  • A standalone server must be started with the standalone-ha or standalone-full-ha profile.
  • A server group in a managed domain must use the ha or full-ha profile, and the ha-sockets or full-ha-sockets socket binding group. JBoss EAP 6 ships with a cluster-enabled server group called other-server-group which meets these requirements.

Note

Where Management CLI commands are given, they assume you use a managed domain. If you use a standalone server, remove the /profile=full-ha portion of the commands.

Procedure 17.14. Configure a Worker Node

  1. Configure the network interfaces.

    By default, the network interfaces all default to 127.0.0.1. Every physical host that hosts either a standalone server or one or more servers in a server group needs its interfaces to be configured to use its public IP address, which the other servers can see.
    To change the IP address of a JBoss EAP 6 host, you need to shut it down and edit its configuration file directly. This is because the Management API which drives the Management Console and Management CLI relies on a stable management address.
    Follow these steps to change the IP address on each server in your cluster to the master's public IP address.
    1. Start the JBoss EAP server using the profile described earlier in this topic.
    2. Launch the Management CLI, using the EAP_HOME/bin/jboss-cli.sh command in Linux or the EAP_HOME\bin\jboss-cli.bat command in Microsoft Windows Server. Type connect to connect to the domain controller on the localhost, or connect IP_ADDRESS to connect to a domain controller on a remote server.
    3. Modify the external IP address for the management, public and unsecure interfaces by typing the following commands. Be sure to replace EXTERNAL_IP_ADDRESS in the command with the actual external IP address of the host.
      /interface=management:write-attribute(name=inet-address,value="${jboss.bind.address.management:EXTERNAL_IP_ADDRESS}"
      /interface=public:write-attribute(name=inet-address,value="${jboss.bind.address.public:EXTERNAL_IP_ADDRESS}"
      /interface=unsecure:write-attribute(name=inet-address,value="${jboss.bind.address.unsecure:EXTERNAL_IP_ADDRESS}"
      reload
      You should see the following result for each command.
       "outcome" => "success"
    4. For hosts that participate in a managed domain but are not the master, you must change the host name from master to a unique name. This name must be unique across slaves and will be used for the slave to identify to the cluster, so make a note of the name you use.
      1. Start the JBoss EAP slave host using the following syntax:
        bin/domain.sh --host-config=HOST_SLAVE_XML_FILE_NAME
        For example:
        bin/domain.sh --host-config=host-slave01.xml
      2. Launch the Management CLI.
      3. Use the following syntax to replace the host name:
        /host=master:write-attribute(name="name",value=UNIQUE_HOST_SLAVE_NAME)
        For example:
        /host=master:write-attribute(name="name",value="host-slave01")
        You should see the following result.
         "outcome" => "success"
        This modifies the XML in the host-slave01.xml file as follows:
        <host name="host-slave01" xmlns="urn:jboss:domain:1.6">
    5. For newly configured hosts that need to join a managed domain, you must remove the local element and add the remote element host attribute that points to the domain controller. This step does not apply for a standalone server.
      1. Start the JBoss EAP slave host using the following syntax:
        bin/domain.sh --host-config=HOST_SLAVE_XML_FILE_NAME
        For example:
        bin/domain.sh --host-config=host-slave01.xml
      2. Launch the Management CLI.
      3. Use the following syntax specify the domain controller:
        /host=UNIQUE_HOST_SLAVE_NAME/:write-remote-domain-controller(host=DOMAIN_CONTROLLER_IP_ADDRESS,port=${jboss.domain.master.port:9999},security-realm="ManagementRealm") 
        For example:
        /host=host-slave01/:write-remote-domain-controller(host="192.168.1.200",port=${jboss.domain.master.port:9999},security-realm="ManagementRealm") 
        You should see the following result.
         "outcome" => "success"
        This modifies the XML in the host-slave01.xml file as follows:
        <domain-controller>
            <remote host="192.168.1.200" port="${jboss.domain.master.port:9999}" security-realm="ManagementRealm"/>
        </domain-controller>
  2. Configure authentication for each slave server.

    Each slave server needs a username and password created in the domain controller's or standalone master's ManagementRealm. On the domain controller or standalone master, run the EAP_HOME/bin/add-user.sh command. Add a user with the same username as the slave, to the ManagementRealm. When asked if this user will need to authenticate to an external JBoss EAP 6 instance, answer yes. An example of the input and output of the command is below, for a slave called slave1, with password changeme.
    user:bin user$ ./add-user.sh
    
    What type of user do you wish to add? 
     a) Management User (mgmt-users.properties) 
     b) Application User (application-users.properties)
    (a): a
    
    Enter the details of the new user to add.
    Realm (ManagementRealm) : 
    Username : slave1
    Password : changeme
    Re-enter Password : changeme
    About to add user 'slave1' for realm 'ManagementRealm'
    Is this correct yes/no? yes
    Added user 'slave1' to file '/home/user/jboss-eap-6.0/standalone/configuration/mgmt-users.properties'
    Added user 'slave1' to file '/home/user/jboss-eap-6.0/domain/configuration/mgmt-users.properties'
    Is this new user going to be used for one AS process to connect to another AS process e.g. slave domain controller?
    yes/no? yes
    To represent the user add the following to the server-identities definition <secret value="Y2hhbmdlbWU=" />
    
  3. Copy the Base64-encoded <secret> element from the add-user.sh output.

    If you plan to specify the Base64-encoded password value for authentication, copy the <secret> element value from the last line of the add-user.sh output as you will need it in the step below.
  4. Modify the slave host's security realm to use the new authentication.

    You can specify the secret value in one of the following ways:
    • Specify the Base64-encoded password value in the server configuration file using the Management CLI.

      1. Launch the Management CLI, using the EAP_HOME/bin/jboss-cli.sh command in Linux or the EAP_HOME\bin\jboss-cli.bat command in Microsoft Windows Server. Type connect to connect to the domain controller on the localhost, or connect IP_ADDRESS to connect to a domain controller on a remote server.
      2. Specify the secret value by typing the following command. Be sure to replace the SECRET_VALUE with the secret value returned from the add-user output from the previous step.
        /host=master/core-service=management/security-realm=ManagementRealm/server-identity=secret:add(value="SECRET_VALUE") 
        reload --host=master
        You should see the following result for each command.
         "outcome" => "success"
    • Configure the host to get the password from the vault.

      1. Use the vault.sh script to generate a masked password. It will generate a string like the following: VAULT::secret::password::ODVmYmJjNGMtZDU2ZC00YmNlLWE4ODMtZjQ1NWNmNDU4ZDc1TElORV9CUkVBS3ZhdWx0.
        You can find more information on password vaults in the Security Architecture and other JBoss EAP security documentation.
      2. Launch the Management CLI, using the EAP_HOME/bin/jboss-cli.sh command in Linux or the EAP_HOME\bin\jboss-cli.bat command in Microsoft Windows Server. Type connect to connect to the domain controller on the localhost, or connect IP_ADDRESS to connect to a domain controller on a remote server.
      3. Specify the secret value by typing the following command. Be sure to replace the SECRET_VALUE with the masked password generated in the previous step.
        /host=master/core-service=management/security-realm=ManagementRealm/server-identity=secret:add(value="${VAULT::secret::password::SECRET_VALUE}") 
        reload --host=master
        You should see the following result for each command.
         "outcome" => "success"

        Note

        When creating a password in the vault, it must be specified in plain text, not Base64-encoded.
    • Specify the password as a system property.

      The following examples use server.identity.password as the system property name for the password.
      1. Specify the system property for the password in the server configuration file using the Management CLI.
        1. Launch the Management CLI, using the EAP_HOME/bin/jboss-cli.sh command in Linux or the EAP_HOME\bin\jboss-cli.bat command in Microsoft Windows Server. Type connect to connect to the domain controller on the localhost, or connect IP_ADDRESS to connect to a domain controller on a remote server.
        2. Type the following command to configure the secret identity to use the system property.
          /host=master/core-service=management/security-realm=ManagementRealm/server-identity=secret:add(value="${server.identity.password}") 
          reload --host=master
          You will see the following result for each command.
           "outcome" => "success"
      2. When you specify the password as a system property, you can configure the host in either of the following ways:
        • Start the server entering the password in plain text as a command line argument, for example:
          -Dserver.identity.password=changeme

          Note

          The password must be entered in plain text and will be visible to anyone who issues a ps -ef command.
        • Place the password in a properties file and pass the properties file URL as a command line argument.
          1. Add the key/value pair to a properties file. For example:
            server.identity.password=changeme
          2. Start the server with the command line arguments
            --properties=URL_TO_PROPERTIES_FILE
            .
  5. Restart the server.

    The slave will now authenticate to the master using its host name as the username and the encrypted string as its password.
Result

Your standalone server, or servers within a server group of a managed domain, are now configured as mod_cluster worker nodes. If you deploy a clustered application, its sessions are replicated to all cluster nodes for failover, and it can accept requests from an external Web server or load balancer. Each node of the cluster discovers the other nodes using automatic discovery, by default.To configure automatic discovery, and the other specific settings of the mod_cluster subsystem, see Section 17.6.2, “Configure the mod_cluster Subsystem”. To configure the Apache HTTP Server,see Section 17.4.7, “Use an External Web Server as the Web Front-end for JBoss EAP 6 Applications”.

17.6.7. Migrate Traffic between Clusters

Summary

After creating a new cluster using JBoss EAP 6, you can migrate traffic from the previous cluster to the new one as part of an upgrade process. In this task, you will see the strategy that can be used to migrate this traffic with minimal outage or downtime.

Prerequisites

Procedure 17.15. Upgrade Process for Clusters - Load Balancing Groups

  1. Setup your new cluster using the steps described in the prerequisites.
  2. In both ClusterNEW and ClusterOLD, ensure that the configuration option sticky-session is set to true (this option is set to true by default). Enabling this option means that all new requests made to a cluster node in any of the clusters will continue to go to the respective cluster node.
    /profile=full-ha/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=sticky-session,value=true)
    
  3. Assuming that all the cluster nodes in ClusterOLD are members of ClusterOLD load balancing group. One can set this configuration either via CLI or with an xml configuration (either ha or full-ha profiles in domain mode and either standalone-ha or standalone-full-ha in standalone mode):
    /profile=full-ha/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=load-balancing-group,value=ClusterOLD)
    				
    				<subsystem xmlns="urn:jboss:domain:modcluster:1.2">
    					<mod-cluster-config load-balancing-group="ClusterOLD" advertise-socket="modcluster" connector="ajp">
    						<dynamic-load-provider>
    							<load-metric type="busyness"/>
    						</dynamic-load-provider>
           		</mod-cluster-config>
        		</subsystem>
    
  4. Add the nodes in ClusterNEW to the mod_cluster configuration individually using the process described here: Section 17.6.6, “Configure a mod_cluster Worker Node”. Additionally use the aforementioned procedure and set their load balancing group to ClusterNEW.
    At this point, one can see an output similar to the undermentioned shortened example on the mod_cluster-manager console:
    				
    				mod_cluster/<version>
    				
        LBGroup ClusterOLD: [Enable Nodes]   [Disable Nodes]   [Stop Nodes]
            Node node-1-jvmroute (ajp://node1.oldcluster.example:8009):
                [Enable Contexts]   [Disable Contexts]   [Stop Contexts]
                Balancer: qacluster, LBGroup: ClusterOLD, Flushpackets: Off, ..., Load: 100
                Virtual Host 1:
                    Contexts:
                        /my-deployed-application-context, Status: ENABLED Request: 0 [Disable]   [Stop]
            
            Node node-2-jvmroute (ajp://node2.oldcluster.example:8009):
                [Enable Contexts]   [Disable Contexts]   [Stop Contexts]
                Balancer: qacluster, LBGroup: ClusterOLD, Flushpackets: Off, ..., Load: 100
                Virtual Host 1:
                    Contexts:
                        /my-deployed-application-context, Status: ENABLED Request: 0 [Disable]   [Stop]
            
            
        LBGroup ClusterNEW: [Enable Nodes]   [Disable Nodes]   [Stop Nodes]
            Node node-3-jvmroute (ajp://node3.newcluster.example:8009):
                [Enable Contexts]   [Disable Contexts]   [Stop Contexts]
                Balancer: qacluster, LBGroup: ClusterNEW, Flushpackets: Off, ..., Load: 100
                Virtual Host 1:
                    Contexts:
                        /my-deployed-application-context, Status: ENABLED Request: 0 [Disable]   [Stop]
            
            Node node-4-jvmroute (ajp://node4.newcluster.example:8009):
                [Enable Contexts]   [Disable Contexts]   [Stop Contexts]
                Balancer: qacluster, LBGroup: ClusterNEW, Flushpackets: Off, ..., Load: 100
                Virtual Host 1:
                    Contexts:
                        /my-deployed-application-context, Status: ENABLED Request: 0 [Disable]   [Stop]
    
  5. There are old active sessions within the ClusterOLD group and any new sessions are created either within the ClusterOLD or CLusterNEW group. Next, we want to disable the whole ClusterOLD group, so as we can power down its cluster nodes without causing any error to currently active client's sessions.
    Click on the [Disable Nodes] link for LBGroup ClusterOLD on mod_cluster-manager web console.
    From this point on, only requests belonging to already established sessions will be routed to members of ClusterOLD load balancing group. Any new client's sessions will be created in the ClusterNEW group only. As soon as there are no active sessions within ClusterOLD group, we can safely remove its members.

    Note

    Using [Stop Nodes] would command the load balancer to stop routing any requests to this domain immediately. This will force a failover to another load balancing group which will cause session data loss to clients, provided there is no session replication between ClusterNEW and ClusterOLD.
Default Load Balancing Group

In case the current ClusterOLD setup does not contain any load balancing group settings (one can see LBGroup:, on mod_cluster-manager console), one can still take advantage of disabling the ClusterOLD nodes. In this case, click on [Disable Contexts] for each of the Cluster OLD nodes. Contexts of these nodes will be disabled and once there are no active sessions present, they will be ready for removal. New client's sessions will be created only on nodes with enabled contexts, presumably Cluster NEW members in this example.

Using JBoss EAP CLI

In addition to the possibility of using mod_cluster-manager web console, one can leverage CLI in order to disable a particular context. The undermentioned operation is called stop-context, but it makes the cluster node to send DISABLE-APP command to the load balancer, having exactly the same effect as clicking on [Disable] link next to a particular context on mod_cluster-manager console (note that virtual host aliases, e.g. default-host were removed from the aforementioned mod_cluster-manager console output example).

/profile=full-ha/subsystem=modcluster/:stop-context(context=/my-deployed-application-context, virtualhost=default-host, waittime=50)

Conclusion

To stop a particular context, cluster node or a whole load balancing group means to force the balancer to stop routing any request to it immediately, thus forcing failover to another available context. To disable a particular context, cluster node or a whole load balancing group means to tell the balancer that no new sessions should be crated on this particular context/node/load balancing group.

Result

You have successfully upgraded a JBoss EAP 6 Cluster.

17.6.8. Configure fail_on_status Parameter for mod_cluster

The fail_on_status parameter lists those HTTP status codes which, when returned by a worker node in a cluster, will mark that node as having failed. The load balancer will then send future requests to another worker node in the cluster. The failed worker node will remain in a NOTOK state until it sends the load balancer a STATUS message.

Note

The fail_on_status parameter cannot be used with HP-UX v11.3 hpws httpd B.2.2.15.15 from Hewlett-Packard as it does not support the feature.
The fail_on_status parameter must be configured in the httpd configuration file of your load balancer. Multiple HTTP status codes for fail_on_status can be specified as a comma-separated list. The following example specifies the HTTP status codes 203 and 204 for fail_on_status.

Example 17.6. fail_on_status Configuration

ProxyPass / balancer://MyBalancer stickysession=JSESSIONID|jsessionid nofailover=on failonstatus=203,204
ProxyPassReverse / balancer://MyBalancer
ProxyPreserveHost on