Chapter 3. mod_cluster Connector
The mod_cluster connector is a reduced configuration, intelligent load-balancing solution for JBoss EAP and Apache HTTP Server Tomcat, and is based on technology originally developed by the JBoss mod_cluster community project.
The mod_cluster module load-balances HTTP requests to JBoss EAP and Apache HTTP Server Tomcat worker nodes, utilizing Apache HTTP Server as the proxy server.
3.1.1. Key Features
The mod_cluster connector has several advantages over the mod_jk connector:
- The mod_cluster Management Protocol (MCMP) is an additional connection between the Tomcat servers and the Apache HTTP Server with the mod_cluster module enabled. It is used by the Tomcat servers to transmit server-side load figures and lifecycle events back to Apache HTTP Server via a custom set of HTTP methods.
- Dynamic configuration of Apache HTTP Server with mod_cluster allows Tomcat servers that have mod_cluster listeners to join the load balancing arrangement without manual configuration.
- Tomcat servers perform the load calculations, rather than relying on Apache HTTP Server. This makes load balancing metrics more accurate than other connectors.
- The mod_cluster connector gives fine-grained application lifecycle control. Each Tomcat server forwards web application context lifecycle events to the Apache HTTP Server, informing it to start or stop routing requests for a given context. This prevents end users from seeing HTTP errors due to unavailable resources.
- AJP, HTTP, or HTTPS transports can be used.
On the proxy server, mod_cluster consists of four Apache modules.
Table 3.1. Components
| || |
The Shared Memory Manager module shares real-time worker node information with multiple Apache HTTP Server processes.
| || |
The Cluster Manager module receives and acknowledges messages from worker nodes, including node registrations, node load data, and node application life cycle events.
| || |
The Proxy Balancer Module handles request routing to cluster nodes. The Proxy Balancer selects the appropriate destination node based on application location in the cluster, the current state of each of the cluster nodes, and the Session ID (if a request is part of an established session).
| || |
The Proxy Advertisement Module broadcasts the existence of the proxy server via UDP multicast messages. The server advertisement messages contain the IP address and port number where the proxy server is listening for responses from worker nodes that want to join the load-balancing cluster.
See the Apache HTTP Server Modules appendix for detailed information about the available modules, including user-configurable parameters.
3.1.3. Character Limits
mod_cluster uses shared memory to keep the nodes description. The shared memory is created at the startup of Apache HTTP Server, and the structure of each item is fixed. When defining proxy server and worker node properties, ensure that you follow these character limits:
Maximum alias length: 100 characters (alias corresponds to the network name of the respective virtual host; the name is defined in the
Maximum context length: 40 characters (for example, if
myapp.waris deployed in
/myappis included in the context).
Maximum balancer name length: 40 characters (the balancer property in
JVMRoutestring length: 80 characters (
Maximum domain name length: 20 characters (the domain property in
Maximum hostname length for a node: 64 characters (hostname address in the
Maximum port length for a node: 7 characters (the port property in the
8009is 4 characters).
Maximum scheme length for a node: 6 characters (the protocol of the connector; possible values are
Maximum cookie name length: 30 characters (the header cookie name for session ID. Default value:
Maximum path name length: 30 characters (the parameter name for the session ID. Default value:
Maximum length of a session ID: 120 characters (session ID resembles the following:
3.2. Downloading and Installing mod_cluster
The mod_cluster module is included in the Apache HTTP Server part of a JBoss Core Services installation.
Follow the procedures in the JBoss Core Services Installation Guide to download and install Apache HTTP Server for your operating system.
3.3. Configuring Load Balancing Using Apache HTTP Server and mod_cluster
In Apache HTTP Server 2.1 and higher, mod_cluster is configured correctly for Apache HTTP Server by default. To set a custom configuration, see Configuring a Basic Proxy Server.
For more information on configuring a Tomcat worker node with mod_cluster, see Configuring Worker Nodes.
Red Hat customers can also use the Load Balancer Configuration Tool on the RedHat Customer Portal to quickly generate optimal configuration templates for mod_cluster, as well as Tomcat worker nodes.
When using this tool for Apache HTTP Server 2.4.29, ensure you select
2.4.x as the Apache version, and select
Tomcat as the back-end configuration.
3.3.1. Configuring a Basic Proxy Server
Proxy server configuration consists of one mandatory and one optional step:
- Configure a Proxy Server listener to receive worker node connection requests and worker node feedback.
- Optional: Disable server advertisement.
The proxy server advertises itself using UDP multicast. When UDP multicast is available between the proxy server and the worker nodes, server advertisement adds worker nodes without requiring further configuration on the proxy server, and requires only minimal configuration on the worker nodes.
If UDP multicast is not available or undesirable, configure the worker nodes with a static list of proxy servers. In either case, the proxy server does not need to be configured with a list of worker nodes.
184.108.40.206. Configuring a Load-balancing Proxy Using mod_cluster
- Install Apache HTTP Server, and configure the mod_cluster modules for your installation. See the Apache HTTP Server Installation Guide for details.
To configure the load-balancing proxy using mod_cluster a Virtual Host for the management channel must be configured:
This address and port combination is only for mod_cluster management messages, not general traffic.
Create a Listen directive for the proxy server:
Edit your mod_cluster configuration file (usually
JBCS_HOME/httpd/conf.d/mod_cluster.conf) to add the following:
IP_ADDRESSis the address of the server network interface to communicate with the worker nodes, and
PORT_NUMBERis the port on that interface to listen on.Note
The port must be open for incoming TCP connections.
Create a virtual host:
Add the following to your mod_cluster configuration file:
<VirtualHost IP_ADDRESS:PORT_NUMBER> <Directory /> Require ip IP_ADDRESS </Directory> KeepAliveTimeout 60 MaxKeepAliveRequests 0 ManagerBalancerName mycluster AdvertiseFrequency 5 EnableMCPMReceive On </VirtualHost>
PORT_NUMBERare the values from the
Optional: Disable server advertisement:
AdvertiseFrequencydirective makes the server periodically send server advertisement messages via UDP multicast. By default, this occurs every 10 seconds.
These server advertisement messages contain the
PORT_NUMBERspecified in the
VirtualHostdefinition. Worker nodes configured to respond to server advertisements use this information to register themselves with the proxy server.
To disable server advertisement, add the following directive to the
If server advertisements are disabled, or UDP multicast is not available on the network between the proxy server and the worker nodes, configure worker nodes with a static list of proxy servers.
Optional: Configure Apache HTTP Server Logging
You can configure the Apache HTTP Server that is doing the load balancing to log which worker node handled a request. This may be useful when troubleshooting your load balancer.
To enable this for mod_cluster, you can add the following to your Apache HTTP Server
- The name of the balancer that served the request.
- The name of the worker node that served the request.
For more information on Apache HTTP Server logging (including log rotation), see http://httpd.apache.org/docs/2.4/logs.html.
Stop and start the Apache HTTP Server service:
See the JBoss Core Services Installation Guide for detailed instructions.
3.3.2. Configuring Worker Nodes
220.127.116.11. Configuring a Tomcat Worker Node
Follow this procedure to install mod_cluster on a Apache HTTP Server node, and configure it for non-clustered operation.
Apache HTTP Server Tomcat worker nodes only support a subset of mod_cluster functionality. Full mod_cluster functionality is available with JBoss EAP.
Supported Worker Node types
- Apache HTTP Server Tomcat service.
mod_cluster Apache HTTP Server Node Limitations
- Non-clustered mode only.
- Only one load metric can be used at a time when calculating the load balance factor.
- Install a supported instance of Apache HTTP Server.
- Understand the proxy configuration parameters.
To configure a Tomcat worker node:
Add a listener to Tomcat:
Add the following
Listenerelement beneath the other
<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="true" stickySession="true" stickySessionForce="false" stickySessionRemove="true" />
Give the worker a unique identity:
JWS_HOME/tomcat<VERSION>/conf/server.xmland add the
jvmRouteattribute and value to the
Engineelement, as shown below:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="worker01">
STATUS MCMPmessage frequency:
Tomcat worker nodes periodically send status messages containing their current load status to the Apache HTTP Server balancer. The default frequency of these messages is 10 seconds. If you have hundreds of worker nodes, the
STATUS MCMPmessages can increase traffic congestion on your Apache HTTP Server network.
You can configure the
MCMPmessage frequency by modifying the
org.jboss.modcluster.container.catalina.status-frequencyJava system property. By default, the property accepts values in seconds*10. For example, setting the property to
1means 10 seconds. The example below demonstrates setting the frequency to 60 seconds.
Optional: Configure the firewall for proxy server advertisements:
A proxy server using mod_cluster can advertise itself via UDP multicast. Most operating system firewalls block this by default. To enable server advertising and receive these multicast messages, open port
23364for UDP connections on the worker node’s firewall.
For Red Hat Enterprise Linux 6:
/sbin/iptables -A INPUT -m state --state NEW -m udp -p udp --dport 23364 -j ACCEPT -m comment -comment receive mod_cluster proxy server advertisements
If automatic proxy discovery is not used, configure worker nodes with a static list of proxies. In this case you can safely ignore the following warning message:
[warning] mod_advertise: ServerAdvertise Address or Port not defined, Advertise disabled!!!
For Red Hat Enterprise Linux 7:
firewall-cmd --permanent --zone=public --add-port=23364/udp
For Microsoft Windows, using PowerShell
Start-Process "$psHome\powershell.exe" -Verb Runas -ArgumentList '-command "NetSh Advfirewall firewall add rule name="UDP Port 23364" dir=in action=allow protocol=UDP localport=23364"' Start-Process "$psHome\powershell.exe" -Verb Runas -ArgumentList '-command "NetSh Advfirewall firewall add rule name="UDP Port 23364" dir=out action=allow protocol=UDP localport=23364"'
18.104.22.168. Configuring a Worker Node with a Static Proxy List
Server advertisement allows worker nodes to dynamically discover and register themselves with proxy servers. If UDP multicast is not available or server advertisement is disabled, then worker nodes must be configured with a static list of proxy server addresses and ports.
Use the following procedure to configure a Apache HTTP Server worker node to operate with a static list of proxy servers.
To configure a worker node with a static proxy list:
Define a mod_cluster Listener, and disable dynamic proxy discovery:
JWS_HOME/tomcat<VERSION>/conf/server.xml, and add or change the
ModClusterListener. Set the
false. For example:
<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="false" stickySession="true" stickySessionForce="false" stickySessionRemove="true"/>
Create a static proxy server list:
Add a comma separated list of proxies to the Listener in the form of
proxyListproperty. For example:
<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="false" stickySession="true" stickySessionForce="false" stickySessionRemove="true" proxyList="10.33.144.3:6666,10.33.144.1:6666"/>