-
Language:
English
-
Language:
English
Red Hat Training
A Red Hat training course is available for Red Hat JBoss Web Server
HTTP Connectors Load Balancing Guide
HTTP load balancing for JBoss Enterprise Application Platform and Red Hat JBoss Web Server 2.1.0
Abstract
Part I. Apache Tomcat Connector
Chapter 1. Overview
1.1. Overview
1.2. Download and Install
mod_jk
is included in the native installation binaries for JBoss Enterprise Application Platform and JBoss Enterprise Web Server.
Chapter 2. Configure Load Balancing Using Apache and mod_jk
2.1. Configure Load Balancing Using Apache and mod_jk
Procedure 2.1. Configure Apache to Load mod_jk
- Open
HTTPD_DIST/conf/httpd.conf
and add a single line at the end of the file.# Include mod_jk's specific configuration file Include conf/mod-jk.conf
- Create a new file named
HTTPD_DIST/conf/mod-jk.conf
- Add the following configuration to the
mod-jk.conf
file.Important
TheLoadModule
directive must reference the mod_jk library directory location applicable to the native binary you installed.Note
TheJkMount
directive specifies which URLs Apache should forward to the mod_jk module. Based on the directive's configuration, mod_jk forwards the received URL onto the correct Servlet containers.To enable Apache to serve static content (or PHP content) directly, and only use the load balancer for Java applications, the suggested configuration specifies all requests with URL path/application/*
are sent to the mod_jk load-balancer.Only mod_jk is used as a load balancer, forward all URLs to mod_jk by specifying/*
in the directive.# Load mod_jk module # Specify the filename of the mod_jk lib LoadModule jk_module modules/mod_jk.so # Where to find workers.properties JkWorkersFile conf/workers.properties # Where to put jk logs JkLogFile logs/mod_jk.log # Set the jk log level [debug/error/info] JkLogLevel info # Select the log format JkLogStampFormat "[%a %b %d %H:%M:%S %Y]" # JkOptions indicates to send SSK KEY SIZE JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories # JkRequestLogFormat JkRequestLogFormat "%w %V %T" # Mount your applications JkMount /application/* loadbalancer # Add shared memory. # This directive is present with 1.2.10 and # later versions of mod_jk, and is needed for # for load balancing to work properly JkShmFile logs/jk.shm # Add jkstatus for managing runtime data <Location /jkstatus/> JkMount status Order deny,allow Deny from all Allow from 127.0.0.1 </Location>
Optional: JKMountFile Directive
In addition to theJkMount
directive, use theJkMountFile
directive to specify a mount points configuration file. The configuration file contains multiple Tomcat forwarding URL mappings.- Navigate to
HTTPD_DIST/conf
. - Create a file named
uriworkermap.properties
. - Specify the URL to forward and the worker name using the following syntax example as a guide.The example block will configure mod_jk to forward requests to
/jmx-console
and/web-console
to Apache.The syntax required takes the form/url=worker_name
.# Simple worker configuration file # Mount the Servlet context to the ajp13 worker /jmx-console=loadbalancer /jmx-console/*=loadbalancer /web-console=loadbalancer /web-console/*=loadbalancer
- In
HTTPD_DIST/conf/mod-jk.conf
, append the following directive.# Use external file for mount points. # It will be checked for updates each 60 seconds. # The format of the file is: /url=worker # /examples/*=loadbalancer JkMountFile conf/uriworkermap.properties
2.2. Configuring Worker Nodes in mod_jk
2.2.1. Configuring Worker Nodes in mod_jk
Procedure 2.2. Configure mod_jk Worker Nodes
workers.properties
directives, as specified in Section A.1, “workers.properties” and configure mod_jk (see Section 2.1, “Configure Load Balancing Using Apache and mod_jk”).
- Navigate to
HTTPD_DIST/conf/
. - Create a file named
workers.properties
. - Append the following information into the
workers.properties
file.# Define list of workers that will be used # for mapping requests worker.list=loadbalancer,status # Define Node1 # modify the host as your host IP or DNS name. worker.node1.port=8009 worker.node1.host=node1.mydomain.com worker.node1.type=ajp13 worker.node1.ping_mode=A worker.node1.lbfactor=1 # Define Node2 # modify the host as your host IP or DNS name. worker.node2.port=8009 worker.node2.host=node2.mydomain.com worker.node2.type=ajp13 worker.node2.ping_mode=A worker.node2.lbfactor=1 # Load-balancing behavior worker.loadbalancer.type=lb worker.loadbalancer.balance_workers=node1,node2 worker.loadbalancer.sticky_session=1 # Status worker for managing load balancer worker.status.type=status
2.3. Configuring Apache Tomcat to Work with mod_jk
conf/
server.xml
file, which contains the following code for this purpose:
<connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
jvmRoute
attribute at your engine as follows:
<Engine name="Catalina" jvmRoute="node1" >
jvmRoute
attribute value must match the worker name set in workers.properties
.
Chapter 3. WebSocket on Tomcat
3.1. About WebSocket
3.2. Implementing WebSocket on Tomcat
- Configuring write timeout
- Configuring incoming binary messages
- Configuring incoming text messages
- Configuring additional programmatic deployment
- Configuring callbacks for asynchronous writes
- Configuring timeout for IO operations while establishing the connections
You can change the write timeout in blocking mode by using the org.apache.tomcat.websocket.BLOCKING_SEND_TIMEOUT
property. The property accepts values in milliseconds. The default value is 20000 milliseconds (20 seconds).
To configure incoming binary messages, MessageHandler.Partial
must be defined. If MessageHandler.Partial
is not defined then incoming binary messages must be buffered so that the entire message is delivered in a single call to MessageHandler.Whole
.
org.apache.tomcat.websocket.binaryBufferSize
.
To configure incoming text messages, MessageHandler.Partial
must be defined. If MessageHandler.Partial
is not defined then incoming text messages must be buffered so that the entire message is delivered in a single call to MessageHandler.Whole
.
org.apache.tomcat.websocket.textBufferSize
.
Java WebSocket specification 1.0 does not allow programmatic deployment after the first endpoint has started a WebSocket handshake. However, Tomcat by default allows additional programmatic deployment. Additional programmatic deployment can be done by using the servlet context initialization parameter org.apache.tomcat.websocket.noAddAfterHandshake
.
org.apache.tomcat.websocket.STRICT_SPEC_COMPLIANCE
to true to change the default setting.
Callbacks for asynchronous writes need to be performed on a different thread to the thread that initiated the write. The container thread pool is not exposed via the Servlet API. Hence the WebSocket implementation has to provide its own thread pool.
org.apache.tomcat.websocket.executorCoreSize
The core size of the executor thread pool. If not set, the default of 0 (zero) is used.org.apache.tomcat.websocket.executorMaxSize
The maximum permitted size of the executor thread pool. If not set, the default of 10 is used.org.apache.tomcat.websocket.executorKeepAliveTimeSeconds
The maximum time an idle thread will remain in the executor thread pool until it is terminated. If not specified, the default of 60 seconds is used.
The timeout for IO operations while establishing the connections is controlled by userProperties
of the provided javax.websocket.ClientEndpointConfig
. You can change timeout by changing the org.apache.tomcat.websocket.IO_TIMEOUT_MS
property. The property accepts the values in milliseconds. The default value is 5000 (5 seconds).
userProperties
of the provided javax.websocket.ClientEndpointConfig
.
org.apache.tomcat.websocket.SSL_CONTEXT
org.apache.tomcat.websocket.SSL_PROTOCOLS
org.apache.tomcat.websocket.SSL_TRUSTSTORE
org.apache.tomcat.websocket.SSL_TRUSTSTORE_PWD
org.apache.tomcat.websocket.SSL_TRUSTSTORE
and org.apache.tomcat.websocket.SSL_TRUSTSTORE_PWD
properties are ignored if the org.apache.tomcat.websocket.SSL_CONTEXT
property is set.
Part II. JBoss HTTP Connector
Chapter 4. Overview
4.1. Overview
4.2. Key Features
Table 4.1. Features
Feature | Description |
---|---|
Apache HTTP Server-based | The JBoss HTTP Connector mod-cluster uses Apache as the proxy server. |
Real-time load-balancing calculation |
The JBoss HTTP Connector mod_cluster creates a feedback network between the worker nodes and the proxy server. The mod_cluster service is deployed on each of the worker nodes. This service feeds real time load information to the proxy server. The proxy server then allocates work, based on the current load on each worker node. This real time adaptive load distribution results in increased optimization of resources.
The information reported by the worker nodes and the load-balancing policy used by the proxy are customizable.
|
Routing based on real-time application life cycle | The JBoss HTTP Connector mod_cluster service deployed on the worker nodes relays application lifecycle events to the proxy server. This allows the server to dynamically update its routing table. When an application is undeployed on a node, the proxy server does notroute traffic for that application to that node |
Automatic Proxy Discovery |
The proxy server is configured to announce its presence via UDP multicast. New worker nodes discover the proxy server and automatically add themselves to the load-balancing cluster. This reduces the configuration and maintenance needed. When UDP multicast is not available or is undesirable, worker nodes are configured with a static list of proxies.
|
Multiple Protocol Support | The JBoss HTTP Connector mod_cluster uses either HTTP, HTTPS, or Apache JServ Protocol (AJP) for communication between the proxy and the worker nodes. |
4.3. Components
Table 4.2. Components
Component | Description |
---|---|
mod_slotmem.so | The Shared Memory Manager module shares the real time worker node information with multiple Apache server processes |
mod_manager.so |
The Cluster Manager module receives and acknowledges messages from nodes, including worker node registrations, worker node load data, and worker node application life cycle events
|
mod_proxy_cluster.so | The Proxy Balancer Module handles request routing to cluster nodes. The Proxy Balancer selects the appropriate destination node based on application location in the cluster, current state of each of the cluster nodes, and the Session ID (if a request is part of an established session). |
mod_advertise.so |
The Proxy Advertisement Module broadcasts the existence of the proxy server via UDP multicast messages. The server advertisement messages contain the IP address and port number where the proxy is listening for responses from nodes that want to join the load-balancing cluster.
|
mod_cluster
offered as part of JBoss Enterprise Web Server is used instead of a JBoss HTTP Connector client service for the worker node service. The mod_cluster in JBoss Enterprise Web Server consists of the following two parts:
- Native - Apache HTTP Server module (balancer logic)
- Java - Tomcat6/7 Listener library (worker logic)
4.4. Limitations
mod_cluster
uses shared memory to keep the nodes description, the shared memory is created at the startup of httpd
and the structure of each item is fixed. Therefore, when defining proxy server and worker node properties, make sure to follow these character limits:
- Maximum Alias length: 100 characters (Alias corresponds to the network name of the respective virtual host; the name is defined in the Host element)
- Maximum context length: 40 characters (for example, if
myapp.war
is deployed in/myapp
, then/myapp
is the context) - Maximum balancer name length: 40 characters (the balancer property in mbean)
- Maximum JVMRoute string length: 80 character (JVMRoute in the <Engine> element)
- Maximum domain name length: 20 characters (the domain property in mbean)
- Maximum hostname length for a node: 64 characters (hostname address in the <Connector> element)
- Maximum port length for a node: 7 characters (8009 is 4 characters, the port property in the <Connector> element)
- Maximum scheme length for a node: 6 characters (possible values are
http
,https
,ajp
, the protocol of the connector) - Maximum cookie name length: 30 characters (the header cookie name for session ID default value:
JSESSIONID
fromorg.apache.catalina.Globals.SESSION_COOKIE_NAME
) - Maximum path name length: 30 characters (the parameter name for the session ID default value:
JSESSIONID
fromorg.apache.catalina.Globals.SESSION_PARAMETER_NAME
) - Maximum length of a session ID: 120 characters (session ID resembles the following:
BE81FAA969BF64C8EC2B6600457EAAAA.node01
)
Chapter 5. Proxy Server Components
5.1. Apache Modules
5.1.1. Apache Modules
5.1.2. mod_manager.so
LoadModule manager_module modules/mod_manager.so
<VirtualHost>
element are as follows:
- EnableMCPMReceive
- Allows the
VirtualHost
to receive the mod_cluster Protocol Message (MCPM) from nodes. Add oneEnableMCPMRecieve
attribute to the httpd configuration to allowmod_cluster
to operate correctly.EnableMCPMRecieve
must be added in theVirtualHost
configuration, at the location whereadvertise
is configured. - MaxMCMPMaxMessSize
- Defines the maximum size of mod_cluster Management Protocol (MCMP) messages. The default value for this is calculate from other
Max
directives. The minumum value for this is1024
. - AllowDisplay
- Toggles the additional display on the
mod_cluster-manager
main page. The default value isoff
which causes only the versions to display on themod_cluster-manager
main page. - AllowCmd
- Toggles permissions for commands using
mod_cluster-manager
URL. The default value ison
, which allows commands. - ReduceDisplay
- Toggles the reduction of information displayed on the
mod_cluster-manager
page. Reducing the information allows more nodes to display on the page. The default value isoff
which allows all the available information to display. - MemManagerFile
- Defines the location for the files in which mod_manager stores configuration details. mod_manager also uses this location for generated keys for shared memory and lock files. This must be an absolute path name. It is recommended that this path be on a local drive, and not a NFS share. The default value is
/logs/
. - Maxcontext
- The maximum number of contexts JBoss mod_cluster will use. The default value is
100
. - Maxnode
- The maximum number of worker nodes JBoss mod_cluster will use. The default value is
20
. - Maxhost
- The maximum number of hosts (aliases) JBoss mod_cluster will use. This is also the maximum number of load balancers. The default value is
10
. - Maxsessionid
- The maximum number of active session identifiers stored. A session is considered inactive when no information is received from that session within five minutes. The default value is
0
, which disables this logic. - ManagerBalancerName
- The name of the load balancer to use when the worker node does not provide a load balancer name. The default value is
mycluster
. - PersistSlots
- When set to
on
, nodes, aliases and contexts are persisted in files. The default value isoff
. - CheckNonce
- When set to
on
, session identifiers are checked to ensure that they are unique, and have not occurred before. The default ison
.Warning
Setting this directive tooff
can leave your server vulnerable to replay attacks. - SetHandler mod_cluster-manager
- Defines a handler to display information about worker nodes in the cluster. This is defined in the
Location
element:<Location $LOCATION> SetHandler mod_cluster-manager Order deny,allow Deny from all Allow from 127.0.0.1 </Location>
When accessing the $LOCATION defined in theLocation
element in your browser, you will see something like the following. (In this case, $LOCATION was also defined asmod_cluster-handler
.)
Maxsessionid
is 0
.
5.1.3. mod_proxy_cluster.so
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
<VirtualHost>
element to change load balancing behavior.
mod_proxy_cluster directives
- CreateBalancers
- Defines how load balancers are created in the Apache HTTP Server virtual hosts. The following values are valid in
CreateBalancers
:- 0
- Create load balancers in all virtual hosts defined in Apache HTTP Server. Remember to configure the load balancers in the
ProxyPass
directive. - 1
- Do not create balancers. When using this value, you must also define the load balancer name in the
ProxyPass
orProxyPassMatch
. - 2
- Create only the main server. This is the default value for
CreateBalancers
.
- UseAlias
- Defines whether to check that the defined
Alias
corresponds to theServerName
. The following values are valid forUseAlias
:- 0
- Ignore Alias information from worker nodes. This is the default value for
UseAlias
. - 1
- Verify that the defined alias corresponds to a worker node's server name.
- LBstatusRecalTime
- Defines the interval in seconds between the proxy calculating the status of a worker node. The default interval is 5 seconds.
- ProxyPassMatch; ProxyPass
ProxyPass
maps remote servers into the local server namespace. If the local server has an addresshttp://local.com/
, then the followingProxyPass
directive would convert a local request forhttp://local.com/requested/file1
into a proxy request forhttp://worker.local.com/file1
.ProxyPass /requested/ http://worker.local.com/
ProxyPassMatch
uses Regular Expressions to match local paths to which the proxied URL should apply.For either directive,!
indicates that a specified path is local, and a request for that path should not be routed to a remote server. For example, the following directive specifies that.gif
files should be served locally.ProxyPassMatch ^(/.*\.gif)$ !
5.1.4. mod_advertise.so
VirtualHost
element. Its identifier in the following code snippet is advertise_module
.
LoadModule advertise_module modules/mod_advertise.so
- ServerAdvertise
- Defines how the advertising mechanism is used.When set to
On
, the advertising mechanism is used to tell worker nodes to send status information to this proxy. You can also specify a hostname and port with the following syntax:ServerAdvertise On http://hostname:port/
. This is only required when using a name-based virtual host, or when a virtual host is not defined.The default value isOff
. When set tooff
, the proxy does not advertise its location. - AdvertiseGroup
- Defines the multicast address to advertise on. The syntax is
AdvertiseGroup address:port
, where address should correspond toAdvertiseGroupAddress
, and port should correspond toAdvertisePort
in your worker nodes.If your worker node is JBoss Enterprise Application Platform-based, and the-u
switch is used at startup, the defaultAdvertiseGroupAddress
is the value passed via the-u
switch.The default value is224.0.1.105:23364
. If port is not specified, the default port specified is23364
. - AdvertiseFrequency
- The interval (in seconds) between multicast messages advertising the IP address and port. The default value is
10
. - AdvertiseSecurityKey
- Defines a string used to identify the JBoss HTTP Connector mod_cluster in JBoss Web. By default this directive is not set and no information is sent.
- AdvertiseManagerUrl
- Defines the URL that the worker node should use to send information to the proxy server. By default this directive is not set and no information is sent.
- AdvertiseBindAddress
- Defines the address and port over which to send multicast messages. The syntax is
AdvertiseBindAddress address:port
. This allows an address to be specified on machines with multiple IP addresses. The default value is0.0.0.0:23364
.
5.1.5. mod_proxy.so
proxy_module
.
Mod_proxy
directives such as ProxyIOBufferSize
are used to configure mod_cluster
.
5.1.6. mod_proxy_ajp.so
Mod_proxy.so
is required to use this module.
5.2. Proxy Server Components Installation and Default Configuration
5.2.1. Proxy Server Components Installation and Default Configuration
Chapter 6. Configure Basic Proxy Server
6.1. Basic Proxy Configuration Overview
- Configure a Proxy Server listener to receive worker node connection requests and worker node feedback.
- Optional: Disable server advertisement.
The proxy server advertises itself using UDP multicast. When UDP multicast is available on the network between the proxy server and the worker nodes, Server Advertisement adds worker nodes without further configuration required on the proxy server, and minimal configuration on the worker nodes.
6.2. Configure a Load-balancing Proxy Using the HTTP Connector
The following are a series of prerequisites for this procedure:
- Install JBoss Enterprise Web Server. See JBoss Enterprise Web Server Installation Guide for details.
- Install JBoss HTTP Connector modules. See Section 1.2, “Download and Install” for details.
Procedure 6.1. Configure a Load-balancing Proxy Using the HTTP Connector
Create a Listen Directive for the Proxy Server
Edit the configuration fileJBOSS_EWS_DIST/httpd/conf.d/JBoss_HTTP.conf
and add the following:Listen IP_ADDRESS:PORT_NUMBER
Where IP_ADDRESS is the IP address of a server network interface to communicate with the worker nodes, and PORT_NUMBER is the port on that interface to listen on.Note
The port PORT_NUMBER must be open on the server firewall for incoming TCP connections.Example 6.1. Example Listen Directive
Listen 10.33.144.3:6666
Create Virtual Host
Add the following to the fileJBOSS_EWS_DIST/httpd/conf.d/JBoss_HTTP.conf
:<VirtualHost IP_ADDRESS:PORT_NUMBER> <Directory /> Order deny,allow Deny from all Allow from 10.33.144. </Directory> KeepAliveTimeout 60 MaxKeepAliveRequests 0 ManagerBalancerName mycluster AdvertiseFrequency 5 EnableMCPMReceive On </VirtualHost>
Where IP_ADDRESS and PORT_NUMBER are the values from the Listen directive.Optional: Disable Server Advertisement
TheAdvertiseFrequency
directive, set to five seconds, makes the server to periodically send server advertisement messages via UDP multicast.These server advertisement messages contain the IP_ADDRESS and PORT_NUMBER specified in the VirtualHost definition. Worker nodes configured to respond to server advertisements use this information to register themselves with the proxy server.To disable server advertisement, add the following directive to theVirtualHost
definition:ServerAdvertise Off
If server advertisements are disabled, or UDP multicast is not available on the network between the proxy server and the worker nodes, configure worker nodes with a static list of proxy servers. See Section 8.1, “Static Proxy Configuration” for directions.Restart the JBoss Enterprise Web Server Apache service
See the JBoss Enterprise Web Server documentation for detailed directions.
Chapter 7. Install Node with Basic Configuration
7.1. Worker Node Requirements
Supported Worker Node types
- JBoss Enterprise Web Server Tomcat service
Note
JBoss HTTP Connector Enterprise Web Server Node Limitations
- Non-clustered mode only.
- Only one load metric can be used at a time when calculating the load balance factor.
7.2. Install and Configure a Worker Node
7.2.1. Configure a Tomcat Worker Node
Note
The following is a list of prerequisites for this task:
- Install a supported JBoss Enterprise Web Server.
- Understand the Proxy Configuration parameters discussed in Appendix B: Java Properties Reference.
Procedure 7.1. Configure a Worker Node for Tomcat
Add a Listener to Tomcat
Add the followingListener
element beneath the other Listener elements inJBOSS_EWS_DIST/tomcat6/conf/server.xml
.<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="true" stickySession="true" stickySessionForce="false" stickySessionRemove="true" />
Give the Worker a Unique Identity
EditJBOSS_EWS_DIST/tomcat6/conf/server.xml
and add ajvmRoute
attribute and value to theEngine
element, as shown:<Engine name="Catalina" defaultHost="localhost" jvmRoute="worker01">
Optional Step: Configure Firewall for Proxy Server Advertisements
A proxy server using the JBoss HTTP Connector can advertise itself via UDP multicast. To receive these multicast messages, open port 23364 for UDP connections on the worker node's firewall.For Linux Users/sbin/iptables -A INPUT -m state --state NEW -m udp -p udp --dport 23364 -j ACCEPT -m comment -comment "receive mod_cluster proxy server advertisements"
If Automatic Proxy Discovery is not used, configure worker nodes with a static list of proxies. In this case you can safely ignore the following warning message:[warning] mod_advertise: ServerAdvertise Address or Port not defined, Advertise disabled!!!
Set STATUS MCMP Messages frequency
The tomcat worker nodes send status messages periodically to Apache HTTP Server balancer containing their current load status. The default frequency of these messages is 10 seconds. With many active worker nodes which are in hundreds, the STATUS MCMP Messages might increase the traffic congestion on Apache HTTP Server network.To set the MCMP message frequency, modify theorg.jboss.modcluster.container.catalina.status-frequency
property. By default, the property accepts values in seconds*10. Example, value = 1 means 10 seconds.-Dorg.jboss.modcluster.container.catalina.status-frequency=6
Procedure 7.2. Configuring Firewall on Microsoft Windows using PowerShell
- Switch off firewall for debug purpose to determine whether the current network behavior is related to the firewall configuration.
Start-Process "$psHome\powershell.exe" -Verb Runas -ArgumentList '-command "NetSh Advfirewall set allprofiles state off"'
- Allow UDP connections on port 23364. For example:
Start-Process "$psHome\powershell.exe" -Verb Runas -ArgumentList '-command "NetSh Advfirewall firewall add rule name="UDP Port 23364" dir=in action=allow protocol=UDP localport=23364"' Start-Process "$psHome\powershell.exe" -Verb Runas -ArgumentList '-command "NetSh Advfirewall firewall add rule name="UDP Port 23364" dir=out action=allow protocol=UDP localport=23364"'
Procedure 7.3. Configure the Firewall on Red Hat Enterprise Linux 7 to Allow mod_cluster Advertising
- To allow mod_cluster advertising on Red Hat Enterprise Linux 7, you must enable the UDP port in the firewal as follows:
firewall-cmd --permanent --zone=public --add-port=23364/udp
Note
224.0.1.105:23364 is the default address and port for mod_cluster balancer advertising UDP multicast.
Chapter 8. Advanced Configuration
8.1. Static Proxy Configuration
Ensure that the following prerequisites are satisfied before initiating the procedure.
- JBoss Enterprise Web Server worker node configured. See Section 7.1, “Worker Node Requirements” for directions.
- Understand the Proxy Configuration parameters discussed in the Java Properties Reference Appendix.
Procedure 8.1. Configure Web Server Worker Node with Static Proxy List
Disable Dynamic Proxy Discovery
Edit the fileJBOSS_EWS_DIST/tomcat6/conf/server.xml
. and set theadvertise
property of the ModClusterListener to false:Define a mod_cluster listener
Add a <Listener> element to theserver.xml
file.<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="false" stickySession="true" stickySessionForce="false" stickySessionRemove="true"/>
Create a static proxy server list
Add a comma separated list of proxies in the form of IP_ADDRESS:PORT as theproxyList
property of the ModClusterListener <Listener> element.Example 8.1. Example Static Proxy List
<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="false" stickySession="true" stickySessionForce="false" stickySessionRemove="true" proxyList="10.33.144.3:6666,10.33.144.1:6666"/>
Chapter 9. Configuring httpd for SSL connections
9.1. Configuring httpd for SSL connections
Procedure 9.1. To configure httpd for ssl connections:
- Install mod_ssl using the following command:
# rpm -qa | grep mod_ssl
- Edit the
HTTPD_HOME/conf.d/ssl.conf
file and addServerName
,SSLCertificateFile
, andSSLCertificateKeyFile
Example SSL configuration <VirtualHost _default_:443> #ServerName www.example.com:443 SSLCertificateFile /etc/pki/tls/certs/localhost.crt SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
ServerName
must match the Common Name (CN) of the SSL certificate. If theServerName
does not match the CN, the client browsers display a message "domain mismatch".- The
SSLCertificateFile
is the private key associate with the certificate (the public key). - Verify that the Listen directive in the
ssl.conf
file is correct as per your setup. For example, if an IP address is specified, it must match the IP address the httpd service is bound to.
- Restart httpd using the following command:
# service httpd restart
Part III. Online Certificate Status Protocol
Chapter 10. Online Certificate Status Protocol
10.1. About Online Certificate Status Protocol
10.2. Using Online Certificate Status Protocol for httpd
Note
[ usr_cert ] ... authorityInfoAccess=OCSP;URI:http://HOST:PORT ... [ v3_OCSP ] basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment extendedKeyUsage = OCSP Signing
10.3. Configure httpd to validate OCSP certificates
Procedure 10.1. To httpd to validate OCSP certificates
- Use the SSLOCSPEnable attribute to enable OCSP validation.
# Require valid client certificates (mutual auth) SSLVerifyClient require SSLVerifyDepth 3 # Enable OCSP SSLOCSPEnable on SSLOCSPDefaultResponder http://10.10.10.25:3456 SSLOCSPOverrideResponder on
10.4. Verify the configuration
# openssl ocsp -issuer cacert.crt -cert client.cert -url http://HOST:PORT -CA ocsp_ca.cert -VAfile ocsp.cert
Note
-issuer
is the Certificate Authority certificate.-cert
is the Client certificate which you want to verify.-url
is the http server validating Certificate (OCSP).-CA
is the CA certificate for verifying the httpd server certificate.-VAfile
is the OCSP Responder certificate.
Part IV. Using Jsvc with JBoss Enterprise Web Server
Chapter 11. Jsvc
11.1. About Jsvc
For Red Hat Enterprise Linux:
jboss-ews-2.1/extras/jsvc
jboss-ews-2.1/tomcat6/bin/jsvc
jboss-ews-2.1/tomcat7/bin/jsvc
Note
extra/jsvc
in the following way:
./jboss-ews-2.1/tomcat6/bin/jsvc -> ../../extras/jsvc
./jboss-ews-2.1/tomcat7/bin/jsvc -> ../../extras/jsvc
For Solaris:
./jboss-ews-2.1/sbin/jsvc
./jboss-ews-2.1/share/apache-tomcat-6.0.41/bin/jsvc
./jboss-ews-2.1/share/apache-tomcat-7.0.54/bin/jsvc
Note
sbin/jsvc
in the following way:
./jboss-ews-2.1/share/apache-tomcat-7.0.54/bin/jsvc -> ../../../sbin/jsvc
./jboss-ews-2.1/share/apache-tomcat-6.0.41/bin/jsvc -> ../../../sbin/jsvc
11.2. Use Jsvc with Tomcat 6 and 7
11.2.1. Run Jsvc with Tomcat 6 and 7
/opt/jboss-ews-2.1/share/tomcat<VERSION>/bin/daemon.sh start
11.2.2. Configure Jsvc with Tomcat 6 and 7
daemon.sh
script to run Jsvc:
Table 11.1. daemon.sh configuration parameters used by Tomcat 6 and 7
Parameter Name | Environment Variable | Default Value | Description |
---|---|---|---|
--java-home | JAVA_HOME | Based on the value of the PATH variable. | The Java home directory location. |
--catalina-home | CATALINA_HOME | Determined by the location of the script. | The Tomcat installation directory location. |
--catalina-base | CATALINA_BASE | Based on the value of the PATH variable. | The directory that contains the specific configuration and set up information if multiple servers are using the same installation. |
--catalina-pid | - | $CATALINA_BASE/logs/catalina-daemon.pid | The file where the process ID (PID) for the running instance of Tomcat is stored. |
--tomcat-user | - | tomcat | The user Tomcat uses. |
--service-start-wait-time | - | This is a wrapper to the --wait parameter. The --wait parameter accepts values in seconds. |
Chapter 12. Working Examples
12.1. Complete Working Example
A proxy server listening on localhost:
LoadModule slotmem_module modules/mod_slotmem.so LoadModule manager_module modules/mod_manager.so LoadModule proxy_cluster_module modules/mod_proxy_cluster.so LoadModule advertise_module modules/mod_advertise.so Listen 127.0.0.1:6666 <VirtualHost 127.0.0.1:6666> <Directory /> Order deny,allow Deny from all Allow from 127.0.0.1 </Directory> KeepAliveTimeout 60 MaxKeepAliveRequests 0 ManagerBalancerName mycluster ServerAdvertise On AdvertiseFrequency 5 EnableMCPMReceive On <Location /mod_cluster-manager> SetHandler mod_cluster-manager Order deny,allow Deny from all Allow from 127.0.0.1 </Location> </VirtualHost>
Edit the $CATALINA_HOME/conf/server.xml
file and add the following listener element to configure the worker for Tomcat:
<Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" advertise="true"/>
Following are a set of example firewall rules using iptables
, for a cluster node on the 192.168.1.0/24 subnet.
/sbin/iptables -I INPUT 5 -p udp -d 224.0.1.0/24 -j ACCEPT -m comment --comment "mod_cluster traffic" /sbin/iptables -I INPUT 6 -p udp -d 224.0.0.0/4 -j ACCEPT -m comment --comment "JBoss Cluster traffic" /sbin/iptables -I INPUT 9 -p udp -s 192.168.1.0/24 -j ACCEPT -m comment --comment "cluster subnet for inter-node communication" /sbin/iptables -I INPUT 10 -p tcp -s 192.168.1.0/24 -j ACCEPT -m comment --comment "cluster subnet for inter-node communication" /etc/init.d/iptables save
12.2. mod_auth_kerb Example
12.2.1. About the mod_auth_kerb Example
12.2.2. mod_auth_kerb Example Prerequisites
- Install mod_auth_kerb on Red Hat Enterprise Linux.
- Install curl with GSS-negotiated support.
- Configure and run a Kerberos or LDAP server (for example ApacheDS) on the same host as your JBoss Enterprise Web Server.
- Create the following LDAP users:
- Create the user
krbtgt
:dn: uid=krbtgt,ou=Users,dc=example,dc=com objectClass: top objectClass: person objectClass: inetOrgPerson objectClass: krb5principal objectClass: krb5kdcentry cn: KDC Service sn: Service uid: krbtgt userPassword: secret krb5PrincipalName: krbtgt/EXAMPLE.COM@EXAMPLE.COM krb5KeyVersionNumber: 0
- Create the user
ldap
:dn: uid=ldap,ou=Users,dc=example,dc=com objectClass: top objectClass: person objectClass: inetOrgPerson objectClass: krb5principal objectClass: krb5kdcentry cn: LDAP sn: Service uid: ldap userPassword: randall krb5PrincipalName: ldap/localhost@EXAMPLE.COM krb5KeyVersionNumber: 0
- Create the user
HTTP
:dn: uid=HTTP,ou=Users,dc=example,dc=com objectClass: top objectClass: person objectClass: inetOrgPerson objectClass: krb5principal objectClass: krb5kdcentry cn: HTTP sn: Service uid: HTTP userPassword: secretpwd krb5PrincipalName: HTTP/localhost@EXAMPLE.COM krb5KeyVersionNumber: 0
- Create user
hnelson
(test user):dn: uid=hnelson,ou=Users,dc=example,dc=com objectClass: top objectClass: person objectClass: inetOrgPerson objectClass: krb5principal objectClass: krb5kdcentry cn: Horatio Nelson sn: Nelson uid: hnelson userPassword: secret krb5PrincipalName: hnelson@EXAMPLE.COM krb5KeyVersionNumber: 0
12.2.3. Configure the Kerberos Client
Procedure 12.1. Configure the Kerberos Client
Create the Kerberos Configuration File
Create thekrb5.conf
configuration file in the/etc
directory and add the following to the file:[logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] default_realm = EXAMPLE.COM default_tgs_enctypes = des-cbc-md5,des3-cbc-sha1-kd default_tkt_enctypes = des-cbc-md5,des3-cbc-sha1-kd dns_lookup_realm = false dns_lookup_kdc = false allow_weak_crypto = yes ticket_lifetime = 24h renew_lifetime = 7d forwardable = yes [realms] EXAMPLE.COM = { kdc = localhost:60088 admin_server = localhost:60088 } [domain_realm] .example.com = EXAMPLE.COM example.com = EXAMPLE.COM
Create a Key Tab
Create a key tab in the/etc/httpd
folder with the following contents:ktutil ktutil: addent -password -p HTTP/localhost@EXAMPLE.COM -k 0 -e des-cbc-md5 Password for HTTP/localhost@EXAMPLE.COM: secretpwd ktutil: list slot KVNO Principal ---- ---- --------------------------------------------------------------------- 1 0 HTTP/localhost@EXAMPLE.COM ktutil: wkt krb5.keytab ktutil: quit Under root user: chgrp apache /etc/httpd/krb5.keytab chmod 640 /etc/httpd/krb5.keytab
Check the Hosts File
Ensure that the following host configuration is included in the/etc/
hosts
file:127.0.0.1 localhost
12.2.4. Configure mod_auth_kerb
Procedure 12.2. Configure mod_auth_kerb
- Create the
auth_kerb.conf
configuration file in the/etc/httpd/conf.d/
folder and add the following information to the file:# # The mod_auth_kerb module implements Kerberos authentication over # HTTP, following the "Negotiate" protocol. # LoadModule auth_kerb_module modules/mod_auth_kerb.so <Location /kerberostest> # SSLRequireSSL AuthType Kerberos AuthName "Kerberos Login" KrbMethodNegotiate On KrbMethodK5Passwd Off KrbAuthRealms EXAMPLE.COM KrbServiceName HTTP Krb5KeyTab /etc/httpd/krb5.keytab require valid-user </Location>
12.2.5. Test the Kerberos Authentication
Procedure 12.3. Test the Kerberos Authentication
Create a Test Page
Create a test page namedauth_kerb_page.html
in the$EWS_HOME/httpd/www/html/kerberostest/
.Add the Contents of the Test Page
Add the following contents to the test page (auth_kerb_page.html
:<html> <body> <h1>mod_auth_kerb successfully authenticated!</h1> </body> </html>
Optional: Set Log Level
Optionally, set the log level for debugging in the$EWS_HOME/httpd/conf/httpd.conf
file.Start httpd
As the root user, start the JBoss Enterprise Web Server httpd as follows:# $EWS_HOME/httpd/sbin/apachectl start
Test Authentication
Test the authentication as follows:- Initiate Kerberos authentication for the test user
hnelson
:$ kinit hnelson
- View the details for the test user
hnelson
:$ klist
A result similar to the following appears:Ticket cache: FILE:/tmp/krb5cc_18602 Default principal: hnelson@EXAMPLE.COM Valid starting Expires Service principal 06/03/13 14:21:13 06/04/13 14:21:13 krbtgt/EXAMPLE.COM@EXAMPLE.COM renew until 06/10/13 14:21:13
Testing httpd Kerberos Authentication
Test httpd Kerberos authentication as follows:$ curl --negotiate -u : http://localhost/kerberostest/auth_kerb_page.html
If working correctly, the following result appears:<html> <body> <h1>mod_auth_kerb successfully authenticated!</h1> </body> </html>
Appendix A. Reference
A.1. workers.properties
HTTPD_DIST/conf/workers.properties
. This file specifies where the different Servlet containers are located, and how calls should be load-balanced across them.
workers.properties
file contains two sections:
- Global Properties
- This section contains directives that apply to all workers.
- Worker Properties
- This section contains directives that apply to each individual worker.
worker.worker_name.directive
- worker
- The constant prefix for all worker properties.
- worker_name
- The arbitrary name given to the worker. For example: node1, node_01, Node_1.
- directive
- The specific directive required.
Note
worker.properties
configuration directives, refer directly to the Apache Tomcat Connector - Reference Guide
worker.properties Global Directives
- worker.list
- Specifies the list of worker names used by mod_jk. The workers in this list are available to map requests to.
Note
A single node configuration, which is not managed by a load balancer, must be set toworker.list=[worker name]
.
workers.properties Mandatory Directives
- type
- Specifies the type of worker, which determines the directives applicable to the worker. The default value is
ajp13
, which is the preferred worker type to select for communication between the web server and Apache httpd Server.Other values includeajp14
,lb
,status
.For detailed information about ajp13, see The Apache Tomcat Connector - AJP Protocol Reference
workers.properties Connection Directives
- host
- The hostname or IP address of the worker. The worker node must support the ajp13 protocol stack. The default value is
localhost
.You can specify theport
directive as part of the host directive by appending the port number after the hostname or IP address. For example:worker.node1.host=192.168.2.1:8009
orworker.node1.host=node1.example.com:8009
- port
- The port number of the remote server instance listening for defined protocol requests. The default value is
8009
, which is the default listen port for AJP13 workers. If you are using AJP14 workers, this value must be set to8011
. - ping_mode
- Specifies the conditions under which connections are probed for their current network health.The probe uses an empty AJP13 packet for the CPing, and expects a CPong in return, within a specified timeout.You specify the conditions by using a combination of the directive flags. The flags are not comma-separated. For example, a correct directive flag set is
worker.node1.ping_mode=CI
- C (connect)
- Specifies the connection is probed once after connecting to the server. You specify the timeout using the
connect_timeout
directive, otherwise the value forping_timeout
is used. - P (prepost)
- Specifies the connection is probed before sending each request to the server. You specify the timeout using the
prepost_timeout
directive, otherwise the value forping_timeout
is used. - I (interval)
- Specifies the connection is probed during regular internal maintenance cycles. You specify the idle time between each interval using the
connection_ping_interval
directive, otherwise the value forping_timeout
is used. - A (all)
- The most common setting, which specifies all directive flags are applied. For information about the
*_timeout
advanced directives, refer directly to Apache Tomcat Connector - Reference Guide.
- ping_timeout
- Specifies the time to wait for CPong answers to a CPing connection probe (see
ping_mode
). The default value is 10000 (milliseconds).
worker.properties Load Balancing Directives
- lbfactor
- Specifies the load-balancing factor for an individual worker, and is only specified for a member worker of a load balancer.This directive defines the relative amount of HTTP request load distributed to the worker compared to other workers in the cluster.A common example where this directive applies is where you want to differentiate servers with greater processing power than others in the cluster. For example, if you require a worker to take three times the load than other workers, specify
worker.worker name.lbfactor=3
- balance_workers
- Specifies the worker nodes that the load balancer must manage. The directive can be used multiple times for the same load balancer, and consists of a comma-separated list of worker names as specified in the workers.properties file.
- sticky_session
- Specifies whether requests for workers with SESSION IDs are routed back to the same worker. The default is
0
(false). When set to1
(true), load balancer persistence is enabled.For example, if you specifyworker.loadbalancer.sticky_session=0
, each request is load balanced between each node in the cluster. In other words, different requests for the same session will go to different servers based on server load.Ifworker.loadbalancer.sticky_session=1
, each session is persisted (locked) to one server until the session is terminated, providing that server is available.
Appendix B. Java Properties Reference
B.1. Proxy Configuration
- During server startup
- When a proxy is detected through the advertise mechanism
- During error recovery, when a proxy's configuration is reset.
Table B.1. Proxy Configuration Values for Tomcat
Value | Default | Description |
---|---|---|
stickySession | true | Specifies whether subsequent requests for a given session should be routed to the same node, if possible. |
stickySessionRemove | false | Specifies whether the httpd proxy should remove session stickiness if the balancer is unable to route a request to the node to which it is stuck. This property is ignored if stickySession is false . |
stickySessionForce | true | Specifies whether the httpd proxy should return an error if the balancer is unable to route a request to the node to which it is stuck. This property is ignored if stickySession is false . |
workerTimeout | -1 | Specifies the number of seconds to wait for a worker to become available to handle a request. When all the workers of a balancer are unusable, mod_cluster will retry after a while (workerTimeout/100) to find an usable worker. A value of -1 indicates that the httpd will not wait for a worker to be available and will return an error if no workers are available. |
maxAttempts | 1 | Specifies the number of times the httpd proxy will attempt to send a given request to a worker before aborting. The minimum value is 1 : try once before aborting. |
flushPackets | false | Specifies whether packet flushing is enabled or disabled. |
flushWait | -1 | Specifies the time to wait before flushing packets. A value of -1 means wait forever. |
ping | 10 | Time to wait (in seconds) for a pong answer to a ping. |
smax | - | Specifies the soft maximum idle connection count. The maximum value is determined by the httpd thread configuration (ThreadsPerChild or 1 ). |
ttl | 60 | Specifies the time (in seconds) idle connections persist, above the smax threshold. |
nodeTimeout | -1 | Specifies the time (in seconds) mod_cluster waits for the back-end server response before returning an error. mod_cluster always uses a cping/cpong before forwarding a request. The connectiontimeout value used by mod_cluster is the ping value. |
balancer | mycluster | Specifies the name of the load-balancer. |
loadBalancingGroup | - | Specifies the load balancing among jvmRoutes within the same load balancing group. A loadBalancingGroup is conceptually equivalent to a mod_jk domain directive. |
B.2. mod_cluster Proxy and Proxy Discovery Configuration Attributes
Table B.2. mod_cluster Proxy Discovery Configuration Attributes
Attribute | Property | Default Value |
---|---|---|
proxy-list | proxyList | - |
proxy-url | proxyURL | - |
advertise | advertise | true |
advertise-security-key | advertiseSecurityKey | - |
excluded-contexts | excludedContexts | - |
auto-enable-contexts | autoEnableContexts | true |
stop-context-timeout | stopContextTimeout | 10 seconds (in seconds) |
socket-timeout | nodeTimeout | 20 seconds (in milliseconds) |
Note
nodeTimeout
is not defined the ProxyTimeout
directive, Proxy
is used. If ProxyTimeout
is not defined the server timeout (Timeout) is used (default 300 seconds). nodeTimeout
, ProxyTimeout
or Timeout
is set at the socket level.
Table B.3. mod_cluster Proxy Configuration Attributes
Attribute | Property | Default Value |
---|---|---|
sticky-session | stickySession | true |
sticky-session-remove | stickySessionRemove | false |
sticky-session-force | stickySessionForce | true |
node-timeout | workerTimeout | -1 |
max-attempts | maxAttempts | 1 |
flush-packets | flushPackets | false |
flush-wait | flushWait | -1 |
ping | ping | 10 |
smax | smax | -1 (uses the default value) |
ttl | ttl | -1 (uses the default value) |
domain | loadBalancingGroup | - |
load-balancing-group | loadBalancingGroup | - |
B.3. Load Configuration
mod_cluster
is configured with Tomcat:
Table B.4. Load Configuration for Tomcat
Attribute | Default Value | Description |
---|---|---|
loadMetricClass | org.jboss.modcluster.load.metric.impl.BusyConnectorsLoadMetric | This is the class name of an object that is implementing org.jboss.load.metric.LoadMetric. |
loadMetricCapacity | 1 | This is the capacity of the load metric defined via the loadMetricClass property. |
loadHistory | 9 | This is the number of historic load values that must be considered in the load balance factor computation. |
loadDecayFactor | 2 | This is the factor by which the historic load values decrease in significance. |
Appendix C. Revision History
Revision History | |||||||
---|---|---|---|---|---|---|---|
Revision 3.0.0-4.1 | Wed Feb 11 2015 | Lucas Costi | |||||
| |||||||
Revision 3.0.0-4 | Tue Oct 07 2014 | Husnain Husnain Ali | |||||
| |||||||
Revision 3.0.0-3 | Tue Aug 19 2014 | Rakesh Ghatvisave | |||||
| |||||||
Revision 3.0.0-2 | Mon Jul 14 2014 | Mandar Joshi | |||||
|