Red Hat Training

A Red Hat training course is available for JBoss Enterprise Application Platform Common Criteria Certification

Chapter 22. HTTP Services

HTTP session replication is used to replicate the state associated with web client sessions to other nodes in a cluster. Thus, in the event one of your nodes crashes, another node in the cluster will be able to recover. Two distinct functions must be performed:
  • Session state replication
  • Load-balancing HTTP Requests
State replication is directly handled by JBoss. When you run JBoss in the all configuration, session state replication is enabled by default. Just configure your web application as <distributable> in its web.xml (see Section 22.2, “Configuring HTTP session state replication”), deploy it, and its session state is automatically replicated across all JBoss instances in the cluster.
However, load-balancing is a different story; it is not handled by JBoss itself and requires an external load balancer. This function could be provided by specialized hardware switches or routers (Cisco LoadDirector for example) or by specialized software running on commodity hardware. As a very common scenario, we will demonstrate how to set up a software load balancer using Apache httpd and mod_jk.

Note

A load-balancer tracks HTTP requests and, depending on the session to which the request is linked, it dispatches the request to the appropriate node. This is called load-balancing with sticky-sessions or session affinity: once a session is created on a node, every future request will also be processed by that same node. Using a load-balancer that supports sticky-sessions but not configuring your web application for session replication allows you to scale very well by avoiding the cost of session state replication: each request for a session will always be handled by the same node. But in case a node dies, the state of all client sessions hosted by this node (the shopping carts, for example) will be lost and the clients will most probably need to login on another node and restart with a new session. In many situations, it is acceptable not to replicate HTTP sessions because all critical state is stored in a database or on the client. In other situations, losing a client session is not acceptable and, in this case, session state replication is the price one has to pay.

22.1. Configuring load balancing using Apache and mod_jk

Apache is a well-known web server which can be extended by plugging in modules. One of these modules, mod_jk has been specifically designed to allow the forwarding of requests from Apache to a Servlet container. Furthermore, it is also able to load-balance HTTP calls to a set of Servlet containers while maintaining sticky sessions, which is what is most interesting for us in this section.

22.1.1. Download the software

First of all, make sure that you have Apache installed. You can download Apache directly from Apache web site at http://httpd.apache.org/. Its installation is pretty straightforward and requires no specific configuration. As several versions of Apache exist, we advise you to use the latest stable 2.2.x version. We will assume, for the next sections, that you have installed Apache in the APACHE_HOME directory.
Next, download mod_jk binaries. Several versions of mod_jk exist as well. We strongly advise the use of mod_jk 1.2.x, as both earlier versions of mod_jk, and mod_jk2, are deprecated, unsupported and no further development is going on in the community. The mod_jk 1.2.x binary can be downloaded from http://www.apache.org/dist/jakarta/tomcat-connectors/jk/binaries/. Rename the downloaded file to mod_jk.so and copy it under APACHE_HOME/modules/.

22.1.2. Configure Apache to load mod_jk

Modify APACHE_HOME/conf/httpd.conf and add a single line at the end of the file:
# Include mod_jk's specific configuration file  
Include conf/mod-jk.conf
Next, create a new file named APACHE_HOME/conf/mod-jk.conf:
# Load mod_jk module
# Specify the filename of the mod_jk lib
LoadModule jk_module modules/mod_jk.so
 
# Where to find workers.properties
JkWorkersFile conf/workers.properties

# Where to put jk logs
JkLogFile logs/mod_jk.log
 
# Set the jk log level [debug/error/info]
JkLogLevel info 
 
# Select the log format
JkLogStampFormat  "[%a %b %d %H:%M:%S %Y]"
 
# JkOptions indicates to send SSK KEY SIZE
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
 
# JkRequestLogFormat
JkRequestLogFormat "%w %V %T"
               
# Mount your applications
JkMount /application/* loadbalancer
 
# You can use external file for mount points.
# It will be checked for updates each 60 seconds.
# The format of the file is: /url=worker
# /examples/*=loadbalancer
JkMountFile conf/uriworkermap.properties               

# Add shared memory.
# This directive is present with 1.2.10 and
# later versions of mod_jk, and is needed for
# for load balancing to work properly
JkShmFile logs/jk.shm 
              
# Add jkstatus for managing runtime data
<Location /jkstatus/>
    JkMount status
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
</Location>
Please note that two settings are very important:
  • The LoadModule directive must reference the mod_jk library you have downloaded in the previous section. You must indicate the exact same name with the "modules" file path prefix.
  • The JkMount directive tells Apache which URLs it should forward to the mod_jk module (and, in turn, to the Servlet containers). In the above file, all requests with URL path /application/* are sent to the mod_jk load-balancer. This way, you can configure Apache to serve static contents (or PHP contents) directly and only use the loadbalancer for Java applications. If you only use mod_jk as a loadbalancer, you can also forward all URLs (i.e., /*) to mod_jk.
In addition to the JkMount directive, you can also use the JkMountFile directive to specify a mount points configuration file, which contains multiple Tomcat forwarding URL mappings. You just need to create a uriworkermap.properties file in the APACHE_HOME/conf directory. The format of the file is /url=worker_name. To get things started, paste the following example into the file you created:
# Simple worker configuration file

# Mount the Servlet context to the ajp13 worker
/jmx-console=loadbalancer
/jmx-console/*=loadbalancer
/web-console=loadbalancer
/web-console/*=loadbalancer
This will configure mod_jk to forward requests to /jmx-console and /web-console to Tomcat.
You will most probably not change the other settings in mod_jk.conf. They are used to tell mod_jk where to put its logging file, which logging level to use and so on.

22.1.3. Configure worker nodes in mod_jk

Next, you need to configure mod_jk workers file conf/workers.properties. This file specifies where the different Servlet containers are located and how calls should be load-balanced across them. The configuration file contains one section for each target servlet container and one global section. For a two nodes setup, the file could look like this:
# Define list of workers that will be used
# for mapping requests
worker.list=loadbalancer,status

# Define Node1
# modify the host as your host IP or DNS name.
worker.node1.port=8009
worker.node1.host=node1.mydomain.com
worker.node1.type=ajp13
worker.node1.ping_mode=A
worker.node1.lbfactor=1 

# Define Node2
# modify the host as your host IP or DNS name.
worker.node2.port=8009
worker.node2.host=node2.mydomain.com
worker.node2.type=ajp13
worker.node2.ping_mode=A
worker.node2.lbfactor=1

# Load-balancing behaviour
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=node1,node2
worker.loadbalancer.sticky_session=1
#worker.list=loadbalancer

# Status worker for managing load balancer
worker.status.type=status
Basically, the above file configures mod_jk to perform weighted round-robin load balancing with sticky sessions between two servlet containers (that is, JBoss Enterprise Application Platform instances) node1 and node2 listening on port 8009.
In the workers.properties file, each node is defined using the worker.XXX naming convention where XXX represents an arbitrary name you choose for each of the target Servlet containers. For each worker, you must specify the host name (or IP address) and the port number of the AJP13 connector running in the Servlet container.
The lbfactor attribute is the load-balancing factor for this specific worker. It is used to define the priority (or weight) a node should have over other nodes. The higher this number is for a given worker relative to the other workers, the more HTTP requests the worker will receive. This setting can be used to differentiate servers with different processing power.
The ping_mode attribute enables CPing/CPong. It determines when established connections are probed to determine whether they are still working. In this case, ping_mode is set to A, which means that the connection is probed once after connecting to the backend, before sending each request to the backend, and at regular intervals during the internal maintenance cycle.
The last part of the conf/workers.properties file defines the loadbalancer worker. The only thing you must change is the worker.loadbalancer.balanced_workers line: it must list all workers previously defined in the same file. Load-balancing will happen over these workers.
The sticky_session property specifies the cluster behavior for HTTP sessions. If you specify worker.loadbalancer.sticky_session=0, each request will be load balanced between node1 and node2; i.e., different requests for the same session will go to different servers. But when a user opens a session on one server, it is always necessary to always forward this user's requests to the same server, as long as that server is available. This is called a "sticky session", as the client is always using the same server he reached on his first request. To enable session stickiness, you need to set worker.loadbalancer.sticky_session to 1.

Note

A non-loadbalanced setup with a single node requires a worker.list=node1 entry.

22.1.4. Configuring JBoss to work with mod_jk

Finally, we must configure the JBoss Enterprise Application Platform instances on all clustered nodes so that they can expect requests forwarded from the mod_jk loadbalancer.
On each clustered JBoss node, we have to name the node according to the name specified in workers.properties. For instance, on JBoss instance node1, edit the JBOSS_HOME/server/all/deploy/jbossweb.sar/server.xml file (replace /all with your own server name if necessary). Locate the <Engine> element and add an attribute jvmRoute:
<Engine name="jboss.web" defaultHost="localhost" jvmRoute="node1">
...
</Engine>
You also need to be sure the AJP connector in server.xml is enabled (i.e., uncommented). It is enabled by default.
 
<!-- Define an AJP 1.3 Connector on port 8009 --> 
<Connector protocol="AJP/1.3" port="8009" address="${jboss.bind.address}"
   redirectPort="8443" />
At this point, you have a fully working Apache with mod_jk load-balancer setup that will balance call to the Servlet containers of your cluster while taking care of session stickiness (clients will always use the same Servlet container).

Note

For more updated information on using mod_jk 1.2 with JBoss AS, please refer to the JBoss wiki page at http://www.jboss.org/community/wiki/UsingModjk12WithJBoss.

22.1.5. Configuring the NSAPI connector on Solaris

This section shows you how to configure the NSAPI connector to use a JBoss Enterprise Platform as a worker node for a Sun Java System Web Server (SJWS) master node.

Note

Sun Java System Web Server has recently been renamed to the Oracle iPlanet Web Server.
In this section, all of the server instances are on the same machine. To use different machines for each instance, use the -b switch to bind your instance of JBoss Enterprise Platform to a public IP address. Remember to edit the workers.properties file on the SJWS machine to reflect these changes in IP address.

22.1.5.1. Prerequisites

This section assumes that:
  • Your worker node(s) are already installed with a JBoss Enterprise Platform 5.1 or later. The Native components are not a requirement of the NSAPI connector. Refer to the Installation Guide for assistance with this prerequisite.
  • Your master node is already installed with any of the following technology combinations, and the appropriate Native binary for its operating system and architecture. Refer to the Installation Guide for assistance with this prerequisite.
    • Solaris 9 x86 with Sun Java System Web Server 6.1 SP12
    • Solaris 9 SPARC 64 with Sun Java System Web Server 6.1 SP12
    • Solaris 10 x86 with Sun Java System Web Server 7.0 U8
    • Solaris 10 SPARC 64 with Sun Java System Web Server 7.0 U8

22.1.5.2. Configure JBoss Enterprise Platform as a Worker Node

This section shows you how to safely configure your JBoss Enterprise Platform instance as a worker node for use with Sun SJWS.

Procedure 22.1. Configure a JBoss Enterprise Platform instance as a worker node

  1. Create a server profile for each worker node

    Make a copy of the server profile that you wish to configure as a worker node. (This procedure uses the default server profile.)
    [user@workstation jboss-eap-5.1]$ cd jboss-as/server
    [user@workstation server]$ cp -r default/ default-01
    [user@workstation server]$ cp -r default/ default-02
  2. Give each instance a unique name

    Edit the following line in the deploy/jbossweb.sar/server.xml file of each new worker instance:
    <Engine name="jboss.web" defaultHost="localhost">
    Add a unique jvmRoute value, as shown. This value is the identifier for this node in the cluster.
    For the default-01 server profile:
    <Engine name="jboss.web" defaultHost="localhost" jvmRoute="worker01">
    For the default-02 server profile:
    <Engine name="jboss.web" defaultHost="localhost" jvmRoute="worker02">
  3. Enable session handling

    Edit the following line in the deployers/jbossweb.deployer/META-INF/war-deployers-jboss-beans.xml file of each worker node:
    <property name="useJK">false</property>
    This property controls whether special session handling is used to coordinate with mod_jk and other connector variants. Set this property to true in both worker nodes:
    <property name="useJK">true</property>
  4. Start your worker nodes

    Start each worker node in a separate command line interface. Ensure that each node is bound to a different IP address with the -b switch.
    [user@workstation jboss-eap-5.1]$ ./jboss-as/bin/run.sh -b 127.0.0.1 -c default-01
    [user@workstation jboss-eap-5.1]$ ./jboss-as/bin/run.sh -b 127.0.0.100 -c default-02

22.1.5.3. Configure Sun Java System Web Server for Clustering

The procedures in the following sections assume that the contents of the Native zip appropriate for your operating system and architecture have been extracted to /tmp/connectors/jboss-ep-native-5.1/. This path is referred to as NATIVE in the procedures that follow. These procedures also assume that the /tmp/connectors directory is used to store logs, properties files and NSAPI locks.
These procedures also assume that your installation of Sun Java System Web Server is in one of the following locations, depending on your version of Solaris:
  • for Solaris 9 x86 or SPARC 64: /opt/SUNWwbsrv61/
  • for Solaris 10 x86 or SPARC 64: /opt/SUNWwbsrv70/
This path is referred to as SJWS in the procedures that follow.

Procedure 22.2. Initial clustering configuration

  1. Disable servlet mappings

    Under Built In Servlet Mappings in the SJWS/PROFILE/config/default-web.xml file, disable the mappings for the following servlets, as shown in the code sample:
    • default
    • invoker
    • jsp
    <!-- ==================== Built In Servlet Mappings ===================== -->
    
    <!-- The servlet mappings for the built in servlets defined above. -->
    
    <!-- The mapping for the default servlet -->
    <!--servlet-mapping>
     <servlet-name>default</servlet-name>
     <url-pattern>/</url-pattern>
    </servlet-mapping-->
    
    <!-- The mapping for the invoker servlet -->
    <!--servlet-mapping>
     <servlet-name>invoker</servlet-name>
     <url-pattern>/servlet/*</url-pattern>
    </servlet-mapping-->
    
    <!-- The mapping for the JSP servlet -->
    <!--servlet-mapping>
     <servlet-name>jsp</servlet-name>
     <url-pattern>*.jsp</url-pattern>
    </servlet-mapping-->
  2. Load the required modules and properties

    Append the following lines to the SJWS/PROFILE/config/magnus.conf file:
    Init fn="load-modules" funcs="jk_init,jk_service" shlib="NATIVE/lib/nsapi_redirector.so" shlib_flags="(global|now)"
    Init fn="jk_init" worker_file="/tmp/connectors/workers.properties" log_level="debug" log_file="/tmp/connectors/nsapi.log" shm_file="/tmp/connectors/jk_shm"
    These lines define the location of the nsapi_redirector.so module used by the jk_init and jk_service functions, and the location of the workers.properties file, which defines the worker nodes and their attributes.

    Note

    The lib directory in the NATIVE/lib/nsapi_redirector.so path applies only to 32-bit machines. On 64-bit machines, this directory is called lib64.
22.1.5.3.1. Configure a basic cluster with NSAPI
Use the following procedure to configure a basic cluster, where requests for particular paths are forwarded to particular worker nodes. In Procedure 22.3, “Configure a basic cluster with NSAPI”, worker02 serves the /nc path, while worker01 serves /status and all other paths defined in the first part of the obj.conf file.

Procedure 22.3. Configure a basic cluster with NSAPI

  1. Define the paths to serve via NSAPI

    Edit the SJWS/PROFILE/config/obj.conf file. Define paths that should be served via NSAPI at the end of the default Object definition, as shown:
    <Object name="default">
          [...]
       NameTrans fn="assign-name" from="/status" name="jknsapi"
       NameTrans fn="assign-name" from="/images(|/*)" name="jknsapi"
       NameTrans fn="assign-name" from="/css(|/*)" name="jknsapi"
       NameTrans fn="assign-name" from="/nc(|/*)" name="jknsapi"
       NameTrans fn="assign-name" from="/jmx-console(|/*)" name="jknsapi"
    </Object>
    You can map the path of any application deployed on your JBoss Enterprise Platform instance in this obj.conf file. In the example code, the /nc path is mapped to an application deployed under the name nc.
  2. Define the worker that serves each path

    Edit the SJWS/PROFILE/config/obj.conf file and add the following jknsapi Object definition after the default Object definition.
    <Object name="jknsapi">
       ObjectType fn=force-type type=text/plain
       Service fn="jk_service" worker="worker01" path="/status"
       Service fn="jk_service" worker="worker02" path="/nc(/*)"
       Service fn="jk_service" worker="worker01"
    </Object>
    This jknsapi Object defines the worker nodes used to serve each path that was assigned to name="jknsapi" in the default Object.
    In the example code, the third Service definition does not specify a path value, so the worker node defined (worker01) serves all of the paths assigned to jknsapi by default. In this case, the first Service definition in the example code, which assigns the /status path to worker01, is superfluous.
  3. Define the workers and their attributes

    Create a workers.properties file in the location you defined in Step 2. Define the list of worker nodes and each worker node's properties in this file, like so:
    # An entry that lists all the workers defined
    worker.list=worker01, worker02
    
    # Entries that define the host and port associated with these workers
    worker.worker01.host=127.0.0.1
    worker.worker01.port=8009
    worker.worker01.type=ajp13
    
    worker.worker02.host=127.0.0.100
    worker.worker02.port=8009
    worker.worker02.type=ajp13
22.1.5.3.2. Configure a Load-balanced Cluster with NSAPI

Procedure 22.4. Configure a load-balancing cluster with NSAPI

  1. Define the paths to serve via NSAPI

    Edit the SJWS/PROFILE/config/obj.conf file. Define paths that should be served via NSAPI at the end of the default Object definition, as shown:
    <Object name="default">
          [...]
       NameTrans fn="assign-name" from="/status" name="jknsapi"
       NameTrans fn="assign-name" from="/images(|/*)" name="jknsapi"
       NameTrans fn="assign-name" from="/css(|/*)" name="jknsapi"
       NameTrans fn="assign-name" from="/nc(|/*)" name="jknsapi"
       NameTrans fn="assign-name" from="/jmx-console(|/*)" name="jknsapi"
       NameTrans fn="assign-name" from="/jkmanager/*" name="jknsapi"
    </Object>
    You can map the path of any application deployed on your JBoss Enterprise Platform instance in this obj.conf file. In the example code, the /nc path is mapped to an application deployed under the name nc.
  2. Define the worker that serves each path

    Edit the SJWS/PROFILE/config/obj.conf file and add the following jknsapi Object definition after the default Object definition.
    <Object name="jknsapi">
       ObjectType fn=force-type type=text/plain
       Service fn="jk_service" worker="status" path="/jkmanager(/*)"
       Service fn="jk_service" worker="router"
    </Object>
    This jknsapi Object defines the worker nodes used to serve each path that was assigned to name="jknsapi" in the default Object.
  3. Define the workers and their attributes

    Create a workers.properties file in the location you defined in Step 2. Define the list of worker nodes and each worker node's properties in this file, like so:
    # The advanced router LB worker
    worker.list=router,status
    
    # Define a worker using ajp13
    worker.worker01.port=8009
    worker.worker01.host=127.0.0.1
    worker.worker01.type=ajp13
    worker.worker01.ping_mode=A
    worker.worker01.socket_timeout=10
    worker.worker01.lbfactor=3
    
    # Define another worker using ajp13
    worker.worker02.port=8009
    worker.worker02.host=127.0.0.100
    worker.worker02.type=ajp13
    worker.worker02.ping_mode=A
    worker.worker02.socket_timeout=10
    worker.worker02.lbfactor=1
    
    # Define the LB worker
    worker.router.type=lb
    worker.router.balance_workers=worker01,worker02
    
    # Define the status worker
    worker.status.type=status
22.1.5.3.3. Restart Sun Java System Web Server
Once your Sun Java System Web Server instance is configured, restart it so that your changes take effect.
For Sun Java System Web Server 6.1:
SJWS/PROFILE/stop
SJWS/PROFILE/start
For Sun Java System Web Server 7.0:
SJWS/PROFILE/bin/stopserv
SJWS/PROFILE/bin/startserv