Red Hat Training
A Red Hat training course is available for Red Hat JBoss Enterprise Application Platform
Chapter 6. Clustering in Web Applications
6.1. Session Replication
6.1.1. About HTTP Session Replication
Session replication ensures that client sessions of distributable applications are not disrupted by failovers of nodes in a cluster. Each node in the cluster shares information about ongoing sessions, and can take over sessions if a node disappears.
Session replication is the mechanism by which mod_cluster, mod_jk, mod_proxy, ISAPI, and NSAPI clusters provide high availability.
6.1.2. Enable Session Replication in Your Application
To take advantage of JBoss EAP High Availability (HA) features and enable clustering of your web application, you must configure your application to be distributable.
Make your Application Distributable
Indicate that your application is distributable. If your application is not marked as distributable, its sessions will never be distributed. Add the
<distributable/>
element inside the<web-app>
tag of your application’sweb.xml
descriptor file:Example: Minimum Configuration for a Distributable Application
<?xml version="1.0"?> <web-app xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_3_0.xsd" version="3.0"> <distributable/> </web-app>
Next, if desired, modify the default replication behavior. If you want to change any of the values affecting session replication, you can override them inside a
<replication-config>
element inside<jboss-web>
in an application’sWEB-INF/jboss-web.xml
file. For a given element, only include it if you want to override the defaults.Example:
<replication-config>
Values<jboss-web xmlns="http://www.jboss.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.jboss.com/xml/ns/javaee http://www.jboss.org/j2ee/schema/jboss-web_10_0.xsd"> <replication-config> <replication-granularity>SESSION</replication-granularity> </replication-config> </jboss-web>
The <replication-granularity>
parameter determines the granularity of data that is replicated. It defaults to SESSION
, but can be set to ATTRIBUTE
to increase performance on sessions where most attributes remain unchanged.
Valid values for <replication-granularity>
can be :
-
SESSION
: The default value. The entire session object is replicated if any attribute is dirty. This policy is required if an object reference is shared by multiple session attributes. The shared object references are maintained on remote nodes since the entire session is serialized in one unit. -
ATTRIBUTE
: This is only for dirty attributes in the session and for some session data, such as the last-accessed timestamp.
Immutable Session Attributes
For JBoss EAP7, session replication is triggered when the session is mutated or when any mutable attribute of the session is accessed. Session attributes are assumed to be mutable unless one of the following is true:
The value is a known immutable value:
-
null
-
java.util.Collections.EMPTY_LIST
,EMPTY_MAP
,EMPTY_SET
-
The value type is or implements a known immutable type:
-
java.lang.Boolean
,Character
,Byte
,Short
,Integer
,Long
,Float
,Double
-
java.lang.Class
,Enum
,StackTraceElement
,String
-
java.io.File
,java.nio.file.Path
-
java.math.BigDecimal
,BigInteger
,MathContext
-
java.net.Inet4Address
,Inet6Address
,InetSocketAddress
,URI
,URL
-
java.security.Permission
-
java.util.Currency
,Locale
,TimeZone
,UUID
-
java.time.Clock
,Duration
,Instant
,LocalDate
,LocalDateTime
,LocalTime
,MonthDay
,Period
,Year
,YearMonth
,ZoneId
,ZoneOffset
,ZonedDateTime
-
java.time.chrono.ChronoLocalDate
,Chronology
,Era
-
java.time.format.DateTimeFormatter
,DecimalStyle
-
java.time.temporal.TemporalField
,TemporalUnit
,ValueRange
,WeekFields
-
java.time.zone.ZoneOffsetTransition
,ZoneOffsetTransitionRule
,ZoneRules
-
The value type is annotated with:
-
@org.wildfly.clustering.web.annotation.Immutable
-
@net.jcip.annotations.Immutable
-
6.2. HTTP Session Passivation and Activation
6.2.1. About HTTP Session Passivation and Activation
Passivation is the process of controlling memory usage by removing relatively unused sessions from memory while storing them in persistent storage.
Activation is when passivated data is retrieved from persisted storage and put back into memory.
Passivation occurs at different times in an HTTP session’s lifetime:
- When the container requests the creation of a new session, if the number of currently active sessions exceeds a configurable limit, the server attempts to passivate some sessions to make room for the new one.
- When a web application is deployed and a backup copy of sessions active on other servers is acquired by the newly deploying web application’s session manager, sessions may be passivated.
A session is passivated if the number of active sessions exceeds a configurable maximum.
Sessions are always passivated using a Least Recently Used (LRU) algorithm.
6.2.2. Configure HTTP Session Passivation in Your Application
HTTP session passivation is configured in your application’s WEB-INF/jboss-web.xml
and META-INF/jboss-web.xml
file.
Example: jboss-web.xml
File
<jboss-web xmlns="http://www.jboss.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.jboss.com/xml/ns/javaee http://www.jboss.org/j2ee/schema/jboss-web_10_0.xsd"> <max-active-sessions>20</max-active-sessions> </jboss-web>
The <max-active-sessions>
element dictates the maximum number of active sessions allowed, and is used to enable session passivation. If session creation would cause the number of active sessions to exceed <max-active-sessions/>
, then the oldest session known to the session manager will passivate to make room for the new session.
The total number of sessions in memory includes sessions replicated from other cluster nodes that are not being accessed on this node. Take this into account when setting <max-active-sessions>
. The number of sessions replicated from other nodes also depends on whether REPL
or DIST
cache mode is enabled. In REPL
cache mode, each session is replicated to each node. In DIST
cache mode, each session is replicated only to the number of nodes specified by the owners
parameter. See Configure the Cache Mode in the JBoss EAP Config Guide for information on configuring session cache modes. For example, consider an eight node cluster, where each node handles requests from 100 users. With REPL
cache mode, each node would store 800 sessions in memory. With DIST
cache mode enabled, and the default owners
setting of 2
, each node stores 200 sessions in memory.
6.3. Public API for Clustering Services
JBoss EAP 7 introduces a refined public clustering API for use by applications. The new services are designed to be lightweight, easily injectable, with no external dependencies.
org.wildfly.clustering.group.Group
The group service provides a mechanism to view the cluster topology for a JGroups channel, and to be notified when the topology changes.
@Resource(lookup = "java:jboss/clustering/group/channel-name") private Group channelGroup;
org.wildfly.clustering.dispatcher.CommandDispatcher
The
CommandDispatcherFactory
service provides a mechanism to create a dispatcher for executing commands on nodes in the cluster. The resultingCommandDispatcher
is a command-pattern analog to the reflection-basedGroupRpcDispatcher
from previous JBoss EAP releases.@Resource(lookup = "java:jboss/clustering/dispatcher/channel-name") private CommandDispatcherFactory factory; public void foo() { String context = "Hello world!"; try (CommandDispatcher<String> dispatcher = this.factory.createCommandDispatcher(context)) { dispatcher.executeOnCluster(new StdOutCommand()); } } public static class StdOutCommand implements Command<Void, String> { @Override public Void execute(String context) { System.out.println(context); return null; } }
6.4. HA Singleton Service
A clustered singleton service, also known as a high-availability (HA) singleton, is a service deployed on multiple nodes in a cluster. The service is provided on only one of the nodes. The node running the singleton service is usually called the master node.
When the master node either fails or shuts down, another master is selected from the remaining nodes and the service is restarted on the new master. Other than a brief interval when one master has stopped and another has yet to take over, the service is provided by one, but only one, node.
HA Singleton ServiceBuilder API
JBoss EAP 7 introduces a new public API for building singleton services that simplifies the process significantly.
The SingletonServiceBuilder
implementation installs its services so they will start asynchronously, preventing deadlocking of the Modular Service Container (MSC).
HA Singleton Service Election Policies
If there is a preference for which node should start the ha-singleton, you can set the election policy in the ServiceActivator
class.
JBoss EAP provides two election policies:
Simple Election Policy
The simple election policy selects a master node based on the relative age. The required age is configured in the position property, which is the index in the list of available nodes where,
- position = 0 – refers to the oldest node (the default)
position = 1 – refers to the 2nd oldest etc.
Position can also be negative to indicate the youngest nodes.
- position = -1 – refers to the youngest node
- position = -2 – refers to the 2nd youngest node etc.
Random Election Policy
The random election policy elects a random member to be the provider of a singleton service.
Create an HA Singleton Service Application
The following is an abbreviated example of the steps required to create and deploy an application as a cluster-wide singleton service. This example service activates a scheduled timer that is started only once in the cluster.
Create an
HATimerService
service that implements theorg.jboss.msc.service.Service
interface and contains thegetValue()
,start()
, andstop()
methods.Service Class Code Example
public class HATimerService implements Service<String> { private static final Logger LOGGER = Logger.getLogger(HATimerService.class.toString()); public static final ServiceName SINGLETON_SERVICE_NAME = ServiceName.JBOSS.append("quickstart", "ha", "singleton", "timer"); /** * A flag whether the service is started. */ private final AtomicBoolean started = new AtomicBoolean(false); /** * @return the name of the server node */ public String getValue() throws IllegalStateException, IllegalArgumentException { LOGGER.info(String.format("%s is %s at %s", HATimerService.class.getSimpleName(), (started.get() ? "started" : "not started"), System.getProperty("jboss.node.name"))); return System.getProperty("jboss.node.name"); } public void start(StartContext arg0) throws StartException { if (!started.compareAndSet(false, true)) { throw new StartException("The service is still started!"); } LOGGER.info("Start HASingleton timer service '" + this.getClass().getName() + "'"); final String node = System.getProperty("jboss.node.name"); try { InitialContext ic = new InitialContext(); ((Scheduler) ic.lookup("global/jboss-cluster-ha-singleton-service/SchedulerBean!org.jboss.as.quickstarts.cluster.hasingleton.service.ejb.Scheduler")) .initialize("HASingleton timer @" + node + " " + new Date()); } catch (NamingException e) { throw new StartException("Could not initialize timer", e); } } public void stop(StopContext arg0) { if (!started.compareAndSet(true, false)) { LOGGER.warning("The service '" + this.getClass().getName() + "' is not active!"); } else { LOGGER.info("Stop HASingleton timer service '" + this.getClass().getName() + "'"); try { InitialContext ic = new InitialContext(); ((Scheduler) ic.lookup("global/jboss-cluster-ha-singleton-service/SchedulerBean!org.jboss.as.quickstarts.cluster.hasingleton.service.ejb.Scheduler")).stop(); } catch (NamingException e) { LOGGER.info("Could not stop timer:" + e.getMessage()); } } } }
Create a service activator that implements the
org.jboss.msc.service.ServiceActivator
interface and installs theHATimerService
as a clustered singleton in theactivate()
method. This example specifies thatnode1
should start the singleton service.Service Activator Code Example
public class HATimerServiceActivator implements ServiceActivator { private final Logger log = Logger.getLogger(this.getClass().toString()); @Override public void activate(ServiceActivatorContext context) { log.info("HATimerService will be installed!"); HATimerService service = new HATimerService(); ServiceName factoryServiceName = SingletonServiceName.BUILDER.getServiceName("server", "default"); ServiceController<?> factoryService = context.getServiceRegistry().getRequiredService(factoryServiceName); SingletonServiceBuilderFactory factory = (SingletonServiceBuilderFactory) factoryService.getValue(); ServiceName ejbComponentService = ServiceName.of("jboss", "deployment", "unit", "jboss-cluster-ha-singleton-service.jar", "component", "SchedulerBean", "START"); factory.createSingletonServiceBuilder(HATimerService.SINGLETON_SERVICE_NAME, service) .electionPolicy(new PreferredSingletonElectionPolicy(new SimpleSingletonElectionPolicy(), new NamePreference("node1/singleton"))) .build(new DelegatingServiceContainer(context.getServiceTarget(), context.getServiceRegistry())) .setInitialMode(ServiceController.Mode.ACTIVE) .addDependency(ejbComponentService) .install(); } }
Create a file named
org.jboss.msc.service.ServiceActivator
in the application’sMETA-INF/services/
directory and add a line containing the fully qualified name of theServiceActivator
class created in the previous step.META-INF/services/org.jboss.msc.service.ServiceActivator File Example
org.jboss.as.quickstarts.cluster.hasingleton.service.ejb.HATimerServiceActivator
Create a
Scheduler
interface that contains theinitialize()
andstop()
methods.Scheduler Interface Code Example
public interface Scheduler { void initialize(String info); void stop(); }
Create a
Singleton
bean that implements theScheduler
interface. This bean is used as the cluster-wide singleton timer.ImportantThe
Singleton
bean must not have a remote interface and you must not reference its local interface from another EJB in any application. This prevents a lookup by a client or other component and ensures theHATimerService
has total control of theSingleton
.Singleton Bean Code Example
@Singleton public class SchedulerBean implements Scheduler { private static Logger LOGGER = Logger.getLogger(SchedulerBean.class.toString()); @Resource private TimerService timerService; @Timeout public void scheduler(Timer timer) { LOGGER.info("HASingletonTimer: Info=" + timer.getInfo()); } @Override public void initialize(String info) { ScheduleExpression sexpr = new ScheduleExpression(); // set schedule to every 10 seconds for demonstration sexpr.hour("*").minute("*").second("0/10"); // persistent must be false because the timer is started by the HASingleton service timerService.createCalendarTimer(sexpr, new TimerConfig(info, false)); } @Override public void stop() { LOGGER.info("Stop all existing HASingleton timers"); for (Timer timer : timerService.getTimers()) { LOGGER.fine("Stop HASingleton timer: " + timer.getInfo()); timer.cancel(); } } }
See the cluster-ha-singleton
quickstart that ships with JBoss EAP for a complete working example of this application. The quickstart provides detailed instructions to build and deploy the application.
6.5. HA Singleton Deployments
JBoss EAP 7 adds the ability to deploy a given application as a singleton deployment.
When deployed to a group of clustered servers, a singleton deployment will only deploy on a single node at any given time. If the node on which the deployment is active stops or fails, the deployment will automatically start on another node.
The policies for controlling HA singleton behavior are managed by a new singleton subsystem. A deployment may either specify a specific singleton policy or use the default subsystem policy. A deployment identifies itself as singleton deployment via a /META-INF/singleton-deployment.xml
deployment descriptor which is most easily applied to an existing deployment as a deployment overlay. Alternatively, the requisite singleton configuration can be embedded within an existing jboss-all.xml
.
Defining or Choosing a Singleton Deployment
To define a deployment as a singleton deployment, include a
/META-INF/singleton-deployment.xml
descriptor in your application archive.Example: Singleton Deployment Descriptor
<?xml version="1.0" encoding="UTF-8"?> <singleton-deployment xmlns="urn:jboss:singleton-deployment:1.0"/>
Example: Singleton Deployment Descriptor with a Specific Singleton Policy
<?xml version="1.0" encoding="UTF-8"?> <singleton-deployment policy="my-new-policy" xmlns="urn:jboss:singleton-deployment:1.0"/>
Alternatively, you can also add a
singleton-deployment
element to yourjboss-all.xml
descriptor.Example: Defining
singleton-deployment
injboss-all.xml
<?xml version="1.0" encoding="UTF-8"?> <jboss xmlns="urn:jboss:1.0"> <singleton-deployment xmlns="urn:jboss:singleton-deployment:1.0"/> </jboss>
Example: Defining
singleton-deployment
injboss-all.xml
with a Specific Singleton Policy<?xml version="1.0" encoding="UTF-8"?> <jboss xmlns="urn:jboss:1.0"> <singleton-deployment policy="my-new-policy" xmlns="urn:jboss:singleton-deployment:1.0"/> </jboss>
Creating a Singleton Deployment
JBoss EAP provides two election policies:
Simple Election Policy
The
simple-election-policy
chooses a specific member, indicated by theposition
attribute, on which a given application will be deployed. Theposition
attribute determines the index of the node to be elected from a list of candidates sorted by descending age, where0
indicates the oldest node,1
indicates the second oldest node,-1
indicates the youngest node,-2
indicates the second youngest node, and so on. If the specified position exceeds the number of candidates, a modulus operation is applied.Example: Create a New Singleton Policy with a
simple-election-policy
and Position Set to-1
, Using the Management CLIbatch /subsystem=singleton/singleton-policy=my-new-policy:add(cache-container=server) /subsystem=singleton/singleton-policy=my-new-policy/election- policy=simple:add(position=-1) run-batch
NoteTo set the newly created policy
my-new-policy
as the default, run this command:/subsystem=singleton:write-attribute(name=default, value=my-new-policy)
Example: Configure a
simple-election-policy
with Position Set to-1
Usingstandalone-ha.xml
<subsystem xmlns="urn:jboss:domain:singleton:1.0"> <singleton-policies default="my-new-policy"> <singleton-policy name="my-new-policy" cache-container="server"> <simple-election-policy position="-1"/> </singleton-policy> </singleton-policies> </subsystem>
Random Election Policy
The
random-election-policy
chooses a random member on which a given application will be deployed.Example: Creating a New Singleton Policy with a
random-election-policy
, Using the Management CLIbatch /subsystem=singleton/singleton-policy=my-other-new-policy:add(cache-container=server) /subsystem=singleton/singleton-policy=my-other-new-policy/election-policy=random:add() run-batch
Example: Configure a
random-election-policy
Usingstandalone-ha.xml
<subsystem xmlns="urn:jboss:domain:singleton:1.0"> <singleton-policies default="my-other-new-policy"> <singleton-policy name="my-other-new-policy" cache-container="server"> <random-election-policy/> </singleton-policy> </singleton-policies> </subsystem>
NoteThe
default-cache
attribute of thecache-container
needs to be defined before trying to add the policy. Without this, if you are using a custom cache container, you might end up getting error messages.
Preferences
Additionally, any singleton election policy may indicate a preference for one or more members of a cluster. Preferences may be defined either via node name or via outbound socket binding name. Node preferences always take precedent over the results of an election policy.
Example: Indicate Preference in the Existing Singleton Policy Using the Management CLI
/subsystem=singleton/singleton-policy=foo/election-policy=simple:list-add(name=name-preferences, value=nodeA) /subsystem=singleton/singleton-policy=bar/election-policy=random:list-add(name=socket-binding-preferences, value=binding1)
Example: Create a New Singleton Policy with a simple-election-policy
and name-preferences
, Using the Management CLI
batch /subsystem=singleton/singleton-policy=my-new-policy:add(cache-container=server) /subsystem=singleton/singleton-policy=my-new-policy/election-policy=simple:add(name-preferences=[node1, node2, node3, node4]) run-batch
To set the newly created policy my-new-policy
as the default, run this command:
/subsystem=singleton:write-attribute(name=default, value=my-new-policy)
Example: Configure a random-election-policy
with socket-binding-preferences
Using standalone-ha.xml
<subsystem xmlns="urn:jboss:domain:singleton:1.0"> <singleton-policies default="my-other-new-policy"> <singleton-policy name="my-other-new-policy" cache-container="server"> <random-election-policy> <socket-binding-preferences>binding1 binding2 binding3 binding4</socket-binding-preferences> </random-election-policy> </singleton-policy> </singleton-policies> </subsystem>
Quorum
Network partitions are particularly problematic for singleton deployments, since they can trigger multiple singleton providers for the same deployment to run at the same time. To defend against this scenario, a singleton policy may define a quorum that requires a minimum number of nodes to be present before a singleton provider election can take place. A typical deployment scenario uses a quorum of N/2 + 1, where N is the anticipated cluster size. This value can be updated at runtime, and will immediately affect any singleton deployments using the respective singleton policy.
Example: Quorum Declaration in the standalone-ha.xml
File
<subsystem xmlns="urn:jboss:domain:singleton:1.0"> <singleton-policies default="default"> <singleton-policy name="default" cache-container="server" quorum="4"> <simple-election-policy/> </singleton-policy> </singleton-policies> </subsystem>
Example: Quorum Declaration Using the Management CLI
/subsystem=singleton/singleton-policy=foo:write-attribute(name=quorum, value=3)
6.6. Apache mod_cluster-manager Application
6.6.1. About mod_cluster-manager Application
The mod_cluster-manager application is an administration web page, which is available on Apache HTTP Server. It is used for monitoring the connected worker nodes and performing various administration tasks, such as enabling or disabling contexts, and configuring the load-balancing properties of worker nodes in a cluster.
Exploring mod_cluster-manager Application
The mod_cluster-manager application can be used for performing various administration tasks on worker nodes.
Figure - mod_cluster Administration Web Page
- [1] mod_cluster/1.3.1.Final: The version of the mod_cluster native library.
- [2] ajp://192.168.122.204:8099: The protocol used (either AJP, HTTP, or HTTPS), hostname or IP address of the worker node, and the port.
- [3] jboss-eap-7.0-2: The worker node’s JVMRoute.
- [4] Virtual Host 1: The virtual host(s) configured on the worker node.
- [5] Disable: An administration option that can be used to disable the creation of new sessions on the particular context. However, the ongoing sessions do not get disabled and remain intact.
-
[6] Stop: An administration option that can be used to stop the routing of session requests to the context. The remaining sessions will failover to another node unless the
sticky-session-force
property is set totrue
. - [7] Enable Contexts Disable Contexts Stop Contexts: The operations that can be performed on the whole node. Selecting one of these options affects all the contexts of a node in all its virtual hosts.
[8] Load balancing group (LBGroup): The
load-balancing-group
property is set in themodcluster
subsystem in JBoss EAP configuration to group all worker nodes into custom load balancing groups. Load balancing group (LBGroup) is an informational field that gives information about all set load balancing groups. If this field is not set, then all worker nodes are grouped into a single default load balancing group.NoteThis is only an informational field and thus cannot be used to set
load-balancing-group
property. The property has to be set inmodcluster
subsystem in JBoss EAP configuration.[9] Load (value): The load factor on the worker node. The load factor(s) are evaluated as below:
-load > 0 : A load factor with value 1 indicates that the worker node is overloaded. A load factor of 100 denotes a free and not-loaded node. -load = 0 : A load factor of value 0 indicates that the worker node is in standby mode. This means that no session requests will be routed to this node until and unless the other worker nodes are unavailable. -load = -1 : A load factor of value -1 indicates that the worker node is in an error state. -load = -2 : A load factor of value -2 indicates that the worker node is undergoing CPing/CPong and is in a transition state.
For JBoss EAP 7.0, it is also possible to use Undertow as load balancer.