17.5. Clustering

17.5.1. Use TCP Communication for the Clustering Subsystem

By default, cluster nodes monitor each other's status using the UDP protocol. Some networks only allow TCP to be used. In this situation, you can add the TCPPING protocol stack to your configuration and use it as the default mechanism. These configuration options are available in the command-line based Management CLI.
The mod_cluster subsystem also uses UDP communication by default, and you can choose to use TCP here as well.
Refer to the following two procedures to configure JGroups and mod_cluster subsystems to use TCP for network communication:

17.5.2. Configure the JGroups Subsystem to Use TCP

By default, the JGroups subsystem communicates using multicast UDP. Use the following procedure to configure the JGroups subsystem to use unicast TCP instead.
To configure the mod_cluster subsystem to use TCP as well, see Section 17.5.3, “Disable Advertising for the mod_cluster Subsystem”.
  1. Modify the following script to suit your environment.

    Copy the following script into a text editor. If you use a different profile on a managed domain, change the profile name. If you use a standalone server, remove the /profile=full-ha portion of the commands. Modify the properties listed at the bottom of the command as follows. Each of these properties is optional.
    initial_hosts
    A comma-separated list of hosts, using the syntax HOST[PORT], that are considered well-known and will be available to look up the initial membership.
    port_range
    If desired, you can assign a port range. If you assign a port range of 2, and the initial port for a host is 7600, then TCPPING will attempt to contact the host on ports 7600-7602. The port range applies to each address specified in initial_hosts. This property is optional.
    timeout
    An optional timeout value, in milliseconds, for cluster members.
    num_initial_members
    The number of nodes before the cluster is considered to be complete. This property is optional.
    batch
    ##  If  tcp  is already added then you can remove it  ##
    /profile=full-ha/subsystem=jgroups/stack=tcp:remove
    /profile=full-ha/subsystem=jgroups/stack=tcp:add(transport={"type" =>"TCP", "socket-binding" => "jgroups-tcp"})
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=TCPPING)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=MERGE2)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=FD_SOCK,socket-binding=jgroups-tcp-fd)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=FD)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=VERIFY_SUSPECT)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=BARRIER)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=pbcast.NAKACK)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=UNICAST2)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=pbcast.STABLE)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=pbcast.GMS)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=UFC)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=MFC)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=FRAG2)
    /profile=full-ha/subsystem=jgroups/stack=tcp/:add-protocol(type=RSVP)
    /profile=full-ha/subsystem=jgroups:write-attribute(name=default-stack,value=tcp)
    run-batch
    /profile=full-ha/subsystem=jgroups/stack=tcp/protocol=TCPPING/property=initial_hosts/:add(value="HostA[7600],HostB[7600]")
    /profile=full-ha/subsystem=jgroups/stack=tcp/protocol=TCPPING/property=port_range/:add(value=0)
    /profile=full-ha/subsystem=jgroups/stack=tcp/protocol=TCPPING/property=timeout/:add(value=3000)
    /profile=full-ha/subsystem=jgroups/stack=tcp/protocol=TCPPING/property=num_initial_members/:add(value=3)
  2. Run the script in batch mode.

    Warning

    The servers running the profile have to be shutdown before executing the batch file.
    In a terminal emulator, navigate to the directory containing the jboss-cli.sh script and enter the command
    ./jboss-cli.sh -c --file=SCRIPT_NAME
    where SCRIPT_NAME is the name and path containing the script.
Result

The TCPPING stack is now available to the JGroups subsystem. If it is used, the JGroups subsystem uses TCP for all network communication. To configure the mod_cluster subsystem to use TCP as well, see Section 17.5.3, “Disable Advertising for the mod_cluster Subsystem”.

17.5.3. Disable Advertising for the mod_cluster Subsystem

By default, the mod_cluster subsystem's balancer uses multicast UDP to advertise its availability to the background workers. If you wish, you can disable advertisement. Use the following procedure to configure this behavior.

Procedure 17.11. 

  1. Modify the Apache HTTP Server configuration.

    Modify the Apache HTTP Server configuration to disable server advertising and to use a proxy list instead. The proxy list is configured on the worker, and contains all of the mod_cluster-enabled Web servers to which the worker can talk.
    The mod_cluster configuration for the Web server is located in HTTPD_HOME. See Section 17.6.3, “Install the mod_cluster Module Into Apache HTTP Server or JBoss Enterprise Web Server (Zip)” and Section 17.6.5, “Configure Server Advertisement Properties for Your mod_cluster-enabled Web Server” for more information about the file itself. Open the file containing the virtual host which listens for MCPM requests (using the EnableMCPMReceive directive), and disable server advertising by changing the ServerAdvertise directive as follows.
    ServerAdvertise Off
  2. Disable advertising within the mod_cluster subsystem of JBoss EAP 6, and provide a list of proxies.

    You can disable advertising for the mod_cluster subsystem and provide a list of proxies, by using the web-based Management Console or the command-line Management CLI. The list of proxies is necessary because the mod_cluster subsystem will not be able to automatically discover proxies if advertising is disabled.
    • Management Console

      If you use a managed domain, you can only configure mod_cluster in profiles where it is enabled, such as the ha and full-ha profiles.
      1. Log in to the Management Console and select the Configuration tab at the top of the screen. If you use a managed domain, select either the ha or full-ha profile from the Profile drop-down menu at the top left.
      2. Expand the Subsystems menu then expand the Web menu and select mod_cluster.
      3. Click Edit under the Advertising tab under mod_cluster. To disable advertising, clear the check box next to Advertise, and click Save.
        mod_cluster Advertising Configuration Screen

        Figure 17.1. mod_cluster Advertising Configuration Screen

      4. Click the Proxies tab. Click Edit and enter a list of proxy servers in the Proxy List field. The correct syntax is a comma-separated list of HOSTNAME:PORT strings, like the following:
        10.33.144.3:6666,10.33.144.1:6666
        Click the Save button to finish.
    • Management CLI

      The following two Management CLI commands create the same configuration as the Management Console instructions above. They assume that you run a managed domain and that your server group uses the full-ha profile. If you use a different profile, change its name in the commands. If you use a standalone server using the standalone-ha profile, remove the /profile=full-ha portion of the commands.
      /profile=full-ha/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=advertise,value=false)
      
      /profile=full-ha/subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=proxy-list,value="10.33.144.3:6666,10.33.144.1:6666")
Result

The Apache HTTP Server balancer no longer advertises its presence to worker nodes and UDP multicast is no longer used.

Note

In order to set the attribute advertise="false", you must also set the attribute proxy-list="address:port". If the proxy-list attribute is empty, the advertise="false" attribute is ignored. To disable the mod_cluster subsystem altogether, you may remove it from the server configuration.

17.5.4. Switch UDP to TCP for HornetQ Clustering

The following example uses the default standalone-full-ha.xml file shipped with EAP 6.

Note

If security is enabled, you must set the cluster-password attribute:
<cluster-password>${jboss.messaging.cluster.password:ChangeMe>}</cluster-password>
  1. Remove the broadcast-groups and discovery-groups:

    <broadcast-groups>
        <broadcast-group name="bg-group1">
            <socket-binding>messaging-group</socket-binding>
            <broadcast-period>5000</broadcast-period>
            <connector-ref>netty</connector-ref>
        </broadcast-group>
    </broadcast-groups>
    <discovery-groups>
        <discovery-group name="dg-group1">
            <socket-binding>messaging-group</socket-binding>
            <refresh-timeout>10000</refresh-timeout>
        </discovery-group>
    </discovery-groups
    
  2. Optionally, remove the "messaging-group" socket-binding:

    <socket-binding name="messaging-group" port="0" multicast-address="${jboss.messaging.group.address:231.7.7.7}" multicast-port="${jboss.messaging.group.port:9876}"/>
  3. Configure the appropriate Netty connector(s) - one for each of the other nodes in the cluster.

    For example, if the cluster is 3 nodes then configure 2 Netty connectors, etc., if the cluster is 2 nodes then configure 1 Netty connector, etc. Here is a sample configuration for a 3-node cluster:
    <netty-connector name="other-cluster-node1" socket-binding="other-cluster-node1"/>
    <netty-connector name="other-cluster-node2" socket-binding="other-cluster-node2"/>
  4. Configure the related socket bindings.

    Note

    The system property substitution can be used for either "host" or "port", if required.
    <outbound-socket-binding name="other-cluster-node1">
        <remote-destination host="otherNodeHostName1" port="5445"/>
    </outbound-socket-binding>
    <outbound-socket-binding name="other-cluster-node2">
        <remote-destination host="otherNodeHostName2" port="5445"/>
    </outbound-socket-binding>
  5. Configure the cluster-connection to use these connectors instead of the discovery-group, which is used by default:

    <cluster-connection name="my-cluster">
        <address>jms</address>
        <connector-ref>netty</connector-ref>
        <static-connectors>
            <connector-ref>other-cluster-node1</connector-ref>
            <connector-ref>other-cluster-node2</connector-ref>
        </static-connectors>
    </cluster-connection>
    This process has to be repeated on each of the cluster nodes so that each node has connectors to every other node in the cluster.

    Note

    Do not configure a node with a connection to itself. This is considered as a misconfiguration.