JBoss EAP 7 TCP Clustering

Solution Verified - Updated -

Environment

  • Red Hat JBoss Enterprise Application Platform (EAP)
    • 7
  • Infinispan Clustering with JGroups
  • TCP-based protocol

Issue

  • How do I configure / setup TCP-based clustering?
  • Please provide instructions for configuration without multicast.

Resolution

  1. Specify the tcp stack is the default stack:

    <channel name="ee" stack="tcp"/>
    
  2. Set node name

    • In standalone (HA-based) configuration:

      <server name="node_1" xmlns="urn:jboss:domain:4.1">
      
    • In managed domain mode, the servers are listed in the host-*.xml in the servers tag. The default configuration specifies names.

      <servers>
          <server name="server-one" group="main-server-group"/>
          <server name="server-two" group="other-server-group">
              <socket-bindings port-offset="150"/>
          </server>
      </servers>
      
  3. Decide on a protocol to use

    1. TCPGOSSIP - see documentation

      This is the newer method "that uses the TCPGOSSIP protocol to use an external gossip router to discover the members of a cluster." It's easier to configure in managed domain mode because you don't have to use variable to set the other server ips. All the info you need can be found in the documentation.

    2. TCPPING - see documentation

      This is the older method, and requires each node to know all the other nodes. The documentation link in the title has the CLI commands, and these are preferable so that the configuration can be automated, and minor version upgrades will be easier.

      Standalone mode / Basics

      1. Setup the TCP stack. In most cases, this involves switching out the MPING protocol for the TCPPING protocol. For more information in regards to the available JGroups protocols, refer to JGroups documentation.

        <stack name="tcp">
            <transport type="TCP" socket-binding="jgroups-tcp"/>
            <protocol type="TCPPING">
                <property name="initial_hosts">192.168.1.5[7600],192.168.1.9[7600]</property>
                <property name="port_range">0</property>
            </protocol>
            <protocol type="MERGE3"/>
            <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
            <protocol type="FD"/>
            <protocol type="VERIFY_SUSPECT"/>
            <protocol type="pbcast.NAKACK2"/>
            <protocol type="UNICAST3"/>
            <protocol type="pbcast.STABLE"/>
            <protocol type="pbcast.GMS"/>
            <protocol type="MFC"/>
            <protocol type="FRAG2"/>
        </stack>
        
      2. Set the IP address of the private interface (which is used by the JGroups component by default). The IP address should correlate to one of the IP addresses specified in the initial_hosts configuration:

        <interface name="private">
            <inet-address value="${jboss.bind.address.private:192.168.1.5}"/>
        </interface>
        
      3. Repeat the above steps for the other nodes within the cluster. Once complete, start up each node and deploy a clustered application. Nodes should form a cluster. Logs would look something like such:

        INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (thread-2,ee,node_1) ISPN000094: Received new cluster view for channel server: [node_1|1] (2) [node_1, node_2]
        INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (thread-2,ee,node_1) ISPN000094: Received new cluster view for channel web: [node_1|1] (2) [node_1, node_2]
        INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (thread-2,ee,node_1) ISPN000094: Received new cluster view for channel hibernate: [node_1|1] (2) [node_1, node_2]
        INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (thread-2,ee,node_1) ISPN000094: Received new cluster view for channel ejb: [node_1|1] (2) [node_1, node_2]
        

      Managed Domain Mode Specifics

      1. Apply the aforementioned changes to the JGroups subsystem of the profile being used.
      2. If the same profile is used for multiple clusters, set the initial_hosts property as a system property:

        <protocol type="TCPPING">
            <property name="initial_hosts">${jboss.cluster.tcp.initial_hosts}</property>
        

        Set the system property at the server-group level:

        <server-groups>
            <server-group name="a-server-group" profile="ha">
                <socket-binding-group ref="ha-sockets"/>
                <system-properties>
                    <property name="jboss.cluster.tcp.initial_hosts" value="192.168.1.5[7600],192.168.1.9[7600]" />
                </system-properties>
        
      3. You will need to define a private interface within the host XML configuration of the host controller (discussed below). The IP address of the private interface should correlate to one of the IP addresses listed in initial_hosts.

        <interfaces>
        ....
            <interface name="private">
                <inet-address value="${jboss.bind.address.private:192.168.1.5}"/>
            </interface>
        </interfaces>
        

      Notes:

      • initial_hosts correlates to a listing of all members within the cluster.
      • 7600 correlates to the default jgroups-tcp port. Depending on one's configuration, port offsets, etc., this port may be different in some environments.
      • One can specify the system property jboss.server.name instead of setting name within the standalone configuration.
      • The property num_initial_members (from previous JBoss EAP versions) has been deprecated.*
      • The timeout property for the TCPPING protocol has been deprecated. If desired, join_timeout can be set in the pbcast.GMS protocol:

        <protocol type="pbcast.GMS">
            <property name="join_timeout">
                2000
            </property>
        </protocol>
        

Root Cause

TCP mode can be used when multicast isn't preferable or allowed by security policies.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

1 Comments

This is really helpful !