35.2. Configure Cross-Datacenter Replication

35.2.1. Configure Cross-Datacenter Replication (Remote Client-Server Mode)

In Red Hat JBoss Data Grid's Remote Client-Server mode, cross-datacenter replication is set up as follows:

Procedure 35.1. Set Up Cross-Datacenter Replication

  1. Set Up RELAY

    Add the following configuration to the standalone.xml file to set up RELAY:
    <subsystem xmlns="urn:infinispan:server:jgroups:8.0">
        <channels default="cluster">
            <channel name="cluster"/>
            <channel name="xsite" stack="tcp"/>
        </channels>
        <stacks default="udp">
            <stack name="udp">
                <transport type="UDP" socket-binding="jgroups-udp"/>
                <...other protocols...>
                <relay site="LON">
                    <remote-site name="NYC" channel="xsite"/>
                    <remote-site name="SFO" channel="xsite"/>
                </relay>
            </stack>
        </stacks>
    </subsystem>{
    The RELAY protocol creates an additional stack (running parallel to the existing UDP stack) to communicate with the remote site. If a TCP based stack is used for the local cluster, two TCP based stack configurations are required: one for local communication and one to connect to the remote site. For an illustration, see Section 35.1, “Cross-Datacenter Replication Operations”
  2. Set Up Sites

    Use the following configuration in the standalone.xml file to set up sites for each distributed cache in the cluster:
    <distributed-cache name="namedCache">
         <!-- Additional configuration elements here -->
         <backups>
            <backup site="{FIRSTSITENAME}" strategy="{SYNC/ASYNC}" />
            <backup site="{SECONDSITENAME}" strategy="{SYNC/ASYNC}" />
         </backups>
    </distributed-cache>
  3. Configure Local Site Transport

    Add the name of the local site in the transport element to configure transport:
    <transport executor="infinispan-transport" 
               lock-timeout="60000" 
               cluster="LON" 
               stack="udp"/>
A cross-datacenter example configuration may be found at $JDG_SERVER/docs/examples/configs/clustered-xsite.xml.

35.2.2. Configure Cross-Data Replication (Library Mode)

35.2.2.1. Configure Cross-Datacenter Replication Declaratively

When configuring Cross-Datacenter Replication, the relay.RELAY2 protocol creates an additional stack (running parallel to the existing TCP stack) to communicate with the remote site. If a TCP-based stack is used for the local cluster, two TCP based stack configurations are required: one for local communication and one to connect to the remote site.
In JBoss Data Grid's Library mode, cross-datacenter replication is set up as follows:

Procedure 35.2. Setting Up Cross-Datacenter Replication

  1. Configure the Local Site

    <infinispan
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="urn:infinispan:config:8.0 http://www.infinispan.org/schemas/infinispan-config-8.0.xsd"
          xmlns="urn:infinispan:config:8.0">
    
      <jgroups>
        <stack-file name="udp" path="jgroups-with-relay.xml"/>
      </jgroups>
    
      <cache-container default-cache="default">
        <transport cluster="infinispan-cluster" lock-timeout="50000" 
                   stack="udp" node-name="node1" 
                   machine="machine1" rack="rack1" site="LON"/>
          <local-cache name="default">
            <backups>
              <backup site="NYC" strategy="SYNC" failure-policy="IGNORE" timeout="12003"/>
              <backup site="SFO" strategy="ASYNC"/>
            </backups>
          </local-cache>
    
       <!-- Additional configuration information here -->
    </infinispan>
    1. Add the site attribute to the transport element to define the local site (in this example, the local site is named LON).
    2. Cross-site replication requires a non-default JGroups configuration. Define the jgroups element and define a custom stack-file, passing in the name of the file to be referenced and the location to this custom configuration. In this example, the JGroups configuration file is named jgroups-with-relay.xml.
    3. Configure the cache in site LON to back up to the sites NYC and SFO.
    4. Configure the back up caches:
      1. Configure the cache in site NYC to receive back up data from LON:
              <local-cache name="backupNYC">
                <backups/>
                <backup-for remote-cache="default" remote-site="LON"/>
              </local-cache>
      2. Configure the cache in site SFO to receive back up data from LON:
              <local-cache name="backupSFO">
                <backups/>
                <backup-for remote-cache="default" remote-site="LON"/>
              </local-cache>
  2. Add the Contents of the Configuration File

    As a default, Red Hat JBoss Data Grid includes JGroups configuration files such as default-configs/default-jgroups-tcp.xml and default-configs/default-jgroups-udp.xml in the infinispan-embedded-{VERSION}.jar package.
    Copy the JGroups configuration to a new file (in this example, it is named jgroups-with-relay.xml) and add the provided configuration information to this file. Note that the relay.RELAY2 protocol configuration must be the last protocol in the configuration stack.
    <config> 
        ... 
        <relay.RELAY2 site="LON" 
                  config="relay.xml"
                  relay_multicasts="false" />
    </config>
  3. Configure the relay.xml File

    Set up the relay.RELAY2 configuration in the relay.xml file. This file describes the global cluster configuration.
    <RelayConfiguration> 
        <sites> 
            <site name="LON" 
                  id="0"> 
                <bridges> 
                    <bridge config="jgroups-global.xml" 
                            name="global"/> 
                    </bridges> 
            </site>  
            <site name="NYC" 
                  id="1"> 
                <bridges> 
                    <bridge config="jgroups-global.xml" 
                            name="global"/> 
                    </bridges> 
            </site>  
            <site name="SFO" 
                  id="2"> 
                <bridges> 
                    <bridge config="jgroups-global.xml" 
                            name="global"/> 
                </bridges> 
            </site> 
        </sites> 
    </RelayConfiguration>
  4. Configure the Global Cluster

    The file jgroups-global.xml referenced in relay.xml contains another JGroups configuration which is used for the global cluster: communication between sites.
    The global cluster configuration is usually TCP-based and uses the TCPPING protocol (instead of PING or MPING) to discover members. Copy the contents of default-configs/default-jgroups-tcp.xml into jgroups-global.xml and add the following configuration in order to configure TCPPING:
    <config> 
        <TCP bind_port="7800" ... /> 
        <TCPPING initial_hosts="lon.hostname[7800],nyc.hostname[7800],sfo.hostname[7800]"
                 ergonomics="false" /> 
              <!-- Rest of the protocols --> 
    </config>
    Replace the hostnames (or IP addresses) in TCPPING.initial_hosts with those used for your site masters. The ports (7800 in this example) must match the TCP.bind_port.
    For more information about the TCPPING protocol, see Section 30.2.1.3, “Using the TCPPing Protocol”.