31.2. Configure Cross-Datacenter Replication

31.2.1. Configure Cross-Datacenter Replication (Remote Client-Server Mode)

In Red Hat JBoss Data Grid's Remote Client-Server mode, cross-datacenter replication is set up as follows:

Procedure 31.1. Set Up Cross-Datacenter Replication

  1. Set Up RELAY

    Add the following configuration to the standalone.xml file to set up RELAY:
    <subsystem xmlns="urn:jboss:domain:jgroups:1.2" 
    	   default-stack="udp">
    	<stack name="udp">
    		<transport type="UDP" 
    			   socket-binding="jgroups-udp"/>
    		<!-- Additional configuration elements here -->
    		<relay site="LON">
    		   <remote-site name="NYC" stack="tcp" cluster="global"/>
    		   <remote-site name="SFO" stack="tcp" cluster="global"/>
    		   <property name="relay_multicasts">false</property>
    		</relay>
    	</stack>
    </subsystem>
    The RELAY protocol creates an additional stack (running parallel to the existing TCP stack) to communicate with the remote site. If a TCP based stack is used for the local cluster, two TCP based stack configurations are required: one for local communication and one to connect to the remote site. For an illustration, see Section 31.1, “Cross-Datacenter Replication Operations”
  2. Set Up Sites

    Use the following configuration in the standalone.xml file to set up sites for each distributed cache in the cluster:
    <distributed-cache>
         <!-- Additional configuration elements here -->
         <backups>
            <backup site="{FIRSTSITENAME}" strategy="{SYNC/ASYNC}" />
            <backup site="{SECONDSITENAME}" strategy="{SYNC/ASYNC}" />
         </backups>
    </distributed-cache>
  3. Configure Local Site Transport

    Add the name of the local site in the transport element to configure transport:
    <transport executor="infinispan-transport" 
               lock-timeout="60000" 
               cluster="LON" 
               stack="udp"/>

31.2.2. Configure Cross-Data Replication (Library Mode)

31.2.2.1. Configure Cross-Datacenter Replication Declaratively

When configuring Cross-Datacenter Replication, the relay.RELAY2 protocol creates an additional stack (running parallel to the existing TCP stack) to communicate with the remote site. If a TCP-based stack is used for the local cluster, two TCP based stack configurations are required: one for local communication and one to connect to the remote site.
In JBoss Data Grid's Library mode, cross-datacenter replication is set up as follows:

Procedure 31.2. Setting Up Cross-Datacenter Replication

  1. Configure the Local Site

    <infinispan>
       <global>      
          <site local="SFO" />
          <transport clusterName="default"> 
             <properties> 
                 <property name="configurationFile" value="jgroups-with-relay.xml"/>     
              </properties> 
          </transport>
          <!-- Additional configuration information here -->
       </global>
       <!-- Additional configuration information here -->
       <namedCache name="lonBackup">
          <sites>
             <backupFor remoteSite="LON" 
    		    remoteCache="lon" />
          </sites>
       </namedCache>
    </infinispan>
    1. Add the site element to the global element to add the local site (in this example, the local site is named LON).
    2. Cross-site replication requires a non-default JGroups configuration. Add the transport element and set up the path to the configuration file as the configurationFile property. In this example, the JGroups configuration file is named jgroups-with-relay.xml.
    3. Configure the cache in site LON to back up to the sites NYC and SFO.
    4. Configure the back up caches:
      1. Configure the cache in site NYC to receive back up data from LON.
      2. Configure the cache in site SFO to receive back up data from LON.
  2. Add the Contents of the Configuration File

    As a default, Red Hat JBoss Data Grid includes JGroups configuration files such as default-configs/default-jgroups-tcp.xml and default-configs/default-jgroups-udp.xml in the infinispan-embedded-{VERSION}.jar package.
    Copy the JGroups configuration to a new file (in this example, it is named jgroups-with-relay.xml) and add the provided configuration information to this file. Note that the relay.RELAY2 protocol configuration must be the last protocol in the configuration stack.
    <config> 
        ... 
        <relay.RELAY2 site="LON" 
                  config="relay.xml"
                  relay_multicasts="false" />
    </config>
  3. Configure the relay.xml File

    Set up the relay.RELAY2 configuration in the relay.xml file. This file describes the global cluster configuration.
    <RelayConfiguration> 
        <sites> 
            <site name="LON" 
                  id="0"> 
                <bridges> 
                    <bridge config="jgroups-global.xml" 
                            name="global"/> 
                    </bridges> 
            </site>  
            <site name="NYC" 
                  id="1"> 
                <bridges> 
                    <bridge config="jgroups-global.xml" 
                            name="global"/> 
                    </bridges> 
            </site>  
            <site name="SFO" 
                  id="2"> 
                <bridges> 
                    <bridge config="jgroups-global.xml" 
                            name="global"/> 
                </bridges> 
            </site> 
        </sites> 
    </RelayConfiguration>
  4. Configure the Global Cluster

    The file jgroups-global.xml referenced in relay.xml contains another JGroups configuration which is used for the global cluster: communication between sites.
    The global cluster configuration is usually TCP-based and uses the TCPPING protocol (instead of PING or MPING) to discover members. Copy the contents of default-configs/default-jgroups-tcp.xml into jgroups-global.xml and add the following configuration in order to configure TCPPING:
    <config> 
        <TCP bind_port="7800" ... /> 
        <TCPPING initial_hosts="lon.hostname[7800],nyc.hostname[7800],sfo.hostname[7800]"
                 ergonomics="false" /> 
              <!-- Rest of the protocols --> 
    </config>
    Replace the hostnames (or IP addresses) in TCPPING.initial_hosts with those used for your site masters. The ports (7800 in this example) must match the TCP.bind_port.
    For more information about the TCPPING protocol, see Section 26.2.1.3, “Using the TCPPing Protocol”

31.2.2.2. Configure Cross-Datacenter Replication Programmatically

The programmatic method to configure cross-datacenter replication in Red Hat JBoss Data Grid is as follows:

Procedure 31.3. Configure Cross-Datacenter Replication Programmatically

  1. Identify the Node Location

    Declare the site the node resides in:
    globalConfiguration.site().localSite("LON");
  2. Configure JGroups

    Configure JGroups to use the RELAY protocol:
    globalConfiguration.transport().addProperty("configurationFile", jgroups-with-relay.xml);
  3. Set Up the Remote Site

    Set up JBoss Data Grid caches to replicate to the remote site:
    ConfigurationBuilder lon = new ConfigurationBuilder();
    lon.sites().addBackup()
          .site("NYC")
          .backupFailurePolicy(BackupFailurePolicy.WARN)
          .strategy(BackupConfiguration.BackupStrategy.SYNC)
          .replicationTimeout(12000)
          .sites().addInUseBackupSite("NYC")
        .sites().addBackup()
          .site("SFO")
          .backupFailurePolicy(BackupFailurePolicy.IGNORE)
          .strategy(BackupConfiguration.BackupStrategy.ASYNC)
          .sites().addInUseBackupSite("SFO")
  4. Optional: Configure the Backup Caches

    JBoss Data Grid implicitly replicates data to a cache with same name as the remote site. If a backup cache on the remote site has a different name, users must specify a backupFor cache to ensure data is replicated to the correct cache.

    Note

    This step is optional and only required if the remote site's caches are named differently from the original caches.
    1. Configure the cache in site NYC to receive backup data from LON:
      ConfigurationBuilder NYCbackupOfLon = new ConfigurationBuilder();
      NYCbackupOfLon.sites().backupFor().remoteCache("lon").remoteSite("LON");
    2. Configure the cache in site SFO to receive backup data from LON:
      ConfigurationBuilder SFObackupOfLon = new ConfigurationBuilder();
      SFObackupOfLon.sites().backupFor().remoteCache("lon").remoteSite("LON");
  5. Add the Contents of the Configuration File

    As a default, Red Hat JBoss Data Grid includes JGroups configuration files such as default-configs/default-jgroups-tcp.xml and default-configs/default-jgroups-udp.xml in the infinispan-embedded-{VERSION}.jar package.
    Copy the JGroups configuration to a new file (in this example, it is named jgroups-with-relay.xml) and add the provided configuration information to this file. Note that the relay.RELAY2 protocol configuration must be the last protocol in the configuration stack.
    <config> 
        <!-- Additional configuration information here --> 
        <relay.RELAY2 site="LON" 
                  config="relay.xml"
                  relay_multicasts="false" /> 
    </config>
  6. Configure the relay.xml File

    Set up the relay.RELAY2 configuration in the relay.xml file. This file describes the global cluster configuration.
    <RelayConfiguration> 
        <sites> 
            <site name="LON" 
                  id="0"> 
                <bridges> 
                    <bridge config="jgroups-global.xml" 
                            name="global"/> 
                    </bridges> 
            </site>  
            <site name="NYC" 
                  id="1"> 
                <bridges> 
                    <bridge config="jgroups-global.xml" 
                            name="global"/> 
                    </bridges> 
            </site>  
            <site name="SFO" 
                  id="2"> 
                <bridges> 
                    <bridge config="jgroups-global.xml" 
                            name="global"/> 
                </bridges> 
            </site> 
        </sites> 
    </RelayConfiguration>
  7. Configure the Global Cluster

    The file jgroups-global.xml referenced in relay.xml contains another JGroups configuration which is used for the global cluster: communication between sites.
    The global cluster configuration is usually TCP-based and uses the TCPPING protocol (instead of PING or MPING) to discover members. Copy the contents of default-configs/default-jgroups-tcp.xml into jgroups-global.xml and add the following configuration in order to configure TCPPING:
    <config> 
        <TCP bind_port="7800" <!-- Additional configuration information here --> /> 
        <TCPPING initial_hosts="lon.hostname[7800],nyc.hostname[7800],sfo.hostname[7800]"
                 ergonomics="false" /> 
              <!-- Rest of the protocols --> 
    </config>
    Replace the hostnames (or IP addresses) in TCPPING.initial_hosts with those used for your site masters. The ports (7800 in this example) must match the TCP.bind_port.
    For more information about the TCPPING protocol, see Section 26.2.1.3, “Using the TCPPing Protocol”