Chapter 3. Performing Cross-Site Replication Operations

Bring sites online and offline. Transfer cache state to remote sites.

3.1. Performing Cross-Site Operations with the CLI

The Data Grid command line interface lets you remotely connect to Data Grid servers, manage sites, and push state transfer to backup locations.

Prerequisites

  • Start the Data Grid CLI.
  • Connect to a running Data Grid cluster.

3.1.1. Bringing Backup Locations Offline and Online

Take backup locations offline manually and bring them back online.

Procedure

  1. Create a CLI connection to Data Grid.

  2. Check if backup locations are online or offline with the site status command: 

    //containers/default]> site status --cache=cacheName --site=NYC
    Note

    --site is an optional argument. If not set, the CLI returns all backup locations.

  3. Manage backup locations as follows:

    • Bring backup locations online with the bring-online command: 

      //containers/default]> site bring-online --cache=customers --site=NYC

    • Take backup locations offline with the take-offline command: 

      //containers/default]> site take-offline --cache=customers --site=NYC

For more information and examples, run the help site command.

3.1.2. Pushing State to Backup Locations

Transfer cache state to remote backup locations.

Procedure

  1. Create a CLI connection to Data Grid.

  2. Use the site command to push state transfer, as in the following example: 

    //containers/default]> site push-site-state --cache=cacheName --site=NYC

For more information and examples, run the help site command.

Reference

Data Grid Command Line Interface

3.2. Performing Cross-Site Operations with the REST API

Data Grid servers provide a REST API that allows you to perform cross-site operations.

3.2.1. Getting Status of All Backup Locations

Retrieve the status of all backup locations with GET requests. 

GET /v2/caches/{cacheName}/x-site/backups/

Data Grid responds with the status of each backup location in JSON format, as in the following example: 

{
  "NYC": "online",
  "LON": "offline"
}

Table 3.1. Returned Status

ValueDescription

online

All nodes in the local cluster have a cross-site view with the backup location.

offline

No nodes in the local cluster have a cross-site view with the backup location.

mixed

Some nodes in the local cluster have a cross-site view with the backup location, other nodes in the local cluster do not have a cross-site view. The response indicates status for each node.

3.2.2. Getting Status of Specific Backup Locations

Retrieve the status of a backup location with GET requests. 

GET /v2/caches/{cacheName}/x-site/backups/{siteName}

Data Grid responds with the status of each node in the site in JSON format, as in the following example: 

{
  "NodeA":"offline",
  "NodeB":"online"
}

Table 3.2. Returned Status

ValueDescription

online

The node is online.

offline

The node is offline.

failed

Not possible to retrieve status. The remote cache could be shutting down or a network error occurred during the request.

3.2.3. Taking Backup Locations Offline

Take backup locations offline with POST requests and the ?action=take-offline parameter. 

POST /v2/caches/{cacheName}/x-site/backups/{siteName}?action=take-offline

3.2.4. Bringing Backup Locations Online

Bring backup locations online with the ?action=bring-online parameter. 

POST /v2/caches/{cacheName}/x-site/backups/{siteName}?action=bring-online

3.2.5. Pushing State to Backup Locations

Push cache state to a backup location with the ?action=start-push-state parameter. 

POST /v2/caches/{cacheName}/x-site/backups/{siteName}?action=start-push-state

3.2.6. Canceling State Transfer

Cancel state transfer operations with the ?action=cancel-push-state parameter. 

POST /v2/caches/{cacheName}/x-site/backups/{siteName}?action=cancel-push-state

3.2.7. Getting State Transfer Status

Retrieve status of state transfer operations with the ?action=push-state-status parameter. 

GET /v2/caches/{cacheName}/x-site/backups?action=push-state-status

Data Grid responds with the status of state transfer for each backup location in JSON format, as in the following example: 

{
   "NYC":"CANCELED",
   "LON":"OK"
}

Table 3.3. Returned Status

ValueDescription

SENDING

State transfer to the backup location is in progress.

OK

State transfer completed successfully.

ERROR

An error occurred with state transfer. Check log files.

CANCELLING

State transfer cancellation is in progress.

3.2.8. Clearing State Transfer Status

Clear state transfer status for sending sites with the ?action=clear-push-state-status parameter. 

POST /v2/caches/{cacheName}/x-site/local?action=clear-push-state-status

3.2.9. Modifying Take Offline Conditions

Sites go offline if certain conditions are met. Modify the take offline parameters to control when backup locations automatically go offline.

Procedure

  1. Check configured take offline parameters with GET requests and the take-offline-config parameter. 

    GET /v2/caches/{cacheName}/x-site/backups/{siteName}/take-offline-config

    The Data Grid response includes after_failures and min_wait fields as follows: 

    {
  "after_failures": 2,
  "min_wait": 1000
}

  2. Modify take offline parameters in the body of PUT requests. 

    PUT /v2/caches/{cacheName}/x-site/backups/{siteName}/take-offline-config

3.2.10. Canceling State Transfer from Receiving Sites

If the connection between two backup locations breaks, you can cancel state transfer on the site that is receiving the push.

Cancel state transfer from a remote site and keep the current state of the local cache with the ?action=cancel-receive-state parameter. 

POST /v2/caches/{cacheName}/x-site/backups/{siteName}?action=cancel-receive-state

3.2.11. Getting Status of Backup Locations

Retrieve the status of all backup locations from Cache Managers with GET requests. 

GET /rest/v2/cache-managers/{cacheManagerName}/x-site/backups/

Data Grid responds with status in JSON format, as in the following example: 

{
   "SFO-3":{
      "status":"online"
   },
   "NYC-2":{
      "status":"mixed",
      "online":[
         "CACHE_1"
      ],
      "offline":[
         "CACHE_2"
      ]
   }
}

Table 3.4. Returned Status

ValueDescription

online

All nodes in the local cluster have a cross-site view with the backup location.

offline

No nodes in the local cluster have a cross-site view with the backup location.

mixed

Some nodes in the local cluster have a cross-site view with the backup location, other nodes in the local cluster do not have a cross-site view. The response indicates status for each node.

3.2.12. Taking Backup Locations Offline

Take backup locations offline with the ?action=take-offline parameter. 

POST /rest/v2/cache-managers/{cacheManagerName}/x-site/backups/{siteName}?action=take-offline

3.2.13. Bringing Backup Locations Online

Bring backup locations online with the ?action=bring-online parameter. 

POST /rest/v2/cache-managers/{cacheManagerName}/x-site/backups/{siteName}?action=bring-online

3.2.14. Starting State Transfer

Push state of all caches to remote sites with the ?action=start-push-state parameter. 

POST /rest/v2/cache-managers/{cacheManagerName}/x-site/backups/{siteName}?action=start-push-state

3.2.15. Canceling State Transfer

Cancel ongoing state transfer operations with the ?action=cancel-push-state parameter. 

POST /rest/v2/cache-managers/{cacheManagerName}/x-site/backups/{siteName}?action=cancel-push-state

3.3. Performing Cross-Site Operations with JMX

Data Grid provides JMX tooling to perfrom cross-site operations such as pushing state transfer and bringing sites online.

3.3.1. Configuring Data Grid to Register JMX MBeans

Data Grid can register JMX MBeans that you can use to collect statistics and perform administrative operations. However, you must enable statistics separately to JMX otherwise Data Grid provides 0 values for all statistic attributes.

Procedure

  • Enable JMX declaratively or programmatically.

Declaratively

<cache-container>
  <jmx enabled="true" /> 1
</cache-container>

1
Registers Data Grid JMX MBeans.

Programmatically

GlobalConfiguration globalConfig = new GlobalConfigurationBuilder()
  .jmx().enable() 1
  .build();

1
Registers Data Grid JMX MBeans.

3.3.2. Performing Cross-Site Operations

Perform cross-site operations via JMX clients.

Prerequisites

  • Configure Data Grid to register JMX MBeans

Procedure

  1. Connect to Data Grid with any JMX client.

  2. Invoke operations from the following MBeans:

    • XSiteAdmin provides cross-site operations for caches.

    • GlobalXSiteAdminOperations provides cross-site operations for Cache Managers.

      For example, to bring sites back online, invoke bringSiteOnline(siteName).

See the Data Grid JMX Components documentation for details about available cross-site operations.

Reference