Chapter 12. Configuring Cross-Site Replication

Set up cross-site replication to back up data between Data Grid clusters running in different locations.

For example, you use Data Grid Operator to manage a Data Grid cluster at a data center in London, LON. At another data center in New York City, NYC, you also use Data Grid Operator to manage a Data Grid cluster. In this case, you can add LON and NYC as backup locations for each other.

Important

Cross-site replication functionality is currently Technology Preview. Contact Red Hat support for more information.

Prerequisites

12.1. Data Grid Cluster and Project Naming

Data Grid Operator expects Data Grid clusters in each site to have the same cluster names and be running in matching namespaces.

For example, in the LON site you create a Data Grid cluster with metadata.name: mydatagrid in a OpenShift project named "my-xsite". In this case you must create Data Grid clusters in other backup locations, such as NYC, with identical names in matching namespaces.

In effect, you must create Data Grid cluster names and OpenShift namespaces at each backup location that mirror one another.

12.2. Creating Service Account Tokens

Traffic between independent OpenShift installations occurs through a Kubernetes API. OpenShift Container Platform clusters use tokens to authenticate with and access the API.

To enable cross-site replication between Data Grid clusters you must add tokens to the namespace on each site. For example, LON needs a secret with the token for NYC. NYC also needs a secret with the token for LON.

Procedure

  1. Create service accounts on each OpenShift instance.

    For example, create a service account on LON as follows:

    $ oc create sa lon
    serviceaccount/lon created
  2. Add the view role to service accounts.

    For example, if your Data Grid cluster runs in the "my-xsite" namespace, add the view role to the service account on LON as follows:

    $ oc policy add-role-to-user view system:serviceaccount:my-xsite:lon
  3. Retrieve tokens from each service account.

    The following example shows the service account token for LON:

    $ oc sa get-token lon
    
    eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9...
  4. Create secrets that contain service account tokens for the backup locations.

    1. Log in to OpenShift Container Platform at NYC.
    2. Add the service account token to a lon-token secret.

      oc create secret generic lon-token --from-literal=token=eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9...
    3. Repeat the preceding steps to create a nyc-token secret on LON.

After you add service account tokens to each backup location, the OpenShift instances can authenticate with each other so that Data Grid clusters can form cross-site views.

12.3. Adding Backup Locations to Data Grid Clusters

Configure Data Grid clusters as backup locations so that they can communicate over a dedicated JGroups transport channel for replicating data.

Procedure

  1. Configure Data Grid clusters at each site with the Infinispan CR as necessary.

    For example, create lon.yaml to configure LON and nyc.yaml to configure NYC. Both configurations must include the following:

    • .spec.service.sites.local names the local site for Data Grid clusters.
    • .spec.service.sites.locations provides the location of all site masters. Data Grid nodes use this information to connect with each other and form cross-site views.
  2. Instantiate Data Grid clusters at each site, for example:

    1. Apply the Infinispan CR for LON.

      $ oc apply -f lon.yaml
    2. Log in to OpenShift Container Platform at NYC.
    3. Apply the Infinispan CR for NYC.

      $ oc apply -f nyc.yaml
  3. Verify that Data Grid clusters form a cross-site view.

    For example, do the following on LON:

    $ oc logs example-rhdatagrid-0 | grep x-site
    
    INFO  [org.infinispan.XSITE] (jgroups-5,example-rhdatagrid-0-<id>) ISPN000439: Received new x-site view: [NYC]
    INFO  [org.infinispan.XSITE] (jgroups-7,example-rhdatagrid-0-<id>) ISPN000439: Received new x-site view: [NYC, LON]

12.3.1. Cross-Site Replication Resources

spec:
  ...
  service:
    type: DataGrid 1
    sites:
      local:
        name: LON 2
        expose:
          type: LoadBalancer 3
      locations: 4
      - name: LON 5
        url: openshift://api.site-a.devcluster.openshift.com:6443 6
        secretName: lon-token 7
      - name: NYC
        url: openshift://api.site-b.devcluster.openshift.com:6443
        secretName: nyc-token
1
Specifies Data Grid service. Data Grid supports cross-site replication with Data Grid service clusters only.
2
Names the local site for a Data Grid cluster.
3
Specifies LoadBalancer as the service that handles communication between backup locations.
4
Provides connection information for all backup locations.
5
Specifies a backup location that matches .spec.service.sites.local.name.
6
Specifies the URL of the Kubernetes API for the backup location.
7
Specifies the secret that contains the service account token for the backup site.