Chapter 11. Configuring Data Grid for OpenShift Clusters

11.1. Configuring Cluster Discovery

Data Grid for OpenShift can use either the Kubernetes or DNS discovery mechanisms for clustering. These discovery mechanisms enable images to automatically join clusters.

Data Grid for OpenShift templates and services use DNS by default. If you deploy Data Grid for OpenShift directly from the image or custom template, you must configure the appropriate discovery mechanism.

11.1.1. Configuring DNS_PING

To configure the DNS discovery mechanism for clustering, do the following:

  1. Set openshift.DNS_PING as the value for the JGROUPS_PING_PROTOCOL environment variable.

    JGROUPS_PING_PROTOCOL=openshift.DNS_PING
  2. Specify the name of the ping service for the cluster as the value for the OPENSHIFT_DNS_PING_SERVICE_NAME environment variable.

    OPENSHIFT_DNS_PING_SERVICE_NAME=${PING_SERVICE_NAME}
  3. Specify the port number where the ping service is exposed as the value for the OPENSHIFT_DNS_PING_SERVICE_PORT environment variable. The default value is 8888.

    OPENSHIFT_DNS_PING_SERVICE_PORT=${PING_SERVICE_NAME}
  4. Define a ping service that exposes the ping port, as in the following example:

    apiVersion: v1
    kind: Service
    spec:
      clusterIP: None
      ports:
        - name: ping
          port: 8888
          protocol: TCP
          targetPort: 8888
      selector: deploymentConfig=datagrid-service
    metadata:
      annotations:
        description: The JGroups ping port for clustering.
        service.alpha.kubernetes.io/tolerate-unready-endpoints: 'true'
    Important

    You should configure clusterIP: None so that the service is headless. Likewise, the ping port must be named and include the service.alpha.kubernetes.io/tolerate-unready-endpoints: 'true' annotation.

11.1.2. Configuring KUBE_PING

To configure the Kubernetes discovery mechanism for clustering, do the following:

  1. Set openshift.KUBE_PING as the value for the JGROUPS_PING_PROTOCOL environment variable.

    JGROUPS_PING_PROTOCOL=openshift.KUBE_PING
  2. Specify the OpenShift project name as the value for the OPENSHIFT_KUBE_PING_NAMESPACE environment variable. If you do not set this variable, the server behaves like a single-node cluster.

    OPENSHIFT_KUBE_PING_NAMESPACE=${PING_NAMESPACE}
  3. Specify a cluster label with the OPENSHIFT_KUBE_PING_LABELS environment variable. If you do not set this variable, pods outside the application but in the same namespace attempt to join.

    OPENSHIFT_KUBE_PING_LABELS=labelKey=labelValue
  4. Grant authorization to the service account the pod is running under so that it can access the Kubernetes REST API. For example, grant authorization to datagrid-service-account as follows:

    oc policy add-role-to-user view \
      system:serviceaccount:$(oc project -q):datagrid-service-account \
      -n $(oc project -q)
  5. Ensure port 8888 is defined as a ping port on the pod container, as follows:

    ports:
        - containerPort: 8888
          name: ping
          protocol: TCP

11.2. Configuring JGroups Encryption

Data Grid for OpenShift uses JGroups technology to secure traffic between clustered servers with the following options:

Authentication

Uses the JGroups AUTH protocol that requires nodes to authenticate with a password when joining the cluster.

You configure authentication with the JGROUPS_CLUSTER_PASSWORD environment variable. This environment variable sets a password for nodes to use when joining the cluster. The password must be the same across the cluster.

Symmetric encryption

Uses the JGroups SYM_ENCRYPT protocol to secure traffic with a JGroups keystore (.jceks). This is the default encryption protocol.

The JGroups AUTH protocol is optional with symmetric encryption.

The JGroups keystore contains credentials that each node in the cluster uses to secure communication.

Asymmetric encryption

Uses the JGroups ASYM_ENCRYPT protocol to secure traffic with public/private key encryption.

The JGroups AUTH protocol is required with asymmetric encryption.

The coordinator node generates a secret key. When a node joins the cluster, it requests the secret key from the coordinator and provides its public key. The coordinator encrypts the secret key with the public key and returns it to the node. The node then decrypts and installs the secret so that it can securely communicate with other nodes in the cluster.

11.2.1. Setting Up Symmetric Encryption

To use symmetric encryption, do the following:

  1. Create a JGroups keystore (.jceks) that contains credentials to encrypt traffic.

    You can use the Java keytool to generate a JGroups keystore.

  2. Deploy the JGroups keystore to OpenShift as a secret.

    1. Log in to your OpenShift cluster.
    2. Create a secret for the JGroups keystore. For example, to create a secret named jgroups-secret from a keystore named jgroups.jceks, do the following:

      $ oc create secret generic jgroups-secret \
        --from-file=jgroups.jceks
    3. Link the secret to the default service account.

      $ oc secrets link default jgroups-secret
    4. Mount the secret to the container.

      $ oc set volumes dc/datagrid \
        --add -t secret \
        --secret-name='jgroups-secret' \
        --mount-path='/keystores/jgroups'
  3. Set the value of the JGROUPS_ENCRYPT_PROTOCOL environment variable to SYM_ENCRYPT for each node in the cluster.
  4. Configure each node in the cluster to use the JGroups keystore with the following environment variables:

    JGROUPS_ENCRYPT_KEYSTORE
    Specifes the JGroups keystore for encrypting cluster traffic.
    JGROUPS_ENCRYPT_KEYSTORE_DIR
    Specifies the directory where the JGroups keystore resides.
    JGROUPS_ENCRYPT_SECRET
    Matches the OpenShift secret for the keystore.
    JGROUPS_ENCRYPT_NAME
    Matches the username for the keystore.
    JGROUPS_ENCRYPT_PASSWORD
    Matches the keystore password.
  5. If required, set a password for nodes to use when joining the cluster with the JGROUPS_CLUSTER_PASSWORD environment variable.

11.2.2. Setting Up Asymmetric Encryption

To use asymmetric encryption, do the following:

  1. Configure authentication with the JGROUPS_CLUSTER_PASSWORD environment variable.
  2. Set the value of the JGROUPS_ENCRYPT_PROTOCOL environment variable to ASYM_ENCRYPT for each node in the cluster.