Chapter 11. Configuring Data Grid for OpenShift Clusters
11.1. Configuring Cluster Discovery
Data Grid for OpenShift can use either the Kubernetes or DNS discovery mechanisms for clustering. These discovery mechanisms enable images to automatically join clusters.
Data Grid for OpenShift templates and services use DNS by default. If you deploy Data Grid for OpenShift directly from the image or custom template, you must configure the appropriate discovery mechanism.
11.1.1. Configuring DNS_PING
To configure the DNS discovery mechanism for clustering, do the following:
Set
openshift.DNS_PING
as the value for theJGROUPS_PING_PROTOCOL
environment variable.JGROUPS_PING_PROTOCOL=openshift.DNS_PING
Specify the name of the ping service for the cluster as the value for the
OPENSHIFT_DNS_PING_SERVICE_NAME
environment variable.OPENSHIFT_DNS_PING_SERVICE_NAME=${PING_SERVICE_NAME}
Specify the port number where the ping service is exposed as the value for the
OPENSHIFT_DNS_PING_SERVICE_PORT
environment variable. The default value is8888
.OPENSHIFT_DNS_PING_SERVICE_PORT=${PING_SERVICE_NAME}
Define a ping service that exposes the ping port, as in the following example:
apiVersion: v1 kind: Service spec: clusterIP: None ports: - name: ping port: 8888 protocol: TCP targetPort: 8888 selector: deploymentConfig=datagrid-service metadata: annotations: description: The JGroups ping port for clustering. service.alpha.kubernetes.io/tolerate-unready-endpoints: 'true'
ImportantYou should configure
clusterIP: None
so that the service is headless. Likewise, the ping port must be named and include theservice.alpha.kubernetes.io/tolerate-unready-endpoints: 'true'
annotation.
11.1.2. Configuring KUBE_PING
To configure the Kubernetes discovery mechanism for clustering, do the following:
Set
openshift.KUBE_PING
as the value for theJGROUPS_PING_PROTOCOL
environment variable.JGROUPS_PING_PROTOCOL=openshift.KUBE_PING
Specify the OpenShift project name as the value for the
OPENSHIFT_KUBE_PING_NAMESPACE
environment variable. If you do not set this variable, the server behaves like a single-node cluster.OPENSHIFT_KUBE_PING_NAMESPACE=${PING_NAMESPACE}
Specify a cluster label with the
OPENSHIFT_KUBE_PING_LABELS
environment variable. If you do not set this variable, pods outside the application but in the same namespace attempt to join.OPENSHIFT_KUBE_PING_LABELS=labelKey=labelValue
Grant authorization to the service account the pod is running under so that it can access the Kubernetes REST API. For example, grant authorization to datagrid-service-account as follows:
oc policy add-role-to-user view \ system:serviceaccount:$(oc project -q):datagrid-service-account \ -n $(oc project -q)
Ensure port
8888
is defined as a ping port on the pod container, as follows:ports: - containerPort: 8888 name: ping protocol: TCP
11.2. Configuring JGroups Encryption
Data Grid for OpenShift uses JGroups technology to secure traffic between clustered servers with the following options:
- Authentication
Uses the JGroups
AUTH
protocol that requires nodes to authenticate with a password when joining the cluster.You configure authentication with the
JGROUPS_CLUSTER_PASSWORD
environment variable. This environment variable sets a password for nodes to use when joining the cluster. The password must be the same across the cluster.- Symmetric encryption
Uses the JGroups
SYM_ENCRYPT
protocol to secure traffic with a JGroups keystore (.jceks
). This is the default encryption protocol.The JGroups
AUTH
protocol is optional with symmetric encryption.The JGroups keystore contains credentials that each node in the cluster uses to secure communication.
- Asymmetric encryption
Uses the JGroups
ASYM_ENCRYPT
protocol to secure traffic with public/private key encryption.The JGroups
AUTH
protocol is required with asymmetric encryption.The coordinator node generates a secret key. When a node joins the cluster, it requests the secret key from the coordinator and provides its public key. The coordinator encrypts the secret key with the public key and returns it to the node. The node then decrypts and installs the secret so that it can securely communicate with other nodes in the cluster.
11.2.1. Setting Up Symmetric Encryption
To use symmetric encryption, do the following:
Create a JGroups keystore (
.jceks
) that contains credentials to encrypt traffic.You can use the Java keytool to generate a JGroups keystore.
Deploy the JGroups keystore to OpenShift as a secret.
- Log in to your OpenShift cluster.
Create a secret for the JGroups keystore. For example, to create a secret named
jgroups-secret
from a keystore namedjgroups.jceks
, do the following:$ oc create secret generic jgroups-secret \ --from-file=jgroups.jceks
Link the secret to the default service account.
$ oc secrets link default jgroups-secret
Mount the secret to the container.
$ oc set volumes dc/datagrid \ --add -t secret \ --secret-name='jgroups-secret' \ --mount-path='/keystores/jgroups'
-
Set the value of the
JGROUPS_ENCRYPT_PROTOCOL
environment variable toSYM_ENCRYPT
for each node in the cluster. Configure each node in the cluster to use the JGroups keystore with the following environment variables:
JGROUPS_ENCRYPT_KEYSTORE
- Specifes the JGroups keystore for encrypting cluster traffic.
JGROUPS_ENCRYPT_KEYSTORE_DIR
- Specifies the directory where the JGroups keystore resides.
JGROUPS_ENCRYPT_SECRET
- Matches the OpenShift secret for the keystore.
JGROUPS_ENCRYPT_NAME
- Matches the username for the keystore.
JGROUPS_ENCRYPT_PASSWORD
- Matches the keystore password.
-
If required, set a password for nodes to use when joining the cluster with the
JGROUPS_CLUSTER_PASSWORD
environment variable.
11.2.2. Setting Up Asymmetric Encryption
To use asymmetric encryption, do the following:
-
Configure authentication with the
JGROUPS_CLUSTER_PASSWORD
environment variable. -
Set the value of the
JGROUPS_ENCRYPT_PROTOCOL
environment variable toASYM_ENCRYPT
for each node in the cluster.