Chapter 3. Setting Up Data Grid for OpenShift Services

3.1. Data Grid for OpenShift Services

Data Grid services are stateful applications that you can easily scale up or down without losing data.

cache-service

An easy-to-use Data Grid for OpenShift cluster designed to accelerate application response time with high-performance caching.

  • Data in memory is distributed evenly across nodes. You define the initial size of the Data Grid cluster when you create the service. Distribution is also synchronous. When propagating data to another node, the sending node waits for the operation to complete before the thread continues.
  • Single copies of cache entries by default. If a pod restarts, data in that pod is lost. For more resiliency with your data, you can easily enable replication when you create the service.
  • Cache entries are stored off-heap for JVM efficiency. When the cache size reaches the amount of memory available to the pod, entries are evicted. You can optionally change the eviction policy to throw a ContainerFullException.
datagrid-service
A full distribution of Data Grid for OpenShift that lets you create multiple different cache configurations. Gives you advanced capabilities such as indexing and querying as well as Prometheus monitoring.

3.1.1. Container Storage

cache-service and datagrid-service containers have storage volumes mounted at /opt/datagrid/standalone/data.

The volume size is 1GB by default. You can adjust the size with datagrid-service but not cache-service.

Ephemeral or Permanent
When you remotely create caches, you control whether they are ephemeral or permanent. Permanent caches survive container restarts because the cache definitions are saved in the storage volume. Default caches are always permanent.
Persistent
datagrid-service lets you persist cache entries and indexes to the storage volume. If you require more guarantees for your data, you can optionally persist to external file-based storage or a database via a cache store.

3.1.2. Partition Handling

By default, Data Grid for OpenShift services use a partition handling configuration to ensure data consistency.

  • DENY_READ_WRITES conflict resolution strategy that denies read and write operations for cache entries unless all the owners of a segment are in the same partition.
  • REMOVE_ALL merge policy that removes entries from the cache when conflicts are detected.
Note

Network partitioning applies only when data is replicated across a cluster.

3.1.3. Confirming Service Availability

The templates for cache-service and datagrid-service are available on Red Hat OpenShift Online and Red Hat OpenShift Container Platform in the openshift namespace.

Run the following command to verify that the service templates are available:

$ oc get templates -n openshift | grep 'cache-service\|datagrid-service'

3.1.3.1. Importing Templates

If necessary, import cache-service and datagrid-service as follows:

  1. Log in to your OpenShift cluster.
  2. Import the service templates:

    $ for resource in cache-service-template.yaml \
      datagrid-service-template.yaml
    do
      oc create -n openshift -f \
      https://raw.githubusercontent.com/jboss-container-images/jboss-datagrid-7-openshift-image/7.3-v1.8/services/${resource}
    done
    Tip

    Overwrite existing templates with oc replace --force.

3.2. Creating Cache Services

Use cache-service to quickly set up clusters that give you optimal performance and ease of use with minimal configuration.

  1. Create the service with the new-app command.
  2. Set template parameters and environment variables as appropriate.

For Example:

  • Create cache-service with minimal configuration:

    $ oc new-app cache-service \
      -p APPLICATION_USER=${USERNAME} \
      -p APPLICATION_PASSWORD=${PASSWORD}
  • Create a cache-service cluster with three nodes and data replication:

    $ oc new-app cache-service \
      -p APPLICATION_USER=${USERNAME} \
      -p APPLICATION_PASSWORD=${PASSWORD} \
      -p NUMBER_OF_INSTANCES=3 \
      -p REPLICATION_FACTOR=2

Template Parameters

  • APPLICATION_NAME specifies a name for the application. The default is cache-service.
  • NUMBER_OF_INSTANCES sets the number of nodes in the Data Grid for OpenShift cluster. The default is 1.
  • TOTAL_CONTAINER_MEM configures the total amount of memory, in MiB, available to the container. The default is 512.
  • APPLICATION_USER creates a user to securely access the cache. There is no default value. You must always create a user.
  • APPLICATION_PASSWORD specifies a password for the user. If you do not set a password, the service template randomly generates one and stores it as a secret.
  • REPLICATION_FACTOR specifies the number of copies for each cache entry. The default is 1.
  • EVICTION_POLICY defines how cache-service behaves when the size of the cache reaches the amount of memory available to the pod. There are two values:

    • evict removes entries from the cache. This is the default.
    • reject throws ContainerFullException instead of adding new entries.
Note

On IBM Z or IBM Power, you must provide the IMAGE=registry.redhat.io/jboss-datagrid-7/datagrid73-openj9-11-openshift:latest parameter.

Environment Variables

  • AB_PROMETHEUS_ENABLE allows you to collect JMX metrics to monitor and analyze Data Grid and has the following values:

    false
    Disables monitoring with the default Prometheus agent.
    true
    Enables monitoring with the default Prometheus agent. The Prometheus Operator must be installed and running. You must also Set Up Monitoring after you create the service.
  • AB_PROMETHEUS_JMX_EXPORTER_PORT defines the port on which Data Grid publishes JMX metrics. The default is 9779.

Verifying the Application

Command output displays parameter values and resources when you create cache-service, as in the following example:

--> Deploying template "rhdg-helloworld/cache-service" to project rhdg-helloworld

     Red Hat Cache Service
     ---------
     Red Hat Data Grid is an in-memory, distributed key/value store.

     * With parameters:
        ...

--> Creating resources ...
    secret "cache-service" created
    service "cache-service-ping" created
    service "cache-service" created
    statefulset "cache-service" created
--> Success
    ...

3.3. Creating Data Grid Services

Use datagrid-service to set up a cluster that you can use with different cache configurations and more complex Data Grid capabilities.

  1. Create the service with the new-app command.
  2. Set template parameters and environment variables as appropriate.

For Example:

  • Create datagrid-service with minimal configuration:

    $ oc new-app datagrid-service \
      -p APPLICATION_USER=${USERNAME} \
      -p APPLICATION_PASSWORD=${PASSWORD}
  • Create a datagrid-service cluster with three nodes and monitoring enabled:

    $ oc new-app datagrid-service \
      -p APPLICATION_USER=${USERNAME} \
      -p APPLICATION_PASSWORD=${PASSWORD} \
      -p NUMBER_OF_INSTANCES=3
      -e AB_PROMETHEUS_ENABLE=true

Template Parameters

  • APPLICATION_NAME specifies a name for the application. The default is datagrid-service.
  • NUMBER_OF_INSTANCES sets the number of nodes in the Data Grid for OpenShift cluster. The default is 1.
  • TOTAL_CONTAINER_MEM configures the total amount of memory, in MiB, available to the container. The default is 512.
  • APPLICATION_USER creates a user to securely access the cache. There is no default value. You must always create a user.
  • APPLICATION_PASSWORD specifies a password for the user. If you do not set a password, the service template randomly generates one and stores it as a secret.
  • REPLICATION_FACTOR specifies the number of copies for each cache entry. The default is 2.
  • TOTAL_CONTAINER_STORAGE configures the size, in GiB, of the file-based storage volume. The default is 1.
Note

On IBM Z or IBM Power, you must provide the IMAGE=registry.redhat.io/jboss-datagrid-7/datagrid73-openj9-11-openshift:latest parameter.

Environment Variables

  • AB_PROMETHEUS_ENABLE allows you to collect JMX metrics to monitor and analyze Data Grid and has the following values:

    false
    Disables monitoring with the default Prometheus agent.
    true
    Enables monitoring with the default Prometheus agent. The Prometheus Operator must be installed and running. You must also Set Up Monitoring after you create the service.
  • AB_PROMETHEUS_JMX_EXPORTER_PORT defines the port on which Data Grid publishes JMX metrics. The default is 9779.

Verifying the Application

Command output displays parameter values and resources when you create datagrid-service, as in the following example:

--> Deploying template "rhdg-helloworld/datagrid-service" to project rhdg-helloworld

     Red Hat Data Grid Service
     ---------
     Red Hat Data Grid is an in-memory, distributed key/value store.

     * With parameters:
        ...

--> Creating resources ...
    secret "datagrid-service" created
    service "datagrid-service-ping" created
    service "datagrid-service" created
    statefulset "datagrid-service" created
--> Success
    ...