Chapter 4. Tutorials

4.1. Example Workflow: Deploying binary build of EAP 6.4 / EAP 7.0 Infinispan application together with JDG for OpenShift image

The following example uses CarMart quickstart to deploy EAP 6.4 / EAP 7.0 Infinispan application, accessing a remote JBoss Data Grid server running in the same OpenShift project.

4.1.1. Prerequisite

  1. Create a new project. 

    $ oc new-project jdg-bin-demo
    Note

    For brevity this example will not configure clustering. See dedicated section if data replication across the cluster is desired.

4.1.2. Deploy JBoss Data Grid 7.1 server

  1. Identify the image stream for the JBoss Data Grid 7.1 image. 

    $ oc get is -n openshift | grep grid | cut -d ' ' -f 1
jboss-datagrid71-openshift

  2. Deploy the server. Also specify the following:

    1. carcache as the name of application,
    2. A Hot Rod based connector, and

    3. carcache as the name of the Infinispan cache to configure. 

      $ oc new-app --name=carcache \
--image-stream=jboss-datagrid71-openshift \
-e INFINISPAN_CONNECTORS=hotrod \
-e CACHE_NAMES=carcache
--> Found image d83b4b2 (3 months old) in image stream "openshift/jboss-datagrid71-openshift" under tag "latest" for "jboss-datagrid71-openshift"

    JBoss Data Grid 7.1
    -------------------
    Provides a scalable in-memory distributed database designed for fast access to large volumes of data.

    Tags: datagrid, java, jboss, xpaas

    * This image will be deployed in deployment config "carcache"
    * Ports 11211/tcp, 11222/tcp, 8080/tcp, 8443/tcp, 8778/tcp will be load balanced by service "carcache"
      * Other containers can access this service through the hostname "carcache"

--> Creating resources ...
    deploymentconfig "carcache" created
    service "carcache" created
--> Success
    Run 'oc status' to view your app.

4.1.3. Deploy binary build of EAP 6.4 / EAP 7.0 CarMart application

  1. Clone the source code. 

    $ git clone https://github.com/jboss-openshift/openshift-quickstarts.git
  2. Configure the Red Hat JBoss Middleware Maven repository.

  3. Build the datagrid/carmart application. 

    $ cd openshift-quickstarts/datagrid/carmart/
    $ mvn clean package
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building JBoss JDG Quickstart: carmart 1.2.0.Final
[INFO] ------------------------------------------------------------------------
...
[INFO] Building war: /tmp/openshift-quickstarts/datagrid/carmart/target/jboss-carmart.war
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.360 s
[INFO] Finished at: 2017-06-27T19:11:46+02:00
[INFO] Final Memory: 34M/310M
[INFO] ------------------------------------------------------------------------

  1. Prepare the directory structure on the local file system.

    Application archives in the deployments/ subdirectory of the main binary build directory are copied directly to the standard deployments folder of the image being built on OpenShift. For the application to deploy, the directory hierarchy containing the web application data must be correctly structured.

    Create main directory for the binary build on the local file system and deployments/ subdirectory within it. Copy the previously built WAR archive for the carmart quickstart to the deployments/ subdirectory: 

    $ ls
pom.xml  README.md  README-openshift.md  README-tomcat.md  src  target
    $ mkdir -p jdg-binary-demo/deployments
    $ cp target/jboss-carmart.war jdg-binary-demo/deployments/
    Note

    Location of the standard deployments directory depends on the underlying base image, that was used to deploy the application. See the following table:

    Table 4.1. Standard Location of the Deployments Directory

    Name of the Underlying Base Image(s)Standard Location of the Deployments Directory

    EAP for OpenShift 6.4 and 7.0

    $JBOSS_HOME/standalone/deployments

    Java S2I for OpenShift

    /deployments

    JWS for OpenShift

    $JWS_HOME/webapps

  2. Identify the image stream for EAP 6.4 / EAP 7.0 image. 

    $ oc get is -n openshift | grep eap | cut -d ' ' -f 1
jboss-eap64-openshift
jboss-eap70-openshift

  1. Create new binary build, specifying image stream and application name. 

    $ oc new-build --binary=true \
--image-stream=jboss-eap64-openshift \
--name=eap-app
--> Found image 8fbf0f7 (2 months old) in image stream "openshift/jboss-eap64-openshift" under tag "latest" for "jboss-eap64-openshift"

    JBoss EAP 6.4
    -------------
    Platform for building and running JavaEE applications on JBoss EAP 6.4

    Tags: builder, javaee, eap, eap6

    * A source build using binary input will be created
      * The resulting image will be pushed to image stream "eap-app:latest"
      * A binary build was created, use 'start-build --from-dir' to trigger a new build

--> Creating resources with label build=eap-app ...
    imagestream "eap-app" created
    buildconfig "eap-app" created
--> Success
    Note

    Specify jboss-eap70-openshift as the image stream name in the aforementioned command to use EAP 7.0 image for the application.

  2. Start the binary build. Instruct oc executable to use main directory of the binary build we created in previous step as the directory containing binary input for the OpenShift build. 

    $ oc start-build eap-app --from-dir=jdg-binary-demo/ --follow
Uploading directory "jdg-binary-demo" as binary input for the build ...
build "eap-app-1" started
Receiving source from STDIN as archive ...
Copying all war artifacts from /home/jboss/source/. directory into /opt/eap/standalone/deployments for later deployment...
Copying all ear artifacts from /home/jboss/source/. directory into /opt/eap/standalone/deployments for later deployment...
Copying all rar artifacts from /home/jboss/source/. directory into /opt/eap/standalone/deployments for later deployment...
Copying all jar artifacts from /home/jboss/source/. directory into /opt/eap/standalone/deployments for later deployment...
Copying all war artifacts from /home/jboss/source/deployments directory into /opt/eap/standalone/deployments for later deployment...
'/home/jboss/source/deployments/jboss-carmart.war' -> '/opt/eap/standalone/deployments/jboss-carmart.war'
Copying all ear artifacts from /home/jboss/source/deployments directory into /opt/eap/standalone/deployments for later deployment...
Copying all rar artifacts from /home/jboss/source/deployments directory into /opt/eap/standalone/deployments for later deployment...
Copying all jar artifacts from /home/jboss/source/deployments directory into /opt/eap/standalone/deployments for later deployment...
Pushing image 172.30.82.129:5000/jdg-bin-demo/eap-app:latest ...
Pushed 0/7 layers, 1% complete
Pushed 1/7 layers, 17% complete
Pushed 2/7 layers, 31% complete
Pushed 3/7 layers, 46% complete
Pushed 4/7 layers, 81% complete
Pushed 5/7 layers, 84% complete
Pushed 6/7 layers, 99% complete
Pushed 7/7 layers, 100% complete
Push successful

  3. Create a new OpenShift application based on the build. 

    $ oc new-app eap-app
--> Found image ee25340 (3 minutes old) in image stream "jdg-bin-demo/eap-app" under tag "latest" for "eap-app"

    jdg-bin-demo/eap-app-1:4bab3f63
    -------------------------------
    Platform for building and running JavaEE applications on JBoss EAP 6.4

    Tags: builder, javaee, eap, eap6

    * This image will be deployed in deployment config "eap-app"
    * Ports 8080/tcp, 8443/tcp, 8778/tcp will be load balanced by service "eap-app"
      * Other containers can access this service through the hostname "eap-app"

--> Creating resources ...
    deploymentconfig "eap-app" created
    service "eap-app" created
--> Success
    Run 'oc status' to view your app.

  4. Expose the service as route. 

    $ oc get svc -o name
service/carcache
service/eap-app
    $ oc get route
No resources found.
    $ oc expose svc/eap-app
route "eap-app" exposed
    $ oc get route
NAME      HOST/PORT                                    PATH      SERVICES   PORT       TERMINATION   WILDCARD
eap-app   eap-app-jdg-bin-demo.openshift.example.com             eap-app    8080-tcp                 None

  5. Access the application.

    Access the CarMart application in your browser using the URL http://eap-app-jdg-bin-demo.openshift.example.com/jboss-carmart. You can view / remove existing cars (Home tab), or add a new car (New car tab).

4.2. Example Workflow: Performing JDG rolling upgrade from JDG 6.5 for OpenShift image to JDG 7.1 for OpenShift image using the REST connector

The following example details the procedure to perform a rolling upgrade from JBoss Data Grid 6.5 for OpenShift image to JBoss Data Grid 7.1 for OpenShift image, using the REST connector.

Important

When performing a rolling upgrade it is recommended to not update any cache entries in the source cluster, as this may lead to data inconsistency.

4.2.1. Start / Deploy the Source Cluster

A rolling upgrade to succeed, it assumes the Source Cluster with properties similar to the ones below:

  • The name of the source JBoss Data Grid 6.5 cluster is jdg65-cluster and it has been deployed using the datagrid65-basic template or similar.
  • The name of the replicated cache to synchronize its content during rolling upgrade is clustercache.
  • The REST Infinispan connector has been configured for the application.
  • The service name of the REST connector endpoint on JBoss Data Grid 6.5 cluster is jdg65-cluster.
  • The clustercache replicated cache has been previously populated with some content to synchronize.

to be up and running.

Note

For demonstration purposes source JBoss Data Grid 6.5 for OpenShift cluster with aforementioned properties can be deployed running e.g. the following steps:

  1. Create a dedicated OpenShift project. 

    $ oc new-project jdg-rest-rolling-upgrade-demo

  2. Deploy source JBoss Data Grid 6.5 cluster with the REST connector enabled, utilizing replicated cache named clustercache. 

    $ oc new-app --template=datagrid65-basic \
-p APPLICATION_NAME=jdg65-cluster \
-p INFINISPAN_CONNECTORS=rest \
-p CACHE_NAMES=clustercache \
-e CLUSTERCACHE_CACHE_TYPE=replicated
--> Deploying template "openshift/datagrid65-basic" to project jdg-rest-rolling-upgrade-demo

     datagrid65-basic
     ---------
     Application template for JDG 6.5 applications.


     * With parameters:
        * APPLICATION_NAME=jdg65-cluster
        * HOSTNAME_HTTP=
        * USERNAME=
        * PASSWORD=
        * IMAGE_STREAM_NAMESPACE=openshift
        * INFINISPAN_CONNECTORS=rest
        * CACHE_NAMES=clustercache
        * ENCRYPTION_REQUIRE_SSL_CLIENT_AUTH=
        * MEMCACHED_CACHE=default
        * REST_SECURITY_DOMAIN=
        * JGROUPS_CLUSTER_PASSWORD=kQiUcyhC # generated

--> Creating resources ...
    service "jdg65-cluster" created
    service "jdg65-cluster-memcached" created
    service "jdg65-cluster-hotrod" created
    route "jdg65-cluster" created
    deploymentconfig "jdg65-cluster" created
--> Success
    Run 'oc status' to view your app.

  3. Populate clustercache with some content to synchronize later.

    1. Add some entries using JBoss Data Grid CLI.

      1. Given the following JBoss Data Grid CLI file to add cache entries non-interactively. 

        $ mkdir -p cache-entries
        $ cat << EOD > cache-entries/cache-input.cli
cache clustercache
put key1 val1
put key2 val2
put key3 val3
put key4 val4
EOD

        Please note a rolling upgrade will fail (BZ-1101512 - CLI UPGRADE command fails when testing data stored via CLI with REST encoding) when storing cache data via CLI using the --codec=rest encoding parameter for the put commands in previous step.

        To overcome this issue, we do not specify codec to be used for encoding of cache entries (cache entries will be stored using the default none encoding).

      2. Get the name of the JBoss Data Grid 6.5 pod. 

        $ export JDG65_POD=$(oc get pods -o name \
| grep -Po "[^/]+$" | grep "jdg65" \
| grep -v "deploy")

      3. Copy the cache-input.cli file to JBoss Data Grid 6.5 pod. 

        $ oc rsync --no-perms=true ./cache-entries/ $JDG65_POD:/tmp
sending incremental file list
cache-input.cli

sent 182 bytes  received 40 bytes  444.00 bytes/sec
total size is 75  speedup is 0.34

      4. Add entries to clustercache by executing commands from cli-input.cli file. 

        $ oc rsh $JDG65_POD /opt/datagrid/bin/cli.sh \
--connect=localhost --file=/tmp/cache-input.cli
Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss

    2. Add some entries directly via remote REST client.

      1. Get the host value of the route for jdg65-cluster. 

        $ export JDG65_ROUTE=$(oc get routes \
--no-headers | grep "jdg65" | tr -s ' ' \
| cut -d ' ' -f2)

      2. Add example ExampleKey entry to clustercache using the REST endpoint remotely. 

        $ curl -X PUT -d "ExampleValue" \
$JDG65_ROUTE/rest/clustercache/ExampleKey

4.2.2. Deploy the Target Cluster

Perform the following to deploy JBoss Data Grid 7.1 cluster with the name of jdg71-cluster, using datagrid71-basic template, and clustercache as the name of the replicated cache to synchronize during the rolling upgrade:

Important

The datagrid71-basic template uses ephemeral, in-memory datastore for the most frequently accessed data. Therefore any cache data, synchronized during a rolling upgrade will be available only during lifecycle of a JDG 7.1 pod (from rolling upgrade to pod restart). Use persistent templates (datagrid71-mysql-persistent or datagrid71-postgresql-persistent) to preserve the previously synchronized cache data across pod restarts. 

$ oc new-app --template=datagrid71-basic \
-p APPLICATION_NAME=jdg71-cluster \
-p INFINISPAN_CONNECTORS=rest \
-p CACHE_NAMES=clustercache \
-p CACHE_TYPE_DEFAULT=replicated \
-p MEMCACHED_CACHE=""
--> Deploying template "openshift/datagrid71-basic" to project jdg-rest-rolling-upgrade-demo

     Red Hat JBoss Data Grid 7.1 (Ephemeral, no https)
     ---------
     Application template for JDG 7.1 applications.

     A new data grid service has been created in your project. It supports connector type(s) "rest".


     * With parameters:
        * Application Name=jdg71-cluster
        * Custom http Route Hostname=
        * Username=
        * Password=
        * ImageStream Namespace=openshift
        * Infinispan Connectors=rest
        * Cache Names=clustercache
        * Datavirt Cache Names=
        * Default Cache Type=replicated
        * Encryption Requires SSL Client Authentication?=
        * Memcached Cache Name=
        * REST Security Domain=
        * JGroups Cluster Password=3Aux1ORc # generated

--> Creating resources ...
    service "jdg71-cluster" created
    service "jdg71-cluster-memcached" created
    service "jdg71-cluster-hotrod" created
    route "jdg71-cluster" created
    deploymentconfig "jdg71-cluster" created
--> Success
    Run 'oc status' to view your app.

4.2.3. Configure REST Store for Caches on the Target Cluster

For each cache in the Target Cluster, intended to be synchronized during a rolling upgrade, configure a RestCacheStore with the following settings:

  1. Ensure that the host and port values point to the Source Cluster.
  2. Ensure that the path value points to the REST endpoint of the Source Cluster.

Given the following helper script to add a REST store to all replicated caches defined in CACHE_NAMES array: 

$ mkdir -p update-cache
Note

Edit the definition of the REST_SERVICE variable below to match the name of the REST service endpoint for your environment. Also, edit the definition of CACHE_NAMES variable in the following helper script to contain names of all caches, that should be equipped with the definition of a REST store. 

$ cat << \EOD > ./update-cache/add-rest-store-to-cache.sh
#!/bin/bash

export JDG_CONF=/opt/datagrid/standalone/configuration/clustered-openshift.xml
export REST_SERVICE="jdg65-cluster"
read -r -d '' REST_STORE_ELEM << EOV
<rest-store path="/rest/cachename" shared="true" purge="false" passivation="false">
<connection-pool connection-timeout="60000" socket-timeout="60000" tcp-no-delay="true"/>
<remote-server outbound-socket-binding="remote-store-rest-server"/>
</rest-store>
EOV

declare -a CACHE_NAMES=("clustercache")

for CACHE in "${CACHE_NAMES[@]}"
do
  # Replace 'cachename' with actual cachename
  REST_STORE_ELEM=${REST_STORE_ELEM//cachename/${CACHE}}
  # Replace newline character with newline and two tabs (in escaped form)
  export REST_STORE_ELEM=${REST_STORE_ELEM//$'\n'/\\n\\t\\t}
  # sed pattern to locate cache definition
  CACHE_PATTERN="\(<replicated-cache[[:space:]]name=\"${CACHE}\"[^<]\+\)\(</replicated-cache>\)"
  # Add REST store definition to cache entry
  sed -i "s#${CACHE_PATTERN}#\1\n\t\t${REST_STORE_ELEM}\n\2#g" $JDG_CONF
done

# sed pattern to locate host / port settings for REST connector
REST_HOST_PATTERN="\(<remote-destination host=\"\)remote-host\(\" port=\"8080\"/>\)"
# Set host value to point to the Source Cluster
sed -i "s#${REST_HOST_PATTERN}#\1${REST_SERVICE}\2#g" $JDG_CONF
EOD

, perform the following:

  1. Get the name of the JBoss Data Grid 7.1 pod. 

    $ export JDG71_POD=$(oc get pods -o name \
| grep -Po "[^/]+$" | grep "jdg71" | grep -v "deploy")

  2. Copy the add-rest-store-to-cache.sh script to JBoss Data Grid 7.1 pod. 

    $ oc rsync --no-perms=true update-cache/ $JDG71_POD:/tmp
sending incremental file list

sent 71 bytes  received 11 bytes  54.67 bytes/sec
total size is 892  speedup is 10.88

  3. Run the script to:

    1. Add REST store definition to each replicated cache from CACHE_NAMES array.
    2. Set host and port to point to the Source Cluster. 
    $ oc rsh $JDG71_POD /bin/bash /tmp/add-rest-store-to-cache.sh

  4. Restart the JBoss Data Grid 7.1 server in order the corresponding caches to recognize the REST store configuration. 

    $ oc rsh $JDG71_POD /opt/datagrid/bin/cli.sh \
--connect ':reload'
Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss
{
    "outcome" => "success",
    "result" => undefined
}
    Warning

    When restarting the server it is important to restart just the JBoss Data Grid process within the running container, not the whole container, since in the latter case the JBoss Data Grid container would be recreated with the default configuration from scratch, without the REST store(s) to be defined for specific cache(s).

4.2.4. Do Not Dump the Key Set During REST Rolling Upgrades

The REST rolling upgrades use case is designed to fetch all the data from the Source Cluster without using the recordKnownGlobalKeyset operation.

Warning

Do not invoke the recordKnownGlobalKeyset operation for REST rolling upgrades. If you invoke this operation, it will cause data corruption and REST rolling upgrades will not complete successfully.

4.2.5. Synchronize Cache Data Using the REST Connector

Run the upgrade --synchronize=rest on the Target Cluster for all caches to be migrated. Optionally, use the --all switch to synchronize all caches in the cluster. 

$ oc rsh $JDG71_POD /opt/datagrid/bin/cli.sh -c \
--commands='cd /subsystem=datagrid-infinispan/cache-container=clustered, \
cache clustercache,upgrade --synchronize=rest'
Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss
ISPN019500: Synchronized 5 entries using migrator 'rest' on cache 'clustercache'

4.2.6. Use the Synchronized Data from the JBoss Data Grid 7.1 (Target) cluster

All the requested data have been just synchronized. You can now point the client application(s) to the Target Cluster.

  1. Get the value of key1 from the JBoss Data Grid 7.1 cache via CLI. 

    $ oc rsh $JDG71_POD /opt/datagrid/bin/cli.sh -c \
--commands='cd /subsystem=datagrid-infinispan/cache-container=clustered, \
cache clustercache,get key1' \
| grep '"' | base64 -di; echo
val1

  2. Get the value of ExampleKey from the JBoss Data Grid 7.1 cache via remote REST call.

    1. Get the value of JBoss Data Grid 7.1 route. 

      $ JDG71_ROUTE=$(oc get routes | grep jdg71 \
| tr -s ' ' | cut -d ' ' -f2)

    2. Get the value of ExampleKey via remote REST client. 

      $ curl -X GET \
$JDG71_ROUTE/rest/clustercache/ExampleKey; echo
ExampleValue

4.2.7. Disable the RestCacheStore on the Target Cluster

Once the Target Cluster has obtained all data from the Source Cluster, disable the RestCacheStore (for each cache it has been previously configured) on the Target Cluster using the following command: 

$ oc rsh $JDG71_POD /opt/datagrid/bin/cli.sh -c \
--commands='cd /subsystem=datagrid-infinispan/cache-container=clustered, \
cache clustercache,upgrade --disconnectsource=rest'
Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss
ISPN019501: Disconnected 'rest' migrator source on cache 'clustercache'

The Source Cluster can now be decommissioned.