5.5. Verify your Deployment

Execute the following steps to verify the deployment
  1. Installation Verification for Independent mode
    1. Examine the installation for the app-storage namespace by running the following commands: 
      # switch to the app-storage namespace
oc project app-storage

# get the list of pods here (1 heketi pod)
oc get pods
NAME                    READY          STATUS          RESTARTS          AGE

heketi-storage-1-v5skm  1/1            Running         0                 1h
    2. Examine the installation for the infra-storage namespace by running the following commands This can be done from an OCP master node or the ansible deploy host that has the OC CLI installed. 
      # switch to the infra-storage namespace
oc project infra-storage

# list the pods here (1 heketi pod and 1 glusterblock-provisioner pod)
oc get pods
NAME                                              READY   STATUS       RESTARTS       AGE
glusterblock-registry-provisioner-dc-1-28sfc         1/1  Running      0              1h
heketi-registry-5-lht6s                              1/1  Running      0              1h
    3. Check the existence of the registry PVC backed by OCP infrastructure Red Hat Openshift Container Storage. This volume was statically provisioned by openshift-ansible deployment. 
      oc get pvc -n default
NAME                      STATUS      VOLUME                                    CAPACITY         ACCESSMODES      STORAGECLASS               AGE
registry-claim            Bound       pvc-7ca4c8de-10ca-11e8-84d3-069df2c4f284  25Gi             RWX                                         1h
      Check the registry DeploymentConfig to verify it's using this glusterfs volume. 
      oc describe dc/docker-registry -n default | grep -A3 Volumes
    Volumes:
    registry-storage:
    Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName: registry-claim
  2. Storage Provisioning Verification for Independent Mode
    1. Validate PV provisioning using the glusterfs and glusterblock OCP Storage Class created during the OCP deployment. The two Storage Class resources, glusterfs-storage and glusterfs-storage-block, can be used to create new PV claims for verification of the Red Hat Openshift Container Storage deployment. The new PVC using the glusterfs-storage storageclass will be using storage available to gluster pods in app-storage project. 
      # oc get storageclass

NAME                                TYPE
glusterfs-storage                   kubernetes.io/glusterfs
Glusterfs-storage-block             gluster.org/glusterblock
$ cat pvc-file.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: rhocs-file-claim1
  annotations:
    volume.beta.kubernetes.io/storage-class: glusterfs-storage
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
      # cat pvc-block.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: rhocs-block-claim1
  annotations:
    volume.beta.kubernetes.io/storage-class: glusterfs-storage-block
spec:
  accessModes:
    - ReadWriteOnly
  resources:
    requests:
      storage: 5Gi

      # oc create -f pvc-file.yaml
# oc create -f pvc-block.yaml
      Validate that the two PVCs and respective PVs are created correctly: 
      # oc get pvc
  3. Using the heketi-client for Verification
    1. The heketi-client package needs to be installed on the ansible deploy host or on a OCP master. Once it is installed two new files should be created to easily export the required environment variables to run the heketi-client commands (or heketi-cli). The content of each file as well as useful heketi-cli commands are detailed here.
      Create a new file (e.g. "heketi-exports-app") with the following contents: 
      export HEKETI_POD=$(oc get pods -l glusterfs=heketi-storage-pod -n app-storage -o jsonpath="{.items[0].metadata.name}")
export HEKETI_CLI_SERVER=http://$(oc get route/heketi-storage -n app-storage -o jsonpath='{.spec.host}')
export HEKETI_CLI_KEY=$(oc get pod/$HEKETI_POD -n app-storage -o jsonpath='{.spec.containers[0].env[?(@.name=="HEKETI_ADMIN_KEY")].value}')
export HEKETI_ADMIN_KEY_SECRET=$(echo -n ${HEKETI_CLI_KEY} | base64)
export HEKETI_CLI_USER=admin
      Source the file to create the HEKETI app-storage environment variables: 
      source heketi-exports-app
# see if heketi is alive
curl -w '\n' ${HEKETI_CLI_SERVER}/hello
Hello from Heketi
# ask heketi about the cluster it knows about
heketi-cli cluster list
Clusters:
Id:56ed234a384cef7dbef6c4aa106d4477 [file][block]
# ask heketi about the topology of the RHOCS cluster for apps
heketi-cli topology info
# ask heketi about the volumes already created (one for the heketi db should exist after the OCP initial installation)
heketi-cli volume list
Id:d71a4cbea22af3453615a9020f261b5c Cluster:56ed234a384cef7dbef6c4aa106d4477
Name:heketidbstorage
      Create a new file (e.g. "heketi-exports-infra") with the following contents: 
      export HEKETI_POD=$(oc get pods -l glusterfs=heketi-registry-pod -n infra-storage -o jsonpath="{.items[0].metadata.name}")
export HEKETI_CLI_SERVER=http://$(oc get route/heketi-registry -n infra-storage -o jsonpath='{.spec.host}')
export HEKETI_CLI_USER=admin
export HEKETI_CLI_KEY=$(oc get pod/$HEKETI_POD -n infra-storage -o jsonpath='{.spec.containers[0].env[?(@.name=="HEKETI_ADMIN_KEY")].value}')
export HEKETI_ADMIN_KEY_SECRET=$(echo -n ${HEKETI_CLI_KEY} | base64)
      Source the file to create the HEKETI infra-storage environment variables: 
      source heketi-exports-infra
# see if heketi is alive
curl -w '\n' ${HEKETI_CLI_SERVER}/hello
Hello from Heketi
# ask heketi about the cluster it knows about (the RHOCS cluster for infrastructure)
heketi-cli cluster list
Clusters:
Id:baf91b261cbca2bb4b62caece63f60d0 [file][block]
# ask heketi about the volumes already created
heketi-cli volume list
Id:77baed02f79f4518326d8cc1db6c7af8 Cluster:baf91b261cbca2bb4b62caece63f60d0 Name:heketidbstorage