A.3. Setting up the Heketi Server

After deploying the containers and installing the templates, the system is now ready to load the Heketi topology file. Heketi provides a RESTful management interface which can be used to manage the lifecycle of Red Hat Gluster Storage volumes.
A sample, formatted topology file (topology-sample.json) is installed with the ‘heketi-templates’ package in the /usr/share/heketi/ directory.
{
    "clusters": [
               {
                  "nodes": [
                                {
                        "node": {
                            "hostnames": {
                                "manage": [
                                    "node1.example.com"
                                ],
                                "storage": [
                                    "192.168.121.168"
                                 ]
                             },
                             "zone": 1
                         },
                         "devices": [
                             "/dev/sdb",
                             "/dev/sdc",
                             "/dev/sdd",
                             "/dev/sde"
                         ]
               },...
Edit the topology file based on the Red Hat Gluster Storage pod hostname under the node.hostnames.manage section and node.hostnames.storage section with the IP address. For simplicity, the /usr/share/heketi/topology-sample.json file only sets up 4 nodes with 8 drives each.

Important

Heketi stores its database on a Red Hat Gluster Storage volume. Heketi service does not respond if the volume is down.
To resolve this issue, restart the gluster pods hosting the Heketi volume.
Execute the following steps to set up the Heketi server:
  1. Execute the following command to check if the bootstrap container is running:
    # curl http://deploy-heketi-<project_name>.<sub-domain_name>/hello
    For example:
    # curl http://deploy-heketi-storage-project.cloudapps.mystorage.com/hello
    Hello from Heketi
  2. Execute the following command to load the topology file:
    # export  HEKETI_CLI_SERVER=http://deploy-heketi-<project_name>.<sub_domain_name>
    For example:
    # export HEKETI_CLI_SERVER=http://deploy-heketi-storage-project.cloudapps.mystorage.com
    # heketi-cli topology load --json=topology.json
    For example:
    # heketi-cli topology load --json=topology.json
    Creating cluster ... ID: 94877b3f72b79273e87c1e94201ecd58
     Creating node node1.example.com ... ID: 95cefa174c7210bd53072073c9c041a3
            Adding device /dev/sdb ... OK
            Adding device /dev/sdc ... OK
            Adding device /dev/sdd ... OK
            Adding device /dev/sde ... OK
        Creating node node2.example.com ... ID: f9920995e580f0fe56fa269d3f3f8428
            Adding device /dev/sdb ... OK
            Adding device /dev/sdc ... OK
            Adding device /dev/sdd ... OK
            Adding device /dev/sde ... OK
        Creating node node3.example.com ... ID: 73fe4aa89ba35c51de4a51ecbf52544d
            Adding device /dev/sdb ... OK
            Adding device /dev/sdc ... OK
            Adding device /dev/sdd ... OK
            Adding device /dev/sde ... OK
    
  3. Execute the following command to verify that the topology is loaded:
    # heketi-cli topology info
  4. Execute the following command to create the Heketi storage volume which will store the database on a reliable Red Hat Gluster Storage volume:
    # heketi-cli setup-openshift-heketi-storage
    For example:
    # heketi-cli setup-openshift-heketi-storage
    Saving heketi-storage.json
  5. Execute the following command to create a job which will copy the database from deploy-heketi bootstrap container to the volume.
    # oc create -f heketi-storage.json
    For example:
    # oc create -f heketi-storage.json
    secret "heketi-storage-secret" created
    endpoints "heketi-storage-endpoints" created
    service "heketi-storage-endpoints" created
    job "heketi-storage-copy-job" created
  6. Execute the following command to verify that the job has finished successfully:
    # oc get jobs
    For example:
    # oc get jobs
    NAME                      DESIRED   SUCCESSFUL   AGE
    heketi-storage-copy-job   1            1               2m
  7. Execute the following command to remove all resources used to bootstrap heketi:
    # oc delete all,job,template,secret --selector="deploy-heketi"
    For example:
    # oc delete all,job,template,secret --selector="deploy-heketi"
    deploymentconfig "deploy-heketi" deleted
    route "deploy-heketi" deleted
    service "deploy-heketi" deleted
    pod "deploy-heketi-1-4k1fh" deleted
    job "heketi-storage-copy-job" deleted
    template "deploy-heketi" deleted
    
  8. Execute the following command to deploy the Heketi service which will be used to create persistent volumes for OpenShift:
    # oc process heketi | oc create -f -
    For example:
    # oc process heketi | oc create -f -
    service "heketi" created
    route "heketi" created
    deploymentconfig "heketi" created
  9. Execute the following command to let the client communicate with the container:
    # export  HEKETI_CLI_SERVER=http://heketi-<project_name>.<sub_domain_name>
    For example:
    # export  HEKETI_CLI_SERVER=http://heketi-storage-project.cloudapps.mystorage.com
    # heketi-cli topology info