Red Hat Training

A Red Hat training course is available for Red Hat Gluster Storage

4.4. Setting up the Heketi Server

After deploying the containers and installing the templates, the system is now ready to load the Heketi topology file. Heketi provides a RESTful management interface which can be used to manage the lifecycle of Red Hat Gluster Storage volumes.
A sample, formatted topology file (topology-sample.json) is installed with the ‘heketi-templates’ package in the /usr/share/heketi/ folder. 
{
    "clusters": [
               {
                  "nodes": [
                                {
                        "node": {
                            "hostnames": {
                                "manage": [
                                    "node1.example.com"
                                ],
                                "storage": [
                                    "192.168.121.168"
                                 ]
                             },
                             "zone": 1
                         },
                         "devices": [
                             "/dev/sdb",
                             "/dev/sdc",
                             "/dev/sdd",
                             "/dev/sde"
                         ]
               },...

Note

Edit the topology file based on the Red Hat Gluster Storage pod hostname under the node.hostnames.manage section and node.hostnames.storage section with the IP address. For simplicity, the file only sets up 3 nodes with 4 drives each.
Execute the following steps to set up the Heketi server:
  1. Execute the following command to check if the bootstrap container is running: 
    # curl http://deploy-heketi-<project_name>.<sub-domain_name>/hello
    For example: 
    # curl http://deploy-heketi-storage-project.cloudapps.mystorage.com/hello

Hello from Heketi
  2. Execute the following command to load the topology file: 
    # export  HEKETI_CLI_SERVER=http://deploy-heketi-<project_name>.<sub_domain_name>
    For example: 
    # export HEKETI_CLI_SERVER=http://deploy-heketi-storage-project.cloudapps.mystorage.com
    # heketi-cli topology load --json=topology.json
    For example: 
    # heketi-cli topology load --json=topology.json

 Creating node node1.example.com ... ID: 95cefa174c7210bd53072073c9c041a3
        Adding device /dev/sdb ... OK
        Adding device /dev/sdc ... OK
        Adding device /dev/sdd ... OK
        Adding device /dev/sde ... OK
    Creating node node2.example.com ... ID: f9920995e580f0fe56fa269d3f3f8428
        Adding device /dev/sdb ... OK
        Adding device /dev/sdc ... OK
        Adding device /dev/sdd ... OK
        Adding device /dev/sde ... OK
    Creating node node3.example.com ... ID: 73fe4aa89ba35c51de4a51ecbf52544d
        Adding device /dev/sdb ... OK
        Adding device /dev/sdc ... OK
        Adding device /dev/sdd ... OK
        Adding device /dev/sde ... OK
  3. Execute the following command to verify that the topology is loaded: 
    # heketi-cli topology info
  4. Execute the following command to create the Heketi storage volume which will store the database on a reliable Red Hat Gluster Storage volume: 
    # heketi-cli setup-openshift-heketi-storage
    For example: 
    # heketi-cli setup-openshift-heketi-storage

Saving heketi-storage.json

    Note

    If the Trusted Storage Pool where the heketidbstorage volume is created is down, then the Heketi service will not work. Hence, you must ensure that the Trusted Storage Pool is up before running heketi-cli.
  5. Execute the following command to create a job which will copy the database from deploy-heketi bootstrap container to the volume. 
    # oc create -f heketi-storage.json
    For example: 
    # oc create -f heketi-storage.json

secret "heketi-storage-secret" created
endpoints "heketi-storage-endpoints" created
service "heketi-storage-endpoints" created
job "heketi-storage-copy-job" created
  6. Execute the following command to verify that the job has finished successfully: 
    # oc get jobs
    For example: 
    # oc get jobs
NAME                      DESIRED   SUCCESSFUL   AGE
heketi-storage-copy-job   1            1               2m
  7. Execute the following command to deploy the Heketi service which will be used to create persistent volumes for OpenShift: 
    # oc process heketi -v \
    HEKETI_KUBE_NAMESPACE=<Project name> \
    HEKETI_KUBE_APIHOST='<OpenShift master endpoint address>' \
    HEKETI_KUBE_INSECURE=y \
    HEKETI_KUBE_USER=<user name> \
    HEKETI_KUBE_PASSWORD=<password> | oc create -f -
    For example: 
    oc process heketi -v \
    HEKETI_KUBE_NAMESPACE=storage-project \
    HEKETI_KUBE_APIHOST='https://master.example.com:8443' \
    HEKETI_KUBE_INSECURE=y \
    HEKETI_KUBE_USER=test-admin \
    HEKETI_KUBE_PASSWORD=admin | oc create -f -


service "heketi" created
route "heketi" created
deploymentconfig "heketi" created
  8. Execute the following command to let the client communicate with the container: 
    # export  HEKETI_CLI_SERVER=http://heketi-<project_name>.<sub_domain_name>
    For example: 
    # export  HEKETI_CLI_SERVER=http://heketi-storage-project.cloudapps.mystorage.com
    # heketi-cli topology info
  9. Execute the following command to remove all the bootstrap containers which was used earlier to deploy Heketi 
    # oc delete all,job,template,secret --selector="deploy-heketi"
    For example: 
    # oc delete all,job,template,secret --selector="deploy-heketi"

deploymentconfig "deploy-heketi" deleted
route "deploy-heketi" deleted
service "deploy-heketi" deleted
pod "deploy-heketi-1-4k1fh" deleted
job "heketi-storage-copy-job" deleted
template "deploy-heketi" deleted