Chapter 11. Operations on a Red Hat Gluster Storage Pod in an OpenShift Environment

This chapter lists out the various operations that can be performed on a Red Hat Gluster Storage pod (gluster pod):
  1. To list the pods, execute the following command :
    # oc get pods
    For example:
    # oc get pods
    NAME                                                     READY     STATUS    RESTARTS   AGE
    storage-project-router-1-v89qc                           1/1       Running   0          1d
    glusterfs-dc-node1.example.com                           1/1       Running   0          1d
    glusterfs-dc-node2.example.com                           1/1       Running   1          1d
    glusterfs-dc-node3.example.com                           1/1       Running   0          1d
    heketi-1-k1u14                                           1/1       Running   0          23m
    rhel1                                                    1/1       Running   0          26s
    Following are the gluster pods from the above example:
    glusterfs-dc-node1.example.com                           
    glusterfs-dc-node2.example.com                           
    glusterfs-dc-node3.example.com

    Note

    The topology.json file will provide the details of the nodes in a given Trusted Storage Pool (TSP) . In the above example all the 3 Red Hat Gluster Storage nodes are from the same TSP.
  2. To enter the gluster pod shell, execute the following command:
    # oc rsh <gluster_pod_name>
    For example:
    # oc rsh glusterfs-dc-node1.example.com
    
    sh-4.2#
  3. To get the peer status, execute the following command:
    # gluster peer status
    For example:
    # gluster peer status
    
    Number of Peers: 2
    
    Hostname: node2.example.com
    Uuid: 9f3f84d2-ef8e-4d6e-aa2c-5e0370a99620
    State: Peer in Cluster (Connected)
    Other names:
    node1.example.com
    
    Hostname: node3.example.com
    Uuid: 38621acd-eb76-4bd8-8162-9c2374affbbd
    State: Peer in Cluster (Connected)
    
  4. To list the gluster volumes on the Trusted Storage Pool, execute the following command:
    # gluster volume info
    For example:
    Volume Name: heketidbstorage
    Type: Distributed-Replicate
    Volume ID: 2fa53b28-121d-4842-9d2f-dce1b0458fda
    Status: Started
    Number of Bricks: 2 x 3 = 6
    Transport-type: tcp
    Bricks:
    Brick1: 192.168.121.172:/var/lib/heketi/mounts/vg_1be433737b71419dc9b395e221255fb3/brick_c67fb97f74649d990c5743090e0c9176/brick
    Brick2: 192.168.121.233:/var/lib/heketi/mounts/vg_0013ee200cdefaeb6dfedd28e50fd261/brick_6ebf1ee62a8e9e7a0f88e4551d4b2386/brick
    Brick3: 192.168.121.168:/var/lib/heketi/mounts/vg_e4b32535c55c88f9190da7b7efd1fcab/brick_df5db97aa002d572a0fec6bcf2101aad/brick
    Brick4: 192.168.121.233:/var/lib/heketi/mounts/vg_0013ee200cdefaeb6dfedd28e50fd261/brick_acc82e56236df912e9a1948f594415a7/brick
    Brick5: 192.168.121.168:/var/lib/heketi/mounts/vg_e4b32535c55c88f9190da7b7efd1fcab/brick_65dceb1f749ec417533ddeae9535e8be/brick
    Brick6: 192.168.121.172:/var/lib/heketi/mounts/vg_7ad961dbd24e16d62cabe10fd8bf8909/brick_f258450fc6f025f99952a6edea203859/brick
    Options Reconfigured:
    performance.readdir-ahead: on
     
    Volume Name: vol_9e86c0493f6b1be648c9deee1dc226a6
    Type: Distributed-Replicate
    Volume ID: 940177c3-d866-4e5e-9aa0-fc9be94fc0f4
    Status: Started
    Number of Bricks: 2 x 3 = 6
    Transport-type: tcp
    Bricks:
    Brick1: 192.168.121.168:/var/lib/heketi/mounts/vg_3fa141bf2d09d30b899f2f260c494376/brick_9fb4a5206bdd8ac70170d00f304f99a5/brick
    Brick2: 192.168.121.172:/var/lib/heketi/mounts/vg_7ad961dbd24e16d62cabe10fd8bf8909/brick_dae2422d518915241f74fd90b426a379/brick
    Brick3: 192.168.121.233:/var/lib/heketi/mounts/vg_5c6428c439eb6686c5e4cee56532bacf/brick_b3768ba8e80863724c9ec42446ea4812/brick
    Brick4: 192.168.121.172:/var/lib/heketi/mounts/vg_7ad961dbd24e16d62cabe10fd8bf8909/brick_0a13958525c6343c4a7951acec199da0/brick
    Brick5: 192.168.121.168:/var/lib/heketi/mounts/vg_17fbc98d84df86756e7826326fb33aa4/brick_af42af87ad87ab4f01e8ca153abbbee9/brick
    Brick6: 192.168.121.233:/var/lib/heketi/mounts/vg_5c6428c439eb6686c5e4cee56532bacf/brick_ef41e04ca648efaf04178e64d25dbdcb/brick
    Options Reconfigured:
    performance.readdir-ahead: on
  5. To get the volume status, execute the following command:
    # gluster volume status <volname>
    For example:
    # gluster volume status vol_9e86c0493f6b1be648c9deee1dc226a6
    
    Status of volume: vol_9e86c0493f6b1be648c9deee1dc226a6
    Gluster process                             TCP Port  RDMA Port  Online  Pid
    ------------------------------------------------------------------------------
    Brick 192.168.121.168:/var/lib/heketi/mounts/v
    g_3fa141bf2d09d30b899f2f260c494376/brick_9f
    b4a5206bdd8ac70170d00f304f99a5/brick        49154     0          Y       3462
    Brick 192.168.121.172:/var/lib/heketi/mounts/v
    g_7ad961dbd24e16d62cabe10fd8bf8909/brick_da
    e2422d518915241f74fd90b426a379/brick        49154     0          Y       115939
    Brick 192.168.121.233:/var/lib/heketi/mounts/v
    g_5c6428c439eb6686c5e4cee56532bacf/brick_b3
    768ba8e80863724c9ec42446ea4812/brick        49154     0          Y       116134
    Brick 192.168.121.172:/var/lib/heketi/mounts/v
    g_7ad961dbd24e16d62cabe10fd8bf8909/brick_0a
    13958525c6343c4a7951acec199da0/brick        49155     0          Y       115958
    Brick 192.168.121.168:/var/lib/heketi/mounts/v
    g_17fbc98d84df86756e7826326fb33aa4/brick_af
    42af87ad87ab4f01e8ca153abbbee9/brick        49155     0          Y       3481
    Brick 192.168.121.233:/var/lib/heketi/mounts/v
    g_5c6428c439eb6686c5e4cee56532bacf/brick_ef
    41e04ca648efaf04178e64d25dbdcb/brick        49155     0          Y       116153
    NFS Server on localhost                     2049      0          Y       116173
    Self-heal Daemon on localhost               N/A       N/A        Y       116181
    NFS Server on node1.example.com                                        2049      0          Y       3501
    Self-heal Daemon on node1.example.com                                  N/A       N/A        Y       3509
    NFS Server on 192.168.121.172                  2049      0          Y       115978
    Self-heal Daemon on 192.168.121.172            N/A       N/A        Y       115986
     
    Task Status of Volume vol_9e86c0493f6b1be648c9deee1dc226a6
    ------------------------------------------------------------------------------
    There are no active volume tasks
    
    
  6. To use the snapshot feature, load the snapshot module using the following command:
    # - modprobe dm_snapshot

    Important

    Restrictions for using Snapshot

    • After a snapshot is created, it must be accessed though the user-serviceable snapshots feature only. This can be used to copy the old versions of files into the required location.
      Reverting the volume to a snapshot state is not supported and should never be done as it might damage the consistency of the data.
    • On a volume with snapshots, volume changing operations, such as volume expansion, must not be performed.
  7. To take the snapshot of the gluster volume, execute the following command:
    # gluster snapshot create <snapname> <volname>
    For example:
    # gluster snapshot create snap1 vol_9e86c0493f6b1be648c9deee1dc226a6
    
    snapshot create: success: Snap snap1_GMT-2016.07.29-13.05.46 created successfully
  8. To list the snapshots, execute the following command:
    # gluster snapshot list
    For example:
    # gluster snapshot list
    
    snap1_GMT-2016.07.29-13.05.46
    snap2_GMT-2016.07.29-13.06.13
    snap3_GMT-2016.07.29-13.06.18
    snap4_GMT-2016.07.29-13.06.22
    snap5_GMT-2016.07.29-13.06.26
  9. To delete a snapshot, execute the following command:
    # gluster snap delete <snapname>
    For example:
    # gluster snap delete snap1_GMT-2016.07.29-13.05.46
    
    Deleting snap will erase all the information about the snap. Do you still want to continue? (y/n) y
    snapshot delete: snap1_GMT-2016.07.29-13.05.46: snap removed successfully
  10. You can set up Container-Native Storage volumes for geo-replication to a non-Container-Native Storage remote site. Geo-replication uses a master–slave model. Here, the Container-Native Storage volume acts as the master volume. To set up geo-replication, you must run the geo-replication commands on gluster pods. To enter the gluster pod shell, execute the following command:
     # oc rsh <gluster_pod_name>
  11. Brick multiplexing is a feature that allows including multiple bricks into one process. This reduces resource consumption, allowing you to run more bricks than earlier with the same memory consumption.
    Brick multiplexing is enabled by default from Container-Native Storage 3.6. If you want to turn it off, execute the following command:
    # gluster volume set all cluster.brick-multiplex off
  12. The auto_unmount option in glusterfs libfuse, when enabled, ensures that the file system is unmounted at FUSE server termination by running a separate monitor process that performs the unmount.
    The GlusterFS plugin in Openshift enables the auto_unmount option for gluster mounts.