Installation Guide

Red Hat Gluster Storage 3.5

Installing Red Hat Gluster Storage 3.5

Gluster Storage Documentation Team

Red Hat Customer Content Services

Abstract

This guide describes the prerequisites and provides step-by-step instructions to install Red Hat Gluster Storage using different methods.
Making open source more inclusive
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright's message

Chapter 1. Planning Red Hat Gluster Storage Installation

This chapter outlines the minimum hardware and software installation requirements for a successful installation, configuration, and operation of a Red Hat Gluster Storage Server environment.

1.1. About Red Hat Gluster Storage

Red Hat Gluster Storage is a software-only, scale-out storage that provides flexible and affordable unstructured data storage for the enterprise. Red Hat Gluster Storage 3.5 provides new opportunities to unify data storage and infrastructure, increase performance, and improve availability and manageability in order to meet a broader set of an organization’s storage challenges and requirements.
GlusterFS, a key building block of Red Hat Gluster Storage, is based on a stackable user space design and can deliver exceptional performance for diverse workloads. GlusterFS aggregates various storage servers over network interconnects into one large parallel network file system. The POSIX compatible GlusterFS servers, which use XFS file system format to store data on disks, can be accessed using industry standard access protocols including NFS and CIFS.
Red Hat Gluster Storage can be deployed in the private cloud or datacenter using Red Hat Gluster Storage Server for On-Premise. Red Hat Gluster Storage can be installed on commodity servers or virtual machines and storage hardware resulting in a powerful, massively scalable, and highly available NAS environment. Additionally, Red Hat Gluster Storage can be deployed in the public cloud using Red Hat Gluster Storage Server for Public Cloud, for example, within the Amazon Web Services (AWS) cloud. It delivers all the features and functionality possible in a private cloud or datacenter to the public cloud by providing massively scalable and highly available NAS in the cloud.
Red Hat Gluster Storage Server for On-Premise

Red Hat Gluster Storage Server for On-Premise enables enterprises to treat physical storage as a virtualized, scalable, and centrally managed pool of storage by using commodity server and storage hardware.

Red Hat Gluster Storage Server for Public Cloud

Red Hat Gluster Storage Server for Public Cloud packages GlusterFS as an Amazon Machine Image (AMI) for deploying scalable NAS in the AWS public cloud. This powerful storage server provides a highly available, scalable, virtualized, and centrally managed pool of storage for Amazon users.

Storage node

A storage node can be a physical server, a virtual machine, or a public cloud machine image running one instance of Red Hat Gluster Storage.

1.2. Prerequisites

Ensure that your environment meets the following requirements:

Important

For production use and high availability, a minimum of three servers per trusted storage pool is required.
For quorum reasons, multi-site clusters with synchronous replication require an odd number of sites (minimum of three) to survive a site failure without manual intervention.

1.2.1. Multi-site Cluster Latency

Ensure the latency of the multi-site cluster is no more than 5ms. Execute the following command on each site to determine the latency with the other site: 
# ping -c100 -q site_ip_address/site_hostname

Note

For more information, see KCS#413623.

1.2.2. File System Requirements

XFS - Format the back-end file system using XFS for glusterFS bricks. XFS can journal metadata, resulting in faster crash recovery. The XFS file system can also be defragmented and expanded while mounted and active.

Note

Red Hat assists existing Gluster Storage Software Appliance customers using ext3 or ext4 to upgrade to a supported version of Red Hat Gluster Storage using the XFS back-end file system.

1.2.3. Logical Volume Manager

Format glusterFS bricks using XFS on the Logical Volume Manager to prepare for the installation.

1.2.4. Network Time Configuration

Synchronize time across all Red Hat Gluster Storage servers using the Network Time Protocol (NTP) or Chrony daemon. For more information about these features see Choosing Between NTP Daemons

1.2.4.1. Configuring time synchronization using Chrony

Note

For Red Hat Enterprise Linux 8, chrony is enabled by default and Network Time Protocol (NTP) is no longer supported.
For information on configuring chrony for time synchronization for all Gluster nodes, see Migrating to chrony.
After configuring chrony, verify that time is synchronized for all Gluster nodes by using the following command from the primary node to all other nodes: 
# clockdiff node-hostname

1.2.4.2. Configuring time synchronization using Network Time Protocol

For Red Hat Enterprise Linux 7
Use a remote server over the Network Time Protocol (NTP) to synchronize the system clock. For more information on NTP, refer to Configuring NTP Using ntpd. Set the ntpd daemon to automatically synchronize the time during the boot process as follows:
  1. Edit the NTP configuration file /etc/ntp.conf using a text editor such as vim or nano. 
    # nano /etc/ntp.conf
  2. Add or edit the list of public NTP servers in the ntp.conf file as follows: 
    server 0.rhel.pool.ntp.org
server 1.rhel.pool.ntp.org
server 2.rhel.pool.ntp.org
    The Red Hat Enterprise Linux 6 version of this file already contains the required information. Edit the contents of this file if customization is required. For more information regarding supported Red Hat Enterprise Linux version for a particular Red Hat Gluster Storage release, see Section 1.7, “Red Hat Gluster Storage Support Matrix”.
  3. Optionally, increase the initial synchronization speed by appending the iburst directive to each line: 
    server 0.rhel.pool.ntp.org iburst
server 1.rhel.pool.ntp.org iburst
server 2.rhel.pool.ntp.org iburst
  4. After the list of servers is complete, set the required permissions in the same file. Ensure that only localhost has unrestricted access: 
    restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict -6 ::1
  5. Save all changes, exit the editor, and restart the NTP daemon: 
    # service ntpd restart
  6. Ensure that the ntpd daemon starts at boot time: 
    # chkconfig ntpd on
Use the ntpdate command for a one-time synchronization of NTP. For more information about this feature, see the Red Hat Enterprise Linux Deployment Guide.

1.3. Hardware Compatibility

Hardware specifications change almost daily, it is recommended that all systems be checked for compatibility. The most recent list of supported hardware can be found in the Red Hat Gluster Storage Server Compatible Physical, Virtual Server and Client OS Platforms List , available online at https://access.redhat.com/knowledge/articles/66206. You must ensure that your environments meets the hardware compatibility outlined in this article. Hardware specifications change rapidly and full compatibility is not guaranteed.
Hardware compatibility is a particularly important concern if you have an older or custom-built system.

1.4. Port Information

Red Hat Gluster Storage Server uses the listed ports. Ensure that firewall settings do not prevent access to these ports.
Firewall configuration tools differ between Red Hat Entperise Linux 6 and Red Hat Enterprise Linux 7.
For Red Hat Enterprise Linux 6, use the iptables command to open a port: 
# iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 5667 -j ACCEPT
# service iptables save
For Red Hat Enterprise Linux 7 and Red Hat Enterprise Linux 8, if default ports are not already in use by other services, it is usually simpler to add a service rather than open a port: 
# firewall-cmd --zone=zone_name --add-service=glusterfs
# firewall-cmd --zone=zone_name --add-service=glusterfs --permanent
However, if the default ports are already in use, you can open a specific port with the following command: 
# firewall-cmd --zone=zone_name --add-port=port/protocol
# firewall-cmd --zone=zone_name --add-port=port/protocol --permanent
For example: 
# firewall-cmd --zone=public --add-port=5667/tcp
# firewall-cmd --zone=public --add-port=5667/tcp --permanent

Table 1.1. Open the following ports on all storage servers

Connection sourceTCP PortsUDP PortsRecommended forUsed for
Any authorized network entity with a valid SSH key22-All configurationsRemote backup using geo-replication
Any authorized network entity; be cautious not to clash with other RPC services.111111All configurationsRPC port mapper and RPC bind
Any authorized SMB/CIFS client139 and 445137 and 138Sharing storage using SMB/CIFSSMB/CIFS protocol
Any authorized NFS clients20492049Sharing storage using Gluster NFS (Deprecated) or NFS-GaneshaExports using NFS protocol
All servers in the Samba-CTDB cluster4379-Sharing storage using SMB and Gluster NFS (Deprecated)CTDB
Any authorized network entity24007-All configurationsManagement processes using glusterd
Any authorized network entity24009-All configurationsGluster events daemon
NFSv3 clients662662Sharing storage using NFS-Ganesha and Gluster NFS (Deprecated)statd
NFSv3 clients3280332803Sharing storage using NFS-Ganesha and Gluster NFS (Deprecated)NLM protocol
NFSv3 clients sending mount requests-32769Sharing storage using Gluster NFS (Deprecated)Gluster NFS MOUNT protocol
NFSv3 clients sending mount requests2004820048Sharing storage using NFS-GaneshaNFS-Ganesha MOUNT protocol
NFS clients875875Sharing storage using NFS-GaneshaNFS-Ganesha RQUOTA protocol (fetching quota information)
Servers in pacemaker/corosync cluster2224-Sharing storage using NFS-Ganeshapcsd
Servers in pacemaker/corosync cluster3121-Sharing storage using NFS-Ganeshapacemaker_remote
Servers in pacemaker/corosync cluster-5404 and 5405Sharing storage using NFS-Ganeshacorosync
Servers in pacemaker/corosync cluster21064-Sharing storage using NFS-Ganeshadlm
Any authorized network entity49152 - 49664-All configurationsBrick communication ports. The total number of ports required depends on the number of bricks on the node. One port is required for each brick on the machine.

Table 1.2. Open the following ports on NFS-Ganesha and Gluster NFS (Deprecated) storage clients

Connection sourceTCP PortsUDP PortsRecommended forUsed for
NFSv3 servers662662Sharing storage using NFS-Ganesha and Gluster NFS (Deprecated)statd
NFSv3 servers3280332803Sharing storage using NFS-Ganesha and Gluster NFS (Deprecated)NLM protocol

1.5. Red Hat Gluster Storage Software Components and Versions

In order to upgrade or use Red Hat Gluster Storage in conjunction with other software, you may need to know the details of each version. This table provides a summary:

Table 1.3. Version Details

RHGS versionglusterfs and glusterfs-fuseRHGS op-versionSMBNFSgDeployAnsible
3.43.12.2-1831302SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0gdeploy-2.0.2-27-
3.4 Batch 1 Update3.12.2-2531303SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0gdeploy-2.0.2-30-
3.4 Batch 2 Update3.12.2-3231304SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0gdeploy-2.0.2-31gluster-ansible-infra-1.0.2-2
3.4 Batch 3 Update3.12.2-4031305SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0gdeploy-2.0.2-31gluster-ansible-infra-1.0.2-2
3.4 Batch 4 Update3.12.2-47 31305 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0gdeploy-2.0.2-32gluster-ansible-infra-1.0.3-3
3.4.4 Async Update3.12.2-47.5 31306 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0gdeploy-2.0.2-32gluster-ansible-infra-1.0.3-3
3.5 (RHEL 6)6.0-22 70000 SMB 1, 2.0, 2.1, 3.0, 3.1.1---
3.5 (RHEL 7)6.0-21 70000 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-35gluster-ansible-infra-1.0.4-3
3.5 Batch Update 1 (RHEL 7)6.0-29 70000 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-35gluster-ansible-infra-1.0.4-5
3.5 Async Update (RHEL 7)6.0-30.1 70000 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-35gluster-ansible-infra-1.0.4-5
3.5 Batch 2 Update (RHEL 7)6.0-37 70000 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-35gluster-ansible-infra-1.0.4-5
3.5 Batch 2 Update (RHEL 8)6.0-37 70000 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-3.0.0-7gluster-ansible-infra-1.0.4-10
3.5.2 Async Update (RHEL 7)6.0-37.1 70000 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-35gluster-ansible-infra-1.0.4-5
3.5.2 Async Update (RHEL 8)6.0-37.1 70000 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-3.0.0-7gluster-ansible-infra-1.0.4-17
3.5 Batch 3 Update (RHEL 7)6.0-49.2 70000 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-36gluster-ansible-infra-1.0.4-5
3.5 Batch 3 Update (RHEL 8)6.0-4970000 SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-3.0.0-8gluster-ansible-infra-1.0.4-11
3.5 Batch 4 Update (RHEL 7)6.0-5670100SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-36gluster-ansible-infra-1.0.4-5
3.5 Batch 4 Update (RHEL 8)6.0-5670100SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-3.0.0-8gluster-ansible-infra-1.0.4-19
3.5.4 Async Update (RHEL 7)6.0-56.270100SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-36gluster-ansible-infra-1.0.4-19
3.5.4 Async Update (RHEL 8)6.0-56.270100SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-36gluster-ansible-infra-1.0.4-19
3.5 Batch 5 Update (RHEL 7)6.0-5970200SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-36 gluster-ansible-infra-1.0.4-19
3.5 Batch 5 Update (RHEL 8)6.0-5970200SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-3.0.0-8 gluster-ansible-infra-1.0.4-19
3.5 Batch 6 Update (RHEL 7)6.0-6170200SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-36 gluster-ansible-infra-1.0.4-19
3.5 Batch 6 Update (RHEL 8)6.0-6170200SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-3.0.0-8 gluster-ansible-infra-1.0.4-19
3.5 Batch 7 Update (RHEL 7)6.0-6370200SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-2.0.2-36 gluster-ansible-infra-1.0.4-19
3.5 Batch 7 Update (RHEL 8)6.0-6370200SMB 1, 2.0, 2.1, 3.0, 3.1.1NFSv3, NFSv4.0, NFSv4.1gdeploy-3.0.0-11 gluster-ansible-infra-1.0.4-21

Note

For more information regarding the version details of previous Red Hat Gluster Storage releases, see https://access.redhat.com/solutions/543123. Red Hat Gluster Storage is not supported on Red Hat Enterprise Linux 6 (RHEL 6) from 3.5 Batch Update 1 onwards.

1.6. Feature Compatibility Support

Red Hat Gluster Storage supports a number of features. Most features are supported with other features, but there are some exceptions. This section clearly identifies which features are supported and compatible with other features to help you in planning your Red Hat Gluster Storage deployment.

Important

Virtual Data Optimizer (VDO) volumes, which are supported in Red Hat Enterprise Linux 7.5, are not currently supported in Red Hat Gluster Storage. VDO is supported only when used as part of Red Hat Hyperconverged Infrastructure for Virtualization 2.0. See Understanding VDO for more information.
Features in the following table are supported from the specified version and later.

Table 1.4. Features supported by Red Hat Gluster Storage version

FeatureVersion
Arbiter bricks3.2
Bitrot detection3.1
Erasure coding3.1
Google Compute Engine3.1.3
Metadata caching3.2
Microsoft Azure3.1.3
NFS version 43.1
SELinux3.1
Sharding3.2.0
Snapshots3.0
Snapshots, cloning3.1.3
Snapshots, user-serviceable3.0.3
Tiering (Deprecated)3.1.2
Volume Shadow Copy (VSS)3.1.3

Table 1.5. Features supported by volume type

Volume TypeShardingTiering (Deprecated)QuotaSnapshotsGeo-RepBitrot
Arbitrated-ReplicatedYesNoYesYesYesYes
DistributedNoYesYesYesYesYes
Distributed-DispersedNoYesYesYesYesYes
Distributed-ReplicatedYesYesYesYesYesYes
ReplicatedYesYesYesYesYesYes
ShardedN/ANoNoNoYesNo
Tiered (Deprecated)NoN/ALimited[a]Limited[a]Limited[a]Limited[a]

Note

Red Hat Gluster Storage is not supported on Red Hat Enterprise Linux 6 (RHEL 6) from 3.5 Batch Update 1 onwards.

Table 1.6. Features supported by client protocol

FeatureFUSEGluster-NFSNFS-GaneshaSMB
ArbiterYesYesYesYes
Bitrot detectionYesYesNoYes
dm-cacheYesYesYesYes
Encryption (TLS-SSL)YesYesYesYes
Erasure codingYesYesYesYes
Export subdirectoryYesYesYesN/A
Geo-replicationYesYesYesYes
Quota (Deprecated)

Warning

Using QUOTA feature is considered to be deprecated in Red Hat Gluster Storage 3.5.3. Red Hat no longer recommends to use this feature and does not support it on new deployments and existing deployments that upgrade to Red Hat Gluster Storage 3.5.3.
YesYesYesYes
RDMA (Deprecated)

Warning

Using RDMA as a transport protocol is considered deprecated in Red Hat Gluster Storage 3.5. Red Hat no longer recommends the use of this feature, and does not support it on new deployments and existing deployments that upgrade to Red Hat Gluster Storage 3.5.3.
YesNoNoNo
SnapshotsYesYesYesYes
Snapshot cloningYesYesYesYes
Snapshot mountYesNoNoNo
Tiering (Deprecated)

Warning

Using tiering feature is considered deprecated in Red Hat Gluster Storage 3.5. Red Hat no longer recommends the use of this feature, and does not support it on new deployments and existing deployments that upgrade to Red Hat Gluster Storage 3.5.3.
YesYesN/AN/A

1.7. Red Hat Gluster Storage Support Matrix

This section lists all the supported Red Hat Enterprise Linux version for a particular Red Hat Gluster Storage release.

Table 1.7. Red Hat Gluster Storage Support Matrix

Red Hat Enterprise Linux version Red Hat Gluster Storage version
6.5 3.0
6.6 3.0.2, 3.0.3, 3.0.4
6.73.1, 3.1.1, 3.1.2
6.83.1.3
6.93.2
6.93.3
6.93.3.1
6.103.4, 3.5
7.13.1, 3.1.1
7.23.1.2
7.23.1.3
7.33.2
7.43.2
7.43.3
7.43.3.1
7.53.3.1, 3.4
7.63.3.1, 3.4
7.73.5, 3.5.1
7.83.5.1, 3.5.2
7.93.5.3, 3.5.4, 3.5.5, 3.5.6, 3.5.7
8.23.5.2, 3.5.3
8.33.5.3
8.43.5.4
8.53.5.5, 3.5.6
8.63.5.7

Chapter 2. Installing Red Hat Gluster Storage

Red Hat Gluster Storage can be installed in a data center using Red Hat Gluster Storage Server On-Premise.
This chapter describes the three different methods for installing Red Hat Gluster Storage Server: using an ISO image, using a PXE server, or using the Red Hat Satellite Server.
For information on launching Red Hat Gluster Storage Server for Public Cloud, see the Red Hat Gluster Storage Administration Guide.

Warning

Gluster-NFS is considered deprecated as of Red Hat Gluster Storage 3.5. Red Hat no longer recommends the use of Gluster-NFS, and does not support its use in new deployments and existing deployments that upgrade to Red Hat Gluster Storage 3.5.3. You can use NFS-Ganesha as an alternative.

Important

  • Technology preview packages will also be installed with this installation of Red Hat Gluster Storage Server. For more information about the list of technology preview features, see chapter Technology Previews in the Red Hat Gluster Storage 3.5 Release Notes.
  • When you clone a virtual machine that has Red Hat Gluster Storage Server installed, you need to remove the /var/lib/glusterd/glusterd.info file (if present) before you clone. If you do not remove this file, all cloned machines will have the same UUID. The file will be automatically recreated with a UUID on initial start-up of the glusterd daemon on the cloned virtual machines.

2.1. Obtaining Red Hat Gluster Storage

This chapter details the steps to obtain the Red Hat Gluster Storage software.

2.1.1. Obtaining Red Hat Gluster Storage Server for On-Premise

Visit the Software & Download Center in the Red Hat Customer Service Portal (https://access.redhat.com/downloads) to obtain the Red Hat Gluster Storage Server for On-Premise installation ISO image files. Use a valid Red Hat Subscription to download the full installation files, obtain a free evaluation installation, or follow the links in this page to purchase a new Red Hat Subscription.
To download the Red Hat Gluster Storage Server installation files using a Red Hat Subscription or a Red Hat Evaluation Subscription:
  1. Visit the Red Hat Customer Service Portal at https://access.redhat.com/login and enter your user name and password to log in.
  2. Click Downloads to visit the Software & Download Center.
  3. In the Red Hat Gluster Storage Server area, click Download Software to download the latest version of the software.

2.1.2. Obtaining Red Hat Gluster Storage Server for Public Cloud

Red Hat Gluster Storage Server for Public Cloud is pre-integrated, pre-verified, and ready to run the Amazon Machine Image (AMI). This AMI provides a fully POSIX-compatible, highly available, scale-out NAS and object storage solution for the Amazon Web Services (AWS) public cloud infrastructure.
For more information about obtaining access to AMI, see https://access.redhat.com/knowledge/articles/145693.

2.2. Installing from an ISO Image

Important

The ISO method is no longer available for Red Hat Enterprise Linux 6. For more information, see chapter Deprecated Features in the Red Hat Gluster Storage 3.5 Release Notes.

2.2.1. Installing Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux

Important

Virtual Data Optimizer (VDO) volumes, which are supported in Red Hat Enterprise Linux 7.5, are not currently supported in Red Hat Gluster Storage. VDO is supported only when used as part of Red Hat Hyperconverged Infrastructure for Virtualization 2.0. See Understanding VDO for more information.
To install Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux
  1. Download the ISO image file for Red Hat Gluster Storage Server as described in Section 2.1, “Obtaining Red Hat Gluster Storage”
  2. In the Welcome to Red Hat Gluster Storage 3.5 screen, select the language that will be used for the rest of the installation and click Continue. This selection will also become the default for the installed system, unless changed later.

    Note

    One language is pre-selected by default on top of the list. If network access is configured at this point (for example, if you booted from a network server instead of local media), the pre-selected language will be determined based on automatic location detection using the GeoIP module.
  3. The Installation Summary screen is the central location for setting up an installation.
    Installation Summary For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 8

    Figure 2.1. Installation Summary For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 8

    Installation Summary for Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 7

    Figure 2.2. Installation Summary for Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 7

    Instead of directing you through consecutive screens, the Red Hat Gluster Storage 3.5 installation program on Red Hat Enterprise Linux 7.7 and later allows you to configure the installation in the order you choose.
    Select a menu item to configure a section of the installation. When you have completed configuring a section, or if you would like to complete that section later, click the Done button located in the upper left corner of the screen.
    Only sections marked with a warning symbol are mandatory. A note at the bottom of the screen warns you that these sections must be completed before the installation can begin. The remaining sections are optional. Beneath each section's title, the current configuration is summarized. Using this you can determine whether you need to visit the section to configure it further.
    The following list provides a brief information of each of the menu item on the Installation Summary screen:
    • Date & Time

      Important

      Setting up NTP is mandatory for Gluster installation.
      To configure NTP, perform the following steps:
      1. Click Date & Time and specify a time zone to maintain the accuracy of the system clock.
      2. Toggle the Network Time switch to ON.

        Note

        By default, the Network Time switch is enabled if you are connected to the network.
      3. Click the configuration icon to add new NTP servers or select existing servers.
      4. Once you have made your addition or selection, click Done to return to the Installation Summary screen.

      Note

      NTP servers might be unavailable at the time of installation. In such a case, enabling them will not set the time automatically. When the servers are available, the date and time will be updated.

    • Language Support

      To install support for additional locales and language dialects, select Language Support.

    • Keyboard Configuration

      To add multiple keyboard layouts to your system, select Keyboard.

    • Installation Source

      To specify a file or a location to install Red Hat Enterprise Linux from, select Installation Source. On this screen, you can choose between locally available installation media, such as a DVD or an ISO file, or a network location.

    • Network & Hostname

      To configure essential networking features for your system, select Network & Hostname.

      Important

      When the Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 7.7 and later, installation finishes and the system boots for the first time, any network interfaces which you configured during the installation will be activated. However, the installation does not prompt you to configure network interfaces on some common installation paths - for example, when you install Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 7.5 from a DVD to a local hard drive.
      When you install Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 7.7 and later, from a local installation source to a local storage device, be sure to configure at least one network interface manually if you require network access when the system boots for the first time. You will also need to set the connection to connect automatically after boot when editing the configuration.
    • Software Selection

      To specify which packages will be installed, select Software Selection. If you require the following optional Add-Ons, then select the required Add-Ons and click Done:

      • RH-Gluster-AD-Integration
      • RH-Gluster-NFS-Ganesha
      • RH-Gluster-Samba-Server
    • Installation Destination

      To select the disks and partition the storage space on which you will install Red Hat Gluster Storage, select Installation Destination. For more information, see the Installation Destination section in the Red Hat Enterprise Linux 7 Installation Guide.

    • Kdump

      Kdump is a kernel crash dumping mechanism which, in the event of a system crash, captures information that can be invaluable in determining the cause of the crash. Use this option to select whether or not to use Kdump on the system

  4. After making the necessary configurations, click Begin Installation on the Installation Summary screen.

    Warning

    Up to this point in the installation process, no lasting changes have been made on your computer. When you click Begin Installation, the installation program will allocate space on your hard drive and start to transfer Red Hat Gluster Storage into this space. Depending on the partitioning option that you chose, this process might include erasing data that already exists on your computer.
    To revise any of the choices that you made up to this point, return to the relevant section of the Installation Summary screen. To cancel installation completely, click Quit or switch off your computer.
    If you have finished customizing the installation and are certain that you want to proceed, click Begin Installation.
    After you click Begin Installation, allow the installation process to complete. If the process is interrupted, for example, by you switching off or resetting the computer, or by a power outage, you will probably not be able to use your computer until you restart and complete the Red Hat Gluster Storage installation process
  5. Once you click Begin Installation, the progress screen appears. Red Hat Gluster Storage reports the installation progress on the screen as it writes the selected packages to your system. Following is a brief description of the options on this screen:
    • Root Password

      The Root Password menu item is used to set the password for the root account. The root account is used to perform critical system management and administration tasks. The password can be configured either while the packages are being installed or afterwards, but you will not be able to complete the installation process until it has been configured.

    • User Creation

      Creating a user account is optional and can be done after installation, but it is recommended to do it on this screen. A user account is used for normal work and to access the system. Best practice suggests that you always access the system via a user account and not the root account.

  6. After the installation is completed, click Reboot to reboot your system and begin using Red Hat Gluster Storage.

2.3. Subscribing to the Red Hat Gluster Storage Server Channels

After you have successfully installed Red Hat Gluster Storage, you must subscribe to the required channels:

Note

If you used Red Hat Satellite or Layered Installation method to install Red Hat Gluster Storage 3.x, then you can skip the steps mentioned in this chapter. For more information regarding layered installation, see Section 2.4, “Installing Red Hat Gluster Storage Server on Red Hat Enterprise Linux (Layered Install)”
Using Subscription Manager

  1. Register the System with Subscription Manager

    Run the following command and enter your Red Hat Network user name and password to register the system with Subscription Manager: 
    # subscription-manager register

  2. Identify Available Entitlement Pools

    Run the following commands to find entitlement pools containing the repositories required to install Red Hat Gluster Storage: 
    # subscription-manager list --available | grep -A8 "Red Hat Enterprise Linux Server"
# subscription-manager list --available | grep -A8 "Red Hat Storage"

  3. Attach Entitlement Pools to the System

    Use the pool identifiers located in the previous step to attach the Red Hat Enterprise Linux Server and Red Hat Gluster Storage entitlements to the system. Run the following command to attach the entitlements: 
    # subscription-manager attach --pool=[POOLID]
    For example: 
    # subscription-manager attach --pool=8a85f9814999f69101499c05aa706e47

  4. Enable the Required Channels for Red Hat Gluster Storage on Red Hat Enterprise Linux

    For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 6.7 and later

    1. Run the following commands to enable the repositories required to install Red Hat Gluster Storage: 
      # subscription-manager repos --enable=rhel-6-server-rpms
# subscription-manager repos --enable=rhel-scalefs-for-rhel-6-server-rpms
# subscription-manager repos --enable=rhs-3-for-rhel-6-server-rpms
    2. If you require Samba, then enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-6-server-rpms
    3. NFS-Ganesha is not supported on Red Hat Enterprise Linux 6 based installations.
    For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 7.7 and later

    1. Run the following commands to enable the repositories required to install Red Hat Gluster Storage 
      # subscription-manager repos --enable=rhel-7-server-rpms
# subscription-manager repos --enable=rh-gluster-3-for-rhel-7-server-rpms
    2. If you require Samba, then enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-7-server-rpms
    3. For Red Hat Gluster Storage 3.5, if NFS-Ganesha is required, then enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-nfs-for-rhel-7-server-rpms --enable=rhel-ha-for-rhel-7-server-rpms
    4. For Red Hat Gluster Storage 3.5, if you require CTDB, then enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-7-server-rpms
    For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 8.2 and later

    1. Run the following commands to enable the repositories required to install Red Hat Gluster Storage 
      # subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms
# subscription-manager repos --enable=rhel-8-for-x86_64-appstream-rpms
# subscription-manager repos --enable=rh-gluster-3-for-rhel-8-x86_64-rpms
    2. If you require Samba, then enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-8-x86_64-rpms
    3. For Red Hat Gluster Storage 3.5, if NFS-Ganesha is required, then enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-nfs-for-rhel-8-x86_64-rpms --enable=rhel-8-for-x86_64-highavailability-rpms
    4. For Red Hat Gluster Storage 3.5, if you require CTDB, then enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-8-x86_64-rpms

  5. Verify if the Channels are Enabled

    Run the following command to verify if the channels are enabled: 
    # yum repolist
Using Red Hat Satellite Server

  1. Configure the Client System to Access Red Hat Satellite

    Configure the client system to access Red Hat Satellite. Refer section Registering Clients with Red Hat Satellite Server in Red Hat Satellite 5.6 Client Configuration Guide.

  2. Register to the Red Hat Satellite Server

    Run the following command to register the system to the Red Hat Satellite Server: 
    # rhn_register

  3. Register to the Standard Base Channel

    In the select operating system release page, select All available updates and follow the prompts to register the system to the standard base channel for Red Hat Enterprise Linux 6 - rhel-6-server-rpms. The standard base channel for Red Hat Enterprise Linux 7 is rhel-7-server-rpms. The standard base channel for Red Hat Enterprise Linux 8 is rhel-8-for-x86_64-baseos-rpms.

  4. Subscribe to the Required Red Hat Gluster Storage Server Channels

    For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 6.7 and later

    1. Run the following command to subscribe the system to the required Red Hat Gluster Storage server channel: 
      # rhn-channel --add \
--channel rhel-6-server-rpms \
--channel rhel-scalefs-for-rhel-6-server-rpms \
--channel rhs-3-for-rhel-6-server-rpms
    2. If you require Samba, then execute the following command to enable the required channel: 
      # rhn-channel --add --channel rh-gluster-3-samba-for-rhel-6-server-rpms
    3. NFS-Ganesha is not supported on Red Hat Enterprise Linux 6 based installations.
    For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 7.7 and later

    1. Run the following command to subscribe the system to the required Red Hat Gluster Storage server channels for Red Hat Enterprise Linux 7: 
      # rhn-channel --add \
--channel rhel-7-server-rpms \
--channel rh-gluster-3-for-rhel-7-server-rpms
    2. If you require Samba, then execute the following command to enable the required channel: 
      # rhn-channel --add --channel rh-gluster-3-samba-for-rhel-7-server-rpms
    3. For Red Hat Gluster Storage 3.5, for NFS-Ganesha enable the following channel: 
      # rhn-channel --add --channel rhel-x86_64-server-7-rh-gluster-3-nfs --channel rhel-x86_64-server-ha-7
    4. For Red Hat Gluster Storage 3.5, if CTDB is required, then enable the following channel: 
      # rhn-channel --add --channel rh-gluster-3-samba-for-rhel-7-server-rpms
    For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 8.2 and later

    1. Run the following command to subscribe the system to the required Red Hat Gluster Storage server channels for Red Hat Enterprise Linux 8: 
      # rhn-channel --add \
            --channel rh-gluster-3-for-rhel-8-x86_64-rpms
    2. If you require Samba, then execute the following command to enable the required channel: 
      # rhn-channel --add --channel rh-gluster-3-samba-for-rhel-8-x86_64-rpms
    3. For Red Hat Gluster Storage 3.5, for NFS-Ganesha enable the following channel: 
      # rhn-channel --add --channel rh-gluster-3-nfs-for-rhel-8-x86_64-rpms --channel rhel-8-for-x86_64-highavailability-rpms
    4. For Red Hat Gluster Storage 3.5, if CTDB is required, then enable the following channel: 
      # rhn-channel --add --channel rh-gluster-3-samba-for-rhel-8-x86_64-rpms

  5. Verify if the System is Registered Successfully

    Run the following command to verify if the system is registered successfully: 
    # rhn-channel --list
rhel-x86_64-server-7
rhel-x86_64-server-7-rh-gluster-3

2.4. Installing Red Hat Gluster Storage Server on Red Hat Enterprise Linux (Layered Install)

Layered install involves installing Red Hat Gluster Storage over Red Hat Enterprise Linux. For more information regarding supported Red Hat Enterprise Linux version for a particular Red Hat Gluster Storage release, see Section 1.7, “Red Hat Gluster Storage Support Matrix”.

Important

Virtual Data Optimizer (VDO) volumes, which are supported in Red Hat Enterprise Linux 7.5, are not currently supported in Red Hat Gluster Storage. VDO is supported only when used as part of Red Hat Hyperconverged Infrastructure for Virtualization 2.0. See Understanding VDO for more information.

Important

It is recommended to create a separate /var partition that is large enough (50GB - 100GB) for log files, geo-replication related miscellaneous files, and other files.

  1. Perform a base install of Red Hat Enterprise Linux Server

    Red Hat Gluster Storage is supported on Red Hat Enterprise Linux 7 (RHEL 7) and Red Hat Enterprise Linux 8 (RHEL 8). We highly recommend you to choose the latest RHEL versions.

    Important

    Red Hat Gluster Storage is not supported on Red Hat Enterprise Linux 6 (RHEL 6) from 3.5 Batch Update 1 onwards. See, Table 1.3, “Version Details”

  2. Register the System with Subscription Manager

    To register the system with subscription manager, refer Section 2.3, “Subscribing to the Red Hat Gluster Storage Server Channels”

  3. Kernel Version Requirement

    Red Hat Gluster Storage requires the kernel-2.6.32-431.17.1.el6 version or higher to be used on the system. Verify the installed and running kernel versions by running the following command: 
    # rpm -q kernel
kernel-2.6.32-431.el6.x86_64
kernel-2.6.32-431.17.1.el6.x86_64
    # uname -r
2.6.32-431.17.1.el6.x86_64

  4. Update all packages

    Ensure that all packages are up to date by running the following command. 
    # yum update

    Important

    If any kernel packages are updated, reboot the system with the following command. 
    # shutdown -r now

  5. Install Red Hat Gluster Storage

    Run the following command to install Red Hat Gluster Storage: 
    # yum install redhat-storage-server
    1. To install Samba, see Chapter 3, Deploying Samba on Red Hat Gluster Storage
    2. To install NFS-Ganesha, see Chapter 4, Deploying NFS-Ganesha on Red Hat Gluster Storage

  6. Reboot

    Reboot the system.

2.5. Installing from a PXE Server

To boot your computer using a PXE server, you need a properly configured server and a network interface in your computer that supports PXE.
Configure the computer to boot from the network interface. This option is in the BIOS, and may be labeled Network Boot or Boot Services. Once you properly configure PXE booting, the computer can boot the Red Hat Gluster Storage Server installation system without any other media.
To boot a computer from a PXE server:
  1. Ensure that the network cable is attached. The link indicator light on the network socket should be lit, even if the computer is not switched on.
  2. Switch on the computer.
  3. A menu screen appears. Press the number key that corresponds to the preferred option.
If your computer does not boot from the netboot server, ensure that the BIOS is configured so that the computer boots first from the correct network interface. Some BIOS systems specify the network interface as a possible boot device, but do not support the PXE standard. See your hardware documentation for more information.

2.6. Installing from Red Hat Satellite Server

Ensure that the firewall settings are configured so that the required ports are open. For a list of port numbers, see Section 1.4, “Port Information”.

2.6.1. Using Red Hat Satellite Server 6.x

Note

For detailed steps for Satellite, see Red Hat Satellite documentation.
  1. Create a new manifest file and upload the manifest in the Satellite 6 server.
  2. Search for the required Red Hat Gluster Storage repositories and enable them.
  3. Synchronize all repositories enabled for Red Hat Gluster Storage.
  4. Create a new content view and add all the required products.
  5. Publish the content view and create an activation key.
  6. Register the required clients 
    # rpm -Uvh satellite-server-host-address/pub/katello-ca-consumer-latest.noarch.rpm
# subscription-manager register --org=”Organization_Name” --activationkey=”Activation_Key”
  7. Identify available entitlement pools 
    # subscription-manager list --available
  8. Attach entitlement pools to the system 
    # subscription-manager attach --pool=Pool_ID
  9. Subscribe to required channels
    1. Enable the RHEL and Gluster channel
      For Red Hat Enterprise Linux 8 
      # subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms --enable=rhel-8-for-x86_64-appstream-rpms
# subscription-manager repos --enable=rh-gluster-3-for-rhel-8-x86_64-rpms
      For Red Hat Enterprise Linux 7 
      # subscription-manager repos --enable=rhel-7-server-rpms --enable=rh-gluster-3-for-rhel-7-server-rpms
    2. If you require Samba, enable its repository
      For Red Hat Enterprise Linux 8 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-8-x86_64-rpms
      For Red Hat Enterprise Linux 7 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-7-server-rpms
    3. If you require NFS-Ganesha, enable its repository
      For Red Hat Enterprise Linux 8 
      # subscription-manager repos --enable=rh-gluster-3-nfs-for-rhel-8-x86_64-rpms
      For Red Hat Enterprise Linux 7 
      # subscription-manager repos --enable=rh-gluster-3-nfs-for-rhel-7-server-rpms
    4. If you require HA, enable its repository
      For Red Hat Enterprise Linux 8 
      # subscription-manager repos --enable=rhel-8-for-x86_64-highavailability-rpms
      For Red Hat Enterprise Linux 7 
      # subscription-manager repos --enable=rhel-ha-for-rhel-7-server-rpms
    5. If you require gdeploy, enable the Ansible repository
      For Red Hat Enterprise Linux 8 
      # subscription-manager repos --enable=ansible-2-for-rhel-8-x86_64-rpms
      For Red Hat Enterprise Linux 7 
      # subscription-manager repos --enable=rhel-7-server-ansible-2-rpms
  10. Install Red Hat Gluster Storage 
    # yum install redhat-storage-server

2.6.2. Using Red Hat Satellite Server 5.x

Creating the Activation Key

For more information on how to create an activation key, see Activation Keys in the Reference Guide.

  • In the Details tab of the Activation Keys screen, select Red Hat Enterprise Linux Server (v.6 for 64-bit x86_64) from the Base Channels drop-down list.
    Base Channels

    Figure 2.3. Base Channels

  • In the Child Channels tab of the Activation Keys screen, select the following child channels: 
    RHEL Server Scalable File System (v. 6 for x86_64)
Red Hat Gluster Storage Server 3 (RHEL 6 for x86_64)
    If you require the Samba package, then select the following child channel: 
    Red Hat Gluster 3 Samba (RHEL 6 for x86_64)
    Child Channels

    Figure 2.4. Child Channels

  • In the Packages tab of the Activation Keys screen, enter the following package name: 
    redhat-storage-server
    Package

    Figure 2.5. Package

    • If you require the Samba package, then enter the following package name: 
      samba
Creating the Kickstart Profile

For more information on creating a kickstart profile, see Kickstart in the Reference Guide.

  • When creating a kickstart profile, the following Base Channel and Tree must be selected.
    Base Channel: Red Hat Enterprise Linux Server (v.6 for 64-bit x86_64)
    Tree: ks-rhel-x86_64-server-6-6.5
  • Do not associate any child channels with the kickstart profile.
  • Associate the previously created activation key with the kickstart profile.

Important

  • By default, the kickstart profile chooses md5 as the hash algorithm for user passwords.
    You must change this algorithm to sha512 by providing the following settings in the auth field of the Kickstart Details, Advanced Options page of the kickstart profile: 
    --enableshadow --passalgo=sha512
  • After creating the kickstart profile, you must change the root password in the Kickstart Details, Advanced Options page of the kickstart profile and add a root password based on the prepared sha512 hash algorithm.
Installing Red Hat Gluster Storage Server using the Kickstart Profile

For more information on installing Red Hat Gluster Storage Server using a kickstart profile, see Kickstart in the Reference Guide.

2.7. Managing the glusterd Service

After installing Red Hat Gluster Storage, the glusterd service automatically starts on all the servers in the trusted storage pool. The service can be manually started and stopped by using the glusterd service commands. For more information on creating trusted storage pools, see the Red Hat Gluster Storage 3.5 Administration Guide.
Use Red Hat Gluster Storage to dynamically change the configuration of glusterFS volumes without restarting servers or remounting volumes on clients. The glusterFS daemon glusterd also offers elastic volume management.
Use the gluster CLI commands to decouple logical storage volumes from physical hardware. This allows the user to grow, shrink, and migrate storage volumes without any application downtime. As storage is added to the cluster, the volumes are distributed across the cluster. This distribution ensures that the cluster is always available despite changes to the underlying hardware.

2.7.1. Manually Starting and Stopping glusterd

Use the following instructions to manually start and stop the glusterd service.
  • Manually start glusterd as follows: 
    # /etc/init.d/glusterd start
    or 
    # service glusterd start
  • Manually stop glusterd as follows: 
    # /etc/init.d/glusterd stop
    or 
    # service glusterd stop

2.8. Installing Ansible to Support Gdeploy

Note

Consult with your IT department to confirm your organization’s supported download instructions for Ansible.
gDeploy depends on Ansible to execute the playbooks and modules. You must install Ansible 2.5 or higher to use gDeploy.
  1. Execute the following command to enable the repository required to install Ansible:
    For Red Hat Enterprise Linux 8 
    # subscription-manager repos --enable=ansible-2-for-rhel-8-x86_64-rpms
    For Red Hat Enterprise Linux 7 
    # subscription-manager repos --enable=rhel-7-server-ansible-2-rpms
  2. Install ansible by executing the following command: 
    # yum install ansible

2.9. Installing Native Client

Native Client is a FUSE-based client in the user space. Native Client is the recommended method for accessing Red Hat Gluster Storage volumes when high concurrency and high write performance is required.

Note

  • Red Hat Gluster Storage server supports the Native Client version which is the same as the server version and the preceding version of Native Client . For list of releases see: https://access.redhat.com/solutions/543123.
  • From Red Hat Gluster Storage 3.5 batch update 7 onwards, glusterfs-6.0-62 and higher version of glusterFS Native Client is only available via rh-gluster-3-client-for-rhel-8-x86_64-rpms for Red Hat Gluster Storage based on Red Hat Enterprise Enterprise Linux (RHEL 8) and rh-gluster-3-client-for-rhel-7-server-rpms for Red Hat Gluster Storage based on RHEL 7.

Use the Command Line to Register and Subscribe a System to Red Hat Subscription Management

Register the system using the command line, and subscribe to the correct repositories.

Prerequisites

  • Know the user name and password of the Red Hat Subscription Manager account with Red Hat Gluster Storage entitlements.
  1. Run the subscription-manager register command to list the available pools. Select the appropriate pool and enter your Red Hat Subscription Manager user name and password to register the system with Red Hat Subscription Manager. 
    # subscription-manager register
  2. Depending on your client, run one of the following commands to subscribe to the correct repositories.
    • For Red Hat Enterprise Linux 8 clients: 
      # subscription-manager repos --enable=rh-gluster-3-client-for-rhel-8-x86_64-rpms
    • For Red Hat Enterprise Linux 7.x clients: 
      # subscription-manager repos --enable=rhel-7-server-rpms --enable=rh-gluster-3-client-for-rhel-7-server-rpms

      Note

      The following command can also be used, but Red Hat Gluster Storage may deprecate support for this repository in future releases. 
      # subscription-manager repos --enable=rhel-7-server-rh-common-rpms
    • For Red Hat Enterprise Linux 6.1 and later clients: 
      # subscription-manager repos --enable=rhel-6-server-rpms --enable=rhel-6-server-rhs-client-1-rpms
    For more information, see Section 3.1 Registering and attaching a system using the Command Line in Using and Configuring Red Hat Subscription Management.
  3. Verify that the system is subscribed to the required repositories. 
    # yum repolist

Use the Web Interface to Register and Subscribe a System

Register the system using the web interface, and subscribe to the correct channels.

Prerequisites

  • Know the user name and password of the Red Hat Subsrciption Management (RHSM) account with Red Hat Gluster Storage entitlements.
  1. Log on to Red Hat Subscription Management (https://access.redhat.com/management).
  2. Click the Systems link at the top of the screen.
  3. Click the name of the system to which the Red Hat Gluster Storage Native Client channel must be appended.
  4. Click Alter Channel Subscriptions in the Subscribed Channels section of the screen.
  5. Expand the node for Additional Services Channels for Red Hat Enterprise Linux 8 for x86_64, Red Hat Enterprise Linux 7 for x86_64, Red Hat Enterprise Linux 6 for x86_64 or Red Hat Enterprise Linux 5 for x86_64 depending on the client platform.
  6. Click the Change Subscriptions button to finalize the changes.
    When the page refreshes, select the Details tab to verify the system is subscribed to the appropriate channels.

Install Native Client Packages

Install Native Client packages from Red Hat Network
  • Run the yum install command to install the native client RPM packages. 
    # yum install glusterfs glusterfs-fuse

Chapter 3. Deploying Samba on Red Hat Gluster Storage

3.1. Prerequisites

To install Samba on Red Hat Gluster Storage you require access to the installation media either through an ISO or a properly configured software repository. The Red Hat Gluster Storage server requirements are:
  1. You must install Red Hat Gluster Storage Server on the target server.

    Warning

    • For layered installation of Red Hat Gluster Storage, ensure to have only the default Red Hat Enterprise Linux server installation, without the Samba or CTDB packages installed from Red Hat Enterprise Linux.
    • The Samba version 3 is being deprecated from Red Hat Gluster Storage 3.0 Update 4. Further updates will not be provided for samba-3.x. It is recommended that you upgrade to Samba-4.x, which is provided in a separate channel or repository, for all updates including the security updates.
    • CTDB version 2.5 is not supported from Red Hat Gluster Storage 3.1 Update 2. To use CTDB in Red Hat Gluster Storage 3.1.2 and later, you must upgrade the system to CTDB 4.x, which is provided in the Samba channel of Red Hat Gluster Storage.
    • Downgrade of Samba from Samba 4.x to Samba 3.x is not supported.
    • Ensure that Samba is upgraded on all the nodes simultaneously, as running different versions of Samba in the same cluster will lead to data corruption.
  2. Enable the channel where the Samba packages are available:
    For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 6.x

    1. If you have registered your machine using Red Hat Subscription Manager or Satellite server-6.x, enable the repository by running the following command: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-6-server-rpms
    For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 7.7 and later

    1. If you have registered your machine using Red Hat Subscription Manager or Satellite server-7.x, enable the repository by running the following command: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-7-server-rpms
    For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 8.2 and later

    1. If you have registered your machine using Red Hat Subscription Manager or Satellite server-8.x, enable the repository by running the following command: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-8-x86_64-rpms

3.2. Installing Samba Using ISO

During the installation of Red Hat Gluster Storage, select the RH-Gluster-Samba-Server component, in the Customizing the Software Selection screen.
If you want to integrate with Active Directory, select both RH-Gluster-Samba-Server and RH-Gluster-AD-Integration components.
Description

Figure 3.1. Customize Packages

For more information about installing Red Hat Gluster Storage using an ISO, see Section 2.2, “Installing from an ISO Image”

3.3. Installing Samba Using yum

To install Samba using yum, install the Samba group using the following command: 
# yum groupinstall RH-Gluster-Samba-Server
If you require Samba Active Directory integration with gluster, execute the following command: 
# yum groupinstall RH-Gluster-AD-Integration
For more information about installing Red Hat Gluster Storage using yum, see Section 2.4, “Installing Red Hat Gluster Storage Server on Red Hat Enterprise Linux (Layered Install)”.
Samba can also be installed by following these steps:

  1. To install the basic Samba packages, execute the following command: 
    # yum install samba
  2. If you require the smbclient on the server, then execute the following command: 
    # yum install samba-client
  3. If you require an Active directory setup, then execute the following commands: 
    # yum install samba-winbind
# yum install samba-winbind-clients
# yum install samba-winbind-krb5-locator
  4. Verify if the following packages are installed. 
    samba-libs
samba-winbind-krb5-locator
samba-winbind-modules
samba-vfs-glusterfs
samba-winbind
samba-client
samba-common
samba-winbind-clients
samba

Note

For instructions how to configure Samba, see the RedHat Gluster Storage 3.5 Admin Guide SMB.

Chapter 4. Deploying NFS-Ganesha on Red Hat Gluster Storage

NFS-Ganesha can be installed using any of the following methods:
  • Installing NFS-Ganesha using yum
  • Installing NFS-Ganesha during an ISO Installation

Warning

Gluster-NFS is considered deprecated as of Red Hat Gluster Storage 3.5. Red Hat no longer recommends the use of Gluster-NFS, and does not support its use in new deployments and existing deployments that upgrade to Red Hat Gluster Storage 3.5.3. You can use NFS-Ganesha as an alternative.

4.1. Prerequisites

To install NFS-Ganesha on Red Hat Gluster Storage you require access to the installation media either through an ISO or a properly configured software repository.
For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 8.2 and later

Enable the channel where the NFS-Ganesha packages are available:

  1. If you have registered your machine using Red Hat Subscription Manager, enable the repository by running the following command: 
    # subscription-manager repos --enable=rh-gluster-3-nfs-for-rhel-8-x86_64-rpms
    To add the HA repository, execute the following command: 
    # subscription-manager repos --enable=rhel-8-for-x86_64-highavailability-rpms
  2. If you have registered your machine using Satellite server, enable the channel by running the following command: 
    # rhn-channel --add --channel rh-gluster-3-nfs-for-rhel-8-x86_64-rpms
    To subscribe to the HA channel, execute the following command: 
    # rhn-channel --add --channel rhel-8-for-x86_64-highavailability-rpms
For Red Hat Gluster Storage 3.5 on Red Hat Enterprise Linux 7.7 and later

Enable the channel where the NFS-Ganesha packages are available:

  1. If you have registered your machine using Red Hat Subscription Manager, enable the repository by running the following command: 
    # subscription-manager repos --enable=rh-gluster-3-nfs-for-rhel-7-server-rpms
    To add the HA repository, execute the following command: 
    # subscription-manager repos --enable=rhel-ha-for-rhel-7-server-rpms
  2. If you have registered your machine using Satellite server, enable the channel by running the following command: 
    # rhn-channel --add --channel rhel-x86_64-server-7-rh-gluster-3-nfs
    To subscribe to the HA channel, execute the following command: 
    # rhn-channel --add --channel rhel-x86_64-server-ha-7

4.2. Installing NFS-Ganesha during an ISO Installation

For more information about installing Red Hat Gluster Storage using an ISO image, see Section 2.2, “Installing from an ISO Image”.
  1. While installing Red Hat Storage using an ISO, in the Customizing the Software Selection screen, select RH-Gluster-NFS-Ganesha and click Next.
  2. Proceed with the remaining installation steps for installing Red Hat Gluster Storage. For more information on how to install Red Hat Storage using an ISO, see Installing from an ISO Image.

4.3. Installing NFS-Ganesha using yum

Execute the following steps to install NFS-Ganesha:
  1. The glusterfs-ganesha package can be installed using the following command: 
    # yum install glusterfs-ganesha
    NFS-Ganesha is installed along with the above package. nfs-ganesha-gluster and HA packages are also installed.

    Important

    Run the following command to install nfs-ganesha-selinux on Red Hat Enterprise Linux 7: 
    # yum install nfs-ganesha-selinux
    Run the following command to install nfs-ganesha-selinux on Red Hat Enterprise Linux 8: 
     # dnf install glusterfs-ganesha

Note

For instructions on how to configure NFS Ganesha, see NFS Ganesha in Red Hat Gluster Storage 3.5 Administration Guide.

Chapter 5. Upgrading to Red Hat Gluster Storage 3.5

This chapter describes the procedure to upgrade RHGS 3.5 with same RHEL platform major version. To upgrade from RHEL 6 based RHGS to RHEL 7 based RHGS, refer to Upgrading Red Hat Gluster Storage

Important

Red Hat Enterprise Linux 8 (RHEL 8) is supported only for new installations of Red Hat Gluster Storage 3.5.2 and above.
Upgrades to RHEL 8 based Red Hat Gluster Storage are supported from Red Hat Gluster Storage 3.5.2 onwards. Upgrade is not supported from RHEL 7 based RHGS cluster to RHEL 8 based RHGS cluster.

Upgrade support limitations

5.1. Offline Upgrade to Red Hat Gluster Storage 3.5

Warning

Before you upgrade, be aware of changed requirements that exist after Red Hat Gluster Storage 3.1.3. If you want to access a volume being provided by a Red Hat Gluster Storage 3.1.3 or higher server, your client must also be using Red Hat Gluster Storage 3.1.3 or higher. Accessing volumes from other client versions can result in data becoming unavailable and problems with directory operations. This requirement exists because Red Hat Gluster Storage 3.1.3 contained a number of changes that affect how the Distributed Hash Table works in order to improve directory consistency and remove the effects seen in BZ#1115367 and BZ#1118762.

Important

In Red Hat Enterprise Linux 7 based Red Hat Gluster Storage 3.1 and higher, updating reloads firewall rules. All runtime-only changes made before the reload are lost, so ensure that any changes you want to keep are made persistently.

5.1.1. Upgrading to Red Hat Gluster Storage 3.5 for Systems Subscribed to Red Hat Subscription Manager

Procedure 5.1. Before you upgrade

  1. Back up the following configuration directory and files in a location that is not on the operating system partition.
    • /var/lib/glusterd
    • /etc/samba
    • /etc/ctdb
    • /etc/glusterfs
    • /var/lib/samba
    • /var/lib/ctdb
    • /var/run/gluster/shared_storage/nfs-ganesha

    Note

    With the release of 3.5 Batch Update 3, the mount point of shared storage is changed from /var/run/gluster/ to /run/gluster/ .
    If you use NFS-Ganesha, back up the following files from all nodes:
    • /run/gluster/shared_storage/nfs-ganesha/exports/export.*.conf
    • /etc/ganesha/ganesha.conf
    • /etc/ganesha/ganesha-ha.conf
    If upgrading from Red Hat Gluster Storage 3.3 to 3.4 or subsequent releases, back up all xattr by executing the following command individually on the brick root(s) for all nodes: 
    # find ./ -type d ! -path "./.*" ! -path "./" | xargs getfattr -d -m. -e hex > /var/log/glusterfs/xattr_dump_brick_name
  2. Unmount gluster volumes from all clients. On a client, use the following command to unmount a volume from a mount point. 
    # umount mount-point
  3. If you use NFS-Ganesha, run the following on a gluster server to disable the nfs-ganesha service: 
    # gluster nfs-ganesha disable
  4. On a gluster server, disable the shared volume. 
    # gluster volume set all cluster.enable-shared-storage disable
  5. Stop all volumes 
    # for vol in `gluster volume list`; do gluster --mode=script volume stop $vol; sleep 2s; done
  6. Verify that all volumes are stopped. 
    # gluster volume info
  7. Stop the glusterd services on all servers using the following command: 
    # service glusterd stop
# pkill glusterfs
# pkill glusterfsd
  8. Stop the pcsd service. 
    # systemctl stop pcsd

Procedure 5.2. Upgrade using yum

Note

Verify that your system is not on the legacy Red Hat Network Classic update system. 
# migrate-rhs-classic-to-rhsm --status
If you are still on Red Hat Network Classic, run the following command to migrate to Red Hat Subscription Manager. 
# migrate-rhs-classic-to-rhsm --rhn-to-rhsm
Then verify that your status has changed. 
# migrate-rhs-classic-to-rhsm --status
  1. If you use Samba:
    1. For Red Hat Enterprise Linux 6.7 or higher, enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-6-server-rpms
      For Red Hat Enterprise Linux 7, enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-7-server-rpms
      For Red Hat Enterprise Linux 8, enable the following repository: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-8-x86_64-rpms
    2. Ensure that Samba is upgraded on all the nodes simultaneously, as running different versions of Samba in the same cluster will lead to data corruption.
      Stop the CTDB and SMB services.
      On Red Hat Enterprise Linux 7 and Red Hat Enterprise Linux 8: 
      # systemctl stop ctdb
      On Red Hat Enterprise Linux 6: 
      # service ctdb stop
      To verify that services are stopped, run: 
      # ps axf | grep -E '(ctdb|smb|winbind|nmb)[d]'
  2. Upgrade the server to Red Hat Gluster Storage 3.5. 
    # yum update
    Wait for the update to complete.

    Important

    Run the following command to install nfs-ganesha-selinux on Red Hat Enterprise Linux 7: 
    # yum install nfs-ganesha-selinux
    Run the following command to install nfs-ganesha-selinux on Red Hat Enterprise Linux 8: 
     # dnf install glusterfs-ganesha
  3. If you use Samba/CTDB, update the following files to replace META="all" with META="<ctdb_volume_name>", for example, META="ctdb":
    • /var/lib/glusterd/hooks/1/start/post/S29CTDBsetup.sh - This script ensures the file system and its lock volume are mounted on all Red Hat Gluster Storage servers that use Samba, and ensures that CTDB starts at system boot.
    • /var/lib/glusterd/hooks/1/stop/pre/S29CTDB-teardown.sh - This script ensures that the file system and its lock volume are unmounted when the CTDB volume is stopped.

      Note

      For RHEL based Red Hat Gluster Storage upgrading to 3.5 batch update 4 with Samba, the write-behind translator has to manually disabled for all existing samba volumes. 
      # gluster volume set <volname> performance.write-behind off
  4. Reboot the server to ensure that kernel updates are applied.
  5. Ensure that glusterd and pcsd services are started. 
    # systemctl start glusterd
# systemctl start pcsd

    Note

    During upgrade of servers, the glustershd.log file throws some “Invalid argument” errors during every index crawl (10 mins by default) on the upgraded nodes. It is *EXPECTED* and can be *IGNORED* until the op-version bump up, after which these errors are not triggered. Sample error message: 
    [2021-05-25 17:58:38.007134] E [MSGID: 114031] [client-rpc-fops_v2.c:216:client4_0_mkdir_cbk] 0-spvol-client-40: remote operation failed. Path: (null) [Invalid argument]
    If you are in op-version '70000' or lower, do not bump up the op-version to '70100' or higher until all the servers and clients are upgraded to the newer version.
  6. When all nodes have been upgraded, run the following command to update the op-version of the cluster. This helps to prevent any compatibility issues within the cluster. 
    # gluster volume set all cluster.op-version 70200

    Note

    70200 is the cluster.op-version value for Red Hat Gluster Storage 3.5. Ater upgrading the cluster-op version, enable the granular-entry-heal for the volume via the given command: 
    gluster volume heal $VOLNAME granular-entry-heal enable
    The feature is now enabled by default post upgrade to Red Hat Gluster Storage 3.5, but this will come into affect only after bumping up the op-version. Refer to Section 1.5, “Red Hat Gluster Storage Software Components and Versions” for the correct cluster.op-version value for other versions.

    Important

    If the op-version is bumped up to '70100' after upgrading the servers and before upgrading the clients, some internal metadata files under the root of the mount point named '.glusterfs-anonymous-inode-(gfid)' exposed to the older clients. The clients must not do any I/O or remove or touch contents in this directory. The clients must upgrade to 3.5.4 or higher version, then this directory becomes invisible to the clients.
  7. If you want to migrate from Gluster NFS to NFS Ganesha as part of this upgrade, install the NFS-Ganesha packages as described in Chapter 4, Deploying NFS-Ganesha on Red Hat Gluster Storage, and configure the NFS Ganesha cluster using the information in the NFS Ganesha section of the Red Hat Gluster Storage 3.5 Administration Guide.
  8. Start all volumes. 
    # for vol in `gluster volume list`; do gluster --mode=script volume start $vol; sleep 2s; done
  9. If you are using NFS-Ganesha:
    1. Copy the volume's export information from your backup copy of ganesha.conf to the new /etc/ganesha/ganesha.conf file.
      The export information in the backed up file is similar to the following: 
      %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.v1.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.v2.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.v3.conf"

      Note

      With the release of 3.5 Batch Update 3, the mount point of shared storage is changed from /var/run/gluster/ to /run/gluster/ .
    2. Copy the backup volume export files from the backup directory to /etc/ganesha/exports by running the following command from the backup directory: 
      # cp export.* /etc/ganesha/exports/
  10. Enable the shared volume. 
    # gluster volume set all cluster.enable-shared-storage enable
  11. Ensure that the shared storage volume is mounted on the server. If the volume is not mounted, run the following command: 
    # mount -t glusterfs hostname:gluster_shared_storage /var/run/gluster/shared_storage
  12. Ensure that the /var/run/gluster/shared_storage/nfs-ganesha directory is created. 
    # cd /var/run/gluster/shared_storage/
# mkdir nfs-ganesha
  13. Enable firewall settings for new services and ports. See Getting Started in the Red Hat Gluster Storage 3.5 Administration Guide.
  14. If you use Samba/CTDB:
    1. Mount /gluster/lock before starting CTDB by executing the following commands: 
      # mount <ctdb_volume_name>
# mount -t glusterfs server:/ctdb_volume_name /gluster/lock/
    2. Verify that the lock volume mounted correctly by checking for lock in the output of the mount command on any Samba server. 
      # mount | grep 'lock'
...
<hostname>:/<ctdb_volume_name>.tcp on /gluster/lock type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
    3. If all servers that host volumes accessed via SMB have been updated, then start the CTDB and Samba services by executing the following commands.
      On Red Hat Enterprise Linux 7 and Red Hat Enterprise Linux 8: 
      # systemctl start ctdb
      On Red Hat Enterprise Linux 6: 
      # service ctdb start
    4. To verify that the CTDB and SMB services have started, execute the following command: 
      ps axf | grep -E '(ctdb|smb|winbind|nmb)[d]'
  15. If you use NFS-Ganesha:
    1. Copy the ganesha.conf and ganesha-ha.conf files, and the /etc/ganesha/exports directory to the /var/run/gluster/shared_storage/nfs-ganesha directory. 
      # cd /etc/ganesha/
# cp ganesha.conf  ganesha-ha.conf /var/run/gluster/shared_storage/nfs-ganesha/
# cp -r exports/ /var/run/gluster/shared_storage/nfs-ganesha/
    2. Update the path of any export entries in the ganesha.conf file. 
      # sed -i 's/\/etc\/ganesha/\/var\/run\/gluster\/shared_storage\/nfs-ganesha/' /var/run/gluster/shared_storage/nfs-ganesha/ganesha.conf
    3. Run the following to clean up any existing cluster configuration: 
      /usr/libexec/ganesha/ganesha-ha.sh --cleanup /var/run/gluster/shared_storage/nfs-ganesha
    4. If you have upgraded to Red Hat Enterprise Linux 7.4 or later, set the following SELinux Boolean: 
      # setsebool -P ganesha_use_fusefs on
    5. Start the nfs-ganesha service and verify that all nodes are functional. 
      # gluster nfs-ganesha enable
    6. Enable NFS-Ganesha on all volumes. 
      # gluster volume set volname ganesha.enable on

5.1.2. Upgrading to Red Hat Gluster Storage 3.5 for Systems Subscribed to Red Hat Network Satellite Server

Procedure 5.3. Before you upgrade

  1. Back up the following configuration directory and files in a location that is not on the operating system partition.
    • /var/lib/glusterd
    • /etc/samba
    • /etc/ctdb
    • /etc/glusterfs
    • /var/lib/samba
    • /var/lib/ctdb
    • /var/run/gluster/shared_storage/nfs-ganesha

    Note

    With the release of 3.5 Batch Update 3, the mount point of shared storage is changed from /var/run/gluster/ to /run/gluster/ .
    If you use NFS-Ganesha, back up the following files from all nodes:
    • /run/gluster/shared_storage/nfs-ganesha/exports/export.*.conf
    • /etc/ganesha/ganesha.conf
    • /etc/ganesha/ganesha-ha.conf
    If upgrading from Red Hat Gluster Storage 3.3 to 3.4 or subsequent releases, back up all xattr by executing the following command individually on the brick root(s) for all nodes: 
    # find ./ -type d ! -path "./.*" ! -path "./" | xargs getfattr -d -m. -e hex > /var/log/glusterfs/xattr_dump_brick_name
  2. Unmount gluster volumes from all clients. On a client, use the following command to unmount a volume from a mount point. 
    # umount mount-point
  3. If you use NFS-Ganesha, run the following on a gluster server to disable the nfs-ganesha service: 
    # gluster nfs-ganesha disable
  4. On a gluster server, disable the shared volume. 
    # gluster volume set all cluster.enable-shared-storage disable
  5. Stop all volumes. 
    # for vol in `gluster volume list`; do gluster --mode=script volume stop $vol; sleep 2s; done
  6. Verify that all volumes are stopped. 
    # gluster volume info
  7. Stop the glusterd services on all servers using the following command: 
    # service glusterd stop
# pkill glusterfs
# pkill glusterfsd
  8. Stop the pcsd service. 
    # systemctl stop pcsd

Procedure 5.4. Upgrade using Satellite

  1. Create an Activation Key at the Red Hat Network Satellite Server, and associate it with the following channels. For more information, see Section 2.6, “Installing from Red Hat Satellite Server”
    • For Red Hat Enterprise Linux 6.7 or higher: 
      Base Channel: Red Hat Enterprise Linux Server (v.6 for 64-bit x86_64)

Child channels:
RHEL Server Scalable File System (v. 6 for x86_64)
Red Hat Gluster Storage Server 3 (RHEL 6 for x86_64)
      If you use Samba, add the following channel: 
      Red Hat Gluster 3 Samba (RHEL 6 for x86_64)
    • For Red Hat Enterprise Linux 7: 
      Base Channel: Red Hat Enterprise Linux Server (v.7 for 64-bit x86_64)

Child channels:
RHEL Server Scalable File System (v. 7 for x86_64)
Red Hat Gluster Storage Server 3 (RHEL 7 for x86_64)
      If you use Samba, add the following channel: 
      Red Hat Gluster 3 Samba (RHEL 7 for x86_64)
  2. Unregister your system from Red Hat Network Satellite by following these steps:
    1. Log in to the Red Hat Network Satellite server.
    2. Click on the Systems tab in the top navigation bar and then the name of the old or duplicated system in the System List.
    3. Click the delete system link in the top-right corner of the page.
    4. To confirm the system profile deletion by clicking the Delete System button.
  3. Run the following command on your Red Hat Gluster Storage server, using your credentials and the Activation Key you prepared earlier. This re-registers the system to the Red Hat Gluster Storage 3.5 channels on the Red Hat Network Satellite Server. 
    # rhnreg_ks --username username --password password --force --activationkey Activation Key ID
  4. Verify that the channel subscriptions have been updated.
    On Red Hat Enterprise Linux 6.7 and higher, look for the following channels, as well as the rh-gluster-3-samba-for-rhel-6-server-rpms channel if you use Samba. 
    # rhn-channel --list
rhel-6-server-rpms
rhel-scalefs-for-rhel-6-server-rpms
rhs-3-for-rhel-6-server-rpms
    On Red Hat Enterprise Linux 7, look for the following channels, as well as the rh-gluster-3-samba-for-rhel-7-server-rpms channel if you use Samba. 
    # rhn-channel --list
rhel-7-server-rpms
rh-gluster-3-for-rhel-7-server-rpms
  5. Upgrade to Red Hat Gluster Storage 3.5. 
    # yum update

    Important

    Run the following command to install nfs-ganesha-selinux on Red Hat Enterprise Linux 7: 
    # yum install nfs-ganesha-selinux
    Run the following command to install nfs-ganesha-selinux on Red Hat Enterprise Linux 8: 
     # dnf install glusterfs-ganesha
  6. Reboot the server and run volume and data integrity checks.
  7. When all nodes have been upgraded, run the following command to update the op-version of the cluster. This helps to prevent any compatibility issues within the cluster. 
    # gluster volume set all cluster.op-version 70200

    Note

    70200 is the cluster.op-version value for Red Hat Gluster Storage 3.5. Ater upgrading the cluster-op version, enable the granular-entry-heal for the volume via the given command: 
    gluster volume heal $VOLNAME granular-entry-heal enable
    The feature is now enabled by default post upgrade to Red Hat Gluster Storage 3.5, but this will come into affect only after bumping up the op-version. Refer to Section 1.5, “Red Hat Gluster Storage Software Components and Versions” for the correct cluster.op-version value for other versions.
  8. Start all volumes. 
    # for vol in `gluster volume list`; do gluster --mode=script volume start $vol; sleep 2s; done
  9. If you are using NFS-Ganesha:
    1. Copy the volume's export information from your backup copy of ganesha.conf to the new /etc/ganesha/ganesha.conf file.
      The export information in the backed up file is similar to the following: 
      %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.v1.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.v2.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.v3.conf"

      Note

      With the release of 3.5 Batch Update 3, the mount point of shared storage is changed from /var/run/gluster/ to /run/gluster/ .
    2. Copy the backup volume export files from the backup directory to /etc/ganesha/exports by running the following command from the backup directory: 
      # cp export.* /etc/ganesha/exports/
  10. Enable the shared volume. 
    # gluster volume set all cluster.enable-shared-storage enable
  11. Ensure that the shared storage volume is mounted on the server. If the volume is not mounted, run the following command: 
    # mount -t glusterfs hostname:gluster_shared_storage /var/run/gluster/shared_storage
  12. Ensure that the /var/run/gluster/shared_storage/nfs-ganesha directory is created. 
    # cd /var/run/gluster/shared_storage/
# mkdir nfs-ganesha
  13. Enable firewall settings for new services and ports. See Getting Started in the Red Hat Gluster Storage 3.5 Administration Guide.
  14. If you use Samba/CTDB:
    1. Mount /gluster/lock before starting CTDB by executing the following commands: 
      # mount <ctdb_volume_name>
# mount -t glusterfs server:/ctdb_volume_name /gluster/lock/
    2. Verify that the lock volume mounted correctly by checking for lock in the output of the mount command on any Samba server. 
      # mount | grep 'lock'
...
<hostname>:/<ctdb_volume_name>.tcp on /gluster/lock type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
    3. If all servers that host volumes accessed via SMB have been updated, then start the CTDB and Samba services by executing the following commands.
      On Red Hat Enterprise Linux 7 and Red Hat Enterprise Linux 8: 
      # systemctl start ctdb
      On Red Hat Enterprise Linux 6: 
      # service ctdb start
    4. To verify that the CTDB and SMB services have started, execute the following command: 
      ps axf | grep -E '(ctdb|smb|winbind|nmb)[d]'
  15. If you use NFS-Ganesha:
    1. Copy the ganesha.conf and ganesha-ha.conf files, and the /etc/ganesha/exports directory to the /var/run/gluster/shared_storage/nfs-ganesha directory. 
      # cd /etc/ganesha/
# cp ganesha.conf  ganesha-ha.conf /var/run/gluster/shared_storage/nfs-ganesha/
# cp -r exports/ /var/run/gluster/shared_storage/nfs-ganesha/
    2. Update the path of any export entries in the ganesha.conf file. 
      # sed -i 's/\/etc\/ganesha/\/var\/run\/gluster\/shared_storage\/nfs-ganesha/' /var/run/gluster/shared_storage/nfs-ganesha/ganesha.conf
    3. Run the following to clean up any existing cluster configuration: 
      /usr/libexec/ganesha/ganesha-ha.sh --cleanup /var/run/gluster/shared_storage/nfs-ganesha
    4. If you have upgraded to Red Hat Enterprise Linux 7.4 or later, set the following SELinux Boolean: 
      # setsebool -P ganesha_use_fusefs on
  16. Start the ctdb service (and nfs-ganesha service, if used) and verify that all nodes are functional. 
    # systemctl start ctdb
# gluster nfs-ganesha enable
  17. If this deployment uses NFS-Ganesha, enable NFS-Ganesha on all volumes. 
    # gluster volume set volname ganesha.enable on

5.1.3. Special consideration for Offline Software Upgrade

5.1.3.1. Migrate CTDB configuration files

If you are upgrading CTDB to 4.9.x and beyond from an older version of CTDB, perform the following steps post the upgrade:
  1. Make a temporary directory to migrate configuration files. 
    # mkdir /tmp/ctdb-migration
  2. Run the CTDB configuration migration script. 
    # ./usr/share/doc/ctdb-4.9.x/examples/config_migrate.sh -o /tmp/ctdb-migration /etc/sysconfig/ctdb
    The script assumes that the CTDB configuration directory is /etc/ctdb. If this is not correct for your setup, specify an alternative configuration directory with the -d option, for example: 
    # ./usr/share/doc/ctdb-4.9.x/examples/config_migrate.sh -o /tmp/ctdb-migration /etc/sysconfig/ctdb -d ctdb-config-dir
  3. Verify that the /tmp/ctdb-migration directory now contains the following files:
    • commands.sh
    • ctdb.conf
    • script.options
    • ctdb.tunables (if additional changes are required)
    • ctdb.sysconfig (if additional changes are required)
    • README.warn (if additional changes are required)
  4. Back up the current configuration files. 
    # mv /etc/ctdb/ctdb.conf /etc/ctdb/ctdb.conf.default
  5. Install the new configuration files. 
    # mv /tmp/ctdb-migration/ctdb.conf /etc/ctdb/ctdb.conf
# mv script.options /etc/ctdb/
  6. Make the commands.sh file executable, and run it. 
      # chmod +x /tmp/ctdb-migration/commands.sh
# ./tmp/ctdb-migration/commands.sh
  7. If /tmp/ctdb-migration/ctdb.tunables exists, move it to the /etc/ctdb directory. 
    # cp /tmp/ctdb-migration/ctdb.tunables /etc/ctdb
  8. If /tmp/ctdb-migration/ctdb.sysconfig exists, back up the old /etc/sysconfig/ctdb file and replace it with /tmp/ctdb-migration/ctdb.sysconfig. 
    # mv /etc/sysconfig/ctdb /etc/sysconfig/ctdb.old
# mv /tmp/ctdb-migration/ctdb.sysconfig /etc/sysconfig/ctdb
    Otherwise, back up the old /etc/sysconfig/ctdb file and replace it with /etc/sysconfig/ctdb.rpmnew. 
    # mv /etc/sysconfig/ctdb /etc/sysconfig/ctdb.old
# mv /etc/sysconfig/ctdb.rpmnew /etc/sysconfig/ctdb

5.2. In-Service Software Upgrade from Red Hat Gluster Storage 3.4 to Red Hat Gluster Storage 3.5

Important

Upgrade all Red Hat Gluster Storage servers before updating clients.
In-service software upgrade refers to the ability to progressively update a Red Hat Gluster Storage Server cluster with a new version of the software without taking the volumes hosted on the cluster offline. In most cases normal I/O operations on the volume continue even when the cluster is being updated.
I/O that uses CTDB may pause for the duration of an upgrade or update. This affects clients using Gluster NFS or Samba.

Note

After performing inservice upgrade of NFS-Ganesha, the new configuration file is saved as "ganesha.conf.rpmnew" in /etc/ganesha folder. The old configuration file is not overwritten during the inservice upgrade process. However, post upgradation, you have to manually copy any new configuration changes from "ganesha.conf.rpmnew" to the existing ganesha.conf file in /etc/ganesha folder.

5.2.1. Pre-upgrade Tasks

Ensure you perform the following steps based on the set-up before proceeding with the in-service software upgrade process.

5.2.1.1. Upgrade Requirements for Red Hat Gluster Storage 3.5

The upgrade requirements to upgrade to Red Hat Gluster Storage 3.5 from the preceding update are as follows:
  • In-service software upgrade is supported for pure and distributed versions of arbiter, erasure-coded (disperse) and three way replicated volume types. It is not supported for a pure distributed volume.
  • If you want to use snapshots for your existing environment, each brick must be an independent thin provisioned logical volume (LV). If you do not plan to use snapshots, thickly provisioned volumes remain supported.
  • A Logical Volume that contains a brick must not be used for any other purpose.
  • Linear LVM and thin LV are supported with Red Hat Gluster Storage 3.4 and later. For more information, see https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/logical_volume_manager_administration/index#LVM_components
  • When server-side quorum is enabled, ensure that bringing one node down does not violate server-side quorum. Add dummy peers to ensure the server-side quorum is not violated until the completion of rolling upgrade using the following command: 
    # gluster peer probe dummynode

    Note

    If you have a geo-replication session, then to add a node follow the steps mentioned in the sectionStarting Geo-replication for a New Brick or New Node in the Red Hat Gluster Storage Administration Guide.
    For example, when the server-side quorum percentage is set to the default value (>50%), for a plain replicate volume with two nodes and one brick on each machine, a dummy node that does not contain any bricks must be added to the trusted storage pool to provide high availability of the volume using the command mentioned above.
    In a three node cluster, if the server-side quorum percentage is set to 77%, bringing down one node would violate the server-side quorum. In this scenario, you have to add two dummy nodes to meet server-side quorum.
  • Stop any geo-replication sessions running between the master and slave. 
    # gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL stop
  • Ensure that there are no pending self-heals before proceeding with in-service software upgrade using the following command: 
    # gluster volume heal volname info
  • Ensure the Red Hat Gluster Storage server is registered to the required channels.
    On Red Hat Enterprise Linux 6: 
    rhel-6-server-rpms
rhel-scalefs-for-rhel-6-server-rpms
rhs-3-for-rhel-6-server-rpms
    On Red Hat Enterprise Linux 7: 
    rhel-7-server-rpms
rh-gluster-3-for-rhel-7-server-rpms
    On Red Hat Enterprise Linux 8: 
    rhel-8-for-x86_64-baseos-rpms
rhel-8-for-x86_64-appstream-rpms
rh-gluster-3-for-rhel-8-x86_64-rpms
    To subscribe to the channels, run the following command: 
    # subscription-manager repos --enable=repo-name

5.2.1.2. Restrictions for In-Service Software Upgrade

The following lists some of the restrictions for in-service software upgrade:
  • In-service upgrade for NFS-Ganesha clusters is supported only from Red Hat Gluster Storage 3.4 and beyond. If you are upgrading from Red Hat Gluster Storage 3.1 and you use NFS-Ganesha, offline upgrade to Red Hat Gluster Storage 3.4. Then use in-service upgrade method to upgrade to Red Hat Gluster Storage 3.5.
  • Erasure coded (dispersed) volumes can be upgraded while in-service only if the disperse.optimistic-change-log ,disperse.eager-lock and disperse.other-eager-lock options are set to off. Wait for at least two minutes after disabling these options before attempting to upgrade to ensure that these configuration changes take effect for I/O operations.
  • Post upgrade the values can be changed to resemble the pre-upgrade values on erasure coded volumes, but ensure that disperse.optimistic-change-log and disperse.other-eager-lock options are set to on.
  • Ensure that the system workload is low before performing the in-service software upgrade, so that the self-heal process does not have to heal too many entries during the upgrade. Also, with high system workload healing is time-consuming.
  • Do not perform any volume operations on the Red Hat Gluster Storage server.
  • Do not change hardware configurations.
  • Do not run mixed versions of Red Hat Gluster Storage for an extended period of time. For example, do not have a mixed environment of Red Hat Gluster Storage 3.3, Red Hat Gluster Storage 3.4, and Red Hat Gluster Storage 3.5 for a prolonged time.
  • Do not combine different upgrade methods.
  • It is not recommended to use in-service software upgrade for migrating to thin provisioned volumes, but to use offline upgrade scenario instead. For more information see, Section 5.1, “Offline Upgrade to Red Hat Gluster Storage 3.5”

5.2.1.3. Configuring repo for Upgrading using ISO

To configure the repo to upgrade using ISO, execute the following steps:

Note

Upgrading Red Hat Gluster Storage using ISO can be performed only from the immediately preceding release. This means that upgrading to Red Hat Gluster Storage 3.5 using ISO can only be done from Red Hat Gluster Storage 3.4. For a complete list of supported Red Hat Gluster Storage releases, see Section 1.5, “Red Hat Gluster Storage Software Components and Versions”.
  1. Mount the ISO image file under any directory using the following command: 
    # mount -o loop <ISO image file> <mount-point>
    For example: 
    # mount -o loop rhgs-3.5-rhel-7-x86_64-dvd-1.iso /mnt
  2. Set the repo options in a file in the following location: 
     /etc/yum.repos.d/<file_name.repo>
  3. Add the following information to the repo file: 
    [local]
name=local
baseurl=file:///mnt
enabled=1
gpgcheck=0

5.2.1.4. Preparing and Monitoring the Upgrade Activity

Before proceeding with the in-service software upgrade, prepare and monitor the following processes:
  • Check the peer and volume status to ensure that all peers are connected and there are no active volume tasks. 
    # gluster peer status
    # gluster volume status
  • Check the rebalance status using the following command: 
    # gluster volume rebalance r2 status
Node   Rebalanced-files  size      scanned   failures    skipped   status   run time in secs
---------   -----------    ---------   --------   ---------  ------  --------  --------------
10.70.43.198         0       0Bytes       99       0           0    completed     1.00
10.70.43.148         49      196Bytes    100       0           0    completed     3.00
  • If you need to upgrade an erasure coded (dispersed) volume, set the disperse.optimistic-change-log, disperse.eager-lock and disperse.other-eager-lock options to off. Wait for at least two minutes after disabling these options before attempting to upgrade to ensure that these configuration changes take effect for I/O operations. 
    # gluster volume set volname disperse.optimistic-change-log off
# gluster volume set volname disperse.eager-lock off
# gluster volume set volname disperse.other-eager-lock off
  • Ensure that there are no pending self-heals by using the following command: 
    # gluster volume heal volname info
    The following example shows no pending self-heals. 
    # gluster volume heal drvol info
Gathering list of entries to be healed on volume drvol has been successful

Brick 10.70.37.51:/rhs/brick1/dir1
Number of entries: 0

Brick 10.70.37.78:/rhs/brick1/dir1
Number of entries: 0

Brick 10.70.37.51:/rhs/brick2/dir2
Number of entries: 0

Brick 10.70.37.78:/rhs/brick2/dir2
Number of entries: 0

5.2.2. Service Impact of In-Service Upgrade

In-service software upgrade impacts the following services. Ensure you take the required precautionary measures.
Gluster NFS (Deprecated)

Warning

Gluster-NFS is considered deprecated as of Red Hat Gluster Storage 3.5. Red Hat no longer recommends the use of Gluster-NFS, and does not support its use in new deployments and existing deployments that upgrade to Red Hat Gluster Storage 3.5.3.
When you use Gluster NFS (Deprecated) to mount a volume, any new or outstanding file operations on that file system will hang uninterruptedly during in-service software upgrade until the server is upgraded.

Samba / CTDB

Ongoing I/O on Samba shares will fail as the Samba shares will be temporarily unavailable during the in-service software upgrade, hence it is recommended to stop the Samba service using the following command: 

# service ctdb stop   ;Stopping CTDB will also stop the SMB service.

Distribute Volume

In-service software upgrade is not supported for distributed volume. If you have a distributed volume in the cluster, stop that volume for the duration of the upgrade. 

# gluster volume stop <VOLNAME>
Virtual Machine Store

The virtual machine images are likely to be modified constantly. The virtual machine listed in the output of the volume heal command does not imply that the self-heal of the virtual machine is incomplete. It could mean that the modifications on the virtual machine are happening constantly.

Hence, if you are using a gluster volume for storing virtual machine images, then it is recommended to power-off all virtual machine instances before in-service software upgrade.

Important

It is recommended to turn off all virtual machine instances to minimize the changes that occur during an In-Service Upgrade. If you choose to keep the VM instances up and running , the time taken to heal will increase and at times may cause a situation where the pending heal remains a constant number and takes a long time to close to 0.

5.2.3. In-Service Software Upgrade

Note

Only offline mode is supported for samba.
The following steps have to be performed on each node of the replica pair:
  1. Back up the following configuration directory and files in a location that is not on the operating system partition.
    • /var/lib/glusterd
    • /etc/samba
    • /etc/ctdb
    • /etc/glusterfs
    • /var/lib/samba
    • /var/lib/ctdb
    • /var/run/gluster/shared_storage/nfs-ganesha

    Note

    With the release of 3.5 Batch Update 3, the mount point of shared storage is changed from /var/run/gluster/ to /run/gluster/ .
    If you use NFS-Ganesha, back up the following files from all nodes:
    • /run/gluster/shared_storage/nfs-ganesha/exports/export.*.conf
    • /etc/ganesha/ganesha.conf
    • /etc/ganesha/ganesha-ha.conf

  2. Important

    If you are updating from RHEL 8.3 to RHEL 8.4, then follow these additional steps.
    1. On one node in the cluster, edit /etc/corosync/corosync.conf. Add a line "token: 3000" to the totem stanza, for example: 
      totem {
            version: 2
            secauth: off
            cluster_name: rhel8-cluster
            transport: knet
            token: 3000
          }
    2. Run `pcs cluster sync`. It is optional to verify /etc/corosync/corosync.conf on all nodes has the new token: 3000 line.
    3. Run `pcs cluster reload corosync`
    4. Run `corosync-cmapctl | grep totem.token, confirm that "runtime.config.totem.token (u32) = 3000" as output.
  3. If the node is part of an NFS-Ganesha cluster, place the node in standby mode. 
    # pcs node standby
  4. Ensure that there are no pending self-heal operations. 
    # gluster volume heal volname info
  5. If this node is part of an NFS-Ganesha cluster:
    1. Disable the PCS cluster and verify that it has stopped. 
      # pcs cluster disable
# pcs status
    2. Stop the nfs-ganesha service. 
      # systemctl stop nfs-ganesha
  6. Stop all gluster services on the node and verify that they have stopped. 
    # systemctl stop glusterd
# pkill glusterfs
# pkill glusterfsd
# pgrep gluster
  7. Verify that your system is not using the legacy Red Hat Classic update software. 
    # migrate-rhs-classic-to-rhsm --status
    If your system uses this legacy software, migrate to Red Hat Subscription Manager and verify that your status has changed when migration is complete. 
    # migrate-rhs-classic-to-rhsm --rhn-to-rhsm
    # migrate-rhs-classic-to-rhsm --status
  8. Update the server using the following command: 
    # yum update

    Important

    Run the following command to install nfs-ganesha-selinux on Red Hat Enterprise Linux 7: 
    # yum install nfs-ganesha-selinux
    Run the following command to install nfs-ganesha-selinux on Red Hat Enterprise Linux 8: 
       # dnf install glusterfs-ganesha
  9. If the volumes are thick provisioned, and you plan to use snapshots, perform the following steps to migrate to thin provisioned volumes:

    Note

    Migrating from thick provisioned volume to thin provisioned volume during in-service software upgrade takes a significant amount of time based on the data you have in the bricks. If you do not plan to use snapshots, you can skip this step. However, if you plan to use snapshots on your existing environment, the offline method to upgrade is recommended. For more information regarding offline upgrade, see Section 5.1, “Offline Upgrade to Red Hat Gluster Storage 3.5”
    Contact a Red Hat Support representative before migrating from thick provisioned volumes to thin provisioned volumes using in-service software upgrade.
    1. Unmount all the bricks associated with the volume by executing the following command: 
      # umount mount_point
    2. Remove the LVM associated with the brick by executing the following command: 
      # lvremove logical_volume_name
      For example: 
      # lvremove /dev/RHS_vg/brick1
    3. Remove the volume group by executing the following command: 
      # vgremove -ff volume_group_name
      For example: 
      # vgremove -ff RHS_vg
    4. Remove the physical volume by executing the following command: 
      # pvremove -ff physical_volume
    5. If the physical volume (PV) is not created, then create the PV for a RAID 6 volume by executing the following command, else proceed with the next step: 
      # pvcreate --dataalignment 2560K /dev/vdb
      For more information, see Formatting and Mounting Bricks in the Red Hat Gluster Storage 3.5 Administration Guide.
    6. Create a single volume group from the PV by executing the following command: 
      # vgcreate volume_group_name disk
      For example: 
      # vgcreate RHS_vg /dev/vdb
    7. Create a thinpool using the following command: 
      # lvcreate -L size --poolmetadatasize md_size --chunksize chunk_size -T pool_device
      For example: 
      # lvcreate -L 2T --poolmetadatasize 16G --chunksize 256  -T /dev/RHS_vg/thin_pool
    8. Create a thin volume from the pool by executing the following command: 
      # lvcreate -V size -T pool_device -n thinvol_name
      For example: 
      # lvcreate -V 1.5T -T /dev/RHS_vg/thin_pool -n thin_vol
    9. Create filesystem in the new volume by executing the following command: 
      # mkfs.xfs -i size=512 thin_vol
      For example: 
      # mkfs.xfs -i size=512 /dev/RHS_vg/thin_vol
      The back-end is now converted to a thin provisioned volume.
    10. Mount the thin provisioned volume to the brick directory and setup the extended attributes on the bricks. For example: 
      # setfattr -n trusted.glusterfs.volume-id \ -v 0x$(grep volume-id /var/lib/glusterd/vols/volname/info \ | cut -d= -f2 | sed 's/-//g') $brick
  10. Disable glusterd. 
    # systemctl disable glusterd
    This prevents it starting during boot time, so that you can ensure the node is healthy before it rejoins the cluster.
  11. Reboot the server. 
    # shutdown -r now "Shutting down for upgrade to Red Hat Gluster Storage 3.5"

  12. Important

    Perform this step only for each thick provisioned volume that has been migrated to thin provisioned volume in the previous step.
    Change the Automatic File Replication extended attributes from another node, so that the heal process is executed from a brick in the replica subvolume to the thin provisioned brick.
    1. Create a FUSE mount point to edit the extended attributes. 
      # mount -t glusterfs HOSTNAME_or_IPADDRESS:/VOLNAME /MOUNTDIR
    2. Create a new directory on the mount point, and ensure that a directory with such a name is not already present. 
      # mkdir /MOUNTDIR/name-of-nonexistent-dir
    3. Delete the directory and set the extended attributes. 
      # rmdir /MOUNTDIR/name-of-nonexistent-dir
      # setfattr -n trusted.non-existent-key -v abc /MOUNTDIR
# setfattr -x trusted.non-existent-key /MOUNTDIR
    4. Ensure that the extended attributes of the brick in the replica subvolume is not set to zero. 
      # getfattr -d -m. -e hex brick_path
      In the following example, the extended attribute trusted.afr.repl3-client-1 for /dev/RHS_vg/brick2 is not set to zero: 
      # getfattr -d -m. -e hex /dev/RHS_vg/brick2
getfattr: Removing leading '/' from absolute path names
# file: /dev/RHS_vg/brick2
trusted.afr.dirty=0x000000000000000000000000
trusted.afr.repl3-client-1=0x000000000000000400000002
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x000000010000000000000000ffffffff
trusted.glusterfs.volume-id=0x924c2e2640d044a687e2c370d58abec9
  13. Start the glusterd service. 
    # systemctl start glusterd
  14. Verify that you have upgraded to the latest version of Red Hat Gluster Storage. 
    # gluster --version
  15. Ensure that all bricks are online. 
    # gluster volume status
    For example: 
    # gluster volume status
Status of volume: r2

Gluster process                                         Port    Online  Pid
------------------------------------------------------------------------------
Brick 10.70.43.198:/brick/r2_0                          49152   Y       32259
Brick 10.70.42.237:/brick/r2_1                          49152   Y       25266
Brick 10.70.43.148:/brick/r2_2                          49154   Y       2857
Brick 10.70.43.198:/brick/r2_3                          49153   Y       32270
NFS Server on localhost                                 2049    Y       25280
Self-heal Daemon on localhost                           N/A     Y       25284
NFS Server on 10.70.43.148                              2049    Y       2871
Self-heal Daemon on 10.70.43.148                        N/A     Y       2875
NFS Server on 10.70.43.198                              2049    Y       32284
Self-heal Daemon on 10.70.43.198                        N/A     Y       32288

Task Status of Volume r2
------------------------------------------------------------------------------
There are no active volume tasks
  16. Start self-heal on the volume. 
    # gluster volume heal volname
  17. Ensure that self-heal on the volume is complete. 
    # gluster volume heal volname info
    The following example shows a completed self heal operation. 
     # gluster volume heal drvol info
Gathering list of entries to be healed on volume drvol has been successful

Brick 10.70.37.51:/rhs/brick1/dir1
Number of entries: 0

Brick 10.70.37.78:/rhs/brick1/dir1
Number of entries: 0

Brick 10.70.37.51:/rhs/brick2/dir2
Number of entries: 0

Brick 10.70.37.78:/rhs/brick2/dir2
Number of entries: 0
  18. Verify that shared storage is mounted. 
    # mount | grep /run/gluster/shared_storage
  19. If this node is part of an NFS-Ganesha cluster:
    1. If the system is managed by SELinux, set the ganesha_use_fusefs Boolean to on. 
      # setsebool -P ganesha_use_fusefs on
    2. Start the NFS-Ganesha service. 
      # systemctl start nfs-ganesha
    3. Enable and start the cluster. 
      # pcs cluster enable
# pcs cluster start
    4. Release the node from standby mode. 
      # pcs node unstandby
    5. Verify that the pcs cluster is running, and that the volume is being exported correctly after upgrade. 
      # pcs status
# showmount -e
      NFS-ganesha enters a short grace period after performing these steps. I/O operations halt during this grace period. Wait until you see NFS Server Now NOT IN GRACE in the ganesha.log file before continuing.
  20. Optionally, enable the glusterd service to start at boot time. 
    # systemctl enable glusterd
  21. Repeat the above steps on the other node of the replica pair. In the case of a distributed-replicate setup, repeat the above steps on all the replica pairs.

  22. Important

    If you are updating from RHEL 8.3 to RHEL 8.4, then follow these additional steps.
    1. Restore the totem-token timeout to the original value, for example, by deleting the token: 3000 line from /etc/corosync/corosync.conf
    2. Run `pcs cluster sync`. It is optional to verify /etc/corosync/corosync.conf on all nodes has the new token: 3000 line.
    3. Run `pcs cluster reload corosync`.
    4. Run `corosync-cmapctl | grep totem.token to verify the changes.
  23. When all nodes have been upgraded, run the following command to update the op-version of the cluster. This helps to prevent any compatibility issues within the cluster. 
    # gluster volume set all cluster.op-version 70200

    Note

    70200 is the cluster.op-version value for Red Hat Gluster Storage 3.5. Ater upgrading the cluster-op version, enable the granular-entry-heal for the volume via the given command: 
    gluster volume heal $VOLNAME granular-entry-heal enable
    The feature is now enabled by default post upgrade to Red Hat Gluster Storage 3.5, but this will come into affect only after bumping up the op-version. Refer to Section 1.5, “Red Hat Gluster Storage Software Components and Versions” for the correct cluster.op-version value for other versions.

    Note

    If you want to enable snapshots, see Managing Snapshots in the Red Hat Gluster Storage 3.5 Administration Guide.

    Important

    Remount the Red Hat Gluster Storage volume on the client side, in the case of updating the Red Hat Gluster Storage nodes enabled with network encryption using TLS/SSL. For more information, refer to Configuring Network Encryption in Red Hat Gluster Storage
  24. If the client-side quorum was disabled before upgrade, then upgrade it by executing the following command: 
    # gluster volume set volname cluster.quorum-type auto
  25. If a dummy node was created earlier, then detach it by executing the following command: 
    # gluster peer detach <dummy_node name>
  26. If the geo-replication session between master and slave was disabled before upgrade, then configure the meta volume and restart the session:
    1. Run the following command: 
      # gluster volume set all cluster.enable-shared-storage enable
    2. If you use a non-root user to perform geo-replication, run this command on the primary slave to set permissions for the non-root group. 
      # gluster-mountbroker setup <mount_root> <group>
    3. Run the following command: 
      # gluster volume geo-replication Volume1 example.com::slave-vol config use_meta_volume true
    4. Run the following command: 
      # gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL start
  27. If you disabled the disperse.optimistic-change-log, disperse.eager-lock and disperse.other-eager-lock options in order to upgrade an erasure-coded (dispersed) volume, re-enable these settings. 
    # gluster volume set volname disperse.optimistic-change-log on
# gluster volume set volname disperse.eager-lock on
# gluster volume set volname disperse.other-eager-lock on

5.2.4. Special Consideration for In-Service Software Upgrade

The following sections describe the in-service software upgrade steps for a CTDB setup.

Note

Only offline mode is supported.

5.2.4.1. Upgrading the Native Client

All clients must use the same version of glusterfs-fuse. Red Hat strongly recommends that you upgrade servers before upgrading clients. If you are upgrading from Red Hat Gluster Storage 3.1 Update 2 or earlier, you must upgrade servers and clients simultaneously. For more information regarding upgrading native client, refer the steps mentioned below.
Before updating the Native Client, subscribe the clients to the channels mentioned in Section 6.2.1, “Installing Native Client”.

Warning

If you want to access a volume being provided by a server using Red Hat Gluster Storage 3.1.3 or higher, your client must also be using Red Hat Gluster Storage 3.1.3 or higher. Accessing these volumes from earlier client versions can result in data becoming unavailable and problems with directory operations. This requirement exists because Red Hat Gluster Storage 3.1.3 changed how the Distributed Hash Table works in order to improve directory consistency and remove the effects.

  1. Unmount gluster volumes

    Unmount any gluster volumes prior to upgrading the native client. 
    # umount /mnt/glusterfs

  2. Upgrade the client

    Run the yum update command to upgrade the native client: 
    # yum update glusterfs glusterfs-fuse

  3. Remount gluster volumes

    Remount the volumes as discussed in Section 6.2.3, “Mounting Red Hat Gluster Storage Volumes”.

5.3. Minor version updates of RHGS 3.5.z

For RHGS 3.5.z minor version updates on RHEL 7 and RHEL 8, follow the procedure described in Section 5.2, “In-Service Software Upgrade from Red Hat Gluster Storage 3.4 to Red Hat Gluster Storage 3.5”

Note

In the case of kernel updates make sure to reboot the host after software update is completed.

Chapter 6. Upgrading Red Hat Gluster Storage to Red Hat Enterprise Linux 7

Important

RHEL 8 is supported only for new installations of Red Hat Gluster Storage 3.5.2. Upgrades to RHEL 8 based Red Hat Gluster Storage 3.5.2 are not supported. Red Hat Gluster Storage is not supported on Red Hat Enterprise Linux 6 (RHEL 6) from 3.5 Batch Update 1 onwards, refer to Table 1.3, “Version Details”
Prerequisites to upgrade from Red Hat Enterprise Linux 6 based Red Hat Gluster Storage to Red Hat Enterprise Linux 7 based Red Hat Gluster Storage are as follow:
  • RHEL6 channel subscriptions are required.
  • Upgrade servers before upgrading clients.
Upgrading from Red Hat Enterprise Linux 6 to Red Hat Enterprise Linux 7 is an offline procedure that results in Gluster node downtime, because volumes are offline during the upgrade process.

6.1. Preparing System for Upgrade

Important

If the system is already registered through Red Hat Subscription Manager, skip the registration steps and start the backup of the Gluster configuration files.

  1. Migrate from Red Hat Network Classic to Red Hat Subscription Manager

    Verify that your system is not on the legacy Red Hat Network Classic update system: 
    # migrate-rhs-classic-to-rhsm --status
    If the system is on Red Hat Network Classic, migrate to Red Hat Subscription Manager using: Migrating from RHN to RHSM in Red Hat Enterprise Linux.

  2. Register the system with Red Hat Subscription Manager

    To register the system with Red Hat Network, execute the following command. Enter your Red Hat Network username and password that have the Red Hat Enterprise Linux entitlements: 
    # subscription-manager register --username=user_name --password=password

  3. Identify the available entitlement pools

    Find the entitlement pools containing the Red Hat Enterprise Linux 6 repositories: 
    # subscription-manager list --available

  4. Attach the entitlement pool to the system

    Use the pool identifier to attach the Red Hat Enterprise Linux 6 entitlements. 
    # subscription-manager attach --pool=pool_ID

  5. Enable the repositories

    Enable the Red Hat Enterprise Linux 6, scalable file system, and Red Hat Gluster Storage repositories: 
    # subscription-manager repos --enable=rhel-6-server-rpms --enable=rhel-scalefs-for-rhel-6-server-rpms --enable=rhs-3-for-rhel-6-server-rpms

  6. Back up the Gluster configuration files

    Note

    It is recommended to make a complete backup before you update your system. Refer to https://access.redhat.com/solutions/1484053 to know one possible approach.
    1. Ensure that the following configuration directories and files are backed up:
      • /var/lib/glusterd
      • /etc/glusterfs
    2. For systems with samba-ctdb enabled cluster, create a new directory to store the backup: 
      # mkdir backup_folder_name
# cd backup_folder_name
    3. Execute the following command to take a backup of samba-ctdb data: 
      for each in `ctdb getdbmap | grep PERSISTENT | cut -d" " -f2 | cut -d":" -f2`; do echo $each ; ctdb backup_folder_name $each ${each}.bak; done

  7. Stop all Gluster services, volumes, and processes

    1. Stop any geo-replication sessions: 
      # gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL stop
    2. Stop all volumes: 
      # for vol in `gluster volume list`; do gluster --mode=script volume stop $vol; sleep 2s; done
    3. Stop the Gluster processes: 
      # service glusterd stop
# pkill glusterfs
# pkill glusterfsd
    4. For systems with a samba-ctdb enabled cluster, stop the CTDB services: 
      # service ctdb stop

  8. Update the system

    Update the system to the latest minor version of Red Hat Enterprise Linux 6: 
    # yum update

  9. Reboot

    Reboot the system: 
    # reboot

  10. Verify the version number

    Check the current version number of the updated Red Hat Enterprise Linux 6 system: 
    # cat /etc/redhat-release
    Ensure the version number is 6.10.

6.2. Performing System Assessment

  1. Install the preupgrade-assistant

    The preupgrade-assistant is a tool that scans the existing Red Hat Enterprise Linux 6 system and checks whether it is ready to be upgraded.
    Subscribe to the required repositories and install the preupgrade-assistant tool using the following commands: 
    # subscription-manager repos --enable rhel-6-server-extras-rpms --enable rhel-6-server-optional-rpms
# yum install preupgrade-assistant preupgrade-assistant-el6toel7

  2. Run the preupgrade tool

    Launch the preupgrade tool: 
    # preupg -v

  3. Assess the results

    1. View the following result of the preupgrade tool in a browser: 
      /root/preupgrade/result.html
    2. Make a note of all the components that are marked as Failed and Needs Action.
    3. Share the findings with the Red Hat executive assisting with the upgrade process.

  4. Check the SELinux policy in Red Hat Enterprise Linux 7

    SELinux policies have changed between Red Hat Enterprise Linux 6 and Red Hat Enterprise Linux 7. For a working SELinux policy in Red Hat Enterprise Linux 7, execute the following command: 
    # semodule -r sandbox

  5. Uninstall Cluster and HA related packages

    Migration is not possible with Cluster and HA related packages installed on the system.
    Uninstall Cluster and HA related packages by executing the following command: 
    # yum remove modcluster ricci openais corosync

6.3. Upgrading from Red Hat Enterprise Linux 6.X to Red Hat Enterprise Linux 7.X

  1. Install migration tool

    # yum install redhat-upgrade-tool
# yum install yum-utils

  2. Disable all repositories

    Disable all the enabled repositories: 
    # yum-config-manager --disable \*

  3. Download the operating system as an ISO file

    Follow these steps to download the latest ISO file for one of the following operating systems.

    Red Hat Enterprise Linux 7 based Red Hat Gluster Storage 3.5

    1. Visit the Red Hat Customer Service Portal at https://access.redhat.com/login and enter your user name and password to log in.
    2. Click Downloads to go to the Software & Download Center.
    3. Click Red Hat Gluster Storage.
    4. Select 3.5 for RHEL 7 (latest) from the Version dropdown menu.

      Important

      If you are upgrading to Red Hat Gluster Storage 3.5, select version 3.4 for RHEL 7. Bug 1762637 means that the Preupgrade Assistant only allows upgrading to Red Hat Enterprise Linux 7.6 at this time.
    5. Click Download Now beside the Red Hat Gluster Storage Server 3.5 on RHEL 7 Installation DVD.

    Red Hat Enterprise Linux 7

    1. Visit the Red Hat Customer Service Portal at https://access.redhat.com/login and enter your user name and password to log in.
    2. Click Downloads to go to the Software & Download Center.
    3. Click Versions 7 and below, beside Red Hat Enterprise Linux.
    4. Select 7.x (latest) from the Version dropdown menu.

      Important

      If you are upgrading to Red Hat Gluster Storage 3.5, select version 7.6. Bug 1762637 means that the Preupgrade Assistant only allows upgrading to Red Hat Enterprise Linux 7.6 at this time.
    5. Click Download Now beside the Red Hat Enterprise Linux 7.x Binary DVD.

  4. Upgrade to Red Hat Enterprise Linux 7 using ISO

    Upgrade to Red Hat Enterprise Linux 7 using the Red Hat upgrade tool and reboot after the upgrade process is completed: 
    # redhat-upgrade-tool --iso ISO_filepath --cleanup-post
# reboot

    Important

    The upgrade process is time-consuming depending on your system's configuration and amount of data.

6.4. Upgrading to Red Hat Gluster Storage 3.5

  1. Disable all repositories

    # subscription-manager repos --disable=’*’

  2. Subscribe to the Red Hat Enterprise Linux 7 channel

    # subscription-manager repos --enable=rhel-7-server-rpms

  3. Check for stale Red Hat Enterprise Linux 6 packages

    Check for any stale Red Hat Enterprise Linux 6 packages post upgrade: 
    # rpm -qa | grep el6

    Important

    If the output lists packages of Red Hat Enterprise Linux 6 variant, contact Red Hat Support for further course of action on these packages.

  4. Update and reboot

    Update the Red Hat Enterprise Linux 7 packages and reboot. 
    # yum update
# reboot

  5. Verify the version number

    Ensure that the latest version of Red Hat Enterprise Linux 6 is shown when you view the `redhat-release` file: 
    # cat /etc/redhat-release

  6. Subscribe to the required channels

    1. Subscribe to the Gluster channel: 
      # subscription-manager repos --enable=rh-gluster-3-for-rhel-7-server-rpms
    2. If you use Samba, enable its repository. 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-7-server-rpms
    3. If you use NFS-Ganesha, enable its repository. 
      # subscription-manager repos --enable=rh-gluster-3-nfs-for-rhel-7-server-rpms --enable=rhel-ha-for-rhel-7-server-rpms
    4. If you use gdeploy, enable the Ansible repository: 
      # subscription-manager repos --enable=rhel-7-server-ansible-2-rpms

  7. Install and update Gluster

    1. If you used a Red Hat Enterprise Linux 7 ISO, install Red Hat Gluster Storage 3.5 using the following command: 
      # yum install redhat-storage-server
      This is already installed if you used a Red Hat Gluster Storage 3.5 ISO based on Red Hat Enterprise Linux 7.
    2. Update Red Hat Gluster Storage to the latest packages using the following command: 
      # yum update

  8. Verify the installation and update

    1. Check the current version number of the updated Red Hat Gluster Storage system: 
      # cat /etc/redhat-storage-release

      Important

      The version number should be 3.5.
    2. Ensure that no Red Hat Enterprise Linux 6 packages are present: 
      # rpm -qa | grep el6

      Important

      If the output lists packages of Red Hat Enterprise Linux 6 variant, contact Red Hat Support for further course of action on these packages.

  9. Install and configure Firewalld

    1. Install and start the firewall daemon using the following commands: 
      # yum install firewalld
# systemctl start firewalld
    2. Add the Gluster process to the firewall: 
      # firewall-cmd --zone=public --add-service=glusterfs --permanent
    3. Add the required services and ports to firewalld. For more information see Considerations for Red Hat Gluster Storage
    4. Reload the firewall using the following commands: 
      # firewall-cmd --reload

  10. Start the Gluster processes

    1. Start the glusterd process: 
      # systemctl start glusterd

  11. Update Gluster op-version

    Update the Gluster op-version to the required maximum version using the following commands: 
    # gluster volume get all cluster.max-op-version
# gluster volume set all cluster.op-version op_version

    Note

    70200 is the cluster.op-version value for Red Hat Gluster Storage 3.5. Ater upgrading the cluster-op version, enable the granular-entry-heal for the volume via the given command: 
    gluster volume heal $VOLNAME granular-entry-heal enable
    The feature is now enabled by default post upgrade to Red Hat Gluster Storage 3.5, but this will come into affect only after bumping up the op-version. Refer to Section 1.5, “Red Hat Gluster Storage Software Components and Versions” for the correct cluster.op-version value for other versions.

  12. Set up Samba and CTDB

    If the Gluster setup on Red Hat Enterprise Linux 6 had Samba and CTDB configured, you should have the following available on the updated Red Hat Enterprise Linux 7 system:
    • CTDB volume
    • /etc/ctdb/nodes file
    • /etc/ctdb/public_addresses file
    Perform the following steps to reconfigure Samba and CTDB:
    1. Configure the firewall for Samba: 
      # firewall-cmd --zone=public  --add-service=samba --permanent
# firewall-cmd --zone=public  --add-port=4379/tcp --permanent
    2. Subscribe to the Samba channel: 
      # subscription-manager repos --enable=rh-gluster-3-samba-for-rhel-7-server-rpms
    3. Update Samba to the latest packages: 
      # yum update
    4. Configure CTDB for Samba. For more information, see Configuring CTDB on Red Hat Gluster Storage Server in Setting up CTDB for Samba. You must skip creating the volume as the volumes present before the upgrade would be persistent after the upgrade.
    5. In the following files, replace all in the statement META="all" with the volume name: 
      /var/lib/glusterd/hooks/1/start/post/S29CTDBsetup.sh
/var/lib/glusterd/hooks/1/stop/pre/S29CTDB-teardown.sh
      For example, the volume name is ctdb_volname, the META="all" in the files should be changed to META="ctdb_volname".
    6. Restart the CTDB volume using the following commands: 
      # gluster volume stop volume_name
# gluster volume start volume_name
    7. Start the CTDB process: 
      # systemctl start ctdb
    8. Share the volume over Samba if required. See Sharing Volumes over SMB.

  13. Start the volumes and geo-replication

    1. Start the required volumes using the following command: 
      # gluster volume start volume_name
    2. Mount the meta-volume: 
      # mount /var/run/gluster/shared_storage/

      Note

      With the release of 3.5 Batch Update 3, the mount point of shared storage is changed from /var/run/gluster/ to /run/gluster/ .
      If this command does not work, review the content of the /etc/fstab file and ensure that the entry for the shared storage is configured correctly, and re-run the mount command. The line for the meta volume in the /etc/fstab file should look like the following: 
      hostname:/gluster_shared_storage   /var/run/gluster/shared_storage/   glusterfs   defaults   0 0
    3. Restore the geo-replication session: 
      # gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL start
      For more information on geo-replication, see Preparing to Deploy Geo-replication.

Chapter 7. Upgrading from Red Hat Gluster Storage 3.4 to Red Hat Gluster Storage 3.5 in a Red Hat Enterprise Virtualization-Red Hat Gluster Storage Environment

This section describes the upgrade methods for a Red Hat Gluster Storage and Red Hat Enterprise Virtualization integrated environment. You can upgrade Red Hat Gluster Storage 3.4 to Red Hat Gluster Storage 3.5 using an ISO or yum.

Warning

Before you upgrade, be aware of changed requirements that exist after Red Hat Gluster Storage 3.1.3. If you want to access a volume being provided by a Red Hat Gluster Storage 3.1.3 or higher server, your client must also be using Red Hat Gluster Storage 3.1.3 or higher. Accessing volumes from other client versions can result in data becoming unavailable and problems with directory operations. This requirement exists because Red Hat Gluster Storage 3.1.3 contained a number of changes that affect how the Distributed Hash Table works in order to improve directory consistency and remove the effects seen in BZ#1115367 and BZ#1118762.

Important

RHEL 8 is supported only for new installations of Red Hat Gluster Storage 3.5.2. Upgrades to RHEL 8 based Red Hat Gluster Storage 3.5.2 are not supported .

Important

In Red Hat Enterprise Linux 7 based Red Hat Gluster Storage, updating to 3.1 or higher reloads firewall rules. All runtime-only changes made before the reload are lost.

7.1. Prerequisites

  • Verify that no self-heal operations are in progress. 
    # gluster volume heal volname info
  • Ensure that the gluster volume corresponding to Glusterfs Storage Domain does not have any pending self heal by executing the following command: 
    # gluster volume heal volname info summary

7.2. Upgrading an in-service integrated environment

Follow the instructions below to upgrade an integrated Red Hat Virtualization and Red Hat Gluster Storage environment that is online and in-service.

  1. Upgrade the Red Hat Gluster Storage nodes

    Perform the following steps for each Red Hat Gluster Storage node, one node at a time. If you have multiple replica sets, upgrade each node in a replica set before moving on to another replica set.
    1. Stop all gluster services on the storage node that you want to upgrade by running the following commands as the root user on that node. 
      # systemctl stop glusterd
# pkill glusterfs
# pkill glusterfsd
# pgrep gluster
    2. In the Administration Portal, click Storage → Hosts and select the storage node to upgrade.
    3. Click Management → Maintenance and click OK.
    4. Upgrade the storage node using the procedure at Section 5.2, “In-Service Software Upgrade from Red Hat Gluster Storage 3.4 to Red Hat Gluster Storage 3.5”.
    5. Reboot the storage node.
    6. In the Administration Portal, click Storage → Hosts and select the storage node.
    7. Click Management → Activate.
    8. Click the name of the storage node → Bricks, and verify that the Self-Heal Info column of all bricks is listed as OK before upgrading the next storage node.

  2. Upgrade Gluster client software on virtualization hosts

    After upgrading all storage nodes, update the client software on all virtualization hosts.

    1. For Red Hat Virtualization hosts

      1. In the Administration Portal, click Compute → Hosts and select the node to update.
      2. On Red Hat Virtualization versions earlier than 4.2, click Management → Maintenance and click OK.
      3. Click Installation → Upgrade and click OK.
        The node is automatically updated with the latest packages and reactivated when the update is complete.

    2. For Red Hat Enterprise Linux hosts

      1. In the Administration Portal, click Compute → Hosts and select the node to update.
      2. Click Management → Maintenance and click OK.
      3. Subscribe the host to the correct repositories to receive updates.
      4. Update the host. 
        # yum update
      5. In the Administration Portal, click Management → Activate.

7.3. Upgrading an offline integrated environment

Follow the instructions below to upgrade an integrated Red Hat Virtualization and Red Hat Gluster Storage environment that is offline and out-of-service.

7.3.1. Upgrading using an ISO

  1. Using Red Hat Enterprise Virtualization Manager, stop all the virtual machine instances.
    The Red Hat Gluster Storage volume on the instances will be stopped during the upgrade.
  2. Using Red Hat Enterprise Virtualization Manager, move the data domain of the data center to Maintenance mode.
  3. Using Red Hat Enterprise Virtualization Manager, stop the volume (the volume used for data domain) containing Red Hat Gluster Storage nodes in the data center.
  4. Using Red Hat Enterprise Virtualization Manager, move all Red Hat Gluster Storage nodes to Maintenance mode.
  5. Perform the ISO Upgrade as mentioned in section Section 5.2.1.3, “Configuring repo for Upgrading using ISO”.
  6. Re-install the Red Hat Gluster Storage nodes from Red Hat Enterprise Virtualization Manager.

    Note

    • Re-installation for the Red Hat Gluster Storage nodes should be done from Red Hat Enterprise Virtualization Manager. The newly upgraded Red Hat Gluster Storage 3.5 nodes lose their network configuration and other configuration details, such as firewall configuration, which was done while adding the nodes to Red Hat Enterprise Virtualization Manager. Re-install the Red Hat Gluster Storage nodes to have the bootstrapping done.
    • You can re-configure the Red Hat Gluster Storage nodes using the option provided under Action Items, as shown in Figure 7.1, “Red Hat Gluster Storage Node before Upgrade ”, and perform bootstrapping.
    Red Hat Gluster Storage Node before Upgrade

    Figure 7.1. Red Hat Gluster Storage Node before Upgrade

  7. Perform the steps above in all Red Hat Gluster Storage nodes.
  8. Start the volume once all the nodes are shown in Up status in Red Hat Enterprise Virtualization Manager.
  9. Upgrade the native client bits for Red Hat Enterprise Linux 6 or Red Hat Enterprise Linux 7, depending on the hypervisor you are using.

    Note

    If Red Hat Enterprise Virtualization Hypervisor is used as hypervisor, then install a suitable build of Red Hat Enterprise Virtualization Hypervisor containing the latest native client.
    Red Hat Gluster Storage Node after Upgrade

    Figure 7.2. Red Hat Gluster Storage Node after Upgrade

  10. Using Red Hat Enterprise Virtualization Manager, activate the data domain and start all the virtual machine instances in the data center.

7.3.2. Upgrading using yum

  1. Using Red Hat Enterprise Virtualization Manager, stop all virtual machine instances in the data center.
  2. Using Red Hat Enterprise Virtualization Manager, move the data domain backed by gluster volume to Maintenance mode.
  3. Using Red Hat Enterprise Virtualization Manager, move all Red Hat Gluster Storage nodes to Maintenance mode.
  4. Perform yum update as mentioned in Section 5.1.1, “Upgrading to Red Hat Gluster Storage 3.5 for Systems Subscribed to Red Hat Subscription Manager”.
  5. Once the Red Hat Gluster Storage nodes are rebooted and up, Activate them using Red Hat Enterprise Virtualization Manager.

    Note

    Re-installation of Red Hat Gluster Storage nodes is required, as the network configurations and bootstrapping configurations done prior to upgrade are preserved, unlike ISO upgrade.
  6. Using Red Hat Enterprise Virtualization Manager, start the volume.
  7. Upgrade the native client bits for Red Hat Enterprise Linux 6 or Red Hat Enterprise Linux 7, based on the Red Hat Enterprise Linux server hypervisor used.

    Note

    If Red Hat Enterprise Virtualization Hypervisor is used as hypervisor, reinstall Red Hat Enterprise Virtualization Hypervisor containing the latest version of Red Hat Gluster Storage native client.
  8. Activate the data domain and start all the virtual machine instances.

Chapter 8. Enabling SELinux

After upgrading to Red Hat Gluster Storage 3.x, follow the steps mentioned here for SELinux support. These steps are not mandatory and are required only if SELinux support is required.
This procedure assumes that the selinux-policy-targeted, selinux-policy, libselinux, libselinux-python provided by the python3-libselinux package in Red Hat Enterprise Linux 8 (RHEL 8), libselinux-utils, policycoreutils, policycoreutils-python replaced by the policycoreutils-python-utils and python3-policycoreutils packages in RHEL8, setroubleshoot, setroubleshoot-server, setroubleshoot-plugins packages are installed. To verify that the packages are installed, use the following command: 
# rpm -q package_name

Important

If the system was initially installed without SELinux, particularly the selinux-policy package, one additional step is necessary to enable SELinux. To make sure SELinux is initialized during system startup, the dracut utility has to be run to put SELinux awareness into the initramfs file system. Failing to do so causes SELinux to not start during system startup.
  1. Before SELinux is enabled, each file on the file system must be labeled with an SELinux context. Before this happens, confined domains may be denied access, preventing your system from booting correctly. To prevent this, configure SELINUX=permissive in /etc/selinux/config: 
    # This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - No SELinux policy is loaded.
SELINUX=permissive
# SELINUXTYPE= can take one of these two values:
#       targeted - Targeted processes are protected,
#       mls - Multi Level Security protection
SELINUXTYPE=targeted
  2. As the Linux root user, reboot the system. During the next boot, file systems are labeled. The label process labels each file with an SELinux context: 
    *** Warning -- SELinux targeted policy relabel is required.
*** Relabeling could take a very long time, depending on file
*** system size and speed of hard drives.
****
    Each * (asterisk) character on the bottom line represents 1000 files that have been labeled. In the above example, four * characters represent 4000 files have been labeled. The time it takes to label all files depends on the number of files on the system and the speed of hard drives. On modern systems, this process can take as short as 10 minutes.
  3. In permissive mode, the SELinux policy is not enforced, but denial messages are still logged for actions that would have been denied in enforcing mode. Before changing to enforcing mode, as the Linux root user, run the following command to confirm that SELinux did not deny actions during the last boot: 
    # grep "SELinux is preventing" /var/log/messages
    If SELinux did not deny any actions during the last boot, this command returns no output.
  4. If there were no denial messages in /var/log/messages, configure SELINUX=enforcing in /etc/selinux/config: 
    # This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
#       targeted - Targeted processes are protected,
#       mls - Multi Level Security protection.
SELINUXTYPE=targeted
  5. Reboot your system. After reboot, confirm that getenforce returns Enforcing 
    ~]$ getenforce
Enforcing
For more information on changing SELINUX states and modes in REHL 7, refer to Permanent Changes in SELINUX States and Modes in the SELinux User's and Adminstrator's Guide
For more information on changing SELINUX states and modes in REHL 8, refer to Permanent Changes in SELINUX States and Modes in the Using SELinux

Chapter 9. Using the Gluster Command Line Interface

The Gluster command line interface (CLI) simplifies configuration and management of the storage environment. The Gluster CLI is similar to the LVM (Logical Volume Manager) CLI or the ZFS CLI, but operates across multiple storage servers. The Gluster CLI can be used when volumes are mounted (active) and not mounted (inactive). Red Hat Gluster Storage automatically synchronizes volume configuration information across all servers.
Use the Gluster CLI to create new volumes, start and stop existing volumes, add bricks to volumes, remove bricks from volumes, and change translator settings. Additionally, the Gluster CLI commands can create automation scripts and use the commands as an API to allow integration with third-party applications.

Note

Appending --mode=script to any CLI command ensures that the command executes without confirmation prompts.
Running the Gluster CLI

Run the Gluster CLI on any Red Hat Gluster Storage Server by either invoking the commands or running the Gluster CLI in interactive mode. The gluster command can be remotely used via SSH.

Run commands directly as follows, after replacing COMMAND with the required command: 
# gluster COMMAND
The following is an example using the peer status command: 
# gluster peer status
Gluster CLI Interactive Mode

Alternatively, run the Gluster CLI in interactive mode using the following command: 

# gluster
If successful, the prompt changes to the following: 
gluster>
When the prompt appears, execute gluster commands from the CLI prompt as follows: 
gluster> COMMAND
As an example, replace the COMMAND with a command such as peer status to view the status of the peer server:
  1. Start Gluster CLI's interactive mode: 
    # gluster
  2. Request the peer server status: 
    gluster> peer status
  3. The peer server status displays.
The following is another example, replacing the COMMAND with a command such as help to view the gluster help options.
  1. Start Gluster CLI's interactive mode: 
    # gluster
  2. Request the help options: 
    gluster> help
  3. A list of gluster commands and options displays.

Appendix A. Revision History

Revision History
Revision 3.5-0Wed Oct 30 2019Red Hat Gluster Storage Documentation Team
Updated documentation for Red Hat Gluster Storage 3.5.

Legal Notice

Copyright © 2015-2019 Red Hat, Inc.
This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.