Red Hat Gluster Storage 3.1

Administration Guide

Configuring and Managing Red Hat Gluster Storage

Divya Muntimadugu

Red Hat Customer Content Services

Bhavana Mohanraj

Red Hat Customer Content Services

Anjana Suparna Sriram

Red Hat Customer Content Services


Red Hat Gluster Storage Administration Guide describes the configuration and management of Red Hat Gluster Storage for On-Premise.
I. Overview
1. Platform Introduction
1.1. About Red Hat Gluster Storage
1.2. About glusterFS
1.3. About On-premise Installation
2. Red Hat Gluster Storage Architecture and Concepts
2.1. Red Hat Gluster Storage Architecture
2.2. Red Hat Gluster Storage for On-premise Architecture
2.3. Storage Concepts
3. Key Features
3.1. Elasticity
3.2. No Metadata with the Elastic Hashing Algorithm
3.3. Scalability
3.4. High Availability and Flexibility
3.5. Flexibility
3.6. No Application Rewrites
3.7. Simple Management
3.8. Modular, Stackable Design
II. Red Hat Gluster Storage Administration On-Premise
4. Getting Started with Red Hat Gluster Storage Server
4.1. Port Information
4.2. Starting and Stopping the glusterd service
5. Trusted Storage Pools
5.1. Adding Servers to the Trusted Storage Pool
5.2. Removing Servers from the Trusted Storage Pool
6. Red Hat Gluster Storage Volumes
6.1. Setting up Gluster Storage Volumes using gdeploy
6.1.1. Getting Started
6.1.2. Setting up a Trusted Storage Pool
6.1.3. Setting up the Backend
6.1.4. Creating Volumes
6.1.5. Mounting Clients
6.1.6. Configuring a Volume
6.1.7. Configuration File
6.2. Managing Volumes using Heketi
6.2.1. Prerequisites
6.2.2. Installing Heketi
6.2.3. Starting the Heketi Server
6.2.4. Setting up the Topology
6.2.5. Creating a Volume
6.2.6. Deleting a Volume
6.3. About Encrypted Disk
6.4. Formatting and Mounting Bricks
6.4.1. Creating Bricks Manually
6.5. Creating Distributed Volumes
6.6. Creating Replicated Volumes
6.6.1. Creating Two-way Replicated Volumes
6.6.2. Creating Three-way Replicated Volumes
6.7. Creating Distributed Replicated Volumes
6.7.1. Creating Two-way Distributed Replicated Volumes
6.7.2. Creating Three-way Distributed Replicated Volumes
6.8. Creating Dispersed Volumes
6.9. Creating Distributed Dispersed Volumes
6.10. Starting Volumes
7. Accessing Data - Setting Up Clients
7.1. Native Client
7.1.1. Installing Native Client
7.1.2. Upgrading Native Client
7.1.3. Mounting Red Hat Gluster Storage Volumes
7.2. NFS
7.2.1. Setting up CTDB for NFS
7.2.2. Using NFS to Mount Red Hat Gluster Storage Volumes
7.2.3. Troubleshooting NFS
7.2.4. NFS-Ganesha
7.3. SMB
7.3.1. Setting up CTDB for Samba
7.3.2. Sharing Volumes over SMB
7.3.3. Mounting Volumes using SMB
7.3.4. Starting and Verifying your Configuration
7.3.5. Disabling SMB Shares
7.3.6. Accessing Snapshots in Windows
7.4. POSIX Access Control Lists
7.4.1. Setting POSIX ACLs
7.4.2. Retrieving POSIX ACLs
7.4.3. Removing POSIX ACLs
7.4.4. Samba and ACLs
8. Integrating Red Hat Gluster Storage with Windows Active Directory
8.1. Prerequisites
8.2. Integration
8.2.1. Configure Authentication
8.2.2. Join Active Directory Domain
8.2.3. Verify/Test Active Directory and Services
9. Configuring Network Encryption in Red Hat Gluster Storage
9.1. Prerequisites
9.2. Configuring Network Encryption for a New Trusted Storage Pool
9.2.1. Enabling Management Encryption
9.2.2. Enabling I/O encryption for a Volume
9.3. Configuring Network Encryption for an existing Trusted Storage Pool
9.3.1. Enabling I/O encryption for a Volume
9.3.2. Enabling Management Encryption
9.4. Expanding Volumes
9.4.1. Certificate Signed with a Common Certificate Authority
9.4.2. Self-signed Certificates
9.5. Authorizing a New Client
9.5.1. Certificate Signed with a Common Certificate Authority
9.5.2. Self-signed Certificates
10. Managing Red Hat Gluster Storage Volumes
10.1. Configuring Volume Options
10.2. Configuring Transport Types for a Volume
10.3. Expanding Volumes
10.3.1. Expanding a Tiered Volume
10.4. Shrinking Volumes
10.4.1. Shrinking a Geo-replicated Volume
10.4.2. Shrinking a Tiered Volume
10.4.3. Stopping a remove-brick Operation
10.5. Migrating Volumes
10.5.1. Replacing a Subvolume on a Distribute or Distribute-replicate Volume
10.5.2. Replacing an Old Brick with a New Brick on a Replicate or Distribute-replicate Volume
10.5.3. Replacing an Old Brick with a New Brick on a Distribute Volume
10.5.4. Replacing an Old Brick with a New Brick on a Dispersed or Distributed-dispersed Volume
10.6. Replacing Hosts
10.6.1. Replacing a Host Machine with a Different Hostname
10.6.2. Replacing a Host Machine with the Same Hostname
10.7. Rebalancing Volumes
10.7.1. Rebalance Throttling
10.7.2. Displaying Status of a Rebalance Operation
10.7.3. Stopping a Rebalance Operation
10.8. Setting up Shared Storage Volume
10.9. Stopping Volumes
10.10. Deleting Volumes
10.11. Managing Split-brain
10.11.1. Preventing Split-brain
10.11.2. Recovering from File Split-brain
10.11.3. Triggering Self-Healing on Replicated Volumes
10.12. Non Uniform File Allocation (NUFA)
11. Managing Containerized Red Hat Gluster Storage
11.1. Prerequisites
11.2. Starting a Container
11.3. Creating a Trusted Storage Pool
11.4. Creating a Volume
11.5. Mounting a Volume
12. Managing Tiering
12.1. Tiering Architecture
12.2. Key Benefits of Tiering
12.3. Tiering Limitations
12.4. Attaching a Tier to a Volume
12.4.1. Attaching a Tier to a Geo-replicated Volume
12.5. Configuring a Tiering Volume
12.5.1. Configuring Watermarks
12.5.2. Configuring Promote and Demote Frequency
12.5.3. Configuring Read and Write Frequency
12.5.4. Configuring Target Data Size
12.5.5. Configuring the File Count per Cycle
12.6. Displaying Tiering Status Information
12.7. Detaching a Tier from a Volume
12.7.1. Detaching a Tier of a Geo-replicated Volume
13. Configuring Red Hat Gluster Storage for Enhancing Performance
13.1. Disk Configuration
13.1.1. Hardware RAID
13.1.2. JBOD
13.2. Brick Configuration
13.3. Network
13.4. Memory
13.4.1. Virtual Memory Parameters
13.5. Small File Performance Enhancements
13.5.1. Enabling Lookup Optimization
13.6. Replication
14. Managing Geo-replication
14.1. About Geo-replication
14.2. Replicated Volumes vs Geo-replication
14.3. Preparing to Deploy Geo-replication
14.3.1. Exploring Geo-replication Deployment Scenarios
14.3.2. Geo-replication Deployment Overview
14.3.3. Prerequisites
14.3.4. Setting Up your Environment
14.3.5. Configuring a Meta-Volume
14.4. Starting Geo-replication
14.4.1. Starting a Geo-replication Session
14.4.2. Verifying a Successful Geo-replication Deployment
14.4.3. Displaying Geo-replication Status Information
14.4.4. Configuring a Geo-replication Session
14.4.5. Stopping a Geo-replication Session
14.4.6. Deleting a Geo-replication Session
14.5. Starting Geo-replication on a Newly Added Brick or Node
14.5.1. Starting Geo-replication for a New Brick or New Node
14.5.2. Starting Geo-replication for a New Brick on an Existing Node
14.6. Scheduling Geo-replication as a Cron Job
14.7. Disaster Recovery
14.7.1. Failover: Promoting a Slave to Master
14.7.2. Failback: Resuming Master and Slave back to their Original State
14.8. Creating a Snapshot of Geo-replicated Volume
14.9. Example - Setting up Cascading Geo-replication
14.10. Recommended Practices
14.11. Troubleshooting Geo-replication
14.11.1. Tuning Geo-replication performance with Change Log
14.11.2. Triggering Explicit Sync on Entries
14.11.3. Synchronization Is Not Complete
14.11.4. Issues with File Synchronization
14.11.5. Geo-replication Status is Often Faulty
14.11.6. Intermediate Master is in a Faulty State
14.11.7. Remote gsyncd Not Found
15. Managing Directory Quotas
15.1. Enabling Quotas
15.2. Setting Limits
15.3. Setting the Default Soft Limit
15.4. Displaying Quota Limit Information
15.4.1. Displaying Quota Limit Information Using the df Utility
15.5. Setting Timeout
15.6. Setting Alert Time
15.7. Removing Disk Limits
15.8. Disabling Quotas
16. Managing Sharding
16.1. Supported use cases
16.2. Configuration Options
16.3. Finding the pieces of a sharded file
17. Managing Snapshots
17.1. Prerequisites
17.2. Creating Snapshots
17.3. Cloning a Snapshot
17.4. Listing of Available Snapshots
17.5. Getting Information of all the Available Snapshots
17.6. Getting the Status of Available Snapshots
17.7. Configuring Snapshot Behavior
17.8. Activating and Deactivating a Snapshot
17.9. Deleting Snapshot
17.9.1. Deleting Multiple Snapshots
17.10. Restoring Snapshot
17.11. Accessing Snapshots
17.12. Scheduling of Snapshots
17.12.1. Prerequisites
17.12.2. Snapshot Scheduler Options
17.13. User Serviceable Snapshots
17.13.1. Enabling and Disabling User Serviceable Snapshot
17.13.2. Viewing and Retrieving Snapshots using NFS / FUSE
17.13.3. Viewing and Retrieving Snapshots using CIFS for Windows Client
17.14. Troubleshooting
18. Monitoring Red Hat Gluster Storage
18.1. Prerequisites
18.2. Installing Nagios
18.2.1. Installing Nagios Server
18.2.2. Configuring Red Hat Gluster Storage Nodes for Nagios
18.3. Monitoring Red Hat Gluster Storage Trusted Storage Pool
18.3.1. Configuring Nagios
18.3.2. Verifying the Configuration
18.3.3. Using Nagios Server GUI
18.4. Monitoring Notifications
18.4.1. Configuring Nagios Server to Send Mail Notifications
18.4.2. Configuring Simple Network Management Protocol (SNMP) Notification
18.5. Nagios Advanced Configuration
18.5.1. Creating Nagios User
18.5.2. Changing Nagios Password
18.5.3. Configuring SSL
18.5.4. Integrating LDAP Authentication with Nagios
18.6. Configuring Nagios Manually
18.7. Troubleshooting Nagios
18.7.1. Troubleshooting NSCA and NRPE Configuration Issues
19. Monitoring Red Hat Gluster Storage Gluster Workload
19.1. Running the Volume Profile Command
19.1.1. Start Profiling
19.1.2. Displaying the I/O Information
19.1.3. Stop Profiling
19.2. Running the Volume Top Command
19.2.1. Viewing Open File Descriptor Count and Maximum File Descriptor Count
19.2.2. Viewing Highest File Read Calls
19.2.3. Viewing Highest File Write Calls
19.2.4. Viewing Highest Open Calls on a Directory
19.2.5. Viewing Highest Read Calls on a Directory
19.2.6. Viewing Read Performance
19.2.7. Viewing Write Performance
19.3. gstatus Command
19.3.1. gstatus Command
19.3.2. Executing the gstatus command
19.4. Listing Volumes
19.5. Displaying Volume Information
19.6. Retrieving Volume Options Value
19.6.1. Retrieving Value of Specific Volume Option
19.6.2. Retrieving Values of All the Volume Options
19.7. Performing Statedump on a Volume
19.8. Displaying Volume Status
19.9. Troubleshooting issues in the Red Hat Gluster Storage Trusted Storage Pool
19.9.1. Troubleshooting a network issue in the Red Hat Gluster Storage Trusted Storage Pool
20. Detecting Data Corruption with BitRot
20.1. Enabling and Disabling the BitRot daemon
20.2. Modifying BitRot Detection Behavior
20.3. Restore a bad file
21. Managing Red Hat Gluster Storage Logs
21.1. Log Rotation
21.2. Red Hat Gluster Storage Component Logs and Location
21.3. Configuring the Log Format
21.4. Configuring the Log Level
21.5. Suppressing Repetitive Log Messages
21.6. Geo-replication Logs
21.6.1. Viewing the Geo-replication Master Log Files
21.6.2. Viewing the Geo-replication Slave Log Files
22. Managing Red Hat Gluster Storage Volume Life-Cycle Extensions
22.1. Location of Scripts
22.2. Prepackaged Scripts
23. Red Hat Gluster Storage Utilities
23.1. Glusterfind Configuration Options
23.1.1. Adding or Replacing a Brick from an Existing Glusterfind Session
III. Data Access with Other Interfaces
24. Managing Object Store
24.1. Architecture Overview
24.2. Components of Object Store
24.3. Advantages of using Object Store
24.4. Limitations
24.5. Prerequisites
24.6. Configuring the Object Store
24.6.1. Configuring a Proxy Server
24.6.2. Configuring the Authentication Service
24.6.3. Configuring Object Servers
24.6.4. Configuring Container Servers
24.6.5. Configuring Account Servers
24.6.6. Configuring Swift Object and Container Constraints
24.6.7. Configuring Object Expiration
24.6.8. Exporting the Red Hat Gluster Storage Volumes
24.6.9. Starting and Stopping Server
24.7. Starting the Services Automatically
24.8. Working with the Object Store
24.8.1. Creating Containers and Objects
24.8.2. Creating Subdirectory under Containers
24.8.3. Working with Swift ACLs
25. Administering the Hortonworks Data Platform on Red Hat Gluster Storage
25.1. Deployment Scenarios
25.1.1. Red Hat Gluster Storage Trusted Storage Pool with Two Additional Servers
25.1.2. Red Hat Gluster Storage Trusted Storage Pool with One Additional Server
25.1.3. Red Hat Gluster Storage Trusted Storage Pool only
25.1.4. Deploying Hadoop on an existing Red Hat Gluster Storage Trusted Storage Pool
25.1.5. Deploying Hadoop on a New Red Hat Gluster Storage Trusted Storage Pool
25.2. Administration of HDP Services with Ambari on Red Hat Gluster Storage
25.3. Managing Users of the System
25.4. Running Hadoop Jobs Across Multiple Red Hat Gluster Storage Volumes
25.5. Scaling Up and Scaling Down
25.5.1. Scaling Up
25.5.2. Scaling Down
25.6. Creating a Snapshot of Hadoop enabled Red Hat Gluster Storage Volumes
25.7. Creating Quotas on Hadoop enabled Red Hat Gluster Storage Volume
IV. Appendices
26. Troubleshooting
26.1. Identifying locked file and clear locks
26.2. Retrieving File Path from the Gluster Volume
26.2.1. Retrieving Known File Name
26.2.2. Retrieving Unknown File Name
26.2.3. Retrieving File Path using gfid String
27. Recommended Configurations - Dispersed Volume
28. Nagios Configuration Files
29. Manually Recovering File Split-brain
A. Revision History