Red Hat Storage 2.1

Administration Guide

Configuring and Managing Red Hat Storage Server

Edition 1

Divya Muntimadugu

Red Hat Engineering Content Services

Bhavana Mohanraj

Red Hat Engineering Content Services

Anjana Suparna Sriram

Red Hat Engineering Content Services

Legal Notice

Copyright © 2013-2014 Red Hat Inc.
This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack Logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.


Red Hat Storage Administration Guide describes the configuration and management of Red Hat Storage Server for On-Premise and Public Cloud.
1. Audience
2. License
3. Document Conventions
3.1. Typographic Conventions
3.2. Pull-quote Conventions
3.3. Notes and Warnings
4. Getting Help and Giving Feedback
4.1. Do You Need Help?
4.2. We Need Feedback
I. Introduction
1. Platform Introduction
1.1. About Red Hat Storage
1.2. About glusterFS
1.3. On-premise Installation
1.4. Public Cloud Installation
2. Red Hat Storage Architecture
2.1. Red Hat Storage Server for On-premise Architecture
2.2. Red Hat Storage Server for Public Cloud Architecture
3. Key Features
3.1. Elasticity
3.2. No Metadata with the Elastic Hashing Algorithm
3.3. Scalability
3.4. High Availability and Flexibility
3.5. Flexibility
3.6. No Application Rewrites
3.7. Simple Management
3.8. Modular, Stackable Design
4. Use Case Examples
4.1. Use Case 1: Using Red Hat Storage for Data Archival
4.1.1. Key Features of Red Hat Storage Server for Nearline Storage and Archival
4.2. Use Case 2: Using Red Hat Storage for High Performance Computing
4.2.1. Key Features of Red Hat Storage Server for High Performance Computing
4.3. Use Case 3: Using Red Hat Storage for Content Clouds
4.3.1. Key Features of Red Hat Storage Server for Content Clouds
5. Storage Concepts
II. Red Hat Storage Administration On-Premise
6. The glusterd Service
6.1. Starting and Stopping the glusterd service
7. Trusted Storage Pools
7.1. Adding Servers to the Trusted Storage Pool
7.2. Removing Servers from the Trusted Storage Pool
8. Red Hat Storage Volumes
8.1. Formatting and Mounting Bricks
8.2. Encrypted Disk
8.3. Creating Distributed Volumes
8.4. Creating Replicated Volumes
8.5. Creating Distributed Replicated Volumes
8.6. Creating Striped Volumes
8.7. Creating Distributed Striped Volumes
8.8. Creating Striped Replicated Volumes
8.9. Creating Distributed Striped Replicated Volumes
8.10. Starting Volumes
9. Accessing Data - Setting Up Clients
9.1. Securing Red Hat Storage Client Access
9.2. Native Client
9.2.1. Installing Native Client
9.2.2. Upgrading Native Client
9.2.3. Mounting Red Hat Storage Volumes
9.3. NFS
9.3.1. Using NFS to Mount Red Hat Storage Volumes
9.3.2. Troubleshooting NFS
9.3.3. NFS Ganesha
9.4. SMB
9.4.1. Automatic and Manual Exporting
9.4.2. Mounting Volumes using SMB
9.5. Configuring Automated IP Failover for NFS and SMB
9.5.1. Setting Up CTDB
9.5.2. Starting and Verifying your Configuration
9.6. POSIX Access Control Lists
9.6.1. Setting POSIX ACLs
9.6.2. Retrieving POSIX ACLs
9.6.3. Removing POSIX ACLs
9.6.4. Samba and ACLs
10. Managing Red Hat Storage Volumes
10.1. Configuring Volume Options
10.2. Expanding Volumes
10.3. Shrinking Volumes
10.3.1. Stopping a remove-brick Operation
10.4. Migrating Volumes
10.4.1. Replacing a Subvolume on a Distribute or Distribute-replicate Volume
10.4.2. Replacing an Old Brick with a New Brick on a Replicate or Distribute-replicate Volume
10.4.3. Replacing an Old Brick with a New Brick on a Distribute Volume
10.5. Replacing Hosts
10.5.1. Replacing a Host Machine with a Different IP Address
10.5.2. Replacing a Host Machine with the Same IP Address
10.6. Rebalancing Volumes
10.6.1. Displaying Status of a Rebalance Operation
10.6.2. Stopping a Rebalance Operation
10.7. Stopping Volumes
10.8. Deleting Volumes
10.9. Managing Split-brain
10.9.1. Preventing Split-brain
10.9.2. Recovering from File Split-brain
10.9.3. Triggering Self-Healing on Replicated Volumes
10.10. Non Uniform File Allocation (NUFA)
11. Configuring Red Hat Storage for Enhancing Performance
11.1. Brick Configuration
11.2. Network
11.3. RAM on the Nodes
11.4. Number of Clients
11.5. Replication
11.6. Hardware RAID
12. Managing Geo-replication
12.1. About Geo-replication
12.2. Replicated Volumes vs Geo-replication
12.3. Preparing to Deploy Geo-replication
12.3.1. Exploring Geo-replication Deployment Scenarios
12.3.2. Geo-replication Deployment Overview
12.3.3. Prerequisites
12.3.4. Configuring the Environment and Creating a Geo-replication Session
12.4. Starting Geo-replication
12.4.1. Starting a Geo-replication Session
12.4.2. Verifying a Successful Geo-replication Deployment
12.4.3. Displaying Geo-replication Status Information
12.4.4. Configuring a Geo-replication Session
12.4.5. Stopping a Geo-replication Session
12.4.6. Deleting a Geo-replication Session
12.5. Starting Geo-replication on a Newly Added Brick
12.5.1. Starting Geo-replication for a New Brick on a New Node
12.5.2. Starting Geo-replication for a New Brick on an Existing Node
12.6. Disaster Recovery
12.6.1. Promoting a Slave to Master
12.6.2. Failover and Failback
12.7. Example - Setting up Cascading Geo-replication
12.8. Recommended Practices
12.9. Troubleshooting Geo-replication
12.9.1. Locating Log Files
12.9.2. Synchronization Is Not Complete
12.9.3. Issues with File Synchronization
12.9.4. Geo-replication Status is Often Faulty
12.9.5. Intermediate Master is in a Faulty State
12.9.6. Remote gsyncd Not Found
13. Managing Directory Quotas
13.1. Enabling Quotas
13.2. Setting Limits
13.3. Setting the Default Soft Limit
13.4. Displaying Quota Limit Information
13.4.1. Displaying Quota Limit Information Using the df Utility
13.5. Setting Timeout
13.6. Setting Alert Time
13.7. Removing Disk Limits
13.8. Disabling Quotas
14. Monitoring Your Red Hat Storage Workload
14.1. Running the Volume Profile Command
14.1.1. Start Profiling
14.1.2. Displaying the I/O Information
14.1.3. Stop Profiling
14.2. Running the Volume Top Command
14.2.1. Viewing Open File Descriptor Count and Maximum File Descriptor Count
14.2.2. Viewing Highest File Read Calls
14.2.3. Viewing Highest File Write Calls
14.2.4. Viewing Highest Open Calls on a Directory
14.2.5. Viewing Highest Read Calls on a Directory
14.2.6. Viewing Read Performance
14.2.7. Viewing Write Performance
14.3. Listing Volumes
14.4. Displaying Volume Information
14.5. Performing Statedump on a Volume
14.6. Displaying Volume Status
15. Managing Red Hat Storage Volume Life-Cycle Extensions
15.1. Location of Scripts
15.2. Prepackaged Scripts
III. Red Hat Storage Administration on Public Cloud
16. Launching Red Hat Storage Server for Public Cloud
16.1. Launching Red Hat Storage Instances
16.2. Verifying that Red Hat Storage Instance is Running
17. Provisioning Storage
18. Stopping and Restarting Red Hat Storage Instance
IV. Data Access with Other Interfaces
19. Managing Object Store
19.1. Architecture Overview
19.2. Components of Object Storage
19.3. Advantages of using Object Store
19.4. Limitations
19.5. Prerequisites
19.6. Configuring the Object Store
19.6.1. Configuring a Proxy Server
19.6.2. Configuring the Authentication Service
19.6.3. Configuring an Object Server
19.6.4. Configuring a Container Server
19.6.5. Configuring an Account Server
19.6.6. Configuring Swift Object and Container Constrains
19.6.7. Exporting the Red Hat Storage Volumes
19.6.8. Starting and Stopping Server
19.7. Starting the Services Automatically
19.8. Working with the Object Store
19.8.1. Creating Containers and Objects
19.8.2. Creating Subdirectory under Containers
19.8.3. Working with Swift ACLs
V. Appendices
20. Troubleshooting
20.1. Managing Red Hat Storage Logs
20.1.1. Rotating Logs
20.2. Troubleshooting File Locks
A. Revision History