Jump To Close Expand all Collapse all Table of contents Administration Guide I. Preface Expand section "I. Preface" Collapse section "I. Preface" 1. Preface Expand section "1. Preface" Collapse section "1. Preface" 1.1. About Red Hat Gluster Storage 1.2. About glusterFS 1.3. About On-premises Installation II. Overview Expand section "II. Overview" Collapse section "II. Overview" 2. Architecture and Concepts Expand section "2. Architecture and Concepts" Collapse section "2. Architecture and Concepts" 2.1. Architecture 2.2. On-premises Architecture 2.3. Storage Concepts III. Configure and Verify Expand section "III. Configure and Verify" Collapse section "III. Configure and Verify" 3. Verifying Port Access 4. Adding Servers to the Trusted Storage Pool Expand section "4. Adding Servers to the Trusted Storage Pool" Collapse section "4. Adding Servers to the Trusted Storage Pool" 4.1. Adding Servers to the Trusted Storage Pool 4.2. Removing Servers from the Trusted Storage Pool 5. Setting Up Storage Volumes Expand section "5. Setting Up Storage Volumes" Collapse section "5. Setting Up Storage Volumes" 5.1. Setting up Gluster Storage Volumes using gdeploy Expand section "5.1. Setting up Gluster Storage Volumes using gdeploy" Collapse section "5.1. Setting up Gluster Storage Volumes using gdeploy" 5.1.1. Getting Started 5.1.2. Setting up a Trusted Storage Pool 5.1.3. Setting up the Backend 5.1.4. Creating Volumes 5.1.5. Mounting Clients 5.1.6. Configuring a Volume 5.1.7. Configuration File 5.1.8. Deploying NFS Ganesha using gdeploy 5.1.9. Deploying Samba / CTDB using gdeploy 5.1.10. Enabling SSL on a Volume 5.2. Managing Volumes using Heketi Expand section "5.2. Managing Volumes using Heketi" Collapse section "5.2. Managing Volumes using Heketi" 5.2.1. Prerequisites 5.2.2. Installing Heketi 5.2.3. Starting the Heketi Server 5.2.4. Setting up the Topology 5.2.5. Creating a Volume 5.2.6. Deleting a Volume 5.3. About Encrypted Disk 5.4. Formatting and Mounting Bricks Expand section "5.4. Formatting and Mounting Bricks" Collapse section "5.4. Formatting and Mounting Bricks" 5.4.1. Creating Bricks Manually 5.4.2. Using Subdirectory as the Brick for Volume 5.4.3. Reusing a Brick from a Deleted Volume 5.4.4. Cleaning An Unusable Brick 5.5. Creating Distributed Volumes 5.6. Creating Replicated Volumes Expand section "5.6. Creating Replicated Volumes" Collapse section "5.6. Creating Replicated Volumes" 5.6.1. Creating Two-way Replicated Volumes 5.6.2. Creating Three-way Replicated Volumes 5.6.3. Creating Sharded Replicated Volumes 5.7. Creating Distributed Replicated Volumes Expand section "5.7. Creating Distributed Replicated Volumes" Collapse section "5.7. Creating Distributed Replicated Volumes" 5.7.1. Creating Two-way Distributed Replicated Volumes 5.7.2. Creating Three-way Distributed Replicated Volumes 5.8. Creating Arbitrated Replicated Volumes Expand section "5.8. Creating Arbitrated Replicated Volumes" Collapse section "5.8. Creating Arbitrated Replicated Volumes" 5.8.1. Arbitrated volume requirements 5.8.2. Arbitration logic 5.8.3. Creating an arbitrated replicated volume 5.8.4. Creating multiple arbitrated replicated volumes across fewer total nodes 5.8.5. Converting to an arbitrated volume 5.8.6. Tuning recommendations for arbitrated volumes 5.9. Creating Dispersed Volumes 5.10. Creating Distributed Dispersed Volumes 5.11. Starting Volumes 6. Creating Access to Volumes Expand section "6. Creating Access to Volumes" Collapse section "6. Creating Access to Volumes" 6.1. Native Client Expand section "6.1. Native Client" Collapse section "6.1. Native Client" 6.1.1. Installing Native Client 6.1.2. Upgrading Native Client 6.1.3. Mounting Red Hat Gluster Storage Volumes 6.2. NFS Expand section "6.2. NFS" Collapse section "6.2. NFS" 6.2.1. Setting up CTDB for NFS 6.2.2. Using NFS to Mount Red Hat Gluster Storage Volumes 6.2.3. Troubleshooting NFS 6.2.4. NFS-Ganesha 6.3. SMB Expand section "6.3. SMB" Collapse section "6.3. SMB" 6.3.1. Setting up CTDB for Samba 6.3.2. Sharing Volumes over SMB 6.3.3. Mounting Volumes using SMB 6.3.4. Starting and Verifying your Configuration 6.3.5. Disabling SMB Shares 6.3.6. Accessing Snapshots in Windows 6.3.7. Tuning Performance 6.4. POSIX Access Control Lists Expand section "6.4. POSIX Access Control Lists" Collapse section "6.4. POSIX Access Control Lists" 6.4.1. Setting ACLs with setfacl 6.4.2. Checking current ACLs with getfacl 6.4.3. Mounting volumes with ACLs enabled 6.4.4. Checking ACL enablement on a mounted volume 6.5. Managing Object Store Expand section "6.5. Managing Object Store" Collapse section "6.5. Managing Object Store" 6.5.1. Architecture Overview 6.5.2. Components of Object Store 6.5.3. Advantages of using Object Store 6.5.4. Limitations 6.5.5. Swift API Support Matrix 6.5.6. Prerequisites 6.5.7. Configuring the Object Store 6.5.8. Starting the Services Automatically 6.5.9. Working with the Object Store 7. Integrating Red Hat Gluster Storage with Windows Active Directory Expand section "7. Integrating Red Hat Gluster Storage with Windows Active Directory" Collapse section "7. Integrating Red Hat Gluster Storage with Windows Active Directory" 7.1. Prerequisites 7.2. Integration Expand section "7.2. Integration" Collapse section "7.2. Integration" 7.2.1. Configure Authentication 7.2.2. Join Active Directory Domain 7.2.3. Verify/Test Active Directory and Services IV. Manage Expand section "IV. Manage" Collapse section "IV. Manage" 8. Managing Snapshots Expand section "8. Managing Snapshots" Collapse section "8. Managing Snapshots" 8.1. Prerequisites 8.2. Creating Snapshots 8.3. Cloning a Snapshot 8.4. Listing of Available Snapshots 8.5. Getting Information of all the Available Snapshots 8.6. Getting the Status of Available Snapshots 8.7. Configuring Snapshot Behavior 8.8. Activating and Deactivating a Snapshot 8.9. Deleting Snapshot Expand section "8.9. Deleting Snapshot" Collapse section "8.9. Deleting Snapshot" 8.9.1. Deleting Multiple Snapshots 8.10. Restoring Snapshot 8.11. Accessing Snapshots 8.12. Scheduling of Snapshots Expand section "8.12. Scheduling of Snapshots " Collapse section "8.12. Scheduling of Snapshots " 8.12.1. Prerequisites 8.12.2. Snapshot Scheduler Options 8.13. User Serviceable Snapshots Expand section "8.13. User Serviceable Snapshots" Collapse section "8.13. User Serviceable Snapshots" 8.13.1. Enabling and Disabling User Serviceable Snapshot 8.13.2. Viewing and Retrieving Snapshots using NFS / FUSE 8.13.3. Viewing and Retrieving Snapshots using CIFS for Windows Client 8.14. Troubleshooting 9. Managing Directory Quotas Expand section "9. Managing Directory Quotas" Collapse section "9. Managing Directory Quotas" 9.1. Enabling and Disabling Quotas 9.2. Before Setting a Quota on a Directory 9.3. Limiting Disk Usage Expand section "9.3. Limiting Disk Usage" Collapse section "9.3. Limiting Disk Usage" 9.3.1. Setting Disk Usage Limits 9.3.2. Viewing Current Disk Usage Limits 9.3.3. Setting Quota Check Frequency (Timeouts) 9.3.4. Setting Logging Frequency (Alert Time) 9.3.5. Removing Disk Usage Limits 10. Managing Geo-replication Expand section "10. Managing Geo-replication" Collapse section "10. Managing Geo-replication" 10.1. About Geo-replication 10.2. Replicated Volumes vs Geo-replication 10.3. Preparing to Deploy Geo-replication Expand section "10.3. Preparing to Deploy Geo-replication" Collapse section "10.3. Preparing to Deploy Geo-replication" 10.3.1. Exploring Geo-replication Deployment Scenarios 10.3.2. Geo-replication Deployment Overview 10.3.3. Prerequisites 10.3.4. Setting Up your Environment 10.3.5. Configuring a Meta-Volume 10.4. Starting Geo-replication Expand section "10.4. Starting Geo-replication" Collapse section "10.4. Starting Geo-replication" 10.4.1. Starting a Geo-replication Session 10.4.2. Verifying a Successful Geo-replication Deployment 10.4.3. Displaying Geo-replication Status Information 10.4.4. Configuring a Geo-replication Session 10.4.5. Stopping a Geo-replication Session 10.4.6. Deleting a Geo-replication Session 10.5. Starting Geo-replication on a Newly Added Brick or Node Expand section "10.5. Starting Geo-replication on a Newly Added Brick or Node" Collapse section "10.5. Starting Geo-replication on a Newly Added Brick or Node" 10.5.1. Starting Geo-replication for a New Brick or New Node 10.5.2. Starting Geo-replication for a New Brick on an Existing Node 10.6. Scheduling Geo-replication as a Cron Job 10.7. Disaster Recovery Expand section "10.7. Disaster Recovery" Collapse section "10.7. Disaster Recovery" 10.7.1. Failover: Promoting a Slave to Master 10.7.2. Failback: Resuming Master and Slave back to their Original State 10.8. Creating a Snapshot of Geo-replicated Volume 10.9. Example - Setting up Cascading Geo-replication 10.10. Recommended Practices 10.11. Troubleshooting Geo-replication Expand section "10.11. Troubleshooting Geo-replication" Collapse section "10.11. Troubleshooting Geo-replication" 10.11.1. Tuning Geo-replication performance with Change Log 10.11.2. Triggering Explicit Sync on Entries 10.11.3. Synchronization Is Not Complete 10.11.4. Issues with File Synchronization 10.11.5. Geo-replication Status is Often Faulty 10.11.6. Intermediate Master is in a Faulty State 10.11.7. Remote gsyncd Not Found 11. Managing Red Hat Gluster Storage Volumes Expand section "11. Managing Red Hat Gluster Storage Volumes" Collapse section "11. Managing Red Hat Gluster Storage Volumes" 11.1. Configuring Volume Options 11.2. Configuring Transport Types for a Volume 11.3. Expanding Volumes Expand section "11.3. Expanding Volumes" Collapse section "11.3. Expanding Volumes" 11.3.1. Expanding a Tiered Volume 11.3.2. Expanding a Dispersed or Distributed-dispersed Volume 11.4. Shrinking Volumes Expand section "11.4. Shrinking Volumes" Collapse section "11.4. Shrinking Volumes" 11.4.1. Shrinking a Geo-replicated Volume 11.4.2. Shrinking a Tiered Volume 11.4.3. Stopping a remove-brick Operation 11.5. Migrating Volumes Expand section "11.5. Migrating Volumes" Collapse section "11.5. Migrating Volumes" 11.5.1. Replacing a Subvolume on a Distribute or Distribute-replicate Volume 11.5.2. Replacing an Old Brick with a New Brick on a Replicate or Distribute-replicate Volume 11.5.3. Replacing an Old Brick with a New Brick on a Distribute Volume 11.5.4. Replacing an Old Brick with a New Brick on a Dispersed or Distributed-dispersed Volume 11.5.5. Reconfiguring a Brick in a Volume 11.6. Replacing Hosts Expand section "11.6. Replacing Hosts" Collapse section "11.6. Replacing Hosts" 11.6.1. Replacing a Host Machine with a Different Hostname 11.6.2. Replacing a Host Machine with the Same Hostname 11.7. Rebalancing Volumes Expand section "11.7. Rebalancing Volumes" Collapse section "11.7. Rebalancing Volumes" 11.7.1. Rebalance Throttling 11.7.2. Displaying Status of a Rebalance Operation 11.7.3. Stopping a Rebalance Operation 11.8. Setting up Shared Storage Volume 11.9. Stopping Volumes 11.10. Deleting Volumes 11.11. Managing Split-brain Expand section "11.11. Managing Split-brain" Collapse section "11.11. Managing Split-brain" 11.11.1. Preventing Split-brain 11.11.2. Recovering from File Split-brain 11.11.3. Triggering Self-Healing on Replicated Volumes 11.12. Recommended Configurations - Dispersed Volume 12. Managing Red Hat Gluster Storage Logs Expand section "12. Managing Red Hat Gluster Storage Logs" Collapse section "12. Managing Red Hat Gluster Storage Logs" 12.1. Log Rotation 12.2. Red Hat Gluster Storage Component Logs and Location 12.3. Configuring the Log Format 12.4. Configuring the Log Level 12.5. Suppressing Repetitive Log Messages 12.6. Geo-replication Logs Expand section "12.6. Geo-replication Logs" Collapse section "12.6. Geo-replication Logs" 12.6.1. Viewing the Geo-replication Master Log Files 12.6.2. Viewing the Geo-replication Slave Log Files 13. Managing Red Hat Gluster Storage Volume Life-Cycle Extensions Expand section "13. Managing Red Hat Gluster Storage Volume Life-Cycle Extensions" Collapse section "13. Managing Red Hat Gluster Storage Volume Life-Cycle Extensions" 13.1. Location of Scripts 13.2. Prepackaged Scripts 14. Managing Containerized Red Hat Gluster Storage Expand section "14. Managing Containerized Red Hat Gluster Storage" Collapse section "14. Managing Containerized Red Hat Gluster Storage" 14.1. Prerequisites 14.2. Starting a Container 14.3. Creating a Trusted Storage Pool 14.4. Creating a Volume 14.5. Mounting a Volume 15. Detecting BitRot Expand section "15. Detecting BitRot" Collapse section "15. Detecting BitRot" 15.1. Enabling and Disabling the BitRot daemon 15.2. Modifying BitRot Detection Behavior 15.3. Restoring a bad file 16. Incremental Backup Assistance using Glusterfind Expand section "16. Incremental Backup Assistance using Glusterfind" Collapse section "16. Incremental Backup Assistance using Glusterfind" 16.1. Glusterfind Configuration Options Expand section "16.1. Glusterfind Configuration Options" Collapse section "16.1. Glusterfind Configuration Options" 16.1.1. Adding or Replacing a Brick from an Existing Glusterfind Session 17. Managing Tiering Expand section "17. Managing Tiering" Collapse section "17. Managing Tiering" 17.1. Tiering Architecture 17.2. Key Benefits of Tiering 17.3. Tiering Limitations 17.4. Attaching a Tier to a Volume Expand section "17.4. Attaching a Tier to a Volume" Collapse section "17.4. Attaching a Tier to a Volume" 17.4.1. Attaching a Tier to a Geo-replicated Volume 17.5. Configuring a Tiering Volume Expand section "17.5. Configuring a Tiering Volume" Collapse section "17.5. Configuring a Tiering Volume" 17.5.1. Configuring Watermarks 17.5.2. Configuring Promote and Demote Frequency 17.5.3. Configuring Read and Write Frequency 17.5.4. Configuring Target Data Size 17.5.5. Configuring the File Count per Cycle 17.6. Displaying Tiering Status Information 17.7. Detaching a Tier from a Volume Expand section "17.7. Detaching a Tier from a Volume" Collapse section "17.7. Detaching a Tier from a Volume" 17.7.1. Detaching a Tier of a Geo-replicated Volume V. Monitor and Tune Expand section "V. Monitor and Tune" Collapse section "V. Monitor and Tune" 18. Monitoring Red Hat Gluster Storage Expand section "18. Monitoring Red Hat Gluster Storage" Collapse section "18. Monitoring Red Hat Gluster Storage" 18.1. Prerequisites 18.2. Installing Nagios Expand section "18.2. Installing Nagios" Collapse section "18.2. Installing Nagios" 18.2.1. Installing Nagios Server 18.2.2. Configuring Red Hat Gluster Storage Nodes for Nagios 18.3. Monitoring Red Hat Gluster Storage Trusted Storage Pool Expand section "18.3. Monitoring Red Hat Gluster Storage Trusted Storage Pool" Collapse section "18.3. Monitoring Red Hat Gluster Storage Trusted Storage Pool" 18.3.1. Configuring Nagios 18.3.2. Verifying the Configuration 18.3.3. Using Nagios Server GUI 18.4. Monitoring Notifications Expand section "18.4. Monitoring Notifications" Collapse section "18.4. Monitoring Notifications" 18.4.1. Configuring Nagios Server to Send Mail Notifications 18.4.2. Configuring Simple Network Management Protocol (SNMP) Notification 18.5. Nagios Advanced Configuration Expand section "18.5. Nagios Advanced Configuration" Collapse section "18.5. Nagios Advanced Configuration" 18.5.1. Creating Nagios User 18.5.2. Changing Nagios Password 18.5.3. Configuring SSL 18.5.4. Integrating LDAP Authentication with Nagios 18.6. Configuring Nagios Manually 18.7. Troubleshooting Nagios Expand section "18.7. Troubleshooting Nagios" Collapse section "18.7. Troubleshooting Nagios" 18.7.1. Troubleshooting NSCA and NRPE Configuration Issues 19. Monitoring Red Hat Gluster Storage Gluster Workload Expand section "19. Monitoring Red Hat Gluster Storage Gluster Workload" Collapse section "19. Monitoring Red Hat Gluster Storage Gluster Workload" 19.1. Running the Volume Profile Command Expand section "19.1. Running the Volume Profile Command" Collapse section "19.1. Running the Volume Profile Command" 19.1.1. Start Profiling 19.1.2. Displaying the I/O Information 19.1.3. Stop Profiling 19.2. Running the Volume Top Command Expand section "19.2. Running the Volume Top Command" Collapse section "19.2. Running the Volume Top Command" 19.2.1. Viewing Open File Descriptor Count and Maximum File Descriptor Count 19.2.2. Viewing Highest File Read Calls 19.2.3. Viewing Highest File Write Calls 19.2.4. Viewing Highest Open Calls on a Directory 19.2.5. Viewing Highest Read Calls on a Directory 19.2.6. Viewing Read Performance 19.2.7. Viewing Write Performance 19.3. gstatus Command Expand section "19.3. gstatus Command" Collapse section "19.3. gstatus Command" 19.3.1. gstatus Command 19.3.2. Executing the gstatus command 19.4. Listing Volumes 19.5. Displaying Volume Information 19.6. Obtaining Node Information 19.7. Retrieving Volume Options Value Expand section "19.7. Retrieving Volume Options Value" Collapse section "19.7. Retrieving Volume Options Value" 19.7.1. Retrieving Value of Specific Volume Option 19.7.2. Retrieving Values of All the Volume Options 19.8. Performing Statedump on a Volume 19.9. Displaying Volume Status 19.10. Troubleshooting issues in the Red Hat Gluster Storage Trusted Storage Pool Expand section "19.10. Troubleshooting issues in the Red Hat Gluster Storage Trusted Storage Pool" Collapse section "19.10. Troubleshooting issues in the Red Hat Gluster Storage Trusted Storage Pool" 19.10.1. Troubleshooting a network issue in the Red Hat Gluster Storage Trusted Storage Pool 20. Tuning for Performance Expand section "20. Tuning for Performance" Collapse section "20. Tuning for Performance" 20.1. Disk Configuration Expand section "20.1. Disk Configuration" Collapse section "20.1. Disk Configuration" 20.1.1. Hardware RAID 20.1.2. JBOD 20.2. Brick Configuration 20.3. Network 20.4. Memory Expand section "20.4. Memory" Collapse section "20.4. Memory" 20.4.1. Virtual Memory Parameters 20.5. Small File Performance Enhancements Expand section "20.5. Small File Performance Enhancements" Collapse section "20.5. Small File Performance Enhancements" 20.5.1. Enabling Lookup Optimization 20.6. Replication 20.7. Directory Operations Expand section "20.7. Directory Operations" Collapse section "20.7. Directory Operations" 20.7.1. Enabling Metadata Caching 21. Nagios Configuration Files VI. Security Expand section "VI. Security" Collapse section "VI. Security" 22. Configuring Network Encryption in Red Hat Gluster Storage Expand section "22. Configuring Network Encryption in Red Hat Gluster Storage" Collapse section "22. Configuring Network Encryption in Red Hat Gluster Storage" 22.1. Prerequisites 22.2. Configuring Network Encryption for a New Trusted Storage Pool Expand section "22.2. Configuring Network Encryption for a New Trusted Storage Pool" Collapse section "22.2. Configuring Network Encryption for a New Trusted Storage Pool" 22.2.1. Enabling Management Encryption 22.2.2. Enabling I/O encryption for a Volume 22.3. Configuring Network Encryption for an existing Trusted Storage Pool Expand section "22.3. Configuring Network Encryption for an existing Trusted Storage Pool" Collapse section "22.3. Configuring Network Encryption for an existing Trusted Storage Pool" 22.3.1. Enabling I/O encryption for a Volume 22.3.2. Enabling Management Encryption 22.4. Expanding Volumes Expand section "22.4. Expanding Volumes" Collapse section "22.4. Expanding Volumes" 22.4.1. Certificate Signed with a Common Certificate Authority 22.4.2. Self-signed Certificates 22.5. Authorizing a New Client Expand section "22.5. Authorizing a New Client" Collapse section "22.5. Authorizing a New Client" 22.5.1. Certificate Signed with a Common Certificate Authority 22.5.2. Self-signed Certificates VII. Troubleshoot Expand section "VII. Troubleshoot" Collapse section "VII. Troubleshoot" 23. Resolving Common Issues Expand section "23. Resolving Common Issues" Collapse section "23. Resolving Common Issues" 23.1. Identifying locked file and clear locks 23.2. Retrieving File Path from the Gluster Volume Expand section "23.2. Retrieving File Path from the Gluster Volume" Collapse section "23.2. Retrieving File Path from the Gluster Volume" 23.2.1. Retrieving Known File Name 23.2.2. Retrieving Unknown File Name 23.2.3. Retrieving File Path using gfid String VIII. Appendices Expand section "VIII. Appendices" Collapse section "VIII. Appendices" 24. Starting and Stopping the glusterd service 25. Manually Recovering File Split-brain A. Revision History Legal Notice Settings Close Language: English Format: Multi-page Single-page PDF Format: Multi-page Single-page PDF Language and Page Formatting Options Language: English Format: Multi-page Single-page PDF Format: Multi-page Single-page PDF Red Hat Training A Red Hat training course is available for Red Hat Gluster Storage Part VII. Troubleshoot Previous Next