Global Network Block Device
Using GNBD with Red Hat Global File System
1. About This Guide
3. Software Versions
Table 1. Software Versions
refers to RHEL4 and higher
refers to GFS 6.1 and higher
4. Related Documentation
- Red Hat Enterprise Linux Installation Guide — Provides information regarding installation of Red Hat Enterprise Linux.
- Red Hat Enterprise Linux Introduction to System Administration — Provides introductory information for new Red Hat Enterprise Linux system administrators.
- Red Hat Enterprise Linux System Administration Guide — Provides more detailed information about configuring Red Hat Enterprise Linux to suit your particular needs as a user.
- Red Hat Enterprise Linux Reference Guide — Provides detailed information suited for more experienced users to reference when needed, as opposed to step-by-step instructions.
- Red Hat Enterprise Linux Security Guide — Details the planning and the tools involved in creating a secured computing environment for the data center, workplace, and home.
- Red Hat Cluster Suite Overview — Provides a high level overview of the Red Hat Cluster Suite.
- Configuring and Managing a Red Hat Cluster — Provides information about installing, configuring and managing Red Hat Cluster components.
- Global File System: Configuration and Administration — Provides information about installing, configuring, and maintaining Red Hat GFS (Red Hat Global File System).
- LVM Administrator's Guide: Configuration and Administration — Provides a description of the Logical Volume Manager (LVM), including information on running LVM in a clustered environment.
- Using Device-Mapper Multipath — Provides information about using the Device-Mapper Multipath feature of Red Hat Enterprise Linux.
- Linux Virtual Server Administration — Provides information on configuring high-performance systems and services with the Linux Virtual Server (LVS).
- Red Hat Cluster Suite Release Notes — Provides information about the current release of Red Hat Cluster Suite.
Chapter 1. Using GNBD with Red Hat GFS
Table 1.1. GNBD Software Subsystem Components
|GNBD|| ||Kernel module that implements the GNBD device driver on clients.|
| ||Command to create, export and manage GNBDs on a GNBD server.|
| ||Command to import and manage GNBDs on a GNBD client.|
| ||A server daemon that allows a node to export local storage over the network.|
- Chapter 2, Considerations for Using GNBD with Device-Mapper Multipath, which describes some of the issues you should take into account when configuring multipathed GNBD server nodes
- Chapter 3, GNBD Driver and Command Usage, which describes the restrictions that apply when you are running GFS on a GNBD server node
- Chapter 4, Running GFS on a GNBD Server Node, which describes the user commands that configure GNBD
Chapter 2. Considerations for Using GNBD with Device-Mapper Multipath
- Linux page caching, as desribed in Section 2.1, “Linux Page Caching”.
- Fencing GNBD server nodes, as described in Section 2.2, “Fencing GNBD Server Nodes”.
- GNBD device names; export names for GNBD devices must be unique. Additionally, you must specify the
-Uwhen using the
gnbd_exportcommand. Exporting GNBD devices is described in Chapter 3, GNBD Driver and Command Usage.
2.1. Linux Page Caching
-coption of the
gnbd_exportcommand). All GNBDs that are part of a logical volume must run with caching disabled. Data corruption occurs if the GNBDs are run with caching enabled. Refer to Section 3.1, “Exporting a GNBD from a Server” for more information about using the
gnbd_exportcommand for GNBD with device-mapper multipath.
2.2. Fencing GNBD Server Nodes
fence_scsifence agent. In addition, you cannot use the GNBD fencing device (
fence_gnbdfence agent) to fence a GNBD server node. For information about configuring fencing for GNBD server nodes, refer to the Global File System manual.
Chapter 3. GNBD Driver and Command Usage
gnbd_serv— Implements the GNBD server. It is a user-space daemon that allows a node to export local storage over a network.
gnbd.ko— Implements the GNBD device driver on GNBD clients (nodes using GNBD devices).
3.1. Exporting a GNBD from a Server
gnbd_servdaemon must be running on a node before it can export storage as a GNBD. You can start the
gnbd_servgnbd_serv: startup succeeded
gnbd_exportcommand is used to export it.
gnbd_exportcommand exports with caching turned off.
- Specifies a storage device to export.
- Specifies an arbitrary name selected for the GNBD. It is used as the device name on GNBD clients. This name must be unique among all GNBDs exported in a network.
- Export the device as read-only.
- Enable caching. Reads from the exported GNBD and takes advantage of the Linux page cache.By default, the
gnbd_exportcommand does not enable caching.
WarningWhen you configure GNBD servers with device-mapper multipath, do not specify the
-coption, as this lead sto data corruption. All GNBDs that are part of a logical volume must run with caching disabled.
NoteIf you have been using GFS 5.2 or earlier and do not want to change your GNBD setup you should specify the
-coption. Before GFS Release 5.2.1, Linux caching was enabled by default for
gnbd_export. If the
-coption is not specified, GNBD runs with a noticeable performance decrease. Also, if the
-coption is not specified, the exported GNBD runs in timeout mode, using the default timeout value (the
-toption). For more information about the
gnbd_exportcommand and its options, refer to the
- Manually sets the Universal Identifier for an exported device. This option is used with
-e. The UID is used by device-mapper multipath to determine which devices belong in a multipath map. A device must have a UID to be multipathed. However, for most SCSI devices the default Get UID command,
/usr/sbin/gnbd_get_uid, will return an appropriate value.
NoteThe UID refers to the device being exported, not the GNBD itself. The UIDs of two GNBD devices should be equal, only if they are exporting the same underlying device. This means that both GNBD servers are connected to the same physical device.
WarningThis option should only be used for exporting shared storage devices, when the
commandoption does not work. This should almost never happen for SCSI devices. If two GNBD devices are not exporting the same underlying device, but are given the same UID, data corruption will occur.
- Gets the UID command. The UID command is a command the
gnbd_exportcommand will run to get a Universal Identifier for the exported device. The UID is necessary to use device-mapper multipath with GNBD. The command must use the full path of any executeable that you wish to run. A command can contain the %M, %m or %n escape sequences. %M will be expanded to the major number of the exported device, %m will be expaned to the minor number of the exported device, and %n will be expanded to the
sysfsname for the device. If no command is given, GNBD will use the default command
/usr/sbin/gnbd_get_uid. This command will work for most SCSI devices.
gamma. Cache is disabled by default.
gnbd_export -d /dev/sdc2 -e gamma -U
deltawith cache enabled.
gnbd_export -d /dev/sdb1 -e delta -c
deltawith cache enabled.
gnbd_export -d /dev/sdb2 -e delta -c
3.2. Importing a GNBD on a Client
gnbd.kokernel module must be loaded on a node before it can import GNBDs. When GNBDs are imported, device nodes are created for them in
/dev/gnbd/with the name assigned when they were exported.
- Specifies a GNBD server by hostname or IP address from which to import GNBDs. All GNBDs exported from the server are imported on the client running this command.
gnbd_import -i nodeA
Chapter 4. Running GFS on a GNBD Server Node
- A GNBD server node must have local access to all storage devices needed to mount a GFS file system. The GNBD server node must not import (
gnbd_importcommand) other GNBD devices to run the file system.
- The GNBD server must export all the GNBDs in uncached mode, and it must export the raw devices, not logical volume devices.
- GFS must be run on top of a logical volume device, not raw devices.
Appendix A. Revision History
|Revision 1.0-0||Wed Apr 01 2009|
- device-mapper multipath, Considerations for Using GNBD with Device-Mapper Multipath
- driver and command usage, GNBD Driver and Command Usage
- exporting from a server daemon, Exporting a GNBD from a Server
- GFS, using on a GNBD server node, Running GFS on a GNBD Server Node
- GNBD, using with Red Hat GFS, Using GNBD with Red Hat GFS
- gnbd.ko module, GNBD Driver and Command Usage, Importing a GNBD on a Client
- gnbd_export command , GNBD Driver and Command Usage, Usage
- gnbd_import command , GNBD Driver and Command Usage, Usage
- gnbd_serv daemon, GNBD Driver and Command Usage, Exporting a GNBD from a Server
- importing on a client module, Importing a GNBD on a Client
- Linux page caching, Linux Page Caching
- software subsystem components, Using GNBD with Red Hat GFS