Chapter 3. GNBD Driver and Command Usage

The Global Network Block Device (GNBD) driver allows a node to export its local storage as a GNBD over a network so that other nodes on the network can share the storage. Client nodes importing the GNBD use it like any other block device. Importing a GNBD on multiple clients forms a shared storage configuration through which GFS can be used.
The GNBD driver is implemented through the following components.
  • gnbd_serv — Implements the GNBD server. It is a user-space daemon that allows a node to export local storage over a network.
  • gnbd.ko — Implements the GNBD device driver on GNBD clients (nodes using GNBD devices).
Two user commands are available to configure GNBD:
  • gnbd_export (for servers) — User program for creating, exporting, and managing GNBDs on a GNBD server.
  • gnbd_import (for clients) — User program for importing and managing GNBDs on a GNBD client.

3.1. Exporting a GNBD from a Server

The gnbd_serv daemon must be running on a node before it can export storage as a GNBD. You can start the gnbd_serv daemon running gnbd_serv as follows: 
#gnbd_serv
gnbd_serv: startup succeeded
Once local storage has been identified to be exported, the gnbd_export command is used to export it.

Warning

When you configure GNBD servers with device-mapper multipath, you must not use page caching. All GNBDs that are part of a logical volume must run with caching disabled. By default, the gnbd_export command exports with caching turned off.

Note

A server should not import the GNBDs to use them as a client would. If a server exports the devices uncached, the underlying devices may also be used by gfs.

Usage

gnbd_export -d pathname -e gnbdname [-c][-u][-U
pathname
Specifies a storage device to export.
gnbdname
Specifies an arbitrary name selected for the GNBD. It is used as the device name on GNBD clients. This name must be unique among all GNBDs exported in a network.
-o
Export the device as read-only.
-c
Enable caching. Reads from the exported GNBD and takes advantage of the Linux page cache.
By default, the gnbd_export command does not enable caching.

Warning

When you configure GNBD servers with device-mapper multipath, do not specify the -c option, as this lead sto data corruption. All GNBDs that are part of a logical volume must run with caching disabled.

Note

If you have been using GFS 5.2 or earlier and do not want to change your GNBD setup you should specify the -c option. Before GFS Release 5.2.1, Linux caching was enabled by default for gnbd_export. If the -c option is not specified, GNBD runs with a noticeable performance decrease. Also, if the -c option is not specified, the exported GNBD runs in timeout mode, using the default timeout value (the -t option). For more information about the gnbd_export command and its options, refer to the gnbd_export man page.
-u uid
Manually sets the Universal Identifier for an exported device. This option is used with -e. The UID is used by device-mapper multipath to determine which devices belong in a multipath map. A device must have a UID to be multipathed. However, for most SCSI devices the default Get UID command, /usr/sbin/gnbd_get_uid, will return an appropriate value.

Note

The UID refers to the device being exported, not the GNBD itself. The UIDs of two GNBD devices should be equal, only if they are exporting the same underlying device. This means that both GNBD servers are connected to the same physical device.

Warning

This option should only be used for exporting shared storage devices, when the -U command option does not work. This should almost never happen for SCSI devices. If two GNBD devices are not exporting the same underlying device, but are given the same UID, data corruption will occur.
-U Command
Gets the UID command. The UID command is a command the gnbd_export command will run to get a Universal Identifier for the exported device. The UID is necessary to use device-mapper multipath with GNBD. The command must use the full path of any executeable that you wish to run. A command can contain the %M, %m or %n escape sequences. %M will be expanded to the major number of the exported device, %m will be expaned to the minor number of the exported device, and %n will be expanded to the sysfs name for the device. If no command is given, GNBD will use the default command /usr/sbin/gnbd_get_uid. This command will work for most SCSI devices.

Examples

This example is for a GNBD server configured with GNBD multipath. It exports device /dev/sdc2 as GNBD gamma. Cache is disabled by default. 
gnbd_export -d /dev/sdc2 -e gamma -U
This example is for a GNBD server not configured with GNBD multipath. It exports device /dev/sdb2 as GNBD delta with cache enabled. 
gnbd_export -d /dev/sdb1 -e delta -c
This example exports device /dev/sdb2 as GNBD delta with cache enabled. 
gnbd_export -d /dev/sdb2 -e delta -c