Chapter 8. The rbd kernel module

As a storage administrator, you can access Ceph block devices through the rbd kernel module. You can map and unmap a block device, and displaying those mappings. Also, you can get a list of images through the rbd kernel module.

Important

Kernel clients on Linux distributions other than Red Hat Enterprise Linux (RHEL) are permitted but not supported. If issues are found in the storage cluster when using these kernel clients, Red Hat will address them, but if the root cause is found to be on the kernel client side, the issue will have to be addressed by the software vendor.

8.1. Prerequisites

  • A running Red Hat Ceph Storage cluster.

8.2. Create a Ceph Block Device and use it from a Linux kernel module client

As a storage administrator, you can create a Ceph Block Device for a Linux kernel module client in the Red Hat Ceph Storage Dashboard. As a system administrator, you can map that block device on a Linux client, and partition, format, and mount it, using the command line. After this, you can read and write files to it.

Prerequisites

  • A running Red Hat Ceph Storage cluster.
  • A Red Hat Enterprise Linux client.

8.2.1. Creating a Ceph block device for a Linux kernel module client using dashboard

You can create a Ceph block device specifically for a Linux kernel module client using the dashboard web interface by enabling only the features it supports.

Kernel module client supports features like Deep flatten, Layering, Exclusive lock, Object map, and Fast diff.

Object map, Fast diff, and Deep flatten features require Red Hat Enterprise Linux 8.2 and later.

Prerequisites

  • A running Red Hat Ceph Storage cluster.
  • A replicated RBD pool created and enabled.

Procedure

  1. From the Block drop-down menu, select Images.
  2. Click Create.
  3. In the Create RBD window, enter a image name, select the RBD enabled pool, select the supported features:

    Create RBD window
  4. Click Create RBD.

Verification

  • You will get a notification that the image is created successfully.

Additional Resources

8.2.2. Map and mount a Ceph Block Device on Linux using the command line

You can map a Ceph Block Device from a Red Hat Enterprise Linux client using the Linux rbd kernel module. After mapping it, you can partition, format, and mount it, so you can write files to it.

Prerequisites

  • A running Red Hat Ceph Storage cluster.
  • A Ceph block device for a Linux kernel module client using the dashboard is created.
  • A Red Hat Enterprise Linux client.

Procedure

  1. On the Red Hat Enterprise Linux client node, enable the Red Hat Ceph Storage 5 Tools repository:

    Red Hat Enterprise Linux 8

    [root@rbd-client ~]# subscription-manager repos --enable=rhceph-5-tools-for-rhel-8-x86_64-rpms

  2. Install the ceph-common RPM package:

    Red Hat Enterprise Linux 8

    [root@rbd-client ~]# dnf install ceph-common

  3. Copy the Ceph configuration file from a Monitor node to the Client node:

    Syntax

    scp root@MONITOR_NODE:/etc/ceph/ceph.conf /etc/ceph/ceph.conf

    Example

    [root@rbd-client ~]# scp root@cluster1-node2:/etc/ceph/ceph.conf /etc/ceph/ceph.conf
    root@192.168.0.32's password:
    ceph.conf                                                            100%  497   724.9KB/s   00:00
    [root@client1 ~]#

  4. Copy the key file from a Monitor node to the Client node:

    Syntax

    scp root@MONITOR_NODE:/etc/ceph/ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring

    Example

    [root@rbd-client ~]# scp root@cluster1-node2:/etc/ceph/ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring
    root@192.168.0.32's password:
    ceph.client.admin.keyring                                                          100%  151   265.0KB/s   00:00
    [root@client1 ~]#

  5. Map the image:

    Syntax

    rbd map --pool POOL_NAME IMAGE_NAME --id admin

    Example

    [root@rbd-client ~]# rbd map --pool block-device-pool image1 --id admin
    /dev/rbd0
    [root@client1 ~]#

  6. Create a partition table on the block device:

    Syntax

    parted /dev/MAPPED_BLOCK_DEVICE mklabel msdos

    Example

    [root@rbd-client ~]# parted /dev/rbd0 mklabel msdos
    Information: You may need to update /etc/fstab.

  7. Create a partition for an XFS file system:

    Syntax

    parted /dev/MAPPED_BLOCK_DEVICE mkpart primary xfs 0% 100%

    Example

    [root@rbd-client ~]# parted /dev/rbd0 mkpart primary xfs 0% 100%
    Information: You may need to update /etc/fstab.

  8. Format the partition:

    Syntax

    mkfs.xfs /dev/MAPPED_BLOCK_DEVICE_WITH_PARTITION_NUMBER

    Example

    [root@rbd-client ~]# mkfs.xfs /dev/rbd0p1
    meta-data=/dev/rbd0p1            isize=512    agcount=16, agsize=163824 blks
            =                       sectsz=512   attr=2, projid32bit=1
            =                       crc=1        finobt=1, sparse=1, rmapbt=0
            =                       reflink=1
    data     =                       bsize=4096   blocks=2621184, imaxpct=25
            =                       sunit=16     swidth=16 blks
    naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
    log      =internal log           bsize=4096   blocks=2560, version=2
            =                       sectsz=512   sunit=16 blks, lazy-count=1
    realtime =none                   extsz=4096   blocks=0, rtextents=0

  9. Create a directory to mount the new file system on:

    Syntax

    mkdir PATH_TO_DIRECTORY

    Example

    [root@rbd-client ~]# mkdir /mnt/ceph

  10. Mount the file system:

    Syntax

    mount /dev/MAPPED_BLOCK_DEVICE_WITH_PARTITION_NUMBER PATH_TO_DIRECTORY

    Example

    [root@rbd-client ~]# mount /dev/rbd0p1 /mnt/ceph/

  11. Verify that the file system is mounted and showing the correct size:

    Syntax

    df -h PATH_TO_DIRECTORY

    Example

    [root@rbd-client ~]# df -h /mnt/ceph/
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/rbd0p1      10G  105M  9.9G   2% /mnt/ceph

Additional Resources

8.3. Getting a list of images

Get a list of Ceph block device images.

Prerequisites

  • A running Red Hat Ceph Storage cluster.
  • Root-level access to the node.

Procedure

  1. To mount a block device image, first return a list of the images:

    [root@rbd-client ~]# rbd list

8.4. Mapping a block device

Use rbd to map an image name to a kernel module. You must specify the image name, the pool name and the user name. rbd will load the RBD kernel module if it is not already loaded.

Prerequisites

  • A running Red Hat Ceph Storage cluster.
  • Root-level access to the node.

Procedure

  1. Map an image name to a kernel module:

    Syntax

    rbd device map POOL_NAME/IMAGE_NAME --id USER_NAME

    Example

    [root@rbd-client ~]# rbd device map rbd/myimage --id admin

  2. Specify a secret when using cephx authentication by either the keyring or a file containing the secret:

    Syntax

    [root@rbd-client ~]# rbd device map POOL_NAME/IMAGE_NAME --id USER_NAME --keyring PATH_TO_KEYRING

    or

    [root@rbd-client ~]# rbd device map POOL_NAME/IMAGE_NAME --id USER_NAME --keyfile PATH_TO_FILE

8.5. Displaying mapped block devices

You can display which block device images are mapped to the kernel module with the rbd command.

Prerequisites

  • A running Red Hat Ceph Storage cluster.
  • Root-level access to the node.

Procedure

  1. Display the mapped block devices:

    [root@rbd-client ~]# rbd device list

8.6. Unmapping a block device

You can unmap a block device image with the rbd command, by using the unmap option and providing the device name.

Prerequisites

  • A running Red Hat Ceph Storage cluster.
  • Root-level access to the node.

Procedure

  1. Unmap the block device image:

    Syntax

    rbd device unmap /dev/rbd/POOL_NAME/IMAGE_NAME

    Example

    [root@rbd-client ~]# rbd device unmap /dev/rbd/pool1/image1