Appendix E. Manually Installing Ceph Block Device

The following procedure shows how to install and mount a thin-provisioned, resizable Ceph Block Device.

Important

Ceph Block Devices must be deployed on separate nodes from the Ceph Monitor and OSD nodes. Running kernel clients and kernel server daemons on the same node can lead to kernel deadlocks.

Prerequisites

Procedure

  1. Create a Ceph Block Device user named client.rbd with full permissions to files on OSD nodes (osd 'allow rwx') and output the result to a keyring file: 

    ceph auth get-or-create client.rbd mon 'profile rbd' osd 'profile rbd pool=<pool_name>' \
-o /etc/ceph/rbd.keyring

    Replace <pool_name> with the name of the pool that you want to allow client.rbd to have access to, for example rbd: 

    # ceph auth get-or-create \
client.rbd mon 'allow r' osd 'allow rwx pool=rbd' \
-o /etc/ceph/rbd.keyring

    See the User Management section in the Red Hat Ceph Storage 3 Administration Guide for more information about creating users.

  2. Create a block device image: 

    rbd create <image_name> --size <image_size> --pool <pool_name> \
--name client.rbd --keyring /etc/ceph/rbd.keyring

    Specify <image_name>, <image_size>, and <pool_name>, for example: 

    $ rbd create image1 --size 4096 --pool rbd \
--name client.rbd --keyring /etc/ceph/rbd.keyring
    Warning

    The default Ceph configuration includes the following Ceph Block Device features:

    • layering
    • exclusive-lock
    • object-map
    • deep-flatten
    • fast-diff

    If you use the kernel RBD (krbd) client, you will not be able to map the block device image because the current kernel version included in Red Hat Enterprise Linux 7.3 does not support object-map, deep-flatten, and fast-diff.

    To work around this problem, disable the unsupported features. Use one of the following options to do so:

    • Disable the unsupported features dynamically: 

      rbd feature disable <image_name> <feature_name>

      For example: 

      # rbd feature disable image1 object-map deep-flatten fast-diff
    • Use the --image-feature layering option with the rbd create command to enable only layering on newly created block device images.

    • Disable the features be default in the Ceph configuration file: 

      rbd_default_features = 1

    This is a known issue, for details see the Known Issues chapter in the Release Notes for Red Hat Ceph Storage 3.

    All these features work for users that use the user-space RBD client to access the block device images.

  3. Map the newly created image to the block device: 

    rbd map <image_name> --pool <pool_name>\
--name client.rbd --keyring /etc/ceph/rbd.keyring

    For example: 

    # rbd map image1 --pool rbd --name client.rbd \
--keyring /etc/ceph/rbd.keyring
    Important

    Kernel block devices currently only support the legacy straw bucket algorithm in the CRUSH map. If you have set the CRUSH tunables to optimal, you must set them to legacy or an earlier major release, otherwise, you will not be able to map the image.

    Alternatively, replace straw2 with straw in the CRUSH map. For details, see the Editing a CRUSH Map chapter in the Storage Strategies guide for Red Hat Ceph Storage 3.

  4. Use the block device by creating a file system: 

    mkfs.ext4 -m5 /dev/rbd/<pool_name>/<image_name>

    Specify the pool name and the image name, for example: 

    # mkfs.ext4 -m5 /dev/rbd/rbd/image1

    This can take a few moments.

  5. Mount the newly created file system: 

    mkdir <mount_directory>
mount /dev/rbd/<pool_name>/<image_name> <mount_directory>

    For example: 

    # mkdir /mnt/ceph-block-device
# mount /dev/rbd/rbd/image1 /mnt/ceph-block-device

For additional details, see the Block Device Guide for Red Hat Ceph Storage 3.