Appendix E. Manually Installing Ceph Block Device
The following procedure shows how to install and mount a thin-provisioned, resizable Ceph Block Device.
Ceph Block Devices must be deployed on separate nodes from the Ceph Monitor and OSD nodes. Running kernel clients and kernel server daemons on the same node can lead to kernel deadlocks.
- Ensure to perform the tasks listed in the Appendix C, Installing the Ceph Command Line Interface section.
- If you use Ceph Block Devices as a back end for virtual machines (VMs) that use QEMU, increase the default file descriptor. See the Ceph - VM hangs when transferring large amounts of data to RBD disk Knowledgebase article for details.
Create a Ceph Block Device user named
client.rbdwith full permissions to files on OSD nodes (
osd 'allow rwx') and output the result to a keyring file:
ceph auth get-or-create client.rbd mon 'profile rbd' osd 'profile rbd pool=<pool_name>' \ -o /etc/ceph/rbd.keyring
<pool_name>with the name of the pool that you want to allow
client.rbdto have access to, for example
# ceph auth get-or-create \ client.rbd mon 'allow r' osd 'allow rwx pool=rbd' \ -o /etc/ceph/rbd.keyring
See the User Management section in the Red Hat Ceph Storage 3 Administration Guide for more information about creating users.
Create a block device image:
rbd create <image_name> --size <image_size> --pool <pool_name> \ --name client.rbd --keyring /etc/ceph/rbd.keyring
<pool_name>, for example:
$ rbd create image1 --size 4096 --pool rbd \ --name client.rbd --keyring /etc/ceph/rbd.keyringWarning
The default Ceph configuration includes the following Ceph Block Device features:
If you use the kernel RBD (
krbd) client, you will not be able to map the block device image because the current kernel version included in Red Hat Enterprise Linux 7.3 does not support
To work around this problem, disable the unsupported features. Use one of the following options to do so:
Disable the unsupported features dynamically:
rbd feature disable <image_name> <feature_name>
# rbd feature disable image1 object-map deep-flatten fast-diff
--image-feature layeringoption with the
rbd createcommand to enable only
layeringon newly created block device images.
Disable the features be default in the Ceph configuration file:
rbd_default_features = 1
This is a known issue, for details see the Known Issues chapter in the Release Notes for Red Hat Ceph Storage 3.
All these features work for users that use the user-space RBD client to access the block device images.
Map the newly created image to the block device:
rbd map <image_name> --pool <pool_name>\ --name client.rbd --keyring /etc/ceph/rbd.keyring
# rbd map image1 --pool rbd --name client.rbd \ --keyring /etc/ceph/rbd.keyringImportant
Kernel block devices currently only support the legacy straw bucket algorithm in the CRUSH map. If you have set the CRUSH tunables to optimal, you must set them to legacy or an earlier major release, otherwise, you will not be able to map the image.
strawin the CRUSH map. For details, see the Editing a CRUSH Map chapter in the Storage Strategies guide for Red Hat Ceph Storage 3.
Use the block device by creating a file system:
mkfs.ext4 -m5 /dev/rbd/<pool_name>/<image_name>
Specify the pool name and the image name, for example:
# mkfs.ext4 -m5 /dev/rbd/rbd/image1
This can take a few moments.
Mount the newly created file system:
mkdir <mount_directory> mount /dev/rbd/<pool_name>/<image_name> <mount_directory>
# mkdir /mnt/ceph-block-device # mount /dev/rbd/rbd/image1 /mnt/ceph-block-device
For additional details, see the Block Device Guide for Red Hat Ceph Storage 3.