Appendix D. Manually Installing Ceph Block Device
The following procedure shows how to install and mount a thin-provisioned, resizable Ceph Block Device.
Using a Red Hat Ceph Storage 3 node as a client is not supported.
Prerequisites
- Ensure to perform the tasks listed in the Appendix C, Installing the Ceph Command Line Interface section.
- If you use Ceph Block Devices as a back end for virtual machines (VMs) that use QEMU, increase the default file descriptor. See the Ceph - VM hangs when transferring large amounts of data to RBD disk Knowledgebase article for details.
Procedure
Create a Ceph Block Device user named
client.rbd
with full permissions to files on OSD nodes (osd 'allow rwx'
) and output the result to a keyring file:ceph auth get-or-create client.rbd mon 'profile rbd' osd 'profile rbd pool=<pool_name>' \ -o /etc/ceph/rbd.keyring
Replace
<pool_name>
with the name of the pool that you want to allowclient.rbd
to have access to, for examplerbd
:# ceph auth get-or-create \ client.rbd mon 'allow r' osd 'allow rwx pool=rbd' \ -o /etc/ceph/rbd.keyring
See the User Management section in the Red Hat Ceph Storage 3 Administration Guide for more information about creating users.
Create a block device image:
rbd create <image_name> --size <image_size> --pool <pool_name> \ --name client.rbd --keyring /etc/ceph/rbd.keyring
Specify
<image_name>
,<image_size>
, and<pool_name>
, for example:$ rbd create image1 --size 4096 --pool rbd \ --name client.rbd --keyring /etc/ceph/rbd.keyring
WarningThe default Ceph configuration includes the following Ceph Block Device features:
-
layering
-
exclusive-lock
-
object-map
-
deep-flatten
-
fast-diff
If you use the kernel RBD (
krbd
) client, you will not be able to map the block device image because the current kernel version included in Red Hat Enterprise Linux 7 does not supportobject-map
,deep-flatten
, andfast-diff
.To work around this problem, disable the unsupported features. Use one of the following options to do so:
Disable the unsupported features dynamically:
rbd feature disable <image_name> <feature_name>
For example:
# rbd feature disable image1 object-map deep-flatten fast-diff
-
Use the
--image-feature layering
option with therbd create
command to enable onlylayering
on newly created block device images. Disable the features be default in the Ceph configuration file:
rbd_default_features = 1
This is a known issue, for details see the Known Issues chapter in the Release Notes for Red Hat Ceph Storage 3.
All these features work for users that use the user-space RBD client to access the block device images.
-
Map the newly created image to the block device:
rbd map <image_name> --pool <pool_name>\ --name client.rbd --keyring /etc/ceph/rbd.keyring
For example:
# rbd map image1 --pool rbd --name client.rbd \ --keyring /etc/ceph/rbd.keyring
ImportantKernel block devices currently only support the legacy straw bucket algorithm in the CRUSH map. If you have set the CRUSH tunables to optimal, you must set them to legacy or an earlier major release, otherwise, you will not be able to map the image.
Alternatively, replace
straw2
withstraw
in the CRUSH map. For details, see the Editing a CRUSH Map chapter in the Storage Strategies guide for Red Hat Ceph Storage 3.Use the block device by creating a file system:
mkfs.ext4 /dev/rbd/<pool_name>/<image_name>
Specify the pool name and the image name, for example:
# mkfs.ext4 /dev/rbd/rbd/image1
This can take a few moments.
Mount the newly created file system:
mkdir <mount_directory> mount /dev/rbd/<pool_name>/<image_name> <mount_directory>
For example:
# mkdir /mnt/ceph-block-device # mount /dev/rbd/rbd/image1 /mnt/ceph-block-device
For additional details, see the Block Device Guide for Red Hat Ceph Storage 3.