Appendix D. Manually Installing Ceph Block Device
The following procedure shows how to install and mount a thin-provisioned, resizable Ceph Block Device.
Ceph Block Devices must be deployed on separate nodes from the Ceph Monitor and OSD nodes. Running kernel clients and kernel server daemons on the same node can lead to kernel deadlocks.
- Ensure to perform the tasks listed in the Appendix C, Installing the Ceph Command Line Interface section.
- If you use Ceph Block Devices as a back end for virtual machines (VMs) that use QEMU, increase the default file descriptor. See the Ceph - VM hangs when transferring large amounts of data to RBD disk Knowledgebase article for details.
Create a Ceph Block Device user named
client.rbdwith full permissions to files on OSD nodes (
osd 'allow rwx') and output the result to a keyring file:
ceph auth get-or-create client.rbd mon 'profile rbd' osd 'profile rbd pool=<pool_name>' \ -o /etc/ceph/rbd.keyring
<pool_name>with the name of the pool that you want to allow
client.rbdto have access to, for example
$ sudo ceph auth get-or-create \ client.rbd mon 'allow r' osd 'allow rwx pool=rbd' \ -o /etc/ceph/rbd.keyring
See the User Management section in the Red Hat Ceph Storage 3 Administration Guide for more information about creating users.
Create a block device image:
rbd create <image_name> --size <image_size> --pool <pool_name> \ --name client.rbd --keyring /etc/ceph/rbd.keyring
<pool_name>, for example:
$ rbd create image1 --size 4096 --pool rbd \ --name client.rbd --keyring /etc/ceph/rbd.keyringWarning
The default Ceph configuration includes the following Ceph Block Device features:
If you use the kernel RBD (
krbd) client, you will not be able to map the block device image because the current kernel version included in Red Hat Enterprise Linux 7.3 does not support
To work around this problem, disable the unsupported features. Use one of the following options to do so:
Disable the unsupported features dynamically:
rbd feature disable <image_name> <feature_name>
# rbd feature disable image1 object-map deep-flatten fast-diff
--image-feature layeringoption with the
rbd createcommand to enable only
layeringon newly created block device images.
Disable the features be default in the Ceph configuration file:
rbd_default_features = 1
This is a known issue, for details see the Known Issues chapter in the Release Notes for Red Hat Ceph Storage 3.
All these features work for users that use the user-space RBD client to access the block device images.
Map the newly created image to the block device:
rbd map <image_name> --pool <pool_name>\ --name client.rbd --keyring /etc/ceph/rbd.keyring
$ sudo rbd map image1 --pool rbd --name client.rbd \ --keyring /etc/ceph/rbd.keyring
Use the block device by creating a file system:
mkfs.ext4 -m5 /dev/rbd/<pool_name>/<image_name>
Specify the pool name and the image name, for example:
$ sudo mkfs.ext4 -m5 /dev/rbd/rbd/image1
This can take a few moments.
Mount the newly created file system:
mkdir <mount_directory> mount /dev/rbd/<pool_name>/<image_name> <mount_directory>
$ sudo mkdir /mnt/ceph-block-device $ sudo mount /dev/rbd/rbd/image1 /mnt/ceph-block-device
For additional details, see the Block Device Guide for Red Hat Ceph Storage 3.