How to create Ceph lvm in Container Environment

Solution In Progress - Updated -

Environment

  • RedHat Ceph Storage 4 (Containerized Deployment)

Issue

  • ceph-volume lvm create command is failed in container deployment.

Resolution

It is needed to execute ceph-volume lvm created with the --no-systemd option as below.

[root@osds-0 ~]# CEPH_VOLUME_DEBUG=true ceph-volume lvm create --data /dev/vdb  --no-systemd
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new xxx-xxx-xxx-xxx
...
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap
...
--> ceph-volume lvm prepare successful for: /dev/vdb
...
--> ceph-volume lvm activate successful for osd ID: 0
--> ceph-volume lvm create successful for: /dev/vdb

Root Cause

When deploying containarized Ceph, it is not necessary to create, enable and start any systemd unit.
It is mandatory to pass the --no-systemd flag though, when executing the ceph-volume lvm create command.

Diagnostic Steps

The following error can be observed when replacing an OSD drive while retaining the OSD ID as per the steps described in Replacing an OSD drive while retaining the OSD ID.

[root@osds-0 ~]# CEPH_VOLUME_DEBUG=true ceph-volume lvm create --data /dev/vdb
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new xxx-xxx-xxx-xxx
...
 stderr: Failed to enable unit: Unit file ceph-volume@lvm-0-xxx-xxx-xxx-xxx.service does not exist.
--> Was unable to complete a new OSD, will rollback changes
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.0 --yes-i-really-mean-it
 stderr: purged osd.0
...
process.run(['systemctl', 'enable', unit])
File "/usr/lib/python3.6/site-packages/ceph_volume/process.py", line 153, in run
raise RuntimeError(msg)
RuntimeError: command returned non-zero exit status: 1
[root@mons-0 ~]

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments