11.4. Removing an OSD from a Ceph Storage node

This procedure demonstrates how to remove an OSD from a node. It assumes the following about the environment:

  • A server (ceph-storage0) has an OSD (ceph-osd@4) running on /dev/sde.
  • The Ceph monitor service (ceph-mon) is running on controller0.
  • There are enough available OSDs to ensure the storage cluster is not at its near-full ratio.

For more information about Ceph OSDs, see Ceph OSDs in the Red Hat Ceph Storage Operations Guide.

Procedure

  1. SSH into ceph-storage0 and log in as root.
  2. Disable and stop the OSD service:

    [root@ceph-storage0 ~]# systemctl disable ceph-osd@4
    [root@ceph-stoarge0 ~]# systemctl stop ceph-osd@4
  3. Disconnect from ceph-storage0.
  4. SSH into controller0 and log in as root.
  5. Identify the name of the Ceph monitor container:

    [root@controller0 ~]# podman ps | grep ceph-mon
    ceph-mon-controller0
    [root@controller0 ~]#
  6. Enable the Ceph monitor container to mark the undesired OSD as out:

    [root@controller0 ~]# podman exec ceph-mon-controller0 ceph osd out 4
    注記

    This command causes Ceph to rebalance the storage cluster and copy data to other OSDs in the cluster. The cluster temporarily leaves the active+clean state until rebalancing is complete.

  7. Run the following command and wait for the storage cluster state to become active+clean:

    [root@controller0 ~]# podman exec ceph-mon-controller0 ceph -w
  8. Remove the OSD from the CRUSH map so that it no longer receives data:

    [root@controller0 ~]# podman exec ceph-mon-controller0 ceph osd crush remove osd.4
  9. Remove the OSD authentication key:

    [root@controller0 ~]# podman exec ceph-mon-controller0 ceph auth del osd.4
  10. Remove the OSD:

    [root@controller0 ~]# podman exec ceph-mon-controller0 ceph osd rm 4
  11. Disconnect from controller0.
  12. SSH into the undercloud as the stack user and locate the heat environment file in which you defined the CephAnsibleDisksConfig parameter.
  13. Notice the heat template contains four OSDs:

    parameter_defaults:
      CephAnsibleDisksConfig:
        devices:
          - /dev/sdb
          - /dev/sdc
          - /dev/sdd
          - /dev/sde
        osd_scenario: lvm
        osd_objectstore: bluestore
  14. Modify the template to remove /dev/sde.

    parameter_defaults:
      CephAnsibleDisksConfig:
        devices:
          - /dev/sdb
          - /dev/sdc
          - /dev/sdd
        osd_scenario: lvm
        osd_objectstore: bluestore
  15. Run openstack overcloud deploy to update the overcloud.

    注記

    This example assumes that you removed the /dev/sde device from all hosts with OSDs. If you do not remove the same device from all nodes, update the heat template as shown and see ??? for information about how to define hosts with a differing devices list.