Chapter 24. OSD migration from FileStore to BlueStore

Procedure

1. Log in as the heat-admin user on a node with Ceph MON containers, such as Controller nodes or standalone Ceph MON nodes. For example, in a standard overcloud deployment, overcloud-controller-1 uses Ceph MON containers.

2. Query the Ceph cluster to see what driver is in use by the OSDs:

   [heat-admin@overcloud-controller-1 ~]$ sudo -i
   [root@overcloud-controller-1 ~]# podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph -f json osd metadata" | jq -c 'sort_by(.hostname) | .[] | ["host", .hostname, "osd_id", .id, "objectstore", .osd_objectstore]'
   [root@overcloud-controller-1 ~]#

3. If any line returns "objectstore": "filestore", that node requires OSD migration.

Procedure

1. Ensure that osd_objectstore in the CephAnsibleDisksConfig parameter does not default to filestore. If the osd_objectstore parameter is present in any of your custom heat environment files, you must define the value bluestore explicitly or remove it:

   parameter_defaults:
     CephAnsibleDisksConfig:
       devices:
         - /dev/sdb
         - /dev/sdc
         - /dev/sdd
       osd_scenario: lvm
      osd_objectstore: bluestore

   Note

   If you have any specific FileStore configuration with, for example, journals, ensure that you update the configuration accordingly. For more information about advanced configurations, see Mapping the Ceph Storage Node Disk Layout in the Deploying an overcloud with containerized Red Hat Ceph guide.

2. Log in to the undercloud as the stack user.

3. Enter the openstack overcloud external-upgrade run command with the ceph_fstobs tag. Replace <NODE_NAME> with the name of the Ceph OSD node you want to upgrade. You can use the openstack server list command to find the node name.

   [stack@undercloud ~] $ openstack overcloud external-upgrade run --tags ceph_fstobs -e ceph_ansible_limit=<NODE_NAME> | tee oc-fstobs.log

4. Log in to a node that has Ceph MON services and query the Ceph cluster to check the status of the OSDs of the node you have upgraded. Before you can start the migration of the next OSD node, you must ensure that the one you have upgraded is successfully migrated:

   [heat-admin@overcloud-controller-1 ~]$ sudo -i
   [root@overcloud-controller-1 ~]# podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph -f json osd metadata" | jq -c '.[] | select(.hostname == "<NODE_NAME>") | ["host", .hostname, "osd_id", .id, "objectstore", .osd_objectstore]'
   [root@overcloud-controller-1 ~]# exit

   Replace <NODE_NAME> with the name of the node that was migrated. If the result shows that the OSDs use BlueStore, its migration is successful.

5. Optional: To view additional details about a specific OSD, enter the following command:

   [root@overcloud-controller-1 ~]# podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph osd metadata <ID>"

   Replace <ID> with the ID of the OSD you want to query.

6. Before you can start the migration process on the next node, you must wait for the cluster to synchronize.

   [root@overcloud-controller-1 ~]# podman exec -it ceph-mon-overcloud-controller-1 sh -c  "ceph  -s"

   Review the command output and ensure that the health of the cluster is `HEALTH_OK` and the PGs are in the `active+clean` state.

7. Before you can start the migration process on the next node, you must wait for the cluster rebalancing process to complete. To follow the status, run the following command:

   [heat-admin@overcloud-controller-0 ~]$ sudo podman exec ceph-mon-<NODE_NAME> ceph -w

   Replace <NODE_NAME> with the name of the node that was migrated.

8. Repeat the migration process for each node in the storage cluster.

Procedure

1. Log in as the heat-admin user to a ceph-mon container that is hosted on one of the Controller nodes.

2. Query the Ceph cluster:

   [heat-admin@overcloud-controller-1 ~]$ sudo -i
   [root@overcloud-controller-1 ~]# podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph -f json osd metadata" | jq -c 'sort_by(.hostname) | .[] | ["host", .hostname, "osd_id", .id, "objectstore", .osd_objectstore]'
   [root@overcloud-controller-1 ~]#