Red Hat Training

A Red Hat training course is available for RHEL 8

Chapter 20. Removing storage devices

You can safely remove a storage device from a running system, which helps prevent system memory overload and data loss.

Note

Before you remove a storage device, you must ensure that you have enough free system memory due to the increased system memory load during an I/O flush. Use the vmstat 1 100 command to view the system’s current memory load. You can also use the free command to view the system’s free memory. Red Hat does not recommend removing a storage device on a system where:

  • Free memory is less than 5% of the total memory in more than 10 samples per 100.
  • Swapping is active (non-zero si and so columns in the vmstat command output).

20.1. Safe removal of storage devices

Safely removing a storage device from a running system requires a top-to-bottom approach, starting from the top layer, which typically is an application or a filesystem, and working towards the bottom layer, which is the physical device.

You can use storage devices in multiple ways, and they can have different virtual configurations on top of physical devices. For example, you can configure a virtual device as a multipath device, make it part of a RAID, or you can make it part of an LVM group. Alternatively, devices can be accessed via a filesystem, or they can be accessed directly (as a “raw” device).

Working from the top layer down, you must ensure that the device that you want to remove is not in use, all pending I/O to the device is flushed, and the operating system is not referencing the storage device.

20.2. Removing a block device

You can safely remove a block device from a running system to help prevent system memory overload and data loss.

Warning

Rescanning the SCSI bus or performing any other action that changes the state of the operating system’s state without following the procedure documented here can cause delays due to I/O timeouts, devices to be removed unexpectedly, or data loss.

Prerequisites

  • If you want to remove a multipath device, and you are unable to access its path devices, disable the multipath device’s queue:

    # multipathd disablequeueing map <multipath-device>

    This enables the device’s I/O to fail, allowing the applications that are using the device to shut down.

Procedure

  1. Ensure that no other applications or services are using the device that you want to remove.
  2. Back up the data from the device that you want to remove.
  3. Unmount any filesystems that are mounted on the device using the umount command.
  4. Remove the device from any md RAID array or from any LVM volume that it belongs to. For example, if the device is a member of an LVM group, and it is a multipath device:

    1. Move the data to another device:

      # pvmove -b /dev/mapper/<from-multipath-device> /dev/mapper/<to-multipath-device>
    2. Remove the device from the volume group:

      # vgreduce <volume-group> /dev/mapper/<from-multipath-device>
    3. (Optional) Remove the LVM metadata from the physical device:

      # pvremove /dev/mapper/<from-multipath-device>
  5. If you are removing a multipath device:

    1. View all the paths to the device using the following command:

      # multipath -l

      The output of this command is required in a later step.

    2. Flush the I/O and remove the multipath device:

      # multipath -f <multipath-device>
  6. If the device is not configured as a multipath device, or if the device is configured as a multipath device and you have previously passed I/O to its individual paths, flush any outstanding I/O to all device paths that are used:

    # blockdev --flushbufs <device>

    This is important for devices accessed directly where the umount or vgreduce commands do not flush the I/O.

  7. If you are removing a SCSI device, complete the following steps:

    1. Remove any reference to the device’s path-based name; for example, /dev/sd, /dev/disk/by-path or the major:minor number, in applications, scripts, or utilities on the system. This ensures that different devices added in the future are not mistaken for the current device.
    2. Remove each path to the device from the SCSI subsystem:

      # echo 1 > /sys/block/<device-name>/device/delete

      where <device-name> is retrieved from the output of the multipath -l command in step 5 if the device was previously used as a multipath device.

  8. Remove the physical device form a running system. Note that the I/O to other devices does not stop when you remove this device.

Additional resources

  • The multipath, pvmove, vgreduce, blockdev, and umount man pages.