Enable LVM2 filtering on OpenStack 16 overcloud nodes - Tech Preview

Updated -

By default the OpenStack 16 overcloud nodes do not use LVM2 volumes but it is possible with some Cinder backends, for example iSCSI or FC, that LVM2 volumes created inside OpenStack guests become visible on the overcloud nodes where the cinder-volume or nova-compute containers are hosted.

The following issues may appear on computes or controllers:

  • LVM appears to see volume groups from my guests
  • LVM is reporting that there are duplicate volume group names
  • Volume detachments fail because LVM is accessing the storage
  • My guests fail to boot due to problems with LVM
  • The LVM on my guest machine is in a partial state due to a missing disk that actually exists
  • Cinder actions fail on devices that have LVM built on them
  • Cinder snapshots fail to remove correctly
  • Error during live-migration with error /etc/multipath.conf does not exist

To prevent this, it is possible to use a feature, initially introduced as Tech Preview in OpenStack Director 16.1z3, which configures an LVM2 filter when deploying or updating the overcloud stack. The feature is, by default, disabled and should be enabled passing LVMFilterEnabled: true in a Heat environment file. When enabled, the LVM2 global_filter will be computed starting from the list of physical devices hosting active LVM2 volumes on the host and this should be sufficient to hide every LVM2 volume but those in use by the host.

$ cat ~/lvm2_filter.yaml
parameter_defaults:
  LVMFilterEnabled: true

Then add the following argument to your openstack overcloud deploy command:

-e lvm2_filter.yaml

After the deployment, to confirm the filter being set up, log any target overcloud node, for example a Compute node that had a single LVM NVMe root volume when using the above snippet the end result would be:

# grep global_filter  /etc/lvm/lvm.conf
global_filter = [ "a|/dev/nvme0n1|","r|.*|" ]

If the computed list is not sufficient, the filter listing can be manually tuned via LVMFilterAllowlist and LVMFilterDenylist parameters, which also support regexp. The deny list defaults to [‘.*’] so that any block device which isn’t in the allow list will be ignored and not scanned by the LVM2 tools.

The template parameters can also be set per-role; for example, to enable the feature only on "Compute" nodes and allow access to in use LVM2 volumes and any device name starting with /dev/nvme, use the following:

$ cat ~/lvm2_filter.yaml
parameter_defaults:
  ComputeParameters:
    LVMFilterEnabled: true
    LVMFilterAllowlist:
      - /dev/nvme.*

Then add the following argument to your openstack overcloud deploy command:

-e lvm2_filter.yaml

After the deployment, to confirm the filter being set up, log any target overcloud node, for example a Compute node when using the above snippet and use the following:

# grep global_filter  /etc/lvm/lvm.conf
global_filter = [ "a|/dev/nvme.*|","r|.*|" ]

Beware that this specific filter would be problematic if your OpenStack deployment uses a Cinder backend driver using NVMe-oF transport protocol, as those volumes would appear as /dev/nvme* and LVM would scan them.

Comments