Getting random LVM and DB2 database corruptions on VMware virtual machines

Solution Verified - Updated -

Issue

  • Getting random LVM and DB2 database corruptions on VMware virtual machines
  • The volume group used on virtual machine randomly shows a missing/unknown PV and the volume group is left in partial state

    $ vgs -v
      Couldn't find device with uuid SBJlmh-0i9k-fPZA-dwXU-lEVn-s4f7-upjqxx.
        There are 1 physical volumes missing.
      WARNING: Couldn't find all devices for LV vg1/lv1 while checking used and assumed devices.
      VG   Attr   Ext   #PV #LV #SN VSize   VFree   VG UUID                                VProfile
      rhel wz--n- 4.00m   1   2   0 <19.00g  <3.00g swOw5B-Oaiq-nFBp-8NJ0-f5hD-Rplt-KSNcnr         
      vg1  wz-pn- 4.00m   3   1   0   1.45g 288.00m FMugrN-VPOu-esnO-1D9k-028K-6zyv-1jpF5Y      <--- Partial VG         
    
  • The disk partitions used for LVM became unknown after a vm reboot.

    $ pvs -v
      Couldn't find device with uuid SBJlmh-0i9k-fPZA-dwXU-lEVn-s4f7-upjqxx.
        There are 1 physical volumes missing.
      WARNING: Couldn't find all devices for LV vg1/lv1 while checking used and assumed devices.
      PV                  VG   Fmt  Attr PSize   PFree   DevSize PV UUID                               
      /dev/sdb1  vg1  lvm2 a--  496.00m 288.00m 500.00m WZY6e9-t13m-34Fc-aNbb-xqx1-PnPV-4fVwf5
      /dev/sdd1  vg1  lvm2 a--  496.00m      0  500.00m IfKIik-zYoo-9ypU-lBJj-6DT1-EWgG-L5Sjqj
      /dev/sdf2 rhel lvm2 a--  <19.00g  <3.00g <19.00g wnxmQJ-eShg-eY4J-NM4G-l4EP-WXmq-3C2572
      [unknown]           vg1  lvm2 a-m  496.00m      0       0  SBJlmh-0i9k-fPZA-dwXU-lEVn-s4f7-upjqxx <---
    
  • Each time the corruption occurs, we are able to recover the PV by recreating the partition and restoring the PV using steps in below article, but would like to know Root Cause Analysis fro the same:
    o How do I restore a volume group

Environment

  • Red Hat Enterprise Linux 6, 7, 8
  • VM hosted in VMware ESX

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content