LVM2 commands return Segmentation fault after "WARNING: Inconsistent metadata found" message on Red Hat Enterprise Linux

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 5
  • lvm2 package prior to lvm2-2.02.46-8.el5
  • Inconsistent metadata reported when running various LVM commands

Issue

  • Why lvm commands return "Segmentation fault" ?

Resolution

  • The following errata advisory addresses several segfault issues in lvm: https://access.redhat.com/errata/RHBA-2009:1393
  • Install the latest versions of lvm2 (lvm2-2.02.46-8 or later), as well as the latest versions of device-mapper and device-mapper-event.

Root Cause

Diagnostic Steps

  • syslog show the following after running lvm commands:
    Apr 26 11:47:27 localhost kernel: lvs[15909]: segfault at 0000000000000000 rip 0000000000000000 rsp 00007fff8c65c048 error 14
Apr 26 11:49:39 localhost kernel: vgdisplay[16103]: segfault at0000000000000000 rip 0000000000000000 rsp 00007fffb6830258 error 14
Apr 26 11:49:39 localhost kernel: vgscan[16104]: segfault at 0000000000000000 rip 0000000000000000 rsp 00007fff064dcf18 error 14
Apr 26 11:49:40 localhost kernel: pvscan[16105]: segfault at 0000000000000000 rip 0000000000000000 rsp 00007fff685f4d08 error 14
Apr 26 11:49:40 localhost kernel: lvs[16106]: segfault at 0000000000000000 rip 0000000000000000 rsp 00007ffff15b3fd8 error 14
Apr 26 11:49:40 localhost kernel: pvs[16107]: segfault at 0000000000000000 rip 0000000000000000 rsp 00007fff80a0a438 error 14
Apr 26 11:49:40 localhost kernel: vgs[16108]: segfault at 0000000000000000 rip 0000000000000000 rsp 00007fffe3289cc8 error 14
  • When running LVM2 commands, you may see output similar to:

    # vgscan
  Reading all physical volumes.  This may take a while...
  Volume group "vg_test" inconsistent
  WARNING: Inconsistent metadata found for VG vg_test - updating to use version 3
  Removing PV /dev/mapper/x (1nxHIW-2Thk-9mS4-2TzO-zHlB-Hsy2-CZsjE7) that no longer belongs to VG vg_test
  Found volume group "vg_test" using metadata type lvm2
Segmentation fault
  • Or messages similar to:
    # vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "vg_rhel4" using metadata type lvm2 
  Found volume group "myvg" using metadata type lvm2     
  Volume group "vg_test" inconsistent                    
  WARNING: Inconsistent metadata found for VG vg_test - updating to use version 4
  Removing PV /dev/mapper/x (T5Fj1R-fvs6-9UZG-oaix-Fmtz-WybW-bhyJ37) that no longer belongs to VG vg_test                                                    
  Found volume group "vg_test" using metadata type lvm2                         
*** glibc detected *** vgscan: double free or corruption (out): 0x006d54c0 ***  
======= Backtrace: =========                                                    
/lib/libc.so.6[0x5fd0f1]                                                        
/lib/libc.so.6(cfree+0x90)[0x600bc0]                                            
/lib/libdevmapper.so.1.02(dm_pool_destroy+0x22)[0x4a6da2]                       
vgscan(vg_release+0x34)[0x80a1624]                                              
vgscan[0x806bc1b]                                                               
vgscan(process_each_vg+0x12c)[0x806c83c]                                        
vgscan(vgscan+0x12c)[0x807389c]                                                 
vgscan(lvm_run_command+0xf18)[0x805f3d8]                                        
vgscan(lvm2_main+0x4ca)[0x805fa2a]                                              
vgscan(main+0x32)[0x8075422]                                                    
/lib/libc.so.6(__libc_start_main+0xdc [0x5a9e8c]                                
vgscan[0x80528c1]                                                               
...

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments