LVM commands hang in cluster after clvmd is killed by oomkiller on Red Hat Enterprise Linux 5

Solution Unverified - Updated -

Issue

  • Our system ran out of memory and oomkiller killed clvmd. Afterwards, LVM commands were hanging and our LVM mirrored LUN's were operating slowly.

Environment

  • Red Hat Enterprise Linux (RHEL) including:
    • Red Hat Enterprise Linux Server 5 (with the High Availability and Resilient Storage Add Ons)
    • Issue was observed on RHEL5.7 with kernel 2.6.18-274.18.1.el5xen, but it is currently unknown if other versions are also affected.
  • Red Hat High Availability cluster with 2 or more nodes
  • Clustered LVM (with clvmd running). Issue has been observed with lvm2-2.02.84-6.el5_7.2, but it is currently unknown if other versions are also affected.
    • cmirror is in use to mirror clustered LVM volumes.
  • System ran out of memory and oomkiller killed clvmd: 
kernel: xend invoked oom-killer: gfp_mask=0x200d2, order=0, oomkilladj=0
kernel: Out of memory: Killed process 24949, UID 0, (clvmd).

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.