Memory Leak observed when using sbd watchdog type stonith

Solution Verified - Updated -

Issue

  • While using sbd watchdog type fencing, the sbd: watcher: Pacemaker thread grows steadily over time, and is never decreased until cluster stop or system reboot:

    Mon Jun 17 12:01:01 PM EEST 2024 VVV
    USER         PID    PPID %CPU %MEM    VSZ   RSS TT       STAT  STARTED     TIME WCHAN                            COMMAND
    root         914       1  0.0  0.2  20808 20456 ?        SL     Jun 14 00:01:13 do_sigtimedwait                  sbd: inquisitor
    root         915     914  0.2  5.5 450088 449472 ?       SL     Jun 14 00:07:50 do_poll.constprop.0              sbd: watcher: Pacemaker <<<===
    root         916     914  0.0  0.5  43288 42884 ?        SL     Jun 14 00:00:50 do_poll.constprop.0              sbd: watcher: Cluster
    
    Mon Jun 17 12:01:21 PM EEST 2024
    USER         PID    PPID %CPU %MEM    VSZ   RSS TT       STAT  STARTED     TIME WCHAN                            COMMAND
    root         914       1  0.0  0.2  20808 20456 ?        SL     Jun 14 00:01:13 do_sigtimedwait                  sbd: inquisitor
    root         915     914  0.2  5.5 450236 449728 ?       SL     Jun 14 00:07:50 do_poll.constprop.0              sbd: watcher: Pacemaker  <<<===
    root         916     914  0.0  0.5  43288 42884 ?        SL     Jun 14 00:00:50 do_poll.constprop.0              sbd: watcher: Cluster
    
  • The rate of growth is very slow, and usually observed at about 128 kb every ~95s ( or ~120 Mb ) a day. If issue persists too long though, this can lead to low memory conditions and memory pressure, and even out of memory ( "OOM" ) events.

Environment

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content