OS hanged (BUG: soft lockup - CPU#0 stuck for 60s! [httpd:xxxx]).

Solution Unverified - Updated -

Issue

  • OS was hung, this occurred several times.
  • Checking a vmcore which was collected by push of NMI switch, it was seen that it was in a state of spin lock during httpd processes as follows.
crash> log
        : <snip>
BUG: soft lockup - CPU#0 stuck for 60s! [httpd:26249]

Pid: 26249, comm:                httpd
EIP: 0060:[<c0620ac3>] CPU: 0
EIP is at _spin_lock+0xa/0xf
 EFLAGS: 00000282    Tainted: P       (2.6.18-238.el5 #1)
EAX: c0706880 EBX: db804d50 ECX: c9ffd960 EDX: 00000001
ESI: c0706880 EDI: f75ae600 EBP: f2ab6568 DS: 007b ES: 007b
CR0: 8005003b CR2: b7fa2000 CR3: 123d5000 CR4: 000006d0
 [<c04ee6d2>] _atomic_dec_and_lock+0x2a/0x44
 [<c048c87b>] dput+0x22/0xed
 [<c0477e2f>] __fput+0x15c/0x184
 [<c0475751>] filp_close+0x4e/0x54
 [<c047698a>] sys_close+0x71/0xa0
 [<c0404f4b>] syscall_call+0x7/0xb
 =======================
BUG: soft lockup - CPU#1 stuck for 60s! [httpd:26183]

Pid: 26183, comm:                httpd
EIP: 0060:[<c0620ac3>] CPU: 1
EIP is at _spin_lock+0xa/0xf
 EFLAGS: 00000282    Tainted: P       (2.6.18-238.el5 #1)
EAX: c0706880 EBX: f68468f0 ECX: c0487f8d EDX: d3231f98
ESI: d3443e40 EDI: f4580884 EBP: f45808f8 DS: 007b ES: 007b
CR0: 8005003b CR2: 091f21ac CR3: 1b945000 CR4: 000006d0
 [<f938a51c>] rfs_readdir+0x38/0x1b4 [redirfs]
 [<c0487f8d>] filldir+0x0/0xb9
 [<c0475a55>] do_filp_open+0x2b/0x31
 [<c04880a9>] vfs_readdir+0x63/0x8d
 [<c0487f8d>] filldir+0x0/0xb9
 [<c04882f7>] sys_getdents+0x5f/0x9c
 [<c0404f4b>] syscall_call+0x7/0xb
 =======================
        : <snip>

Environment

  • Red Hat Enterprise Linux 5.6
  • kernel-2.6.18-238.el5

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.