System crashed after softlockup traces caused by postgres process.

Solution Unverified - Updated -

Issue

  • System rebooted unexpectedly, RCA required.
  • System crashed after softlockup message:
BUG: soft lockup - CPU#1 stuck for 70s! [postgres:553375]
CPU 1 
..
Pid: 553375, comm: postgres Not tainted 2.6.32-504.8.1.el6.x86_64 #1 ...
RIP: 0010:[<ffffffff8152ce91>]  [<ffffffff8152ce91>] _spin_lock+0x21/0x30
..
Process postgres (pid: 553375, threadinfo ffff882213882000, task ffff88204025aaa0)
Stack:
 ...
Call Trace:
 [<ffffffff8115a6c2>] ? page_referenced+0xa2/0x360
 [<ffffffff8113e451>] ? shrink_active_list+0x1e1/0x370
 [<ffffffff8113f1d5>] ? shrink_mem_cgroup_zone+0x3c5/0x540
 [<ffffffff8117eecd>] ? mem_cgroup_iter+0xfd/0x280
 [<ffffffff8113f3ca>] ? shrink_zone+0x7a/0x180
 [<ffffffff8113f5e5>] ? do_try_to_free_pages+0x115/0x610
 [<ffffffff81131c8c>] ? get_page_from_freelist+0x15c/0x870
 [<ffffffff8113fcb2>] ? try_to_free_pages+0x92/0x120
 [<ffffffff811340be>] ? __alloc_pages_nodemask+0x47e/0x8d0
 [<ffffffff8116c79a>] ? alloc_pages_vma+0x9a/0x150
 [<ffffffff8114f6fd>] ? handle_pte_fault+0x73d/0xb00
 [<ffffffff810a3da4>] ? hrtimer_start_range_ns+0x14/0x20
 [<ffffffff8114fcea>] ? handle_mm_fault+0x22a/0x300
 [<ffffffff8104d0d8>] ? __do_page_fault+0x138/0x480
 [<ffffffff81448795>] ? sys_sendto+0x185/0x190
 [<ffffffff8152ffde>] ? do_page_fault+0x3e/0xa0
 [<ffffffff8152d395>] ? page_fault+0x25/0x30

Environment

  • Red Hat Enterprise Linux 6
  • postgresql-8.4.20-1.el6_5

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.