NMI Watchdog 在 CPU XX 中检测到硬锁定,位于 _spin_lock_irqsave。
Issue
- 内核 panic,带有以下调用追踪信息。
Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 14
Pid: 35242, comm: oracle Tainted: P --------------- 2.6.32-431.5.1.el6.x86_64 #1
Call Trace:
<NMI> [<ffffffff81527513>] ? panic+0xa7/0x16f
[<ffffffff810153a3>] ? native_sched_clock+0x13/0x80
[<ffffffff810e696d>] ? watchdog_overflow_callback+0xcd/0xd0
[<ffffffff8111c847>] ? __perf_event_overflow+0xa7/0x240
[<ffffffff8101d93d>] ? x86_perf_event_set_period+0xdd/0x170
[<ffffffff8111ce14>] ? perf_event_overflow+0x14/0x20
[<ffffffff81022d87>] ? intel_pmu_handle_irq+0x187/0x2f0
[<ffffffff8152d1f6>] ? kprobe_exceptions_notify+0x16/0x430
[<ffffffff8152bd69>] ? perf_event_nmi_handler+0x39/0xb0
[<ffffffff8152d825>] ? notifier_call_chain+0x55/0x80
[<ffffffff8152d88a>] ? atomic_notifier_call_chain+0x1a/0x20
[<ffffffff810a153e>] ? notify_die+0x2e/0x30
[<ffffffff8152b4eb>] ? do_nmi+0x1bb/0x340
[<ffffffff8152adb0>] ? nmi+0x20/0x30
[<ffffffff8152a50d>] ? _spin_lock_irqsave+0x2d/0x40
<<EOE>> <IRQ> [<ffffffffa02e66df>] ? PowerPlatformTopIodone+0x17f/0x220 [emcp]
[<ffffffffa02ee199>] ? PowerTopIodone+0x29/0x170 [emcp]
[<ffffffffa02ee337>] ? PowerProcessTopIodonePirps+0x57/0xd0 [emcp]
[<ffffffffa02ee6f0>] ? PowerBottomIodoneNew+0x230/0x550 [emcp]
[<ffffffffa02eebe7>] ? PowerPlatformBottomIodone+0xd7/0x140 [emcp]
[<ffffffff812658d7>] ? __blk_run_queue+0x27/0x40
[<ffffffff811c22bd>] ? bio_endio+0x1d/0x40
[<ffffffff81264ebb>] ? req_bio_endio+0x9b/0xe0
[<ffffffff81266437>] ? blk_update_request+0x117/0x490
[<ffffffff812667d7>] ? blk_update_bidi_request+0x27/0xa0
[<ffffffff8127d5c5>] ? __cfq_slice_expired+0x1e5/0x530
[<ffffffff812688cf>] ? blk_end_bidi_request+0x2f/0x80
[<ffffffff81268970>] ? blk_end_request+0x10/0x20
[<ffffffff81389acf>] ? scsi_io_completion+0xaf/0x6c0
[<ffffffff8137ff52>] ? scsi_finish_command+0xc2/0x130
[<ffffffff8138a245>] ? scsi_softirq_done+0x145/0x170
[<ffffffff8126e855>] ? blk_done_softirq+0x85/0xa0
[<ffffffff8127bce0>] ? cfq_kick_queue+0x0/0x60
[<ffffffff8107a8e1>] ? __do_softirq+0xc1/0x1e0
[<ffffffff8100c30c>] ? call_softirq+0x1c/0x30
<EOI> [<ffffffff8100fa75>] ? do_softirq+0x65/0xa0
[<ffffffff8107b5da>] ? local_bh_enable+0x9a/0xb0
[<ffffffff8149123d>] ? __ip_route_output_key+0x17d/0x1b0
[<ffffffff814912a6>] ? ip_route_output_flow+0x36/0x2c0
[<ffffffff81498a00>] ? ip_generic_getfrag+0x0/0xb0
[<ffffffff814bd80a>] ? udp_sendmsg+0x78a/0x930
[<ffffffff814c5eba>] ? inet_sendmsg+0x4a/0xb0
[<ffffffff8122ed0f>] ? selinux_socket_sendmsg+0x1f/0x30
[<ffffffff81448003>] ? sock_sendmsg+0x123/0x150
[<ffffffff81449cb3>] ? sock_recvmsg+0x133/0x160
[<ffffffff8109b290>] ? autoremove_wake_function+0x0/0x40
[<ffffffff8119fec0>] ? pollwake+0x0/0x60
[<ffffffff8119fec0>] ? pollwake+0x0/0x60
[<ffffffff81447e54>] ? move_addr_to_kernel+0x64/0x70
[<ffffffff814497f6>] ? __sys_sendmsg+0x406/0x420
[<ffffffff810129de>] ? copy_user_generic+0xe/0x20
[<ffffffff810179ab>] ? check_for_xstate+0x3b/0x90
[<ffffffff8100b9ce>] ? common_interrupt+0xe/0x13
[<ffffffff810149b9>] ? read_tsc+0x9/0x20
[<ffffffff810a7091>] ? ktime_get_ts+0xb1/0xf0
[<ffffffff81449a19>] ? sys_sendmsg+0x49/0x90
[<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
Environment
- Red Hat Enterprise Linux 6
- Kernel-2.6.32-431.5.1.el6
- EMCpower.LINUX-5.7.1.00.00-029.el6
- EMC Unsigned (U) module "emcp"
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.