While testing addition/deletion of the SAN devices, system crashed with error 'Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 0'
Issue
- While trying to test the addition/deletion of the SAN devices using following steps the system crashed with error 'Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 0':
Steps to Reproduce:
1. Boot on the 6.6 kernel
2. Inject faults on the fabric layer and disable/enable targets ports repeatedly
3. Keep rescanning the SAN devices using following script:
#!/bin/sh -ex
while true; do
date
rescan-scsi-bus.sh -r -a --hosts=7,8
rescan-scsi-bus.sh -a --hosts=7,8
sleep 1
done
3. System panics with following errors:
rport-7:0-3: blocked FC remote port time out: removing target and saving binding
scsi 7:0:5:0: Device offlined - not ready after error recovery
Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 0
Pid: 603, comm: fc_wq_7 Not tainted 2.6.32-504.3.3.el6.x86_64 #1
Call Trace:
<NMI> [<ffffffff81529277>] ? panic+0xa7/0x16f
[<ffffffff81014959>] ? sched_clock+0x9/0x10
[<ffffffff810ea5dd>] ? watchdog_overflow_callback+0xcd/0xd0
[<ffffffff81120d87>] ? __perf_event_overflow+0xa7/0x240
[<ffffffff8101dacd>] ? x86_perf_event_set_period+0xdd/0x170
[<ffffffff811213d4>] ? perf_event_overflow+0x14/0x20
[<ffffffff81024892>] ? intel_pmu_handle_irq+0x202/0x410
[<ffffffff8152e579>] ? perf_event_nmi_handler+0x39/0xb0
[<ffffffff81530035>] ? notifier_call_chain+0x55/0x80
[<ffffffff8153009a>] ? atomic_notifier_call_chain+0x1a/0x20
[<ffffffff810a4eae>] ? notify_die+0x2e/0x30
[<ffffffff8152dcfb>] ? do_nmi+0x1bb/0x340
[<ffffffff8152d5c0>] ? nmi+0x20/0x30
[<ffffffff8138ddf3>] ? scsi_remove_target+0x93/0x210
<<EOE>> [<ffffffffa0235570>] ? fc_starget_delete+0x0/0x30 [scsi_transport_fc]
[<ffffffffa0235596>] ? fc_starget_delete+0x26/0x30 [scsi_transport_fc]
[<ffffffff81097fe0>] ? worker_thread+0x170/0x2a0
[<ffffffff8109eb00>] ? autoremove_wake_function+0x0/0x40
[<ffffffff81097e70>] ? worker_thread+0x0/0x2a0
[<ffffffff8109e66e>] ? kthread+0x9e/0xc0
[<ffffffff8100c20a>] ? child_rip+0xa/0x20
[<ffffffff8109e5d0>] ? kthread+0x0/0xc0
[<ffffffff8100c200>] ? child_rip+0x0/0x20
Environment
- Red Hat Enterprise Linux 6.6
- kernel 2.6.32-504.3.3.el6.x86_64
- kernel-2.6.32-504.8.1.el6.x86_64
- Fiber Channel HBA
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
