System panics with very short scsi timeout and/or high level of console logging
Issue
- System panics with very short scsi timeout and/or high level of console logging
BUG: unable to handle kernel NULL pointer dereference at 00000000000000b0
IP: [<ffffffff8127834f>] blk_done_softirq+0x7f/0xa0
PGD dfd338b067 PUD cd80ed6067 PMD 0
Oops: 0000 [#1] SMP
last sysfs file: /sys/devices/pci0000:00/0000:00:07.0/0000:0b:00.1/host6/rport-6:0-4/target6:0:2/6:0:2:83/state
CPU 50
Modules linked in: hangcheck_timer oracleacfs(P)(U) oracleadvm(P)(U) oracleoks(P)(U) krg_11_0_0_1130_impRHEL6K1smp-x86_64(P)(U) mptctl mptbase nfsd exportfs oracleasm(U) nfs lockd fscache auth_rpcgss nfs_acl sunrpc bonding 8021q garp stp llc ipv6 ext3 jbd dm_round_robin iTCO_wdt iTCO_vendor_support be2net ixgbe dca mdio e1000e ptp pps_core microcode ipmi_devintf serio_raw lpc_ich mfd_core hpilo hpwdt i7core_edac edac_core sg power_meter acpi_ipmi ipmi_si ipmi_msghandler bnx2 shpchp ext4 jbd2 mbcache sr_mod cdrom sd_mod lpfc scsi_transport_fc scsi_tgt crc_t10dif pata_acpi ata_generic ata_piix hpsa radeon ttm drm_kms_helper drm i2c_algo_bit i2c_core dm_multipath dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
Pid: 205, comm: ksoftirqd/50 Tainted: P --------------- 2.6.32-504.1.3.el6.x86_64 #1 HP ProLiant DL980 G7
RIP: 0010:[<ffffffff8127834f>] [<ffffffff8127834f>] blk_done_softirq+0x7f/0xa0
RSP: 0018:ffff88a070c03f18 EFLAGS: 00010216
RAX: 0000000000000000 RBX: ffff88a070c03f18 RCX: ffff88adba447490
RDX: ffff88a070c03f18 RSI: ffff881fd299bbc0 RDI: ffff88a070c137c0
RBP: ffff88a070c03f38 R08: 0000000000000000 R09: 0000000000000000
R10: ffff881fd1a74400 R11: 00000000055b9b1f R12: ffffffff81a830a0
R13: 0000000000000020 R14: 0000000000000100 R15: 0000000000000004
FS: 0000000000000000(0000) GS:ffff88a070c00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00000000000000b0 CR3: 000000cd920a0000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process ksoftirqd/50 (pid: 205, threadinfo ffff881fd3a42000, task ffff881fd3a3eaa0)
Stack:
ffff88adba447490 ffff88ae8b7d14d0 0000000000000100 0000000000000029
<d> ffff88a070c03fa8 ffffffff8107d8b1 0000000000000032 ffff88a070c03f68
<d> 0000003200000005 ffff881fd3a43fd8 ffff881fd3a43fd8 0000000000011440
Call Trace:
<IRQ>
[<ffffffff8107d8b1>] __do_softirq+0xc1/0x1e0
[<ffffffff8100c30c>] call_softirq+0x1c/0x30
<EOI>
[<ffffffff8100fc15>] ? do_softirq+0x65/0xa0
[<ffffffff8107d470>] ksoftirqd+0x80/0x110
[<ffffffff8107d3f0>] ? ksoftirqd+0x0/0x110
[<ffffffff8109e66e>] kthread+0x9e/0xc0
[<ffffffff8100c20a>] child_rip+0xa/0x20
[<ffffffff8109e5d0>] ? kthread+0x0/0xc0
[<ffffffff8100c200>] ? child_rip+0x0/0x20
Code: e0 48 39 d8 74 34 66 0f 1f 44 00 00 48 8d 78 f0 48 8b 4f 10 48 8b 57 18 48 89 51 08 48 89 0a 48 89 47 10 48 89 47 18 48 8b 47 38 <ff> 90 b0 00 00 00 48 8b 45 e0 48 39 d8 75 d2 48 83 c4 18 5b c9
RIP [<ffffffff8127834f>] blk_done_softirq+0x7f/0xa0
RSP <ffff88a070c03f18>
CR2: 00000000000000b0
Environment
- Red Hat Enterprise Linux 6
- Short scsi timeout set (4 seconds in this case)
- Great deal of logging to the console (not required but exacerbates issue)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
