Kernel Panic: unable to handle kernel paging request
I'm running RHEL6.8, kernel 2.6.32-642.3.1.el6.x86_64 on VMWare. Over the weekend, one of my production servers restarted due to a kernel panic. I'm not particularly in analyzing core dump files, so I was hoping someone could point me in a useful direction based on what information I have gathered.
Here is the kernel panic message:
BUG: unable to handle kernel paging request at ffff88082bf4e030 IP: [] 0xffff88082bf4e030 PGD 1a8e063 PUD 2e067 PMD 800000082be001e3 Oops: 0011 [#1] SMP last sysfs file: /sys/devices/system/node/node0/meminfo CPU 1 Modules linked in: nfsd exportfs autofs4 nfs lockd fscache auth_rpcgss nfs_acl sunrpc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 microc ode vmware_balloon sg i2c_piix4 shpchp ext4 jbd2 mbcache sd_mod crc_t10dif sr_mod cdrom vmxnet3 vmw_pvscsi pata_acpi ata_generic ata_piix vmwgfx ttm drm_kms_helper drm i2c_core dm_mirror dm_r egion_hash dm_log dm_mod [last unloaded: speedstep_lib] Pid: 0, comm: swapper Not tainted 2.6.32-642.3.1.el6.x86_64 #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform RIP: 0010:[ ] [ ] 0xffff88082bf4e030 RSP: 0018:ffff880028223c68 EFLAGS: 00010097 RAX: ffff88082bf4e000 RBX: 000003e8fde218f7 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 00000000ffffffff RDI: ffff88082bf4e000 RBP: ffff880028223c70 R08: 0000000000000001 R09: 0000000000000001 R10: 0000000000000000 R11: 0000000000000000 R12: 000000002a7d4f68 R13: ffff88082b3cdb00 R14: 0000000000000000 R15: 0000000000000000 FS: 0000000000000000(0000) GS:ffff880028220000(0000) knlGS:0000000000000000 CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b CR2: ffff88082bf4e030 CR3: 00000001a2521000 CR4: 00000000000407e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper (pid: 0, threadinfo ffff880834aac000, task ffff880834aa9520) Stack: ffffffff8105eb69 ffff880028223c80 ffffffff810a077f ffff880028223c90 ffffffff8127bbc5 ffff880028223cf0 ffffffff812965c1 ffffffff81b0ecc0 ffffffff81b0ece8 ffffffff81b0ecc0 0000000000000206 ffff880028223d00 Call Trace: [ ] ? __wake_up_common+0x59/0x90 [ ] queue_work+0x1f/0x30 [ ] kblockd_schedule_work+0x15/0x20 [ ] cfq_completed_request+0x201/0x7e0 [ ] elv_completed_request+0x60/0x70 [ ] __blk_put_request+0x3c/0xd0 [ ] blk_finish_request+0xd9/0x260 [ ] blk_end_bidi_request+0x54/0x80 [ ] blk_end_request+0x10/0x20 [ ] scsi_io_completion+0xc0/0x6c0 [ ] ? scheduler_tick+0xcc/0x260 [ ] scsi_finish_command+0xcd/0x130 [ ] scsi_softirq_done+0x147/0x170 [ ] blk_done_softirq+0x85/0xa0 [ ] __do_softirq+0xe5/0x230 [ ] ? update_ts_time_stats+0x67/0xa0 [ ] call_softirq+0x1c/0x30 [ ] do_softirq+0x65/0xa0 [ ] irq_exit+0x85/0x90 [ ] smp_call_function_single_interrupt+0x35/0x40 [ ] call_function_single_interrupt+0x13/0x20 [ ] ? native_safe_halt+0xb/0x10 [ ] default_idle+0x4d/0xb0 [ ] cpu_idle+0xb6/0x110 [ ] start_secondary+0x2c0/0x316 Code: 88 ff ff 30 e0 f4 2b 08 88 ff ff 30 e0 f4 2b 08 88 ff ff 0f 19 e2 fd e8 03 00 00 00 d0 5f 1f 04 00 00 00 30 e0 f4 2b 08 88 ff ff 01> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 RIP [ ] 0xffff88082bf4e030 RSP CR2: ffff88082bf4e030
Backtrace
PID: 0 TASK: ffff880834aa9520 CPU: 1 COMMAND: "swapper" #0 [ffff880028223850] machine_kexec at ffffffff8103fdcb #1 [ffff8800282238b0] crash_kexec at ffffffff810d1d62 #2 [ffff880028223980] oops_end at ffffffff8154ba80 #3 [ffff8800282239b0] no_context at ffffffff810518cb #4 [ffff880028223a00] __bad_area_nosemaphore at ffffffff81051b55 #5 [ffff880028223a50] bad_area_nosemaphore at ffffffff81051c23 #6 [ffff880028223a60] __do_page_fault at ffffffff810523c0 #7 [ffff880028223b80] do_page_fault at ffffffff8154da0e #8 [ffff880028223bb0] page_fault at ffffffff8154ad15 [exception RIP: unknown or invalid address] RIP: ffff88082bf4e030 RSP: ffff880028223c68 RFLAGS: 00010097 RAX: ffff88082bf4e000 RBX: 000003e8fde218f7 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 00000000ffffffff RDI: ffff88082bf4e000 RBP: ffff880028223c70 R8: 0000000000000001 R9: 0000000000000001 R10: 0000000000000000 R11: 0000000000000000 R12: 000000002a7d4f68 R13: ffff88082b3cdb00 R14: 0000000000000000 R15: 0000000000000000 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #9 [ffff880028223c68] __wake_up_common at ffffffff8105eb69 #10 [ffff880028223cf8] elv_completed_request at ffffffff81276f20 #11 [ffff880028223d08] __blk_put_request at ffffffff8127e6dc #12 [ffff880028223d38] blk_finish_request at ffffffff8127e849 #13 [ffff880028223d88] blk_end_bidi_request at ffffffff8127edc4 #14 [ffff880028223db8] blk_end_request at ffffffff8127ee40 #15 [ffff880028223dc8] scsi_io_completion at ffffffff813a0800 #16 [ffff880028223e48] scsi_finish_command at ffffffff81396c2d #17 [ffff880028223e78] scsi_softirq_done at ffffffff813a0f67 #18 [ffff880028223ea8] blk_done_softirq at ffffffff81285945 #19 [ffff880028223ed8] __do_softirq at ffffffff81085215 #20 [ffff880028223f58] call_softirq at ffffffff8100c38c #21 [ffff880028223f70] do_softirq at ffffffff8100fca5 #22 [ffff880028223f90] irq_exit at ffffffff810850a5 #23 [ffff880028223fa0] smp_call_function_single_interrupt at ffffffff81037655 #24 [ffff880028223fb0] call_function_single_interrupt at ffffffff8100be33 ------ #25 [ffff880834aafe28] call_function_single_interrupt at ffffffff8100be33 [exception RIP: native_safe_halt+11] RIP: ffffffff8104601b RSP: ffff880834aafed8 RFLAGS: 00000246 RAX: 0000000000000000 RBX: ffff880834aafed8 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff81e03228 RBP: ffffffff8100be2e R8: 0000000000000000 R9: 0000000000000000 R10: 00083559a322bda9 R11: ffff880834aafe68 R12: 0000000000000082 R13: ffff880834aafe48 R14: ffff880028231ac8 R15: ffff880834aafe88 ORIG_RAX: ffffffffffffff04 CS: 0010 SS: 0018 #26 [ffff880834aafee0] default_idle at ffffffff8101696d #27 [ffff880834aaff00] cpu_idle at ffffffff81009fe6
I found a similar error in the KnowledgeBase, but it seemed to be based around TrendMicro serverprotect, which I am not using.
Responses