Kernel Oops Analyzer

Updated -

This tool is designed to help you diagnose a kernel crash. When you input a text or a file including one or more kernel oops messages, we will walk you through diagnosing the kernel crash issue.

To use this app, copy and paste a text or choose a file including kernel oop messages and click the Detect button. All kernel messages will be extracted and shown on the right side of the page. If only one kernel oops message is found, the app will automatically find a solution for you; otherwise, if more than one kernel oops messages are found, you can choose one to find a solution.

Solutions may be presented in two forms. If your kernel oops message exactly matches a known issue, the solution will be a compound result with multiple parts: a KCS solution, fixed kernel versions, errata, and Bugzilla bugs. If there is no exact match to your kernel message, we will search the Red Hat Knowledgebase and return several of the most relevant KCS solutions.

If you are not sure how to get kernel oops messages, you can find tips on the lower left side of this page.

Was this helpful?

We appreciate your feedback. Leave a comment if you would like to provide more detail.
It looks like we have some work to do. Leave a comment to let us know how we could improve.
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.

Get notified when this content is updated

Follow

Comments

Subscriber exclusive content

An active Red Hat subscription is required to participate.

Log In

Not sure what this tool actually wants as input. I rarely see kernel 'oops', but rather panics with no ooops strings. Will this help in that case or not?

INFO: task vmtoolsd:1448 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
INFO: task rs:main Q:Reg:1581 blocked for more than 120 seconds.

I've not found yet.

I just used it.

None of our prior kernel panics are correctly referenced with a knowledge base article, in every single case (I tried 15 former cases) this tool tells me to open a new case.

Even though every case has eventually been closed with a reference to a knowledge base article. I can provide you with the details if this is helpful to improve the tool.

Aug 15 08:34:46 calorr06 kernel: BUG: soft lockup - CPU#2 stuck for 62s! [oracle:21862]
Aug 15 08:34:46 calorr06 kernel: BUG: soft lockup - CPU#1 stuck for 63s! [events/1:20]
Aug 15 08:34:46 calorr06 kernel: Modules linked in: bridge stp
Aug 15 08:34:46 calorr06 kernel: Modules linked in: bridge stp llc nfs lockd fscache llc nfs lockd fscache auth_rpcgss nfs_acl sunrpc vmsync(U) vsock(U) iptable_filter ip_tables ip6table_filter ip6_tables ipv6 uinput auth_rpcgss ppdev nfs_acl parport_pc sunrpc parport vmsync(U) vsock(U) iptable_filter ip_tables ip6table_filter ip6_tables ipv6 uinput ppdev parport_pc parport microcode microcode vmware_balloon vmware_balloon sg sg vmci vmci(U)(U) i2c_piix4 i2c_piix4 i2c_core i2c_core shpchp shpchp ext4 ext4 mbcache mbcache jbd2 jbd2 sd_mod sd_mod crc_t10dif crc_t10dif sr_mod sr_mod cdrom cdrom vmxnet3 vmw_pvscsi pata_acpi ata_generic ata_piix vmxnet3 vmw_pvscsi pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_mirror dm_region_hash dm_log dm_log dm_mod [last unloaded: speedstep_lib]
Aug 15 08:34:46 calorr06 kernel: CPU 1
Aug 15 08:34:46 calorr06 kernel: Modules linked in: bridge dm_mod [last unloaded: speedstep_lib]
Aug 15 08:34:46 calorr06 kernel: CPU 2 stp
Aug 15 08:34:46 calorr06 kernel: Modules linked in: llc bridge nfs stp lockd llc fscache nfs auth_rpcgss lockd nfs_acl fscache sunrpc auth_rpcgss nfs_acl sunrpc vmsync vmsync(U)(U) vsock vsock(U) iptable_filter(U) ip_tables iptable_filter ip_tables ip6table_filter ip6table_filter ip6_tables ip6_tables ipv6 ipv6 uinput uinput ppdev ppdev parport_pc parport_pc parport parport microcode microcode vmware_balloon vmware_balloon sg sg vmci vmci(U)(U) i2c_piix4 i2c_piix4 i2c_core i2c_core shpchp shpchp ext4 ext4 mbcache mbcache jbd2 jbd2 sd_mod sd_mod crc_t10dif crc_t10dif sr_mod cdrom sr_mod vmxnet3 vmw_pvscsi cdrom pata_acpi vmxnet3 ata_generic vmw_pvscsi pata_acpi ata_piix ata_generic ata_piix dm_mirror dm_mirror dm_region_hash dm_region_hash dm_log dm_mod dm_log dm_mod [last unloaded: speedstep_lib]
Aug 15 08:34:46 calorr06 kernel: [last unloaded: speedstep_lib]
Aug 15 08:34:46 calorr06 kernel:
Aug 15 08:34:46 calorr06 kernel:
Aug 15 08:34:46 calorr06 kernel: Pid: 21862, comm: oracle Not tainted 2.6.32-358.2.1.el6.x86_64 #1
Aug 15 08:34:46 calorr06 kernel: Pid: 20, comm: events/1 Not tainted 2.6.32-358.2.1.el6.x86_64 #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
Aug 15 08:34:46 calorr06 kernel: RIP: 0010:[]
Aug 15 08:34:46 calorr06 kernel: RIP: 0010:[] [] skb_put 0x2d/0x70
Aug 15 08:34:46 calorr06 kernel: RSP: 0000:ffff880049643da0 EFLAGS: 00010246
Aug 15 08:34:46 calorr06 kernel: RAX: 000000000000005e RBX: ffff880049643db0 RCX: 0000000000000002
Aug 15 08:34:46 calorr06 kernel: RDX: 0000000000000000 RSI: 000000000000003c RDI: ffff88074bdda480
Aug 15 08:34:46 calorr06 kernel: RBP: ffffffff8100bb93 R08: 0000000000000000 R09: ffff88078c37c0c0
Aug 15 08:34:46 calorr06 kernel: R10: 0000000000000794 R11: 0000000000000000 R12: ffff880049643d20
Aug 15 08:34:46 calorr06 kernel: R13: ffff88078ccc2800 R14: 0000000000000022 R15: ffffffff81516ffb
Aug 15 08:34:46 calorr06 kernel: FS: 00007fbb8eeb6700(0000) GS:ffff880049640000(0000) knlGS:0000000000000000
Aug 15 08:34:46 calorr06 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Aug 15 08:34:46 calorr06 kernel: CR2: 00007f870fc32000 CR3: 0000000742cb2000 CR4: 00000000000007e0
Aug 15 08:34:46 calorr06 kernel: [] tcp_rcv_established 0x784/0x800
Aug 15 08:34:46 calorr06 kernel: RSP: 0018:ffff880049623d20 EFLAGS: 00010246
Aug 15 08:34:46 calorr06 kernel: RAX: 0000000000000000 RBX: ffff880049623d70 RCX: 00000000000001c0
Aug 15 08:34:46 calorr06 kernel: RDX: ffff88074ca0b044 RSI: 0000000000000000 RDI: ffff8807648e80c0
Aug 15 08:34:46 calorr06 kernel: RBP: ffffffff8100bb93 R08: ffff880834ac4f50 R09: 0000000000000000
Aug 15 08:34:46 calorr06 kernel: R10: 00000000000001c0 R11: 0000000000000000 R12: ffff880049623ca0
Aug 15 08:34:46 calorr06 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug 15 08:34:46 calorr06 kernel: R13: ffff88074ca0b044 R14: ffff8807648e80c0 R15: ffffffff81516ffb
Aug 15 08:34:46 calorr06 kernel: FS: 0000000000000000(0000) GS:ffff880049620000(0000) knlGS:0000000000000000
Aug 15 08:34:46 calorr06 kernel: CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
Aug 15 08:34:46 calorr06 kernel: CR2: 00000035b78f8370 CR3: 0000000740671000 CR4: 00000000000007e0
Aug 15 08:34:46 calorr06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Aug 15 08:34:46 calorr06 kernel: Process oracle (pid: 21862, threadinfo ffff88078ce0c000, task ffff88082f6f7540)
Aug 15 08:34:46 calorr06 kernel: Stack:
Aug 15 08:34:46 calorr06 kernel: ffff88082dd90ee0 ffff8808308a0e58 ffff880049643e30 ffffffffa00537e8
Aug 15 08:34:46 calorr06 kernel: ffff88082dd906e0 0000000000000002 ffff88078c37c0c0 ffffffff8128d2b0
Aug 15 08:34:46 calorr06 kernel: 0000004005f23e30 ffff88073f289022 ffff88082dd906e0 0000009900000011
Aug 15 08:34:46 calorr06 kernel: Call Trace:
Aug 15 08:34:46 calorr06 kernel:
Aug 15 08:34:46 calorr06 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug 15 08:34:46 calorr06 kernel: [] ? vmxnet3_rq_rx_complete 0x1d8/0x890 [vmxnet3]
Aug 15 08:34:46 calorr06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Aug 15 08:34:46 calorr06 kernel: Process events/1 (pid: 20, threadinfo ffff880834b66000, task ffff880834b40ae0)
Aug 15 08:34:46 calorr06 kernel: [] ? swiotlb_map_page 0x0/0x100
Aug 15 08:34:46 calorr06 kernel: Stack:
Aug 15 08:34:46 calorr06 kernel: ffff880049623da0 ffffffff000001e0 ffff880834ac5570 006d9119b0c70b62
Aug 15 08:34:46 calorr06 kernel: 006d9119b0ced209 ffff880831161380 ffff8807648e80c0 0000000000000000
Aug 15 08:34:46 calorr06 kernel: 0000000000000000 ffff88074ca0b030 ffff880049623e10 ffffffff8149fe93
Aug 15 08:34:46 calorr06 kernel: Call Trace:
Aug 15 08:34:46 calorr06 kernel:
Aug 15 08:34:46 calorr06 kernel: [] ? tcp_v4_do_rcv 0x2e3/0x430
Aug 15 08:34:46 calorr06 kernel: [] ? vmxnet3_poll_rx_only 0x43/0xc0 [vmxnet3]
Aug 15 08:34:46 calorr06 kernel: [] ? run_timer_softirq 0x227/0x340
Aug 15 08:34:46 calorr06 kernel: [] ? net_rx_action 0x103/0x2f0
Aug 15 08:34:46 calorr06 kernel: [] ? irq_exit 0x48/0x90
Aug 15 08:34:46 calorr06 kernel: [] ? __do_softirq 0xc1/0x1e0
Aug 15 08:34:46 calorr06 kernel: [] ? hrtimer_interrupt 0x14b/0x260
Aug 15 08:34:46 calorr06 kernel: [] ? apic_timer_interrupt 0x13/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? call_softirq 0x1c/0x30
Aug 15 08:34:46 calorr06 kernel: [] ? do_softirq 0x65/0xa0
Aug 15 08:34:46 calorr06 kernel: [] ? tcp_delack_timer 0x0/0x270
Aug 15 08:34:46 calorr06 kernel: [] ? tcp_delack_timer 0xf2/0x270
Aug 15 08:34:46 calorr06 kernel: [] ? run_timer_softirq 0x197/0x340
Aug 15 08:34:46 calorr06 kernel: [] ? irq_exit 0x85/0x90
Aug 15 08:34:46 calorr06 kernel: [] ? __do_softirq 0xc1/0x1e0
Aug 15 08:34:46 calorr06 kernel: [] ? hrtimer_interrupt 0x14b/0x260
Aug 15 08:34:46 calorr06 kernel: [] ? smp_apic_timer_interrupt 0x70/0x9b
Aug 15 08:34:46 calorr06 kernel: [] ? apic_timer_interrupt 0x13/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? call_softirq 0x1c/0x30
Aug 15 08:34:46 calorr06 kernel:
Aug 15 08:34:46 calorr06 kernel: [] ? do_softirq 0x65/0xa0
Aug 15 08:34:46 calorr06 kernel: [] ? irq_exit 0x85/0x90
Aug 15 08:34:46 calorr06 kernel: [] ? smp_apic_timer_interrupt 0x70/0x9b
Aug 15 08:34:46 calorr06 kernel: [] ? apic_timer_interrupt 0x13/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? kmem_cache_alloc 0x9c/0x190
Aug 15 08:34:46 calorr06 kernel:
Aug 15 08:34:46 calorr06 kernel: [] ? jbd2_journal_start 0x81/0x100 [jbd2]
Aug 15 08:34:46 calorr06 kernel: [] ? drain_array 0xf1/0x100
Aug 15 08:34:46 calorr06 kernel: [] ? ext4_journal_start_sb 0x56/0xe0 [ext4]
Aug 15 08:34:46 calorr06 kernel: [] ? cache_reap 0x8e/0x250
Aug 15 08:34:46 calorr06 kernel: [] ? ext4_dirty_inode 0x2a/0x60 [ext4]
Aug 15 08:34:46 calorr06 kernel: [] ? __mark_inode_dirty 0x3b/0x160
Aug 15 08:34:46 calorr06 kernel: [] ? file_update_time 0xf2/0x170
Aug 15 08:34:46 calorr06 kernel: [] ? prepare_to_wait 0x4e/0x80
Aug 15 08:34:46 calorr06 kernel: [] ? physflat_send_IPI_mask 0xe/0x10
Aug 15 08:34:46 calorr06 kernel: [] ? __generic_file_aio_write 0x230/0x490
Aug 15 08:34:46 calorr06 kernel: [] ? cache_reap 0x0/0x250
Aug 15 08:34:46 calorr06 kernel: [] ? worker_thread 0x170/0x2a0
Aug 15 08:34:46 calorr06 kernel: [] ? apic_timer_interrupt 0xe/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? autoremove_wake_function 0x0/0x40
Aug 15 08:34:46 calorr06 kernel: [] ? mutex_lock 0x21/0x50
Aug 15 08:34:46 calorr06 kernel: [] ? worker_thread 0x0/0x2a0
Aug 15 08:34:46 calorr06 kernel: [] ? kthread 0x96/0xa0
Aug 15 08:34:46 calorr06 kernel: [] ? generic_file_aio_write 0x88/0x100
Aug 15 08:34:46 calorr06 kernel: [] ? child_rip 0xa/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? kthread 0x0/0xa0
Aug 15 08:34:46 calorr06 kernel: [] ? ext4_file_write 0x61/0x1e0 [ext4]
Aug 15 08:34:46 calorr06 kernel: [] ? child_rip 0x0/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? do_sync_write 0xfa/0x140
Aug 15 08:34:46 calorr06 kernel: [] ? autoremove_wake_function 0x0/0x40
Aug 15 08:34:46 calorr06 kernel: Code:
Aug 15 08:34:46 calorr06 kernel: 00 [] ? security_file_permission 0x16/0x20
Aug 15 08:34:46 calorr06 kernel: 00
Aug 15 08:34:46 calorr06 kernel: [] ? vfs_write 0xb8/0x1a0
Aug 15 08:34:46 calorr06 kernel: 3b [] ? sys_pwrite64 0x82/0xa0
Aug 15 08:34:46 calorr06 kernel: 83 [] ? system_call_fastpath 0x16/0x1b
Aug 15 08:34:46 calorr06 kernel: 60 04 00 00 0f
Aug 15 08:34:46 calorr06 kernel: Code: 85 71 89 fd e5 ff 48 83 ec ff 8b 83 1c 10 05 00 48 00 89 83 18 89 1c 24 05 4c 00 89 64 24 00 08 e8 0f 1f 44 00 00 8b 57 6c 4c 58 8d c0 ff 8b 48 89 a7 83 d0 10 05 00 00 00 00 00 e9 8b 54 fd ff 9f c8 00 ff <48> 83 bb b8 04 00 00 00 00 0f 00 84 f3 fc ff ff 85 31 d2 75 ff 25 44 89 8d 55 04 1e b8 <01> 77 e8 68 3b
Aug 15 08:34:46 calorr06 kernel: 87 Call Trace:
Aug 15 08:34:46 calorr06 kernel: cc 00 00 00 89 87 c8 00 00 [] ? tcp_v4_do_rcv 0x2e3/0x430
Aug 15 08:34:46 calorr06 kernel: 00 [] ? irq_exit 0x48/0x90
Aug 15 08:34:46 calorr06 kernel: 77 15 89 db 49 [] ? apic_timer_interrupt 0x13/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? tcp_delack_timer 0x0/0x270
Aug 15 08:34:46 calorr06 kernel: [] ? tcp_delack_timer 0xf2/0x270
Aug 15 08:34:46 calorr06 kernel: 8d
Aug 15 08:34:46 calorr06 kernel: Call Trace:
Aug 15 08:34:46 calorr06 kernel: [] ? run_timer_softirq 0x197/0x340
Aug 15 08:34:46 calorr06 kernel: [] ? __do_softirq 0xc1/0x1e0
Aug 15 08:34:46 calorr06 kernel: [] ? hrtimer_interrupt 0x14b/0x260
Aug 15 08:34:46 calorr06 kernel: [] ? vmxnet3_rq_rx_complete 0x1d8/0x890 [vmxnet3]
Aug 15 08:34:46 calorr06 kernel: [] ? call_softirq 0x1c/0x30
Aug 15 08:34:46 calorr06 kernel: [] ? do_softirq 0x65/0xa0
Aug 15 08:34:46 calorr06 kernel: [] ? irq_exit 0x85/0x90
Aug 15 08:34:46 calorr06 kernel: [] ? smp_apic_timer_interrupt 0x70/0x9b
Aug 15 08:34:46 calorr06 kernel: [] ? apic_timer_interrupt 0x13/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? swiotlb_map_page 0x0/0x100
Aug 15 08:34:46 calorr06 kernel: [] ? vmxnet3_poll_rx_only 0x43/0xc0 [vmxnet3]
Aug 15 08:34:46 calorr06 kernel: [] ? run_timer_softirq 0x227/0x340
Aug 15 08:34:46 calorr06 kernel: [] ? drain_array 0xf1/0x100
Aug 15 08:34:46 calorr06 kernel: [] ? cache_reap 0x8e/0x250
Aug 15 08:34:46 calorr06 kernel: [] ? net_rx_action 0x103/0x2f0
Aug 15 08:34:46 calorr06 kernel: [] ? __do_softirq 0xc1/0x1e0
Aug 15 08:34:46 calorr06 kernel: [] ? prepare_to_wait 0x4e/0x80
Aug 15 08:34:46 calorr06 kernel: [] ? cache_reap 0x0/0x250
Aug 15 08:34:46 calorr06 kernel: [] ? worker_thread 0x170/0x2a0
Aug 15 08:34:46 calorr06 kernel: [] ? autoremove_wake_function 0x0/0x40
Aug 15 08:34:46 calorr06 kernel: [] ? hrtimer_interrupt 0x14b/0x260
Aug 15 08:34:46 calorr06 kernel: [] ? call_softirq 0x1c/0x30
Aug 15 08:34:46 calorr06 kernel: [] ? do_softirq 0x65/0xa0
Aug 15 08:34:46 calorr06 kernel: [] ? irq_exit 0x85/0x90
Aug 15 08:34:46 calorr06 kernel: [] ? smp_apic_timer_interrupt 0x70/0x9b
Aug 15 08:34:46 calorr06 kernel: [] ? apic_timer_interrupt 0x13/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? worker_thread 0x0/0x2a0
Aug 15 08:34:46 calorr06 kernel: [] ? kthread 0x96/0xa0
Aug 15 08:34:46 calorr06 kernel: [] ? child_rip 0xa/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? kmem_cache_alloc 0x9c/0x190
Aug 15 08:34:46 calorr06 kernel: [] ? kthread 0x0/0xa0
Aug 15 08:34:46 calorr06 kernel: [] ? jbd2_journal_start 0x81/0x100 [jbd2]
Aug 15 08:34:46 calorr06 kernel: [] ? ext4_journal_start_sb 0x56/0xe0 [ext4]
Aug 15 08:34:46 calorr06 kernel: [] ? child_rip 0x0/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? ext4_dirty_inode 0x2a/0x60 [ext4]
Aug 15 08:34:46 calorr06 kernel: [] ? __mark_inode_dirty 0x3b/0x160
Aug 15 08:34:46 calorr06 kernel: [] ? file_update_time 0xf2/0x170
Aug 15 08:34:46 calorr06 kernel: [] ? physflat_send_IPI_mask 0xe/0x10
Aug 15 08:34:46 calorr06 kernel: [] ? __generic_file_aio_write 0x230/0x490
Aug 15 08:34:46 calorr06 kernel: [] ? apic_timer_interrupt 0xe/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? mutex_lock 0x21/0x50
Aug 15 08:34:46 calorr06 kernel: [] ? generic_file_aio_write 0x88/0x100
Aug 15 08:34:46 calorr06 kernel: [] ? ext4_file_write 0x61/0x1e0 [ext4]
Aug 15 08:34:46 calorr06 kernel: [] ? do_sync_write 0xfa/0x140
Aug 15 08:34:46 calorr06 kernel: [] ? autoremove_wake_function 0x0/0x40
Aug 15 08:34:46 calorr06 kernel: [] ? security_file_permission 0x16/0x20
Aug 15 08:34:46 calorr06 kernel: [] ? vfs_write 0xb8/0x1a0
Aug 15 08:34:46 calorr06 kernel: [] ? sys_pwrite64 0x82/0xa0
Aug 15 08:34:46 calorr06 kernel: [] ? system_call_fastpath 0x16/0x1b

obviously, this is a cpu softlockup warning, rather than a kernel oops message.