BUG: unable to handle kernel paging request, panic in update_curr()
Issue
- A vmcore was not collected becuase the customer does not configure kdump.conf so that vmcore is collected. However, instead he was configuring kdump.conf so that ring buffer (dmesg) information was collected.
- According to the ring buffer (kdump-dmesg-2013-10-22.0407 file), following panic messages were recorded. We could not find the similar case as this panic from redhat bugzilla and nec's case.
<1>BUG: unable to handle kernel paging request at ffffffff5183f000
<1>IP: [<ffffffff810568e4>] update_curr+0x144/0x1f0
<4>PGD 1a87067 PUD 0
<4>Oops: 0000 [#1] SMP
<4>last sysfs file: /sys/module/vt/parameters/underline
<4>CPU 3
<4>Modules linked in: des_generic ecb md4 nls_utf8 cifs softdog clpkhb(U) liscal(P)(U) ipmi_devintf sunrpc bonding 8021q garp stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ipt_REJECT iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 uinput power_meter microcode sg sb_edac edac_core i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support tg3 ioatdma dca ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix megaraid_sas dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
<4>
<4>Pid: 24294, comm: flush-218:1 Tainted: P --------------- 2.6.32-358.2.1.el6.x86_64 #1 NEC Express5800/R120d-2E [N8100-1814Y]/GA-7PPSV-NJ
<4>RIP: 0010:[<ffffffff810568e4>] [<ffffffff810568e4>] update_curr+0x144/0x1f0
<4>RSP: 0018:ffff880028383db8 EFLAGS: 00010082
<4>RAX: ffff8803796bd540 RBX: fffffffff9f88248 RCX: ffff8801ffeec240
<4>RDX: 00000000000192d8 RSI: 0000000000000000 RDI: ffff8803796bd578
<4>RBP: ffff880028383de8 R08: ffffffff8160bb65 R09: 0000000000000000
<4>R10: 0000000000000010 R11: 0000000000000000 R12: ffff880028396768
<4>R13: 00000000000f9906 R14: 000df78a74f7bb14 R15: ffff8803796bd540
<4>FS: 0000000000000000(0000) GS:ffff880028380000(0000) knlGS:0000000000000000
<4>CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<4>CR2: ffffffff5183f000 CR3: 00000001fac21000 CR4: 00000000000407e0
<4>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>Process flush-218:1 (pid: 24294, threadinfo ffff880379f88000, task ffff8803796bd540)
<4>Stack:
<4> ffff880028383dc8 ffffffff81013783 ffff8803796bd578 ffff880028396768
<4><d> 0000000000000000 0000000000000000 ffff880028383e18 ffffffff81056e9b
<4><d> ffff880028396700 0000000000000003 0000000000016700 0000000000000003
<4>Call Trace:
<4> <IRQ>
<4> [<ffffffff81013783>] ? native_sched_clock+0x13/0x80
<4> [<ffffffff81056e9b>] task_tick_fair+0xdb/0x160
<4> [<ffffffff8105ace1>] scheduler_tick+0xc1/0x260
<4> [<ffffffff810a7ff0>] ? tick_sched_timer+0x0/0xc0
<4> [<ffffffff810811fe>] update_process_times+0x6e/0x90
<4> [<ffffffff810a8056>] tick_sched_timer+0x66/0xc0
<4> [<ffffffff8109b3ae>] __run_hrtimer+0x8e/0x1a0
<4> [<ffffffff810a184f>] ? ktime_get_update_offsets+0x4f/0xd0
<4> [<ffffffff8107700f>] ? __do_softirq+0x11f/0x1e0
<4> [<ffffffff8109b716>] hrtimer_interrupt+0xe6/0x260
<4> [<ffffffff81516ffb>] smp_apic_timer_interrupt+0x6b/0x9b
<4> [<ffffffff8100bb93>] apic_timer_interrupt+0x13/0x20
<4> <EOI>
<4> [<ffffffff81168620>] ? kmem_cache_alloc_node_trace+0xa0/0x200
<4> [<ffffffff8116882d>] __kmalloc_node+0x4d/0x60
<4> [<ffffffff8143d8cd>] __alloc_skb+0x6d/0x190
<4> [<ffffffff8143e9e6>] skb_copy+0x36/0xa0
<4> [<ffffffffa013627c>] tg3_start_xmit+0xa8c/0xd50 [tg3]
<4> [<ffffffff81448ec8>] dev_hard_start_xmit+0x308/0x530
<4> [<ffffffff8146724a>] sch_direct_xmit+0x15a/0x1c0
<4> [<ffffffff8144cbd0>] dev_queue_xmit+0x3b0/0x550
<4> [<ffffffff814851c0>] ? ip_finish_output+0x0/0x310
<4> [<ffffffff814852fc>] ip_finish_output+0x13c/0x310
<4> [<ffffffff81485588>] ip_output+0xb8/0xc0
<4> [<ffffffff8148484f>] ? __ip_local_out+0x9f/0xb0
<4> [<ffffffff81484885>] ip_local_out+0x25/0x30
<4> [<ffffffff81484d60>] ip_queue_xmit+0x190/0x420
<4> [<ffffffff8113b489>] ? zone_statistics+0x99/0xc0
<4> [<ffffffff8112c74f>] ? free_hot_page+0x2f/0x60
<4> [<ffffffff81499a4e>] tcp_transmit_skb+0x3fe/0x7b0
<4> [<ffffffff8149be0b>] tcp_write_xmit+0x1fb/0xa20
<4> [<ffffffff8149c7c0>] __tcp_push_pending_frames+0x30/0xe0
<4> [<ffffffff814942f3>] tcp_data_snd_check+0x33/0x100
<4> [<ffffffff81497efd>] tcp_rcv_established+0x3ed/0x800
<4> [<ffffffff8149fe93>] tcp_v4_do_rcv+0x2e3/0x430
<4> [<ffffffff8149bc82>] ? tcp_write_xmit+0x72/0xa20
<4> [<ffffffff81438e35>] release_sock+0x65/0xe0
<4> [<ffffffff8148d12c>] tcp_sendmsg+0x73c/0xa20
<4> [<ffffffff814359e3>] sock_sendmsg+0x123/0x150
<4> [<ffffffff81096ca0>] ? autoremove_wake_function+0x0/0x40
<4> [<ffffffff81096ca0>] ? autoremove_wake_function+0x0/0x40
<4> [<ffffffffa02dbe91>] ? liscal_spin_unlock_irqrestore+0x11/0x20 [liscal]
<4> [<ffffffff81082034>] ? mod_timer+0x144/0x220
<4> [<ffffffffa02de97d>] liscal_send2+0xfd/0x210 [liscal]
<4> [<ffffffffa02dc977>] liscal_send_data_part+0x47/0xc0 [liscal]
<4> [<ffffffffa02d6e75>] liscal_send_data+0x265/0x430 [liscal]
<4> [<ffffffffa02db8ae>] liscal_make_request+0x79e/0x940 [liscal]
<4> [<ffffffff8125cdde>] generic_make_request+0x25e/0x530
<4> [<ffffffff8111c575>] ? mempool_alloc_slab+0x15/0x20
<4> [<ffffffffa00c21d8>] ? __ext4_journal_stop+0x68/0xa0 [ext4]
<4> [<ffffffff8125d13d>] submit_bio+0x8d/0x120
<4> [<ffffffff811b4d16>] submit_bh+0xf6/0x150
<4> [<ffffffff811b6a70>] __block_write_full_page+0x1e0/0x3b0
<4> [<ffffffff811b63b0>] ? end_buffer_async_write+0x0/0x190
<4> [<ffffffffa00ab270>] ? noalloc_get_block_write+0x0/0x60 [ext4]
<4> [<ffffffffa00ab270>] ? noalloc_get_block_write+0x0/0x60 [ext4]
<4> [<ffffffff811b76b0>] block_write_full_page_endio+0xe0/0x120
<4> [<ffffffffa00a6c80>] ? ext4_bh_delay_or_unwritten+0x0/0x30 [ext4]
<4> [<ffffffff811b7705>] block_write_full_page+0x15/0x20
<4> [<ffffffffa00acaa2>] ext4_writepage+0x172/0x400 [ext4]
<4> [<ffffffffa00ace77>] mpage_da_submit_io+0x147/0x1d0 [ext4]
<4> [<ffffffffa00af2de>] mpage_da_map_and_submit+0x17e/0x470 [ext4]
<4> [<ffffffff8127cf55>] ? radix_tree_gang_lookup_tag_slot+0x95/0xe0
<4> [<ffffffff81119860>] ? find_get_pages_tag+0x40/0x130
<4> [<ffffffff81119860>] ? find_get_pages_tag+0x40/0x130
<4> [<ffffffffa00af63d>] mpage_add_bh_to_extent+0x6d/0xf0 [ext4]
<4> [<ffffffffa00af98f>] write_cache_pages_da+0x2cf/0x470 [ext4]
<4> [<ffffffffa00afe02>] ext4_da_writepages+0x2d2/0x620 [ext4]
<4> [<ffffffff8112e1d1>] do_writepages+0x21/0x40
<4> [<ffffffff811ac7cd>] writeback_single_inode+0xdd/0x290
<4> [<ffffffff811acbde>] writeback_sb_inodes+0xce/0x180
<4> [<ffffffff811acd3b>] writeback_inodes_wb+0xab/0x1b0
<4> [<ffffffff811ad0db>] wb_writeback+0x29b/0x3f0
<4> [<ffffffff8150d960>] ? thread_return+0x4e/0x76e
<4> [<ffffffff81081ae2>] ? del_timer_sync+0x22/0x30
<4> [<ffffffff811ad3c9>] wb_do_writeback+0x199/0x240
<4> [<ffffffff811ad4d3>] bdi_writeback_task+0x63/0x1b0
<4> [<ffffffff81096b67>] ? bit_waitqueue+0x17/0xd0
<4> [<ffffffff8113cb20>] ? bdi_start_fn+0x0/0x100
<4> [<ffffffff8113cba6>] bdi_start_fn+0x86/0x100
<4> [<ffffffff8113cb20>] ? bdi_start_fn+0x0/0x100
<4> [<ffffffff81096936>] kthread+0x96/0xa0
<4> [<ffffffff8100c0ca>] child_rip+0xa/0x20
<4> [<ffffffff810968a0>] ? kthread+0x0/0xa0
<4> [<ffffffff8100c0c0>] ? child_rip+0x0/0x20
<4>Code: 00 8b 15 34 2d a4 00 85 d2 74 34 48 8b 50 08 8b 5a 18 48 8b 90 10 09 00 00 48 8b 4a 50 48 85 c9 74 1d 48 63 db 66 90 48 8b 51 20 <48> 03 14 dd c0 dd bf 81 4c 01 2a 48 8b 49 78 48 85 c9 75 e8 48
<1>RIP [<ffffffff810568e4>] update_curr+0x144/0x1f0
<4> RSP <ffff880028383db8>
<4>CR2: ffffffff5183f000
Environment
- Red Hat Enterprise Linux 6.4
- kernel-2.6.32-358.2.1.el6.x86_64
- 3rd party kernel module - 'liscal'
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.