BUG: unable to handle kernel paging request, panic in update_curr()

Solution Unverified - Updated -

Issue

  • A vmcore was not collected becuase the customer does not configure kdump.conf so that vmcore is collected. However, instead he was configuring kdump.conf so that ring buffer (dmesg) information was collected.
  • According to the ring buffer (kdump-dmesg-2013-10-22.0407 file), following panic messages were recorded. We could not find the similar case as this panic from redhat bugzilla and nec's case.
<1>BUG: unable to handle kernel paging request at ffffffff5183f000
<1>IP: [<ffffffff810568e4>] update_curr+0x144/0x1f0
<4>PGD 1a87067 PUD 0 
<4>Oops: 0000 [#1] SMP 
<4>last sysfs file: /sys/module/vt/parameters/underline
<4>CPU 3 
<4>Modules linked in: des_generic ecb md4 nls_utf8 cifs softdog clpkhb(U) liscal(P)(U) ipmi_devintf sunrpc bonding 8021q garp stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ipt_REJECT iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 uinput power_meter microcode sg sb_edac edac_core i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support tg3 ioatdma dca ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix megaraid_sas dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
<4>
<4>Pid: 24294, comm: flush-218:1 Tainted: P           ---------------    2.6.32-358.2.1.el6.x86_64 #1 NEC Express5800/R120d-2E [N8100-1814Y]/GA-7PPSV-NJ 
<4>RIP: 0010:[<ffffffff810568e4>]  [<ffffffff810568e4>] update_curr+0x144/0x1f0
<4>RSP: 0018:ffff880028383db8  EFLAGS: 00010082
<4>RAX: ffff8803796bd540 RBX: fffffffff9f88248 RCX: ffff8801ffeec240
<4>RDX: 00000000000192d8 RSI: 0000000000000000 RDI: ffff8803796bd578
<4>RBP: ffff880028383de8 R08: ffffffff8160bb65 R09: 0000000000000000
<4>R10: 0000000000000010 R11: 0000000000000000 R12: ffff880028396768
<4>R13: 00000000000f9906 R14: 000df78a74f7bb14 R15: ffff8803796bd540
<4>FS:  0000000000000000(0000) GS:ffff880028380000(0000) knlGS:0000000000000000
<4>CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<4>CR2: ffffffff5183f000 CR3: 00000001fac21000 CR4: 00000000000407e0
<4>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>Process flush-218:1 (pid: 24294, threadinfo ffff880379f88000, task ffff8803796bd540)
<4>Stack:
<4> ffff880028383dc8 ffffffff81013783 ffff8803796bd578 ffff880028396768
<4><d> 0000000000000000 0000000000000000 ffff880028383e18 ffffffff81056e9b
<4><d> ffff880028396700 0000000000000003 0000000000016700 0000000000000003
<4>Call Trace:
<4> <IRQ> 
<4> [<ffffffff81013783>] ? native_sched_clock+0x13/0x80
<4> [<ffffffff81056e9b>] task_tick_fair+0xdb/0x160
<4> [<ffffffff8105ace1>] scheduler_tick+0xc1/0x260
<4> [<ffffffff810a7ff0>] ? tick_sched_timer+0x0/0xc0
<4> [<ffffffff810811fe>] update_process_times+0x6e/0x90
<4> [<ffffffff810a8056>] tick_sched_timer+0x66/0xc0
<4> [<ffffffff8109b3ae>] __run_hrtimer+0x8e/0x1a0
<4> [<ffffffff810a184f>] ? ktime_get_update_offsets+0x4f/0xd0
<4> [<ffffffff8107700f>] ? __do_softirq+0x11f/0x1e0
<4> [<ffffffff8109b716>] hrtimer_interrupt+0xe6/0x260
<4> [<ffffffff81516ffb>] smp_apic_timer_interrupt+0x6b/0x9b
<4> [<ffffffff8100bb93>] apic_timer_interrupt+0x13/0x20
<4> <EOI> 
<4> [<ffffffff81168620>] ? kmem_cache_alloc_node_trace+0xa0/0x200
<4> [<ffffffff8116882d>] __kmalloc_node+0x4d/0x60
<4> [<ffffffff8143d8cd>] __alloc_skb+0x6d/0x190
<4> [<ffffffff8143e9e6>] skb_copy+0x36/0xa0
<4> [<ffffffffa013627c>] tg3_start_xmit+0xa8c/0xd50 [tg3]
<4> [<ffffffff81448ec8>] dev_hard_start_xmit+0x308/0x530
<4> [<ffffffff8146724a>] sch_direct_xmit+0x15a/0x1c0
<4> [<ffffffff8144cbd0>] dev_queue_xmit+0x3b0/0x550
<4> [<ffffffff814851c0>] ? ip_finish_output+0x0/0x310
<4> [<ffffffff814852fc>] ip_finish_output+0x13c/0x310
<4> [<ffffffff81485588>] ip_output+0xb8/0xc0
<4> [<ffffffff8148484f>] ? __ip_local_out+0x9f/0xb0
<4> [<ffffffff81484885>] ip_local_out+0x25/0x30
<4> [<ffffffff81484d60>] ip_queue_xmit+0x190/0x420
<4> [<ffffffff8113b489>] ? zone_statistics+0x99/0xc0
<4> [<ffffffff8112c74f>] ? free_hot_page+0x2f/0x60
<4> [<ffffffff81499a4e>] tcp_transmit_skb+0x3fe/0x7b0
<4> [<ffffffff8149be0b>] tcp_write_xmit+0x1fb/0xa20
<4> [<ffffffff8149c7c0>] __tcp_push_pending_frames+0x30/0xe0
<4> [<ffffffff814942f3>] tcp_data_snd_check+0x33/0x100
<4> [<ffffffff81497efd>] tcp_rcv_established+0x3ed/0x800
<4> [<ffffffff8149fe93>] tcp_v4_do_rcv+0x2e3/0x430
<4> [<ffffffff8149bc82>] ? tcp_write_xmit+0x72/0xa20
<4> [<ffffffff81438e35>] release_sock+0x65/0xe0
<4> [<ffffffff8148d12c>] tcp_sendmsg+0x73c/0xa20
<4> [<ffffffff814359e3>] sock_sendmsg+0x123/0x150
<4> [<ffffffff81096ca0>] ? autoremove_wake_function+0x0/0x40
<4> [<ffffffff81096ca0>] ? autoremove_wake_function+0x0/0x40
<4> [<ffffffffa02dbe91>] ? liscal_spin_unlock_irqrestore+0x11/0x20 [liscal]
<4> [<ffffffff81082034>] ? mod_timer+0x144/0x220
<4> [<ffffffffa02de97d>] liscal_send2+0xfd/0x210 [liscal]
<4> [<ffffffffa02dc977>] liscal_send_data_part+0x47/0xc0 [liscal]
<4> [<ffffffffa02d6e75>] liscal_send_data+0x265/0x430 [liscal]
<4> [<ffffffffa02db8ae>] liscal_make_request+0x79e/0x940 [liscal]
<4> [<ffffffff8125cdde>] generic_make_request+0x25e/0x530
<4> [<ffffffff8111c575>] ? mempool_alloc_slab+0x15/0x20
<4> [<ffffffffa00c21d8>] ? __ext4_journal_stop+0x68/0xa0 [ext4]
<4> [<ffffffff8125d13d>] submit_bio+0x8d/0x120
<4> [<ffffffff811b4d16>] submit_bh+0xf6/0x150
<4> [<ffffffff811b6a70>] __block_write_full_page+0x1e0/0x3b0
<4> [<ffffffff811b63b0>] ? end_buffer_async_write+0x0/0x190
<4> [<ffffffffa00ab270>] ? noalloc_get_block_write+0x0/0x60 [ext4]
<4> [<ffffffffa00ab270>] ? noalloc_get_block_write+0x0/0x60 [ext4]
<4> [<ffffffff811b76b0>] block_write_full_page_endio+0xe0/0x120
<4> [<ffffffffa00a6c80>] ? ext4_bh_delay_or_unwritten+0x0/0x30 [ext4]
<4> [<ffffffff811b7705>] block_write_full_page+0x15/0x20
<4> [<ffffffffa00acaa2>] ext4_writepage+0x172/0x400 [ext4]
<4> [<ffffffffa00ace77>] mpage_da_submit_io+0x147/0x1d0 [ext4]
<4> [<ffffffffa00af2de>] mpage_da_map_and_submit+0x17e/0x470 [ext4]
<4> [<ffffffff8127cf55>] ? radix_tree_gang_lookup_tag_slot+0x95/0xe0
<4> [<ffffffff81119860>] ? find_get_pages_tag+0x40/0x130
<4> [<ffffffff81119860>] ? find_get_pages_tag+0x40/0x130
<4> [<ffffffffa00af63d>] mpage_add_bh_to_extent+0x6d/0xf0 [ext4]
<4> [<ffffffffa00af98f>] write_cache_pages_da+0x2cf/0x470 [ext4]
<4> [<ffffffffa00afe02>] ext4_da_writepages+0x2d2/0x620 [ext4]
<4> [<ffffffff8112e1d1>] do_writepages+0x21/0x40
<4> [<ffffffff811ac7cd>] writeback_single_inode+0xdd/0x290
<4> [<ffffffff811acbde>] writeback_sb_inodes+0xce/0x180
<4> [<ffffffff811acd3b>] writeback_inodes_wb+0xab/0x1b0
<4> [<ffffffff811ad0db>] wb_writeback+0x29b/0x3f0
<4> [<ffffffff8150d960>] ? thread_return+0x4e/0x76e
<4> [<ffffffff81081ae2>] ? del_timer_sync+0x22/0x30
<4> [<ffffffff811ad3c9>] wb_do_writeback+0x199/0x240
<4> [<ffffffff811ad4d3>] bdi_writeback_task+0x63/0x1b0
<4> [<ffffffff81096b67>] ? bit_waitqueue+0x17/0xd0
<4> [<ffffffff8113cb20>] ? bdi_start_fn+0x0/0x100
<4> [<ffffffff8113cba6>] bdi_start_fn+0x86/0x100
<4> [<ffffffff8113cb20>] ? bdi_start_fn+0x0/0x100
<4> [<ffffffff81096936>] kthread+0x96/0xa0
<4> [<ffffffff8100c0ca>] child_rip+0xa/0x20
<4> [<ffffffff810968a0>] ? kthread+0x0/0xa0
<4> [<ffffffff8100c0c0>] ? child_rip+0x0/0x20
<4>Code: 00 8b 15 34 2d a4 00 85 d2 74 34 48 8b 50 08 8b 5a 18 48 8b 90 10 09 00 00 48 8b 4a 50 48 85 c9 74 1d 48 63 db 66 90 48 8b 51 20 <48> 03 14 dd c0 dd bf 81 4c 01 2a 48 8b 49 78 48 85 c9 75 e8 48 
<1>RIP  [<ffffffff810568e4>] update_curr+0x144/0x1f0
<4> RSP <ffff880028383db8>
<4>CR2: ffffffff5183f000

Environment

  • Red Hat Enterprise Linux 6.4
  • kernel-2.6.32-358.2.1.el6.x86_64
  • 3rd party kernel module - 'liscal'

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content