[RHEL6] Kernel panic occurred at netpoll_poll_dev.

Solution Verified - Updated -

Issue

  • A kernel panic occured with RIP in netpoll_poll_dev.
  • The following is the data from the resulting core:
crash> sys
      KERNEL: /usr/lib/debug/lib/modules/2.6.32-220.4.2.el6.i686/vmlinux
    DUMPFILE: vmcore_yknsshp1._xxx_  [PARTIAL DUMP]
        CPUS: 8
        DATE: Fri Sep 14 15:30:08 2012
      UPTIME: 3 days, 19:59:53
LOAD AVERAGE: 0.01, 0.02, 0.02
       TASKS: 414
    NODENAME: yknsshp1
     RELEASE: 2.6.32-220.4.2.el6.i686
     VERSION: #1 SMP Mon Feb 6 16:46:31 EST 2012
     MACHINE: i686  (2400 Mhz)
      MEMORY: 4 GB
       PANIC: "Oops: 0002 [#1] SMP " (check log for details)
crash>
crash> bt
PID: 40     TASK: f7170030  CPU: 5   COMMAND: "events/5"
 #0 [f7193b24] crash_kexec at c049a98c
 #1 [f7193b78] oops_end at c0831f82
 #2 [f7193b8c] no_context at c04331fd
 #3 [f7193bb0] bad_area_nosemaphore at c043342a
 #4 [f7193bb8] __do_page_fault at c04338a3
 #5 [f7193c38] do_page_fault at c0833905
 #6 [f7193c50] error_code (via page_fault) at c0831395
    EAX: f4cb5020  EBX: f490abcc  ECX: 00000101  EDX: f490abe0  EBP: f4cb505c
    DS:  007b      ESI: f490abe0  ES:  007b      EDI: 00000000  GS:  00e0
    CS:  0060      EIP: c079ab6a  ERR: ffffffff  EFLAGS: 00010002
 #7 [f7193c84] netpoll_poll_dev at c079ab6a
 #8 [f7193c9c] enqueue_entity at c04502e0
 #9 [f7193ccc] bond_poll_controller at f916b3c8 [bonding]
#10 [f7193cdc] netpoll_poll_dev at c079aafe
#11 [f7193d1c] netpoll_send_skb_on_dev at c079b0a8
#12 [f7193d5c] bond_dev_queue_xmit at f916c7fa [bonding]
#13 [f7193d78] bond_start_xmit at f916ca99 [bonding]
#14 [f7193d8c] __alloc_skb at c077fa8a
#15 [f7193da8] netpoll_send_skb_on_dev at c079b0eb
#16 [f7193dbc] netpoll_send_udp at c079b2fb
#17 [f7193de8] write_msg at f7e8f2cc [netconsole]
#18 [f7193e0c] __call_console_drivers at c0454c69
#19 [f7193e24] release_console_sem at c0455178
#20 [f7193e48] vprintk at c04556ff
#21 [f7193ec8] printk at c082e171
#22 [f7193ed4] e1000_print_hw_hang at f85f590d [e1000e]
#23 [f7193f68] worker_thread at c0471809
#24 [f7193fc0] kthread at c0475d22
#25 [f7193fe4] kernel_thread_helper at c0409ffd
crash>
crash> log
    :
  (snip)
    :
e1000e 0000:03:00.0: eth2: Detected Hardware Unit Hang:
  TDH                  <6a>
  TDT                  <6b>
  next_to_use          <6b>
  next_to_clean        <69>
buffer_info[next_to_clean]:
  time_stamp           <13b90319>
  next_to_watch        <6a>
  jiffies              <13b9080e>
  next_to_watch.status <0>
MAC Status             <80383>
PHY Status             <792d>
PHY 1000BASE-T Status  <3800>
PHY Extended Status    <3000>
PCI Status             <10>
BUG: unable to handle kernel NULL pointer dereference at 00000004
IP: [<c079ab6a>] netpoll_poll_dev+0x9a/0x420
*pdpt = 00000000347ce001 *pde = 000000013dd5b067
Oops: 0002 [#1] SMP
last sysfs file: /sys/devices/pci0000:ff/0000:ff:06.3/irq
Modules linked in: fuse clpka(U) nfsd lockd nfs_acl auth_rpcgss exportfs iptable_filter ip_tables clpkhb(U) cpufreq_powersave liscal(P)(U) netconsole configfs autofs4 sunrpc cpufreq_ondemand acpi_cpufreq mperf bonding ipv6 uinput power_meter microcode sg i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support ioatdma i7core_edac edac_core e1000e igb dca ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix megaraid_sas(U) dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]

Pid: 40, comm: events/5 Tainted: P           ----------------   2.6.32-220.4.2.el6.i686 #1 NEC Express5800/R120b-1 [N8100-1718]/MS-91E7-001
EIP: 0060:[<c079ab6a>] EFLAGS: 00010002 CPU: 5
EIP is at netpoll_poll_dev+0x9a/0x420
EAX: f4cb5020 EBX: f490abcc ECX: 00000101 EDX: f490abe0
ESI: f490abe0 EDI: 00000000 EBP: f4cb505c ESP: f7193c90
 DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Process events/5 (pid: 40, ti=f7192000 task=f7170030 task.ti=f7192000)
Stack:
 00000991 d3c180fc f34ba59c c04502e5 9173651f 00000010 f4cb5020 ffffffff
<0> 00000001 000001aa d3c1811c f4cfea00 00000000 f4db13e0 f4de9140 f916b3cd
<0> f4a58400 f4db1020 f4de9160 c079ab00 c043badd f34ba570 d3c180c0 2581a254
Call Trace:
 [<c04502e5>] ? enqueue_entity+0x155/0x490
 [<f916b3cd>] ? bond_poll_controller+0x4d/0x60 [bonding]
 [<c079ab00>] ? netpoll_poll_dev+0x30/0x420
 [<c043badd>] ? check_preempt_curr+0x4d/0x70
 [<c0427c33>] ? lapic_next_event+0x13/0x20
 [<c0483c1c>] ? clockevents_program_event+0x8c/0x120
 [<c079b0ad>] ? netpoll_send_skb_on_dev+0x12d/0x210
 [<f916c7ff>] ? bond_dev_queue_xmit+0x19f/0x1b0 [bonding]
 [<f916ca9e>] ? bond_start_xmit+0x28e/0x530 [bonding]
 [<c077fa8f>] ? __alloc_skb+0x4f/0x140
 [<c079b0ee>] ? netpoll_send_skb_on_dev+0x16e/0x210
 [<c079b300>] ? netpoll_send_udp+0x170/0x1d0
 [<f7e8f2d1>] ? write_msg+0x91/0xe0 [netconsole]
 [<f7e8f240>] ? write_msg+0x0/0xe0 [netconsole]
 [<c0454c6b>] ? __call_console_drivers+0x5b/0x70
 [<c045517d>] ? release_console_sem+0xfd/0x210
 [<c0455704>] ? vprintk+0x1a4/0x490
 [<c07758a0>] ? pci_read+0x30/0x40
 [<c06096ee>] ? pci_bus_read_config_word+0x6e/0x80
 [<c082e176>] ? printk+0x17/0x21
 [<f85f5912>] ? e1000_print_hw_hang+0x1f2/0x200 [e1000e]
 [<f85f5720>] ? e1000_print_hw_hang+0x0/0x200 [e1000e]
 [<c047180b>] ? worker_thread+0x11b/0x230
 [<c0475f60>] ? autoremove_wake_function+0x0/0x40
 [<c04716f0>] ? worker_thread+0x0/0x230
 [<c0475d24>] ? kthread+0x74/0x80
 [<c0475cb0>] ? kthread+0x0/0x80
 [<c0409fff>] ? kernel_thread_helper+0x7/0x10
Code: 00 00 64 a1 18 9d b0 c0 39 43 18 74 e7 8d 73 14 89 f0 e8 fa 5c 09 00 85 c0 74 d9 8b 44 24 18 f6 43 08 01 8b b8 40 02 00 00 74 b6 <83> 4f 04 02 f0 ff 05 0c 8f c3 c0 8d 4b 08 f0 80 4b 08 04 8b 54
EIP: [<c079ab6a>] netpoll_poll_dev+0x9a/0x420 SS:ESP 0068:f7193c90
CR2: 0000000000000004
crash>

Environment

  • Red Hat Enterprise Linux 6.2
  • kernel-2.6.32-220.4.2.el6.i686

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.