Why does my system with a GRO enabled network interface and VLANs trigger a kernel panic with RIP :8021q:vlan_gro_frags+0x2d/0x21d ?

Solution Verified - Updated -

Issue

  • Systems with 10G ethernet card (currently found with qlge and be2net drivers) configured with bonding + vlan triggers kernels panic periodically. Due to this panic, system is unusable.

    #3 [ffff81084e189ef0] default_idle at ffffffff8006bdde
     #4 [ffff8
    Unable to handle kernel NULL pointer dereference at 00000000000003c8 RIP: 
     [<ffffffff88306c20>] :8021q:vlan_gro_frags+0x2d/0x21d
    PGD 10320cb067 PUD 10334d6067 PMD 0 
    Oops: 0000 [1] SMP 
    last sysfs file: /class/net/br2000/bridge/topology_change_detected
    CPU 0 
    Modules linked in: tun lockd sunrpc bonding xt_physdev bridge ipt_REJECT xt_tcpudp xt_state ip_conntrack nfnetlink xt_multiport iptable_filter ip_tables x_tables be2iscsi ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic ipv6 xfrm_nalgo crypto_api uio cxgb3i cxgb3 libiscsi_tcp libiscsi2 scsi_transport_iscsi2 scsi_transport_iscsi parport_pc lp parport ksm(U) kvm_amd(U) kvm(U) shpchp sg tpm_tis i2c_core amd64_edac_mod be2net tpm tpm_bios edac_mc 8021q squashfs dm_snapshot ext3 jbd dm_multipath dm_mod usb_storage ahci libata ohci_hcd ehci_hcd cciss uhci_hcd qla2xxx scsi_transport_fc scsi_dh_rdac scsi_dh loop sr_mod scsi_mod cdrom
    Pid: 18125, comm: ifup Tainted: G      2.6.18-238.el5 #1
    RIP: 0010:[<ffffffff88306c20>]  [<ffffffff88306c20>] :8021q:vlan_gro_frags+0x2d/0x21d
    RSP: 0000:ffffffff804a5de0  EFLAGS: 00010286
    RAX: 0000000000000001 RBX: ffff81043eee8dc0 RCX: 0000000000000106
    RDX: ffff8108314f6000 RSI: 0000000000000000 RDI: ffff81103267e820
    RBP: ffff81083f0f0510 R08: ffff810009000830 R09: ffff810433e62900
    R10: 0000000000000000 R11: 00000000ffffffff R12: 0000000000000014
    R13: ffff8108338a0000 R14: 0000000000000001 R15: 0000000000000007
    FS:  00002b5d89f7bf50(0000) GS:ffffffff80425000(0000) knlGS:0000000000000000
    CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
    CR2: 00000000000003c8 CR3: 000000103fd6e000 CR4: 00000000000006e0
    Process ifup (pid: 18125, threadinfo ffff811032d44000, task ffff81103908a820)
    Stack:  000000000000003c ffff81083f0f47d0 ffff81043eee8dc0 000000000000003c
     ffff81083f0f0500 ffffffff88342f95 0000000000000000 0000004000000300
     ffff81000f128060 0000000000000082 ffff81000f128064 000000008009906d
    Call Trace:
     <IRQ>  [<ffffffff88342f95>] :be2net:be_poll_rx+0x265/0x849
     [<ffffffff88343608>] :be2net:be_poll+0x8f/0x11e
     [<ffffffff8006eb88>] do_timer_jiffy+0x9/0x1d
     [<ffffffff883443ba>] :be2net:be_poll_compat+0x24/0x3d
     [<ffffffff8000c979>] net_rx_action+0xac/0x1b3
     [<ffffffff883451eb>] :be2net:be_msix+0x5b/0x80
     [<ffffffff80012464>] __do_softirq+0x89/0x133
     [<ffffffff8005e2fc>] call_softirq+0x1c/0x28
     [<ffffffff8006d5f5>] do_softirq+0x2c/0x7d
     [<ffffffff8006d485>] do_IRQ+0xec/0xf5
     [<ffffffff8005d615>] ret_from_intr+0x0/0xa
     <EOI>  [<ffffffff80064ac8>] _spin_lock_irqsave+0x0/0xd
     [<ffffffff80022214>] __up_read+0x19/0x7f
     [<ffffffff8006723e>] do_page_fault+0x4fe/0x874
     [<ffffffff80028d4f>] do_sigaction+0x76/0x199
     [<ffffffff8005dde9>] error_exit+0x0/0x84
    
    
    Code: 48 8b 86 c8 03 00 00 48 85 c0 0f 84 a6 01 00 00 48 83 78 18 
    RIP  [<ffffffff88306c20>] :8021q:vlan_gro_frags+0x2d/0x21d
     RSP <ffffffff804a5de0>
    
  • Usually this panic occurs on hosts configured for virtualization (KVM, Xen or RHEV).

Environment

  • Red Hat Enterprise Linux 5.4 or later.
  • Network interface with:
    • GRO (generic receive offload) enabled.
    • VLANs defined.
    • Promiscuous mode.

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In