Repeated vmxnet3 tx hang and resetting

Solution Verified - Updated -

Issue

  • BUG_ON, softlockup hangs, hung tasks, and WARNING: at net/sched/sch_generic.c:261 dev_watchdog+0x26b/0x280() (Not tainted) with VMware VMs
  • Repeated vmxnet3 TX hang, with the following printed in logs:

    NETDEV WATCHDOG: eth0 (vmxnet3): transmit queue 1 timed out
    vmxnet3 0000:0b:00.0 eth0: tx hang
    vmxnet3 0000:0b:00.0 eth0: resetting
    vmxnet3 0000:0b:00.0 eth0: tx hang
    
  • Kernel panic during driver resetting NIC at:

    kernel BUG at drivers/net/vmxnet3/vmxnet3_drv.c:756!
    RIP: 0010:[<ffffffffa0091869>]  [<ffffffffa0091869>] vmxnet3_xmit_frame+0x9e9/0xe90 [vmxnet3]
    Call Trace:
     <IRQ> 
     [<ffffffff81461934>] dev_hard_start_xmit+0x224/0x480
     [<ffffffff8147e5da>] sch_direct_xmit+0x15a/0x1c0
     [<ffffffff8147e6ab>] __qdisc_run+0x6b/0xe0
     [<ffffffff81459910>] net_tx_action+0x130/0x1c0
     [<ffffffff8107d8b1>] __do_softirq+0xc1/0x1e0
     [<ffffffff810b034a>] ? tick_program_event+0x2a/0x30
     [<ffffffff8100c30c>] call_softirq+0x1c/0x30
     [<ffffffff8100fb55>] do_softirq+0x65/0xa0
     [<ffffffff8107d765>] irq_exit+0x85/0x90
     [<ffffffff81533bca>] smp_apic_timer_interrupt+0x4a/0x60
     [<ffffffff8100bb93>] apic_timer_interrupt+0x13/0x20
     <EOI> 
    RIP  [<ffffffffa0091869>] vmxnet3_xmit_frame+0x9e9/0xe90 [vmxnet3]
    
  • Intermittent network connection and loss of network connectivity on VMware VM

Environment

  • Red Hat Enterprise Linux 6, seen on
    • kernel-2.6.32-504.3.3.el6.x86_64
    • kernel-2.6.32-573.7.1.el6.x86_64
  • Red Hat Enterprise Linux 7, seen on
    • kernel-3.10.0-514.el7.x86_64
  • Other kernel versions may be affected
  • VMware virtualization
    • ESXi 5.5.0 build 2403361
    • EXSi 5.5.0 build 3568722
    • ESXi 6.0 P02
  • vmxnet3 paravirtualized network interface

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In