soft lockup with _raw_spin_lock in ovs_flow_stats_update
Issue
- A soft lockup occurred.
BUG: soft lockup - CPU#3 stuck for 22s! [revalidator_106:28613]
Modules linked in: vfio_pci vfio_iommu_type1 vfio binfmt_misc nls_utf8 isofs loop nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_mac xt_physdev vhost_net macvtap macvlan tun veth openvswitch vxlan ip_tunnel gre ipt_REJECT xt_comment xt_conntrack xt_multiport iptable_filter ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_CHECKSUM iptable_mangle ip_tables vfat fat coretemp kvm_intel kvm bnx2x mdio tg3 iTCO_wdt iTCO_vendor_support snd_hda_intel snd_hda_codec crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul snd_hwdep i2c_i801 snd_seq ipmi_devintf snd_seq_device ptp snd_pcm pps_core glue_helper ablk_helper cryptd snd_page_alloc snd_timer snd soundcore hpilo hpwdt shpchp pcspkr lpc_ich mfd_core
ioatdma acpi_power_meter dca ipmi_si ipmi_msghandler mperf bridge stp llc xfs libcrc32c sd_mod crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm drm hpsa i2c_core wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 28613 Comm: revalidator_106 Tainted: G W -------------- 3.10.0-123.el7.x86_64 #1
Hardware name: HP ProLiant DL380 Gen9, BIOS P89 11/03/2014
task: ffff882025d471c0 ti: ffff882026d36000 task.ti: ffff882026d36000
RIP: 0010:[<ffffffff815e90ea>] [<ffffffff815e90ea>] _raw_spin_lock+0x3a/0x50
RSP: 0018:ffff88103f8c3ae8 EFLAGS: 00000206
RAX: 0000000000001ab1 RBX: 0000000000000000 RCX: 000000000000005e
RDX: 0000000000000060 RSI: 0000000000000060 RDI: ffff881026614858
RBP: ffff88103f8c3ae8 R08: ffff8810117d92d0 R09: ffff88103f8c3a30
R10: 0000000000000010 R11: ffff8810273f0000 R12: ffff88103f8c3a58
R13: ffffffff815f2d9d R14: ffff88103f8c3ae8 R15: ffff881026614840
FS: 00007fa18e7fc700(0000) GS:ffff88103f8c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa17400ce30 CR3: 0000002025c19000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88103f8c3b18 ffffffffa03ff29f ffff881023d8e200 ffffe8f0000c29b0
ffff8810264fed80 ffff88103f8c3b50 ffff88103f8c3c00 ffffffffa03fe9a4
000000043b33536a ffff880fc94bf2c4 ffff881023d8f200 ffff881023d8f200
Call Trace:
<IRQ>
[<ffffffffa03ff29f>] ovs_flow_stats_update+0x4f/0xd0 [openvswitch]
[<ffffffffa03fe9a4>] ovs_dp_process_received_packet+0x84/0x120 [openvswitch]
[<ffffffffa040501a>] ovs_vport_receive+0x2a/0x30 [openvswitch]
[<ffffffffa0405f21>] netdev_frame_hook+0xc1/0x120 [openvswitch]
[<ffffffff814cf9c2>] __netif_receive_skb_core+0x282/0x870
[<ffffffff8101a0d9>] ? read_tsc+0x9/0x20
[<ffffffff814cffc8>] __netif_receive_skb+0x18/0x60
[<ffffffff814d0050>] netif_receive_skb+0x40/0xd0
[<ffffffff814d0aa8>] napi_gro_receive+0x58/0x80
[<ffffffffa05e89fc>] bnx2x_rx_int+0xfec/0x1860 [bnx2x]
[<ffffffff812b8655>] ? cpumask_next_and+0x35/0x50
[<ffffffff8101a0d9>] ? read_tsc+0x9/0x20
[<ffffffff810b6513>] ? ktime_get+0x43/0xc0
[<ffffffff810392f3>] ? lapic_next_deadline+0x33/0x40
[<ffffffff810b007e>] ? swsusp_save+0x1fe/0x280
[<ffffffffa05eb01a>] bnx2x_poll+0xfa/0x3c0 [bnx2x]
[<ffffffff814d041a>] net_rx_action+0x15a/0x250
[<ffffffff81067047>] __do_softirq+0xf7/0x290
[<ffffffff815f3a5c>] call_softirq+0x1c/0x30
[<ffffffff81014d25>] do_softirq+0x55/0x90
[<ffffffff810673e5>] irq_exit+0x115/0x120
[<ffffffff815f4358>] do_IRQ+0x58/0xf0
[<ffffffff815e94ad>] common_interrupt+0x6d/0x6d
<EOI>
Environment
- Red Hat Enterprise Linux 7.0
- openvswitch
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.