soft lockup with _raw_spin_lock in ovs_flow_stats_update

Solution Unverified - Updated -

Issue

  • A soft lockup occurred.
BUG: soft lockup - CPU#3 stuck for 22s! [revalidator_106:28613]
Modules linked in: vfio_pci vfio_iommu_type1 vfio binfmt_misc nls_utf8 isofs loop nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_mac xt_physdev vhost_net macvtap macvlan tun veth openvswitch vxlan ip_tunnel gre ipt_REJECT xt_comment xt_conntrack xt_multiport iptable_filter ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_CHECKSUM iptable_mangle ip_tables vfat fat coretemp kvm_intel kvm bnx2x mdio tg3 iTCO_wdt iTCO_vendor_support snd_hda_intel snd_hda_codec crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul snd_hwdep i2c_i801 snd_seq ipmi_devintf snd_seq_device ptp snd_pcm pps_core glue_helper ablk_helper cryptd snd_page_alloc snd_timer snd soundcore hpilo hpwdt shpchp pcspkr lpc_ich mfd_core
ioatdma acpi_power_meter dca ipmi_si ipmi_msghandler mperf bridge stp llc xfs libcrc32c sd_mod crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm drm hpsa i2c_core wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 28613 Comm: revalidator_106 Tainted: G        W   --------------   3.10.0-123.el7.x86_64 #1
Hardware name: HP ProLiant DL380 Gen9, BIOS P89 11/03/2014
task: ffff882025d471c0 ti: ffff882026d36000 task.ti: ffff882026d36000
RIP: 0010:[<ffffffff815e90ea>]  [<ffffffff815e90ea>] _raw_spin_lock+0x3a/0x50
RSP: 0018:ffff88103f8c3ae8  EFLAGS: 00000206
RAX: 0000000000001ab1 RBX: 0000000000000000 RCX: 000000000000005e
RDX: 0000000000000060 RSI: 0000000000000060 RDI: ffff881026614858
RBP: ffff88103f8c3ae8 R08: ffff8810117d92d0 R09: ffff88103f8c3a30
R10: 0000000000000010 R11: ffff8810273f0000 R12: ffff88103f8c3a58
R13: ffffffff815f2d9d R14: ffff88103f8c3ae8 R15: ffff881026614840
FS:  00007fa18e7fc700(0000) GS:ffff88103f8c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa17400ce30 CR3: 0000002025c19000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88103f8c3b18 ffffffffa03ff29f ffff881023d8e200 ffffe8f0000c29b0
ffff8810264fed80 ffff88103f8c3b50 ffff88103f8c3c00 ffffffffa03fe9a4
000000043b33536a ffff880fc94bf2c4 ffff881023d8f200 ffff881023d8f200
Call Trace:
<IRQ> 

[<ffffffffa03ff29f>] ovs_flow_stats_update+0x4f/0xd0 [openvswitch]
[<ffffffffa03fe9a4>] ovs_dp_process_received_packet+0x84/0x120 [openvswitch]
[<ffffffffa040501a>] ovs_vport_receive+0x2a/0x30 [openvswitch]
[<ffffffffa0405f21>] netdev_frame_hook+0xc1/0x120 [openvswitch]
[<ffffffff814cf9c2>] __netif_receive_skb_core+0x282/0x870
[<ffffffff8101a0d9>] ? read_tsc+0x9/0x20
[<ffffffff814cffc8>] __netif_receive_skb+0x18/0x60
[<ffffffff814d0050>] netif_receive_skb+0x40/0xd0
[<ffffffff814d0aa8>] napi_gro_receive+0x58/0x80
[<ffffffffa05e89fc>] bnx2x_rx_int+0xfec/0x1860 [bnx2x]
[<ffffffff812b8655>] ? cpumask_next_and+0x35/0x50
[<ffffffff8101a0d9>] ? read_tsc+0x9/0x20
[<ffffffff810b6513>] ? ktime_get+0x43/0xc0
[<ffffffff810392f3>] ? lapic_next_deadline+0x33/0x40
[<ffffffff810b007e>] ? swsusp_save+0x1fe/0x280
[<ffffffffa05eb01a>] bnx2x_poll+0xfa/0x3c0 [bnx2x]
[<ffffffff814d041a>] net_rx_action+0x15a/0x250
[<ffffffff81067047>] __do_softirq+0xf7/0x290
[<ffffffff815f3a5c>] call_softirq+0x1c/0x30
[<ffffffff81014d25>] do_softirq+0x55/0x90
[<ffffffff810673e5>] irq_exit+0x115/0x120
[<ffffffff815f4358>] do_IRQ+0x58/0xf0
[<ffffffff815e94ad>] common_interrupt+0x6d/0x6d
<EOI> 

Environment

  • Red Hat Enterprise Linux 7.0
  • openvswitch

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content