ovs-vswitchd is crashing in loop with a Segmentation Fault

Solution In Progress - Updated -

Issue

  • VMs connectivity on DPDK deployment on RHOSP16.1 doesn't work

  • ovs-vswitchd crashes in loop with:

ovs-vswitchd.log:2021-07-12T04:05:30.635Z|00002|daemon_unix|ERR|fork child died before signaling startup (killed (Segmentation fault))
ovs-vswitchd.log:2021-07-12T09:13:00.083Z|00001|fatal_signal(pmd-c21/id:9)|WARN|terminating with    signal 11 (Segmentation fault)
ovs-vswitchd.log:2021-07-12T11:23:02.073Z|00001|fatal_signal(pmd-c21/id:9)|WARN|terminating with signal 11 (Segmentation fault)
ovs-vswitchd.log:2021-07-12T14:40:04.137Z|00001|fatal_signal(pmd-c21/id:9)|WARN|terminating with signal 11 (Segmentation fault)
ovs-vswitchd.log:2021-07-12T17:58:10.211Z|00001|fatal_signal(pmd-c21/id:9)|WARN|terminating with signal 11 (Segmentation fault)
ovs-vswitchd.log:2021-07-12T18:15:23.894Z|00002|daemon_unix|ERR|fork child died before signaling startup (killed (Segmentation fault))
ovs-vswitchd.log:2021-07-12T18:15:32.653Z|00002|daemon_unix|ERR|fork child died before signaling startup (killed (Segmentation fault))
ovs-vswitchd.log:2021-07-12T18:36:04.115Z|00002|daemon_unix|ERR|fork child died before signaling startup (killed (Segmentation fault))
ovs-vswitchd.log:2021-07-12T18:36:08.581Z|00002|daemon_unix|ERR|fork child died before signaling startup (killed (Segmentation fault))
ovs-vswitchd.log:2021-07-12T18:37:41.564Z|00001|fatal_signal(pmd-c57/id:7)|WARN|terminating with signal 11 (Segmentation fault)
ovs-vswitchd.log:2021-07-12T18:37:42.678Z|00002|daemon_unix|ERR|fork child died before signaling startup (killed (Segmentation fault))
ovs-vswitchd.log:2021-07-12T19:09:43.539Z|00001|fatal_signal(pmd-c21/id:3)|WARN|terminating with signal 11 (Segmentation fault)
ovs-vswitchd.log:2021-07-12T19:28:08.718Z|00002|daemon_unix|ERR|fork child died before signaling startup (killed (Segmentation fault))
  • Running gdb against a core file returns the following backtrace:
(gdb) bt
#0  dp_packet_set_size (v=7236, b=0x0) at ../lib/dp-packet.h:495
#1  netdev_linux_batch_rxq_recv_sock (rx=<optimized out>, mtu=<optimized out>, batch=0x7ffdc1774780) at ../lib/netdev-linux.c:1293
#2  0x000055664fe2bd1f in netdev_linux_rxq_recv (rxq_=0x556651ffd830, batch=0x7ffdc1774780, qfill=0x0) at ../lib/netdev-linux.c:1493
#3  0x000055664fd7ddd5 in netdev_rxq_recv (rx=<optimized out>, batch=batch@entry=0x7ffdc1774780, qfill=<optimized out>) at ../lib/netdev.c:726
#4  0x000055664fd4ee89 in dp_netdev_process_rxq_port (pmd=pmd@entry=0x7f0dbaf89010, rxq=0x556651ffd7a0, port_no=13) at ../lib/dpif-netdev.c:4461
#5  0x000055664fd4fc7c in dpif_netdev_run (dpif=<optimized out>) at ../lib/dpif-netdev.c:5469
#6  0x000055664fd03f0c in type_run (type=<optimized out>) at ../ofproto/ofproto-dpif.c:370
#7  0x000055664fcebe0f in ofproto_type_run (datapath_type=datapath_type@entry=0x5566520ab3a0 "netdev") at ../ofproto/ofproto.c:1772
#8  0x000055664fcd955c in bridge_run__ () at ../vswitchd/bridge.c:3242
#9  0x000055664fcdfccd in bridge_run () at ../vswitchd/bridge.c:3307
#10 0x000055664f9a9f05 in main (argc=<optimized out>, argv=<optimized out>) at ../vswitchd/ovs-vswitchd.c:127

Environment

  • Red Hat OpenStack Platform 16.1 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content