Failed to create DPDK port

Solution In Progress - Updated -

Issue

  • Instance fails to spawn with the following error:
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa] Traceback (most recent call last):
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6233, in _create_domain_and_network
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]     network_info)
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]   File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]     next(self.gen)
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 478, in wait_for_instance_event
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]     actual_event = event.wait()
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]   File "/usr/lib/python3.6/site-packages/eventlet/event.py", line 125, in wait
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]     result = hub.switch()
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]   File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 298, in switch
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]     return self.greenlet.switch()
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa] eventlet.timeout.Timeout: 300 seconds
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa] 
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa] During handling of the above exception, another exception occurred:
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa] 
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa] Traceback (most recent call last):
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2645, in _build_resources
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]     yield resources
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2419, in _build_and_run_instance
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]     block_device_info=block_device_info)
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3474, in spawn
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]     power_on=power_on)
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6254, in _create_domain_and_network
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa]     raise exception.VirtualInterfaceCreateException()
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2020-05-07 18:33:30.185 7 ERROR nova.compute.manager [instance: e7674f5f-7076-4cb1-b6eb-0ce83ff9d2fa] 
  • The following errors are seen in /var/log/openvswitch/ovs-vswitchd.log:
2020-05-07T18:28:24.258Z|00213|dpdk|INFO|VHOST_CONFIG: vhost-user client: socket created, fd: 9
2020-05-07T18:28:24.258Z|00214|netdev_dpdk|INFO|vHost User device 'vhu301f908b-37' created in 'client' mode, using client socket '/var/lib/vhost_sockets/vhu301f908b-37'
2020-05-07T18:28:24.260Z|00215|dpdk|WARN|VHOST_CONFIG: failed to connect to /var/lib/vhost_sockets/vhu301f908b-37: No such file or directory
2020-05-07T18:28:24.260Z|00216|dpdk|INFO|VHOST_CONFIG: /var/lib/vhost_sockets/vhu301f908b-37: reconnecting...
2020-05-07T18:28:24.361Z|00217|netdev_dpdk|ERR|Failed to create mempool "ovs7e2a6c3500021580262144" with a request of 262144 mbufs
2020-05-07T18:28:24.361Z|00218|netdev_dpdk|ERR|Failed to create memory pool for netdev vhu301f908b-37, with MTU 1500 on socket 0: File name too long
2020-05-07T18:28:24.361Z|00219|dpif_netdev|ERR|Failed to set interface vhu301f908b-37 new configuration
2020-05-07T18:28:24.361Z|00220|dpif_netdev|INFO|Core 73 on numa node 1 assigned port 'dpdk1' rx queue 0 (measured processing cycles 5616064).
2020-05-07T18:28:24.361Z|00221|dpif_netdev|INFO|Core 1 on numa node 0 assigned port 'dpdk0' rx queue 0 (measured processing cycles 4158580).
2020-05-07T18:28:24.361Z|00222|bridge|WARN|could not add network device vhu301f908b-37 to ofproto (No such device)
2020-05-07T18:28:24.372Z|00223|netdev_linux|INFO|Dropped 18 log messages in last 126391 seconds (most recently, 126391 seconds ago) due to excessive rate
...
2020-05-07T18:28:24.501Z|00242|netdev_dpdk|ERR|Failed to create mempool "ovs7e2a6c3500021580262144" with a request of 262144 mbufs
2020-05-07T18:28:24.501Z|00243|netdev_dpdk|ERR|Failed to create memory pool for netdev vhu301f908b-37, with MTU 1500 on socket 0: File name too long
2020-05-07T18:28:24.501Z|00244|dpif_netdev|ERR|Failed to set interface vhu301f908b-37 new configuration
2020-05-07T18:28:24.501Z|00245|dpif_netdev|INFO|Core 73 on numa node 1 assigned port 'dpdk1' rx queue 0 (measured processing cycles 5616064).
2020-05-07T18:28:24.501Z|00246|dpif_netdev|INFO|Core 1 on numa node 0 assigned port 'dpdk0' rx queue 0 (measured processing cycles 4158580).
2020-05-07T18:28:24.501Z|00247|bridge|WARN|could not add network device vhu301f908b-37 to ofproto (No such device)
2020-05-07T18:28:24.504Z|00248|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on vxlan_sys_4789 device failed: No such device
2020-05-07T18:28:24.506Z|00249|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on vxlan_sys_4789 device failed: No such device
2020-05-07T18:28:25.260Z|00001|dpdk|INFO|VHOST_CONFIG: /var/lib/vhost_sockets/vhu301f908b-37: connected

Environment

  • Red Hat OpenStack Platform 16.0 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In