neutron namespace cleanup is failing

Solution Verified - Updated -

Issue

  • Every now and then, neutron-netns-cleanupbreaks:
()[root@overcloud-controller-0 /]# neutron-netns-cleanup
2019-10-04 00:44:24.546 589450 INFO neutron.common.config [-] Logging enabled!
2019-10-04 00:44:24.546 589450 INFO neutron.common.config [-] /usr/bin/neutron-netns-cleanup version 12.0.6
2019-10-04 00:44:24.547 589450 INFO oslo.privsep.daemon [-] Running privsep helper: ['sudo', 'privsep-helper', '--privsep_context', 'neutron.privileged.default', '--privsep_sock_path', '/tmp/tmph30jXp/privsep.sock']
2019-10-04 00:44:24.989 589450 INFO oslo.privsep.daemon [-] Spawned new privsep daemon via rootwrap
2019-10-04 00:44:24.938 589866 INFO oslo.privsep.daemon [-] privsep daemon starting
2019-10-04 00:44:24.941 589866 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
2019-10-04 00:44:24.943 589866 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN/none
2019-10-04 00:44:24.943 589866 INFO oslo.privsep.daemon [-] privsep daemon running as pid 589866
2019-10-04 00:44:25.361 589450 ERROR neutron.agent.linux.utils [-] Exit code: 1; Stdin: ; Stdout: ; Stderr: setting the network namespace "qdhcp-ad638670-6128-4ed4-9864-ba654993e0c5" failed: Invalid argument

2019-10-04 00:44:25.363 589450 CRITICAL neutron [-] Unhandled error: ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: ; Stderr: setting the network namespace "qdhcp-ad638670-6128-4ed4-9864-ba654993e0c5" failed: Invalid argument
2019-10-04 00:44:25.363 589450 ERROR neutron Traceback (most recent call last):
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/bin/neutron-netns-cleanup", line 10, in <module>
2019-10-04 00:44:25.363 589450 ERROR neutron     sys.exit(main())
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/cmd/netns_cleanup.py", line 290, in main
2019-10-04 00:44:25.363 589450 ERROR neutron     cleanup_network_namespaces(conf)
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/cmd/netns_cleanup.py", line 260, in cleanup_network_namespaces
2019-10-04 00:44:25.363 589450 ERROR neutron     if eligible_for_deletion(conf, ns, conf.force)]
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/cmd/netns_cleanup.py", line 117, in eligible_for_deletion
2019-10-04 00:44:25.363 589450 ERROR neutron     return force or ip.namespace_is_empty()
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 212, in namespace_is_empty
2019-10-04 00:44:25.363 589450 ERROR neutron     return not self.get_devices()
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 139, in get_devices
2019-10-04 00:44:25.363 589450 ERROR neutron     return []
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-10-04 00:44:25.363 589450 ERROR neutron     self.force_reraise()
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-10-04 00:44:25.363 589450 ERROR neutron     six.reraise(self.type_, self.value, self.tb)
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 132, in get_devices
2019-10-04 00:44:25.363 589450 ERROR neutron     log_fail_as_error=self.log_fail_as_error).split()
2019-10-04 00:44:25.363 589450 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 147, in execute
2019-10-04 00:44:25.363 589450 ERROR neutron     returncode=returncode)
2019-10-04 00:44:25.363 589450 ERROR neutron ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: ; Stderr: setting the network namespace "qdhcp-ad638670-6128-4ed4-9864-ba654993e0c5" failed: Invalid argument
2019-10-04 00:44:25.363 589450 ERROR neutron 
2019-10-04 00:44:25.363 589450 ERROR neutron

Environment

  • Red Hat OpenStack Platform 13.0 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In