Possible network outage during neutron restart
Issue
In our environment we have HA with pacemaker. When we restarted the mysql to change the certificate, using pacemaker, neutron was restarted automatically. This operation HAVEN'T performed a neutron fail-over to the other node, because the other node was in standby mode. As far as we know, this type of restart doesn't impacts in the network connectivity.
2016-02-02 08:19:42.864 20865 INFO neutron.service [-] Neutron service started, listening on 0.0.0.0:9696
During all this time, we were doing connectivity test from one instance in our environment to Internet. NO packet loss in all the intervention.
But, one of our tenants have complained about connectivity problems. They have 4 mongodb nodes. These nodes uses floating IPs in our ext_inet network to connect with the other ones. They have experienced network outages during the neutron restart, as can be see in these logs, e.g.
2016-02-02T08:20:04.205+0100 [rsHealthPoll] couldn't connect to pro-pd-mongo-dev-02:27097: couldn't connect to server pro-pd-mongo-dev-02:27097 (x.x.x.247) failed, connection attempt failed
2016-02-02T08:20:09.206+0100 [rsHealthPoll] replSet info pro-pd-mongo-dev-02:27097 is down (or slow to respond):
2016-02-02T08:20:09.206+0100 [rsHealthPoll] replSet member pro-pd-mongo-dev-02:27097 is now in state DOWN
2016-02-02T08:20:10.147+0100 [rsHealthPoll] warning: Failed to connect to x.x.x.178:27097, reason: errno:113 No route to host
We know that the connectivity from our instances to Internet using this ext_inet network wasn't affected, because we were doing tests. But we weren't doing tests from Internet to the floating public IPs.
So, it's possible that the neutron restarts affected in some way to these public floating IPs?
Environment
Red Hat OpenStack
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.