HAproxy routers performance seems to deteriorate over time and haproxy mapping files doesn't match
Issue
- We have two haproxy routers deployed with the same routes (each one of them is in a different node), when a route is published it's published almost instantly in
etcd
, thenroutes.json
in each node and thenhaproxy.config
and mapping files are reloaded as expected. - Over time this performance seems to deteriorate:
- Routes aren't published that fast, it takes at least 2-5 minutes to publish routes in one haproxy, and 5-10 minutes in the other. For some reason it seems one haproxy is always refreshing its configuration taking twice the time.
- HAproxy mapping files doesn't match. After a router restart the mapping files shows the same routes (as expected), but when the performance decreases there are routes published in one haproxy that aren't published in the other, causing we can't reach some pods when accessing through the haproxy with missing routes.
- Workaround is restarting haproxys routers, then they work as expected.
Environment
- OpenShift 3.0
-OpenStack - Red Hat Enterprise Linux (RHEL) 7.1
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.