Red Hat Openshift Online - [ALERT] Proxy express has no server available
Environment
- Red Hat OpenShift Online
Issue
- My servers are down and I am unable to bring them up:
[ALERT] 005/015432 (350193) : proxy 'express' has no server available!
[WARNING] 005/020435 (350193) : Server express/local-gear is DOWN for maintenance.
[WARNING] 005/020435 (472765) : config : log format ignored for proxy 'stats' since it has no log address.
[WARNING] 005/020435 (472765) : config : log format ignored for proxy 'express' since it has no log address.
[WARNING] 005/020435 (472765) : Server express/local-gear is DOWN, reason: Layer7 wrong status, code: 404, info: "Not Found", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 005/020435 (472765) : proxy 'express' has no server available!
[WARNING] 005/021114 (472765) : Server express/local-gear is DOWN for maintenance.
[WARNING] 005/021124 (472765) : Server express/local-gear is UP (leaving maintenance).
[WARNING] 005/021130 (472765) : Server express/local-gear is DOWN, reason: Layer7 wrong status, code: 404, info: "Not Found", check duration: 4875ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 005/021130 (472765) : proxy 'express' has no server available!
- Cannot access or login to my high availability application
- Gear is stuck down for maintenance
Resolution
Ensure that the HAProxy is able to perform health checks on the application:
sshinto the main gear of the application.
$ rhc ssh [appname]
- Update the
haproxy conf: change thehttpchk GEToption from/to a valid location, such as anindex.htmlorindex.jspfile
$ cd ~/haproxy/conf
$ vim haproxy.cfg
. . .
option httpchk GET /path/to/viable/resource
. . .
- Restart the
HAProxy
$ rhc cartridge-restart --cartridge haproxy
- Verify the status of your gears by going to http://
- .rhcloud.com/haproxy-status
Root Cause
Applications with scaling enabled utilize a High Availability Proxy (HAProxy) to perform health checks and other maintenance related to scaling. The health check URL identified in haproxy.cfg is contacted via the loopback IP on the gear. If the health check URL is inaccessible or otherwise non-valid (does not return a 200 status message on a curl over the loopback interface), the gear will fail the health check.
Diagnostic Steps
- Verify the status of your gears by going to http://
- .rhcloud.com/haproxy-status - Investigate the location pointed to by the
httpchk GEToption in thehaproxy/conf/haproxy.cfgfile
$ curl <location indicated by the httpchk GET option>
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
