500 internal server error from APIs and web GUI

Solution In Progress - Updated -

Issue

  • Getting 500 Internal Server Errors from the api and webui like:
$ grep ' 500 ' httpd-logs/httpd/ssl_access_log| grep 11/Jan/2016:11
xxxx - - [11/Jan/2016:11:00:01 -0700] "POST /rpc/api HTTP/1.0" 500 2846

$ grep "Mon Jan 11 11" httpd-logs/httpd/ssl_error_log 
[Mon Jan 11 11:10:01 2016] [error] ajp_read_header: ajp_ilink_receive failed
[Mon Jan 11 11:10:01 2016] [error] (70007)The timeout specified has expired: proxy: read response failed from (null) (localhost)
[Mon Jan 11 11:29:51 2016] [error] [client xxxx] mod_wsgi (pid=26241): Exception occurred processing WSGI script '/usr/share/rhn/wsgi/xmlrpc.py'.
[Mon Jan 11 11:29:51 2016] [error] [client xxxx] Traceback (most recent call last):
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/share/rhn/wsgi/xmlrpc.py", line 22, in application
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     return wsgiHandler.handle(environ, start_response, "xmlrpc", "server.xmlrpc")
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/share/rhn/wsgi/wsgiHandler.py", line 37, in handle
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     ret = parseServ(req)
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/lib/python2.6/site-packages/spacewalk/server/apacheHandler.py", line 80, in headerParserHandler
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     rhnSQL.initDB()
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/__init__.py", line 142, in initDB
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     __init__DB(backend, host, port, username, password, database, sslmode, sslrootcert)
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/__init__.py", line 56, in __init__DB
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     __DB.connect()
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/driver_cx_Oracle.py", line 390, in connect
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     log_debug(1, "Connecting to database", self.dbtxt)
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/lib/python2.6/site-packages/spacewalk/common/rhnLog.py", line 116, in log_debug
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     LOG.logMessage(*args)
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/lib/python2.6/site-packages/spacewalk/common/rhnLog.py", line 205, in logMessage
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     self.writeMessage(msg)
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/lib/python2.6/site-packages/spacewalk/common/rhnLog.py", line 213, in writeMessage
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     self.writeToLog(msg)
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]   File "/usr/lib/python2.6/site-packages/spacewalk/common/rhnLog.py", line 219, in writeToLog
[Mon Jan 11 11:29:51 2016] [error] [client xxxx]     self.fd.write("%s\\n" % msg)
[Mon Jan 11 11:29:51 2016] [error] [client xxxx] IOError: [Errno 28] No space left on device
[Mon Jan 11 11:29:52 2016] [error] [client xxxx] mod_wsgi (pid=26243): Exception occurred processing WSGI script '/usr/share/rhn/wsgi/xmlrpc.py'.
[Mon Jan 11 11:29:52 2016] [error] [client xxxx] Traceback (most recent call last):
[Mon Jan 11 11:29:52 2016] [error] [client xxxx]   File "/usr/share/rhn/wsgi/xmlrpc.py", line 22, in application
[Mon Jan 11 11:29:52 2016] [error] [client xxxx]     return wsgiHandler.handle(environ, start_response, "xmlrpc", "server.xmlrpc")
[Mon Jan 11 11:29:52 2016] [error] [client xxxx]   File "/usr/share/rhn/wsgi/wsgiHandler.py", line 37, in handle
[Mon Jan 11 11:29:52 2016] [error] [client xxxx]     ret = parseServ(req)

Environment

  • Red Hat Satellite 5.7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.