hammer ping shows pulp failure with `500 Internal Server Error` and `mod_wsgi (pid=18246): Exception occurred processing WSGI script '/srv/pulp/webservices.wsgi'` error is observed.

Solution Verified - Updated -

Environment

  • Red Hat Satellite 6.1

Issue

  • Getting We're sorry, but something went wrong error while accessing Content Hosts page from Satellite 6 WebUI.

  • Following is the output of hammer ping command:

    # hammer ping
   candlepin:
   Status:          ok
   Server Response: Duration: 45ms
   candlepin_auth:
   Status:          ok
   Server Response: Duration: 45ms
   pulp:
   Status:          FAIL
   Server Response: Message: 500 Internal Server Error
   elasticsearch:
   Status:          ok
   Server Response: Duration: 37ms
   foreman_tasks:
   Status:          ok
   Server Response: Duration: 1ms

  • Below errors are found in /var/log/messages file.

    mod_wsgi (pid=18246): Exception occurred processing WSGI script '/srv/pulp/webservices.wsgi'.
 Traceback (most recent call last):
   File "/srv/pulp/webservices.wsgi", line 16, in <module>
     from pulp.server.webservices.application import wsgi_application
   File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 42, in <module>
     logs.start_logging()
   File "/usr/lib/python2.6/site-packages/pulp/server/logs.py", line 67, in start_logging
     facility=CompliantSysLogHandler.LOG_DAEMON)
   File "/usr/lib64/python2.6/logging/handlers.py", line 721, in __init__
     self._connect_unixsocket(address)
   File "/usr/lib64/python2.6/logging/handlers.py", line 737, in _connect_unixsocket
     self.socket.connect(address)
   File "<string>", line 1, in connect
 error: [Errno 2] No such file or directory

Resolution

  • Restore the original /etc/rsyslog.conf file and restart the katello services.

    # katello-service restart

  • Ensure the SELinux context for /dev/log is as follows:

    # ls -lZ /dev/log
  srw-rw-rw-. root root system_u:object_r:devlog_t:s0    /dev/log

Root Cause

  • Some manual changes were made in /etc/rsyslog.conf file and the system was rebooted; after that, the /dev/log directory is missing.

    # ls -lZ /dev/log
  ls: cannot access /dev/log: No such file or directory

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments