Chapter 10. Troubleshooting
tail -fcommand for all log files and then run
yum list. You should then examine all new log entries for potential clues.
# df -h
# /usr/sbin/rhn-satellite status
# service httpd status
- 10.1. Installing and Updating
- 10.2. Services
- Q: Why isn't the Apache Web server running?
- Q: How do I find out what the status of the Red Hat Network Task Engine is?
- Q: How do I find out what the status of the Satellite's Embedded Database is?
- Q: What do I do if yum, up2date, or the push capability of the Red Hat Network Satellite stops working?
- 10.3. Connectivity
- 10.4. Logging and Reporting
- 10.5. Errors
- Q: I'm getting an "Error validating satellite certificate" error during Red Hat Network Satellite installation. How do I fix it?
- Q: I'm getting an "ERROR: server.mount_point not set in the configuration file" error when I try to activate or synchronize the Red Hat Network Satellite. How do I fix it?
- Q: Why does cobbler check give an error saying that it needs a different version of yum-utils?
- Q: I'm getting a "unsupported version" error when I try to activate the Red Hat Network Satellite certificate. How do I fix it?
- Q: I'm getting an "Internal Server Error" complaining about ASCII when I try to edit the kickstart profile. What's going on?
- Q: I'm getting "Host Not Found" or "Could Not Determine FQDN" errors. What do I do now?
- Q: I'm getting a "This server is not an entitled Satellite" when I try to synchronize the Red Hat Network Satellite server. How do fix it?
10.1. Installing and Updating
audit.logfiles available so that Red Hat Support personnel can assist you. You can find the file in
/var/log/audit/audit.logand can attach the file to your Support ticket for engineers to assist you.
/var/satelliteto an NFS mount, and now SELinux is stopping it working properly. What do I need to do?
# /usr/sbin/setsebool -P spacewalk_nfs_mountpoint on
# /usr/sbin/setsebool -P cobbler_use_nfs on
- Red Hat Developer Suite
- Red Hat Application Server
- Red Hat Extras
- JBoss product channels
/etc/hostsfile may be incorrect.
# service taskomatic status
# service oracle status
up2date, or the push capability of the Red Hat Network Satellite stops working?
up2date, or the push capability of the Red Hat Network Satellite ceases to function, it is possible that old log files may be at fault. Stop the jabberd daemon before removing these files. To do so, issue the following commands as root:
# service jabberd stop # rm -f /var/lib/jabberd/db/_db* # service jabberd start
- Attempt to connect to the Red Hat Network Satellite's database at the command line using the correct connection string as found in
# sqlplus username/password@sid
- Ensure the Red Hat Network Satellite is using Network Time Protocol (NTP) and set to the appropriate time zone. This also applies to all client systems and the separate database machine in Red Hat Network Satellite with Stand-Alone Database.
- Confirm the correct package:
rhn-org-httpd-ssl-key-pair-MACHINE_NAME-VER-REL.noarch.rpmis installed on the Red Hat Network Satellite and the corresponding
rhn-org-trusted-ssl-cert-*.noarch.rpmor raw CA SSL public (client) certificate is installed on all client systems.
- Verify the client systems are configured to use the appropriate certificate.
- If also using one or more Red Hat Network Proxy Servers, ensure each Proxy's SSL certificates are prepared correctly. The Proxy should have both its own server SSL key-pair and CA SSL public (client) certificate installed, since it will serve in both capacities. Refer to the SSL Certificates chapter of the Red Hat Network Client Configuration Guide for specific instructions.
- Make sure client systems are not using firewalls of their own, blocking required ports as identified in Section 2.4, “Additional Requirements”.
# rm -rf temporary-directory
/var/rhn-sat-import/as the temporary directory.
SSL_CONNECTerrors, is the result of a Satellite being installed on a machine whose time had been improperly set. During the Satellite installation process, SSL certificates are created with inaccurate times. If the Satellite's time is then corrected, the certificate start date and time may be set in the future, making it invalid.
# openssl x509 -dates -noout -in /usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT
# openssl x509 -dates -noout -in /etc/httpd/conf/ssl.crt/server.crt
10.4. Logging and Reporting
/var/log/rhn/directory. These are rotated logs, which are log files created with a .<NUMBER> extension when the current
rhn_satellite_install.logfile fills up to a size as specified by the
logrotate(8)daemon and the contents written to a rotated log file. For example, the
rhn_satellite_install.log.1contains the oldest rotated log file, while
rhn_satellite_install.log.4contains the most recently rotated log.
Table 10.1. Log Files
|Component/Task||Log File Location|
|Apache Web server|| |
|Red Hat Network Satellite|| |
|Red Hat Network Satellite Installation Program|| |
|Database installation - Embedded Database|| |
|Database population|| |
|Red Hat Network Satellite Synchronization Tool|| |
|Monitoring infrastructure|| |
|Monitoring notifications|| |
|Red Hat Network DB Control - Embedded Database|| |
|Red Hat Network Task Engine (taskomatic)|| |
|XML-RPC transactions|| |
spacewalk-reportcommand to gather and display vital Satellite information at once.
spacewalk-reportyou must have the
spacewalk-reportallows administrators to organize and display reports about content, errata, systems, system event history, and user resources across the Satellite. The
spacewalk-reportcommand is used to generate reports on:
- System Inventory — Lists all of the systems registered to the Satellite.
- Entitlements — Lists all organizations on the Satellite, sorted by system or channel entitlements.
- Errata — Lists all the errata relevant to the registered systems, sorts errata by severity as well as the systems that apply to a particular erratum.
- Users — Lists all the users registered to the Satellite, and lists any systems associated with a particular user.
- System History — Lists all, or a subset, of the system events that have occurred.
# spacewalk-report report_name
|System Inventory|| ||List of systems registered to the server, together with hardware and software information|
|Entitlements|| ||Lists all organizations on the Satellite with their system or channel entitlements|
|Errata in channels|| ||Lists errata in channels|
|All Errata|| ||Complete list of all errata|
|Errata for systems|| ||Lists applicable errata and any registered systems that are affected|
|Users in the system|| ||Lists all users registered to the Satellite|
|Systems administered|| ||Lists systems that can be administered by individual users|
|Kickstart Trees|| ||Lists trees able to be kickstarted|
|System history|| ||Lists system event history|
|System history channels|| ||Lists system event history|
|System history configuration|| ||Lists system configuration event history|
|System history entitlements|| ||Lists system entitlement event history|
|System history errata|| ||Lists system errata event history|
|System history kickstart|| ||Lists system kickstart and provisioning event history|
|System history packages|| ||Lists system package event history|
--list-fields-infoand the report name. The description and list of possible fields in the report will be shown.
spacewalk-report(8)manpage as well as the
--helpparameter of the
spacewalk-reportprogram can be used to get additional information about the program invocations and their options.
web.default_mail_fromoption and appropriate value in
install.logfile, and locating the following error:
ERROR: unhandled exception occurred: Traceback (most recent call last): File "/usr/bin/rhn-satellite-activate", line 45, in ? sys.exit(abs(mod.main() or 0)) File "/usr/share/rhn/satellite_tools/rhn_satellite_activate.py", line 585, in main activateSatellite_remote(options) File "/usr/share/rhn/satellite_tools/rhn_satellite_activate.py", line 291, in activateSatellite_remote ret = s.satellite.deactivate_satellite(systemid, rhn_cert) File "/usr/lib/python2.4/site-packages/rhn/rpclib.py", line 603, in __call__ return self._send(self._name, args) File "/usr/lib/python2.4/site-packages/rhn/rpclib.py", line 326, in _request self._handler, request, verbose=self._verbose) File "/usr/lib/python2.4/site-packages/rhn/transports.py", line 171, in request headers, fd = req.send_http(host, handler) File "/usr/lib/python2.4/site-packages/rhn/transports.py", line 698, in send_http self._connection.connect() File "/usr/lib/python2.4/site-packages/rhn/connections.py", line 193, in connect sock.connect((self.host, self.port)) File "<string>", line 1, in connect socket.timeout: timed out
- Run the install script in disconnected mode, and skip the database installation which has already been done:
# ./install.pl --disconnected --skip-db-install
/etc/rhn/rhn.confwith your preferred text editor, and add or modify the following line:
server.satellite.rhn_parent = satellite.rhn.redhat.comRemove the following line:
disconnected=1If you are using a proxy for the connection to Red Hat Network, you will also need to add or modify the following lines to reflect the proxy settings.
server.satellite.http_proxy = <hostname>:<port> server.satellite.http_proxy_username = <username> server.satellite.http_proxy_password = <password>
- Re-activate the Satellite in connected mode, using the
rhn-satellite-activatecommand as the root user, including the path and filename of the satellite certificate:
# rhn-satellite-activate --rhn-cert=/path/to/file.cert
install.plscript in connected mode, but with the
--answer-file=answer fileoption. Ensure the answer file has the HTTP proxy information specified as follows:
rhn-http-proxy = <hostname>:<port> rhn-http-proxy-username = <username> rhn-http-proxy-password = <password>
mount_pointconfiguration parameter in
/etc/rhn/rhn.confdoes not point to a directory path, or the directory path it points to is not present or does not have permission to access the directory.
mount_pointconfiguration parameter in
/etc/rhn/rhn.conf. If it set to the default value of
/var/satellite, verify that the
/var/satellite/redhatdirectories exist. For all values, check that path to the file is accurate, and that the permissions are set correctly.
cobbler checkgive an error saying that it needs a different version of
cobbler checkcommand can give an error similar to the following:
# cobbler check The following potential problems were detected: #0: yum-utils need to be at least version 1.1.17 for reposync -l, current version is 1.1.16
reposyncpackage. The error is spurious and can be safely ignored. This error will be resolved in future versions of Red Hat Network Satellite.
ERROR: <Fault -2: 'unhandled internal exception: unsupported version: 96'>
RHN_PARENT: satellite.rhn.redhat.com Error reported from RHN: <Fault -2: 'unhandled internal exception: unsupported version: 115'> ERROR: unhandled XMLRPC fault upon remote activation: <Fault -2: 'unhandled internal exception: unsupported version: 115'> ERROR: <Fault -2: 'unhandled internal exception: unsupported version: 115'>
Invalid satellite certificate
'ascii' codec can't encode character u'\u2013'
- Ssh directly onto the Satellite server as the root user:
# ssh email@example.com
- Find the kickstart profile that is causing the problem by looking at the dates of the files in
/var/lib/cobbler/config/profiles.dand locating the one that was edited most recently:
# ls -l /var/lib/cobbler/config/profiles.d/
- Open the profile in your preferred text editor, and locate the following text:
\u2013hostnameChange the entry to read:
- Save changes to the profile and close the file.
- Restart the Red Hat Network Satellite services to pick up the updated profile:
# rhn-satellite restart Shutting down rhn-satellite... Stopping RHN Taskomatic... Stopped RHN Taskomatic. Stopping cobbler daemon: [ OK ] Stopping rhn-search... Stopped rhn-search. Stopping MonitoringScout ... [ OK ] Stopping Monitoring ... [ OK ] Stopping httpd: [ OK ] Stopping tomcat5: [ OK ] Shutting down osa-dispatcher: [ OK ] Shutting down Oracle Net Listener ... [ OK ] Shutting down Oracle DB instance "rhnsat" ... [ OK ] Shutting down Jabber router: [ OK ] Done. Starting rhn-satellite... Starting Jabber services [ OK ] Starting Oracle Net Listener ... [ OK ] Starting Oracle DB instance "rhnsat" ... [ OK ] Starting osa-dispatcher: [ OK ] Starting tomcat5: [ OK ] Starting httpd: [ OK ] Starting Monitoring ... [ OK ] Starting MonitoringScout ... [ OK ] Starting rhn-search... Starting cobbler daemon: [ OK ] Starting RHN Taskomatic... Done.
- Return to the web interface. Note that interface can take some time to resolve the services, but should return to normal after a minute or so.
/etc/hostsfile. You may confirm this by examining
/etc/nsswitch.conf, which defines the methods and the order by which domain names are resolved. Usually, the
/etc/hostsfile is checked first, followed by Network Information Service (NIS) if used, followed by DNS. One of these has to succeed for the Apache Web server to start and the Red Hat Network client applications to work.
/etc/hostsfile. It may look like this:
127.0.0.1 this_machine.example.com this_machine localhost.localdomain \ localhost
127.0.0.1 localhost.localdomain.com localhost
127.0.0.1 localhost.localdomain.com localhost
18.104.22.168 this_machine.example.com this_machine
satellite-syncreports that the server is not activated as an Red Hat Network Satellite, it isn't subscribed to the respective Red Hat Network Satellite channel. If this is a newly installed system then the satellite certificate is possibly not activated on the system. If it was activited earlier, then it has become deactivated.
# yum repolist
# rhn-satellite-activate -vvv --rhn-cert=/path/to/certificate
satellite-debug. To use this tool, issue the command as root. You will see the pieces of information collected and the single tarball created, like so:
# satellite-debug Collecting and packaging relevant diagnostic information. Warning: this may take some time... * copying configuration information * copying logs * querying RPM database (versioning of RHN Satellite, etc.) * querying schema version and database character sets * get diskspace available * timestamping * creating tarball (may take some time): /tmp/satellite-debug.tar.bz2 * removing temporary debug tree Debug dump created, stored in /tmp/satellite-debug.tar.bz2 Deliver the generated tarball to your RHN contact or support channel.
/tmp/directory to your Red Hat representative for immediate diagnosis.