What is an sos report and how to create one in Red Hat Enterprise Linux?
- Red Hat Enterprise Linux (RHEL) 4.6+, 5, 6, 7, 8, 9
- For System Is Not Booting cases
- Follow the directions in: "How to generate sos report from the rescue environment" for cases where the system is not booting or does not stay running long enough to generate an sos report via normal means.
- For Azure Virtual Guests
- Follow the directions in: "How to Create a Microsoft Azure Virtual Machine sos report and Submit the Report to Red Hat" for Azure virtual guests.
- For OpenShift 4 nodes
- Follow the directions in: "How to generate SOS REPORT within OpenShift4 nodes without SSH".
- For Pacemaker cluster nodes
- Follow the directions in: "sosreport on RHEL7 pacemaker cluster does not collect pacemaker data".
- What is
sos report, is it the same as an sosreport?
- How do I collect system log files, configuration details and system information from a Red Hat Enterprise Linux system to provide to Red Hat Technical Support for analysis?
- How can I generate an
- I am opening a service request with Red Hat Technical Support for technical assistance. What data I have to provide for initial analysis of my system?
- How can I control how the
sos reportcommand runs?
- How to install the package containing sos report
- What to do if an sos report hangs
- What commands a particular sos report plugin runs in the background
Table of Contents
- Installing sos package
- What is an sos report, is it different from an sosreport?
- How can I generate an sos report?
- How can I control how the
sos reportcommand runs?
- How can I disable or enable specific plugins?
- How do I use plugin options?
- Why does
sos reportsometimes skip collecting certain command output?
sos reporton my system uses too much CPU time or memory (it is supported with sos version 3.6 or later)
- How do I automatically upload an sos report?
- How do I run newer sos versions on older RHEL releases?
What is an sos report and is it different from an sosreport?
sos report within RHEL 8+ is equivalent to the
sosreport command in RHEL7 and earlier. So sos report is the same as sosreport -- just different command syntax used within the sos package.
- Within RHEL 7 and earlier releases, while the installed package name was sos, the command to capture data from a system was
- Starting with RHEL 8+, the command name was changed to just
sosand the optional argument of
reportis needed to perform the same data collection activities as
sosreportin older sos packages.
- The convention within this document is to utilize the new syntax of
sos reportgenerically unless it is specifically referencing a command example where the older syntax of
- The command syntax of
sosreportwill work on all kernel version, but is deprecated in RHEL 8+ kernels and will result in an "Redirecting to sos report" warning output.
sos report command is a tool that collects configuration details, system information and diagnostic information from a Red Hat Enterprise Linux system. For instance: the running kernel version, loaded modules, and system and service configuration files. The command also runs external programs to collect further information, and stores this output in the resulting archive.
» Why am I being asked to provide an sos report?
The output of
sos report is the common starting point for Red Hat support engineers when performing an initial analysis of a service request for a Red Hat Enterprise Linux system.
The utility provides a standardized way to collect diagnostic information that your Red Hat support engineer will reference throughout their investigation of issues reported in support cases.
The use of
sos report helps to ensure that you are not being continually asked for piecemeal data outputs.
How can I generate an sos report?
sos package has been installed, issue the following commands to run sos report. The first command ensures the latest package is installed so that most current data collection procedures are performed.
# yum update sos # sosreport
Notice Starting in RHEL 8, the
sosreportcommand is deprecated in favor of
sos report. A message to this effect is output if
sosreportis issued on a RHEL 8 or 9 system and "
Redirecting to 'sos report'" is performed.
Notice In RHEL8+, change the above command name to
sos report, with the added space between sos and report, to avoid the "Redirecting to sos report" related messages.
sos reportcommand requires root permissions to gather data correctly. There is no current mechanism to allow non-root executions of
The command will normally complete within a few minutes on Red Hat Enterprise Linux 6. Older versions may take longer to complete. Depending on local configuration and the options specified in some cases the command may take longer to finish. If you are concerned about the run time of the
sos report command contact your Red Hat support representative for assistance.
sos report will generate a compressed file under
/tmp (for RHEL6 and earlier) or under
/var/tmp (for RHEL7 and later). The file should be provided to your support representative (normally as an attachment to an open support case).
sos report results in "no valid plugins were enabled" message
Typically the "no valid plugins" message is output then the
sos reportcommand run without full root privileges. The
sos reportcommand requires root permissions to gather data from the system correctly. Many of the files, commands, and tools that it runs to gather information required root access privilege. For example, if multipath is configured on the system then sos report will gather information from multipath subsystem on the current configuration settings, which it cannot do as a normal user.
$ multipathd -k"show config" need to be root
Please try running again as root, and if you are still experiencing issues, open a support case so that we may investigate.*
sos report fills my available disk space
The size of the archive varies depending on system configuration and any optional
sos reportfeatures that are enabled. For example
--all-logswill greatly increase the size of the archive as it removes size-limiting features for command output and log file collection.
/var/tmpis too small to hold an sos report archive, use the
--tmp-diroption to specify an alternate location with sufficient available space.
» How do I provide an
sos report to Red Hat?
sosis able to directly upload an archive to Red Hat via the
sos report --upload.
- See section How do I automatically upload an sos report? for more in-depth information.
Alternative upload methods that are also available include:
- To post
sos report, or any other file, to an existing support case you can use the redhat-support-tool command line option, the Red Hat Portal UI or several different methods using FTP.
- If the collected
sos reportfile is too big to upload to the case, it may be uploaded to the Red Hat Secure FTP.
- If you use the ftp option, update your support case with the exact filename as this is the only way your support engineer will be able to retrieve it.
» I do not have a case number yet, do I need to provide one for
sos report to run?
The case number input prompt is optional, and if provided the case number will be part of the archive's file name. Omitting the case number will not have any negative effect on running the
Also, the command may be run in batch mode using the
--batchoption in order to avoid the need to enter user and account information interactively.
How can I control how the
sos report command runs?
The sos report command has a plugin structure and allows the user to enable and disable plugins and specify plugin options via the command line. Available plugins and their options may be listed by running the following command:
# sosreport -l
» How can I disable or enable specific plugins?
Users may selectively enable or disable plugins using the
-n/--skip-plugins options respectively. These options take comma-delimited lists of plugin names or may be specified multiple times. For example, to disable the
kvm plugins, use the following:
# sosreport -n kvm,amd
» How do I use plugin options?
Individual plugins may provide additional options that may be specified via the
-k option. These options are listed in the same output as the listing of available plugins (
sosreport -l). The format this option takes is
plugin_name.option_name=value. For example, to enable collection of container logs, use the following:
# sosreport -k podman.logs=on
Options will specify whether they are boolean toggles or expect a string or integer. For boolean toggles, users may use
» Why does
sos report sometimes skip collecting certain command output?
An sos report execution might print to console a message like the following:
[plugin:networking] skipped command 'nft list ruleset': required kernel modules or services not present (kmods=[nf_tables] services=).
sos report aims to not alter the system it runs on in any way. Some commands called by certain plugins may automatically trigger a change (load a kernel module in the example above) and are thus gated by default. If making these changes is acceptable for your environment, and you would like to have
sos report make them in order to collect the skipped commands, run
sos report like so:
# sos report --allow-system-changes
sos report on my system uses too much CPU time or memory»* (it is supported with sos version 3.6 or later)
sos report will run up to 4 plugins in parallel in an effort to reduce total runtime. This may cause memory contention or high CPU usage on certain systems depending on which plugins are being run (and particularly if each of those plugins collects journal output). To reduce the number of simultaneous collections use the
--threads option. For example, to run plugins one at a time, use the following:
# sosreport --threads=1
Installing sos package
sos package must be installed in order to run the
sos report command. You can check to see if the
sos package is installed and whether there is any problems with the installation using the following command:
# rpm -qa | grep sos sos-3.2-35.el7_2.3.noarch << sos package is installed # rpm -V sos << run verification on installed package
» Red Hat Enterprise Linux 5 and later
If the system is registered with RHSM, use the
# yum install sos
If the system is not registered with RHSM, the
sos package can be downloaded from the RHN website or found on the installation CDs or DVD. The rpm command may be used to install the package on any version of Red Hat Enterprise Linux:
# rpm -Uvh sos-<version>.noarch.rpm
Sos Package Update
The sos report command constantly gets updated with important bug fixes and data collection updates to ensure that all the proper data collection needed by our support engineers is performed. Not using the latest sos report often impeded support's ability to review a case in a timely manner and will often require a secondary request for information or an update and redo of sos report. Making sure the latest sos package is installed prevents these issues and inconveniences
# yum update sos
» Red Hat Enterprise Linux 4 Update 6 and later 4.x kernels
If the system is registered with Red Hat Subscription Manager (RHSM),
sos can be installed using the
# up2date sos
How do I automatically upload an sos report to Red Hat Support?
sos-3.9 and later, sos supports the
--upload option to automatically upload an archive once it is generated. Note that a local copy of the archive will still exist. This option requires the use of the
python3-requests package, which is defined as a weak dependency for the sos rpm. Systems that are not configured to resolve weak dependencies, or do not enable the AppStream repository, will need to manually install this package.
When run on a Red Hat Enterprise Linux system, using the
--upload option will prompt the user for their Customer Portal credentials:
# sos report --upload sosreport (version 4.2) [...] Please enter the case id that you are generating this report for : 123456 Enter your Red Hat Customer Portal username (empty to use public dropbox): your-username Please provide the upload password for your-username: [...] Your sosreport has been generated and saved in: /var/tmp/sosreport-myhost-123456-2022-08-11-sgczqyo.tar.xz Attempting upload to Red Hat Customer Portal Uploaded archive successfully
By providing a case number and login credentials which are capable of accessing the specified case number, sos will upload the resulting archive directly to the case upon completion. If either of these are not provided or are incorrect (e.g. an incorrect case number for the credentials provided), then sos will fallback to uploading the archive to Red Hat's secure FTP server:
# sos report --upload sosreport (version 4.2) [...] Please enter the case id that you are generating this report for : 123456 Enter your Red Hat Customer Portal username (empty to use public dropbox): incorrect-user Please provide the upload password for your-username: [...] Your sosreport has been generated and saved in: /var/tmp/sosreport-myhost-123456-2022-08-11-hyrtzji.tar.xz Attempting upload to Red Hat Customer Portal Upload to Red Hat Customer Portal failed. Trying sftp://sftp.access.redhat.com Attempting upload to Red Hat Secure FTP Unable to retrieve Red Hat auth token using provided credentials. Will try anonymous. User 'xAnrDdnP'used for anonymous upload. Please inform your support engineer so they may retrieve the data. Uploaded archive successfully
Note: The anonymous user is a temporary user that may not be re-used, and must be provided to your support engineer in the relevant support case so that the engineer will be able to retrieve and review the archive.
The following methods are available for providing
--uploadas demonstrated above, and use the interactive prompts
- Use the
--upload-passto provide the credentials without a prompt. This may be useful when using
--batchto avoid interactive executions of
sos report. Note however, that this will result in the plaintext password being captured by various process information collections, such as
- Use the
SOSUPLOADPASSWORDenvironment variables. If set, the
--uploadfunction will not prompt for credentials, and these values will not appear in any collections within the archive.
Uploads will fail if network or firewall configuration does not allow access to
api.access.redhat.com (for direct-to-case uploads) or
sftp.access.redhat.com (for SFTP (fallback) uploads).
How do I run newer sos versions on older RHEL releases?
If your RHEL system is version locked (e.g. EUS), the latest
sos package may not available in the older repos. Similarly, recent additions or changes to
sos may not always be backported to older RHEL releases for various reasons. However, it is still possible to use the latest
sos package on any version of RHEL by use of the
support-tools container image.
This container image is consistently maintained and updated with the latest available
sos package and is available from the registry.redhat.io container image registry.
In order to collect host-level information with sos via the support-tools container, the image must be deployed in a specific manner. Simply installing the
sos package inside a different container will result in collections occurring within the container, which is unlikely to be useful. The requirements for successful host-level collections from inside the container image are baked into the
RUN label of the image, and the image may be easily deployed using this label as follows:
# podman container runlabel RUN registry.redhat.io/rhel8/support-tools --- You will now be attached to a root shell inside the container --- # sos report
sos report completes, it will show the location of the archive being at
/host/var/tmp/. The container mounts the host's
/ filesystem to
/host when deployed using the
podman command above - so even if the container is exited the tarball will exist on the host under the (normal)
What to do if sos report hangs
First, verify that there is enough free space available in either
/tmp (RHEL 6) or
/var/tmp (RHEL7 and later), and if not, use the
--tmp-dir option as detailed earlier.
sos report is hanging/stalling on a specific plugin (currently running plugins are displayed during execution), try running
sos report with the problem plugin(s) disabled using the
-n/--skip-plugins option as detailed earlier.
Note that each plugin has a default timeout of 5 minutes (controllable by the
timeout plugin option available to all plugins). Please allow enough time for this timeout threshold to be hit, and
sos report should automatically terminate that plugin's execution. In the event that this termination does not happen, use the option highlighted above.
sos report can not be completed at all for one reason or another, please see How to gather data from a Red Hat Enterprise Linux system for troubleshooting if the
sos report process goes to hung state? for further guidance.
I cannot run
sos report at all and would like to collect a bare minimum of data to provide
Please see sos report fails. What data should I provide in its place? for an alternative bare-minimum data collection script.
For issue-specific guidance in the case of
sos report failure, please see Additional and alternative steps for gathering an sos report for a Red Hat Enterprise Linux support case
What commands does a particular sos report plugin run in the background?
There are following ways to understand what a particular plugin does when its is executed through the
sos report command.
Check the source of
sospackage that provides
For a specific version of source code, see downloads section on Customer Portal.
sos.logfile in the sos report. For example, for plugin
scsithere would be something similar to as shown below
-- directories and files collected -- 2020-07-30 11:04:53,632 INFO: [plugin:scsi] collecting path '/sys/bus/scsi' 2020-07-30 11:04:53,656 INFO: [plugin:scsi] collecting path '/sys/class/scsi_disk' 2020-07-30 11:04:53,663 INFO: [plugin:scsi] collecting path '/proc/scsi' 2020-07-30 11:04:53,666 INFO: [plugin:scsi] collecting path '/sys/class/scsi_generic' 2020-07-30 11:04:53,675 INFO: [plugin:scsi] collecting path '/sys/class/scsi_host' 2020-07-30 11:04:53,677 INFO: [plugin:scsi] collecting path '/sys/class/scsi_device' -- these were the commands executed in the background -- 2020-07-30 11:04:53,686 INFO: [plugin:scsi] collecting output of 'lsscsi -i' 2020-07-30 11:04:53,710 INFO: [plugin:scsi] collecting output of 'sg_map -x' 2020-07-30 11:04:53,765 INFO: [plugin:scsi] collecting output of 'udevadm info -a /sys/class/scsi_host/host3' 2020-07-30 11:04:53,789 INFO: [plugin:scsi] collecting output of 'udevadm info -a /sys/class/scsi_host/host1'
sos reportonly with that particular plugin and check the files and directories and the output of the commands in the collected archive.
To generate the sos report with particular plugin, do
# sosreport -o plugin-name
To see the list of plugins, do
# sosreport -l
- Red Hat Enterprise Linux
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.