[abrt] sosreport fails at redhat.py:67:__init__:KeyError: 'filesystem'

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 6 or 7
  • any EL7 sos version older than 3.4
  • so far any EL6 sos version

Issue

  • sosreport fails with an exception at redhat.py:67:__init__:KeyError: 'filesystem' on Red Hat Enterprise Linux 6 or 7
# /usr/sbin/sosreport 
Traceback (most recent call last):
  File "/usr/sbin/sosreport", line 25, in <module>
    main(sys.argv[1:])
  File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 1636, in main
    sos = SoSReport(args)
  File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 697, in __init__
    self.policy = sos.policies.load(sysroot=self.opts.sysroot)
  File "/usr/lib/python2.7/site-packages/sos/policies/__init__.py", line 38, in load
    cache['policy'] = policy(sysroot=sysroot)
  File "/usr/lib/python2.7/site-packages/sos/policies/redhat.py", line 158, in __init__
    super(RHELPolicy, self).__init__(sysroot=sysroot)
  File "/usr/lib/python2.7/site-packages/sos/policies/redhat.py", line 67, in __init__
    if pkgs['filesystem']['version'][0] == '3':
KeyError: 'filesystem'

Resolution

Root Cause

  • sosreport collects initial list of installed packages (rpm -qa like command) within timeout of 30 seconds
  • if that command timeouts, no output is collected
  • but the output is referred either way later on
  • detection of filesystem version is the first place where such a reference fails / raises exception if no output is collected

Diagnostic Steps

  • ABRT would list out a sosreport crash on system.
# abrt-cli list --since 1478241960
id f2f98f7e74a7116d88a9f2c8903840dd5fbe85c9
reason:         redhat.py:67:__init__:KeyError: 'filesystem'
time:           Sun 23 Oct 2016 12:53:37 AM CEST
cmdline:        /usr/bin/python /usr/sbin/sosreport -l
package:        sos-3.2-35.el7
uid:            0 (root)
count:          13
Directory:      /var/spool/abrt/Python-2016-10-23-00:53:37-24010
Run 'abrt-cli report /var/spool/abrt/Python-2016-10-23-00:53:37-24010' for creating a case in Red Hat Customer Portal
  • From /var/log/messages :
Oct 18 14:40:22 dtzfuesql02 python: detected unhandled Python exception in '/usr/sbin/sosreport'
Oct 18 14:40:30 dtzfuesql02 python: communication with ABRT daemon failed: timed out
Oct 18 14:40:36 dtzfuesql02 abrt-server: sosreport run failed with exit code 1, log follows:
Oct 18 14:40:36 dtzfuesql02 abrt-server: Traceback (most recent call last):
Oct 18 14:40:36 dtzfuesql02 abrt-server:  File "/usr/sbin/sosreport", line 25, in <module>
Oct 18 14:40:36 dtzfuesql02 abrt-server:    main(sys.argv[1:])
Oct 18 14:40:36 dtzfuesql02 abrt-server:  File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 1592, in main
Oct 18 14:40:36 dtzfuesql02 abrt-server:    sos = SoSReport(args)
Oct 18 14:40:36 dtzfuesql02 abrt-server:  File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 676, in __init__
Oct 18 14:40:36 dtzfuesql02 abrt-server:    self.policy = sos.policies.load(sysroot=self.opts.sysroot)
Oct 18 14:40:36 dtzfuesql02 abrt-server:  File "/usr/lib/python2.7/site-packages/sos/policies/__init__.py", line 38, in load
Oct 18 14:40:36 dtzfuesql02 abrt-server:    cache['policy'] = policy(sysroot=sysroot)
Oct 18 14:40:36 dtzfuesql02 abrt-server:  File "/usr/lib/python2.7/site-packages/sos/policies/redhat.py", line 157, in __init__
Oct 18 14:40:36 dtzfuesql02 abrt-server:    super(RHELPolicy, self).__init__(sysroot=sysroot)
Oct 18 14:40:36 dtzfuesql02 abrt-server:  File "/usr/lib/python2.7/site-packages/sos/policies/redhat.py", line 67, in __init__
Oct 18 14:40:36 dtzfuesql02 abrt-server:    if pkgs['filesystem']['version'][0] == '3':
Oct 18 14:40:36 dtzfuesql02 abrt-server: KeyError: 'filesystem'
  • Component
  • sos

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments