Basic SELinux Troubleshooting in CLI

Updated -

SELinux isolates all processes running on the system to mitigate attacks which take advantage of privilege escalation. Privilege escalation means that a process gains more access rights than it should have.

To prevent this, SELinux enforces Mandatory Access Control (MAC) mechanism over all processes. It labels every process, file, or directory according to rules specified in a security policy known as the SELinux policy.

The SELinux policy also specifies how processes interact with each other and how they can access files and directories. SELinux denies every action that it is not explicitly allowed by the SELinux policy.

The most common causes why SELinux denies an action are:

  • processes, files, or directories are labeled with incorrect SELinux context
  • confined processes are configured in a different way than what is expected by the default SELinux policy
  • there is a bug in the SELinux policy or in an application

Troubleshooting SELinux AVC Messages on the Command Line

When SELinux denies an action, an Access Vector Cache (AVC) message is logged to the /var/log/audit/audit.log and /var/log/messages files or the journald daemon logs it. If you suspect that SELinux denied an action that you attempted to do, follow these basic troubleshooting steps:

  1. Use the ausearch utility to find any recent AVC messages and confirm that SELinux denies the action:

    # ausearch -m AVC,USER_AVC -ts recent
    time->Thu Feb 18 14:24:24 2016
    type=AVC msg=audit(1455805464.059:137): avc:  denied  { append } for  pid=861 comm="httpd" name="error_log" dev="sdb1" ino=20747 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=file permissive=0
    

    The -m option specifies what kind of information ausearch returns.The -ts option specifies the time stamp. For example -ts recent returns AVC messages from the last 10 minutes or -ts today returns messages from the whole day.

  2. Use the journalctl utility to view more information about the AVC message:

    # journalctl -t setroubleshoot --since= [time]
    

    Replace [time] with the time from the AVC message found in the first step. In this example, SELinux prevented the httpd process from accessing the /var/log/httpd/error_log file:

        # journalctl -t setroubleshoot --since=14:20
    -- Logs begin at Fri 2016-01-15 01:17:17 UTC, end at Thu 2016-02-18 14:25:21 UTC. --
    Feb 18 14:24:24 fedora.23.virt setroubleshoot[866]: SELinux is preventing httpd from append access on the file error_log. For complete SELinux messages. run sealert -l e9d8fa2e-3608-4ffa-9e72-31a1b85e460b
    
  3. Use the sealert utility to further inspect the AVC message:

    # sealert -l [message_ID]
    

    Replace [message_ID] with the number of the AVC message. The output will look similarly as in the examples below:

    • In this example, SELinux prevented the httpd process from accessing the /var/log/httpd/error_log file because it was incorrectly labeled with the var_log_t SELinux type:

      # sealert -l e9d8fa2e-3608-4ffa-9e72-31a1b85e460b
          SELinux is preventing httpd from open access on the file /var/log/httpd/error_log.
      
      ***** Plugin restorecon (99.5 confidence) suggests   **************************
      
      If you want to fix the label.
          /var/log/httpd/error.log default label should be httpd_log_t.
          Then you can run restorecon.
          Do
          # /sbin/restorecon -v /var/log/httpd/error_log
      
      [trimmed for clarity]
      
    • In this example, SELinux prevented the plugin-containe process from connecting to the network using the TCP protocol and from using the Bluejeans service because the mozilla_plugin_can_network_connect and mozilla_plugin_use_bluejeans Booleans were not enabled:

      # sealert -l fc46b9d4-e5a1-4738-95a7-26616d0858b0
      SELinux is preventing plugin-containe from name_connect access on the tcp_socket port 5000.
      
      *****  Plugin catchall_boolean (9.19 confidence) suggests   ******************
      
      If you want to allow mozilla plugin domain to connect to the network using TCP.
      Then you must tell SELinux about this by enabling the 'mozilla_plugin_can_network_connect' boolean.
      You can read 'mozilla_selinux' man page for more details.
      Do
      setsebool -P mozilla_plugin_can_network_connect 1
      
      *****  Plugin catchall_boolean (9.19 confidence) suggests   ******************
      
      If you want to allow mozilla plugin to use Bluejeans.
      Then you must tell SELinux about this by enabling the 'mozilla_plugin_use_bluejeans' boolean.
      You can read 'mozilla_selinux' man page for more details.
      Do
      setsebool -P mozilla_plugin_use_bluejeans 1
      
      [trimmed for clarity]
      
    • In this example, SELinux denied the passwd process to access the /home/user/output.txt file because there is no rule in the SELinux policy that allows passwd to write to files labeled with the user_home_t SELinux type:

      # sealert -l 1dd524dd-1784-44ef-b6d1-fff9238ed927
      
      SELinux is preventing passwd from write access on the file /home/user/output.txt.
      
      *****  Plugin catchall (100. confidence) suggests   **************************
      
      If you believe that passwd should be allowed write access on the output.txt file by default.
      Then you should report this as a bug.
      You can generate a local policy module to allow this access.
      Do
      allow this access for now by executing:
      # grep passwd /var/log/audit/audit.log | audit2allow -M mypol
      # semodule -i mypol.pp
      
      [trimmed for clarity]
      
  4. Perform actions according to suggestions provided by sealert. For example, use the restorecon utility to fix incorrectly labeled files or enable particular Booleans. If there is no suitable hint provided by sealert or you are not sure how to implement the suggestions, contact our support. If you believe that there is a bug in the SELinux policy, report a bug.

  5. Repeat the action you attempted to do before SELinux denied it. If SELinux is still preventing the action, report a bug.

Additional Information:

9 Comments

Great article, terrific insight, thank you for sharing about ausearch utility. A bit of a cosmetic problem though, or maybe I missed something: you are listing an SELinux infraction via sealert -l. How do you pass the UUID of the infraction as an argument to this command? This may be very simple and basic, but I have tried passing it by selecting the line that contains that UUID by enclosing a regex in a command substitution step (both backticks and $(...) ways), through xargs and it just isn't accepting the input.

Hi Gjorgi,

I'm glad that you find this article useful. As per you question, if you by UUID mean the ID of the denial message, then you just pass it as it is, for example:

sealert - l 1dd524dd-1784-44ef-b6d1-fff9238ed927

Also, journalctl provides the exact sealert command you can run to investigate the problem more, for example:

# journalctl -t setroubleshoot --since=14:20
-- Logs begin at Fri 2016-01-15 01:17:17 UTC, end at Thu 2016-02-18 14:25:21 UTC. --
Feb 18 14:24:24 fedora.23.virt setroubleshoot[866]: SELinux is preventing httpd from append access on the file error_log. For complete SELinux messages. run **sealert -l e9d8fa2e-3608-4ffa-9e72-31a1b85e460b**

Hope that helps,

Bara

What I was looking for is a way to programaticaly pass the message ID (which is a very long string) to sealert -l rather than retyping the entire string by hand. In a situation where for example, we don't have the luxury of a graphical interface, so there's no copy-paste on click. So, in the above example, I am looking for a way to avoid re-typing "e" followed by "9" followed by "d" followed by "8" et cetera, et cetera... when I want to further inspect the infraction via sealert -l

Ah, I got it now :) Unfortunately, that's something I don't know, however, I'll check with our SELinux guys and let you know.

Gjori,

I asked around but no one was able to tell me how to do it. I'm sorry I cannot help you. Hope you find a solution soon.

Bara

pipe your output to

grep -w -oE "\b[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}\b"

Which takes longer, re-typing the message ID or typing this string above? I am sure it will match the message ID string but...

there are 2 benefits of mentioned regex:

  1. you can use it for your CI/CD pipeline.. or whatever automation way you use.
  2. you can make
alias grep_uuid='grep -w -oE "\b[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}\b"'

and thus you don't have to retype it again.

"--since TIME" takes a lot of time to type the timestamp.

I found faster ways to see the recommended actions: 1. use "-f" flag to journalctl which will live tail the events, as you probably want to see live wtf is going on right now. or use "--since today" if looking for past events today. 2. tail -f & grep the messages log file which contains the plugins with actions.