Logs flooded with systemd messages: Created slice & Starting Session

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 7

Issue

  • On my RHEL7 newely installed system I am seeing the following in /var/log/messages all the time

    Jul 24 08:50:01 example.com systemd: Created slice user-0.slice.
    Jul 24 08:50:01 example.com systemd: Starting Session 150 of user root.
    Jul 24 08:50:01 example.com systemd: Started Session 150 of user root.
    Jul 24 09:00:01 example.com systemd: Created slice user-0.slice.
    Jul 24 09:00:02 example.com systemd: Starting Session 151 of user root.
    Jul 24 09:00:02 example.com systemd: Started Session 151 of user root.
    

Resolution

  • These messages are normal and expected -- they will be seen any time a user logs in

  • To suppress these log entries in /var/log/messages, create a discard filter with rsyslog, e.g., run the following command:

    echo 'if $programname == "systemd" and ($msg contains "Starting Session" or $msg contains "Started Session" or $msg contains "Created slice" or $msg contains "Starting user-" or $msg contains "Starting User Slice of" or $msg contains "Removed session" or $msg contains "Removed slice User Slice of" or $msg contains "Stopping User Slice of") then stop' >/etc/rsyslog.d/ignore-systemd-session-slice.conf
    
    

    Then restart the rsyslog service

    systemctl restart rsyslog
    

Note In case above rule only works for local syslog logging(/var/log/messages) but doesn't work for remote syslog server then we can understand this rule is read after syslog events are sent to syslog server. So in order to avoid this add the rule directly in rsyslog.conf above the line of remote syslog sending configuration. (. @@sys-log server)

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

14 Comments

The proposed command produces the following error:

rsyslogd-2307: warning: ~ action is deprecated, consider using the 'stop' statement instead [try http://www.rsyslog.com/e/2307 ]

Indeed. (Though to be clear, it's a warning; not an error.) Old habits die hard. I've updated the article. Thanks.

I have updated the filter to: if $programname == "systemd" and ($msg startswith "Starting Session" or $msg startswith "Started Session" or $msg startswith "Created slice" or $msg startswith "Starting user-" or $msg startswith "Removed slice" or $msg startswith "Stopping user-") then stop

This filter will no longer log entries like:

Apr  5 09:53:01 aixtsmprod01 systemd: Removed slice user-1000.slice.
Apr  5 09:53:01 aixtsmprod01 systemd: Stopping user-1000.slice.

I guess the filter should also contain below 2 entries also:

$msg contains "Removed slice"
$msg contains "Stopping user-"

To remove such log entries from messages logs:

Aug 18 10:20:01 test-prov-el7 systemd: Removed slice user-0.slice.
Aug 18 10:20:01 test-prov-el7 systemd: Stopping user-0.slice.

The overall command post adding the changes would be:

echo 'if $programname == "systemd" and ($msg contains "Starting Session" or $msg contains "Started Session" or $msg contains "Created slice" or $msg contains "Starting user-" or $msg contains "Starting User Slice of" or $msg contains "Removed session" or $msg contains "Removed slice User Slice of" or $msg contains "Stopping User Slice of" or $msg contains "Removed slice" or $msg contains "Stopping user-") then stop' >/etc/rsyslog.d/ignore-systemd-session-slice.conf

In rhel7u4, rsyslog have update to rsyslog v8 version. And the sample log format have changed in rhel7u4. Thanks for the update

The log is printed by the function session_start_scope()of the systemd.

Dec 6 04:01:01 localhost systemd: Created slice user-0.slice. Dec 6 04:01:01 localhost systemd: Starting Session 63 of user root. Dec 6 04:01:01 localhost systemd: Started Session 63 of user root. Dec 6 05:01:01 localhost systemd: Created slice user-0.slice.

This function is used to create cgroup; The function will be called when there a user login because the OS will need to create cgroup for the just login user (The user could a real user or an app which uses dbus messages to communicate with other users). These log just shows the info when the cgroup is created by systemd, so normally, you might be able to just ignore these logs.

Why tune rsyslog when we might tune systemd? THats where the message is created. If it were in SSH session mgt, then we'd tune sshd_config first. or.. maybe not push it into /var/log/messages.

This statement is misleading: "they will be seen any time a user logs in" An interactive user login is not the only event that will generate this message to rsyslog; other events generate that same message. That statement might suggest otherwise to an unknowing system administrator.

seems like in rhel 7.4 the logger deamon is no longer "systemd" but "systemd-logind" which makes the current rsyslog filter invalid:

Nov 30 08:54:04 db161142 systemd-logind[2445]: New session 2172664 of user dbi_backup. Nov 30 08:54:05 db161142 systemd-logind[2445]: New session 2172665 of user dbi_backup. Nov 30 08:54:05 db161142 systemd-logind[2445]: New session 2172666 of user dbi_backup.

I tried to install cinnamon. I ended with a balcnk screen. I pressed ctrl alt F2. I got a blinking cursor at the upper left of the scree. Then the messages appeared: [Ok] Created slice User Slice of root. [ok] Started session 1 of user root [ok] Removed slice User Slice of root] at the bottom of this the cursor is blinking and I can enter nothing. I see it is continuing [ok]Removed slice User of root [ok]Started session 4 of user root [OK] Removed slice User Slice of root What do I do?

Can I ask what was the logic in having this information go to syslog in the first place?
We have filtering tools to ignore specific types of messages but this make as much sense as having email go to a folder by default so you can delete it in bulk.

Why not solve the problem, not the symptom?

Exactly! Tune the service that creates the log. Since we have a service called systemd-logind, can we tune that? Surely the ability to control logging behavior isn't taken away. Might we add logging controls to pam_systemd? This looks like a good place to apply some logic . https://www.freedesktop.org/software/systemd/man/pam_systemd.html - then we can put rules in for logging cron jobs, ssh sessions, console logins, webmin, .. might even catch unintended sessions from exploits? Since Systemd is subsuming everything, this should be considered.

For the record (years later), if you're getting these every 10 minutes (and only every 10 minutes), it's most likely sar's cronjob firing off that systemd feels the need to flood our collective logs with.

If the sar is cause of this logs, I think the kb needs to modify only to filter out cron related root login session logs.