Logs flooded with systemd messages: Created slice, Starting Session
Environment
- Red Hat Enterprise Linux (RHEL) 7, 8, 9
Issue
-
On my newly installed system I am seeing the following in
/var/log/messagesand journal all the timeJul 24 08:50:01 example.com systemd: Created slice user-0.slice. Jul 24 08:50:01 example.com systemd: Starting Session 150 of user root. Jul 24 08:50:01 example.com systemd: Started Session 150 of user root. Jul 24 09:00:01 example.com systemd: Created slice user-0.slice. Jul 24 09:00:02 example.com systemd: Starting Session 151 of user root. Jul 24 09:00:02 example.com systemd: Started Session 151 of user root.
Resolution
-
To suppress these log entries in
/var/log/messagesand journal you have following options:-
enable lingering for the user so
systemd --userinstance will continue running even when user has no active sessions:# loginctl enable-linger testuser -
restrict logging for systemd user manager for the user:
# mkdir /etc/systemd/system/user@1000.service.d # cat > /etc/systemd/system/user@1000.service.d/logging.conf <<EOF [Service] LogLevelMax=notice EOF -
restrict logging for systemd user manager globally:
# sed -i -e 's/#LogLevel=info/LogLevel=notice/' /etc/systemd/user.conf -
in RHEL 7, create a discard filter with rsyslog, e.g., run the following command:
echo 'if $programname == "systemd" and ($msg contains "Starting Session" or $msg contains "Started Session" or $msg contains "Created slice" or $msg contains "Starting user-" or $msg contains "Starting User Slice of" or $msg contains "Removed session" or $msg contains "Removed slice User Slice of" or $msg contains "Stopping User Slice of") then stop' >/etc/rsyslog.d/ignore-systemd-session-slice.confThen restart the rsyslog service:
systemctl restart rsyslogNote In case above rule only works for local syslog logging (
/var/log/messages) but doesn't work for remote syslog server then we can understand this rule is read after syslog events are sent to syslog server. So in order to avoid this add the rule directly in rsyslog.conf above the line of remote syslog sending configuration.(*.* @@sys-log server)
-
Root Cause
- These messages are normal and expected -- they will be seen any time a user logs in or e.g. a
cronjob is executed for the user
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
19 Comments
The proposed command produces the following error:
Indeed. (Though to be clear, it's a warning; not an error.) Old habits die hard. I've updated the article. Thanks.
I have updated the filter to: if $programname == "systemd" and ($msg startswith "Starting Session" or $msg startswith "Started Session" or $msg startswith "Created slice" or $msg startswith "Starting user-" or $msg startswith "Removed slice" or $msg startswith "Stopping user-") then stop
This filter will no longer log entries like:
I guess the filter should also contain below 2 entries also:
To remove such log entries from messages logs:
The overall command post adding the changes would be:
In rhel7u4, rsyslog have update to rsyslog v8 version. And the sample log format have changed in rhel7u4. Thanks for the update
The log is printed by the function session_start_scope()of the systemd.
Dec 6 04:01:01 localhost systemd: Created slice user-0.slice. Dec 6 04:01:01 localhost systemd: Starting Session 63 of user root. Dec 6 04:01:01 localhost systemd: Started Session 63 of user root. Dec 6 05:01:01 localhost systemd: Created slice user-0.slice.
This function is used to create cgroup; The function will be called when there a user login because the OS will need to create cgroup for the just login user (The user could a real user or an app which uses dbus messages to communicate with other users). These log just shows the info when the cgroup is created by systemd, so normally, you might be able to just ignore these logs.
Why tune rsyslog when we might tune systemd? THats where the message is created. If it were in SSH session mgt, then we'd tune sshd_config first. or.. maybe not push it into /var/log/messages.
This statement is misleading: "they will be seen any time a user logs in" An interactive user login is not the only event that will generate this message to rsyslog; other events generate that same message. That statement might suggest otherwise to an unknowing system administrator.
seems like in rhel 7.4 the logger deamon is no longer "systemd" but "systemd-logind" which makes the current rsyslog filter invalid:
Nov 30 08:54:04 db161142 systemd-logind[2445]: New session 2172664 of user dbi_backup. Nov 30 08:54:05 db161142 systemd-logind[2445]: New session 2172665 of user dbi_backup. Nov 30 08:54:05 db161142 systemd-logind[2445]: New session 2172666 of user dbi_backup.
I tried to install cinnamon. I ended with a balcnk screen. I pressed ctrl alt F2. I got a blinking cursor at the upper left of the scree. Then the messages appeared: [Ok] Created slice User Slice of root. [ok] Started session 1 of user root [ok] Removed slice User Slice of root] at the bottom of this the cursor is blinking and I can enter nothing. I see it is continuing [ok]Removed slice User of root [ok]Started session 4 of user root [OK] Removed slice User Slice of root What do I do?
Can I ask what was the logic in having this information go to syslog in the first place?
We have filtering tools to ignore specific types of messages but this make as much sense as having email go to a folder by default so you can delete it in bulk.
Why not solve the problem, not the symptom?
Exactly! Tune the service that creates the log. Since we have a service called systemd-logind, can we tune that? Surely the ability to control logging behavior isn't taken away. Might we add logging controls to pam_systemd? This looks like a good place to apply some logic . https://www.freedesktop.org/software/systemd/man/pam_systemd.html - then we can put rules in for logging cron jobs, ssh sessions, console logins, webmin, .. might even catch unintended sessions from exploits? Since Systemd is subsuming everything, this should be considered.
For the record (years later), if you're getting these every 10 minutes (and only every 10 minutes), it's most likely sar's cronjob firing off that systemd feels the need to flood our collective logs with.
If the sar is cause of this logs, I think the kb needs to modify only to filter out cron related root login session logs.
What about logs like these:
Did anyone manage to suppress these ?
systemd генерит не информативный спам в логи. Надо чинить systemd, а не делать костыли в правилах rsyslog
Hi team.
I have this problem... the the connection is lost immediately
May 18 21:44:15 CDRServer systemd: Created slice User Slice of SplunkLogs. May 18 21:44:15 CDRServer systemd: Started Session 268482 of user SplunkLogs. May 18 21:44:15 CDRServer systemd-logind: New session 268482 of user SplunkLogs. May 18 21:44:15 CDRServer systemd-logind: Removed session 268482.
Do you know the possibles reasons and how to fix it?
Regards!
Hi all.
I have this problem... the connection is lost immediately
Do you know the possible reasons and how to fix it?
Regards!
Did you get any solution Carlos Duarte.? If yes , let me know since I'm also facing the same issue.