10.7. Additional Considerations While Managing Services

During normal operation, systemd maintains an association between a unit abstraction and the underlying processes active on the system.
From: man systemd
Processes systemd spawns are placed in individual Linux control groups named after the unit which they belong to in the private systemd hierarchy. (see cgroups.txt[1] for more information about control groups, or short "cgroups"). systemd uses this to effectively keep track of processes. Control group information is maintained in the kernel, and is accessible via the file system hierarchy (beneath /sys/fs/cgroup/systemd/), or in tools such as ps(1) (ps xawf -eo pid,user,cgroup,args is particularly useful to list all processes and the systemd units they belong to).
The cgroup hierarchy is critical to systemd's view of process and service health. When a process forks itself, it inherits the cgroup of the creating process. With this being the case, all processes associated with a given unit can be verified by reading the contents of the applicable cgroup.procs file, such as:
~]# cat /sys/fs/cgroup/systemd/system.slice/httpd.service/cgroup.procs
11854
11855
11856
11857
11858
11859
The output matches the CGroup information returned during a systemctl status unit operation:
~]# systemctl status httpd
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2019-05-29 12:08:16 EDT; 45s ago
     Docs: man:httpd(8)
           man:apachectl(8)
 Main PID: 11854 (httpd)
   Status: "Total requests: 0; Current requests/sec: 0; Current traffic:   0 B/sec"
   CGroup: /system.slice/httpd.service
           ├─11854 /usr/sbin/httpd -DFOREGROUND
           ├─11855 /usr/sbin/httpd -DFOREGROUND
           ├─11856 /usr/sbin/httpd -DFOREGROUND
           ├─11857 /usr/sbin/httpd -DFOREGROUND
           ├─11858 /usr/sbin/httpd -DFOREGROUND
           └─11859 /usr/sbin/httpd -DFOREGROUND

May 29 12:08:16 localhost systemd[1]: Starting The Apache HTTP Server...
May 29 12:08:16 localhost systemd[1]: Started The Apache HTTP Server.
To directly view these groupings of processes system-wide, the systemd-cgls utility can be used:
~]# systemd-cgls | head -17
├─1 /usr/lib/systemd/systemd --switched-root --system --deserialize 22
├─user.slice
│ └─user-0.slice
│   └─session-168.scope
│     ├─ 3049 login -- root
│     ├─11884 -bash
│     ├─11943 systemd-cgls
│     └─11944 head -17
└─system.slice
  ├─httpd.service
  │ ├─11854 /usr/sbin/httpd -DFOREGROUND
  │ ├─11855 /usr/sbin/httpd -DFOREGROUND
  │ ├─11856 /usr/sbin/httpd -DFOREGROUND
  │ ├─11857 /usr/sbin/httpd -DFOREGROUND
  │ ├─11858 /usr/sbin/httpd -DFOREGROUND
  │ └─11859 /usr/sbin/httpd -DFOREGROUND
  ├─rhnsd.service
In order for systemd to function properly, the service must be started or stopped through the systemd system to maintain the correct process to unit grouping. Any operation that takes external action results in the necessary cgroup structure not being created. This happens because systemd is not aware of the special nature of the processes being started.
As an example of the above constraint, stopping the httpd service and then issuing /usr/sbin/httpd directly results in the following:
~]# systemctl stop httpd
~]# /usr/sbin/httpd
# systemd-cgls | head -17
├─1 /usr/lib/systemd/systemd --switched-root --system --deserialize 22
├─user.slice
│ └─user-0.slice
│   └─session-168.scope
│     ├─ 3049 login -- root
│     ├─11884 -bash
│     ├─11957 /usr/sbin/httpd
│     ├─11958 /usr/sbin/httpd
│     ├─11959 /usr/sbin/httpd
│     ├─11960 /usr/sbin/httpd
│     ├─11961 /usr/sbin/httpd
│     ├─11962 /usr/sbin/httpd
│     ├─11963 systemd-cgls
│     └─11964 head -17
└─system.slice
  ├─rhnsd.service
  │ └─3261 rhnsd
Note that the httpd process is now visible under the user-0.slice and a session-168.scope. This service is treated as a user started process, as opposed to a system service, that systemd should monitor and manage directly. Some failures that can occur due to this misalignment include, but are not limited to:
  • Services are not properly shutdown during system shutdown or restart events.
  • Unexpected signals are delivered during user logout such as SIGHUP and SIGTERM.
  • Processes that fail are not automatically restarted despite having a Restart= directive

Note

Non-graceful application shutdown events can result in a large number of subsequent application failures, such as client-side failures, data loss, and on-disk corruption.