systemd replacement for /sbin/init
Incorporate systemd as a replacement for /sbin/init, even as a tech preview or install option.
Responses
As systemd is already in the latest Fedora releases, I would assume it would naturally follow through to the release of RHEL 7.
Heck no! Every location I've been in has been a mixed environment. If a Red Hat tech comes in with all the dumb RH specific tools and no knowledge of standard Unix methods, they are useless as an admin.
Also, from a server standpoint, the hardware takes long enough to make a reboot a slow process. There's no real gain in a few moments on the init system when the hardware takes 5-10 minutes to boot anyway.
Keep the existing init system and everything that is Unix common. Keep RH techs useful in the real world of mixed environments.
Expert
823 points
I agree that hardware boot time (BIOS or EFI init & POST) is the dominant variable...but what about VMs? They don't have that issue and could benefit from a faster OS boot.
On the other hand, even for VMs booting only happens once every month or three (assuming you reboot to apply kernel security patches every now & then)...so we're back to, why put a lot of effort & extra complexity into something that doesn't need to be super-optimized for servers?
Prioritizing decisions like this is not nearly as easy as it looks, which is why I am very happy to see Red Hat at least soliciting direct customer input to help in the process.
Guru
3262 points
Leam: which Red Hat specific tools are you referring to as dumb?
While I can see the benefit of having standard tools and nomenclature across different Unix variants, some vendors who have adopted "standard" tools have already ditched SysV init functionality in favor of greener pastures (see: Solaris & SMF). There are components of systemd and SMF that are a serious, marked improvement over conventional SysV init, including:
- Dependency order. Services sometimes depend on one another for proper operation, and a robust system should know each service's dependencies. If an underlying service fails, it needs to be corrected before other services that depend upon it are affected.
- Configurable boot verbosity
- Delegation of tasks to non-root users. A service can be configured to run within a limited set of privileges, rather than as the all-powerful root user. If a service has been compromised, the amount of damage that can be inflicted by the intruder will be minimized if the service's power is constrained to that of a more limited user.
- Parallel starting of services. This speeds up the boot process by starting multiple services simultaneously, allowing idle CPU time resulting from a service that is temporarily blocked to be relinquished for use by other services that can start independently of the blocked service.
- Automatic service restart after failure. Works in conjunction with the Solaris Fault Manager, allowing software recovery in the event of hardware faults (cpu, memory), admin error such as accidental kills, and software core dumps.
(this list is taken directly from the SMF Wiki)
Automatic service restart is a big one for us. In a 4000+ node pure EL environment, it is "somewhat common" to see a service fail, and having to deal with homebrew or engineering-provided service watchdogs (or needing to write Puppet ensure directives) for two dozen different products is a headache that could be easily trivialized away.
The init system is crappy, inflexible and old, there should be little doubt about that.
You talk of 'standard Unix' methods while
- Solaris has since 10 (2004) moved over to the Service Management Facility
- Debian, Ubuntu, RHEL6 and a lot of others have, often haphazardly, moved over to Upstart
- HP-UX (to my knowledge) is still on init, but it's implementation is in some very crucial ways different
- AIX acts like it's on init, but it's really not, and the system is vastly different.
So init might've been standard once, but it hasn't been for a long time as better solutions have (rightly) been developed.
Besides, all of these, including systemd, provide some form of backward compatibility.
Bottom line is it's already a mess, and that should not stop Red Hat from bettering their product. Any admin who isn't capable of coping with mixed environments (where the boot/service system is probably not at the top of the worry-list anyway) should either be severely limited in his freedom of administration or be demoted to 1st line helpdesk (or janitor).
SystemD is much much more than just about improving boot times, and before you call something 'dumb' you should know and try that.
I'm not saying it's perfect, far from it, but either SystemD or Upstart are most definitely the way forward.
I don't think it is a good idea to consider limiting the software that is used in RHEL due to limited cross-system skill and training of some administrators. If they are professional administrators then I would expect them to be competent enough to know the differences between RHEL and other linux distributions. Otherwise it presents an opportunity for RedHat training to bring people up to speed.
That said, Lennart is actively promoting systemd to other distributors as well. The boot time performance benefit is not a primary benefit, it is secondary at best. It is not expected to be a RHEL specific feature.
-alan
You need to work with more than just legacy RedHat. As others have pointed out, several other (commercial) flavors of *N*X replaced init a couple of iterations ago - mostly for the better (sometimes, it's really nice to have service auto restart themselves if they've failed or enforce start-sequences for related/layered services).
As to hardware taking too long to reboot: you can avoid that with `kexec` (one of the RH employees posted a topic on that not too long ago). I know in our case, it takes an HP DLxxx system's reboot from being a 15-minute process to being a two-minute process.
As to "keep RH techs useful": you need to hire better techs. Qualified administrators adapt quickly and easily to new methods. They may grouse about it, but the good ones tend to like new things that ultimately make their lives easier.
SysV was a silly idea and systemd is just flogging the poor dead horse.
BSD style init is much more suitable for servers and vastly simpler to configure. One central config file and two run levels is clean, neat and simple.
Expert
823 points
Oh, please no! I really don't like the BSD init setup--it's too simple, and therefore actually fairly difficult to manage programmatically (tools like 'chkconfig' operating on a single file with no sequence numbering system? Third-party commercial software installers getting the mods right?). How 'bout seeing all the daemons--both 'system' and 'local'--that are supposed to be running by just doing an 'ls' (which generally fits on one screen, unlike the BSD rc files, or even 'chkconfig' output).
I think SysV init actually strikes a nice balance between simplicity and flexibility. I'm less fond of the modern whiz-bang options like 'launchd' (Apple) or SMF (Sun), since they add a layer of opacity that can be a bit annoying.
*nix booting is a procedural event, and any layer that adds opacity to that is a step in the wrong direction in my opinion. If as much effort has been put into fixing the little things that are frustrating in BSD init (or sysv) as has been put into making init easier to manage for non-kooliad-drinking admins, we wouldn't have 15 different init systems to deal with.
Let's face it; linux admins should be able to manage their own hardware. Dbus belongs on ubuntu desktops for grandmothers and non-technical people. It should not be forced on advanced admins under init, who don't want to run it; I would be very sad to see it be forced upon future redhat releases.
BSD init is clean and simple, and it doesn't try to hide the true nature of the boot process from the user. It adds an extremely easy to understand way for admins to granularly configure how the system boots and that's all we should need.
I work in an enviroment with 5 uinix versions, keeping to the same startup approach helps the admins who have to move between OS types consistantly; and even in the RHEL space we have 3, 4,5 and 6; so having a different startup approach for RHEL 7 would just make life even more complex - of course, if we only had RHEL 7 it would not be a problem.
When changing startup solutions, you need to consider the time taken for vendors to change their code to use the new approach, this seems to take a very long time, so whatever you do, you will need to support the current approach for a few years.
I agree with Leam, keep the standard sysv init, its been in use for years and many developers are used to the platform. What is the point in changing it, are there any pros dramatic enough for a change of this magnitude to something that runs great as it is?
Pro
678 points
RE: "Heck no! Every location I've been in has been a mixed environment. If a Red Hat tech comes in with all the dumb RH specific tools and no knowledge of standard Unix methods, they are useless as an admin."
Systemd is designed around existing FreeDesktop.org standards and interfaces, DBus et al. It's what most of us have wanted for Linux for much of the past decade as GNOME 2 and other desktop environments became standard. Even for console-only, non-X servers, Systemd is the right move forward.
- http://www.freedesktop.org/wiki/Software/systemd
Like many things Red Hat addresses with standard organizations like Apache, FreeDesktop, Xorg, etc..., they become universal standards. Systemd is yet another that everyone will adopt, it's just that powerful, just like DBus messaging before it.
Legacy, script-based init is a problem on its own. Real-time, interactive service and facility control is not something to be shunned. And it's hardly Red Hat-centric. ;)
-- Bryan
P.S. I still remember people complaining about Red Hat "going off on its own" with GLibC 2, ANSI C++ compliance and other things. What are all distros doing today? Red Hat leads by example, with a lot of developers. ;)
With the enterprise environment in mind, most of us will be dealing with admins that have to 'shift gears' between releases. As we already do with many other things, ie. LVM didn't become a norm in the environments I support until RHEL5. So we have RHEL3/4 with traditional partitions and RHEL5/6 with LVM everywhere. It can cause issues if someone starts doing the wrong thing on the wrong server.
I am not a fan of Systemd, at all. Boot speed is not a concern in the enterprise. That is evident by looking at the reboot time of a SUN or AIX box. It takes 32 approvals to reboot a server anyway. And as for running every service as root, someone above might want to take a look at "runas". But I believe it comes down to how it is implemented and closely the management of the services can "mimic" the legacy SysV management.
Change is unfortunately here to stay.
Pro
678 points
Boot speed is not the reason for Systemd. If boot speed was the only reason, then Upstart (Fedora 9-14 and Enterprise Linux 6) works "good enough." But Upstart never addressed many reasons for moving away from a legacy, script-based init system. Based on some of the comments here, I can only assume some individuals have not managed Enterprise Linux 6 yet, because it's already using a new init replacement in Upstart.
Systemd finally brings a fully integrated, message-based service interaction and management system. Again, it's the primary reason for moving away from Upstart, and really justifies why we shouldn't be using legacy, script-based init. I always felt Upstart did not, and the move wasn't justified. Service management without service messaging and control is incomplete. Hence why Systemd is even more justified.
As far as the commentary on LVM, volume management exists for a reason. Beyond just Linux, most enterprise platforms use volume management. DeviceMapper facilities in kernel 2.6 (EL4+) pretty much made LVM2 and Multipath a no-brainer adoption. You either have the facilities and capability, or you do not. In fact, kinda makes the same point.
Without Systemd, you have no way to manage services and their interaction. Kinda like without LVM, you have no way to manage volumes in the same regard. "Raw" init has the same problem as "raw" storage, and they should be avoided at all costs for that reason.
systemd has a lot of design points that make it much, much, better for server installations. Just some of them:
- A clean execution environment for services, whether started at boot or by the administrator. No more leaking of environment details into the service.
- Full monitoring of basic server state. The init daemon knows, and can be queried to, when the service started, when it stopped, if it failed, what the exit code was, and so on. It also means that service state can't be fooled by a missing pid file, etc.
- Availability of this init system data via a real API, not just via parsing commandline output and reading PID files, leading to better service management tools.
- cgroup monitoring of services. By using the kernel cgroup functionality, it can properly track a service and it's children, whether they fork away or not. This means service stopping is more complete and reliable.
- Full support for a variety of features for limiting services - uid/gid, ulimit, cgroups, private namespaces for filesystems and networks, among others
Guru
2135 points
To the nay-sayers - please read up on the subject. systemd provides MUCH more than "just" a speedy replacement for init.
I'm used to init.d as much as anyone. But having got my head round the way systemd works, it's clear why it was developed.
People resist change. But this is a GOOD change.
There are backwards compatibility wrappers to ensure chkconfig and service <name> restart work as before.
The introduction of cgroups to help control things is just excellent.
This change gets all of my votes.
Duncan, please elaborate, if you are going to make a post like that, give us some examples.. I ask because I am torn about this change. I have used systemd before, and I am very familiar with init, so I prefer init because of years of use, but I am not opposed to change. I am, though, opposed to change for change sake. If it is implemented poorly, or just because they want to, that warrants no change to me.
So please, elaborate, give me some ideas as to why you think it is such a "GOOD" change.
Guru
2135 points
Systemd goes much further than simply replacing the mechanism by which daemons start.
Systemd changes things from what is essentially a single thread of startup (i.e. start a, then b, then c, then d etc.) to a tree of dependencies. The dependency structure allows the system to start processes whenever they are capable of being started rather than simply waiting their turn.
This view, however, puts too much focus on the booting of a machine for me. It is something that I welcome for desktop and laptop users, but is of little relevance for my servers.
What is of huge benefit to my servers, on the other hand, is the use of cgroups. This gives me the ability to control and monitor system resources much better than before. To me, this is one of the killer features. To quote from the http://fedoraproject.org/wiki/Systemd page:
For each process spawned, it controls: The environment, resource limits, working and root directory, umask, OOM killer adjustment, nice level, IO class and priority, CPU policy and priority, CPU affinity, timer slack, user id, group id, supplementary group ids, readable/writable/inaccessible directories, shared/private/slave mount flags, capabilities/bounding set, secure bits, CPU scheduler reset of fork, private /tmp name-space, cgroup control for various subsystems. Also, you can easily connect
stdin/stdout/stderrof services tosyslog,/dev/kmsg, arbitrary TTYs. If connected to a TTY for input systemd will make sure a process gets exclusive access, optionally waiting or enforcing it.
There's a huge amount in there that is just completely new in enabling better control & monitoring of my systems.
It also extends the system control mechanism to cover mount points and snapshotting amongst others. I've not looked too much into the other areas yet as I haven't had a need/chance to.
The point I was making was just that much of the no 'campaign' seems to be based on resisting change rather than an understanding of the technology and infrastructure beneath it all. I also don't mean to have a go at anybody at any stage. It's just that I've used these features on more than just a desktop and have seen first-hand the benefits. I've then gone on to investigate systemd and have looked into what more I can make it do for me.
HTH
D
See, strictly for the one paragraph
For each process spawned, it controls: The environment, resource limits, working and root directory, umask, OOM killer adjustment, nice level, IO class and priority, CPU policy and priority, CPU affinity, timer slack, user id, group id, supplementary group ids, readable/writable/inaccessible directories, shared/private/slave mount flags, capabilities/bounding set, secure bits, CPU scheduler reset of fork, private /tmp name-space, cgroup control for various subsystems. Also, you can easily connect
stdin/stdout/stderrof services tosyslog,/dev/kmsg, arbitrary TTYs. If connected to a TTY for input systemd will make sure a process gets exclusive access, optionally waiting or enforcing it.There's a huge amount in there that is just completely new in enabling better control & monitoring of my systems.
I think think this would be a great idea. I tend to be wary of the changes from Fedora to RHEL sometimes, only because to make the desktop functionality great, some of those features are truly counter-productive for people like me (Systems Engineers). When I want my tune my RHEL servers to peak performance, some of the changes they have done to tune the GUI for instance, I want nothing to do with.
I think cgroups would be a welcome addition, and I could find quick use for this. In my environment we are scanned for security holes daily, and so our policies are much more strict because of this (not a bad thing), but the way the security is done here is not the most ideal, strictly because we need more than LDAP or AD authentication.
From my experience with Fedora 16 and 17 systemd isn't completly baked yet.
I like the dbus integration and think it could be usefull for alot of things especialy for the potential for improving highavalibility tools. There are a few other things I like about it aswell in theory; however the cut over of many of the current startup scrits were done quickly, and weren't well thought out as a result.
There was also a failure to take the opertunity to redesign components that are showing there age a great example of this are the network scripts.
While the scripts worked well when they were written they are still built around thet old ifconfig and route commands. A modern implementation should be built around iproute2. I cant even tell you how many times ive run into situations where the current scripts were deficient for specialized needs that you run into in larger environments. Just this past year I ran into a situation where I had to create a static arp entry on a bridge interface which is easy to do with iproute2 but not possible through the current network scripts. Dont ask why I needed to do that its a very long story but it was the only way to deal with an issue with integrating two odd vendor designed production environments that were never ment to talk to each other. To be fair I was quickly able to write a hack to do it but this is a feature thats been in the kernel for about a decade. There are also new features poping up in the kernel that I can see the current scripts having difficulty with that we will want like VMAC support for HA clusters.
Guru
2135 points
I'd completely agree that it isn't fully baked in Fedora 16 or 17. If 18 ever gets to see the light of day ;-) I suspect that the implementation will be complete.
Also totally agree on the network scripts side of things. It works, but it's not ideal. I've not looked into how things could be better, but your suggestion seems good. I would hope that when RHEL7 rolls out the door a) systemd is fully baked b) items such as network scripts are revisited to improve their functionality.
D
Red Hat
Guru
3719 points
You're wrong about network scripts in RHEL being built around the old ifconfig and route commands. Starting with RHEL5, virtually all the network scripts were rewritten to use iproute2 tools in place of the old net-tools commands.
There are still a few scripts in RHEL 5 & 6 that use ifconfig because of incompatibilities with the ip command, e.g. ifup-aliases; however, if you look, you'll find ifconfig in zero of Fedora18's network scripts.
EDIT: ifup-aliases was fixed in RHEL6 to use the ip command exclusively. In RHEL6, only 4 ifup scripts use ifconfig -- ippp, isdn, plip, and plusb.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.