Why systemd is winning the init wars and other things aren't

February 11, 2014

Recently, an article by Rich Felker called Broken by design:
systemd has been making the rounds.
I have a number of things to say about this article but today
I want to talk about one specific issue it brings up, which is
systemd's novelty (or lack thereof) and why it is succeeding.
To start with, here is the relevant quote from Felker's article:

None of the things systemd "does right" are at all
revolutionary. They've been done many times before. DJB's
daemontools,
runit, and Supervisor, among others, have solved the "legacy init
is broken" problem over and over again (though each with some of
their own flaws). Their failure to displace legacy sysvinit in major
distributions had nothing to do with whether they solved the problem,
and everything to do with marketing. [...]

This is wrong on several levels. To start with and as usual, social
problems are the real problems. In
specific, none of these alternate init systems did the hard work
to actually become a replacement init system for anything much.
Anyone can write an init system, especially a partial one (I did
once, long ago). Getting it adopted by people is the hard part and
none of these alternatives tackled that effectively (if they did
so at all, and some of them certainly didn't). And as Felker admits,
each of these theoretical alternatives have flaws of their own.

(Note that this is not a criticism of those alternate init systems. I
don't think any of them have really been developed with replacing SysV
init in Linux distributions or elsewhere as a goal. DJB daemontools
certainly wasn't; I believe that DJB's attitude towards it, as towards
more or less everything he's developed, can be summed up as 'I showed
you the way, what you do with it is up to you'.)

The reason systemd has succeeded in becoming an SysV init replacement
is simple: it did the work. Not only did it put together a lot
of good ideas regardless of their novelty or lack
thereof but its developers put in the time and effort to convince
people that it was a good idea, the right answer, a good solution
to problems and so on. Then they dealt with lots and lots of practical
concerns, backwards compatibility, corner cases, endless arguments,
and so on and so forth. I want to specifically mention here that
one of the things the systemd people did was write extensive
documentation on systemd's design, how to configure and operate
it, and what sorts of neat things you can do with it. While this
documentation is not perfect, most init systems are an order of
magnitude less well documented.

(I am sure that in some quarters it's popular to believe that
Lennart Poettering bulldozed the Fedora technical people into
adopting his new thing. I do not think that the Fedora technical
people are that easily overrun (or that impressed by Poettering,
especially after PulseAudio), and for that matter at least some
of the Debian technical people feel that systemd is the best
option despite having looked deeply at the alternatives (cf).)

You can call this marketing if you want, although I don't think that
that's a useful label for what is really happening. I call this 'trying'
versus 'not trying'. If you don't try hard and work hard to become a
replacement init system, it should be no surprise when you don't.

(In particular, note that SysV init is not a particularly bad init
system so it should be no surprise when it is not particularly easy
to displace.)

Beyond that I have some degree of experience with one of these
alternate init systems, specifically DJB daemontools, and I've
looked at the documentation for the other two. Speaking as a system
administrator, systemd solves my problems better.
The authors of systemd have looked at problems that are not solved
by SysV init and come up with real solutions to them. Many of these
problems are not solved by any of the alternatives that Felker put
forward. In specific, often the alternatives assume (or require)
cooperative daemon processes in order to fully realize their benefits;
systemd is deliberately designed so that it does not and can fully
manage even existing obstreperous Unix daemons with their willful
backgrounding and other inconvenient behaviors.

(I don't know the field of Linux and Unix init-like systems well enough
to say whether or not features like socket activation and clever use of
control groups are genuinely novel in systemd or simply the first time
I've become aware of them. They do feel novel.)

Since that may not be clear, let me be plain: systemd is a better init
system than the alternatives. It does more to solve real problems and
it does it better. That alone is a good reason for it to win in the
practical world, the one where people care about getting stuff done.
That systemd is not necessarily novel or the first to come up with the
ideas that it embodies is irrelevant to this. Implementation matters
more than ideas.

(Arguably it's an advantage that systemd feels no urge to reinvent
different wheels when perfectly decent ones exist.)

PS: Please note that the reason that Unix itself succeeded is not its
ideas alone, it is that Unix implemented them very well. A number of
Unix's ideas are both great and novel, but a bad implementation would
have doomed the whole enterprise. The fate of good ideas with a bad
implementation is to be reimplemented elsewhere, cf the Xerox Alto and
for that matter the Apple Lisa.

PPS: Also note that the one serious competitor to systemd is Upstart,
which is also the product of a great deal of work and polishing.