Package management. No really. How hard can it be?

We had a little discussion at work some time ago, again, involving package management. No, I’m not talking about the .deb versus .rpm debate. I am talking about source dependencies for JavaScript and PHP projects. But the problem holds equally well for OS package management or build dependencies.

Currently, we are using a fine blend of Composer, Bower and NPM. Coincidentally, another one is peeking around the corner, because Composer uses a tool internally, which uses a manifest.json to identify the source files as coming from some source repository, namely github, so the package is able to update itself1.

Thank heavens, we’re at least agreeing on a common definition format being JSON. At least, the syntax of it, not the semantics…

The knights who say ‘NIH!’

NIH. The “Not invented here” syndrome. There are plenty of tools out there, but lets just do it again, but now the way we think is best. It’s called a syndrome, which should give you an idea of it’s benefits. Syndromes usually don’t constitute good health. That counts for something.

We fail to see that effectively all these tools do the exact same thing. Some file is loaded. It is checked for dependencies on other packages. This is done recursively, ending up in a list of selectable versions, then this list is narrowed down using some kind of version resolution scheme, such as preferring stable over development and explicit version dependencies. When this is complete, a list of installable packages remains which can be downloaded from some kind of source, usually some site hosting the packages or github. Not that hard, right? Right.

There is one big “but”. The description of dependencies are part of the packages, i.e., their source distribution. So now, the source tree must have some kind of declaration of which dependencies it has. And here we go. We are building a PHP project, so we must have some tool that does the job, in PHP. Also, we’re building front end javascript and sass, so we need some tool doing the exact same thing for front end assets. And, last but not least, we tend to use NPM for Javascript tooling that is not hosted in some bower registry. Hence, Composer, Bower and NPM.

Awful. We fail to see that the one thing we need to publish in our source code is the dependencies. If we can all agree on some kind of standard for this, the choice of package management can be mine, whether it is composer, npm, bower, dselect, yum, what-freaking-ever-pm I set my mind on today. But the knights who say NIH, don’t usually sit down to think things through. And they tend to squable over futilities. Me included.

Knights of the round table, then?

Here’s my thought. Let’s think of a standard. And lets use that in any package manager we use. Now, we get rid of all composer.json, package.json, bower.json and whatever Python, Ruby, Erlang or Brainfuck version there is out there and let’s just hold hands and sing a mantra:

A package manager exists to download stuff in the right version for me. Downloading is easy. Dependency resolution is easy (though it might take some time if the graph is big). Agreeing on how to make these dependencies explicit seems to be hard. But it really isn’t. All you need is a file saying “I need this package in that version”. It could look just like this:

symfony/symfony: 2.6.*
jquery: 2.*
node-sass: *@stable

Kinda looks like YML, doesn’t it. But let’s set the preferences of serialization format X versus Y aside. Let’s agree that at least one format is useful. Or, in RFC style, any package manager implementing the spec COULD support multiple file formats reflecting the same structure. It MUST implement at least one.

Now, we need some tool to translate these package names to download locations. Wait. That doesn’t need to be a tool. It could be a hosted service… No wait. Don’t we already know something like that? Like a sources.list in Debian? Or yum.repos in Fedora? Then, finally, we need a way to identify where the packages need to be stored, once downloaded. Whether that’s ./node_modules, ./vendor, /usr/local, /, or "C:\Program Files" really is just Yet Another Config Value.

We don’t need more package managers. We need a specification. A standard. And it really shouldn’t be hard. Hell, I think I almost summed it up already. Once we do that, we can let go of the NIH, get ourselves a pair of halves of a coconut shell2 and start prancing around whinnying and enjoy ourselves just a little bit better.