Open Source and the Long Tail

I’ve been thinking recently about open source software and the notion of the long tail. (See Chris Anderson’s blog, book, and Wired article for the canonical description of the whole “Long Tail” meme).

One way to approach open source and the long tail is to focus on the sheer volume of available open source projects: the notion that there are a few highly visible projects (the equivalent of the best sellers, or the “head” against which the long tail is opposed) and that there are many many more smaller projects with less visibility (the long tail).

A deeper, more interesting relationship between open source and the long tail is the way that open source development methodologies, distribution techniques, and licenses enable the development of custom solutions for niche problems.

In other words, open source enables the development of the long tail of software applications.

It isn’t just that the projects found in the long tail (the “bottom” 80% of projects in SourceForge based on activity) might provide value, but that even the top 20% of open source projects provide a way to solve uncommon needs. Because they provide both the core framework (what we might call the 80% solution) but also license and source code access, to those who might want to develop the 20% which remains.

I can take a solution or framework which addresses the 80% of my needs which are common, and develop just the bits necessary to make it fit my unique, niche, long-tail need.

Simon Phillips described this phenomena in large and highly active open source communities as a code keiretsu: a community, gathered around a source commons . . .[with] many parts acting independently but to each other’s benefit.” When such a community evolves around a core application, a virtuous circle develops: “As the long tail develops, more and more developers join the keiretsu, each targeting a different area of the long tail, each benefiting from the code commons by being fast to their market yet with commodity-level costs.”

Steve Borsch, similarly, offers “the ecosystem delivering plug-ins, add-ons, modules, components, themes and other chunks of functionality that somebody, somewhere wants to use with some open source software project” as “a potentially more profound example of The Long Tail.”

Joe Krause from JotSpot (and formerly Excite) similarly identifies the notion of a long tail for software, pointing out that “there is a long tail of very custom process problems that software is supposed to help business solve.” Further, he points out some very specific reasons that software has failed to address the problem so far:

Software’s long tail has been generally inaccessible because software has been

Too difficult to write

Too expensive to write and distribute

Too brittle or expensive to customize once deployed.

It just hasn’t been economical . . . That’s why . . . the traditional focus has been on dozens of markets of millions instead of millions of markets of dozens.

He goes on, ultimately, to position JotSpot as a solution, describing them as “building a platform to make it easy and affordable to build long-tail software applications.”

While I think JotSpot’s an interesting platform, and I can easily see areas where what they’re building will be quite useful, I’d argue that the platform which makes long-tail applications possible is open source itself. (JotSpot also makes itself available to open source projects for free).
(Another interesting example is Chris Anderson’s presentation, described in this blog entry, at the annual salesforce.com user meeting, which positions AppExchange as enabling the same kind of solution).

Open source distribution has a second-order effect of making it easier to produce highly-customized applications by reducing the cost of development, distribution, and maintenance. That may mean finding a small, less visible project that already does what you need, or, more likely, finding a large, mature, and active community with a project which already does most of what you need and customizing it.

In the interest of full disclosure, it is only fair to mention that from one perspective this is exactly what Optaros, where I work, does. Our assembly methodology is designed to benefit from such effects as I’ve described. But the reality is true whether you are doing it on your own or with the help of a systems integration and consulting firm like ours.