If you are seeing this it is because you have a browser against which I am discrimating. You can still view synthesist.net, but there are no guarantees about the quality of the experience. I use a lot of CSS, which means that the layout that you see may appear strange.

The Natural History of Software Platforms

(c) 2004 David Stutz

A friend recently asked me to explain how Ray Ozzie's interesting observations about software integration, interoperability, and standards might be reconciled with my own beliefs about the inexorable commoditization of successful packaged software and the long, slow, decline of the packaged shrinkwrap software industry. Ray understands the far-reaching impact that technical decisions have on business outcomes as well as anyone that I've ever met, and re-reading his essay was serendipitous. Just last week, at the Open Source Business Conference, I had heard Clay Christensen devote a large part of a long lecture to his theory of "the conservation of attractive profits" (a phenomenon that he also calls "the conservation of modularity"). To me, this theory bears directly upon Ray's assertions about the business impact of software architecture, and more generally upon several "truths" that I've observed during my career in the software industry, but have never been able to explain succinctly.

I believe that anyone trying to create the proverbial "really good business" in software should read and understand Christensen's thoughts. Unfortunately, those who look for it in Christensen's latest book, "The Innovator's Solution," will find only a short write-up, which in my opinion doesn't do the idea justice. (Hopefully, he will expand on it in the future!) In lieu of that, here is a short description of "the conservation of attractive profits" in Christensen's own words [Harvard Business Review]:

Products are most profitable when they are not "good enough" to satisfy customers' needs. This is because to make them performance competitive, engineers must use proprietary, dependent architectures. Use of such architectures makes product differentiation straightforward, because each company pieces its parts together in a unique way.

Once a product's performance is good enough, companies must change the way that they compete. The innovations for which customers will pay premium prices become speed to market and the ability responsively and conveniently to give customers exactly what they need, when they need it. To compete in this way, companies are forced to employ modular architectures for products. Modularity causes the products to become undifferentiable and commoditized. Attractive profits don't evaporate, however...

They move elsewhere in the value chain, often to subsystems from which the modular product is assembled. This is because it is improvements in the subsystems, rather than the modular product's architecture, that drives the assembler's ability to move upmarket towards more attractive profit margins. Hence, the subsystems become decommoditized and attractively profitable.

In line with Christensen's theory, I would assert that every "pure software" company that has had large-scale success first offered their customers enhanced productivity in the form of packaged proprietary software, followed later by a redefinition of that software as a platform to be used by customers for rapid customization and their extensibility needs. (Such platforms, besides solving customer problems, also have the salutary effect of ensnaring those same customers and extenders within their "ecosystems," slowly raising switching costs over time. Think of customers and small-scale participants in these ecosystems as boiled frogs...)

Platforms exist at many layers of a software system, and slowly come and go as their usefulness waxes and wanes over many product cycles. Programs like Ozzie's Groove and Notes, Microsoft Office, or even Nullsoft's Winamp are application-level platforms. The Java API, coupled with a Java virtual machine, is a middleware platform, as is Microsoft's Common Language Runtime and its managed libraries. Windows and Linux, of course, are operating system platforms. Even low-level device subsystems can be platforms; stackable filesystems, "miniport drivers," and constellations of hardware devices that depend upon standardized interconnects such as IEEE 1394 or USB all rely upon platform-like dynamics to sustain their obscure (if not lively) ecosystems.

Platform Gravity

Software that is built atop a platform typically takes the form of modules, framework extensions, or custom configurations. Such software is usually relatively easy to build, and because of this, it is also undifferentiable over the long haul. Platform evangelists earnestly try to convince ISVs of the viability and potentially large marketplace that will exist for components plugins, and extensions, whether these be small Perl or Python scripts, EJBs, Visual Basic "custom controls," browser plugins, plug-and-play devices, or web services. Meanwhile, those sage ISVs who have tracked platforms and tools over time recognize these "opportunities" as low-margin swamp land that will eventually be polluted with nothing but hard-to-maintain, barely functional, code and abandonware. Component marketplaces and integrated solutions built on the shifting sands of not-yet-commoditized software are seldom profitable, and always fraught with maintenance problems.

Despite their flaws, software platforms such as application executables, virtual machines, operating systems, and driver subsystems, represent very real value to those who build and maintain them. Their impossible-to-reproduce code has almost always accreted over time into monolithic and monumental hairballs, crisscrossed with compromises and architectural peculiarities that reflect the history of adaptations to ship schedules, underlying infrastructure, bugfixes and patches, and changing customer requirements. Of course, these hairballs present a very different face to the community that extend and customize them. From the outside, these platform hairballs, Christensen's "decommoditized" pieces of the value chain, are surrounded by well-defined boundaries such as easy-to-use APIs, elegant UIs for customization, and standardized protocols.

Because mature and successful platform ecosystems are so well-defined and so difficult to change, the vendors who build them invariably place themselves at risk of commoditization through cloning. Christensen refers to "modularity" as the necessary ingredient for customization, which ultimately leads to non-differentiability. Within the software industry, however, this usage of the word "modularity" is confusing. To me, a better choice of words would be "cloneable." Because of the monolithic nature of platforms, the emergence of modularity in software is not typically accomplished through retrofitting or platform rework. Instead, it occurs through a hardening of the external shell presented by the platform over time. As a platform succeeds in the marketplace, its APIs, UI, feature-set, file formats, and customization interfaces ossify and become more and more difficult to change. (They may, in fact, ossify so far as to literally harden into hardware appliances!) The process of ossification makes successful platforms easy targets for cloners, and cloning is what spells the beginning of the end for platform profit margins.

Value Waves

The sandwiching of highly proprietary and value-laden pieces of code between standardized module boundaries is the process that sets up Christensen's theories about the migration of value over time. Platforms that succeed will follow their destiny towards being "good enough," at which point their users will start valuing customization and response over new features. The migration of margin-rich opportunities is not always "up the value chain" as Tim O'Reilly and others have asserted. Instead, the value chain acts as a carrier for slowly traveling waves, on which higher-margin business opportunities move both up and down. Examples of this wave phenomenon abound: consider the current commoditization, intense political activity, and shake-out that is occurring within the Java virtual machine ecosystem. Implementations of the Java VM will lose much of their proprietary value over time, but as they do, complex proprietary frameworks written in Java are likely to appear above the level of the newly ubiquitous virtual machines, while below them new hosts will also appear to bring Java code to entirely new markets. Likewise, printer drivers were once commoditized by word processors, Microsoft Office is currently being commoditized by OpenOffice, Internet Explorer will eventually be commoditized by Mozilla, Opera, Safari, and other browsers, and the Unix API has nearly been completely commoditized by Linux. New platforms emerge above and below freshly commoditized layers, exploiting the standardized interfaces and newly "free" infrastructure. Attractive profits are conserved, not destroyed.

Modularity Cannot be Rushed

New platform efforts that set out to be highly modular or deeply object-oriented from day one are doomed to fail, because they are recognized as being vulnerable to cloning by potential ecosystem members and are consequently not adopted because of their lack of business potential. Truly useful standards always come after the fact, and often without a formal standardization process. Successful platforms ossify, leaving standardized patterns of information in their wake.

How can I be so sure about my hypotheses? An informal test that I have used over the years is to check what I'm saying against papers and presentations written by Jim Gray and Butler Lampson. I am convinced that, between the two of them, they have already followed every interesting premise that exists within the domain of computer architecture. Sure enough, Butler Lampson's presentation on components seems to cover much of what I've talked about here!

Ray Ozzie's essay optimistically points out opportunities that exist for new integrated client-side application software that will exploit standardized schema to achieve new levels of interoperability, while at the same time preserving proprietary value by exposing rich data relationships and user interface. It also hints at how and why the architectural decisions embedded into such software can yield sticky network externalities that guarantee business longevity and exponential growth. These are useful observations, but I assert that his conclusions about the failure of componentized architectures is due as much to the platform effect that I am discussing as to technical deficiencies. Furthermore, I believe that the new platforms that he is discussing are just as likely to suffer the same fate.

Microsoft's heavily touted WinFS storage subsystem, should it ever see the light of day, represents an interesting new platform, one that is could be extremely useful and therefore support a large ecosystem. In the absence of commodity (replaceable) implementations, however, its schema-based extensibility will fail to become universal in the same way that Microsoft component models have failed to become drivers of commodity value. Microsoft is very unlikely to seek to erode their margins by what people in Redmond sometimes refer to as "premature standardization." Because of this, although I agree with many of Ray's observations, I disagree with his predicted outcome. Rather than becoming the basis for a powerful network driven by the commodity exchange of schematized XML data, WinFS is more likely to be just one more in a string of proprietary Microsoft extensibility mechanisms.

Profiting from Platforms

The platform effect - ossification, followed by cloning - is how Christensen-style modularity comes to exist in the software industry. What begins as a value-laden proprietary platform becomes a replaceable component over time, and the most successful of these components finally define the units of exchange that power commodity networks.

There is huge value to be captured from commodity networks, but it is not to be found in the production of the underlying software resources. Instead, this value can be found in the distribution of platform-standardized information, and also in the form of political power. Microsoft, for example, recognizes that it is likely to accrue huge profits in the future by controlling distribution networks for digital media and web-published content; their pursuit of the search engine business and of DRM are both strategic moves in this direction that go beyond media players and productivity apps.

In general, the owners and maintainers of mature software platforms should augment the production of packaged software with strategies that exploit distribution of the commodity by-products of their platforms. By doing this, they are likely to avoid the terminal side-effects of commoditization, while continuing to derive benefits from their platform assets.