Testing as the Open Source Killer App

Kim Polece, from SpikeSource, is talking about software testing in open source software. She starts by talking about the architecture of participation. This architecture is characterized by:

Commoditization of software

Network-enabled collaboration

Software customizability

and the shift from an "egosystem" to and open, thriving ecosystem.

Kim shows a power curve and talks about pahses in open source adoption. In the first phase, we buit and buit with, the tall end, left end of the power curve (Linux, php, Python, Mozilla, etc). In the second, phase, further to the roght on the tail of the curve, countless new building materials are piling up on the curve. Kim shows a list of these from just onee company that they talked to. There were dozens of build tools, runtime and class libraries in the list.

There are some problems:

Velocity mismatch. This refers to the release schedules for the multiple open source projects. Coordinating release schedules between components and managing compatibility is difficult to do.

Dependencies. This is not unique to open source, but its compounded by the variation and number of components. When you patch one component of your stack, does the entire stack get hosed?

The largest independent IT shops formalize their DIY proceses for building with open source. Smaller shops don't have that luxery.

This leads to phase thre: IT becomes core and outsources the infrastructure tasks, including testing, certification, and so of open source packages. Testing is the biggest single refacoring shift in computig today. Its at the core of managing dependencies and velocity mismatch. We need testing on a massive scale.

Now a word from our sponsor: this is what SpikeSource does.

Testing has been the ugly stepchild of software for as long as people have been writing code. Microsoft has a 1:1 ratio of QA to developers. The run 500,000 test scenarios for any given product line. Thhere are 100,000 open source products already. How can as scale this?

To solve testing on a masive scale, you need participation by the community and automation. This is just one more architecture of participation, going back to Tim's talk. Testing is just one service among many in the open source market place. Developers and users benefit from a pervasive testing regime.

Testing will do for open source what it did for chip design a generation ago. It made possible chips that couldn't be built before.