Lessons learned and musing about software tools, software testing, computational experiments, optimization, operations research, and other interesting stuff that I run across...

Friday, January 20, 2012

Testing Open Source Software

Software testing is widely recognized as a best practice for software development. Software tests define expected functionality, and they can focus developer efforts by providing an objective assessment of the state of a software project. Additionally, software testing data can provide evidence that a software package can be reliably used. For example, when evaluating whether to try out open source software, I routinely look for software testing data to confirm which platforms the software will run on, the versions of associated software that is used, and test coverage statistics that indicate how much of the the code is tested.

Unfortunately, most open source software projects do not publish software test data. I suspects that this indicates that a small fraction of OSS projects have robust test suites. However, this also reflects another aspect of the OSS community: hosting facilities for open source software do not support web-based testing facilities, like Jenkins, that can be used by developers to remotely launch jobs on test machines with a variety of different configurations. This is not totally unexpected, since testing can be computationally intensive.