Testing Environment

Introduction

One of the features that make the Insight Journal different from any other Journal is that it enforces Reproducibility.

This is one of the most fundamental principles of the scientific method. The result of scientific work must be such that they can be replicated and reproduced by others. When this fundamental condition of the scientific work is skipped, then darkness and confusion arise.

The Insight Journal provides the infrastructure for replicating the work reported in any of the paper submitted for publication.

This process is achieved by making use of CMake, a cross platform build tool.

Implementation

When a paper is submitted to the Journal, it may contain additional files with source code and image data. A script will explore the additional files and if it finds a CMakeLists.txt file on them, then it will attempt to configure, build and test the software. The results of this attempt are posted to the Insight Journal dashboard in a format very similar to the one of the current ITK and VTK Dashboards.

The environment in which the software is tested is equivalent to a virtual Linux machine, where the IJ can provide a controlled, secured and standarized environment.

The code will be configured, compiled and run using the following available tools:

CMake 2.6.2

ITK 4.3.1

ITK 3.14.0

ITK 3.12.0

ITK 3.10.0

ITK 3.6.0

ITK 3.4.0

ITK 3.2.0

ITK 2.8.1

ITK 2.4.1

ITK 2.2.1

ITK 1.8.1

VTK 5.0

VTK 4.4

FLTK 1.1.6

Sun Java 1.4.2

Python 2.4.2

Tcl 8.4

CableSwig (CVS checkout on 3/29/2006)

ImageCompare

It is therefore convenient for authors to test their code using a similar environment before they submit their contribution to the Journal. In this way they can minimize the chances of encountering problems due to differences between their usual build environment, and the one offered by the Insight Journal.

Image Compare

This tool is a simple command line application that reads two images and compares their content. It has a built-in degree of tolerance, so the images don't have to be identical pixel by pixel. The tool is used at the end of execution of every submitted test in order to compare the authors results with the ones that have been produced when running the test in the Testing Environment. You can download Image Compare from here.