Open Source Tools for Model-Based Testing

I promised to provide a list of open source and free model-based testing tools in a QUEST panel session, and have updated this list a few times since. There is wide variation in maturity, stability, and provisioning.

Nearly all of following information is from provider’s web sites – YMMV. The Environment column indicates the source code language and/or run-time environment(s); “*nix” indicates a tool written to run on most Linux/Unix variants. Open source licenses vary widely, click here for an overview.

If you note any errors or omissions, please post a comment and I will revise accordingly. A revision history follows the last table.

Supports conformance testing of reactive software; checks the correct behaviour of a real implementation during its execution based on its formal specification. Can execute a Torx specification as SUT.

Fielded Codebases

A model-based testing and analysis framework. User writes model programs in C#. NModel generates tests from your model program in advance or on-the-fly as the test run executes. Users may add custom test generation strategies in C# to achieve any coverage criteria. SUT not limited to Windows.;C#. Requires .NET, but not Visual Studio. Precursor to Spec Explorer; lacks many features (e.g., Cord Files) supported in Spec Explorer.

Model-driven development and testing environment for critical hw/sw embedded systems. Supports formal methods, SysML, UML, and OCL. Part of a suite of open source tools for embedded system development.

Java

Eclipse

Eclipse Public

Club Codebases

Tools whose codebase has a small user community using it for non-research work.

Generates offline and online test sequences from state machines modeled with GraphML — no UML to learn. User selects any of seven built-in coverage (stopping) criteria for test generation. Tool may be integrated with a Java test harness or called with SOAP as web service from the developer’s server. Use of graphics to represent models limits scalability.

A compiler/simulator/model-checker for a rich subset of UML diagrams. based on the VERIMAG IF model checker. May be used to validate and verify asynchronous timed system models with complex dynamics. Uses rigorous semantics suited for critical real-time embedded systems.

Models are written as Java classes which directly interact with a Java SUT or adapter. Uses the annotations feature of Java 5.0. Processes models to generate calls to the SUT and evaluate response (no Junit code is produced.)

The fMBT (Free Model-based Testing) tool is suitable for testing anything from individual C++ classes to GUI applications, mobile devices, and distributed systems as well as different platforms. fMBT provides a model editor, test generator, adapters for testing through various interfaces, and tools for analyzing logs.

Open Source Modelling Objects (OSMO) includes tools and libraries to model software behaviour with extended state machines in Java for online/offline test generation and non-deterministic online models.

I stumbled upon Gallio and MB Unit today. Gallio is a open source test automation platform for Windows intended to integrate many testing tools, including MbUnit. MbUnit is a Dot Net test automation framework compatible with NUnit and adds many features. Although MbUnit is said to mean “Model based Unit Testing Framework”, neither tool supports generation of tests from models. The site http://code.google.com/p/mb-unit/ lists over 20000 downloads of the most recent (2012) release.

Another candidate for your list is an open source tool that I’ve created: Tcases, a model-based test case generator. Documentation, downloads, and source code are all located here: https://github.com/Cornutum/tcases