CMS Testing

From early Mambo days until today, Joomla! has constantly amended and extended functionality. With Joomla! 1.5, quality and user experience changed to the better, and it did so with version 2.5. Moving to GitHub has made it easier to contribute and to see what things different developers are working on. Now, the focus more and more goes to the next logical step: Automated tests.

Process

In Joomla!, we currently use three kinds of tests.

Unit Tests – testing small chunks of code (units) on the lowest possible level, usually methods, without including other classes or methods. These tests ensure that methods do their job as expected.

Integration Tests – testing the integration of two or more units. These tests cover the interaction between several units.

System Tests – testing the system as a whole. Testing an application through the browser belongs to this group.

To ensure the overall quality of Joomla!, a unit test coverage of more than 90% is needed. The current code base does not always allow unit tests. Caused by the current tight coupling of objects, we are mostly restricted to integration and system tests.

Test Driven Development

For the future it must be the goal to enforce Test Driven Development, ie., a developer writes a unit test first to document the expected behavior, and after that implements the code. Thus, a very high unit test coverage is guaranteed. Evolving the code base will never again break existing functionality, because that would be detected instantly.

Testable Objects

Utilizing Dependency Injection instead of the currently high coupling will enable unit tests throughout the project. An easy way to achieve this would be to make JFactory's methods non-static and pass an (the) instance of the factory as an argument. That way it is no problem to inject a mock factory that delivers mock objects for testing.

No Test – No Fix

To fix a bug, a test must be written to reproduce the problem. Ideally this would be a unit test, but often this will end in a system test. Then a solution is created to make the test succeed, which proves the fix.

Niels Braczek started working as a freelance software developer in 1983. Since 2005 he's bringing his experience with various programming languages (such as COBOL, C, Assembler) and runtime environments (from personal computers to mainframes) into the Joomla! project. As a Clean Code evangelist, Niels was one of the driving forces behind the introduction of automated software testing in Joomla!.