Revision Content

When finished, this page will give an overview of options for automated testing available to Mozilla developers with links to more documentation.

Most automated tests should be executed on <tt>make check</tt>. How to add a build-time test describes the steps required to add an arbitrary test program to the test suite. Depending on what you need to test, you can use one of the testing frameworks available:

XPCshell: make check

With xpcshell test harness you write your unit test in JavaScript. The code later runs in xpcshell, which is an XPConnect-enabled JS shell. This means your code can access XPCOM components, but can't (easily) open windows, test the application's chrome, work with HTML parser or the DOM.

Mochitest

{{wiki.template('Named-source', [ "testing/mochitest/README.txt", "Mochitest" ])}} is a framework based on Mochikit for writing tests. Tests run in the browser, from a local web server. The start scripts that accompany it grant several privileges to the localhost (it creates a fresh profile each run). As a result, unit tests are free to request UniversalXPConnect privileges, open popup windows, etc. Mochitest is a good fit if you really need a full browser to test a problem. For example, a recent test verifies that form submission works in iframes set to display:none.

Reftest

{{wiki.template('Named-source', [ "layout/tools/reftest/README.txt", "Layout Engine Visual Tests (reftest)" ])}} ({{template.Bug(344591)}}). Each test consists of two documents (e.g. HTML) - one of them containing test markup and the other containing reference markup. The system works by comparing the rendering of two documents.

reftest is currently not part of the build system, so in order to run tests in it, you have to reference it from the parent makefile by adding TOOL_DIRS += reftest in it above the rules.mk line. After running <tt>make</tt> in layout/tools, you can run a testsuite via <tt>firefox -reftest <manifest></tt>. An example manifest is here.

The readme in the tree is quite informational, but we may need a page walking through writing a reftest.

jssh

Requires an extension to be enabled at build time (via <tt>--enable-extensions=</tt>).

Works only on trunk. Needs to be post-branching merged back to 1.8

{{template.Bug(343199)}} has the patches to build jssh as a real extension

Further Reading

Please ignore the {{mediawiki.interwiki('wikimo', 'SoftwareTesting:Scratchpad', 'wikimo:SoftwareTesting:Scratchpad')}} page, and look at {{mediawiki.interwiki('wikimo', 'SoftwareTesting', 'wikimo:SoftwareTesting')}} only. The scratchpad is for works-in-progress, and is pretty much guaranteed to be stale or wrong.

There's also {{mediawiki.interwiki('wikimo', 'SoftwareTesting', 'wikimo:SoftwareTesting')}} and Automated testing tips and tricks if you're looking for something to read.

These are some additional efforts underway:

You can write standalone test programs in C/C++. This option can be used to test functionality not exposed via XPCOM.

{{template.Bug(343673)}} tracks one person's effort, but there has been no recent activity

{{template.Bug(346703)}} contains one example of how this might be done

JSUnit can be used to write tests that run as content in the browser. Particularly useful for DOM, parser tests, but can't do anything that requires chrome privileges.

JSUnit will most likely not be a make check target for the time being, since it needs a full browser instance.

Refer to documentation on {{mediawiki.interwiki('wikimo', 'SoftwareTesting:Tools:jsUnit', 'wikimo:SoftwareTesting:Tools:jsUnit')}} for more information.

Revision Source

<p>
</p><p>When finished, this page will give an overview of options for automated testing available to Mozilla developers with links to more documentation.
</p><p>Most automated tests should be executed on <tt>make check</tt>. <a href="en/How_to_add_a_build-time_test">How to add a build-time test</a> describes the steps required to add an arbitrary test program to the test suite. Depending on what you need to test, you can use one of the testing frameworks available:
</p>
<h3 name="XPCshell:_make_check"> XPCshell: make check </h3>
<p>With <a href="en/Writing_xpcshell-based_unit_tests">xpcshell test harness</a> you write your unit test in JavaScript. The code later runs in <a href="en/Xpcshell">xpcshell</a>, which is an <a href="en/XPConnect">XPConnect</a>-enabled JS shell. This means your code can access XPCOM components, but can't (easily) open windows, test the application's chrome, work with HTML parser or the DOM.
</p>
<h3 name="Mochitest"> Mochitest </h3>
<p>{{wiki.template('Named-source', [ "testing/mochitest/README.txt", "Mochitest" ])}} is a framework based on <a class="external" href="http://mochikit.com/">Mochikit</a> for writing tests. Tests run in the browser, from a local web server. The start scripts that accompany it grant several privileges to the localhost (it creates a fresh profile each run). As a result, unit tests are free to request UniversalXPConnect privileges, open popup windows, etc. Mochitest is a good fit if you really need a full browser to test a problem. For example, a recent test verifies that form submission works in iframes set to display:none.
</p>
<h3 name="Reftest"> Reftest </h3>
<p>{{wiki.template('Named-source', [ "layout/tools/reftest/README.txt", "Layout Engine Visual Tests (reftest)" ])}} ({{template.Bug(344591)}}). Each test consists of two documents (e.g. HTML) - one of them containing test markup and the other containing reference markup. The system works by comparing the rendering of two documents.
</p><p>reftest is currently not part of the build system, so in order to run tests in it, you have to reference it from the parent makefile by adding <code>TOOL_DIRS += reftest</code> in it above the rules.mk line. After running <tt>make</tt> in layout/tools, you can run a testsuite via <tt>firefox -reftest &lt;manifest&gt;</tt>. An example manifest is <a class="external" href="https://bugzilla.mozilla.org/attachment.cgi?id=230052">here</a>.
</p>
<ul><li><ul><li> <i>The readme in the tree is quite informational, but we may need a page walking through writing a reftest.</i>
</li></ul>
</li></ul>
<h3 name="jssh"> jssh </h3>
<p><a class="external" href="http://croczilla.com/jssh">jssh</a> runs JavaScript code as chrome.
</p>
<ul><li> Rumored by davel to be slow
</li><li> Requires an extension to be enabled at build time (via <tt>--enable-extensions=</tt>).
</li><li> Works only on trunk. Needs to be post-branching merged back to 1.8
</li><li> {{template.Bug(343199)}} has the patches to build jssh as a real extension
</li></ul>
<p><br>
</p>
<h3 name="Further_Reading"> Further Reading </h3>
<p>Please ignore the {{mediawiki.interwiki('wikimo', 'SoftwareTesting:Scratchpad', 'wikimo:SoftwareTesting:Scratchpad')}} page, and look at {{mediawiki.interwiki('wikimo', 'SoftwareTesting', 'wikimo:SoftwareTesting')}} only. The scratchpad is for works-in-progress, and is pretty much guaranteed to be stale or wrong.
</p><p>There's also {{mediawiki.interwiki('wikimo', 'SoftwareTesting', 'wikimo:SoftwareTesting')}} and <a href="en/Automated_testing_tips_and_tricks">Automated testing tips and tricks</a> if you're looking for something to read.
</p><p>These are some additional efforts underway:
</p>
<ul><li> You can write standalone test programs in C/C++. This option can be used to test functionality not exposed via XPCOM.
<ul><li> <i>{{template.Bug(343673)}} tracks one person's effort, but there has been no recent activity</i>
</li><li> <i>{{template.Bug(346703)}} contains one example of how this might be done</i>
</li></ul>
</li><li> <a class="external" href="http://www.jsunit.net/">JSUnit</a> can be used to write tests that run as content in the browser. Particularly useful for DOM, parser tests, but can't do anything that requires chrome privileges.
<ul><li> JSUnit will most likely not be a make check target for the time being, since it needs a full browser instance.
</li><li> {{mediawiki.interwiki('wikimo', 'SoftwareTesting#Ideas to Collect', 'wikimo:SoftwareTesting#Ideas_to_Collect')}} lists some jsunit examples.
</li><li> Refer to documentation on {{mediawiki.interwiki('wikimo', 'SoftwareTesting:Tools:jsUnit', 'wikimo:SoftwareTesting:Tools:jsUnit')}} for more information.
</li></ul>
</li></ul>