Yet Another JavaScript Testing Framework

Did you notice that there are a lot of JavaScript testing frameworks out there? Why has the JavaScript community not consolidated on a single JavaScript framework the way Java has on JUnit. My feeling is that all of these frameworks are good at something but none solve the complete package. Here is what I want out of JavaScript unit-test framework:

I want to edit my JavaScript code in my favorite IDE, and when I hit Ctrl-S, I want all of the tests to execute across all browsers and return the results immediately.

I don’t know of any JavaScript framework out there which will let me do what I want. In order to achieve my goal I need a JavaScript framework with these three features:

Command Line Control

Most JavaScript test runners consist of JavaScript application which runs completely in the browser. What this means in practice is that I have to go to the browser and refresh the page to get my results. But browsers need an HTML file to display, which means that I have to write HTML file which loads all of my production code and my tests before I can run my tests. Now since browsers are sandboxes, the JavaScript tests runner can only report the result of the test run inside the browser window for human consumption only. This implies that 1) I cannot trigger running of the tests by hitting Ctrl-S in my IDE, I have to Alt-tab to Browser, hit refresh and Alt-tab back to the IDE and 2) I cannot display my test result in my IDE, the results are in the browser in human readable form only.

On my continuous build machine I need to be able to run the same tests and somehow get the failures out of the browser and on to the status page. Most JavaScript test runners have a very poor story here, which makes integrating them into a continuous build very difficult.

What we need, is the ability to control test execution from command line so that I can trigger it from my IDE, or my continuous build machine. And I need test failures to be reported on the command line (not inside the browser where they are unreachable) so that I can display them in IDE or in continuous build status page.

Parallel Execution

Since most JavaScript test runners run fully in the browser I can only run my test on one browser at a time during my development process. In practice this means that you don’t find out about failures in other browser until you have checked in the code to your continuous build machine (if you were able to set it up) and your code executes on all browsers. By that point you have completely forgotten about what you have written and debugging becomes a pain. When I run my test I want to run them on all browser platforms in parallel.

Instant Feedback in IDE

After I hit Ctrl-S on my IDE, my patience for test results is about two seconds before I start to get annoyed. What this means in practice is that you can not wait until the browser launches and runs the tests. The browser needs to be already running. Hitting refresh on your browser manually is very expensive since the browser needs to reload all of the JavaScript code an re-parse it. If you have one HTML file for each TestCase and you have hundred of these TestCases, The browser may be busy for several minutes until it reloads and re-parses the same exact production code once for each TestCase. There is no way you can fit that into the patience of average developer after hitting Ctrl-S.

Jeremie Lenfant-engelmann and I have set out to build a JavaScript test runner which solves exactly these issues so that Ctrl-S causes all of my JavaScript tests to execute in under a second on all browsers. Here is how Jeremie has made this seemingly impossible dream a reality. On startup JsTestDriver captures any number of browsers from any number of platforms and turns them into slaves. As slave the browser has your production code loaded along with all of your test code. As you edit your code and hit Ctrl-S the JsTestDriver reloads only the files which you have modified into the captured browsers slaves, this greatly reduces the amount of network traffic and amount of JavaScript re-parsing which the browser has to do and therefore greatly improves the test execution time. The JsTestDriver than runs all of your test in parallel on all captured browsers. Because JavaScript APIs are non-blocking it is almost impossible for your tests to run slow, since there is nothing to block on, no network traffic and no re-parsing of the JavaScript code. As a result JsTestDriver can easily run hundreds of TestCases per second. Once the tests execute the results are sent over the network to the command which executed the tests either on the command line ready to be show in you IDE or in your continuous build.

HiLooks very nice! Could it be, that there is an older version on google code then in the video? i get a “”–browser” is not a valid option”-message. After starting it with –browserPath, i get a “Error 404 NOT_FOUND”.Thanks!Alain

BTW, it’s very cool.It’s not truly headless, because you have to start the browser(s) that you are testing in. How do you accomplish this, say, in a Windows engironment (or a headless Linux environment)?

[...] Google recently released a new Javascript testing framework, JS Test Driver. It provides incredibly fast execution for Javascript unit tests, and can be run from the command line without the need for manual control of browsers. Check out this introduction to JS Test Driver by Miško Hevery. [...]

[...] integration. Misko talks a little bit more about the motivations behind writing it on his Yet Another JS Testing Framework post. To try out the plugin for yourselves, go add the following update site to [...]

[...] integration. Misko talks a little bit more about the motivations behind writing it on his Yet Another JS Testing Framework post. To try out the plugin for yourselves, go add the following update site to eclipse : [...]

[...] integration. Misko talks a little bit more about the motivations behind writing it on his Yet Another JS Testing Framework post. To try out the plugin for yourselves, go add the following update site to [...]

Yes we would be interested in helping. One thing that JsTestDriver’s goals is to become THE way to run your test, not another assertion framework. For this reason JsTestDriver allows for plugins which allow you to put any assertion framework on top of it. We already have QUnit, YahooTest, and are working on Jasmine. Since we have already taken care of capturing the browser(s) running in continues builds and ability to remotely control the test, IDE plugins, I think it is a very nice fit to TestSwarm. JsTestDriver would take care of Running the test, TestSwarm would create a farm of JsTestDrivers, and than the user could choose any assertion framework on top of it to run their tests.

We even took care of on the fly instrumentation of JS code for code coverage and injection and clean up of DOM between tests, and remote logging. So I think this would be a great fit.

Im relatively new to jstestDriver and Im evaluating JavaScript test frameworks for an upcoming project My question is “does jstestDriver support
dispatching jobs to a testswarm server or is this support planned? If so
how does/will one come about that?

Sorry, we do not yet integrate with test swarm, and currently do not have the bandwidth to add it, but would love to help anyone who is interested in adding that support as we are an open source project. Having said that JsTestDriver supports out of the box running tests on multiple browsers in a continuous build environment, so the need for something like Test Swarm is bit reduced.