I currently have 40 tests with approximately 200 assertions running. When all of them are run, the same two are almost always failing. Run individually, or with about half of all the available tests, they pass. The total running time is typically 8-10 seconds.

If I run *all* the tests with the developer console open (set up so that the browser cache is not used), the same two tests *always* fail - the total running time is closer to 30 seconds).

Almost all of these tests involve the following steps:

1. (re)set some default values2. define the following: var done =assert.async();3. load some files using ajax calls, with async set to false; I'm running these tests locally, using Python http.server.4. executing some code using eval() on the content of one of these files. (Most often, it is Brython's exec() which is used; Brython translates Python code in the browser into Javascript and executes it. As such, it is slower than simply using eval() on some pure javascript code.)5. check some assertions6. end the test with a call to done().

By inspecting closely the result, I found that, when a test has failed, the code run by exec/eval appeared to have been cut short - as though the state was corrupted by another test running (but this may be a red herring). [For example: a successful run would create an array containing 360 objects but a failed one, with the same test, would create an array containing less than 300 objects.]

Questions:1. am I using async()/done() correctly?2. Any suggestion as to what could be done to fix this?

Replies(1)

My use of eval() apparently resulted in some of the tests not being atomic. While I no longer observe any failure, ensuring that I redefine "everything" at the beginning of each test results in a five-fold increase in the time required to run all the tests ... but at least, I no longer get spurious failure reports.