JavaScript Testing Tool Showdown: Sinon.js vs testdouble.js

When unit testing real-world code, there are many situations that make tests hard to write. How do you check if a function was called? How do you test an Ajax call? Or code using setTimeout? That’s when you use test doubles — replacement code that makes hard to test things easy to test.

For many years, Sinon.js has been the de-facto standard in JavaScript tests for creating test doubles. It’s a must-have tool for any JavaScript developer writing tests, as without it, writing tests for real applications would be nigh impossible.

Recently, a new library, aptly named testdouble.js, has been making waves. It boasts a similar feature set as Sinon.js, with a few differences here and there.

In this article, we’ll look into what both Sinon.js and testdouble.js offer, and compare their respective pros and cons. Will Sinon.js remain the superior choice, or will the challenger take the prize?

Note: If you’re unfamiliar with test doubles, I recommend reading my Sinon.js tutorial first. It will help you better understand the concepts we’ll be talking about here.

Terminology Used in This Article

To ensure it’s easy to understand what is being discussed, here’s a quick overview of the terminology used. These are the definitions for Sinon.js, and they can be slightly different elsewhere.

A test double is a replacement for a function used during a test. It can refer to any of the three types mentioned below.

A spy is a test double which allows the checking of effects without affecting the behavior of the target function.

A stub is a test double which replaces the target function’s behavior with something else, such as returning a value.

A mock is a different approach to stubs. Mocks contain built-in verification and can be used instead of a separate assertion.

It should be noted that one of the goals of testdouble.js is to reduce the confusion between this type of terminology.

Sinon.js and testdouble.js at a Glance

Let’s begin with a look at how Sinon.js and testdouble.js compare in basic usage.

Sinon has three separate concepts for test doubles: Spies, stubs and mocks. The idea is that each represents a different usage scenario. This makes the library more familiar to those coming from other languages or who have read books using the same terminology, such as xUnit Test Patterns. But the other side is that these three concepts can also make Sinon more difficult to understand when first using it.

In contrast, testdouble.js opts for an API which is more straightforward. Instead of using concepts like spies or stubs, it uses language much more familiar to JavaScript developers, such as td.function, td.object and td.replace. This makes testdouble potentially easier to pick up, and better suited to certain tasks. But on the other hand, some more advanced uses may not be possible at all (which is sometimes intentional).

The language used by testdouble is more straightforward. We “replace” a function instead of “stubbing” it. We ask testdouble to “explain” a function to get information from it. Other than this, so far it’s fairly similar to Sinon.

This also extends to creating “anonymous” test doubles:

var x = sinon.stub();

vs.

var x = td.function();

Sinon’s spies and stubs have properties which offer more information about them. For example, Sinon provides properties such as stub.callCount, and stub.args. In testdouble’s case, we get this information from td.explain:

One of the bigger differences relates to how you set up your stubs and verifications. With Sinon, you chain commands after a stub, and use an assertion to verify the result. testdouble.js simply has you show it how you want the function to be called — or how to “rehearse” the function call.

This can make testdouble’s API easier to understand, since you don’t need to know what operations you can chain and when.

Comparing Common Testing Tasks in More Detail

On a high level both libraries are reasonably similar. But what about common testing tasks that you might need to do in a real project? Let’s take a look at a few cases where the differences start to show.

testdouble.js has no spies

The first thing to note is testdouble.js has no concept of a “spy”. While Sinon.js allows us to replace a function call so that we get information from it, while keeping the function’s default behavior, this it not possible at all with testdouble.js. When you replace a function with testdouble, it always loses its default behavior.

This is not necessarily a problem however. The most common usage for spies would be using them to verify callbacks were called, which is easily doable with td.function:

While not a big issue, it’s still good to know this difference exists between the two, as otherwise you may be surprised if you expect to be able to use spies in some more specific manner with testdouble.js.

testdouble.js requires more precise inputs

The second difference you’ll run into is testdouble is stricter about inputs.

Both Sinon’s stubs and assertions allow you to be imprecise about what parameters are given. This is easiest illustrated by an example:

By default, Sinon doesn’t care how many extra parameters are given to a function. While it provides functions such as sinon.assert.calledWithExactly, those are not suggested as the default in the documentation. Functions like stub.withArgs also do not come with an “exactly” variant.

testdouble.js on the other hand defaults to requiring the exact parameters specified. This is by design. The idea is that if a function is given some other parameters unspecified in the test, it’s potentially a bug, and should fail the test.

It is possible to allow specifying arbitrary parameters in testdouble.js, it just isn’t the default:

In this case, we use callsArgWith instead of yields. We have to provide the specific index of the call for it to work, which can be a bit fiddly especially on functions with many parameters.

What if we want to call both callbacks with some values?

var x = td.function();
td.when(x(td.callback('a', 'b'), td.callback('foo', 'bar'))).thenReturn();
//callback1 is called with 'a' and 'b'
//callback2 is called with 'foo' and 'bar'
x(callback1, callback2);

With Sinon, this is not possible at all. You can chain multiple calls to callsArgWith, but it will only ever call one of them.

testdouble.js has built-in module replacement

In addition to being able to replace functions using td.replace, testdouble lets you replace entire modules.

This is mainly useful in situations where you have a module which directly exports a function which you need to replace:

module.exports = function() {
//do something
};

If we want to replace this with testdouble, we can use td.replace('path/to/file'), for example…

While Sinon.js can replace functions which are members of some object, it cannot replace a module in a similar fashion to this. To do this when using Sinon, you need to use another module such as proxyquire or rewire

Another thing worth noticing about module replacement is testdouble.js replaces the entire module automatically. If it’s a function export like in the example here, it replaces the function. If it’s an object containing several functions, it replaces all of them. Constructor functions and ES6 classes are also supported. Both proxyquire and rewire require you to individually specify what to replace and how.

Typically the sandbox and sinon.test methods are recommended in practice, as otherwise it’s very easy to accidentally leave stubs or spies in place, which can then cause problems in other tests. This can result in hard to track cascading failures.

testdouble.js only provides one way of cleaning up your test doubles: td.reset(). The recommended way is to call it in an afterEach hook:

This greatly simplifies both set up of test doubles and cleaning up after tests, reducing the likelihood of hard to track bugs.

Pros and Cons

We’ve looked at the functionality in both libraries now. They both offer a similar feature set, but they have a somewhat different design philosophy from each other. Can we break this down into pros and cons?

Let’s first talk about Sinon.js. It provides some additional features over testdouble.js, and some aspects of it are more configurable. This affords it some increased flexibility in more special testing scenarios. Sinon.js also uses language more familiar to those coming from other languages — concepts such as spies, stubs and mocks exist in varying libraries and are discussed in testing related books as well.

The downside of this is added complexity. While its flexibility allows experts to do more things, it also means some tasks are more complicated than in testdouble.js. For those new to the concept of test doubles, it can also have a steeper learning curve. In fact, even someone like me who is very familiar with it can have trouble elaborating some of the differences between sinon.stub and sinon.mock!

testdouble.js instead opts for a somewhat simpler interface. Most of it is reasonably straightforward to use, and feels more intuitive for JavaScript, while Sinon.js can sometimes feel like it was designed with some other language in mind. Thanks to this and some of its design principles, it can be easier to pick up for beginners, and even experienced testers will find many tasks simpler to do. For example, testdouble uses the same API for both setting up test doubles and verifying the results. It can also be less error prone due to its simpler clean-up mechanism.

testdouble’s biggest problems are caused by some of its design principles. For example, the total lack of spies can make it unusable for some who prefer using them instead of stubs. This is something that is very much a matter of opinion, and you may not find a problem at all. Apart from this, testdouble.js is offering some serious competition to Sinon.js despite being a much more recent entry.

testdouble.js technically doesn’t have mocks in the way Sinon.js has them. However, since mocks in Sinon are essentially objects which contain stubs and verifications, a similar effect can be achieved by using td.replace(someObject)

Some similar effects to argument captors can be achieved by using stub.yield (not to be confused with stub.yields)

Summary and Conclusion

Both Sinon.js and testdouble.js provide a fairly similar set of functionality. Neither of them is clearly superior in this sense.

The biggest differences between the two are in their API. Sinon.js is perhaps slightly more verbose, while providing a lot of options on how to do things. This can be both its blessing and curse. testdouble.js has a more streamlined API, which can make it easier to learn and use, but due to its more opinionated design, some may find it problematic.

So which one is right for me?

Do you agree with testdouble’s design principles? If yes, then there’s no reason not to use it. I’ve used Sinon.js in many projects, and I can safely say testdouble.js does at least 95% of everything I’ve done with Sinon.js, and the remaining 5% is probably doable via some easy workaround.

If you’ve found Sinon.js difficult to use, or are looking for a more “JavaScripty” way to do test doubles, then testdouble.js might also be for you. Even as someone who has spent a lot of time learning to use Sinon, I’m inclined to recommend trying testdouble.js and seeing if you like it.

Certain aspects of testdouble.js can however cause headaches for those who know Sinon.js or otherwise are veteran testers. For example the total lack of spies can be a deal breaker. For experts and those who want the maximum amount of flexibility, Sinon.js is still a great choice.

If you want to learn more about how to use test doubles in practice, check out my free Sinon.js in the Real-World guide. Although it uses Sinon.js, you can apply the same techniques and best practices with testdouble.js as well.

Questions? Comments? Are you using testdouble.js already? Would you consider giving it a try after reading this article? Let me know in the comments below.