Javascript performance: callback (async) vs Q ..

Promises/A+ Performance Hits You Should Be Aware Of

The Promises/A+ specification is a fresh and very interesting way of dealing with the asynchronous nature of Javascript. It also provides a sensible way to deal with error handling and exceptions. In this article we will go through the performance hits you should be aware of and as a side-effect do a comparison between the two most popular Promises/A+ implementations, When and Q and how they compare to Async, the lowest abstraction you can get on asynchronicity.

Basic nodejs single thread architecture:

The Case

My motivation for looking deeper into the performance of Promises/A+ was a Job Queuing system i’ve been working on named Kickq. It is expected that the system will get hammered when used on production so stress testing was warranted. After stubbing all the database interactions, essentially making the operation of job creation synchronous, I was getting odd performance results.

The test was simple, create 500 jobs in a loop and measure how long it takes for all the jobs to finish.

The measurements were in the ~550ms range and my eyeballs started to roll. “That’s a synchronous operation, it should finish in less than 3ms, WHAT THE????!?!”. After taking a few moments to let it sip in the suspect was found, it was Promises. I used them as the only pattern to handle asynchronous ops and callbacks throughout the whole project. Brian Cavalier, one of the authors of When.js, helped me pinpoint the real culprit, it was the tick:

Promises/A+ Specification, Note 4.1 In practical terms, an implementation must use a mechanism such as setTimeout, setImmediate, or process.nextTick to ensure that onFulfilled and onRejected are not invoked in the same turn of the event loop as the call to then to which they are passed.

In other words, Promises, per the specification, must be resolved Asynchronously! That comes with a cost, a heavy one apparently.

In the process of studying performance I had to create a performance library, poor mans profiling. And a benchmark test for Promises/A+ implementations that’s already used to optimize the future versions of When.

Creating The Promises/A+ Benchmark

I tried to broaden the definition of the test case. If an application uses the Promises pattern as the only way to manage how the internal parts interact, we can make a few assumptions:

There will be a series of promises chained together, representing the various operations that will be performed by your application.

The Deferred Object is used on each link of the chain to control resolution and how the promise object is exposed.

Throughout the whole chain of promises there can be operations that are actually synchronous, we will measure all cases.

Difference to First Resolved Promise, 500 Loops

Perf Type

Async

When 2.1.0

Q 0.9.5

Promise 3.0.1

Sync Diff

0.01ms

36.62ms

186.43ms

63.96ms

Mixed Diff

5.37ms

41.78ms

226.34ms

83.83ms

Async Diff

22.42ms

58.18ms

241.80ms

93.68ms

Sync Diff vs AsyncLib

1x

3,662x

18,643x

6,396x

Mixed Diff vs AsyncLib

1x

7.78x

42.15x

15.61x

Async Diff vs AsyncLib

1x

2.60x

10.79x

4.18x

Libraries When.js v1.8.1 and Deferred are not included in this table because they resolve promises synchronously. This difference makes the Diff metric inapplicable.