JavaScript Promises ... In Wicked Detail

09 February 2014

I’ve been using promises in my JavaScript code for a while now. They can be a little brain bending at first. I now use them pretty effectively, but when it came down to it, I didn’t fully understand how they work. This article is my resolution to that. If you stick around until the end, you should understand promises well too.

We will be incrementally creating a promise implementation that by the end will mostly meet the Promises/A+ spec, and understand how promises meet the needs of asynchronous programming along the way. This article assumes you already have some familiarity with promises. If you don’t, promisejs.org is a good site to checkout.

Why?

Why bother to understand promises to this level of detail? Really understanding how something works can increase your ability to take advantage of it, and debug it more successfully when things go wrong. I was inspired to write this article when a coworker and I got stumped on a tricky promise scenario. Had I known then what I know now, we wouldn’t have gotten stumped.

The Simplest Use Case

Let’s begin our promise implementation as simple as can be. We want to go from this

There is a problem here. If you trace through the execution, you’ll see that resolve() gets called before then(), which means callback will be null. Let’s hide this problem in a little hack involving setTimeout

This Code is Brittle and Bad

Our naive, poor promise implementation must use asynchronicity to work. It’s easy to make it fail again, just call then() asynchronously and we are right back to the callback being null again. Why am I setting you up for failure so soon? Because the above implementation has the advantage of being pretty easy to wrap your head around. then() and resolve() won’t go away. They are key concepts in promises.

If you open up the console, you’ll see an error about the callback not being a function, because then() was called in a setTimeout.

Promises have State

Our brittle code above revealed something unexpectedly. Promises have state. We need to know what state they are in before proceeding, and make sure we move through the states correctly. Doing so gets rid of the brittleness.

A promise can be pending waiting for a value, or resolved with a value.

Once a promise resolves to a value, it will always remain at that value and never resolve again.

(A promise can also be rejected, but we’ll get to error handling later)

Let’s explicitly track the state inside of our implementation, which will allow us to do away with our hack

It’s getting more complicated, but the caller can invoke then() whenever they want, and the callee can invoke resolve() whenever they want. It fully works with synchronous or asynchronous code.

This is because of the state flag. Both then() and resolve() hand off to the new method handle(), which will do one of two things depending on the situation:

The caller has called then() before the callee calls resolve(), that means there is no value ready to hand back. In this case the state will be pending, and so we hold onto the caller’s callback to use later. Later when resolve() gets called, we can then invoke the callback and send the value on its way.

The callee calls resolve() before the caller calls then(): In this case we hold onto the resulting value. Once then() gets called, we are ready to hand back the value.

Notice setTimeout went away? That’s temporary, it will be coming back. But one thing at a time.

With promises, the order in which we work with them doesn’t matter. We are free to call then() and resolve() whenever they suit our purposes. This is one of the powerful advantages of capturing the notion of eventual results into an object

We still have quite a few more things in the spec to implement, but our promises are already pretty powerful. This system allows us to call then() as many times as we want, we will always get the same value back

This is not completely true for the promise implementation in this article. If the opposite happens, ie the caller calls then() multiple times before resolve() is called, only the last call to then() will be honored. The fix for this is to keep a running list of deferreds inside of the promise instead of just one. I decided to not do that in the interest of keeping the article more simple, it’s long enough as it is :)

Chaining Promises

Since promises capture the notion of asynchronicity in an object, we can chain them, map them, have them run in parallel or sequential, all kinds of useful things. Code like the following is very common with promises

getSomeData is returning a promise, as evidenced by the call to then(), but the result of that first then must also be a promise, as we call then() again (and yet again!) That’s exactly what happens, if we can convince then() to return a promise, things get more interesting.

Hoo, it’s getting a little squirrelly. Aren’t you glad we’re building this up slowly? The real key here is that then() is returning a new promise.

Since then() always returns a new promise object, there will always be at least one promise object that gets created, resolved and then ignored. Which can be seen as wasteful. The callback approach does not have this problem. Another ding against promises. You can start to appreciate why some in the JavaScript community have shunned them.

What value does the second promise resolve to? It receives the return value of the first promise. This is happening at the bottom of handle(), The handler object carries around both an onResolved callback as well as a reference to resolve(). There is more than one copy of resolve() floating around, each promise gets their own copy of this function, and a closure for it to run within. This is the bridge from the first promise to the second. We are concluding the first promise at this line:

var ret = handler.onResolved(value);

In the examples I’ve been using here, handler.onResolved is

function(value) {
console.log("Got a value:", value);
}

in other words, it’s what was passed into the first call to then(). The return value of that first handler is used to resolve the second promise. Thus chaining is accomplished

Who wants that crud in their code? Let’s have the promise implementation seamlessly handle this for us. This is simple to do, inside of resolve() just add a special case if the resolved value is a promise

We’ll keep calling resolve() recursively as long as we get a promise back. Once it’s no longer a promise, then proceed as before.

It is possible for this to be an infinite loop. The Promises/A+ spec recommends implementations detect infinite loops, but it’s not required.

Also worth pointing out, this implementation does not meet the spec. Nor will we fully meet the spec in this regard in the article. For the more curious, I recommend reading the promise resolution procedure.

Notice how loose the check is to see if newValue is a promise? We are only looking for a then() method. This duck typing is intentional, it allows different promise implementations to interopt with each other. It’s actually quite common for promise libraries to intermingle, as different third party libraries you use can each use different promise implementations.

Different promise implementations can interopt with each other, as long as they all are following the spec properly.

Other than the addition of reject() itself, handle() also has to be aware of rejection. Within handle(), either the rejection path or resolve path will be taken depending on the value of state. This value of state gets pushed into the next promise, because calling the next promises’ resolve() or reject() sets its state value accordingly.

When using promises, it’s very easy to omit the error callback. But if you do, you’ll never get any indication something went wrong. At the very least, the final promise in your chain should have an error callback. See the section further down about swallowed errors for more info.

Unexpected Errors Should Also Lead to Rejection

So far our error handling only accounts for known errors. It’s possible an unhandled exception will happen, completely ruining everything. It’s essential that the promise implementation catch these exceptions and reject accordingly.

What is going to happen here? Our callback inside then() is expecting some valid JSON. So it naively tries to parse it, which leads to an exception. But we have an error callback, so we’re good, right?

Nope.That error callback will not be invoked! If you run this example via the above fiddle, you will get no output at all. No errors, no nothing. Pure chilling silence.

Why is this? Since the unhandled exception took place in our callback to then(), it is being caught inside of handle(). This causes handle() to reject the promise that then() returned, not the promise we are already responding to, as that promise has already properly resolved.

Always remember, inside of then()‘s callback, the promise you are responding to has already resolved. The result of your callback will have no influence on this promise

If you want to capture the above error, you need an error callback further downstream

In my experience, this is the biggest pitfall of promises. Read onto the next section for a potentially better solution

done() to the Rescue

Most (but not all) promise libraries have a done() method. It’s very similar to then(), except it avoids the above pitfalls of then().

done() can be called whenever then() can. The key differences are it does not return a promise, and any unhandled exception inside of done() is not captured by the promise implementation. In other words, done() represents when the entire promise chain has fully resolved. Our getSomeJson() example can be more robust using done()

done() also takes an error callback, done(callback, errback), just like then() does, and since the entire promise resolution is, well, done, you are assured of being informed of any errors that erupted.

done() is not part of the Promises/A+ spec (at least not yet), so your promise library of choice might not have it.

Recovering from Rejection

It is possible to recover from a rejected promise. If you pass in an errback to then(), from then on any further promises in this chain will be resolved instead of rejected:

Promise Resolution Needs to be Async

Early in the article we cheated a bit by using setTimeout. Once we fixed that hack, we’ve not used setTimeout since. But the truth is the Promises/A+ spec requires that promise resolution happen asynchronously. Meeting this requirement is simple, we simply need to wrap most of handle()‘s implementation inside of a setTimeout call

This is all that is needed. In truth, real promise libraries don’t tend to use setTimeout. If the library is NodeJS oriented it will possibly use process.nextTick, for browsers it might use the new setImmediate or a setImmediate shim (so far only IE supports setImmediate), or perhaps an asynchronous library such as Kris Kowal’s asap (Kris Kowal also wrote Q, a popular promise library)

Why Is This Async Requirement in the Spec?

It allows for consistency and reliable execution flow. Consider this contrived example

What is the call flow here? Based on the naming you’d probably guess it is invokeSomething() -> invokeSomethingElse() -> wrapItAllUp(). But this all depends on if the promise resolves synchronously or asynchronously in our current implementation. If doAnOperation() works asynchronously, then that is the call flow. But if it works synchronously, then the call flow is actually invokeSomething() -> wrapItAllUp() -> invokeSomethingElse(), which is probably bad.

To get around this, promises always resolve asynchronously, even if they don’t have to. It reduces surprise and allows people to use promises without having to take into consideration asynchronicity when reasoning about their code.

Promises always require at least one more iteration of the event loop to resolve. This is not necessarily true of the standard callback approach.

Before We Wrap Up … then/promise

There are many, full featured, promise libraries out there. The then organization’s promise library takes a simpler approach. It is meant to be a simple implementation that meets the spec and nothing more. If you take a look at their implementation, you should see it looks quite familiar.

At the time this article was written, the final result looked very much like the then/promise implementation. That’s no longer true, they’ve completely rewritten their promise implementation

There are some differences in the real implementation and what is here in this article. That is because there are more details in the Promises/A+ spec that I have not addressed. I recommend reading the spec, it is short and pretty straightforward.

Conclusion

If you made it this far, then thanks for reading! We’ve covered the core of promises, which is the only thing the spec addresses. Most implementations offer much more functionality, such as all(), spread(), race(), denodeify() and much more. I recommend browsing the API docs for Bluebird to see what all is possible with promises.

Once I came to understand how promises worked and their caveats, I came to really like them. They have led to very clean and elegant code in my projects. There’s so much more to talk about too, this article is just the beginning!

Further Reading

More great articles on promises

promisejs.org – great tutorial on promises (already mentioned it a few times)

Q’s Design Rationale – an article much like this one, but goes into even more detail. By Kris Kowal, creator of Q