Embed URL

HTTPS clone URL

SSH clone URL

This article has been given a more permanent home on my blog. Also, since it was first written, the development of the Promises/A+ specification has made the original emphasis on Promises/A seem somewhat outdated.

You're Missing the Point of Promises

Promises are a software abstraction that makes working with asynchronous operations much more pleasant. In the most
basic definition, your code will move from continuation-passing style:

to one where your functions return a value, called a promise, which represents the eventual results of that operation.

var promiseForTweets = getTweetsFor("domenic");

This is powerful since you can now treat these promises as first-class objects, passing them around, aggregating them,
and so on, instead of inserting dummy callbacks that tie together other callbacks in order to do the same.

I've talked about how cool I think promises are at length. This essay isn't about that. Instead, it's
about a disturbing trend I am seeing in recent JavaScript libraries that have added promise support: they completely
miss the point of promises.

Thenables and CommonJS Promises/A

When someone says "promise" in a JavaScript context, usually they mean—or at least think they
mean—CommonJS Promises/A. This is one of the smallest "specs" I've seen. The meat of it is entirely about specifying
the behavior of a single function, then:

A promise is defined as an object that has a function as the value for the property then:

then(fulfilledHandler, errorHandler, progressHandler)

Adds a fulfilledHandler, errorHandler, and progressHandler to be called for completion of a promise. The
fulfilledHandler is called when the promise is fulfilled. The errorHandler is called when a promise fails. The
progressHandler is called for progress events. All arguments are optional and non-function values are ignored. The
progressHandler is not only an optional argument, but progress events are purely optional. Promise implementors are
not required to ever call a progressHandler (the progressHandler may be ignored), this parameter exists so that
implementors may call it if they have progress events to report.

This function should return a new promise that is fulfilled when the given fulfilledHandler or errorHandler
callback is finished. This allows promise operations to be chained together. The value returned from the callback
handler is the fulfillment value for the returned promise. If the callback throws an error, the returned promise will
be moved to failed state.

People mostly understand the first paragraph. It boils down to callback aggregation. You use then to attach
callbacks to a promise, whether for success or for errors (or even progress). When the promise transitions state—which
is out of scope of this very small spec!—your callbacks will be called. This is pretty useful, I guess.

What people don't seem to notice is the second paragraph. Which is a shame, since it's the most important one.

What is the Point of Promises?

The thing is, promises are not about callback aggregation. That's a simple utility. Promises are about something
much deeper, namely providing a direct correspondence between synchronous functions and asynchronous functions.

What does this mean? Well, there are two very important aspects of synchronous functions:

They return values

They throw exceptions

Both of these are essentially about composition. That is, you can feed the return value of one function straight into
another, and keep doing this indefinitely. More importantly, if at any point that process fails, one function in the
composition chain can throw an exception, which then bypasses all further compositional layers until it comes into the
hands of someone who can handle it with a catch.

Now, in an asynchronous world, you can no longer return values: they simply aren't ready in time. Similarly, you can't
throw exceptions, because nobody's there to catch them. So we descend into the so-called "callback hell," where
composition of return values involves nested callbacks, and composition of errors involves passing them up the chain
manually, and oh by the way you'd better never throw an exception or else you'll need to introduce something crazy
like domains.

The point of promises is to give us back functional composition and error bubbling in the async world. They do this
by saying that your functions should return a promise, which can do one of two things:

Become fulfilled by a value

Become rejected with an exception

And, if you have a correctly implemented then function that follows Promises/A, then fulfillment and rejection will
compose just like their synchronous counterparts, with fulfillments flowing up a compositional chain, but being
interrupted at any time by a rejection that is only handled by someone who declares they are ready to handle it.

Note in particular how errors flowed from any step in the process to our catch handler, without explicit by-hand
bubbling code. And with the upcoming ECMAScript 6 revision of JavaScript, plus some party tricks, the code becomes
not only parallel but almost identical.

That Second Paragraph

All of this is essentially enabled by that second paragraph:

This function should return a new promise that is fulfilled when the given fulfilledHandler or errorHandler
callback is finished. This allows promise operations to be chained together. The value returned from the callback
handler is the fulfillment value for the returned promise. If the callback throws an error, the returned promise will
be moved to failed state.

In other words, then is not a mechanism for attaching callbacks to an aggregate collection. It's a mechanism for
applying a transformation to a promise, and yielding a new promise from that transformation.

This explains the crucial first phrase: "this function should return a new promise." Libraries like jQuery (before 1.8)
don't do this: they simply mutate the state of the existing promise. That means if you give a promise out to multiple
consumers, they can interfere with its state. To realize how ridiculous that is, consider the synchronous parallel: if
you gave out a function's return value to two people, and one of them could somehow change it into a thrown exception!
Indeed, Promises/A points this out explicitly:

Once a promise is fulfilled or failed, the promise's value MUST not be changed, just as a values in JavaScript,
primitives and object identities, can not change (although objects themselves may always be mutable even if their
identity isn't).

Now consider the last two sentences. They inform how this new promise is created. In short:

If either handler returns a value, the new promise is fulfilled with that value.

If either handler throws an exception, the new promise is rejected with that exception.

This breaks down into four scenarios, depending on the state of the promise. Here we give their synchronous parallels so
you can see why it's crucially important to have semantics for all four:

Fulfilled, fulfillment handler throws an exception: getting data, and throwing an exception in response to it

Rejected, rejection handler returns a value: a catch clause got the error and handled it

Rejected, rejection handler throws an exception: a catch clause got the error and re-threw it (or a new one)

Without these transformations being applied, you lose all the power of the synchronous/asynchronous parallel, and your
so-called "promises" become simple callback aggregators. This is the problem with jQuery's current "promises": they only
support scenario 1 above, omitting entirely support for scenarios 2–4. This was also the problem with Node.js 0.1's
EventEmitter-based "promises" (which weren't even thenable).

Furthermore, note that by catching exceptions and transforming them into rejections, we take care of both intentional
and unintentional exceptions, just like in sync code. That is, if you write aFunctionThatDoesNotExist() in either
handler, your promise becomes rejected and that error will bubble up the chain to the nearest rejection handler just as
if you had written throw new Error("bad data"). Look ma, no domains!

So What?

Maybe you're breathlessly taken by my inexorable logic and explanatory powers. More likely, you're asking yourself why
this guy is raging so hard over some poorly-behaved libraries.

Here's the problem:

A promise is defined as an object that has a function as the value for the property then

As authors of Promises/A-consuming libraries, we would like to assume this statement to be true: that something that
is "thenable" actually behaves as a Promises/A promise, with all the power that entails.

If you can make this assumption, you can write very extensive libraries that are entirely agnostic
to the implementation of the promises they accept! Whether they be from Q, when.js, or even WinJS, you can
use the simple composition rules of the Promises/A spec to build on promise behavior. For example, here's a generalized
retry function that works with any Promises/A implementation.

Unfortunately, libraries like jQuery break this. This necessitates ugly hacks to detect the presence of objects
masquerading as promises, and who call themselves in their API documentation promises, but aren't really Promises/A
promises. If the consumers of your API start trying to pass you jQuery promises, you have two choices: fail in
mysterious and hard-to-decipher ways when your compositional techniques fail, or fail up-front and block them from using
your library entirely. This sucks.

The Way Forward

So this is why I want to avoid an unfortunate callback aggregator solution ending up in Ember. That's why I
wrote this essay, with some prompting from @felixge for which I am thankful. And that's why, in the hours following
writing this essay, I worked up a general Promises/A compliance suite that we can all use to get on the same
page in the future.

For example, at current time of writing, the latest jQuery version is 1.8.2, and its promises implementation is
completely broken with regard to the error handling semantics. Hopefully, with the above explanation to set the stage
and the test suite in place, this problem can be corrected in jQuery 2.0.

Since the release of that test suite, we've already seen one library, @wycats's rsvp.js, be released with the
explicit goal of providing these features of Promises/A. I'm hopeful others will follow suit. In the meantime, here
are the libraries that pass the test suite, and that I can unreservedly recommend:

Q by @kriskowal and myself: a full-featured promise library with a large, powerful API surface, adapters for
Node.js, progress support, and preliminary support for long stack traces.

rsvp.js by @wycats: a very small and lightweight, but still fully compliant, promise library.

when.js by @briancavalier: an intermediate library with utilities for managing collections of eventual tasks,
as well as support for both progress and cancellation. Does notguarantee asynchronous resolution.

If you are stuck with a crippled "promise" from a library like jQuery, I recommend using one of the above libraries'
assimilation utilities (usually under the name when) to convert to a real promise as soon as possible. For example:

I agree, but even with the understanding of 2nd paragraph, promises don't seem to be "the solution".

For example, WinJS code (~=40Klines) is extensively using Promises/A, and it's just the worst nightmarish code you would ever think of.
Adding Promises/A features to JS hopping it would solve asynch dev is like hopping semaphore would solve any problem in threading environment.

Developers will always misuse them & forget tests because the user may interact with the GUI changing objects sates between two then() in place the developer might not even think of.

imho, all this asynch dev paradigm will collapse if there is no strong language+framework answer to the whole problem.

@syndr0m: WinJS code is nightmarish for entirely different reasons. Their promise implementation is top-notch, and code you write that consumers WinJS promises has the potential to be quite beautiful. (The code you write to consume other aspects of WinJS can be pretty bad though.)

@domenic Thanks very much for writing this down and being so thorough. Having used jQuery's broken promises implementation in the past, I hadn't seen much of an advantage to using them over the usual callback pattern. This makes it crystal clear why promises are so powerful when you actually have a then that works. Cheers.

@mjijackson: thanks for the kind words! I've updated the gist to explain exactly what's still missing in the current jQuery implementation (under "That Second Paragraph"), and also add some advice for how to assimilate jQuery promises into real promises (in the new "The Way Forward" section).

Very interesting post. I'll admit to being someone who didn't properly understand how to implement a Promise API correctly and was just using it as an aggregation of handlers.

I've since re-written the primary implementation of my Promise (in db.js) but I did so modeling it on top of jQuery's current implementation (1.8.2). @domenic are you saying that jQuery is still broken in their implementation or is it now "fixed" (from my investigation it does seem like it's fixed).

@dos-one, @aaronpowell: yes, jQuery is indeed (still) broken. I recently updated the article to spell this out a bit more:

This is the problem with jQuery's current "promises": they only support scenario 1 above, omitting entirely support for scenarios 2–4.

That is, jQuery promises do not support any of the error-related cases, but instead only the simple functional composition cases. So they've managed to do callback aggregation and, using pipe or the >= 1.8 then, they've managed to flatten chains of async operations so you can avoid nesting. But those chains of async operations do not behave like chains of sync operations, in terms of error handling, so while your code gets a bit prettier, it's still missing the point.

I've used promises for quite some time, but ended up going back to nodejs-style callbacks and creating a bunch of higher-order functions that makes handling that easier and reduce the boilerplate code. The two most useful ones, that really made it much easier, deal with error handling/bubbling: (CoffeeScript)

iferr =(errfn, succfn)->(e, a...)->if e?then errfn e else succfn a...throwerr = iferr.bind null, (e)->throw e
# To let an error bubble up, `iferr` decorates the success function and the error function, and decide which one should handle the response# This replaces `if (err) return fn err` that you see all over the placegetLastTweetBody =(user, fn)->
getTweetsFor user, iferr fn, (tweets)->
expandUrlUsingTwitterApi (parseTweetsForUrls tweet)[0], iferr fn, (url)->
doHttpRequest url, fn
# To throw the errors when you don't have any other way to handle them, the function can be decorated with `throwerr`# This replaces `if (err) throw err` that you see all over the place
getLastTweetBody 'domenic', throwerr (body)->console.log body

With those and a couple more functions, I find writing code in that style much more manageable. The code is quite readable, there isn't much boilerplate, and you don't get the overhead of promises (which persist the promise's response, making GC somewhat problematic).

You could add so much clarity to this article by adding something like this at the first mention of jQuery:

At current time of writing, the latest jQuery version is 1.8.2, and its promises implementation (and all versions before it) is broken. I write this, hoping some future version may address the problems detailed below.

It was hard to tell if there are working jQuery versions around, from not knowing when you wrote this, if some OK versions exist, and if 1.8 was. Maybe not impossible, just hard.

Finally got around to reading this. It's a really great explanation. It raises some questions for me, and I'll try to work those up into examples that can be discussed. But this is a good reference for explaining the scope of promises.

hi @domenic, what part of the jquery promise implementation is broken in 1.8.2? You mention that the error handling is "broken", can you please elaborate more? As you'll see by this js fiddle, if an error happens early in the chain it is aggregated properly through the error handlers as expected.

@danshultz: here is the list of failing Promises/A tests for jQuery 1.8.2 from my promise-tests repo:

[Promises/A] Chaining off of a fulfilled promise
when the first fulfillment callback throws an error
1) should call the second rejection callback with that error as the reason
[Promises/A] Chaining off of a rejected promise
when the first rejection callback returns a new value
2) should call the second fulfillment callback with that new value
when the first rejection callback throws a new reason
3) should call the second rejection callback with that new reason
[Promises/A] Chaining off of an eventually-fulfilled promise
when the first fulfillment callback throws an error
4) should call the second rejection callback with that error as the reason
[Promises/A] Chaining off of an eventually-rejected promise
when the first rejection callback returns a new value
5) should call the second fulfillment callback with that new value
when the first rejection callback throws a new reason
6) should call the second rejection callback with that new reason
[Promises/A] Multiple handlers
when there are multiple fulfillment handlers for a fulfilled promise
7) should call them all, in order, with the same fulfillment value
when there are multiple rejection handlers for a rejected promise
8) should call them all, in order, with the same rejection reason

I maintain underscore.deferred, a standalone implementation of jQuery Deferreds that works as an underscore mixin. I had been intrigued by your post and finally got around to making my implementation pass your test suite (the promises part anyway, still failing 2 of the "always-async" tests), but it raised some questions along the way.

Surprisingly, the changes needed to make jQuery Deferred's pass were pretty minor and only break 1 test in the jQuery test suite. The test that breaks is about chaining/filtering against a deferred object.

Here's the problem I'm having: when chaining deferreds with then, why doesn't the second deferred object (the one returned by the first then) continue to honor the state of the first deferred? jQuery's implementation maintains the state of the deferred from the first then to the next unless a new deferred is explicitly returned. Here's an example:

In the old behavior, the state of the previous promise is maintained when chaining then's, whereas in the new behavior you lose this because the second then is always resolved if your handler returns any value that's not a deferred.

Is an uncaught exception the only way the second promise is ever rejected? Does it still count as "simple callback aggregation" if the state is maintained even though a new deferred object is created on each transformation?

Curious what your thoughts are on this... I'm not sure which behavior is more correct, but I'm looking forward to saying that underscore.deferred passes your promises tests!

So indeed, just like in sync code, if you don't re-throw, the error is considered handled.

You can check out more formal specs at Promises/A+, something various promise implementors have been working on over the last month or so to make a more explicit version of Promises/A. It might be helpful in seeing exactly how things are supposed to go.

It's very cool that it took so little extra work to make it pass! I'm also impressed that you managed to create an implementation that allows multiple fulfillment values or rejection reasons, while still passing.

@domenic thanks for the explanation, it definitely makes more sense when its compared to the synchronous counterpart. I've incorporated these changes into the lastest release of underscore.deferred, which passes promise-tests (but still not the extended suite) and still passes over 99% of the jQuery test suite (1163 out of 1167 assertions).

once promise1 is fulfilled; we call the success handlers success1 and success2, and the exception in the second handler will trigger error2. (given that the success and error handlers are called in order they are "then"ed).

The end result would be different if promise2 and promise3 are the mutations of the same object, or two distinct promise objects.
In the former case both will have a "rejected" state, whereas in the latter case promise2 would be in "fulfilled" state and promise3 will be in a "rejected" state. -- and it's the latter we want (i.e. promise1 and promise2 maintaining their own states)

neither of the rejection handlers will ever be called, since promise1 is fulfilled. Instead (in a Promises/A implementation), promise2 will be fulfilled, and promise3 will be rejected. If you want to add rejection handlers to those promises, you need another handler, e.g. promise3.then(undefined, error3);.

basically my examples were fooling me because my handlers were returning (resolved/rejected) promises instead of relying on then() returning a promise for me, am I right?
Matter of fact, your wrap() function is doing exactly that, if the handler's return value is not a promise itself, you wrap the value into a resolved promise, if the handler throws an error, you wrap the error into a rejected promise.

Another BIG issue is that if one of the promise.done() functions fails there is no way to inform the next promise.done() that things have broken and that it needs to handle it with a fail? Am I correct in this summation? A simple yes or no will suffice. Thanks for posting this essay. Cheers.

Awesome article. I have been using when.js for a long time but I was working on a project that was using jQuery promises and I was very confused. Thanks for the clarification. Just an FYI, when.js is now fully async as of 2.0. You can read the change log here https://github.com/cujojs/when.
Thanks!

You say, that promises "give us back functional composition and error bubbling in the async world". Currenty, I'm wondering, how to abort a chain of promises. In the sync world, I could do something like the following, but how to do this with promises?

I'm co-writing a book on jQuery deferreds for O'Reilly. Is there anyone who'd like early access and be willing to cast a critical eye over what we've written? It's not long, currently 73 pages.

We've been aware of the jQuery deferred limitations with error processing from the start (we came to jQuery deferreds from the Twisted world), but it still seems like introducing people to deferreds via jQuery is a reasonable approach, given jQuery's ubiquity. The idea is to bring deferreds to people's attention, show them how to use them and think about them, and then discuss some of the wider picture and point to other deferred packages. The bulk of the book is a cookbook of deferred examples.

Anyway, if anyone would like to help improve the quality of what we've written, that would be fantastic.

@domenic - thanks for a great thread and for all the work on the test suite.