Experiments with Koa and JavaScript Generators

Posted on December 21, 2013

Koa is a new application framework for Node.js. Its whole point is to eliminate the callback madness that plagues many Node.js apps. It achieves this previously-impossible feat by using the powerful generators feature which is new in JavaScript ES6. Koa is built by the same people who created Express, the best-known Node.js application framework, so it’s at least worth a look.

Having written plenty of callback-heavy JavaScript, I was excited to give this thing a go and see how much simpler it could make my code.

What have generators got to do with async?

On the surface, nothing. Generators are a way of returning an arbitrary sequence of results from a function, with the function’s execution suspended in between results. Here’s one:

// This is a generator function. The star preceding the function name means// it will return a 'generator' object, even though none of the code inside// explicitly says so.function*getAllSquareNumbers(){for(var i =1;; i++){// Every time we 'yield', this function's execution pauses until// the generator is restarted by a call to 'next' (see below).
yield i * i;}}// Now let's fetch some valuesvar generator = getAllSquareNumbers();
console.log(generator.next().value);// Outputs '1'
console.log(generator.next().value);// Outputs '4'
console.log(generator.next().value);// Outputs '9'
console.log(generator.next().value);// Outputs '16'// We could go on. That's enough.

Each time you call the generator’s next function, it returns an object of the form { value: ..., done: ... }, where value is the next thing to be yielded by the generator function, and done is a boolean that becomes true if the generator function has exited (e.g., by calling return).

For C# developers, this is exactly equivalent to how the yield keyword has worked since C# 2.0 in 2005.

So again, how does this help with async programming?

When I first heard about Koa eliminating callbacks, I thought “Oh I see, I guess the generator emits values asynchronously“. This turns out to be completely incorrect, and in fact doesn’t even make sense when you have a clear mental picture of generators. But because I was fixated on this (wrong) view of how Koa would work, I was totally confused for some time, and couldn’t make it do what I thought it should do. Allow me to spare you any such confusion.

Generators yield values synchronously.

They cannot yield values asynchronously. It wouldn’t make sense for them to do so, because the code receiving these values doesn’t necessarily know anything about ES6 Generators, and it’s always been a basic axiom of JavaScript that your code’s execution can’t be blocked (except now for functions that opt in by having a star in their name, i.e., generator functions). So when you call generator.next() from regular code, you will receive the next value synchronously.

Well then, for the third time, how are generators going to help us with async code, considering that they have to be synchronous?

What if the sequence of values returned was in fact a sequence of promises, or other objects that describe asynchronous tasks? In that case, the code consuming the sequence could choose to wait for each one to complete before calling .next to get the next one. And that’s how Koa works – your application code is the generator, it emits a series promises (or other things I’ll show below), and Koa waits for each one to complete before resuming your code (passing back to you the result of the previous task).

And how does Koa pass back results into your generator? That’s another feature of ES6 generators. If you pass a value to next, e.g., generator.next(123), then that value is given to the generator code as the value of the currently-blocked yield expression. Hence code like var nextData = yield something; (it emits something, then some time later receives nextData).

Koa Example

Here, then, is a trivial Koa example:

var koa = require('koa'),
app = koa();
app.use(function*(){// Here is the important bit of application logic for this example.// We make use of a series of async operations without callbacks.var city = yield geolocation.getCityAsync(this.req.ip);var forecast = yield weather.getForecastAsync(city);this.body='Today, '+ city +' will be '+ forecast.temperature+' degrees.';});
app.listen(8080);

The application code provided to Koa is a generator function (in this case, an anonymous generator function, hence function *() { ... } ). Each yield expression supplies a promise (or other task description) to Koa and suspends execution. Koa waits for each such promise to complete, then resumes your generator, passing you the most recent async result value.

The net result is that you get to consume asynchronous operations using a syntax that barely differs from if they were synchronous.

Error handling

Also great news is that error handling works really cleanly too. If the async task reports an error (for example, promises have ‘failure’ callbacks), then Koa uses the generator’s throw feature to dispatch an exception to whatever yielded that task. You can therefore catch exceptions just like in synchronous code, or let them cause the request to fail (instead of being silently ignored, as async exceptions often are by mistake).

Doing async Koa, or “What can I yield?”

I looked for this info in the Koa docs and couldn’t find it, so here is a list of the possible things you can yield to Koa and have it work properly.

In fact this turns out to be documented for the co Node module, which is what Koa uses behind the scenes.

1. Promises

Typically the most convenient. If you have any library that describes async tasks as Promise-like objects, you can just yield them to Koa. Example:

Note: This works with any “thenable”, which is the looser notion of a promise defined in the Promises/A+ spec. Every sane promises library conforms to this.

2. Thunks (ignore these if you’re not comfortable passing around function objects)

Thunks? Huh? Koa understands thunks to be functions you call with a single parameter, callback, to receive a result value (or error description) when it’s done.

To begin, let’s imagine you have the following existing code that uses Node.js’s traditional callback style to return a result. Ignore the fact that this totally does not need to be asynchronous. Imagine it’s a database query or something.

Now, Koa can’t understand this function directly, because it has no idea how many (or what) parameters it takes. But you can turn it into a thunk by wrapping it as follows:

// Wrap the async operation in a 'thunk' so that Koa understands itfunction getSquareValueThunk(num){// A 'thunk' returns a function that accepts a callbackreturnfunction(callback){
getSquareValueAsync(num, callback);};}

The whole point of this wrapping is to eliminate all parameters other than callback, so that Koa knows how to consume it. Now you can yield this thunk to Koa inside any of your app.use generators, e.g.,

To clarify, calling getSquareValueThunk(16) does not actually do anything except create and return a new function instance, i.e., a function that takes a single callback parameter. Whenever you yield a Function instance to Koa, then Koa assumes it may invoke it with a single callback parameter, and that callback will eventually be invoked as callback(error, result).

If you don’t like or understand this, just use promises. Promises are almost certainly a better choice for application code. Thunks are only good as a low-level primitive used internally by Koa because they don’t involve any particular promises library.

3. Generators

OK, here it gets interesting. Generators can emit generators which emit generators, etc. This makes it possible to construct async tasks out of other async tasks, with arbitrary nesting.

Imagine we want to invoke getSquareValueThunk multiple times in series, and wrap that series up as a single operation. Easy: just make another generator.

Notice the star preceding the function name. This means it’s legal to use yield inside it, and that it returns a generator. This generator returns a series of thunks (though promises would work equally). Now you can insert this series into your async pipeline by yielding it from any app.use code:

app.use(function*(){var bigNumber = yield getSquareOfSquareOfSquare(16);this.body="16 to the 8th power is "+ bigNumber;});

This is how a nontrivial Koa app will be structured. The top-level app.use generators will yield generators returned by mid-tier application code (such as business logic) which in turn will yield generators returned by lower-level application logic (such as data access). All the way down, your source code looks synchronous (no callbacks), but is actually fully async. Of course you aren’t necessarily doing three-tier architecture, but you get the idea

Just as an aside, this technique could be used to simplify the doHttpRequest function you saw earlier in this blog post. The original version is:

If you have multiple simultaneous tasks and want to wait for all of them, just yield an array of them, e.g.,

app.use(function*(){var urls =['http://example.com/','http://twitter.com/','http://bbc.co.uk/news/'];// On this line, all three requests begin in parallel// Note: doHttpRequest is defined earlier in this blog postvar arrayOfPromises = urls.map(doHttpRequest);// When we yield the array of promises here, Koa will// wait for all of them to completevar arrayOfResponses = yield arrayOfPromises;this.body="Results";for(var i =0; i < urls.length; i++){this.body+='\n'+ urls[i]+' response length is '+ arrayOfResponses[i].body.length;}});

As you can see, the result of yield arrayOfPromises; is another array, with the corresponding entries being the promise results.

The array you yield doesn’t have to be purely promises – it can also contain thunks, generators, other arrays, or key-value objects (see below). The structure can be nested arbitrarily.

5. Key-value objects

If you have multiple simultaneous tasks you want to wait for, and you want to give useful names to each of them, use an object. Example:

It’s just easier to read than referring to your tasks as tasks[0], tasks[1], etc.

Isn’t this a lot like C# 5′s async and await?

Yes, very much like that indeed! The * in function *() { } plays a similar role to C#’s async keyword, and JavaScript’s yield (at least when used inside Koa or a similar framework) plays a similar role to C#’s await.

The usage patterns of */async and yield/await are very similar. The main differences are to do with the underlying implementation. In C#, await handles asynchrony as a language feature and has a built-in standard notion of Task, whereas in JavaScript, yieldcan be used to handle asynchrony (when you supply some library code to describe async tasks and some other library code to wait for them), but it can also be used for other things, and the JavaScript language doesn’t know or care what you’re using yield for.

For C# developers, it would be fun to use the same technique to recreate C# 5′s async/await patterns purely using C# 2.0′s yield keyword.

Learning more about Koa

This blog post is purely about generators and async in Koa. If you want to actually build an app with Koa, go and read the framework’s website. If you’ve never used Express (the earlier incarnation of Koa’s ideas) then be sure to learn about middleware, the central architectural pattern in Koa, so you will see how the framework intends your application to be organised.

Thanks for this info, Steve! There’s not much information on Koa yet, so your experiments are helping me a lot.
I’ve used the thunkify lib to turn regular node functions (e.g. zlib.unzip) into thunk-returning functions, so I can yield them inside a generator. This Koa framework is really going to boost Node.js development!
Hopefully NTVS (Node.js Tools for Visual Studio) is going to support generators functions soon, so I can use it to write/debug my new node.js code

Also good to know: Azure Websites allow you to run Koa, by adding Node.exe (>= 0.11.9) into the “bin” folder of your website, and then adding this line to your iis.yaml:
nodeProcessCommandLine: D:\home\site\wwwroot\bin\node.exe –harmony

Howdy, There’s no doubt that your site may be having browser compatibility issues. Whenever I take a look at your website in Safari, it looks fine however when opening in Internet Explorer, it’s got some overlapping issues. I merely wanted to provide you with a quick heads up! Other than that, excellent site!
free itune codes http://freeitunedownloadcodes.com

Hi are using WordPress for your blog platform? I’m new to the blog world but I’m trying
to get started and create my own. Do you need any coding expertise
to make your own blog? Any help would be greatly appreciated!

Thanks for one’s marvelous posting! I quite enjoyed reading it,
you’re a great author. I will make certain to bookmark your blog and will come back later on.
I want to encourage you continue your great posts, have a
nice morning!

The chosen will perform Duets with the superstar singers in front of a live studio audience
for a chance at a recording contract with Hollywood Records.
‘Song of Leaves’ was the film he produced and it premiered in Caen.
The device pulls the tray through the scanner and
reproduces each image.

Υesterday, while I was at աork, my sister stole my iphone and tested to see if it can survivе a
thirty foot drop, jսst so she can be a youtuƄe ѕensation.
Mү iPad is now destroyed and she has 83 views.
І know this is completely off topic but I had to share it with someone!

Terrific work! This is the type of info that are supposed to be shared around
the web. Disgrace on the search engines for no longer positioning this publish higher!
Come on over and visit my site . Thanks =)

The moves are very similar to the moves Shaun T brings you in INSANITY and INSANITY:
ASYLUM. Remembering how you used to smash that tennis ball across
the court or run a mile breakfast, you might attempt more than your body’s
accustomed to. Some of us can not stand walking for a certain amount of time but what if you didn’t have to walk
down the road or on a track, would you walk then.

hello!,I like yօur writing very so much! percentage we be in contact extra ɑpproximately your post
on AOL? I need an expert іn this area to solve
my problem. Maybe that’s you! Looking ahead to look you.

So any unwelcome sound that reaches the ear is usually considered undesirable and given
the label of “noise”. Door equipment such as peep-holes,
panic bars, locks, and closures can be fitted, using materials and seals that will ensure once again that there is no noise leakage through areas surrounding these fittings.
They should also only require the processing power that you home computer can provide.

You can also add peanut butter into the shakes so
that you can add to the nutrition and calories in your protein shakes.

Your body’s hormones are not ignored, after all a proper balance must be maintained
to build proper muscle mass. According to most body building enthusiasts and experts, the
answer is yes. When you begin a routine to build muscle, you will notice an increase in energy that will help
you perform better and complete tasks that you might
not have finished before. Too many people botch their muscle building
efforts by rushing them. This is why cardio is an important factor in getting a lean body.
Place potatoes in cast iron frying pan and a dash of salt and pepper.
Weightlifters are normally pushing on their own to the
limit or lifting weights to the degree of ‘failure’.
You will discover you aren’t doing as well as you assumed.
Keeping a few on hand will help prevent reaching for cold and greasy pizza during a study break
and provide comfort associated with home life.

They hold everything together, so strengthening them will strengthen your entire system.
Well, first of all, you want to find the best tasting protein supplement
so that you actually look forward to eating or drinking it.
Then there are the dozens of variations of triceps machines from all the leading equipment companies.

To maximize your mass building efforts, activities other than weight lifting and strength training should be avoided, since they will only burn calories and make it harder to gain muscle mass.
Choosing suitable exercises is vital to gaining
weight for skinny girls.

Today, I went to the beachfront with my children.
I found a sea shell and gave it to my 4 year old daughter and said “You can hear the ocean if you put this to your ear.” She put
the shell to her ear and screamed. There was a hermit crab inside and it pinched her ear.
She never wants to go back! LoL I know this is totally off topic but I had to tell someone!

I loved as much as you’ll receive carried out right here.
The sketch is attractive, your authored material stylish.
nonetheless, you command get got an edginess over that you wish be delivering the following.
unwell unquestionably come more formerly again since exactly the same nearly a lot often inside case you shield this
increase.

I was very happy to discover this page. I want to to thank you for your time for this fantastic read!!
I definitely enjoyed every bit of it and I have you bookmarked to check out new
things on your web site.

Cached is achedcay, and similar pages reads
imilarsay agespay. I learned long ago that people outside my congressional district
and Florida’s Governor Crist could care less about my
viewpoint and the ones of others. Ten minutes in I was a crop-planting,
animal husbandry, country commerce mogul.

Set the right path to 412, Eldon street,
Herndon, which can be near to Virginia, Reston, Sterling, Ashburn,
Chantilly, Fairfax, Mclean to try out our best services.

t acquire the resolutions to, or in the wedding you did, weren’t answered for a reassurance.
This is completely unlike the outcome which come combined with the harsh
chemical peels, the glycolic acids and also the other skin
peeling products, which actually damage your skin layer then trigger a surge of particles causing inflammation as a rescue
to your skin.

Great items from you, man. I have take into accout your stuff previous to and you’re simply extremely fantastic.
I really like what you’ve acquired right here, really like what you are stating and
the way in which wherein you are saying it. You’re making it
enjoyable and you still take care of to stay it sensible.
I cant wait to read much more from you. This is really a great web site.

Attractive part of content. I simply stumbled upon your blog
and in accession capital to claim that I get actually enjoyed account your weblog
posts. Anyway I’ll be subscribing on your feeds or even I success you get entry to constantly quickly.

If you wish to buy new complete business furniture and have a tight
budget you then is going for affordable plastic and metal furniture.
They are fantastic for contemporary homes, rooms, and offices that value modern, international style and supple, full-grain leather.

Corner Desks – A corner desk is great for many, especially in a smaller room.

A full, 1080p, magnificent MP4 file of the movie is roughly 8 to 9 Gb when compressed down from the original 25Gb Blu-Ray disc.
Finallyn although the Internet can be a wonderful resourcen the person must do not forget that,
overall, it’s not regulated. Here is often a brief list of my top four favorite
films that I discovered on Netflix:.

In case you have teenagers keep reminding them that swimming in deep water may be
dangerous. A toothbrush that you just are done using in your teeth is going to do fine.
A pre-manufactured frame is put surrounding this swimming pool after digging the hole.

I would like to invite you to be a Speaker on conference which I’m organizing this year.
As you don’t have contact form on your blog I would like to ask how I can send you more details?
Can you please send me your email?

Ӊey! Do you know if they makе any plugins to assist with SEՕ?
I’m trying to get my blog to rank for some targeted keywords but I’m not seeing very good results.
If you knoԝ of any please share. Thanks!

• On 10 August 2007, a forty year old former
Royal Mail postyal administrator fro Scotland collected a
Euro lotto jackpot of €52. However, as travel became easier and travellers became more skilful travel businesses would try
to offer rates in multiple currencies in brochures.

No matter which theory is eventually accepted,
there is no doubt that this is one of the greatest engineering
feats of the ancient world.

Having read this I thought it was really enlightening.
I appreciate you takking the time and energy to
put this article together. I once again find
myself spending a significant amounht of time both reading and posting comments.

I seriously love your website.. Pleasant colors & theme.
Did you develop this web site yourself? Please reply back as I’m wanting
to create my own personal website and want to learn where you got this from or just what the theme is named.
Thank you!