. . . (hopefully) useful stuff for web development and life

Everything You Need To Know About Async & Meteor

Meteor runs on top of Node.js. This means I/O activities such as reading a file or sending a request over the network won’t block the whole program. Instead, we provide callbacks that will be executed in the Event Loop later when those activities finish. Ok, I agree, that may not make a lot of sense. How about some cartoons!

Let’s say our task is to read an encrypted file, then decrypt it and get the secret content:

The Event Loop is just a queue of functions waiting to be executed. Every time we call a function, it is put onto the Event Loop

When we execute getSecretData to decrypt and print out the secret, the function readFile get called and appear on the Event Loop:

That readFile guy doesn’t care about what happens later at all, he just tells the OS to send the file and then go away!

Some moment later, the readFile operation is completed. A guy with the name ‘callback’ will jump into the Event Loop:

Later, when the file is received, our hero appears and finishes off the job

That’s quite nice and get the job done. But what if our task is more sophisticated and require many level of async operations? We might end up with something like this:

The problem of async control flow is it makes the code more difficult to read and maintain. It would be nicer if we can have getSecretData return the secret content and print it out synchronously, like this:

// So sweet. We have getSecretData return the value, then print it out, all synchronously.

// If only life were that simple...

varresult=getSecretData('my-secret-key');// undefined

console.log(result);// undefined

This code does not work, because getSecretData will not wait for the readFile operation to finish. It just goes on and return decryptedSecret as an undefined value. To solve this problem, we need a new hero. Here comes Fiber!

Meet Fiber, the big guy who can carry many functions inside him

A Fiber is a special container function. He can be put into the Event Loop like other normal functions. But Fiber has a special power: He can halt at any point in the middle of his execution, get out of the Event Loop to take a rest, then come back at any time, all at his will (or in fact, the developer’s will). When a Fiber halts, control will be passed to the next function in the Event Loop (which may be a normal function or yet another Fiber).

You properly already see the advantage here: If our Fiber contains a function that performs a time-consuming I/O activity, he can just get out of the Event Loop and wait for the result. In the mean time, we can go on and run the next functions waiting in the queue. Life is short and time is precious! When the I/O activity finishes, our Fiber can jump back in again and resume what he was doing the last time.

elsefuture.return(aes.decrypt(res,key));// signal that the future has finished (resolved)

// the passed argument (the decrypted secret)

// will become the value returned by wait() below

}

returnfuture;// we return the future instance so other code can wait() for this future

};

// The future method is added to the prototype object of every function

// Calling future() on a function will return a Fiber-wrapped version of it

(function(){

// we wait for the future to finish. While we're waiting, control will be yielded

// when this future finishes, wait() will return the value passed to future.return()

varresult=getSecretData('my-secret-key').wait();

console.log(result);

}.future())();

Wait! In the examples above, we have freely modified our getSecretData function. What if you come across an async function that you can’t modify (like functions from external APIs)? No worry, instead of modifying it, we can wrap it up!

Actually there’s more to async than meets the eye. Other useful things that are worth mentioning:

– – –

Future.wrap and Meteor.wrapAsync are very selective

They only do business with native, pure async functions. That is, functions that expect a callback with error and result as arguments. Also, they only work on the server-side (since yielding is not possible on the client – there’re no Fibers living there).

– – –

Meteor.wrapAsync will turn your innocent function into Two-Face !!!

Fortunately, two-faced functions are not as destructive as Harvey Dent. In fact, they’re pretty useful: They can be called synchronously (like what we were doing above) or asynchronously (with a callback passed to them).

On server-side, methods such as HTTP.call and collection.insert/update/remove are all already pre-wrapped this way. Take HTTP.call for example: If you call it without a callback, the method will block until the response is received. If called with a callback, HTTP.call returns immediately, and will later excute the provided callback when network response has arrived.

On client-side, since blocking/yielding is not possible, we always have to provide a callback to these methods.

– – –

Fiber hiccups

By default, method calls from a client are run inside a single Fiber – they’re run one at a time. This Fiber gets access to a set of environment variables that are specific to the currently connected client (e.g. Meteor.userId()). This may result in two common problems:

1) On server-side, calling methods like HTTP.call synchronously will block other subsequent method calls from the current client. This may not be a good thing. If subsequent methods are independent from the current running method, we can save time by using this.unblock(), which will allow other method calls to be run in a new Fiber:

JavaScript

1

2

3

4

5

6

Meteor.methods({

requestSecret:function(){

this.unblock();

returnHTTP.call('GET','http://www.nsa.gov/top-secrets');

}

});

2) “Meteor code must always run within a Fiber”

Looks familiar? This error often occurs when you try to call a third-party API with async callback. You’re not allowed to do this, since the callback function would be executed outside Fiber, without access to necessary environment variables. One way to solve this is wrapping the callback function with Meteor.bindEnvironment, which will return a Fiber-wrapped and environment-packed version of the function. The second way is using Meteor.wrapAsync like what we were doing above (actually wrapAsync already called bindEnvironment internally for us!).

I hope you’ve learned something useful about async and Meteor in this article. Happy coding!

P.S. For anyone wanting to learn more about this topic, here are two interesting classes from EventedMind:

ABOUT THE AUTHOR

Phuc Nguyen

I'm a web developer, a big fan of the Meteor framework, and a former startup founder. Currently I'm looking for an interesting and challenging Meteor project to join. You can read more about me at phucnguyen.info.

HiI have connected my Mac to a Linux box directly using a cable (no home router involved).In Mac, I am setting a static IP address to 10.10.11.13 with subnet mask 255.255.255.0 and in Linux 10.10.11.16 with same mask. My Mac has two NICs (2nd one is connected to router and WWW). How can I add a route in Mac so all the traffic intended for the Linux box goes through first NIC? I am interested in waking the Linux machine up using something like:$ wakeonlan -i 10.10.11.16 00BA::AB:XX:XXIs this routing possible?

Beyond lovely! The print and your Lydia mods are just right, Nettie. I’m so very glad to hear the making and wearing of this gorgeous dress was a source of cheer up for you. What a nod to sewing as therapy!

I came to your Everything You Need To Know About Async & Meteor | Phuc Nguyen’s Blog page and noticed you could have a lot more traffic. I have found that the key to running a website is making sure the visitors you are getting are interested in your subject matter. We can send you targeted traffic and we let you try it for free. Get over 1,000 targeted visitors per day to your website. Check it out here: http://pcgroup.com.uy/15 Unsubscribe here: http://pcgroup.com.uy/2a