Iterators and Generators in Javascript

Introduction

Kenneth Truyers

Iterators and Generators in Javascript

Iterators and Generators in Javascript

Last week I wrote about the yield return statement in c# and how it allows for deferred execution. In that post I explained how it powers LINQ and explained some non-obvious behaviors.

In this week’s post I want to do the same thing but for Javascript. ES6 (ES2015) is becoming more and more mainstream, but in terms of usage I mostly see the more common arrow-functions or block-scoping (with let and const).

However, iterators and generators are also a part of Javascript and I want to go through how we can use them to create deferred execution in Javascript.

Iterators

An iterator is an object that can access one item at a time from a collection while keeping track of its current position. Javascript is a bit ‘simpler’ than c# in this aspect and just requires that you have a method called next to move to the next item to be a valid iterator.

The following is an example of function that creates an iterator from an array:

Iterables

An iterable is an object that defines its iteration behavior. The for..of loop can loop over any iterable. Built-in Javascript objects such as Array and Map are iterables and can thus be looped over by the for..of construct. But we can also create our own iterables. To do that we must define a method on the object called @@iterator or, more conveniently, use the Symbol.iterator as the method name:

Generators

Custom iterators and iterables are useful, but are complicated to build, since you need to take care of the internal state. A generator is a special function that allows you to write an algorithm that maintains its own state. They are factories for iterators. A generator function is a function marked with the * and has at least one yield-statement in it.

A normal function would run endlessly (or until the memory is full), but similar to what I discussed in the post on yield return in C#, the yield-statement gives control back to the caller, so we can break out of the sequence earlier.

Deferred Execution

Since we have the same possibilities for yielding return values in Javascript as in C#, the only what’s missing to be able to recreate LINQ in Javascript are extension methods. Javascript doesn’t have extension methods, but we can do something similar.

What we’d like to do is to be able to write something like this:

generateNumbers().skip(3)
.take(5)
.select(n => n * 3);

It turns out, we can do this, although we need to take a few hurdles.

To attach methods to existing objects (similar to what extension methods do in c#), we can use the prototype in Javascript. Generators however all have a different prototype, so we can’t easily attach new methods to all generators. Therefore, what we need to do is make sure that they all share the same prototype. To do that, we can create a shared prototype and a helper function that assigns the shared prototype to the function:

Now that we have a shared prototype, we can add methods to this prototype. I’m also going to create a helper method for this:

function createFunction(f) {
createChainable(f);
Chainable.prototype[f.name] = function(...args) {
return f.call(this, ...args); };
return f;
}
}
In the above method:
- It makes sure the function itself is also chainable, by calling createChainable
- Then it attaches the method to the shared protoype (using the name of the function). The method receives the arguments, which gets passed on to that method while supplying the correct this-context.
With this in place we can now create our “extension methods” in Javascript:
```javascript
// the base generator
let test = createChainable(function*(){
yield 1;
yield 2;
yield 3;
yield 4;
yield 5;
});
// an 'extension' method
createFunction(function* take(count){
for(let i=0;i<count;i++){
yield this.next().value;
}
});
// an 'extension' method
createFunction(function* select(selector){
for(let item of this){
yield selector(item);
}
});
// now we can iterate over this and this will log 2,4,6)
for(let item of test.take(3).select(n => n*2)){
console.log(item);
}
Note that in the above method, it doesn’t matter whether we first <font face="Courier New">take</font> and then <font face="Courier New">select</font> or the other way around. Because of the deferred execution, it will only fetch 3 values and do only 3 selects.
### Caveat
One problem with the above is that it doesn’t work on standard iterables such as Arrays, Sets and Maps because they don’t share the prototype. The workaround is to write a wrapper-method that wraps the iterable with a method that does use the shared prototype:
```javascript
let wrap = createChainable(function*(iterable){
for(let item of iterable){
yield item;
}
});
With the wrap function, we can now wrap any array, set or map and chain our previous function to it:
```javascript
let myMap = new Map();
myMap.set("1", "test");
myMap.set("2", "test2");
myMap.set("3", "test3");
for(let item of wrap(myMap).select(([key,value]) => key + "--" + value)
.take(3)){
console.log(item);
}
One more thing I want to add is the ability to execute a chain, so that it returns an array (for c# devs: the ToList-method). This method can be added on to the prototype:
```javascript
Chainable.prototype.toArray = function(){
let arr = [];
for(let item of this){
arr.push(item);
}
return arr;
}

Conclusion

If we implement the above, it allows us to write LINQ-style Javascript:

Obviously, this only works in ES2015 and it’s probably not a good idea to actually write LINQ in Javascript using this method (and besides, there are already other implementations of LinqJS), but it does demonstrate the power of Iterators and Generators in Javascript.