As someone very active in both .NET and the JavaScript community, I like to take lessons learned from one environment and apply it to the other. This blog will apply lessons learned from .NET's LINQ to JavaScript collections.

Introduction

We have come a long way in JavaScript when it comes to dealing with collections. Before 2015 we had to write stuff like this:

However, there is still a lot of room for improvement. First of all, every operator creates a new collection.

Depending on the size of the collection and how many operations are applied, this leads to wasted memory.

Second, who says we need the full collection anyway? With paging, people are usually only interested in the first few pages.

The Best Place to Hide a Dead Body is Page Two of Google search results - Unknown

So, how to solve this issue? This is where generators come in.

Generators

Here is a brief explanation about generators. If you know what generator functions are, you can skip this part.

A generator function looks something like this:

function* getPrime() {
yield 2;
yield 3;
yield 5;
yield 7;
}

Notice the * and the yield keyword. This function returns a collection. But it's not an array, it's a Generator. The main difference between an Array and a Generator is how its iterator works.
With an array, the iterator walks through memory. When you request the next item, it simply reads a value from memory. With generators, the iterator walks through code. When you request the next item, it will execute code until it reaches a yield statement and then it returns that value.

Calling the getPrime function immediately returns a Generator. However, nothing in the function body was executed. Only when line 3 is called, the function will start to execute until it reaches yield 2. Then the value 2 is handed to console.log. On line 4, we request the next item. The function will pick up where it left off and continue until it reaches yield 3. This is also known as lazy evaluation.

Notice that this result was never stored. It just fetches and processes the values one by one. This makes generators very suitable for pipelined processing. Also notice that 5 and 7 where never reached and no excessive work was done. This means that generators can be used in combination with potentially infinite lists, like all prime numbers.

But this time the operators don't create temporary collections but instead support lazy evaluation by doing pipelined processing. The for loop asks for the next item, which the select function asks to the where function, that the where retrieves from the original collection.

The where function is basically copy/pasted from the very first code block. Except that any filter can be applied, and that it's now a generator function. The same thing applies to the select function.

No filtering or transformation is done until we reach the for loop at the end of the script. Then, each value in coll will be sent through the pipeline individually.

Great, that's exactly what we wanted. But maybe we can write it in a more natural way.

We've set the functions on the prototype of Array. Looks a lot better right? There is only one problem... It doesn't work. The reason is simple: function* does not return an Array it returns a Generator object. So, the select function is not found on the output of the where function.

version 3

Well, that should be easy to solve, right? You just put it on the Generator object instead. Two problems with that.

The Generator object is not exposed by JavaScript

The first Array will have to be converted to a Generator before you can start chaining the operators

We can get a reference to the Generator object by using a little hack though.

Here, the where and select functions create a generator function. The pipe function chains these generators together without using the Generator's prototype.

Basically you get the following:

let result = selectGen(whereGen(coll))

As you might have noticed, this looks a lot like RxJS's syntax. That's no coincidence, we've basically built the synchronous version of RxJS here. The only difference is that ours is a pull mechanism, and not a push mechanism.

That's it! I hope you learned something. Leave a comment with any thoughts or remarks.