Trade Secrets Of A Developer / Entrepreneur

Finitely Iterating Infinite Data With ES6 Generators

ES6 generators provide a new way of working with data that should be iterated. In fact, generators produce an iterator when you initially call the generator function.

One of the things that you can do with a generator, that might not be obvious, is iterate over a never-ending series of data items while still allowing the iteration to be halted when you want it to be.

Finite Iteration

Normally, iteration takes place over a finite set of items. There are a large number of ways to do this, including for loops, forEach loops and more:

In this example, there is a finite number of items in the list. Iteration happens and then the code continue on.

With generators, though, you can have an infinite set of data over which you can iterate.

Infinite Data

The methods of generating an infinite amount of data are numerous, but a simple example would be to have a counter that increments and then reset on a regular basic (to avoid number overflow, etc).

Once you have an infinite amount of data, you can modify the above code to use a generator and yield it.

From here, you can iterate over the data using a generator iterator.

Infinite Iteration

A generator function returns an iterator object when you execute it. The easiest way to iterate through the data in the iterator is with the new “for of” loop in ES6.

When you run this code, you will get an infinite loop of CPU burning console.logs.

Be sure to have your ctl-c fingers ready, because it won’t stop otherwise!

Finitely Iterating An Infinite Data Set

There are some real possible uses for this beyond burning your CPU up. For example, you may have an “infinite scroll” web app where you continuously load data as the user scrolls.

Data could be loaded in chunks, and then iterated to display a set number of items. Once the user scrolls to a certain spot, continue iterating the existing data instead of having to request more items.

To do this, you would need to skip “for of” loops, however. That method of iteration will literally run forever if you let it. Instead, look at using the iterator API directly.

In this example, the code is manually iterated using the “.next()” method on the iterator returned from the generator function. Once that happens, the current iteration number is checked, and a check is made to ensure iteration is not yet complete. Inside of the while loop, the item is rendered, the “i” counter is incremented and the iterator is moved to the next item.

This loop repeats until the iterator runs out of items, or the “i” counter hits 10. To have it resume iteration on a scroll even or otherwise, a little more encapsulation will need to happen.

Resuming The Iteration

Whether you want to use a “more” button, a “next page” button or a scroll event doesn’t really matter. To have the rendering of items resume, you need to resume iteration. Fortunately, this is exactly what generators allow you to do!

By wrapping the generator code in a slightly nicer abstraction (function set), you can easily resume iteration when you want to.

With this setup, you can “continueRendering” whenever you want!

In this case, the first rendering happens when the page loads (using jQuery) and then when a “more” button is clicked.

Easy Iteration, Many Options

The idea of infinite scrolling, or continuously loading / rendering new items with a “more” button click is certainly not new. Plenty of website have been doing this for a very long time. With generators, though, you have more options for pre-loading the data and only rendering a certain number of items.

With the generator version of this kind of code, iteration takes place when you want it to – when you call the “.next()” method on the iterator. This allows you to more easily handle an unknown number of data items. You no longer have to deal with fixed array sizes and ensuring you are looking at the next item by index. Generator iterators handle the complexity of figuring out which item is the current item.

There are still plenty of other good uses of generators, too, as I’ve talked about before. Any time you want to iterate through items, pause iteration and process them, or even abandon iteration entirely, generators may be a good choice.

Still Need The Basics Of Generators?

If you’d like to start from scratch with generators – see how the protocol for iteration works, how you can use them to better handle asynchronous code, etc – check out my Learning ES6 Live! series on WatchMeCode.

This series includes several episodes to cover the ground-up learning experience of generators. You’ll see how they work, some of the more advanced possibilities with asynchronous code, and more!

About Derick

Derick Bailey is a developer, entrepreneur, author, speaker and technology leader in central Texas (north of Austin). He's been a professional developer since the late 90’s, and has been writing code since the late 80’s. Derick has built software for organizations of all shapes and sizes, including contributions to Microsoft's MDSN library, running several very highly regarded open source projects, creating software solutions for large financial organizations, healthcare orgnaizations, world-class airlines, the U.S. government, and more. These days, Derick spends most of his time working on content for his own entrepreneurial efforts at WatchMeCode.net, playing video games when he gets a chance, and writing code for for his few remaining clients. You can reach Derick at DerickBailey.com or on twitter, @derickbailey.