Introduction to the Reactive Extensions for JavaScript – Buffering

We’ve come a long way in the series on the Reactive Extensions for JavaScript. After spending some time with Ruby and with extension points in other libraries, let’s step back to some of the basic operators again. This time, let’s talk about how we can buffer our input, which is to say we can put our observable values into a buffer based upon either time or by count. This means that instead of flooding our system with calls to OnNext, we instead fill up a buffer based upon the given criteria of time or count, and then call OnNext with that value. Let’s look deeper into how we can use this in today’s post.

Buffering Output

In a traditional pull-based model of synchronous programming, you could have scenarios where yielding the next value could be a potentially expensive blocking operations. Contrasting it to the reactive push based model, we could get flooded with requests as our systems are eagerly sending us data as fast as it can. Potentially this can be a problem which could overload our system. To compensate for this, we have with the Reactive Extensions (for both .NET and JavaScript), a notion of buffering, which is to say based upon some criteria, we can buffer the results and return them as an array. We have the choice of either buffering by count threshold, or by a given time span. Once the criteria has been exceeded (or we reach the end), we have the values yielded to us at once in array form. Let’s first look at the buffering by count.

…With Count?

The first buffering mechanism we have with the Reactive Extensions is by count. To make use of this functionality, we can call the BufferWithCount method which takes a count, and optionally a number to indicate how many you want values to skip in between buffers being yielded. This method then returns to us an array of values inside an observable sequence.

Let’s look at this in action. We’ll take an array with ten numbers and buffer them every two values, so in all we should have yielded to us five arrays. We’ll iterate through our buffer and indicate its value and position in the buffer.

Next, we’ll try specifying the skip count, which is to say how many elements we should skip. For example, if I have a buffer set at two and then I specify four as the skip value, then it will yield 1 and 2 and then skip 3 and 4 before then yielding to us 5 and 6. Let’s show how that code might look:

But this of course isn’t the only mechanism we have in our arsenal. Let’s now talk about buffering with time.

…With Time?

In addition to buffering with count, we have a notion of buffering with time, which we can achieve with the BufferWithTime method. This method takes in a timespan in milliseconds, an optional time shift, which acts like the skip from BufferWithCount, and an optional Rx Scheduler. This then returns to us, as before, an Observable of an array.

To show this in action, we’ll hop back to some of our knowledge gained in our custom schedulers post. We’ll create our custom DelayedScheduler based upon a given delay time. This will help take an existing array and then space it out over time.

With the scheduler now in place, we can take our array and delay each item by 100 milliseconds by calling the FromArray with our scheduler. We can then buffer it with a given timeout of 300 milliseconds and then subscribe much as we have before.

Now, let’s try with a time shift specified. In this example, we’ll set our timeout once again as 300 milliseconds and our shift of 100 milliseconds, which should then allow us to skip the numbers 3 and 7 based upon our timing.

Now we shouldn’t be limited to an either or at this point, why not have the best of both worlds?

…With Both?

As we’ve already covered both buffering with count and time, let’s cover how we can get the best of both worlds and when either the timeout or the buffer size has been hit, then yield us the values in the buffer. We can do that through the use of the BufferWithTimeOurCount which we specify the time in milliseconds, a count, and an optional Rx Scheduler. This then yields to us our Observable of an array.

To show this in action, I’m going to need another custom scheduler, but this time completely random as to allow me to show off both behaviors. We’ll take much of the above code from our DelayedScheduler and instead create a random number up to the delay parameter. Then, every time a new value is to be yielded, then we delay that value by the given random number.

Let’s show this off by taking our standard array that we’ve used so far and use our customer RandomScheduler of a maximum of 100 milliseconds. We’ll then buffer it by either a timeout of 200 milliseconds or 4 items and then subscribe like normally.

And there you have it, a simple walkthrough of using buffers with the Reactive Extensions for JavaScript (and if you squint real hard, .NET as well).

Conclusion

Dealing with asynchronous programming has been in the forefront of many minds in the JavaScript community. At JSConf, there were several examples of frameworks trying to get around the idea of callbacks and instead lean more towards composition. By utilizing the Reactive Extensions for JavaScript, we’re able to compose together asynchronous and event-based operations together and transform the results in interesting ways.

In the reactive programming world, we could easily have situations where we have systems pushing values at us faster than we would like. To compensate for this, we have a notion of buffering in the Reactive Extensions which allow for us to specify either count buffers, timeout buffers, or heck, even both. This gives us some much needed flexibility on how we process the data that is coming at us.