Working with Pagination and LoopBack

I knew LoopBack supported pagination, but not exactly how it did. So, I wanted to build a quick demo for my own learning and I thought it might be useful to share it with others. This will be a relatively simple blog post to demonstrate what I learned.

Pagination is simply the process of presenting a large set of data to the user one “page”, or one set, at a time. For example, if you have a few hundred records, it is a lot less intimidating to the user to show them just ten or so records first. For this demo, I created a simple model, Person. Person has two properties: name and picture. To create a large amount of data (so we can actually paginate), I used the excellent Random User Generator API to spit out about two thousand users. I did this one time of course, saved the result as a huge JSON file, and then created a quick script to import this into my LoopBack application. The net result was 5000 unique names and pictures in my datastore. Now for the fun part.

If you simply run the GET method on the Person API, you’ll get the ginormous JSON packet back. This just confirms that – by default – the API does no filtering on size. As an example of how bad that could be, check out the size of the request in Firefox below:

LoopBack provides two filters to help manage this and provide pagination. The first is limit. By appending filter[limit]=X to our URL, we can limit the result set to a sensible size.

Woot! Much better. That’s displaying the first ten records. So how do you get the next ten? One more filter – skip. By adding filter[skip]=X to the query, I can tell LoopBack to start returning results after the Xth row. Here’s an example of that:

Woot! (Again!) But we’re not quite done yet. How do we know when to stop paging? If you tell LoopBack to skip past the end of the result set, you simply get back an empty array. In theory your front end could stop displaying “Next”-style UI at that result, but if you wanted to provide the user feedback on how many pages exist, you would be out of luck. Luckily, LoopBack comes to the rescue again. Every API will have a Count method out of the box, as demonstrated below.

So by simply making an HTTP GET request to /api/people/count, you’ll get the response back in a simple JSON object: {“count”:5000}.

Making a Demo

Alright, so let’s put this together in an incredibly simple demo application. I began with a simple view:

I’ve got a blank div for my data and two buttons for navigation. Notice they are disabled by default. Now let’s look at the code.

var$backBtn,$fwdBtn;var$results;varcurrentPos=0;vartotal;varperPage=10;$(document).ready(function(){$backBtn=$("#goBackBtn");$fwdBtn=$("#goForwardBtn");$results=$("#results");//step one - how much crap do we have?$.get('http://localhost:3000/api/people/count',function(res){total=res.count;console.log('count is '+total);},'json');fetchData();$backBtn.on('click',moveBack);$fwdBtn.on('click',moveForward);});functionfetchData(){//first, disable both$backBtn.attr('disabled','disabled');$fwdBtn.attr('disabled','disabled');//hide results currently there$results.html('');//now fetch$.get('http://localhost:3000/api/people?filter[limit]='+perPage+'&filter[skip]='+currentPos,function(res){renderData(res);if(currentPos>0)$backBtn.removeAttr('disabled');if(currentPos+perPage<total)$fwdBtn.removeAttr('disabled');},'json');}//could(should) use a nice template langfunctionrenderData(p){s='';p.forEach(function(person){s+='<div class="person"><img src="'+person.picture+'" width="48" height="48">'+person.name+'</div>';});$results.html(s);}functionmoveBack(){currentPos-=perPage;fetchData();}functionmoveForward(){currentPos+=perPage;fetchData();}

This file is a bit more complex, but still relatively simple. We do a quick call to figure out the count of our data. Then we simply run a function, fetchData, that handles fetching the current “page” of data. Once data is returned, we call another function to render it and then figure out which of our navigation buttons should be enabled. It’s not the prettiest demo, but here it is in action.

One last thing

So as a quick FYI – what if you were worried about people trying to fetch all of your data? That could be a performance drain on your server. Luckily LoopBack provides a quick way to handle that. A few days ago, Alex blogged about remote hooks. Remote hooks are a way to apply logic to the REST calls of your API. In our case, we can listen for calls to “find” data and look to see if a limit was applied. Here is the entirety of the logic.

That’s it! The number 10 was chosen arbitrarily – you can use whatever makes sense for you.

One more last thing

Ok, I lied, we aren’t quite done yet. I just discovered an interesting feature of LoopBack called Scopes. Scopes allow you to define aliases for queries. So for example, I could actually make a scope for ‘page2’ that implied a limit of 10 and a skip of 10 as well. You can also supply a *default* scope and correct the issue we mentioned above, setting a default limit. It would look like this in the model JSON file:

"scope":{"limit":10},

That’s simpler, so why would we ever use the other option? Well because this is more than just a default, it is a strict setting, so even if the user wanted more than 10, they can’t request it by appending filter[limit]=X. You may be fine with that, but just keep it in mind.