Dealing with large amounts of data isn’t an easy one-man task, it’s a whole field of research and development. Especially when your constraints are memory and CPU power, all you can do is to approach a good solution that may balance the load on the two (memory and CPU) to give you the best solution. Maybe I’m exaggerating, as hundreds or even thousands aren’t large by any means, but they are still challenging enough to get us into an OutOfMemoryException if we’re not careful. It is true that the smartphones of today can handle most desktop-grade applications with ease, but it comes with a price: battery life, no one will use your app if they find that it’s a battery drainer or a heat source. Also, unless you want to develop for a minority of the market, you will need to support devices manufactured at least in 2014.

As IndieDevs, we can only learn from our mistakes, and better yet, share them so others learn from them as well.

I was developing an Android app that you can find in Google Play now. Your reaction would be, "But this can be done in the course of a weekend," well, not if you care about the users and yourself. What took me most of the time was planning a way to display a large amount of data without any lag or uncomfortable user experience.

In order to manipulate large amounts of data, we are used to streaming it as needed. I knew that I couldn't stream a large amount of data from the internet and keep the experience fluid enough for the users, so the way I started was to first load that data from the internet and store it in a local database before displaying it in the RecyclerView (which benefits other things such as the load on the servers, the battery of the phone, the network usage, offline navigation ...).

The application didn’t freeze while data was loaded; I load small chunks of data that I accumulate in the RecyclerView thanks to RxJava.

When I applied this simple strategy, I was able to display many previews on the screen without any lag. But I wasn’t yet convinced that this could handle whatever I threw at it, so I started looking for other strategies to accommodate this, which is what we will see in the second part of this article.

In the second part, we will see how I managed to load more jokes in the memory by shrinking their size, and display them without disturbing the user experience.