Pages

Wednesday, 2 November 2011

Caching not just information with maxHeap's smartCache

I personally believe that a lot can be learnt from what happens inside a company itself. A lot of information is implicit and priceless. To capture this information is always a good thing, and to learn from it, better.

In our BI framework, we've developed our own caching. The caching is in a position where it can be used by not just the business intelligence framework, but anything inside the company. Ofcourse, mongodb's quick querying and ability to store and visualize complex data helps – and it has been a huge shift, at least for the tools team, in how applications could now behave and be developed.

So with a very fast cache in place, it was only a matter of time before we started capturing what the users of the BI framework, were doing. All that information was otherwise going to waste.

We capture that information now. And not just that, we learn from it. It takes a clever mathematical function to give a piece of information it's value. For instance, we now know which city and which area is important internally by evaluating now only how many times it has been requested but also the volume of data it yields. Once this was implemented, we started learning. Within a day, we knew what exactly we were looking at, most often, and what was returning the most promising results. And with that, you knew exactly what was more important from a business standpoint without the need of a guy with a calculator.