Saturday, January 4, 2014

Testing elasticseach

Real time analytics on real time data has been increasingly used in today's e-commerce. Elasticsearch is a great tool for that. Its distributed architecture, high availability and full text searching capability have gained a lot of followers. Not only it has a long list of partners (27 and growing), it has many thriving communities.

For a customer who runs Elasticsearch software on the cloud infrastructure, it's important to know the capacity of the reserved instances. This will help the customer to reserve less instances and still meet his/her business needs. In this blog, we are going to show how easy it is to run some performance test on ElasticSearch with the NetGend platform. I would like to thank Joel Abrahamsson's excellent tutorial on the basic commands on ElasticSearch to insert/view records and do queries.

We are going to do some simple tests on the ElasticSearch software running on my Ubuntu 12.04 server with Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz. First, let's insert a comprehensive DVD list into the database so that we can do some searchs on it. The dvdlist is in the format of

We were able to achieve 2500+ inserts/second with about 80% CPU usage on ElasticSearch process.

Note that in constructing a JSON message, we don't have to escape the multiple double quotes in the template as on some other test platforms, which could be quite tedious. In the last example of this blog, we will see an even simpler way of constructing a JSON message.