Win-Elastic vs Lin-Search

Thursday, December 11, 2014

How well does Elasticsearch ride on Linux? What about Windows? I've had no luck finding the answer. Now, my boss asked me exactly that as part of a an internal research. We have found oury answers for our specific scenario and we wanted to share them with you.

We wanted to evaluate the capacities of Elasticsearch to solve one of our problems. After several pitfalls and expending 20 times the original budget we finally came up with some rock solid results.

Our Data

We need to search for all the outter fields and all inner fields but the prices. There would be a hundred of items per entry and a dozen prices per item. We have 2 periods one with 28 days (4 weeks) of data and one with 180 days (6 months). For each day we will have 30k of root level object. We created one index per week and 15 shards routed by the tag field, there are 500 tags evenly distributed. It is a total of about 33 million documents per day.

Our Toys

Three nodes clusters were used for the tests. One working horse and two data nodes. No dedicated masters. For this we hired Amazon Web Service (AWS). We chose the biggest memory optimized instances (r3.8xlarge), that is: 32 vCPUs, 244GB RAM and 40 Gigabit Network. The client performing the test was similar. OSes were: Ubuntu Server 14.04 LTS and Microsoft Windows Server 2012 R2 Base, both 64 bit Editions. The tester was a Microsoft Windows Server 2012 R2 Base as well.

Tests

We executed random searches of three different kind of tests: one with small requests that would yield small responses, one medium and one that would return the whole 500 entries. There were no empty results search. The program executing the tests had 10 threads executing constantly for 5 minutes and recording the stats; 3 rounds with 1, 5 and 10 instances of the testing program. That yielded a total of 10, 50 and 100 simultaneous requests at any given moment. The response times went up (as expected) as more simultaneous requests got in.

Enough talk!!

We will group the results by kind (small and big, medium was not very different from small) and test, which will be the pair (OS, days). Time values will be in milliseconds. We will present the response times and the number of request per minute per unit of time the cluster was able to process in two separated sections.

Response times

The lower values are better

The small

The big

Number of requests per minute

The higher values are better

The small

The big

Conclusions

Windows performance was much worse than linux for all tests. The response times were acceptable though, but the number of requests per minutes was not. For big traffic internet applications this number is critical while a couple hundred milliseconds make no difference at all. If you plan to run an Elasticsearch cluster you should probably choose linux. It will perform much better and will be a lot cheaper. I can also share that it was awfully painfull to set up the tests environments and data in Windows, mostly because scripting remote commands in Windows is kind of hard and requires a lot of knowledge (which I don't have) of Windows administration. Linux on the other hand was pretty easy (same level of knowledge from my side) to script via ssh.