Thursday, 26 April 2012

Performance testing MongoDB

So, this morning I was hacking around in the mongo shell. I had come up with three different ways to aggregate the data I wanted, but wasn't sure about which one I should subsequently port to code to use within my application.

So how would I decide on which method to implement? Well, lets just choose the one that performs the best. Ok, how do I do that? Hmmm. I could download and install some of the tools out there, or I could just wrap the shell code in a function and add some timings. OR, I could use the same tool that I use to performance test everything else; JMeter. To me it was a no brainer.

Bundle up into a jar and drop into the apache-jmeter-X.X\lib\ext folder

Update search_paths=../lib/ext/mongodb.jar in jmeter.properties if you place the jar anywhere else.

How I did it

I tend to have a scratch pad project set up in my IDE, so I decided just to go with that. Just to be on the safe side, I imported all the dependencies from:

apache-jmeter-X.X\lib

apache-jmeter-X.X\lib\ext

apache-jmeter-X.X\lib\junit

I then created the two classes and the properties file.

I then exported the jar to apache-jmeter-X.X\lib\ext, and fired up jmeter.

Go through the normal steps to set the test plan up:

Right click Test Plan and add a Thread Group.

Right click the Thread Group and add a Sampler, in this case a MongoDB Script Sampler.

Add your script to the textarea; db.YOUR_COLLECTION_NAME.insert({"jan" : "thinks he is great"})

Run the test

Happy days. You can then use JMeter as you would for any other sampler.

Future enhancements

This is just a hack that took me 37 minutes to get running, plus 24 minutes if you include this post. This can certainly be extended to allow you to enter the replicaset config details for instance and to pull the creation of the connection out so we're not initiating this each time run a test.

Hi Jan. I've seen mongo benchrun, but there is little or no docs, and I don't see how to pipe results so they can be viewed grapihcally, so I have found this really useful. I built the jar from github, but are you likely to be sharing a jar for future releases?

Thank you Jan, I really appreciate your help.I put driver to lib\ext\, and now there are no errors in the log.I can insert documents into collections.But now there is another issue: I never receive any response, for example for db.XXX.find();In listener it always shows response size 16, but only thread name displayed.

I wrote this to work via eval, as such as long as it runs in the shell, I'm happy, I was never interested in returning any results for display in jmeter.

This is on github, so feel free to extend it in anyway you want. It's easy to create a fork.

If you have any specific user cases, then feel free to send them to me, though I can't give any guarantees around when I can implement them.

I'll probably commit some quick changes tonight, so stay tuned.

Please note (and as I alluded to earlier) that you need to ensure that anything added to the script field needs to run via eval. You cannot return a cursor from db.eval() So you cannot do this db.jmeter.find(), but you can do this db.jmeter.find().toArray()

Thank you for your time answering my stupid questions Jan, I really appreciate that.I am not a developer, I just use jmeter for QA; so stuff that is simple for you could be beyond my understanding.I was looking for a jmeter plugin to send requests to mongodb and receive responses, same as I do using JDBC driver.Sorry.

Remember to add toArray() as you cannot return a cursor from db.eval(). So db.tbl1.find() needs to be db.tbl1.find().toArray()Remember to wrap it in a function too, so the whole thing will look like this function() {return db.tbl1.find().toArray();}

Hello Jan,Finally with your help I have everything working for remote mongodb.

I found how to reproduce the fail:If there are two or more mongodb samples in one .jms file, only the last sample returns results, the previous mongodb samples return empty array [].

This is log file record for multiple mongodb samples:2012/06/26 10:22:34 WARN - jmeter.engine.StandardJMeterEngine: Error encountered during shutdown of jan.mongometer.ScriptSampler@69b3af04 java.lang.NullPointerException at jan.mongometer.ScriptSampler.testEnded(ScriptSampler.java:330) at jan.mongometer.ScriptSampler.testEnded(ScriptSampler.java:322) at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfEnd(StandardJMeterEngine.java:236) at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:454) at java.lang.Thread.run(Unknown Source)

Thank you Jan.This is the jmx file:https://www.dropbox.com/sh/jdwgub72wkrvsyd/5xzG4UvWIx

There are two samples, sample-1 sends request to local db-1; sample-2 sends request to local db-2.If both samples enabled, only the second one gets response from the db; the first sample returns [].Each sample works ok if another is disabled.

Hi Jan, I am getting the same symptom that archie mentions on 7 July 2012 00:58 and 26 June 2012 18:30. This prevents me from having multiple mongodb script instances in the same thread. Have you had a chance to fix this?

Thank you for responding so quickly! I need to get this working soon. It will take me a while to get familar enough with the JMeter Sampler requirements to see if the mongoDB script code meets al the requirements so I will attempt it this weekend.

For today, I am looking into other less pretty (more complicated) ways to implement getting mongoDB values into JMeter variables (e.g. BSF Sampler using Groovy script to call mongo directly and pass values back to JMeter)

Below is more information in case you have time this weekend to look into it.

This is the request for the first MongoDB script:db.useraccount.find( { "email" : "ralph@navarrocomputing.com" }, { _id:1 } ).toArray()

This is the actual and expected response data for the first MongoDB script when the second MongoDB script is disabled:[ { "_id" : "b4a634aa-e71b-410e-9ead-cada44b26ea0"}]

I pull the _id value into an ID variable using ATLANTBH's JSON Path Extractor ( https://github.com/ATLANTBH/jmeter-components ) with the following:$[0]._id

Debug PostProcessor shows the variable has been initialized properly so the mongoDB script and JSON path extractor worked:ID=b4a634aa-e71b-410e-9ead-cada44b26ea0

Here is the problem: As soon as I enable a second mongoDB script that does a similar query to get another variable initialized with data from another db/collection, both mongoDB scripts return null:[]

Debug PostProcessor shows the variables also get set to NULL:ID=NULLID2=NULL

There is a NullPointerException that shows up in the JMeter Log Viewer only during the second run when both mongoDB scripts are enabled:

Yes, I added the second script to the same Thread Group by duplicating the first mongoDB script JMeter component and then modifying its fields. The Thread Group is set to 'Continue' after Sampler error.

In the second mongoDB script, the mongoDB server is the same as used in the first script, but the db and collection are different.

Thank you very much for the tool.I started using this tool, but i am getting this error." jan.mongometer.config.MongoSourceElement: MongoDB Source Config testStarted 2013/07/11 11:34:32 ERROR - jmeter.JMeter: Uncaught exception: java.lang.NoClassDefFoundError: com/mongodb/MongoOptions at jan.mongometer.config.MongoSourceElement.testStarted(MongoSourceElement.java:204) at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:209) at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:325) at java.lang.Thread.run(Unknown Source)"

Hi thanks for your tool.but when i try to insert some data i am getting the following error jmeter.JMeter: Uncaught exception: java.lang.NoSuchMethodError: com.mongodb.MongoClientOptions.builder()Lcom/mongodb/MongoClientOptions$Builder; at org.apache.jmeter.protocol.mongodb.config.MongoSourceElement.testStarted(MongoSourceElement.java:139) at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:214) at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:336) at java.lang.Thread.run(Unknown Source)