The great free performance, load and stress testing tools that can be used with SharePoint verdict

Here’s a verdict of a list of tools we’ve tested to see if they’re useful in scenarios where you want to perform performance, load, and stress tests for SharePoint. Note that we’ve deliberately left out the Visual Studio Web Test tooling, as we’re finding lots of companies that don’t have the required licenses to use it. We do feel that every software author of each tool deserves a fair judgment based on facts and arguments, the only thing is, this verdict is not going to provide any of that. We’ve looked at a list of 100+ performance, load, and stress testing tools, and discarded any that were either commercial or not Windows oriented enough for our tastes. For the small list of remaining tools, we will just list them and tell you whether we liked it or not without getting into the in depth arguments why we reached a certain conclusion. Is that fair? Probably not. Here goes:

LoadUI at http://www.loadui.org/ is a great tool with at first, a brilliant user interface. Later on, the UI is still proves to be quite good, but not as great as the various Microsoft tooling we’re accustomed to.

StressStimulus (http://stresstimulus.stimulustechnology.com/) is a Fiddler add-in that looks promising but failed on our machine because it couldn’t install SQL Server Compact (we love Compact, btw). This was caused by the fact that we ran it on a 64 bit computer, and it tries to download the 32 bit version. You can solve it by downloading and installing SQL Server Compact yourself. To complicate things, the 64 bit version of Compact requires the 32 bit version of Compact (aaargh), but after that it works and you can install StressStimulus after all. That was the point when we found out that the free version of it was so limited that we had to forget about the tool.

As far as D-ITG (the Distributed Internet Traffic Generator) goes (http://www.grid.unina.it/software/ITG/), we should have known better when we saw the animated NEW images and should have stopped there.

With the Fast Web Performance Test Tool (FWPTT at http://fwptt.sourceforge.net/), we pressed ‘Start Recording’ and that’s the last we ever heard back from the tool. That is probably caused by the fact that the tool might not believe in communicating with the end user.

The HttpRider tool (http://httprider.codeplex.com/) started out all right, but apparently isn’t able to handle authentication scenarios which renders it quite useless (unless of course, getting unauthorized HTTP responses is your thing).

After all, IxoraRMS (http://www.ixorarms.com/) turns out to be not Windows-oriented enough for our tastes. We want a setup or exe button, okay?

OpenLoad (http://www.opendemand.com/openload/) is a command line tool that allows to specify a single URL and simulates 5 clients. Doesn’t like authentication either. It does work if you’re exactly looking for this type of functionality, but really, if this is what we can get we prefer to write our own tool instead.

OpenSTA (http://opensta.org/) begins by setting our expectations below zero by requiring a system restart. Really, for this type of tool? It’s just not fair. After that, we find out that by default it expects the IE5 browser and complains that it can’t find it (no kidding?) and again we’ve had enough.

Ostinato (http://code.google.com/p/ostinato/) looks kinda cool but managed to wreck our web server in such a way that we decided to restore a previous snapshot. Having said that though, it did look like a powerful tool that requires more studying.

TestMaker (http://www.pushtotest.com/) asked us to download a version that was 600+ MB in size. Apparently we asked for the OS version of the load and stress testing tool, such sizes are ridiculous, so we’re not that desperate yet.

At this point, we remember why Open Source isn’t the end all solution for all of world’s software problems. The first three tools are our top 3 choices, but certainly, when considering this type of tool, it might not be a bad idea to try to get some budget for it and check out a commercial vendor. We’d say that except for WCAT and maybe LoadUI, there’s no free lunch and the free functionality is so limited that it’s nothing more than a teaser to buy commercial versions.

7 responses to “The great free performance, load and stress testing tools that can be used with SharePoint verdict”

Not sure why/how Ostinato wrecked your webserver. If you are interested, feel free to drop by the Ostinato mailing list and we can investigate.

That said, it appears you are trying to stress test sharepoint server with multiple connections simulating clients requiring authentication. At the moment, Ostinato is “dumb” – it just pumps packets as you configure them. It does not have any intelligence to do TCP handshakes, SEQ/ACK ordering etc., much less authentication – so it might not be the right tool for your need.

If you are installing StresStimulus on a 64-bit machine, by design the installer will try to download and install the 32-bit version of SQL CE, followed by the 64-bit version. Our bad that we did not make it more clear.

The latest version 4.2 of StresStimulus made improvements to the Free version. It now has the same test designer as in Pro version and works with up to 250 VUs.