Running Web and Load tests from the Command-Line

This blog post will show you how to run a web test or load test from the command-line instead of from within Visual Studio.Most of the time you will want to run you web or load test from within visual studio, but you can run them from a command-line.Maybe you have a set of 5 web tests that you want to run once a day.You could configure a scheduled task to run a batch file which contained ran the 5 web tests.The command line executable that is used to run tests is called mstest.exe and is installed when you install either VS Team Suite or VS Team Test SKU.Here is a MSDN help link for mstest:http://msdn.microsoft.com/en-us/library/ms182486.aspx

The easiest way to get started with mstest is to start a visual studio command prompt.Go to Start Menu ->All programs -> Microsoft Visual Studio v9.0 -> Visual Studio Tools -> Visual Studio 9.0 Command Prompt.This will launch a command window which has mstest.exe in its path.So type mstest /?This will show each of the available options.Let’s run through a few options of how to run tests.

1)Running a single WebTest

First let’s run a simple web test.To do this, we just need to specify the /TestContainer argument.A .webtest file or .loadtest file is considered a test container in the same way that a dll which contains unit tests is a test container.So if we have a web test called WebTest1.webtest, you would use the following command to run the web test:

Now let’s execute 2 web tests from one call to mstest.You can specify multiple test container arguments on one command-line.So say we wanted to execute WebTest1.webtest and WebTest2.webtest.You would use the following command-line:

One of the big differences with running from the command line is that you are not within a visual studio project.When you run a web test from within Visual Studio, we will try to determine what needs to be deployed with the web test in order for it to run.For example, maybe you have written a custom validation rule or a extraction rule.When you run the test from within visual studio, we will look at the references of the test project and try to figure out what needs to be deployed.

When you are running from the command-line, you will need to be very explicit about what you deploy.So if you have a dll that needs to be deployed for a test to run, you will need to inform mstest.The way you do this is by specifying /runconfig parameter.One of the sections of a run config is deployment items.Use visual studio to create a run config.Then to run a test with a runconfig, you would have a command-line similar to the following:

When in VS, to run a web or load test on a controller/agent setup you need to use a test run configuration.You would open the run config, specify to run the test remotely and specify a controller name.It works the same way from the command-line.You will need to create a run config that has the correct settings.Then you run it the same way as you do when specifying deployment items:

You will notice that the results file(trx file) gets a unique name which contains user, machine and a timestamp.If you want to specify the name of the results file and where it is generated, you can use the /resultsfile parameter.For example, to create a results file called MyResults.trx and have it saved to c:\results, I would do the following: