An app I work on is being developed with a modified version of scrum. If you are not familiar with scrum, it's just an alternative approach to a more traditional watefall model, where a series of features are worked on for a set amount of time known as a sprint. The app is written in C# and makes use of WPF. We use Visual C# 2010 Express edition as an IDE.

If we work on a sprint and add in a few new features, but do not plan to release until a further sprint is complete, then regression testing is not an issue as such. We just test the new features and give the app a good once over. However, if a release is planned that our customers can download - a full regression test is factored in. In the past this wasn't a big deal, it took 3 or 4 days and the devs simply fix up any bugs found in the regression phase, but now, as the app is getting larger and larger and incorporating more and more features, the regression is spanning out for weeks.

I am interested in any methods that people know of or use that can decrease this time. At the moment the only ideas I have are to either start writing Unit Tests, which I have never fully tried out in a commercial environment, or to research the possibilty of any UI Automation API's or tools that would allow me to write a program to perform a series of batch tests. I know literally nothing about the possibilities of UI automation so any information would be valuable. I don't know that much about Unit testing either, how complicated can the tests be? Is it possible to get Unit tests to use the UI? Are there any other methods I should consider?

Thanks for reading, and for any advice in advance.

Edit: Thanks for the information. Does anybody know of any alternatives to what has been mentioned so far (NUnit, RhinoMocks and CodedUI)?

This question came from our site for professional and enthusiast programmers.

1

I am in the exact same boat. We just broke into using the Coded UI library. It is rather heavy and slow to get started but quite good and helpful, once you build up a library of functions that can manipulate the UI. That said, there are many annoyances in what Microsoft's vision of automation is. We just tend to use things that work and ignore the ones that do not.
–
JobMar 9 '11 at 21:40

the only reliable method I know is to hire more QA to arrange professional regression testing. Did use that "method" a lot, works like a charm... not to mention that it also helps developers focus on their work instead of wasting efforts in a fruitless attempts to be jack of all trades (division of labour has been invented long time ago).
–
gnatJun 2 '12 at 6:24

3 Answers
3

Naturally if you test everything manually, it will take you a lot of time. Especially in scrum projects, where testing is frequent and the time constants are short, it is important to use automated testing.

A good testing solution is a combination of unit tests (that test a small part of the code, typically a class), and automated integration tests (which follow end-to-end processes and use cases in your application). Some of the integration tests can definitely be UI tests, and there are frameworks for UI testing that you can use.

You can move from manual tests to automated tests gradually. Start by enforcing a practice that every new code written by the team must have unit testing. Read about Test-Driven Development, it's a bit of a harsh approach but it's a good point to start from when designing your own best practices.

NUnit, combined with RhinoMocks, is an excellent solution both for unit tests and integration tests.

In general, when dealing with large complex applications Unit tests are usually the best way to have comprehensive coverage but also the most expensive in terms of time to write when starting from scratch. You will likely need to refactor a large portion of code that is not written with test-ability in mind. In these cases its often faster and more efficient to invest in Integration and UI tests first; using the quality confidence they provide to re-factor code to make unit testing easier.

The most efficient approach, that I have used on multiple applications now is as follows:

Automate the most time consuming System/End to End Integration tests first

Start automating as you go: New defects should have an automated tests to cover regression; if you are modifying some code which doesnt have a test around it, write a few

You can't automate everything: So do a benefit cost analysis on which test cases you want to automate vs. which ones you can still run manually.

As you start automating tests, consider integrating the tests with a Continuous Integration engine like Teamcity, Bamboo or Cucumber so your tests run on every check-in thus reducing effort required to discover root cause of bugs and fixing them as they are introduced.

I have had success with unit testing my application with MSTest and testing my Javascript with QUnit and picking up the results of those tests with a single MSTest Unit test that uses WatiN (I was suggested Selenium, which has the ability to record tests too, but found WatiN better for automating cleanly), and using these for regression tests.