5 Answers

I think my first question is what kinds of test data do we need generated? What can we do without the tool? What are out gaps?

Try to avoid the term "Test data generation" entirely and use other terms. Find a tool that supports your /what/.

So, for example, in health care, one desire is to have production-like data that is anonymized. A tool that can look at one prod envrionment, assess it, and "spit out" prod-like data in to a test environment tht is anonymized might solve this problem -- that is probably the single biggest use of these tools I know of. But you might not have that problem, and might be able to use perlclip combined with some ruby scripts. :-)

Whne you are considering going for test management tools, obviously your first consideration would be the size of your setup and whether your budget allows you to go for a paid or an open source tool. This would narrow down your probable choices. Then you will have to consider the features of the tool such as the report generation options available and how to share these reports with the various stakeholders. There are reporting tools such as Geckoboard to which you can push data in real-time so that the report generation is automated and you don't have to worry about it on a regular basis. As a technical writer for OnPath Testing, I have written this blog on data dashboards from Geckoboard and how they make your life easier.

But, and you guessed it right, such tools are paid. So, you can consider cloud-based spreadsheets which will afford you shared access among team members and version control also. You can control who gets to access which spreadsheet (edit, view or comment) with access control features. Keeping the spreadsheet updated is also easy as the whole team has access to it. So, maintaining the test data becomes a task with shared responsibility.

When comparing test management tools, you should look for these features.

It's always a good idea to use a demo if possible and test drive the product, so to speak. In this way, you can gauge what changes you would like to see in a customizable solution and measure the value if data collection services available on different platforms. Of course, ease of use is also important, so look for a product that offers QA knowledge and support.

Here's an article that hits right on the vital points when it comes to buying a test case management tool, a buyer's guide:

In order to compare tools create the list of items that are important to you and your compnay.what do you need to see with each project? What are the repeatable factors that you have to use with each cycle of testing. What do you have to capture to,show project success and failure? Other factors to consider are the canned reports that come with the application vs the ones you have to build.

These are some of the factors to consider before purchasing an application.