Wednesday, February 13, 2013

Semi-automated regression testing

Automated regression testing is pretty much expensive stuff, but what to do if you have few of resources but your you're in strong need to have some fast way to assess if your new build got affected with regression? You're lucky enough in case your application produces some standardized output like table-data in csv file format.

The approach may look like this:

Define the reliable output of the application version you're considering the stable one

Keep that data as golden-set

Each time you've got the new dev-build scope your new piece of data to make it relevant to your golden set

Use some comparison tool to detect if your new piece of data has the differences against your golden-set

Analyze the root-cause of the differences in case they are

Pro:

You save the resources from wasting

You know about the new problem in case your application has produced improper data but you have proper one in your golden set

You know about the old problem in case your application has produced proper data but you have improper one in your golden set

Con:

You do not know about the problem in case you've got improper data in both the sets