We have unit and integrations tests in our project and want to work test driven. One questions that raise up is this: When a ticket is created because somebody found a bug, should we create a small test case just for that bug, or should we include this case in the general tests?

4 Answers
4

I believe the answer is "it depends". If you're talking about a unit test, why not? These should be the bulk of your tests anyway, and usually run very fast. But it may not be worth the time or money.

If you're talking about functional tests, probably not. But again, it depends. Was the bug a cross-browser or OS issue? Was it a logical bug or typo? Will the test give you a satisfactory level of confidence that it wouldn't happen again? What is the business impact if the bug happens again? How much does it cost the company to have you write a test for a particular bug? Is there sufficient ROI or risk mitigation to make it worthwhile?

These are the types of questions that you need to prioritize before you can really determine if a particular bug should have a corresponding test, and the business needs usually take precedence over all else.

Exactly. If you always re-test every bug that has ever been fixed, do you necessarily get better quality? It may be better to spend that time testing in other ways. Or to spend less time testing.
–
user246Mar 23 '12 at 16:12

I do not create a new test case for every bug. I believe testing is about managing risk. If I believe the risk of a a bug reappearing is low, I will test the bug when the developer says it is fixed, but I will not re-test it for every subsequent release. Judging risk is subjective, and sometimes my judgement is wrong, but I think this way of testing is a better use of resources than testing for every bug that has ever appeared.

Other organizations may manage risk differently because of the nature of their business, their regulatory environment, or the culture of the company.

Isn't the fact, that this bug existed in the first place, a good indicator for a fragile feature in most cases?
–
Tim BütheMar 23 '12 at 9:36

If by fragile you mean likely to have more bugs, I do not think so. But I think that is a different issue than whether a fixed bug is likely to break again some time in the future.
–
user246Mar 23 '12 at 12:58

I'm not sure what the difference is between a "small test" and "general tests".
Anyway, we add tests based on bugs to our main test suite, with some exceptions.
One of the exceptions is the test not being a "small test", some bugs are difficult and time consuming to replicate or verify.
Another exception is "not our bug", meaning a bug in an adjusting module that we are not responsible for (unless it's really trivial to be tested).