To Bug or not to Bug?

Sheet Happens identified a potential bug observed in passing on a supermarket price label. A perennial issue for testers coming across something (arguably inappropriate text) while actually looking for something else (toilet roll) is how to decide (a) whether it’s an issue at all and (b) if it is, whether this observation is the complete problem, an instance of a more general problem or the tip of a problem iceberg poking out above the surface of the stormy software seas. Or something else altogether, ideally without an annoying alliterative analogy.

Oh yes, (c) what risk to associate with it while (d) expending an appropriate level of effort in any investigation. And (e) without yet knowing how serious the problem is.

In some cases it’ll be obvious. Your word processing application crashed and all your data was lost when you tried to change the text colour? No further thought required. In other cases it’s less obvious. You noticed that instances of some object in the database have one of two formats for a particular field. Both seem reasonable and that field is used only rarely for non-critical tasks. Now you need to think.

So now you’re testing Tesco’s systems for generating price labels and you’ve see the issue but aren’t sure what to do next. Should you report an issue or not? If so, on what basis? One quick approach is have some heuristics or standard questions for yourself. There are numerous techniques for this kind of thing (such as 7 or 5 Whys, HICCUPPS, Kipling’s W’s) and in this case the questions we’d ask and answers we’d give might include:

yes, if it’s cheap and has power to discriminate between the scenarios

yes, if the risk of a likely (given what we know so far) scenario is high

Working through this kind of thought experiment is usually low cost and quick, and it can help to improve your efficiency, work out your next steps, avoid compromising the mission you were on when you tripped over it, enhance your bug reports, aid the triage team’s assessment of the severity and priority of the issue and, ultimately, cause the right bugs to be reported and fixed at the right time given our best knowledge of the context.