Using JIRA for Risk Based Testing – Webinar Recap

Risk based testing (RBT) is a popular testing strategy that prioritizes the tests of features and functions in software, based on the risk of failure, the function of their importance and likelihood or impact of failure.

However, most people don’t know how to incorporate into test cycles and release, while some think that RBT strategies are simply just ways to perform less testing. Moreover, a vast majority of these new risky features and functions are being tracked in agile project management tools like JIRA, VersionOne and Rally.

You can also view the slides below:

It was no surprise that 35% of the webinar audience had ‘Not Enough Time to Test’, making Risk Based Testing the perfect solution to addressing this challenge (see below slide).

The reality is, there will never be enough time to test everything and really, it’s not productive or efficient to do so even if there was.

Enter (RBT) Risk Based Testing.

Looking below, it’s clear how Risk Based Testing differs from traditional testing. Traditional testing focuses on literally testing everything where as RBT focuses on prioritizing the tests of features and functions in software, based on the risk of failure, the function of their importance and likelihood or impact of failure.

Add-ons to Compliment Risk Based Testing:

This add-on help you understand your JIRA projects better. Working as a complement to the primarily text-based reporting and numerical charting built into JIRA, visualizer displays each of your issues as a card on a zooming, scrolling canvas. You can then choose how these cards (issues) will be organized visually, choosing projects, issue types, sprints, assignees, or any other JIRA field to group them. You can choose X and Y axis grouping, and style the cards based on issue data.

As your issues spread out before you on the screen, the relationships between them are much more clear than ever before. Learn more about Visualizer for JIRA on the Atlasssian Marketplace.

Risk Register

This add-on lets you designate selected JIRA projects as risk registers. It carefully configures those projects to support the creation and management of risk issues, which are assigned a custom ‘Risk’ type.

You can work with risk issues in the same way that you do other types of issue: add comments to them, link them to other issues, watch them, attach images, and so on. In addition, risk issues have custom fields that let you assign a probability and an impact level, which together determine the exposure of the risk. You can then assign a treatment strategy to a risk. A custom workflow guides you through this process of analyzing and treating risks, and keeps track of where each risk is in that process.

Webinar Questions Recap:

In your experience, who normally leads/drives the process of rating the stories for risk? PM, Test Team Lead, Dev, etc.? While Dev and QA will need to work together to pull out the risk, I am curious who normally leads/drives this process into the test cycle.

I would suggest that is a mixture of the entire team, but should be lead at the story level by the product owner first. The product owner should be the subject matter expert in the application, and therefore should know historical areas of risk along with potential new issues.

However, the testers and developers also have experience with Risk identification. No one person owns the entire risk evaluation of the stories that need testing by priority.

Also, be aware that the product owners might identify some stories as high risk only because they want the feature to be signed off first to go into production. Risk priority and feature priority are different items that require setting aside biases.

What could be one critical risk for a software finance project that would be a major blocker and how can we go about while testing?

Every company is going to lose of revenue due to something the system not working. For example, an app store will lose revenue when a performance issue prevents someone from downloading their latest update. This finance project you are testing will have loss of revenue due to some problem in the system.

In one slide from the webinar, I showed how you would find define Impact of failure. For test purposes, we should look at the overall loss of revenue that could occur IF this risk is not addressed.

Critical Impact – 5 = The problem caused all major and critical functioning of the application to fail. This resulted in a large loss of revenue.

How will risk based testing work for regression?

In the below RBT model, the goal would be to test the P1 – Critical Impact areas first, with the goal to create automated smoke tests that can be added to a regression suite.

This way, when regression runs every release, we know that that have already incorporated P1 tests in an automated fashion.

Of course not all P1 areas of testing can be automated, but we should look for candidates first.

Who defines (or helps QA to choose) the Impact of failure? QA doesn’t always have visibility on impact of failure from business point of view.

With heuristic Risk Based Testing, James Bach suggests an inside out approach to identify risks. This is where you would grab a developer and ask them questions the exposes risks within the application, as the developer explain how it works.

I would encourage you to do this same paired research with the product owner or other stakeholders of the application.

When I was working with a large retail company, I would ask to go to the physical warehouses and shadow a the line workers and system managers. Believe it or not, the system owners actually like it when testers take initiative to learn about how they use the system! This allowed me to ask questions like:

If this scan fails, what’s your workaround?

If this validation doesn’t happen, is work halted?

What’s the last system failure that crippled your warehouse operations?

Talking with the actual end users and managers of the applications that I tested, unveiled new degrees of Impact that my developer wouldn’t have recognized.

What if there is need of 100% testing but we as a tester lack of time. What should be action from our side?

This all goes back to setting expectations for yourself and management. You can always test more, and test less. The key is to test what’s necessary with a high degree of confidence that your high risk areas have been covered.

With RBT, you are identifying the high risk areas of the application that can be tested within a certain amount of time. As you time shrinks, or risks change, then you plan can adapt. At the end of your time box, it’s possible to achieved “100% testing” since that 100% makes up the testing that you identified as needing priority.

Wouldn’t you want to write simple “happy path” tests for low priority items, and more “in depth” tests for the high priority/severity items?

This will depend upon your test strategy and complexity of the application. Some industries require in depth audit-ability of all test cases, while others do not.

However, I would always recommend incorporating at block of test time for exploratory testing. Exploratory testing would allow you to set time aside for Low and High priority items without being forced created in depth test cases.