Regression Testing

Do you know the importance of Quality Assurance (QA) testing of your app before it gets to the end users?

A product owner lives by his idea and spends thousands of dollars before taking it to the market. Making a quick launch without conducting proper QA testing is definitely not a good idea.

QA testing plays a vital role for the success of your product as it helps in determining the quality of the product and in ensuring customer satisfaction.

There are several manual or automated ways to test an app functionality and performance. Today, I will share an overview of Regression Testing and why is it important for software development lifecycle.

What is Regression Testing?

Regression Testing is a type of software testing in which the functional and non-functional tests are executed again to make sure that there are no bugs after some changes have been made in the code. These tests were initially performed when the application was first developed.

Already executed test cases are re-executed to ensure that the existing functionalities work fine as there are chances of finding bugs after some part of the code or program is changed.

This test ensures that the alterations made in the code have no adverse effects on the existing functionalities of the software.

This testing may be required when:

The code is modified due to change in the requirements e.g. there might be a need of login via Facebook option on the login page now.

The code is modified due to addition of a new feature e.g. a company decides to remove remember password option on login page.

There is a defect fix e.g. login button stops working and needs to be fixed.

There is a performance issue fix e.g. loading a page in 2 seconds instead of 5 minutes originally.

There is an environment change e.g. updating database from MySQL to Oracle.

All of these cases involve some change in production code and once the change is made there are huge chances of finding bugs due to modification. These bugs occur due to the dependencies of different functionalities within the code.

Regression Testing Example

Suppose the login button in a login page of a software stops working due to a defect. Once the developers fix the login button issue, testers check if the previously broken button is now fixed or not.

Meanwhile, regression testing is also performed by the testers to ensure that the functionalities related to login button are also working fine after the developers changed the code in order to fix the login button.

Types of Regression Testing

Regression Testing does become necessary as the modifications may cause the system to work incorrectly. The various regression testing technique types include:

Retest All

Regression Test Selection

Test Case Prioritization

Hybrid Testing

Retest All

In this type, as the name suggests, all of the existing test cases are re-executed. It is quite expensive as it requires huge time and resources but it ensures there are no errors after the code has been modified.

Regression Test Selection

In this technique, just a part of the test suite is run but only if the cost of selecting the part of the test suite is not greater than the Retest All technique. Test cases can be divided into re-usable and obsolete test cases where the latter is not used in succeeding cycles.

Test Case Prioritization technique

In this type, the priorities of the test cases are set on the basis of business impact, critical and frequently used functionalities, in order to improve fault detection ability of a test suite. Higher priority test cases are scheduled for execution earlier than the lower priority test cases. Test cases selection on the basis of priority greatly reduces the regression test suite.

Hybrid Testing

This technique is a combination of Regression Test Selection and Test Case Prioritization. Instead of selecting the entire test suite, select only the test cases that are to be re-executed depending on their priorities.

Regression Testing in Agile

Agile software development is increasingly popular among other development approaches in software development process. Numbers of tests in agile development increases with the passage of time as you have to ensure that the new changes being implemented do not affect other functionalities of the software. So regression testing can have a huge role in the agile development process.

Since there are a lot of build cycles in agile development process, regression testing becomes highly effective as the changes are being made continuously.

In agile development, regression testing allows the entire team to focus on the functionality of the current sprint and target best possible solution while maintaining overall stability. Test automation and continuous execution can help in maximum utilization of the regression testing. For this purpose, regression tests should be scripted and run automatically in order to get rapid feedback. The list of all the important business functions which should be included in the regression test is compiled.

In agile condition, regression testing can be divided into following two categories:

Sprint Level Regression Testing

End-to-End Regression Testing

Sprint Level Regression Testing focuses on testing new functionalities after the last release whereas End-to-End Regression Testing incorporates all the functionalities of the software.

Following approaches can be opted for regression testing:

Traditional Approach

In this approach, each sprint cycle is followed by sprint level regression test. After some successful sprint cycles, the software undergoes end-to-end regression testing. This approach gives testers the flexibility to choose how much automation they actually want to implement in their testing.

Delayed Week Approach

In this approach, there is no timeline to confine the sprint level regression test. Like if a sprint level regression test was meant to be completed in two weeks, but failed to do so, can now be completed in the third week. At the start of testing routines, when the testers are trying to find possible bugs in the software, this approach helps a lot then. Continuous testing lifts the burden of backlogs built during end-to-end regression tests as the bugs found earlier are not left to be dealt with later.

Delayed Sprint Approach

In this approach, the regression cycle is regular and also the regression test cases that were employed for the next sprint have the functionality story of the previous story. There is no need of having separate sprint cycles when using this approach because the regression cycles are delayed by one sprint. With this approach, longer end-to-end regression test cycles can be avoided.

Organizations should think what regression testing approach will keep the continuity of business functions in mind. For best results, defects found during regression tests should not be tolerated at all and should be dealt immediately.

Hence, we know that regression testing can play a huge role in agile development. Apart from its purpose to prove stability, it also helps in focusing on the functionality in the current sprint.

Regression Testing Tools

Some of the important tools for Regression Testing are Selenium, Quick Test Professional (QTP) and Rational Functional Tester (RFT). Selenium is an open source tool used to automate web apps. Regression testing based on browsers can be done with this tool. HP Quick Test Professional (QTP), an automated software, is also used to automate functional and regression test cases using VBScript language. Rational Functional Tester (RFT) by IBM is a Java based functional and regression test case automating software.

Manual execution of test cases can be costly especially when the changes are made more frequently. The testing tools mentioned above help in automating the regression testing cases to save those costs.

Questions & answers

What is regression testing?

Regression Testing is a type of software testing where the functional and non-functional tests are executed again to make sure that the previously developed and fully tested software still performs fine after some changes have been made in the program or code.

For example, a login button on a login page has a defect and has to be fixed. Once the developers fix the login button issue, testers check if the button is now fixed or not. Meanwhile, regression testing is also performed by the testers to ensure that the functionalities related to login button are also working fine after the developers changed the code in order to fix the login button.

Is regression testing part of UAT?

Regression Testing and UAT are completely different concepts. In ideal cases, regression testing is performed first and then, once the final build is created, UAT is performed.

Regression testing is basically retesting around an area where the code has been changed or a defect has been fixed.

On the other hand, when any software is developed and tested, it is handed over to the users for User Acceptance Testing where the users then focus on end-to-end scenarios. This is the last part of the software testing process and helps to demonstrate that required business functions are operating in a manner suited to real-world circumstances and usage.

What are the types of regression testing?

As mentioned above in the section 3, the various regression testing types include

Retest All

Regression Test Selection

Test Case Prioritization

Hybrid Testing.

In Retest All type, all the test cases are re-executed where as in Regression Test Selection type, just a part of the test suite is run but only if the cost of selecting the part of the test suite is not greater than the Retest All technique.

In Test Case Prioritization technique, the priorities of the test cases are set, on the basis of business impact, critical and frequently used functionalities, in order to improve fault detection ability of a test suite.

Hybrid technique is a combination of Regression Test Selection and Test Case Prioritization. Instead of selecting the entire test suite, select only the test cases that are to be re-executed depending on their priorities.

Why is regression testing done?

Regression testing, both manual and automated, is very important for huge software applications because it is usually very difficult to know whether changing part of an issue has created new issue for a different part of application.

Regression testing is important to ensure that there are no bugs found after a code change has been performed. Regression Testing do become necessary as the modifications may cause the system to work incorrectly.

Can regression testing be automated?

Yes, regression can be automated, in-fact, it should be automated for excellent results. An inventory of the workflow present is transformed into test scripts. Automation of the operation of these test scripts is then done on the basis of changes in the application. The test cases that are stable and repeatable are automated. This allows manual testers to utilize their time in exploring more unusual cases in the product environment. There are wide range of automated regression testing tools available that help in recording & playing back selected test cases and simplify regression report generation.

Conclusion

An effective regression testing strategy helps organizations in saving both time and money. According to one of the case studies in banking field, regression testing can save up to 60% of the time in defect solving in addition to 40% of the money. Regression testing boosts the confidence of the programmers that their code is bug-free after its modification. It ensures there are no unexpected problems or bugs.

Regression Testing’s biggest challenge is to achieve maximum test coverage while using minimum test suite. After successive running of regression tests, the size of test suite increases. This is why the execution of regression tests fails at times. Time and budget constraints play a huge role in hindering people from executing regression tests.

Try to develop a test-driven development through Origami Studios (www.origamistudios.us) which is one of the leading companies, around the globe, in providing expert application development solutions. Origami Studios is enlisted on Fortune 500 and is one of the most efficient and reliable companies out there.

At Origami Studios, our engineering team conducts several test cases to verify the quality and accuracy of an app before its launch. If you’re a development company or if you want your app to be tested by professional QA engineers, kindly contact us.