Old but Relevant | The 7 fundamental principles of Software Testing

The world is driven by beliefs. It may be in God or Technology. Over a period of time every society lay down a set of philosophies or values for better organization & operation. Similarly, any technology has its own fundamental principles which have been proven right as the time elapsed. ‘Software Testing’ is no different – it also has a set of 7 fundamental principles that are proven right over the time.

1. Testing shows the presence of defects

In simple terms – Software Testing is the activity of finding defects. Flipping the coin, it doesn’t mean post-testing the application is defect-free. As we say in life – ‘There is always a scope for improvement’, similarly even after rigorous testing there is always a scope of defect occurring or further optimizing the application. The goal of testing is clear – to identify ‘as many’ defects as possible.

Software Testing is not a proof of correctness, it’s all about the assurance that most of the defects have been identified (and rectified) thus reducing the probability of undiscovered defects that might occur in live environment. Why do you think companies don’t sign a contract for a software to be defect-free after testing OR why do they have warranty period even after the release?

Trivia: Do you think Facebook don’t do rigorous testing? If they do, then why to have the famous Bug Bounty program encouraging end-users to identify still-unidentified defects.

Please note this doesn’t mean you can use this principle as an excuse for poor quality, instead focus on designing the test cases that have maximum coverage and identify ‘maximum’ defects!

2. Exhaustive testing in not possible

Exhaustive as in complete or in-depth testing is not possible. Why? You don’t have all the money, time & resources at your perusal – the budget is limited, timelines are defined, resource count is fixed, P&C are quite huge, etc. Even a simple text-box with 5 possible values need data tests, UI tests, security tests, compatibility tests, etc. Keeping different factors in mind, it is not possible to test all the possible combinations, flows, data & scenarios.

What’s the way out then? Prioritize based on the Risk analysis! Use different Test Design techniques to optimize test cases. Focus is to derive optimal amount of testing required based on the risks & priorities, such that it uncovers maximum defects focusing on the most important aspects first.

3. Early testing

Say client asks to develop a login page for his application. Developers build a page with Username & Password fields and deploy it for testing. During testing you observe that there is no ‘Forgot Password’ link provided. Come on, it’s obvious! What do you do? Simple – raise a defect >> client says it’s obvious >> requirements are detailed >> developer changes the code to fix it >> you retest it & close.

Now imagine Testing team is involved from the project start itself. What do you do? As soon as requirements are received, you raise an issue with business team about the ambiguous, missing or confusing requirements >> Business team details out the requirements clearly. Result – It is easier & cheaper to fix the issue if identified earlier in the cycle. Hope you got the point 😉

Software Testing should start as early as possible in the Software Development Life Cycle. E.g. it is much cheaper to change an incorrect requirement than having to change a functionality in a large system that is not working as requested or as designed! That’s why we have requirement analysis, unit tests, design discussions, impact analysis, reviews, etc.

4. Defect clustering

Ever heard about the ‘Pareto principle’, also known as the 80–20 rule? It states that for many events, roughly 80% of the effects come from 20% of the causes.

80% of your sales come from 20% of your clients

80% of a company’s sales come from 20% of its products

80% of the traffic occurs during 20% of the time

Many natural phenomena (population, wealth, sports, business, technology, etc.) have been shown empirically to exhibit such a distribution. Applying this principle to Software Testing – 80% of the defects are found in 20% of the modules. Yeah! That’s roughly true. And how do you identify such modules? It comes with experience my boy 😉 (Or girl)

Trivia: Management consultant Joseph M. Juran suggested the principle and named it after Italian economist Vilfredo Pareto. Pareto noticed that 80% of Italy’s land was owned by 20% of the population. He then carried out surveys on a variety of other countries and found to his surprise that a similar distribution applied.

5. The pesticide paradox

Trivia: The pesticide paradox states that applying pesticide to a pest may end up increasing the abundance of the pest if the pesticide upsets natural predator–prey dynamics in the ecosystem, i.e. the target pest has a naturally occurring predator that is equally affected by the pesticide.

Or in other words, if you keep running the same set of tests over and over again the software gets immune to testing, i.e. as the system evolves, many of the previously reported defects will have been fixed and the old test cases cannot find any new defects.

What’s the way out then? To overcome this “Pesticide Paradox”, the test cases need to be regularly reviewed & revised, adding new & different test cases to help find more defects.

6. Testing is context dependent

Context as in the environment, i.e. application type. All the developed software’s are not identical. You might use different methodologies, approach, method, techniques and types of testing depending upon the application type, whether it’s a Game, Mobile App, Cloud-based, Web-based, desktop application, embedded system, etc. After all testing an ATM machine will be different from testing Flipkart.

7. Absence of errors fallacy

This is a bit tricky 😉 First of all don’t confuse this principle with 1 & 2. In simple terms:

Testing is to identify defects, not to confirm that there are no defects.

Even after complete testing, defects might be present in the system.

Coming to absence of errors fallacy, imagine you developed an e-commerce system & tested it completely. All the identified defects have been fixed & retested. 99% of the defects have been rectified. Management is pretty sure about the product quality w.r.t. defects. Now when the system is demonstrated to client, what if you get a feedback saying “Though it is so-called defect-free, but still this is not what I wanted. I wanted a simple UI that can handle the user load”?

Yeah! Everybody in the Test team was confident about the product ‘quality’ (absence of errors) but at the end it proved to be false (fallacy) – the system is not usable, it doesn’t fulfill client’s expectations 🙁

The lesson: Client’s requirement & expectations are as important as product quality. And it might change over time. Why do you think Agile methodology is so popular now-a-days? It allows continuous interaction with business users and rapid prototyping (minimum marketable feature or MMF) prevents unhappy clients at the end.

Now that you know the 7 fundamental principles of software testing, don’t forget the key take-away:

Focus on designing the test cases that have maximum coverage and identify ‘maximum’ defects.

Related Articles

Continuing on our previous article – Comprehensive list of Software Testing Types | Agile, Acceptance, Beta & Ad-hoc are NOT the ‘Testing Types’, let’s deep dive into the world of different Testing Types. (Hope you have gone through the basics “Attention! Get your facts clear before facing an interview!” to clarify most of the doubts about different Test terminologies.

In software project management, software testing, and software engineering, Verification & Validation (V&V) is the process of checking that a software system meets specifications and that it fulfills its intended purpose. In the context of testing, “Verification and Validation” are very widely and commonly used terms. Most of the times, we consider the terms same, but actually the terms are quite different. In this article we will first explore Verification vs. Validation and then move on to its practical application in Software Testing.

This topic may seem simple to many, but despite of hundreds of web articles – Smoke, Sanity, Retesting & Regression are the most misunderstood topics in Software Testing. There is enormous amount of literature on the subject, but most of them are confusing. The following article makes an attempt to address the confusion. Before understanding these terminologies, first & foremost you need to understand the concept of Software Build.

About STS

Software Testing Studio is an attempt to share some incredible knowledge from industry leaders & experts, which should be helpful for anybody to start his/her career in ‘Software Testing’ or to progress it further. Apart from the technical nitty-gritties, one can also find some intellectual posts by industry experts sharing their Wisdom.