Software Testing Estimation Techniques

Software testing estimation techniques are as old as the binary digits in the history of digital computers. Software testing is one of the important stages during software development life cycle (SDLC) to check and control the quality of the application.

Today, over 30% to 70% of a project’s resources are dedicated towards testing. Still, testing is often considered as an underestimated part of software development. Moreover, academic research in testing is low compared with the importance of the subject in real-life software projects.

Software testing estimation techniques involves experimentally and systematically checking the correctness of software. It is performed by applying test experiments to a software system, by making observations during the execution of the tests and by subsequently assigning a verdict about the correct functioning of the system. The correctness criterion against which the system is to be tested should be given in some kind of system specification.

The specification prescribes what the system has to do and what not, and, consequently, constitutes the basis for any testing activity. Despite its importance, testing is often an under-exposed phase in the software development process. Software testing turns out to be expensive, difficult and problematic while, on the other hand, research and development in testing have been rather immature.

Software Testing Best Practices

In order to implement testing best practices for a software development or mobile app development project, one needs to analyze the risks and complexities about the project by estimating the testing efforts. Software testing estimation techniques play a very important role in building credibility before initiating any software or mobile app testing project. To achieve bug free code for your software and mobile applications, software testing estimating techniques should be implemented by your team.

Today, any software application developed is unique in its own domain. So it becomes difficult and inconvenient to estimate software testing accurately at the very first attempt. Even if the project scope is understood and requirements are clear, it is still arduous to estimate a complex system that is going to be built with its unique set of requirements.

Software Testing Estimation Techniques

One of the most important factors while estimating testing efforts is the hands-on experience on varied projects for the software test life cycle. No longer can one just take a guessing approach about the number of days for any task or working on the old time formula of one third of the development effort.

Although this method is not based on any scientific principle or technique, it is one of the most widely used estimation technique by companies offering software-testing services. Unfortunately the development versus testing effort method has given many failures in software projects in the past, thereby compromising the software or mobile apps on quality.

In the recent years there have been many techniques that have been developed for estimating the software testing timeframe. These techniques are: 3-Point Software Testing Estimation Technique, Use-Case Point Method and Wide Band Delphi Method.

3-Point Software Testing Estimation Technique

3-Point Software Testing Estimation Technique is based on statistical methods in which each testing task is broken down into sub tasks and then three types of estimation are done on each tasks.

Wideband Delphi

In Wideband Delphi Method, work structure is broken down for each task and is distributed to a team comprising of 3-7 members for re-estimating the task. The final estimate is the result of the summarized estimates based on the team consensus. This method speaks more on experience rather than any statistical formula. The Wideband Delphi testing estimation technique logically estimates the group iteration efforts required in a visual manner for the testing team. This test was coined by Barry Boehm and is widely accepted software testing estimation technique to solve complex problems.

How to evaluate a software testing estimation technique?

For any software testing estimation technique, it is highly recommended that the following factors should be taken into account:

Domain knowledge and core requirements

Risks and complexity of the application

Team knowledge on the subject/skills

Historical data for the previous estimation for improvement and accuracy

Estimation should include buffer time

Bug cycles for the project

Resources availability

However these techniques just provide the means for estimating but rely heavily on the team productivity variations, individual skills, complexity of the unknown factors like system environment and downtime.

Despite advances in formal methods and verification techniques, a system developed still needs to be tested before it is implemented in real-life. Testing remains a truly effective means to assure the quality of a software system of non-trivial complexity, as well as one of the most intricate and least understood areas in software engineering. Testing, is an important research area within computer science is likely to become even more important in the future.

Rishabh Software leverages the best practices for software testing and mobile app testing, helping clients globally to achieve their business goals. Our team of software and mobile developers solves complex test cases and simplifies the testing efforts. Get in touch with us to know more on implementing the right software testing estimation technique through our Application testing services customized just for your application.

Get a Free ConsultationTalk to our experts to get the best suited solution for your business.

The techniques above specified can be applied to a single feature or mulitple features. None of the above mentioned techniques is used for finding out the negative scenarios or performing any validations.

The above techniques are used for getting the estimated effort for a certain defined tasks which is to be anlsysed for preparation and execution of test cases. The above formula is derived from PERT analysis and is little bit tweaked in terms of test cases, for getting the effort for set of test cases classified in positive(P), negative(N) and exceptional(E) test cases. Hence the effort is calculated as P+(4*N)+E/6. Values 4 and 6 are the constant and can be interpreted for the difficulty or the complexity of the test cases and can also be taken as optimistic, pessimistic and most likely classification based on the PERT analysis.

For example,

1. Say in a project you have total 705 test cases out of which 400 positive, 180 negative and 125 exceptional set of test cases then the effort calculated will be roughly estimated at 1140.833 with standard deviation of 9.16

2. In another example, say if you have total 7 test cases out of which 4 are positive, 2 negative, and 1exceptional, then the effort will be 12 with standard deviation of 0.16

I hope this solves your query.

Thanks & regards, M Jha.

http://www.rishabhsoft.com M Jha

Hi Sivakumar M,

Thanks for the link that provides detailed information on PERT analysis and the 3 point estimation technique. However the 3-point mentioned in the above article is little bit tweaked for 3-point, as the software testing already deals into positive, negative and exceptional scenarios and then deriving the final effort.

Thanks & regards, M Jha

Venu

Hi Jha, Can you please give one example for Use – Case Point Method estimation techinque like as 3point estimaiton as above

Many thanks, Venu R

http://www.rishabhsoft.com M Jha

Hi Venu,

Consider a project having 10 use cases for which there will be minimum 10 test cases. So as per Use Case Point method, we can derive the efforts as follows: Say for 5 positive test cases it takes 5 minutes to execute, 3 negative test cases it takes 10 minutes and 2 exceptional test cases as 15 minutes respectively which can vary as per complexity of the test case execution then, Unadjusted Actor Weights = 85 (summation of +ve, -ve and exceptional test cases with actor weight which is in minutes) Unadjusted Use Case Weight = 10 (total no. of test cases or use cases) Unadjusted Use Case Points = 85+10 = 95 Adjusted Use Case Point = 95* [0.65 + (0.01 * 50)] = 109.45, where 0.65 is the constant factor for technical complexity and (0.01*50 = 0.5) is the environmental constant for the test case execution in worst case scenario) Total or Final Effort = 109.45 * 2 = 218.5 I hope this will help in understanding the final effort calculation for getting the test execution effort.

Normally we use UCP to calculate the project efforts but in this way you can calculate the efforts for the test efforts needed for the project.

Thanks & regards, M Jha.

bindu

Appreciate Jha for a detailed explanation and example for UCP. It was really useful.

Deepika

Hi, how do you know how many test cases will be needed when the requirements are not clear? How do you provide high level estimation at the initial stage?

http://www.rishabhsoft.com M Jha

Hi Deepika,

Very valid question. A lot of software test professionals are into this dilemma when the requirements are not clear and majority of the projects have a scope creep conundrum. From my perspective when the requirements are not clear then in that case, Three – Point estimation technique is the best way to derive the approximate no. of test cases. Three point estimation facilitates that for every requirement there will be one positive test case and for every positive test case there will be 4 negative test cases. By using the aforementioned formula you can derive the efforts for high level estimation. All the above estimation techniques if used correctly will provide you a tolerance of 5-10% variation or deviation against high level requirements estimated efforts.

Thanks & regards, M Jha.

basu

HI Jha,

How can u say “one positive test case and for every positive test case there will be 4 negative test cases”. Can you please give some example.

Ajit

Hi Basu, thanks for pointing that out. On behalf of M. Jha, please see the response as below:

When there is an uncertainty in requirements then Three – Point estimation technique is the best way to derive the approximate no. of test cases. Here as per the technique “for every positive test case there will be 4 negative test cases”; but practically it may happen for some positive test cases there can be 4 negative test cases. And that’s why we are expecting tolerance of 5-10% variation or deviation against high level requirements estimated efforts.

Hope it helps solve your query. Do let us know if any doubt still exists. Thanks once again.

Rohit

Does this take into account complexity of the test case. What if my positive test case is more complex and will take more time to designe

Rohit

Let me paraphrase my question. I am talking about the 3 point estimation technique, what will I do if I encounter a complex test case that has dependencies on other modules/3rd party softwares. Will this be treated as exceptional test case? If not what is treated as an exceptional test case?

Ajit

Hi Rohit,

Thanks for your time. Please see below the response to your query.

The complexity of test case has to be taken into consideration for positive, negative and exceptional scenarios. As you have rightly said that the complexity increases with 3rd party software integration. This integration complexity can have impact on positive and/or negative and/or exceptional test case.

An exceptional test case would cover special cases or exceptional conditions similar to some of the alternative branches of a use case.

Please let us know, if we’re able to solve your query. Thanks

Khaja Mohideen

Pls explain the test effort estimation methods widely used with an real time example

Kaushal Shah

Thank you for your time. A brief on the widely used estimation methods/techniques are already mentioned in the article. We will be doing a separate article on the same topic in the near future which will cover a real time example.

Sunil Yadav

Hi Mr. Jha,

I gone through your estimation techniques and found very helpful. They are very well described.

I would like to ask regarding the estimation in Agile methodology.

We got the Acceptance criteria as work items for each sprint and I face difficulties to estimate it.

Is there any specific technique for that?

Kaushal Shah

Hi Sunil, Assuming that it is a complex project and you alone are doing the required estimate. Normally for such scenarios we add more resources for doing the estimate. This is the Wideband Delphi method technique for estimation.

As per the Wideband Delphi method, project manager decides the team composition which consists of SMEs for the estimation. Ideally the team size is around 3 to 7 members. For agile it is recommended to engage the entire team in the creation of credible, supported estimates. The purpose is to gain consensus on the estimates for each user story or feature, which would be achieved through a series of iterative steps.

All the team members can follow the below steps 1. As per work line item define acceptance criteria steps 2. Divide criteria into complexity (Complex, Major, Minor) 3. Base on complexity provide estimation for each acceptance criteria