Blog

Are you doing Effective Testing?

Have you ever faced a challenge of timely delivery with well tested software? Welcome, join the club, you are not alone out there who is facing this challenge.

Most of the time, it is very difficult to come up with a good test plan that can cover every possible combination of inputs and external conditions and still meet the software release deadlines. And hence effective testing is one of the challenges we face during software development life cycle.

In this post I am trying to convey an important message about – testing software effectively with well tested deliverables, with concrete example of “Login Dialog”.

However, before I start on this, lets back-up and revise our intention for effective testing.

Why should we care for Effective Testing?

Well, every stakeholder can have different answer to this question.

Software developer: My code must be defect free, I must be known as the one who always delivers quality code.

Tester: I need my next promotion, so I must showcase my skills of how much effectively I can test.

Project Manager/Product Owner/Scrum Master/Company: We need repeat business by delivering quality product to the customer.

Customer: The software I am using must work as advertised. I cannot tolerate any issues that hampers my work.

Who is responsible for effective testing?

Of course, tester is always responsible. In fact, he is the gate keeper who does all the dirty work which sometimes developer is not willing to do.

Product quality is responsibility of entire team, and hence testing is a team effort.

“You cannot have the mindset that QA will find the bugs in your code. Instead, you should absolutely make it your responsibility to find and fix the bugs before your code goes to testing.” – John Sonmez

Project manager/Product owner/Scrum Master: Must ensure that requirements conveyed are the ones which needs to be implemented. They validate and re-validate whether deliverable’s are of good quality.

What is involved in Effective Testing?

Clarity in requirements

This is must, without clarity in requirements it is not possible to deliver quality product that customer can use. Agile methodologies such as Scrum enforces this as part of backlog grooming, however if team is actually doing tailored Agile approach then it is possible that developers work on features without absolute clarity and this might lead to issues later on. Issues may include rigid and inflexible software because of bad design, re-work on features and defects in the system.

Maximum test coverage of functionality in minimum time

There is always pressure to release the software as soon as possible. Hence in limited time team needs to achieve maximum test coverage in the minimum possible time. Here are some ways to maximize test coverage –

Guidelines

Prepare test data/input data.

Consider maximum possible permutations/combinations of required inputs, however some combinations follow similar code paths, hence some of the combinations can be skipped for testing. Always pick the most stringent inputs.

Crystal clear test strategy

This requires multiple strategies and you cannot rely on one or two testing strategies. Here are some minimum recommendations –

Unit testing:

This is the best effort from developer’s side to ensure that newly written code does what it is expected to do, and how the code will react to external impacting factors that may or may not have favorable conditions.

I personally like to have fully automated unit test cases, however for legacy code (with lots of dependencies) for which automated tests can’t be written there must be some plan to perform basic unit testing.

When TDD approach is followed using unit testing frameworks (such as MSTest, NUnit, xUnit et al), this is the most powerful way of development.

Integration testing

It is must to test multiple independent units of system working in cohesive manner.

Examples:

Authentication module used by login dialog needs to be tested with “Authorization Module”

Login Dialog is required to re-authenticate user before data is submitted

Integration Testing of modules in the system

Workflow testing

Test end-to-end workflow of end-user scenarios. (And this may be called a system testing)

Examples:

Administrator logs into the system and approves financial data submitted by Accountant of the company.

POS operator logs into the system and for each customer in the queue prepares bill of purchased goods, applies any coupons for discount and accepts payment from customer and generates the bill.

Performance testing

Identify areas in your software where performance testing is required, and define your goal in this area.

Examples:

System can handle 1000 login requests per second

System can process 1 GB data per second

Technical reviews

It is absolutely must to perform technical reviews. These are far cheaper and more effective than testing if done properly.

Design reviews: When we do design review, we not only validate from requirements standpoint, but also for future extensibility point of view. And this helps a lot as we progress from one version of software to another.

Test case reviews: Whether it is automated unit test case, manual test case document or any other type of test case (Coded UI, Specflow etc.). Often this step is skipped and some of the important aspects are always missed.

Documentation of Results

Documentation of expected results: In Scrum or Kanban type of project ideally this should get recorded as part of acceptance criteria. However, when functionality is complex, there may be additional aspects that needs to be documented. Perhaps these could be implicit requirements that were not captured in the backlogs acceptance criteria.

Documentation of Actual results: Professional teams know the importance of actual results documentation. And if you are using JIRA or VersionOne like tools then you may be already doing this. Just note that don’t skip this one.

Our main focus is on requirements, however we should also verify what application is not intended to do.

The Inverted Testing Pyramid

Cost of Defect at Various Stages

** This image belong to the rightful author(s)

Guidelines To Write Test Cases

Intelligently choose the relatively few test cases

Cover all requirements and features defined in the requirements specifications

Test cases should not be redundant

Maximum focus on scenarios that users are likely to encounter in practice

Analyze all points at which data enters the system and look for ways to attack it