QASource Newsletter

AutoCast - Winter 2019

As per many software testing experts, code coverage is a good way to identify the untested code. But the percentages aimed do not necessarily contribute to test quality. Such coverage goals encourage poorly designed tests, written with the only intention of meeting system requirements instead of testing the software correctly. We agree that unit testing is important but when aiming at quality its limited. A useful alternative is functional test coverage through test automation, which tracks whether tests execute important values or sequences of values corresponding to software features.

Code Coverage As Part Of Continuous Testing

Is your application developing unknown bugs with each sprint for unknown reasons? Not sure if your product is market-ready?

Implementing code coverage by means of unit tests have long been the answer to the above questions. This helped to avoid unknown bugs, but to an extent.

With the inception of agile methodologies, teams started working in small sprints. Automated testing became popular and unit tests were eventually pushed to the back seat. Today, we have tools that perform code coverage from automated functional tests, regressions suites, exploratory testing and even unit testing.

These tools provide a good overview of a continuously evolving code through tables and graphs in dashboards, hence providing a commendable release confidence. The tools perform both unit and automated functional testing coverage on app’s code. Comparison from both verticals is now possible which lets a tester analyze the production release.

A Second Thought On The Traditional Ways Of Code Coverage

Unit tests were initially considered as the only metric for code coverage, which often led to poorly designed tests. Such tests are generally written with the intent of meeting just the system requirements. Unit testing is important, but only when quality goals are limited.

Code coverage that includes test automation as well as functional testing in addition to unit tests is a better alternate here. It ensures that the source-code meets the system requirements as both functional and unit tests are used to produce the code coverage. Test quality is bound to refine as coverage is more comprehensive than just based on unit tests.

New Trends In Code Coverage

As mentioned above, unit tests are limited when aiming at quality through code coverage. So, one must consider integration tests, acceptance tests and even manual tests as a criteria. This is now possible through SeaLights and even JaCoCo now offers the code coverage through functional tests.

SeaLights

This is a continuous testing platform. It integrates with the test environment and measures test coverage across all types of tests. SeaLights provides insights on the extent of the tests and the risk involved with untested code changes.

Features

Code Coverage through automated functional tests and manual tests

Test GAP Analytics

Release Quality Analytics

Quality Trends, Charts, and Dashboards to analyze results

Integrates with Jenkins and Maven

Benefits

Clear stats depicting what application code has been covered through QA automation

JaCoCo

JaCoCo is a code coverage tool that calculates coverage based on unit tests. It’s developed in Java language and provides support for Groovy as well. It provides good reports of code coverage metrices and calculates the percentage of code covered based on methods/branches/lines. JaCoCo adds the lines of code for coverage calculation in byte code of the Java class. When executed, it instruments the class files used in the test with the coverage information. This produces a jacoco.exec file which is then used to generate the code coverage report.

Benefits To QA TeamsAdopting code coverage tools has empowered us in decision making in following terms

Reports generated from these tools inform developers about the code (branches/lines/conditions/methods) or inputs that are mostly, or never used by test automation scenarios. These reports help the teams prioritize the key areas. This ensures:

Product quality

Test quality over sprint lifecycle

Code quality and readability

The QA teams are able to create new test cases of uncovered areas of code

Dead code detection helps us in the following:

Minimizing unknown bugs

Reduced production costs

Higher code coverage and hence confidence in the product

Tips To Implement Code Coverage

Begin with a goal; don't strive for 100% coverage initially

Aim for a tool that considers your functional tests for calculating coverage

Focus on increasing code coverage with each release cycle

Use reports from previous analysis to make estimations each sprint

Select a code coverage tool that supports your

App’s programming languages

Tools like Maven, Jenkins, bug tracking and project management tools

Have Suggestions?

We would love to hear your feedback, questions, comments and suggestions. This will help us to make us better and more useful next time.Share your thoughts and ideas at knowledgecenter@qasource.com

Disclaimer

The logos used in this post are owned by the individual companies of each logo or trademark. The logo is not authorized by, sponsored by, or associated with the trademark owner, but QASource is using the logos only for reviewing purposes. The endorsement of the used logos by QASource is neither intended nor implied.

Request A Quote

Written by QA Experts

QASource Blog, for executives and managers, shares QA strategies, methodologies, and new ideas to inform and help effectively deliver quality products, websites, and applications.

Subscribe to our blog

QASource exists to help organizations like yours enjoy the benefits of a full QA department without the associated setup cost and hassle. With an emphasis on time-bound delivery and customized solutions, we excel at helping our partners manage the quality of their deliverables while keeping costs low.

QASource uses cookies to optimize users' experience. Click "Agree and Proceed" button to confirm your consent to the use of cookies. OR, by continuing to use this website, you implicitly accept the use of cookies. Find out more