10/26/17

Software development practices change over time, so do
the tools and technologies.

Such changes aim to improve productivity, quality,
customer satisfaction, to tackle ever-shorten delivery time, and to deliver
successful products and services. Software testing obviously plays an
important role in achieving these objectives.

The recently released World Quality Report 2017–2018 by Capgemini,
Sogeti, and Marco Focus points out several interesting trends in software
quality and testing. Two of three key trends are increasing test automation and
widespread adoption of agile and DevOps methodologies. As the report shows,
organizations need intelligent automation and smart analytics to speed up
decision making and validation and to better address the challenges of testing
smarter devices and products that are highly integrated and continuously
changing. The report also suggests the need of smart test platforms that are
self-aware and self-adaptive to support the complete application lifecycle.

In the test automation landscape, automation tools
certainly take a center stage. This post summarizes the top test automation
tools and frameworks that have the potential to help organizations to best
position themselves to keep pace with the trends in software testing. The list
includes both open-source and commercial test automation solutions.

1) Selenium

Selenium is possibly the most popular
open-source test automation framework for Web applications. Being originated in
the 2000s and evolved over a decade, Selenium has been an automation framework
of choice for Web automation testers, especially for those who possess advanced
programming and scripting skills. Selenium has become a core framework for
other open-source test automation tools such as Katalon Studio, Watir,
Protractor, and Robot Framework.

Selenium supports multiple system
environments (Windows, Mac, Linux) and browsers (Chrome, Firefox, IE, and
Headless browsers). Its scripts can be written in various programming languages
such as Java, Groovy, Python, C#, PHP, Ruby, and Perl.

While testers have flexibility with Selenium
and they can write complex and advanced test scripts to meet various levels of
complexity, it requires advanced programming skills and effort to build
automation frameworks and libraries for specific testing needs.

Katalon Studio is a powerful test automation
solution for web application, mobile, and web services. Being built on top of
the Selenium and Appium frameworks, Katalon Studio takes advantage of these
solutions for integrated software automation.

The tool supports different levels of
testing skill set. Non-programmers can find it easy to start an automation
testing project (like using Object Spy to record test scripts) while
programmers and advanced automation testers can save time from building new
libraries and maintaining their scripts.

Katalon Studio can be integrated into CI/CD
processes and works well with popular tools in the QA process including qTest,
JIRA, Jenkins, and Git. It offers a nice feature called Katalon
Analytics which provides users comprehensive views of test
execution reports via dashboard including metrics, charts, and graphs.

IBM RFT is a data-driven testing platform
for functional and regression testing. It supports a wide range of application
such as .Net, Java, SAP, Flex, and Ajax. RFT uses Visual Basic .Net
and Java as scripting languages. RFT has a unique feature called Storyboard
testing in which users’ actions on AUT are recorded and visualized in a
storyboard format through application screenshots.

Another interesting feature of RFT is its
integration with IBM Jazz application lifecycle management systems such as IBM
Rational Team Concert and Rational Quality Manager.

TestComplete by SmartBear is a powerful
commercial testing tool for web, mobile, and desktop testing. TestComplete
supports various scripting languages such as JavaScript, VBScript, Python, and
C++Script. Like Katalon Studio, testers can perform keyword-driven and
data-driven testing with TestComplete. The tool also offers an easy-to-use
record and playback feature.

Like UTF, TestComplete’s GUI object
recognition capability can automatically detect and update UI objects which
helps reduce the effort to maintain test scripts when the AUT is changed. It
also integrates with Jenkins in a CI process.

An image-based automated functional testing
tool that enables testers to interact with AUT the same way end users do.
TestPlant eggPlant is completely different from traditional testing tools in
its approach: modeling user’s point of view rather instead of the test scripts
view often seen by testers. This allows testers with less programming skills to
learn and apply test automation intuitively. The tool supports various
platforms like Web, mobile, and POS systems. It offers lab management and CI
integration as well.

Tricentis Tosca is a model-based test
automation tool that provides quite a broad feature set for continuous testing
including dashboards, analytics, and integrations to support agile and DevOps
methodologies.

Tricentis Tosca helps users to optimize the
reusability of test assets. Like many other test automation tools, it supports
a wide range of technologies and applications such as web, mobile, and API.
Tricentis Tosca also has features for integration management, risk analysis,
and distributed execution.

Ranorex is a quite comprehensive commercial
automation tool for web, mobile, and desktop testing. The tool features
advanced capabilities for GUI recognition, reusable test scripts, and
record/playback. Codeless test creation is also a very useful feature that
allows new automation testers to learn and apply test automation to their
projects.

The tool supports Selenium integration for
web application testing. Testers can distribute the execution of their tests
across platforms and browsers using Selenium grid. Ranorex offers a low-pricing
model for businesses.

Robot Framework is an open-source automation
framework that implements the keyword-driven approach for acceptance testing
and acceptance test-driven development (ATDD). Robot Framework provides
frameworks for different test automation needs. But its test capability can be
further extended by implementing additional test libraries using Python and
Java. Selenium WebDriver is a popular external library used in Robot Framework.

Test engineers can leverage Robot Framework
as an automation framework for not only web testing but also for Android and
iOS test automation. Robot Framework can be easy to learn for testers who are
familiar with keyword-driven testing.