Menu

Manual testing is the process of finding defects in a product or application when the tester checks the operation of all components of the program as if he/she were a user. It has been considered as one of the traditional technology practices where the tester often uses predefined testing plan, which indicates the most important aspects of the application for the achievement of highest accuracy. Highly successful products hold a strict testing methodology in order to identify maximum number of bugs using rigorous manual testing practices.

Automated testing has become more and more popular in recent years and this looks like automated testing is dominating manual testing trade. As an ambitious future centric testing community, we will always be curious to know whether manual testing is dead and QA automation holds the future of the testing community.

Manual Tester’s Perspective:

Manual Testing is normally the first thing Organizations do to gain information they need from the system/application developed. Manual Testing is the starting point for any testing related initiatives. The more complex companies build the software, the more they need Manual Testing. Manual Testing will never die as it’s the first gate to software quality. It may be transformed into several new testing facets and techniques but Manual Testing will prevail for a very long time. Though industry trends point towards automated testing, it doesn’t make sense for short projects, because the expenses are too high. At the moment, testing geeks claim 100% automation cannot be done but may be in a decade or two manual testing will be taken over by machine learnings and automation testing. But, test designing cannot be taken over and automated. Simple reason – “If you do the same thing the same way and same number of times, it will stop finding issues”

Is the world ending? No one knows. Yet there are many prophecies.

Does coding happen by itself? No, it’s programmed by human.

Does a self-driving car drive itself? Yes indeed, but it’s configured and tested by humans.

So don’t worry, manual testing will never be dead!

Automation Tester’s Perspective:

Does Google do manual testing?

Is there any value for android manual testing?

Will bigger organizations hire laterals for manual testing?

Automation testing cannot be done before manual testing. Why is that all job openings are for Automation and hardly for Manual?

There’s a need for both manual and automated testing. But we should agree that the job market out there expects most testers to be able to do both these days.

The trend of test automation is changing exponentially and selenium is one that is growing along. In the current trend, the clients are not ready to wait n number of cycles to get the return on their investment. And again the most important fact is that, even as an automation test engineer, he/she should have enough domain knowledge on whatever application they are automating. Gone are those days of doing automation by just seeing the test steps. Automation engineers design sophisticated test frameworks and are skilled software developers. They automate, maintain, and run test cases.

These roles are bound to evolve with continual changes in the software landscape. As more enterprises increase automation rates, fewer test cases will be left for testers to run manually. As more manual testers learn to automate their tests, automation engineers will find that the focus of their work changes as well.

Manual testers and automation engineers will see their job descriptions changing and as a result, become more valuable to their respective companies more than ever before.

Chief Technology Officer’s Perspective: Should development and testing become one?

Many testers are really nervous about their jobs because they don’t know how to code or script. Automated software testing — a key part of agile software development — could easily put these non-programming testers out of a job. We’ve seen some companies that have eliminated software testing altogether because they’ve moved to Agile.

So is the answer turn testers into junior developers? A lot of technology experts started believing in the concept…

Conclusion: Software testers need to become more techie

Testers bring a unique mind set to the software development table. Creating software is a different technique than the critique that testers bring. Do testers need to get more techie? Yes, they do. But they don’t have to become full on programmers. Testers need to up their skills and be more aware of how development works how code functions, and know how the business functions to make money. Yes, they need better technical competence, but that’s only one out of several areas where they need to improve.”

Zado’s Perspective – Z-Automate

Are you worried about phasing out manual testers and hiring Test Automation specialists at a higher cost for Selenium? Do you want to implement test automation strategy with manual testers itself? Are you exploring the options to implement test automation strategy using script less automation strategy?

Z-automate is a selenium based product that exactly addresses the above mentioned questions:

54% of companies have adopted DevOps and the interest around DevOps is increasing rapidly – A research Report.

What is DevOps?

DevOps is a software development methodology which looks at integrating all the software development functions from development to operations within the same cycle. This methodology is an extension of Agile and it requires higher level of coordination across various stakeholders in the software development process (namely Development, QA & Operations)

DevOps Cycle:

DevOps impact on Software Testing:

DevOps encourages everyone to contribute across the chain. So it is important for the Chief Technology Officer to understand the impact of various roles that encompass Software Testing Team. Scenario 1: A developer can configure deployments. Scenario 2: Deployment engineers can add test cases to the QA repository. Scenario 3: QA Engineers can develop test automation scripts and configure their automation test cases into the DevOps chain. What it means is that every single person in the organization is responsible for quality and timeliness of deliverables.

DevOps and Test Automation

To achieve such speed and agility, it is important to automate all testing processes and implement specialized automation testing tools and continuous integration tools to make DevOps possible. This also necessitates building a mature automation testing framework through which one can quickly script new test cases.

In today’s world of Continuous Delivery, DevOps is a stage by stage process to be introduced and implemented within the organization. The best possible way is to start with the Testing and Test Automation process and extend it to the other departments.

Consider this approach:

Start by automating your application deployment to test environments. Take advantage of virtualization and the cloud, and deploy those test environments dynamically, without any hard-coding. The test environments are torn down at the end of the test. In order for this to be achievable, the team needs to collaborate around the application architecture to make it as-easy-as-possible to automate deployment.

Make sure you can run a test automatically first: Implement a strategy from the build – it can be installed, configured and the automated test runs before writing additional tests. Approach differently about what tests should be automated

Provide an automation framework that solves the test automation implementation timeline puzzle. Create a Test Automation framework that can provide reusable components for creating new scripts. An Automation framework that is well created contributes to pay off in the future.

Make it a team commitment that every code change is committed with a test to the framework. There will be no shortage of tests.

DevOps QA/ Test Automation Is About Preventing Defects, Not Finding Them

QA takes a critical role in DevOps organizational structure because they have the visibility and the directive to push code out when it is working, and roll it back when it is not. This is a very different mindset from QA organizations of 10 years ago, whose primary responsibilities involved finding bugs. Today QA groups are charged with preventing defects from reaching the public domain. This mindset will inspire technology organizations to go for DevOps and Test Automation together.

It is interesting to look at how the test automation industry grew over the past few years and where it’s heading in the Microservices & Mobile driven development. Based on the other key indicators, expert’s predictions and industry reports, here are the top 5 software testing & test automation trends that Zado thinks as the next generation trends for industries to watch out for.

Merger of QA and Developers Roles

Quality assurance is considered one of the most important steps in product development. As agile development became normalized, validation worked its way more closely into the actual coding phase using techniques such as Test Driven Development. With the emergence of DevOps, QA is experiencing the merger of development and IT Operations. The developers and operation managers have started adopting testing activity as a part of their day-to-day job. The experts predict that the roles of developers and testers will merge in the years to come.

Mobile Testing

Mobile application testing and test automation have been creating major impact in the software testing industry. The modern mobile apps need to deliver personalized and flawless user experience. It triggers organizations to strategize and implement a formal mobile testing strategy to enhance the test coverage and make the testing efforts uniform. However, mobile application testing is hugely impacted by the latest versions of mobile operating systems and rapid technological transformations. Open-source frameworks and Mobile DevOps will lead the expansion of mobile test automation landscape.

Testing of Big Data & Analytics

The Big Data has three unique characteristics—Volume, Velocity, and Variety. Today’s big data applications are growing dramatically and quality management for Big Data implementations will emerge. Devices, applications, and networks generate a huge source of unstructured and broken data ultimately leading to faulty and costly decisions if quality management is missing. As more and more data is being generated from multiple sources, Organizations will need to manage and aggregate all of the testing activities from one central quality management platform.

Micro Services Testing

There has been a huge shift from service based architectures to smaller and more focused “micro” services. There are many benefits with this approach such as the ability to independently deploy, scale and maintain each component and parallelize development across multiple teams. We will see organizations investing in Micro Services which makes it easy to test and deploy products & applications. They tend to look for approaches for managing the additional testing complexity of multiple independently deployable components and multiple teams acting as guardians for different services.

Selenium Automation

Selenium is skyrocketing in the automation tools industry since enterprises are increasingly using web applications. Supporting the onward rise of web and mobile app test automation, the industry will see increased use Selenium in the quest to lower costs while increasing efficiency and quality. Many tools and frameworks will increase their capability to cover more of the software development process from inception to production.

With this, Z hopes that software testing and test automation professionals seize the emerging opportunities in the global testing market, which is estimated to reach US $50 billion by 2020 by NASSCOM.

Test automation is not about removing manual testers but to make use of their time better. It is possible to automate testing completely, and it has to be a mix of expertise from manual testers and automation testers. If you have decided to automate testing, below are some practical tips for implementing any test automation initiative:

Focus on the Framework/methodology and not the tool

A clearly defined automation framework/methodology that covers how the automation process will be conducted can eliminate most of the frustrations associated with automation by providing stakeholders with an upfront understanding. This would provide a fair idea on what is needed to automate tests, which includes tool selection as well as the rest of the automation process.

Choose tools that are scalable to meet future needs

Ensure that the tools can be customized to your needs and address the following: reusability, scalability, maintainability, visibility, measurability and manageability. Also, look at the support each tool has for scripting in different languages, where there are strengths available internally. Otherwise, you will need to organize an appropriate program that trains the team on the selected tools.

Keep in mind that no single tool will satisfy all the requirements; so choose a tool that meets the majority of the evaluation criteria.

Perform a POC using the selected tool

Even if the tool appears to satisfy the evaluation criteria, it is advised to conduct a few test scenarios using the tool. Proof of concept (POC) should be done in such a way that the test scenarios cover an end to end business scenario, most of the controls and a few common features. POCs can be done with the evaluation versions of most tools and hence it is a fairly easier option to do.

Test automation should not lead test design

Test design must be kept independent of test automation and they form the input for the automation discipline and not the other way around. Essentially, quality is the prerogative of the QA function and it has to be in control without worrying about the automation function and the problems faced by the automation team.

Expect reasonable ROI

Test automation requires substantial investments upfront in terms of tool licenses(commercial) and creating automation scripts. It should be clear to all stakeholders that it will take time to see a defined return on these investments. However, there are intangibles such as improved quality and increased test coverage that can be observed in the short-term.

Look outside your organization for expertise

Instead of quickly building a team ground up, seek a 3rd party with the necessary expertise to manage automation for you initially. Through outsourcing, you can build capabilities internally over time and transition the testing in alignment with your overall organizational goals.

There are a lot of tools available for mobile application automated testing. In this blog post we will try to compare among three of them – Robotium, Calabash and Appium. Let’s look at them individually and find out what are their capabilities in terms of a set of parameters.

Appium: Appium provides an API based on Selenium’s WebDriver JSON wire protocol and supports platforms like Android, iOS and FirefoxOS. Since the standard automation APIs work on all the platforms, there is no need to recompile the app. Also, we can use any WebDriver compatible language like PHP, Java, Ruby, Python, C#, etc. to write the tests. We can use any testing framework. It actually provides a true cross-platform mobile automation.Calabash: Calabash is a free open source project which can help you run tests on iOS, Android and Simulator. It consists of two libraries – calabash android and calabash iOS, which helps in writing tests in domain specific languages. They enable test code to interact with the apps, wherein each of these interactions consists of a number of end user actions like gestures, assertions or screenshots. You can even test the HTML 5 part of the application using the cucumber interface.Robotium: Robotium supports both native and hybrid app through its Android test automation framework. It provides the test case developers an easier way to write functional, system and acceptance test scenarios, with minimal knowledge of the application under test in minimal time. Actually, the framework can handle multiple android activities automatically and can integrate smoothly with Maven or Ant for continuous integration. The test case execution is generally fast and more robust.
Apart from the above discussed tools, we have some major tools for mobile test automation like DevJuice, Eggplant, Frank, MonkeyTalk, ranorex, MonkeyRunner, etc. Let’s take a look at a comparison matrix of a few of them.

In most ISVs & Software Enabled Businesses, developers double up as testers and there is no formal testing process in place

QA is typically given a step-motherly treatment and is looked at only when schedule and time permits

This is most likely to result in increased rework, decreased customer satisfaction, decrease in profits, loss of business, increased bug fixing costs and product recalls.

Independent testing or 3rdparty testing, as it is famously called can help ISVs & Software Enabled Businesses in removing all the pains listed above.

Some of the reasons that substantiate usage of 3rdparty testing are:

Good software development practice calls for separating the development team from the testing team. This allows provision of unbiased view of the product quality and greater visibility of product issues. It also reduces direct staff costs, for the ISVs do not have to staff them internally

Testing lifecycle goes through staffing peaks and troughs and this becomes a huge challenge for in-house testing functions, whereas 3rdparty testing companies has that flexibility

ISV’s can save up to 40% in direct staff cost and around 20% in associated costs by outsourcing it to a 3rd party testing company

3rd party testing companies can help ISVs reduce their time to market with increased product quality, enhancing their competitive advantage

Software product testing is a specialist skill, and cannot be accomplished by the developer community. It is difficult for ISVs to allocate enough funds to internally hire expert resources or train their existing resources, resulting in compromised product quality

Partnership gives you time, flexibility, bandwidth and platform to thoroughly test your product(s)?

3rd party can bring IP in the form of test automation frameworks and the experience of having been there and done that for ISVs

I’d be interested in addressing any specific queries that any of the readers may have.

Ecommerce is heavily dependent on software applications and they form the critical core of the business. Also, the industry is so dynamic that, changes need to be implemented on a daily basis. In such a scenario, a manual testing process will lead to nowhere. In this age of omni-channel retailing, it’s a tough task to be consistent across all the touch points. Some of the most important areas to focus in ecommerce testing are:

Performance Testing

Security Testing

PCI compliance Testing

Load Testing

Usability Testing

Apart from these, considering the multiple touch points of the business model, we need to be consistent in the mobile store applications as well. Since, more customers are opting for a mobile app for the shopping portal; we need to provide an easy, secure and faultless functionality which is consistent over different mobile platforms.

“With this tough competition and the challenges you cannot just take a risk of manually testing your website and keeping your business at stake”.

So naturally, Test Automation becomes the preferred choice, where we need to tackle the challenges in a faster and more comprehensive way. Let’s take a look at how Test Automaton can help an ecommerce site:

Since a lot of changes come into play on a daily basis, we need to execute reusable test scenarios to accelerate test cycles. Automated testing provides you with the same.

Using the functional test case generator, we can build automated test cases and customized business process libraries in lesser time.

Privacy, access control, authentication, integrity and non-repudiation are big issues in ecommerce website. Test data management can play a highly effective role here.

We can make use of different automated frameworks to check cross browser compatibility.

Downtime is too expensive to tolerate w.r.t. ecommerce. Automated testing can help in faster performance measurement and take corrective and preventive measures.

It’s not feasible to take your website off. Test Automation can help in exercising a live site surveillance.

As a retailer you need to keep on checking what’s working for the competitors. You can check with different variations by using Split Testing to increase your engagement rates.

Are you confident that the tool you have selected to test your mobile software will support the mutations?

With the rising number of mobile application, software product companies face a provoking opportunity to magnify their market reach by offering mobile products.

Yet, the twin side of this opportunity is the danger associated with releasing a mobile product too soon, before all the major bugs have been discovered and fixed; today’s consumers are so demanding that a poor experience with mobile can often lead them to reject the entire brand.

This engenders tension within the mobile testing team. Naturally, mobile testing is vastly improved when you’re working with the best possible tools.

Follows the checklist of specific areas to review the mobile testing tool

Cost ‘keep you customer happy by keeping the cost low’ – Tool cost analysis includes maximizing automation (reducing costs), support across multiple platforms, script reusability, and total cost of ownership.

Hybrid tool combinations ‘you can mix and match devices and emulators to enhance runs on different platforms’

Integration with Management Tool – ‘you tool should speak easily to the management and the bug tracking tool’

Ease of Use, support from the vendor, reusability, cross-platform and cross-browser support are the other few criteria which should be considered.

Test automation is not about removing manual testers but to make use of their time better. It is possible to automate testing completely, and it has to be a mix of expertise from manual testers and automation testers. If you have decided to automate testing, below are some practical tips for implementing any test automation initiative:

Focus on the Framework/methodology and not the tool

A clearly defined automation framework/methodology that covers how the automation process will be conducted can eliminate most of the frustrations associated with automation by providing stakeholders with an upfront understanding. This would provide a fair idea on what is needed to automate tests, which includes tool selection as well as the rest of the automation process.

Choose tools that are scalable to meet future needs

Ensure that the tools can be customized to your needs and address the following: reusability, scalability, maintainability, visibility, measurability and manageability. Also, look at the support each tool has for scripting in different languages, where there are strengths available internally. Otherwise, you will need to organize an appropriate program that trains the team on the selected tools.

Keep in mind that no single tool will satisfy all the requirements; so choose a tool that meets the majority of the evaluation criteria.

Perform a POC using the selected tool

Even if the tool appears to satisfy the evaluation criteria, it is advised to conduct a few test scenarios using the tool. Proof of concept (POC) should be done in such a way that the test scenarios cover an end to end business scenario, most of the controls and a few common features. POCs can be done with the evaluation versions of most tools and hence it is a fairly easier option to do.

Test automation should not lead test design

Test design must be kept independent of test automation and they form the input for the automation discipline and not the other way around. Essentially, quality is the prerogative of the QA function and it has to be in control without worrying about the automation function and the problems faced by the automation team.

Expect reasonable ROI

Test automation requires substantial investments upfront in terms of tool licenses(commercial) and creating automation scripts. It should be clear to all stakeholders that it will take time to see a defined return on these investments. However, there are intangibles such as improved quality and increased test coverage that can be observed in the short-term.

Look outside your organization for expertise

Instead of quickly building a team ground up, seek a 3rd party with the necessary expertise to manage automation for you initially. Through outsourcing, you can build capabilities internally over time and transition the testing in alignment with your overall organizational goals.