FAQ For Software Testing and Web Application Testing Services

Feel free to browse our list of Frequently Asked Questions and if you don’t find what you’re looking for, please let us know and we’ll schedule
a convenient time to talk and answer all of your questions.

In our 12-year history, we’ve had a lot of questions about what we do and how we work. We prefer to talk in person about your specific needs but realize not everyone has time for a phone call.

Offshore software testing teams and companies can provide a number of different types of software testing. Everything from manual functional software testing through complex test automation and performance testing. A good rule of thumb when using offshore software testing teams to provide any type of software testing is to have a very clear understanding of the outcome you expect, the time duration in which to deliver the expected outcome and any process or communication constraints that are needed to work within your software development process. Clear expectations and requirements coupled with frequent and effective communication within your existing process can overcome almost all of the problems and hurdles associated with getting the best types of software testing from an offshore software testing team.

Software Testing Solutions Provided via Offshore Software Testing

Offshore software testing teams can provide software testing services or software testing solutions. They are different! Software testing services generally has a focus on the activities and is typically used to expand the coverage of an existing software testing team. Software testing solutions are focused on the outcomes (and sometimes these outcomes can be services) where the “how” of delivering the solution is less important than the actual outcome. We find it very valuable to clearly understand how our clients want us to perform – software testing solutions or offshore software testing services.

What are Common Pitfalls with Traditional Offshore Software Testing?

The traditional offshore software testing model has many pitfalls. Offshore software testing and offshore IT services business models are well over 30 years old. Some aspects have changed with time but a number of elements haven’t changed since offshoring technical services was started many decades ago. The most common pitfall we see in offshore software testing is the “throw it over the wall” approach. This approach worked, most of the time, when the current development methodology was SDLC-70 – not so much anymore. The second biggest pitfall we see is the use of a “utility IT technician” rather than a specialist – particularly in the area of software testing. It should be easy enough that anyone can do it, right? Yeah, not so much from what we’ve seen.

Smarter Offshore Software Testing

We practice a smarter version of offshore software testing that addresses a number of aspects of matching our offshore testing service and software testing consulting with current and modern software development methodologies. In particular, we don’t allow a “throw it over the wall” approach. At a minimum, you’ll see and hear us communicate our status, plans and activities at least twice daily. Additionally, we assign a dedicated QA Manager to each project and account to make sure that our Test Engineers are engineering tests and that all of the management activities are taken care of by an actual manager. We attract, select and train specialized Test Engineers – people who love to find defects and then get them fixed! The difference between a TESTCo Test Engineer and a “utility IT technician” is like the difference between a friend who’s been sick and a real doctor – who do you want to talk with when you really need help?

Is Web Application Testing Different?

Testing web applications is very different than testing enterprise software applications. While the basic software testing procedures is different, the focus, order and contingencies are quite different. Enterprise software applications tend to have a greater depth and lighter set of constraints on the test cases and test plans. Testing web apps is typically much broader and shallower in scope and focus. Additionally, we see that the initial quality requirements for testing web apps is typically much higher on when testing web applications than when testing enterprise software. Here’s why. Most web apps have a much broader audience than enterprise software apps. Broader audiences tend to mean a lower user tolerance for defects and a higher and quicker abandonment rate. Enterprise software is typically captive to a well defined audience and the hard requirement to use the application results in a greater tolerance for defects.

Web Testing Covers a Lot of Ground

Web testing also tends to cover more software testing ground than enterprise software testing. While web apps tend to be broader and shallower than enterprise software, the potential for internet-based performance issues are greater. Web testing, specifically beyond functional and regression testing, tends to include much more performance and connectivity related testing than enterprise software. Enterprise software typically runs on a captive network where web testing uses the publicly available internet and is subject to many more potential disruptions that can appear to be defects. Web testing for the “out of bounds” performance and connectivity related issues, while not necessarily defects, is something that can dramatically increase the adoption rate of a web application and reduce user abandonment issues that can be perceived as defects in the web application.

What are the Advantages of a Dedicated Outsourced Software Testing Company?

It’s easy to think that almost any technical services company could provide software testing – even something as simple as software regression testing from a set of test cases and a test plan that is already written. The test cases just need to be run, right? We think there is a difference in a company that ONLY provides software testing and the other companies that claim to be a software testing company and provide a host of other technical services. We believe that the primary difference is our people and our talent. Since we ONLY provide software testing, we attract and screen for the very best software testing talent – and there is a difference between a software test engineer working for a software testing company and a general software engineer who works for a company with a broad service offering.

A Regression Testing Example

For an example using software regression testing – a typical tester in a broad technical services company will run the test cases and report the defects – exactly as you’ve asked. A test engineer working at TESTCo performing software regression testing will also run those test case according to the test plan AND will offer just a bit more. See if these might be valuable to you – the count and percentage of test cases run, passed, blocked and with defects; notes in the test cases where additional data sets might be valuable for boundary testing; timings on test case execution so you can estimate more accurately next time; daily status and production reports that provide real time adjustment of our software regression testing process to your changing needs; a dedicated Software Quality Assurance Manager who is responsible for making sure that the test engineer performing the software regression testing has everything they need to deliver according to exactly what you need. We think there is a difference between a dedicated software testing company and our competitors who offer a broad range of technical services, including software quality assurance.

How To Get Started with Outsourced Software Testing?

According to an article in IT Pro/silicon.com, one in three offshore or outsourced software testing projects fail. Research firm Ventoro found that “the main cause of offshore testing failures is a lack of preparation and poor execution by the user organization.” Ventoro went on to conclude that some offshore software testing companies can try to “steamroller the initial engagement process” in a rush to get the contract signed. MIT reports that only 37% of offshore testing co-sourced projects succeed and 50% of offshore software testing strategic partnerships fail. How do you like those odds? In the event that you do fail, where is the flaming finger of blame going to point? If your company depends on software for either new revenue or operational efficiency and productivity then it seems like making this type of bet on outsourced software testing and development is overly risky.

Here are a few things you can do to improve the odds in your favor and minimize your risk:

Start with the best outsourced software testing company you can find

Focus on the lowest risk area

Demand daily deliverables

Insist on real value

Use a scorecard

Report and adjust every week

Why Do Offshore Software Testing Project Fail?

Gartner Research and Diamond Cluster International reported in May, 2005 in “Five Reasons Why Offshore Software Testing Deals Fail” that the most common reasons for failed offshore software testing projects were:

Furthermore, they outline how these offshore software testing issues can be avoided through appropriate strategy, advance planning and expectation management with your software testing company.

It’s a good read if you’re the CIO of a Fortune 500 Company. But, most of us aren’t and very few of us have the resources to implement the strategic planning, training and process maturity recommendations made in the report.

Additionally, most of their recommendations involve lowering your expectations and lengthening your timeframe when it comes to offshore software testing services.

Most of us are unwilling to compromise our standards so easily. Few of us have the time or interest to sort out the theories from practical advice we can implement today with our outsourced software testing.

The real truth about why most offshore software testing projects fail is seldom discussed. It’s one of the “dirty secrets” of offshoring.

Knowing this in advance will allow you to correct it or avoid it completely.

Offshore software testing projects fail primarily because of a lack of expertise and experience working with an outsourced software testing company. This is true for both U.S.-based companies as well as offshore software testing companies. Most offshore software testing companies have very little experience delivering successful projects.

This problem is compounded by the antiquated and overly simplistic approach that most customers and vendors use to measure success – “on time and on budget” when it comes to offshore software testing services.

When an offshore testing project fails, the cost is typically greater than the cost of the original problem. Software development slows down, customer satisfaction declines, costs increase and frustration mounts.

When do offshore software testing projects fail? Right from the start – when you select a software testing services partner, not when they actually begin the work.

If an offshore software testing company is not making your business better, then they are either pulling it down or doing something that you could just as easily do yourself. By the way, the same goes for nearshoring as well!

The chain of events that occur when an offshore testing project fails is long and the damage extensive. And, that’s only the tip of the iceberg. Knowing exactly what to look for will give you the ability to act quickly to halt the damage.

To be successful in today’s demanding business environment, you must have the ability to respond quickly to changing market conditions and customer demand.

The rapid pace of the market and increasing customer expectations allows very little room for failures of any size.

A failed offshore software testing project, even a small one, can have a distinct and measurable impact on your current and future business success.

Rather than define success in the traditional manner of “on time and on budget”, we recommend using a more meaningful definition.

Success in an offshore testing project is best defined as:

delivery of value above and beyond the cost of the project AND

complete satisfaction with the process and results AND

repeatable and dependable systems for producing the same results again.

That’s a tough set of criteria. But, you expect that from yourself and your team so why would you expect any less from your offshore software testing company?

The best measures of success for any software quality project are increased revenue, higher customer satisfaction and shortened delivery cycles.

Ultimately, the best and most long-lasting benefit is a process that becomes a stepping stone upon which you can build additional business capacity.

Do you still have questions? That’s alright. TESTCo provides multiple reports and white papers to help you streamline your outsourcing, maximize your efforts, and ensure the efficiency and effectiveness of your business operations.

What are the Challenges to Web Application Testing?

Web applications have unique testing challenges. The problem is that they only reveal themselves in contrast to traditional software testing. It is easy to assume that testing a web application is just like testing an enterprise installed software application. But, that would be wrong. And, you probably wouldn’t know it was wrong unless you had spent some time testing traditional enterprise software applications. In fact, it is only by comparing and distinguishing the differences that you come to appreciate how very different they are.

The Issues Unique to Web Application Testing

Web application testing takes its roots from traditional software testing. But it does require a slightly different focus to be truly effective. Just like driving a vehicle is largely the same, driving a car down the highway is very different than driving a loaded tractor down the side of the highway. You have different areas that you pay attention to so that you get to your destination safely.

Web application testing is broader where software application testing is deeper. This requires that the QA strategy and test plans intentionally look to the broad set of border cases rather than the depth of calculation validation.

Web applications work in uncontrolled environments and software applications work within a more controlled environment. Complete test coverage across all possible environments is not possible. A smart approach to environment and platform testing is needed to ensure that your quality goals are met.

Web applications fail primarily with usability issues where software applications tend to fail with defects. Usability is the prime determinant of most web application success – not defects. Most web applications begin to fail to deliver the customers expected value when they contain too much functionality condensed too heavily into a few screens. Additionally, a seldom considered aspect of web applications is that they are primarily used as convenience tools rather than work tools. This has a big impact on how the test plan is created.

It is only through knowing these differences that you can make a smart decision about how to build a test strategy and plan that avoids the following consequences.

The Consequences of Not Knowing the Differences

The consequences of failing to acknowledge the difference between testing web applications and testing traditional software can be subtle and vexing. Without acknowledging the differences while building a test strategy and plan, you may find that you see these types of problems as you move your software from development into production.

Defects are discovered by customers and are very hard to reproduce. This results in a very expensive process of handing defect reports, calming customers and detailed research to reproduce the defect. Even security defects become a challenge.

Testing takes too long! This produces a large backlog, slower releases, fewer features and getting beat by your competitors.

Web application quality is high but uptake and usage is low. This consequence is the most costly as large initial investments in software and and marketing are wasted as users find themselves confused with the web application.

These consequences are not absolute nor are they insurmountable. In fact, the actions and solutions to avoid the challenges with web application are simple if considered early enough during the development of the test strategy and plan.

Web Application Testing Solutions

Knowing the differences between traditional software testing and web application testing is only the start. In order to avoid the problems outlined above, you’ll need to consider each of the following aspects during test strategy and planning.

Know and understand the testing boundaries. Boundaries can be found in functionality, networks, operating systems, bandwidth, geographic location and even a person’s age! Establish boundaries and then re-evaluate them after each web application testing cycle. They will change!

Sample and measure the environments – then choose smartly. No testing plan can cover every environment. Not every environment is measured the same. Begin early efforts to measure the impact of the web application being tested within each selected environment.

Know and measure the value the user expects from the application. Users use web application with an expectation of receiving some sort of value. Know that value and test the delivery of that value regularly during the testing process.

Most importantly, consistently acting on these solutions requires testing talent. This requires more than just write test cases, run test cases. Taking advantage of these simple solutions requires an experienced Test Engineer supported by an experienced QA Manager.