Marcações

Agile Testing - Mobile application testing challenges

Agile Testing - Mobile application testing challenges

Several months ago, I was challenged by the Release Quality Manager, at a large financial institution, to demonstration the value agile on large projects where several of his development teams were struggling with quality with their mobile application (mobile app) development.

Here is a summary of the challenges, recommendations and outcomes from this engagement.

Current Situation

My customer was in the middle of an extensive agile transformation with many teams adopting a combination of Scrum / XP practices. Most of the teams selected were part of a single large program but much of their development was related to the front end of the business, ie., mobile applications, mobile web interfaces, etc…

Challenges

Testing became a huge challenge for mobile app teams. Unlike traditional development, mobile apps often had test matrices with hundreds and even thousands of permutations (mobile OS level, network carrier, locale, and device orientation) of supported device to test against.

In addition, testing mobile apps was expensive, time consuming and complex activity usually requiring the mobile team to test against a large set of back end services which might also be under development. Managing testing environments and test data also was an issue as functional and scenario based testing usually

Further, mobile apps were being held to a higher quality standard than traditional applications as they were frequently rated by their users for all to see. A bad app rating can reduce user base, deter new users for using it and be detrimental other products or services provided that the vendor.

Recommendations

Coming up with the right testing strategy, requires an assessment of the advantages (and disadvantages) found in each of the testing options, determining the technology that best suits your testing requirements and implementing it with the appropriate rigor.

Here are some of the key recommendations we gave the project team.

1. Use a combination of native and emulators to test.Emulators are more cost effective, because a single platform with different device profiles can be used to test most of the device on the market both today.

o During initial sprints, use an emulated environment to perform most of your testing to take advantage of the speed and device diversity that an emulator can provide.

o Test with real devices later in the sprint and for independent testing to help certify the applications works in a real world environment and to validate critical performance, safety and usability objectives have been met.

2. Automate wherever possible. Define a testing solution that supported script & playback functionality for the real and emulated devices to save time and allow you to execute more test cases. Implement a lightweight test management approach to quickly identify which test cases that must be executed along with an efficient bug tracking for the issues found during testing.

3. Implement lightweight testing strategyWhile there are a number of different approaches that could overcome these challenges, we tried to keep things simple so we started by splitting testing into Functional and Technical. The Functional testing focused on ensuring the expected business value was achieved and relied heavily on scenario / acceptance based testing of each story. This allowed the team to work with the Product Owner on test tasks that provide business benefit.

The Technical testing focused on verifying the technical requirements: things like security and performance criteria, common libraries to the backend and data retention rules.

#1 Manual Functional Testing of acceptance criteria. All of the speed and simplicity of user stories. An efficient communication mechanism for stakeholders.

Our "shift-left testing strategy” focused on moving issue resolution to the lowest cost level using automated testing to help identify issues at the closest sprint and to free-up expensive project resources to focus on implementing projects on-time, in a quality manner.

Summary

Mobile app testing challenges present you with tradeoffs that you should consider to make the right selection of testing methods and techniques to ensure you achieve the level of quality your users have come to expect. Most organizations must create / adapt their own strategy based on their unique situation. However, even in the best situation where the agile team is using TDD and have automated much of their unit and testing, the team cannot usually conduct full system integrations, large-scale performance, and security testing without the help. Effective test planning, service virtualization and test automation are usually required to be successful in the long run.

References:

To learn more about the challenges and various testing approaches read: