Given the wide range of devices on the market and devices being released, testing your Apps has become extremely important. This is an area where functionality and usability may garner low reviews on an app store, but a single defect can result in your app being uninstalled. Careful attention has to be made in your testing plans and quality assurance. The following link covers many of the topics that need to be addressed in general, such as, identify your environment, define test cases, types of testing, assumptions, customer involvement, etc. Also discussed are tools to help in the testing effort. Internal tools, like Hobbes, can help with web-based UI testing. Tough Day can stress your instances with a simulated load. If your testing environment already has experience with 3rd-party tools, like Selenium, these too can be used. Additional tools are listed here that can assist with issues specific to mobile testing.

When developing a mobile app, there are many new concerns specific to devices that have to be addressed along with those of traditional testing.

Functional – Are all requirements met by your app?

Usability – Is the app easy to use and understand by your customer?

Performance – What happens during a spike in usage? Are the app elements, like swipes and carousels, quick and don’t detract away from the experience?

Failure or Interrupts – What happens when there is an incoming call or notification while your app is running? What happens if there is a network outage or power off?

Installation and Updates – How is the install experience? How are updates pushed out?

Technical – Is your app consuming too much power from a device?

Localization – Are all areas in your app translated?

Certification – Has your app been certified? Can customers trust that it follows all data privacy legal requirements?

These questions should be answered during your automated and manual testing.

Automated testing

Some degree of automated testing should be performed to cover the variety of screen sizes, memory constraints, input methods, and operating systems. Not only does it cover much of the test cases, but it can speed up regression testing when new features or devices are introduced. Ideally, your automation tools should reduce or limit duplication of effort. Use tools or frameworks so that your testing effort is applicable across all platforms. The following chart shows a simplified structure of a testing environment for both web-based UI testing and mobile app testing. The left side of the chart shows a series of Selenium nodes with browsers. SeleniumGrid can farm out common, web-based UI tests to any of these nodes. The Selenium hub can also connect to Appium for cross-platform app testing. Only shown are simulators, but you can incorporate adb, for Android, and Xcode utilities for iOS devices. Links are provided later in this document where you can find specific details for the tools mentioned.

Manual testing

In addition to automated testing, your app should go through a cycle of manual testing. Customers running the app on a real device cannot be duplicated by a script. Here too, you have many options. You can use a platform, such as HockeyApp, to define who has access and gather feedback. Or, you can outsource the whole process to a service like UTest, ElusiveStars, or Testin. If you have a group of internal testers, but lack variation of devices, there are cloud services where you can perform manual testing on their pool of devices. One such service that provides this is SauceLabs. You can also build apps remotely to PhoneGap Enterprise and install on local devices as a level of acceptance testing or demoing. See the PhoneGap website for their latest features and documentation. Whatever the approach, manual testing should;

hit a large target of testers,

test against a large pool of devices (ideally real devices, but simulators/emulators if real devices are not available),

provide informative feedback:

crash reports,

analytics/tracking,

usability,

areas of attention,

performance,

data/power consumption, etc.

Tools

There is a wide range of tools available for testing mobile apps. The choice of ones to use will be based on your specific situation: features, price, support, coverage, etc. The following is just a small description of some of the tools and services available.

Selenium

Framework that includes an API for test scripts to feed WebDriver and control various browsers.

You can use this in conjunction with Appium for testing on real devices.

Although not a testing tool, Jenkins is a continuous integration framework providing the backbone for automated tests. Numerous 3rd-party plugins are available to extend functionality. One example, the SeleniumGrid plugin provides an UI to help manage the Selenium hub and nodes.