A Guide to Automate UI testing for Phonegap / Cordova apps

Since our recent article providing automation examples for interacting with webviews was covered by Phonegap, we felt a more thorough guide on how to better automate UI testing might further help the community to leverage mobile test automation for their projects.

This time around, our guide highlights the role of mobile test automation for Phonegap & Cordova hybrid apps. We will look at the role of test automation within such apps, and the motivation driving many more agile teams to adopt test automation. We will also share previous tutorials that will help you to set up your first test case.

Why do we want to automate our Phonegap app?

As part of practising an agile development process, we need the capacity to execute tests fast, so that the developer can, in case of a bug or regression, react quickly. Manual testing capacity reaches its limit quickly when it comes to regressions suites that can be in the hundreds or, dependent on the complexity of your app(or app portfolio), thousands of test cases. Besides executing test cases quickly, mobile teams are ever more aware of device fragmentation, meaning what works perfectly on device of “Manufacturer A” doesn’t particularly mean it works on the device of “Manufacturer B”. With over 24 thousand devices available, it is increasingly difficult to cover a good cross-section of devices. The above challenges and many more provided the impetus for Phonegap (and many other developers) to adopt automated testing sooner, rather than later. Additional testing challenges specifically for hybrid apps are:

Fonts are not embedded on particular Android versions

Pixel density differences across certain devices

CSS Rendering issues, for example:

Sidebar is not on the screen

Elements are cut off, such as:

Rows, columns or headers of tables are cut off

Crashes, primarily due to memory leaks

Buttons that do not work

Increased loading time on specific devices

Obviously not all of these issues can be caught by automated testing. For several of them it’s important to have a setup that can take screenshots for each test step, so that a you can manually spot potential issues. You could also consider using our open source image comparison feature as a way to detect differences automatically.

testmunk dashboard showing screenshots

How to get started automating Phonegap / Cordova apps:

In the following paragraphs, we will highlight the logical steps required to get started with automating your app. This is not referring to unit testing or code testing, but rather automated testing of the functional UI of your app.

Testmunk helps to ensure test cases are made simple to read and to implement via the use of Cucumber and the Calabash testing framework. Calabash is an open source product, and has a great community for both iOS and Android, contributing to its success. Testmunk has contributed to this community by offering tutorials and best practices, and by testing the framework.

Installation:

Calabash supports both iOS and Android devices. However, if your organization has both iOS and Android apps, and you wish to decide which to test the platform against first, we recommend Android. Android is a tiny bit simpler to implement, as Apple forces you to deal with provisioned test devices, amongst other minor hurdles. For that reason, we will primarily focus on Android instructions and code examples in the sections below.

For Android, you will need an .apk file with debug access enabled. Follow the instructions linked below.

Querying elements

Regardless which option you choose, the above installation will walk you through the installation of a calabash ruby gem, which is needed to script and execute tests. Once installed, one of the first things that you need to do, prior to scripting, is to identify the elements that can be interacted with within your app’s UI view.

A practical way of inspecting elements and executing queries is the ‘calabash console’. The console is a runtime session within your app that you can run commands against right from your terminal. Querying the app in this manner is done to identify the elements and IDs that we will later need in order to put the tests together.

After the installation procedure, and installing the Android .apk file on your device, you can start the console by opening a terminal window and executing:

Scripting Automated Testcases

The testcase structure consists of 2 layers. Layer 1 is the human readable ‘cucumber’ language, which allows for both technical and non-technical stakeholders to understand and cooperate in test creation. A cucumber teststep triggers actions defined by the ‘Ruby’ language.

In order to get a sense of what can be done, and the format typically seen, we recommend generating a sample testcase skeleton by executing the command:

calabash-android gen

Cucumber steps are defined in the .feature file, whereas the ruby implementation is defined in the .rb file. Ruby is a little more difficult to understand initially, but when laid out alongside it’s Cucumber teststep, it becomes infinitely more clear.

Execution of your first test

After saving your .rb and .feature file, and after installing the app file on your device, you will be able to start a testrun session simply by executing the following terminal command from your project folder, on the same level, where your .feature folder is located:

$ calabash-android run sample.apk

In your terminal session, you should at this point see the terminal session indicating the step that is being executed. On the device being tested, the actions specified will be performed on your app.

A few words on Cross Platform testing

The obvious beauty of hybrid frameworks such as Phonegap/Cordova is that you can rely on the same codebase for both iOS and Android. This benefit also relates to the test suite. In traditional native app development projects, tests must typically be maintained in two different repositories. For hybrid app projects, one testsuite can be used to execute on both platforms. This does not mean that every calabash/ruby instruction will be the same across platforms. For example, Android often uses the ‘hard press’ back button on the hardware, whereas iOS developers more commonly use the ‘back button’ that is part of the UI itself. However, I think everyone will agree that writing a few custom teststeps for one platform is infinitely preferable to writing completely separate tests for each.

A more in depth article on how to structure ‘cross-platform’ test suites can be found here. However, we recommend waiting before diving into this until you have the first 10-20 tests behind you, and have become more familiar with the general nature of automated app testing. You may be interested in our blogpost on the automated testing of PhoneGap/Cordova apps, or our blogpost on testing webviews using Calabash.

About the author:Martin Poschenrieder has been working in the mobile industry for most of the past decade. He began his career as an intern for one of the few German handset manufacturers, years before Android and iPhone were launched. After involvement with several app projects, he soon realized that one of the biggest pain-points in development was mobile app testing. In order to ease this pain, he started Testmunk. Testmunk is based in Silicon Valley, and provides automated app testing over the cloud.Follow Martin on twitter