Mobile website test automation with cucumber but without the hassle

Introduction

In the test automation world the client often requests to run the tests on mobile devices, next to doing the testing on regular desktop. When running your automation in Cucumber, you have multiple possibilities to solve this. In this article I will talk about these possibilities and a new (simple) way to do it with Lapis Lazuli, a Ruby Gem that cooperates with Selenium Webdriver.

I will talk about the following solutions:

Lapis Lazuli device simulation

Using Appium

Using Browserstack

Using a phone via USB.

Lapis Lazuli device simulation

With the latest release of Lapis Lazuli (version 1.1.0), it is now possible to simulate a mobile device by providing:

Device name

User-agent

Screen width

Screen height

When starting your cucumber script, all you need to do is add DEVICE=iphone5. This will start the browser (Chrome or Firefox) with the dimensions and screen size of the defined device.

Cons

Appium

Another possibility is using Appium. This is a tool that allows you to connect to an emulated device, install or start an application and use it.

Pros

Uses the actual mobile device

Can also be used in combination with Cucumber

A free solution

Cons

Does not work with existing TA scripts

Is more focussed on testing apps, not websites

Difficult to set up in a Continuous Integration environment

Browserstack

This is a service that allows you to emulate any device. Browserstack also allows tools to connect to it and run automated scripts.

Pros

Reasonably easy to set up

Close to testing with the real device (emulation)

Usable in Continuous Integration

Cons

Not a free solution

Because your scripts are running externally, debugging is difficult

Remote testing increases the chance of invalid failure due to latency

Using mobile devices via USB

You could also connect a mobile device to your computer and run your scripts on the mobile device using Google Chrome. To be able to do this you will need to install the device drivers from Android Studio

I did not research a solution to do this with iOS, but most likely, it is only possible if you own a OS X computer.

Cons

Summary

Chrome uses the same version for mobile and for desktop. Because Chrome has become the most common browser for mobile devices, a simulation like Lapis Lazuli offers, is now quite reliable.

For obvious reasons, Lapis Lazuli calls the mobile web testing “Device Simulation”. It is a solid and simple way to test websites for their mobile capabilities. Unfortunately, it is not the real thing. If using real devices is a priority for you and you want it to fit in your Continuous Integration, I would recommend using Browserstack. If you are only running your tests locally, I would recommend running your tests via USB.

Our solution for Continuous Integration, called Calliope, also supports remote browser testing. This function is still in beta testing, but soon you will be able to link your Browserstack account to your Test Automation solution and run all your scrips on any Browserstack device.