e2e testing hybrid apps

By definition, end-to-end testing is meant to assure that the whole integrated system operates correctly. For hybrid app developers, who run JavaScript apps in native-app context, using WebViews, that context should be a part of the tests.

My use case was an AngularJS app running inside a complex, native iOS app. I wanted to achieve full automation.

My goals were:

allow to run the e2e tests in iOS, which includes navigation in the native context,

allow to run the e2e tests in the browser too,

allow to run single test suites,

allow to run all e2e tests,

use it with a single command.

An overview

To achieve the above, I used the WD driver for node.js, Mocha, Appium, and gulp. At first, I first wanted to go with Protractor, but it does not have support for switching contexts from native app to web view, and I ran into issues with the setup. From a perspective, it does seem to be possible to use Protractor, by utilising the library wd-bridge and Protractor’s onPrepare() function, as described here https://github.com/angular/protractor/blob/master/docs/mobile-setup.md#setting-up-protractor-with-appium---iossafari. However, my setup has an advantage — due to not using Protractor, it is not limited to AngularJS, nor any other framework.

The initial test file

The simplest test file will include both the environment setup, and the specs.
The environment setup will be declared in a before() block. That means, that it will be run before all it() blocks, which are in the contained within the same scope. It is important to note that code outside of it() blocks will NOT wait for before() to finish.

The decision which environment to setup — desktop or iOS — we base on an environment variable, in this case _IOSTEST . It is a standard approach, and allows the test to be initiated in multiple ways.

Run the test

To run the test, we need to install mocha and selenium. I use webdriver-manager app, which is bundled with protractor. We also need the libraries used directly in the test file: wd (a node.js client for webdriver), chai, and chai-as-promised

npm install -g mocha
npm install -g protractor

and:

npm install

if you use my example code, otherwise manually:

npm install wd
npm install chai
npm install chai-as-promised

Now let’s start protractor in a terminal window, and let the process live.

webdriver-manager update
webdriver-manager start

Now, the actual test (in the browser).

mocha ./article-steps/initial-test-file.js

Ok, that works! So now let’s test it on iOS!

testing on iOS

We need to install Appium first, when that’s done, we also need to ensure there are no blockers for Appium, and grant access to the simulators. Blissfully, there are simple commands for both actions

npm install -g appium
appium-doctor —ios
sudo authorise_ios

IMPORTANT! Do not install appium with sudo, it will not work correctly.

Now that we have Appium, let’s start it in the same way that we did with webdriver - that is in a new terminal, and let’s let the process live.

appium

Ok. This is it! Let’s run the test in iOS!
(you need to have your own app)

_IOSTEST=1 mocha test-e2e/main/main.js

cool.

Let’s automate it!

Let’s write the gulpfile.js first, so we can see what we need, and install the dependencies next.

The tasks in gulpfile, gulp test-e2e and gulp test-ios, will start webdriver or appium, respectively, unless there already is a session running. The output of those commands is silent, to not pollute the stdout. Ok, let’s run it!

gulp test-ios

So now, we have a test which run against iOS, with a single command.

Separating concerns

The test code is not very DRY. The environment setup part should be separated. Still, we want to be able to execute our test suites in the same way.

We need to modify our test files, to separate module tests from environment setup. Keep in mind, that module tests are ran last. The module test I put in /test-e2e/main/main.test.js:

Running test suites

But one test is not enough. We will want to have multiple tests. It would also be great to allow to run single tests, as well as all of them in one go. We need a form of test suites. My approach to that with protractor was using suites, but here, I rely globs and directory structure. I basically have it like this:

Follow

Webby Awards 2015

Hi, my name is Michal

I am a JavaScript, Python and Go developer.

I work as a full-stack software engineer for Toptal and Airnauts, and also through my IT consultancy company, Trewebs.
Through Trewebs, I collaborate in a team with talented developers, UI designers, artists and photographers.

You can contact me at michal@trewebs.com, or through any of the social media links listed above.