Testing mobile apps with SpecFlow and Appium

Just 10 years ago, a majority of users were surfing the internet on their PC. Today, according to statistics, most visits come from mobile versions of a website or mobile applications. Therefore, the requirements for the stability and quality of these applications are very high.

Usually the quality of the software being developed is monitored by the testers. They write test cases, do manual testing and report bugs. However, for applications in active development, there might be several boring, repetitive and monotonous tests. That kind of work has a high risk of human error. How can this problem be solved?

Test automation

The answer is test automation. Entrust the more complex and critical tasks that will be difficult for the computer to handle to your QA, and let the routine work be done by the machine.

First, you need to determine the technologies you are going to use for the tests. There are a lot of different frameworks for testing on the market. I would recommend these two: SpecFlow and Appium.

SpecFlow

SpecFlow – BDD framework

SpecFlow is an open source framework for behavior-driven development (BDD). It allows you to write human-readable scenarios for your tests, which can also be your living documentation. SpecFlow integrates with Visual Studio.

Appium

Mobile app automation framework

Appium is a test automation framework for use with mobile apps. It allows you to test native, hybrid and mobile web apps. It uses the WebDriver protocol for iOS, Android and Windows apps.

Creating a simple test automation project

Let’s take a closer look at these frameworks and create a simple project for testing Android applications.

* We will assume that you have already installed Appium and the Android SDK. You can read about the Appium setup here and the Android SDK here.

Open Visual Studio and create a new C# class library project. Call it “Mobile.UI.Tests.”

Now let’s add SpecFlow, NUnit and Appium from the NuGet package manager. Find and install the following libraries:

SpecFlow

SpecFlow

WebDriver

NUnit

When everything is done, our references should look like this:

Project references

We also need to install the NUnit Test Adapter and SpecFlow for Visual Studio extensions. They can be found in Tools -> Extensions and Updates -> Online -> Search for “SpecFlow for Visual Studio” and “NUnit Test Adapter.” After the extensions are installed, please restart your IDE.

Next, we have to edit our App.config. Open it and add the following app settings:

appActivity – defines the activity to be executed from the app package

appiumServerUrl – Appium server URL

timeOut – global timeout for different testing purposes (for example, waiting for an element to be visible on the screen)

Your values may differ. In my case, I’m going to run tests on an emulator with the name “emulator-5554” for a YouTube application starting on “HomeActivity,” and my Appium server is set to a local machine.

Now, open the file Default.srprofile (added after the SpecFlow package installation). Edit the “Execution” element. It should look like this:

As the next step, add a new folder called “Appium.” Inside that folder, add another folder called “Drivers.” Then create a C# class called AndroidDriverProvider. This class sets up and returns the driver for interacting with the UI of our Android application.

The MobileApplication class contains our driver and some UI interaction methods that wrap default driver functions like clicking buttons, typing text, waiting for element state, etc. Most of these methods use waiters. This is very useful because UI elements often appear after some delay. Lots of automation engineers are using Thread.Sleep() for these cases, which is a bad practice. Waiters are a good solution for that. They will wait for an element state and throw an exception if the state is not set after a given timeout. For a timeout, I prefer a global config value (“timeOut” in App.config) instead of having multiple constants. That makes the code easier to maintain.

All preparations are done; now we can create tests.

Creating a test scenario

Let’s add a test scenario. For that we need to create a folder called “Features.” Right-click it and select Add -> New Item -> Visual C# Items -> SpecFlow Feature File. Type “Demo” as the feature file name and save it.

* If SpecFlow templates can’t be found in the “New Item” window, please make sure you have installed the SpecFlow plugin for Visual Studio. Above there is a description of how to install the necessary Visual Studio extension.

In my scenarios, I try to avoid mentioning UI elements and instead describe them in terms of business logic. When you write something like “Select item from the dropdown,” there is a big possibility that the UI will be changed and the dropdown could be replaced with another control. In that case, you will have to rewrite your scenarios. Business logic changes less often. If the scenario describes business logic and the UI changes, there is no need to rewrite it – you’ll only have to change the code.

Writing a code for scenarios

Let’s make our scenario executable.

To do this, we need to create classes that will represent the screens (pages) of the application being tested. First, we create a base class from which we will inherit the rest.

As you can see, the BaseScreen class has a property App of type MobileApplication, which is a reference to our driver wrapper object (it contains methods for interacting with the UI). Method InitializeSession()creates an instance of AndroidDriver. DeleteSession() closes the Appium session.

Before we proceed with page classes, let’s check the workflow we are going to test.

As I mentioned before, we are going to test a YouTube application. The following actions must be performed:

Open the YouTube application.

Click on the search icon in the top right corner.

On the new screen in the search text box, type: “Robot Pepper Intern. Episode 01. Arrival.”

Click on the top video.

Check that video player is shown.

Home screen with search icon in top right corner

Screen with a search text box

Search result screen

Video player screen

So, we have four screens to interact with. Let’s start with the home screen.

This class inherits from BaseScreen, so it can use methods from MobileApplication to interact with the UI. Here we have a static, read-only property SearchIcon, which locates the search icon in the tested application. To click on that icon, there is the method ClickSearchIcon().

Like all the previous page classes, VideoScreen inherits from the BaseScreen. It also has an IsPlayerShown() method, which returns true if the video player is shown and false if not.

Our page classes are now implemented. Now we can add a “StepsDefinition” class to bind with our demo feature.

Create a “Steps” folder. Add a new item with the type “SpecFlow Step Definition” and name it “StepsDefinition.” Visual Studio will create a C# class with some binding for features. Let’s implement code for our given/when/then steps from the feature file.

Given user opens YouTube app – in this step we are going to create our web driver and open the YouTube application.

To run the tests, an Android emulator or real device is required with the YouTube application installed. In Visual Studio, go to Test -> Windows -> Test Explorer. Find a test in the opened window and run it. If everything is alright, then it should open the YouTube app on the emulator or your device, execute the tests and show the result. The test must be successful.

Test result in Visual Studio

Conclusion

In this article, we have implemented a very basic project for automating tests. It can be set to run tests in your CI or on the QAs’/developers’ machines as often as you need and for as many tests as you can develop. The results of the automated tests can help to monitor your mobile application’s stability and quality. QAs and developers can receive tests results, analyze them and work together to fix defects and make your application more stable.