Testing Wimdu with Frank

A long, long time ago… we started to work on an iPhone app for the merchants of Wimdu (= our hosts). We want to provide them with a handy tool to manage all their bookings. We call it our “host app”.

While the project comes to the final stages of development we started to think about testing the application. It was clear from the very beginning, that we wanted to build automated tests to easily check every release for stability, quality and reliability.

We had a very specific set of requirements for our tests. We want our mobile app not to be a stand-alone or isolated platform, we will have interaction between the hosts (merchants) using the iPhone and our guests. Those interaction are mainly messages between the two parties as well as the actual bookings. If we want to simulate and test end-to-end user behaviour, we need to run simulations on our web- and mobile app simultaneous. The tests for the web app are written in Ruby, using Watir to hook into Selenium and drive Firefox. So we needed a system that could run the scripts for the web app simulation. Furthermore, we wanted to develop test driven, so we needed quite specific feature descriptions and scenarios for our test cases.

Naturally, the first step was to look into the UI Test Automation that is built into Xcode. Thinking, that as it’s fully integrated into Xcode and tthe perfect tool to work with, we tried to build some small test cases. The limitation of this tool is quite obvious: it can only test the iOS app. There would be a lot of manual steps involved to switch between the test cases of the iOS app and the web app. Hence, for this approach the build in tool box was not sufficient.

Amongst others we looked into Testing with Frank build by Thoughtworks. It was the best fit for our need. It uses the two languages we need, Cucumber to have a Product-Understandable side of the tests, where they can define and read test cases. On the other hand its build on Ruby. We love Ruby, so we tried it.

Frank is charming, what happens is pretty much explained in one picture:

How does Frank work?

In general, the first post that is recommended to get things going with Frank is this post by Peter Hodgson. And don’t forget to install the Xcode command line tools in case until now you did not.

However, after running frank setup we stumbled upon the first problem as we were using Cocoapods for dependency management in our projects. You can fix it by passing the workspace and scheme to Frank like:

If you build Frank in release mode, then the “Frank Driver” will not access the “Frank Serve” any more. Other than that the build will succeed and we can follow Peter’s blog to write our first tests.

Getting started with the Cucumber tests we quickly realized that it would be a good idea to strictly divide the two “languages” Ruby and Cucumber. It creates a bit of an overhead for smallest step definitions, but as soon as things get a little more complicated it helps to organize the code.

In Example:

Cucumber reflects the specification from the Product Manager.
It lives in the default folder features/request.feature

Scenario:
as a host, who has a booking request I want to see the request in the app with the guest name displayed
Given I have a booking request
Then I should see 1 request
And I should see the name of the guest

the step definitions are also in the default folder features/step_definitions/request_steps.rb. Here we decided to write as little Ruby as possible, to not have to places where the actual code is running. We do only require the according class:

require_relative "lib/requests"
requests = Requests.new
Given /^I have a booking request$/ do
requests.pending_booking
steps %{
Given I log into the app with user #{requests.host}
Given I touch "Requests"
}
end
And /^I should see the name of the guest$/ do
requests.find_guest_name
end

We tell the Ruby-Part of our tests to create a new booking request for a new host and gather information. We can also include other Cucumber steps if it fits the purpose of the scenario. The last step is now to write the actual test, it lives, as you can see already in features/step_definitions/lib/requests.rb:

All interaction with our web application is handled in the Webapp class. In this case a new host is created, a new offer for this host and a guest to book the offer. As we want to develop end-to-end user tests we do not create dummy entries in the API for the app but rather interact with both apps at once. The label we use in frankly_map is set in the Xcode project in the first place, so that the element is accessible for us in Frank.

How we work (with Watir) to write tests and simulations in our Webapp will be the next post to follow.