Top 5 iOS Testing Frameworks

As the market matures in the mobile app space, iOS application developers have a multitude of iOS testing frameworks to leverage. Let’s go over some of the most popular frameworks (in alphabetical order), their strengths, features, test coverage options, and how they are licensed.

Appium

From appium.io:

“Appium is an open-source tool for automating native, mobile web, and hybrid applications on iOS and Android platforms. Native apps are those written using the iOS, Android, or Windows SDKs. Mobile web apps are web apps accessed using a mobile browser (Appium supports Safari on iOS and Chrome, or the built-in “Browser” app on Android). Hybrid apps have a wrapper around a “webview”—a native control that enables interaction with web content.”

Sauce Labs is very active in the development of Appium. Jonathan Lipps, Director of Open Source at Sauce, is the Appium project lead, and a number of Sauce Labs developers frequently contribute to the project. Your Appium tests can run across emulators, simulators and real devices on the Sauce Labs platform.

Strengths

You can choose your own underpinning framework, like XCTest, for unit testing.

Cross-platform support allows for reuse of test scenarios across mobile and web channels.

You can write tests with your favorite tools using any WebDriver-compatible language such as Java, Objective-C, and JavaScript.

Apple Xcode Test

From developer.apple.com:
“Use the XCTest framework to write unit tests for your Xcode projects that integrate seamlessly with Xcode’s testing workflow. Tests assert that certain conditions are satisfied during code execution, and record test failures (with optional messages) if those conditions are not satisfied. Tests can also measure the performance of blocks of code to check for performance regressions, and can interact with an application’s UI to validate user interaction flows.”

Strengths

Tooling is incredibly easy to embed in the application

Provides unit, UI, and performance testing

Uses Objective-C and Swift so developers can use their existing skillset

Can run tests automatically as a “bot” in the CI/CD process on Xcode Server

Calabash

From calaba.sh:
“Calabash enables you to write and execute automated acceptance tests of mobile apps. It’s cross-platform, supporting Android and iOS-native apps. It’s also open-sourced and free, developed and maintained by Xamarin.”

(Editor's Note: Xamarin announced that they have discontinued active development of Calabash as of April, 2017).

Strengths

Uses Cucumber to describe the test scenarios using BDD. This is a great feature for QA teams because it makes information clear and easy to digest.

Cross-platform support allows for reuse of test scenarios across mobile and web channels.

Sample Cucumber Feature

Feature: Refund item
Scenario: Customer returns a faulty microwave
Given Customer has bought a microwave for $100
And she has a receipt
When she returns the microwave
Then Customer should be refunded $100

EarlGrey

From the Google Open Source Blog:
“Brewing for quite some time, we are excited to announce EarlGrey, a functional UI iOS testing framework. Several Google apps like YouTube, Google Calendar, Google Photos, Google Translate, Google Play Music and many more have successfully adopted the framework for their functional testing needs.”

Strengths

Easy to include in an iOS project, either directly or using CacaoPods

A flexible framework, with powerful synchronization features across internal components

OCMock

Homepage: ocmock.org
OCMock provides a framework to create stub objects in your iOS app. It comes in two options—a static library that is used for iOS development, and a framework used for OS X development.

Conclusion: Which iOS Testing Framework Should I Use?

Based on your needs and experience with other languages, the iOS testing framework to select may be obvious. Or you may need to try all the frameworks listed to see which one works best for you.

Either way, at this point, the market is mature enough that there is a framework out there that fits the needs of almost any developer, and serves a variety of tasks—whether you simply need to automate testing service calls in iOS, or want to run UI tests in parallel across multiple mobile and web platforms.

Vince Power is an Enterprise Architect at Medavie Blue Cross. His focus is on cloud adoption and technology planning in key areas like core computing (IaaS), identity and access management, application platforms (PaaS), and continuous delivery.