Updated on 2013/04/08: Added command-line for Xcode >= 4.5, how to take screenshots and launch local scripts.

Quick introduction

Automated tests are very useful to test your app “while you sleep”. It enables you to quickly track regressions and performance issues, and also develop new features without worrying to break your app.

Since iOS 4.0, Apple has released a framework called UIAutomation, which can be used to perform automated tests on real devices and on the iPhone Simulator. The documentation on UIAutomation is quite small and there is not a lot of resources on the web. This tutorial will show you how to integrate UIAutomation in your workflow.

1. Your first UIAutomation script

UIAutomation functional tests are written in Javascript. There is a strong relation between UIAutomation and accessibility, so you will use the accessibility labels and values to simulate and check the results of simulated UI interaction.

Let’s go, and write our first test!

Using iOS simulator

Download the companion project TestAutomation.xcodeproj, and open it. The project is a simple tab bar application with 2 tabs.

Insure that the following scheme is selected ’TestAutomation > iPhone 5.0 Simulator’ (Maybe you’ve already switched to 5.1 so it could be also iPhone 5.1)

Re-launch the script ⌘R (you don’t need to save). The script runs and you can stop it after logs appear.

Voilà! You’ve written your first UIAutomation test!

Using an iOS device

You can also run this test with a real device, instead of the simulator. Automated tests are only available on devices that support multitask: iPhone 3GS, iPad, running iOS > 4.0. UIAutomation is unfortunately not available on iPhone 3G, whatever is the OS version.

To run the test on a device:

Connect your iPhone to USB

Select the scheme ’TestAutomation > iOS Device’

Check that the Release configuration is associated with a Developper profile (and not an Ad-Hoc Distribution profile). By default, profiling is done in Release (there is no reason to profile an app in Debug!)

Profile the app (⌘I)

Follow the same steps than previously on the Simulator.

2. Dealing with UIAElement and Accessibility

UIAElement hierarchy

There is a strong relationship between Accessibility and UIAutomation: if a control is accessible with Accessibility, you will be able to set/get value on it, produce action etc… A control that is not “visible” to Accessibility won’t be accessible through automation.

You can allow accessibility/automation on a control whether using Interface Builder, or by setting programmatically the property isAccessibilityElement. You have to pay some attention when setting accessibility to container view (i.e. a view that contains other UIKit elements). Enable accessibility to an entire view can “hide” its subviews from accessibility/automation. For instance, in the project, the view outlet of the controller shouldn’t be accessible, otherwise the sub controls won’t be accessible. If you have any problem, logElementTree is your friend: it dumps all current visible elements that can be accessed.

Each UIKit control that can be accessed can be represented by a Javascript Object, UIAElement. UIAElement has several properties, name, value, elements, parent. Your main window contains a lot of controls, which define a UIKit hierachy. To this UIKit hierarchy, corresponds an UIAElement hierachy. For instance, by calling logElementTree in the previous test, we have the following tree:

You can see now that accessibility properties are used by UIAutomation to target the different controls. That’s very clever, because 1) there is only one framework to learn; 2) by writing your automated tests, you’re also going to insure that your app is accessible! So, each UIAElement can access its children by calling the following functions: buttons(), images(), scrollViews(), textFields(), webViews(), segmentedControls(), sliders(), staticTexts(), switches(), tabBar(), tableViews(), textViews(), toolbar(), toolbars() etc… To access the first tab in the tab bar, you can write:

The UIAElement hierarchy is really important and you’re going to deal with it constantly. And remember, you can dump the hierarchy each time in your script by calling logElementTree on UIAApplication:

UIATarget.localTarget().frontMostApp().logElementTree();

In the simulator, you can also activate the Accessibility Inspector. Launch the simulator, go to ’Settings > General > Accessibility > Accessibility Inspector’ and set it to ’On’.

This little rainbow box is the Accessibility Inspector. When collapsed, Accessibility is off, and when expanded Accessibility is on. To activate/desactivate Accessibility, you just have to click on the arrow button. Now, go to our test app, launch it, and activate the Inspector.

Then, tap on the text field and check the name and value properties of the associated UIAElement (and also the NSObject accessibilityLabel and accessibilityValue equivalent properties). This Inspector will help you to debug and write your scripts.

Simulate user interactions

Let’s go further and simulate user interaction. To tap a button, you simply call tap() on this element:

This script launches the app, selects the first tab if it is not selected, sets the value of the text field to ’Unusually Long Name for a Recipe’ and dismisses the keyboard. Some new functions to notice: delay(Number timeInterval) on UIATarget allows you to introduce some delay between interactions, logMessage( String message) on UIALogger can be used to log message on the test output and logPass(String message) on UIALogger indicates that your script has completed successfully.
You can also see how to a access the different buttons on the keyboard and tap
on it app.keyboard().buttons()["return"].tap();

3. Tips to simplify your life

Introducing Tune-up

Now, you’ve a basic idea of how you could write some tests. You will notice soon that there is a lot of redundancy and glue code in your tests, and you’ll often rewrite code like that:

That’s why we’re going to use a small Javascript library that eases writing UIAutomation tests. Go to https://github.com/alexvollmer/tuneup_js, get the library and copy the tuneup folder aside your tests folder. Now, we can rewrite Test1.js using Tune-Up

Tune-Up avoids you to write the same boilerplate code, plus gives you some extra like various assertions: assertTrue(expression, message), assertMatch(regExp, expression, message), assertEquals(expected, received, message), assertFalse(expression, message), assertNull(thingie, message), assertNotNull(thingie, message)… You can extend the library very easily: for instance, you can add a logDevice method on UIATarget object by adding this function in uiautomation-ext.js:

By the power of the command line

If you want to automate your scripts, you can launch them from the command line. In fact, I recommend to use this option, instead of using the Instruments graphical user interface. Instruments’s UI is slow, and tests keep running even when you’ve reached the end of them. Launching UIAutomation tests on command line is fast, and your scripts will stop at the end of the test.

Depending on you version of Xcode, the command line is slighty different.

You will note that the only difference between the different version of Xcode is the emplacement of Automation.tracetemplate. Before anything else, be sure that Automation.tracetemplate is existing at the path you provide. Once again, Tune-Up will make your life easy, there is a script in tuneup/test_runner/run that will take care of identifying and running the right command line.

The command line works also with the Simulator. You will need to know the absolute path of your app in the simulator file system. The simulator ’simulates’ the device file system in the following folder ~/Library/Application Support/iPhone Simulator/5.1/. Under this directory, you will find the Applications directory that contains a sandbox for each of the apps installed in the simulator. Just identify the repository of the TestAutomation app and type in the simulator:

You can also give an app that is outside the sanding box of the simulator: for instance, let’s say you have a Jenkins server that build your app automatically. Your current build will be locating in something like /Users/jc/.jenkins/jobs/TestAutomation/workspace/TestAutomation/build/TestAutomation.app. You can give /Users/jc/.jenkins/jobs/TestAutomation/workspace/TestAutomation/build/TestAutomation.app as the input of instruments and instruments will do what’s necessary to copy and install the app in the Simulator sandbox.

A final word on the command line. If you don’t precise an output file, the log result will be put in the folder in which you’ve typed the command. You can use -e UIARESULTSPATH results_path to redirect the output of the scripts. You can also use the -l option to give a time-out to your script: for instance -l 60000 will force to terminate the current script in 60s.

I’ve not succeeded to launch multiple scripts in parallel with the command line. Use the whole nights to chain and launch your scripts so you will really test your app “while you sleep”.

Interactively record interaction

Instead of typing your script, you can record the interaction directly on the device or in the simulator, to replay them later. Do to this:

Launch Instruments (⌘I)

Create a new script

Select the Script editor

In the bottom of the script editor, see that red button ?
Press-it!

Now, you can play with your app; you will see the captured interactions appearing in the script window (even rotation event). Press the square button to stop recording.

“When things don’t work, add UIATarget.delay(1);”

While writing your script, you will play with timing, animations and so on. UIAutomation has various functions to get elements and wait for them even if they’re not displayed but the best advice is from this extra presentation:

When things don’t work, add UIATarget.delay(1);!

4. Advanced interactions

Handling unexpected and expected alerts

Handling alert in automated tests has always been difficult: you’ve carefully written your scripts, launch your test suite just before going to bed, and, in the morning, you discover that all your tests has been ruined because your iPhone has received an unexpected text message that has blocked the tests. Well, UIAutomation helps you to deal with that.

and returning false, you ask UIAutomation to automatically dismiss any UIAlertView, so alerts won’t interfere with your tests. Your scripts will run as if there has never been any alert. But alerts can be part of your app and tested workflow so, in some case, you don’t wan’t to automatically dismiss it. To do so, you can test against the title of the alert, tap some buttons and return true. By returning true, you indicate UIAutomation that this alert must be considered as a part of your test and treated accordantly.

For instance, if you want to test the ’Add Something’ alert view by taping on an ’Add’ button, you could write:

Multitasking

Testing multitasking in your app is also very simple: let’s say you want to test that crazy background process you launch each time the app resumes from background and enter in - (void)applicationWillEnterForeground:(UIApplication *)application selector, you can send the app in background, wait for for 10 seconds, and resume it by calling:

UIATarget.localTarget().deactivateAppForDuration(10);

deactivateAppForDuration(duration) will pause the script, simulate the user taps the home button, (and send the app in background), wait, resume the app and resume the test script for you, in one line of code!.

Orientation

You can simulate the rotation of your iPhone. Again, pretty straightforward and easy:

The location of the screenshot will be inferred by the path you add in option -e UIARESULTSPATH results_path

Launching local script

Finally, you can launch any scripts (not only Javascript) that is on your local host. Combined with the capacity to take screenshots, you can imagine powerful automatic tests. You can use performTaskWithPathArgumentsTimeout(path, args, timeout)with path containing the full path of your script, args an array of arguments to pass to your script, and timeout a … timeout!

5. The end

Useful links

This was a pretty long post but I hope that you see the power of UIAutomation and the potential burst in quality that your app can gained. There is not a lot of documentation on UIAutomation, but I’ve listed a bunch of links that may help you.

A video

To conclude this trip to UIAutomation, I don’t resist to show you how we use UIAutomation with Meon in a little video. We use various test, and in this video, we test that the player can play from level 0 to level 120. Heeeelp me, my iPhone is alive!