Thursday, October 12, 2017

I assume that any QA engineer who works in mobile project has implemented in one or another way monkey tests for their applications. Today I'm going to explain how to implement in about 1 hour your own supervised monkey (or pseudo-monkey) tests for Android or iOS apps.

In this article I'll use the following testing frameworks - Espresso together with UIAutomator for Android and XCTest for iOS.

Usually monkey tests are implemented using 3rd party's libraries or scripts which can bring us some disadvantages:

Monkey tests are not the part of the project codebase and not controlled by Espresso or XCTest test frameworks

Is usually a 3rd party library with it's own issues and need for maintenance

Hard to fetch and process test results

Having monkey tests in native testing frameworks like Espresso for Android of XCTest for iOS brings us the following advantages:

Monkey tests are the part of the UI tests codebase. Fully owned and controlled by you

Possibility to use other tests in combination with Monkey (for example use UI test to login and afterwards start Monkey tests)

Friday, January 13, 2017

This post is about setting up android emulator for UI test automation. Properly configured emulator is the basis for reliable tests. Hundreds or thousands of professionally written test cases is great but if they become flaky because of the environment they are running on, their value reduces a lot.

I will give you a couple of advices I'm following in my test automation projects. In general we will go through below topics:

Managing emulator system animations

Controlling soft keyboard appearance

Changing emulator system locale

Tweaking first and second points will reduce to minimum flakiness in our automated tests which can be caused by emulator.

For those who are lazy to read the whole article at the bottom of the post I shared youtube video where I describe the same points with one more additional hint on top :)

1. There are three types of system animation we may control:

window animation scale

transition animation scale

animator duration scale

Emulator system animations can be controlled manually from inside the "Settings -> Developer options" as shown on screen shot below

2. Running UI tests with soft keyboard is the tricky thing. Based on my experience UI tests are failing from time to time due to clicking on the keyboard instead of UI element. That is why you may want to disable it completely.

3. Nowadays most of the applications support more than one language. This fact forces us to test our apps with multiple languages.

During manual testing this is achievable by just changing system language from Settings but the same hardly doable from automated tests. Fortunately android emulator has preinstalled "Custom locale" application which can be used to change system language by simply sending specific intent with extra language parameter to it as below:

Monday, September 7, 2015

Recently I had a need to adapt my Espresso tests to operate on RecyclerView after migration from ListViews. The current actions that are available for RecyclerView based on item position working fine but I don't like to be dependent on position since data in my tests is created dynamically.

I've googled the ViewHolder matchers and found only this link without any practical examples - RecyclerViewActions.

Then based on already created Matcher<Object> used in onData(...) I've created Matcher<VH> which was not so difficult.

Let's assume each item in RecyclerView adapter has subject, represented by TextView. The below matcher will search for item in RecylerView with unique subject which I provide into matcher. Feel free to use it:

Friday, May 29, 2015

Espresso for Android is perfect and fast test automation framework, but it has one important limitation - you are allowed to operate only inside your app under test context.

That means that it is not possible to automate tests for such app features like:

application push notifications

contact synchronization

navigating from another app to your app under test,

since you have to deal with other apps from the mobile device - Notification Bar, Contacts or People app, etc.

In fact it wasn't possible until the release of UIAutomator 2.0. As stated in Android Developers blog post - "...Most importantly, UI Automator is now based on Android Instrumentation...". And because of that we can run UIAutomator tests as well as Espresso tests using Instrumentation test runner.

In addition to that we can combine UIAutomator tests together with Espresso tests and this gives us the real power and control over the phone and application under test.

In the below example I'll explain how this approach was used to automate Contact Synchronization tests for the app I'm working with:

Thursday, April 9, 2015

The Android Lollipop update brought a hard nut to crack for the testers who use Espresso, represented by RecyclerView.

So far the app I'm testing contains this element in couple of places. While I was writing tests for them I found out that it was not so easy and obvious. Thanks to Espresso authors, version 2.0 has the basic RecyclerView actions support which is honestly not enough and is not convenient sometimes.

The activity under test has the following structure - it contains the RecyclerView element which is populated with some data. Let's call it the Feed. Each element in Feed is the Feed Post represented by FrameLayout. Of course every FrameLayout has the same layout elements inside as it's neighbors. It contains header with text and footer with like ImageView and TextView used to show number of likes.

What I just described represented by below image:

Now let's try to implement this test case:

click the like ImageView in 3rd RecyclerView item (FameLayout 3),

check that number of likes changed after the click.

In my tests I don't want to rely on the Post position in the RecyclerView, because I run multiple tests in parallel and some of them can create new Posts and push down the target one. I'd like to rely on some unique element of the Post instead.

Now I'll focus on the layout. Target ImageView and TextView are marked in red color.

Actually I can't click the like image (ImageView with id=like_image) directly, based on it's id, since this will lead us to multiple matches issue.

But what if we climb in the FrameLayout hierarchy up (from like ImageView) until we found any element with the unique value. In our case the unique is the text in Post header (TextView with id=header_text).

Now we'll find the way how to tie like image with header text. We'll go up in the hierarchy from like image using withParent() method until we reach footer (LinearLayout with id=footer) which is the sibling of header (LinearLayout with id=header), which contains child with unique text (follow arrows on the image). And the code is:

Thursday, February 26, 2015

This time I would like to share with you solution for 'Class ref in pre-verified class resolved to unexpected implementation' issue I got while I was testing my multi-module project.
Espresso tests were failing with below issue when I wanted to operate on RecyclerView from com.android.support:appcompat-v7.
In my case RecyclerView dependency was defined not in the core-app module (the core module of the app) but in the other one which is set as a library.

java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation
at com.my.app.fragments.MyFragment.onCreateView(MyFragment.java:190)
at android.support.v4.app.Fragment.performCreateView(Fragment.java:1786)
at android.support.v4.app.FragmentManagerImpl.moveToState(FragmentManager.java:947)
at android.support.v4.app.FragmentManagerImpl.moveToState(FragmentManager.java:1126)
at android.support.v4.app.BackStackRecord.run(BackStackRecord.java:739)
at android.support.v4.app.FragmentManagerImpl.execPendingActions(FragmentManager.java:1489)
at android.support.v4.app.FragmentActivity.onStart(FragmentActivity.java:548)
at com.my.app.activities.BaseActivity.onStart(BaseActivity.java:226)
at android.app.Instrumentation.callActivityOnStart(Instrumentation.java:1171)
at android.support.test.runner.MonitoringInstrumentation.callActivityOnStart(MonitoringInstrumentation.java:358)
at android.app.Activity.performStart(Activity.java:5143)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2184)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2261)
at android.app.ActivityThread.access$600(ActivityThread.java:141)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1256)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.support.test.espresso.base.UiControllerImpl.loopUntil(UiControllerImpl.java:461)
at android.support.test.espresso.base.UiControllerImpl.loopUntil(UiControllerImpl.java:402)
at android.support.test.espresso.base.UiControllerImpl.injectMotionEvent(UiControllerImpl.java:226)
at android.support.test.espresso.action.MotionEvents.sendUp(MotionEvents.java:135)
at android.support.test.espresso.action.MotionEvents.sendUp(MotionEvents.java:118)
at android.support.test.espresso.action.Tap.sendSingleTap(Tap.java:135)
at android.support.test.espresso.action.Tap.access$100(Tap.java:35)
at android.support.test.espresso.action.Tap$1.sendTap(Tap.java:40)
at android.support.test.espresso.action.GeneralClickAction.perform(GeneralClickAction.java:98)
at android.support.test.espresso.ViewInteraction$1.run(ViewInteraction.java:144)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:390)
at java.util.concurrent.FutureTask.run(FutureTask.java:234)
at android.os.Handler.handleCallback(Handler.java:730)
at android.os.Handler.dispatchMessage(Handler.java:92)
at android.os.Looper.loop(Looper.java:137)
at android.app.ActivityThread.main(ActivityThread.java:5103)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:525)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:737)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
at dalvik.system.NativeStart.main(Native Method)

And again solution is similar to one that is posted in my previous post - I have added code below into my core-app module:

Friday, February 13, 2015

Currently I'm moving all my tests to Espresso 2.0 and Junit4. And today I was struggling with one nasty issue that you can face with as well and found the solution.

So, the problem was noticed on pre-Lollipop devices/emulators. My app under test is configured to target the current latest API level 21. And after adapting some tests to Espresso 2.0 and JUnit4 I was able to successfully run it on emulator with Lollipop but trying to run it on devices/emulators lower then API 21 was failing with below issue: