Monthly Archives: October 2016

I just noticed my video from my presentation was released. This was the presentation that I did at #NativeScript Developer Day in Sept of 2016. The subject was Testing your NativeScript app and dealt with both Unit Testing and End to End testing.

I saw a couple of "um" mistakes. I "um" need to "um" work on "um" removing some "um"'s. 😉

Here are the questions at the end (since like an idiot, I also totally spaced about repeating them) are:
1. What about testing Angular2 NativeScript apps.
2. What about Code Coverage .

I got to give a talk on NativeScript testing to a great group of people at NativeScript Developer Days.

Despite my silly technical problems at the beginning, we managed to get back on track and I was able to present all the critical information that was needed to get everyone up and running. I did manage to miss my last slide; so I will make sure to have all that information at the end of each of these posts.

For those who are interested in doing testing and weren't there; or those who were in the talk, and just want to get a quick refresh, or if you just prefer the information written down; then this set of blog posts is for you. I plan on doing a multi-part series on this as I want to cover what I did in my talk in the first two blog posts; but then expand the series to show Angular Native unit testing, how to unit test controls, and how to write unit tests for the NativeScript core.

You can view the #NSDevDay video here . You can download my slide deck from the talk here. And you can download the entire app that I wrote for my talk, including all the tests from here.

Now as you might have guess even though you can do a lot of tests via unit testing, there are some limitations to unit testing. You are typically only testing pieces of the whole. So you really need to make sure the whole app works and the code you used to tie together all the pieces works. And this is called End to End testing.

In my book, "Getting Started with NativeScript", I discuss using Appium, for end to end testing. Since the point where my book was published, one of the NativeScript engineers Hristov Deshev has actually created a really neat plugin. It actually wraps up all the same steps that I came up for in my book. Since it is a plugin, it is way easier to use, since it handles all the configuration and installation for you. You just type tns install appium and it will install everything you need for end to end tests. In this case all the tests for Appium will be stored in the root /e2e-tests folder since they don't actually need to be part of the Application itself. Appium uses Mocha (which is the primary reason why I use mocha for my normal unit tests, I like consistency.). It also uses Chia for the Asserts; so the tests are created the exact same way as I described in the unit testing post; with only a couple minor changes.

The first thing you will notice is that I don't require any of the code from the app; the end to end testing does not run from inside your app; it is 100% external to your app. So, you are at this point requiring the Appium setup and control driver library that Hristov wrote to wrap the configuration complexity.

We still use the describe; but we have to actually setup the driver (that controls the device) in the before function. And we tear it down (or close it down) in the after function. Then we actually do all of our tests...

So in the first test, what we have to do is use the driver we created in the before function; pretty much everything uses the driver. It is the communication channel to the device being tested. Then the command we use is .elementByAccessibilityId('message'). This command will search the iOS or Android layout for any element in the UI that has the "message" accessibility id attribute set. Now, in NativeScript this is actually set using the automationText property. So if you look at my main-dynamic.xml or main-page.xml file; you will see:<Label row="1" id="message" automationText="message" text="{{ text }}" class="message" tap="scrollOff" textWrap="true"/>

Then once the driver finds this element, it looks at the .text() value and that value should become "Back of Card". When the app starts up; the first card chosen is the "Back of Card". So if my app is actually running properly, this test will succeed.

Lets skip to the last test; as understanding it, will explain all the other tests in between. So lets figure out how it works. Now first thing to understand is in the carder app; the numbers, letters and pips on the card are actually using a font. So if you were to switch to Arial as the font, the actual underlying character is a "r" for the hearts pip. So that is why I have "r" and "q" letter used in the tests...

As you might have noticed we do another elementByAccessibilityId("prior") -- we are looking for the prior element which is a button in this case (xml is:<Button automationText="prior" id="prior" text="Prior" tap="prior"/>).

Then once it exists we tap it twice. As you can see you can keep stacking commands; so in this case we actually stacked tap twice. This is important to know, because frequently you will want to do different tests to the same element; or multiple actions to the same element. You can easily chain them.

Next up, we are using the elementsByXPath which searchs the UI for anything that matches the xpath and returns it. And finally we check the number of elements found.

Appium allows you to set/get values of fields, emulate taps & gestures, act like typing in on a keyboard, or just act exactly like an end user would, and then you can verify the results. This allows you to build complex tests for your UI that test the entire "user" exposed functionality.

Now lets go into some specific details on some of these commands that you need to be aware of in Appium and NativeScript testing. The Appium web driver actually has a ton of different selectors; however since Appium was initially developed for the web, only a couple selectors in the Appium documentation actually work for your NativeScript mobile apps. The two selectors you can use reliably is element(s)ByAccessibilityId and element(s)ByXpath. "element" returns only the first element found. "elements" (note the added 's') returns all elements found. As discussed earlier, AccessibilityId uses the NativeScript automationText value to find item(s).

XPath actually allows you to drill down into the UI and find specific items that may have a specific hierarchy and/or certain parents. For example; rather than search for all buttons, you can limit the search to buttons that are inside a GridLayout which is inside a StackLayout area. However, the biggest downside with xpath is that it expects you to have the actual native android or native ios control type name. For example; on Android the NativeScript Button class is actually using the android.widget.button. The native class on iOS it is actually using an UIButton. Now that makes XPath really, really hard to be cross platform test, doesn't it? So to solve that issue; I have written a cool wrapper to help with this issue. It allows you to pass in your NativeScript class name and it will, depending on the platform you are testing against, will return the real underlying native component name. So in this specific test case the xpath was "//" = any level of items, we aren't giving any specific parents (so find this anywhere in the layout). Then my helper class nsAppium.xpath("Label") will give me the actual underlying UI name of a NativeScript Label component, and then finally "[@text='r']" means that element must have a "r" as the text field value. In the case of the card it should find, the two pips on the edge of the card which should be a "r". So this test would pass as long as the prior button worked to bring you to a King of Hearts card...

The next thing you need to be aware of in Appium is that you MUST return the driver results. You will see every one of my Appium tests does a return. In all reality, the entire chain that we are doing is actually a promise chain. So for the test to actually run and then pass/fail, the final result of the promise chain must be evaluated. So ALWAYS return the promise chain, or your tests will say they passed without actually knowing for sure that it actually passed or failed. This is CRITICALLY important you return the final promise!

The final gotcha in Appium is to know is at the top of the test file, the "this.timeout(100000);" is actually very important. Appium can take a while to actually startup the DRIVER to communicate with the device/emulator. And you really do not want the test to timeout (which = failure) before it actually starts running it. So make sure at least for android, that this is a very large value...

A couple notes; Appium launching the driver can be extremely slow. You have to wait a while before it actually appears to be doing anything. Second; If you are using my NativeScript-LiveEdit plugin, the watcher now has a cool ability to be able to launch Appium when you hit the "a" button in the watcher window.

Now all of this can be automated and is highly recommended to be automated in something like local git hooks, or some other CI environment. That way when you commit a change; Unless you have a beefy machine, I would recommend you set it to run on like every 3-5 commits (depending on how frequently you commit, it might be higher). Because Appium is fairly slow to get the whole test started. At worst case I would recommend you run a Appium at least once a day, several hours before you go home...

If you need help setting up a automatic testing and/or CI environment or you would like some training, please contact me.

For those who have upgraded to the all new xCode 8, you may have noticed some of the plugins breaking... The biggest breaking change in NativeScript and xCode 8 is now things deep down in the ObjC runtime that used to be a function call are now a property.

So, for example let say you needed to access UIScreen.mainScreen.

In xCode 7 this was

var mainScreen = UIScreen.mainScreen();

in xCode 8 this is now:

var mainScreen = UIScreen.mainScreen;

Notice, it is no longer a FUNCTION call. It is a PROPERTY. Now how do you make this compatible so your code can run with both xCode 7 and xCode 8.

If you are developing an app; I recommend you use the helper function that Telerik added to NativeScript which they use throughout the core modules.