Posts tagged ‘mobile application testing’

Logs are very important when you are getting any crash in your app or when your app is not behaving correctly.Traditionally for Android apps, we all collect the logs by connecting to DDMS.However sometimes it becomes so difficult to dig out the log when your device do not get connected to your PC due to some cable problem or any other issues with the device.Recently I faced such problem for my Android Tablet Asus Transformer and it really made me so difficult to provide the crash logs to my developers.If you are the one who faced such problem, here is an app which can help you in this situation.

Log Collector:-

Log collector is an app/tool which can provide you application logs whenever required

You are not required to connect with your PC

You can share logs via various means like Email,Facebook,Twitter,Bluetooth,Messaging etc.

You can edit the logs before sending these logs to your developer.

How to get Log Collector?

You can install it from Android Market(Now Google Play). Just type “Log Collector” and search in Google play.This tool/app can be install on Android 1.5 and above.

Screenshots:-

Google Increase the file size limit up to 4 GB in Android Market

Here is yet another rejoicing news for all Android developers. Google has increased the Android app size limit from 50 MB to 4GB to upload in Android Market. Well the maximum allowed APK (Application Package File) file size will still be limited to 50 MB. However the developers will have two expansion files, each up to 2 GB. What does this really mean? Here are some points which will make you understand about what this update from Google can do for developers and users.

Predominantly 50 MB size is enough for any kind of android applications. One of the famous android game Angry Birds Rio has the file size of 19 MB. However there are some types of apps e.g. high-quality 3D interactive games having advanced 3-D graphics, audio, and video which requires more resources.

Before I proceed, It should be clear that Android apps were free to exceed the size limit (50 MB) earlier also; however developers had to host the data on their own.

Traditionally, when user try to download such bigger apps or game from Android market, after downloading the core app, user is prompted to install additional files to complete the download. These additional files had to be hosted by developers .Obviously the file size of these additional files(which may be in GB’S) were not considered while showing the app size of that game/app on Android Market. Hence sometimes even a user may feel fooled as for app/game of 10 MB (as per app size shown in android market) we need to install additional 1-2 GB of data after installing the core apk.

Moreover downloading this additional data were often leading to expiry of the refund time. (Google provides a 15 min Refund window for paid apps. If you download an app and you uninstall within 15 mins, you can automatically get the refund for your purchased price. However For a bigger apps or games, downloading the additional data were often leading to expiry of this refund time as downloading of this additional data may takes several minutes.)

What’s Great Now after this update?

As Google has increased the Android app size limit from 50 MB to 4GB to upload in Android Market, these additional files will now be hosted on Google’s server and developers will not have to host these additional files on their own expenses on some third party server.

As there will be less restriction on the app size, we can expect to have some more good quality application in Android Market.

As all that data can be fully integrated into the APK and the expansion file, user will be able to see the total size of the app in Android Market before purchasing and downloading it.

Most Interestingly, unlike earlier, the refund window will begin only when you have installed all the files.

For Most newer devices, the expansion files will be automatically downloaded when you will download the core app from android market.

In Recently held conference on 15-02-12,NTT DOCOMO has decided to introduce an elaborate remote testing center, located at the University of Aizu in Fukushima Prefecture, Japan to help developers test their content. This Remote Testing Solution which is based on Perfecto Mobile Platform and executed by Accenture will have hundreds of handsets of all software versions, screen sizes and will help developers combat with Android device fragmentation. Here are some highlights of this proposed remote testing center:-

App developer can upload their application on the remote devices and can test their application

The system will allow 60 handsets to be tested at one time

Developers will be able reserve time slots on specific handsets

Developer can upload and test their software on this platform

Developers will be able to run automated batch tests.

Developers will have Android testing Interface through which they can take desired action such as swipes, taps at specific locations and button presses.

Remote testers will be able to use the Android testing interface, which allows for actions

More advanced inputs, like pinching on the touch display, or GPS and accelerometer readings, will not be accessible

This service will enable smooth portability of content adaptation across devices.

Mobile testing center powered by the MobileCloud platform

In short this solution will be similar as Perfecto Mobile cloud or Device Anywhere from where user can remotely test their application.

SeeTest & the Android multi-device challenge

Summary: there are many Android devices. When doing mobile test automation – say with SeeTest from Experitest – it is not practical to test ALL devices. The question is then – How to select a small number of Android devices that will enable to identify issues across 99% of ALL Android devices?

In this short article we will suggest two strategies – Edge strategy (testing the most extreme devices) and Commonality strategy (testing the most common, popular devices). Our suggested strategy to obtain optimal results in Android test automation would be a combination of both strategies.

The Challenge – testing thousands of Android devices

The business module behind the Android mobile platform creates huge challenge when you come to test your application. Your application will run on multiple hardware platforms and software versions.

The question is how to set your risk management strategy when you come to select the devices to perform your testing on?

Ideally you would run your entire regression test on all the possible variation of phone model / all Android OS versions / and all vendor firmware versions.

To get an idea of what we are looking at:

As of Q3 2011 there are:

~130 (without tablets) different HW modules.

7 Android Platform versions: 1.5, 1.6, 2.1, 2.2, 2.3, 3.0 and 3.1.

The vendor firmware can also vary and can be updated from time to time – a realistic assumption would be ~2 firmware per device.

So in order to test all the permutations of Android devices we are looking at around 1800 combinations (=130 HW device models x 7 SW OS versions x 2 firmware)

Needless to say this is totally impractical. No mobile automation engineer can seriously target such a large set of devices.

The Solution – identify relevant device factors

The question then is how to narrow down this huge list of thousands of Android devices and select a subset to test on?

The critical thing when narrowing the list is to identify which factors of the device might affect your application behavior and make sure to cover all the possible permutations of such factors. Tackling the factors instead of the devices themselves enables to narrow down to a subset of say 8 devices and provide coverage for ~90% of the devices.

Following is a list of the factors that can affect your application behavior:

Screen size – One of your main concerns is the screen side of the devices that will execute your application. One of the great pitfalls of building an Android application is how to build the view layout so it will render correctly on all screen sizes.The screens sizes can vary from QVGA (240×320), WVGA (480×800), HVGA (320×480), FWVGA (480×854) , in tablets you will find 1024×600 and what probably will be the standard for tablets 1280×800.So running the same application on 240×320 screens and in the same time on 1280×800 screens is huge challenge.

To emphasize it please look at the relative size difference (the red rectangular representing the smaller size and the black presenting the larger size):

Android OS versions – The android platform is changing very fast. The difference from version 1.5 to version 2.3 is huge. Lots of new capabilities like graphics HW acceleration were added and can affect your application.

CPU – Mobile devices in general are very sensitive to processing power. We can see phones with single core running at 600 MHz to phones that have duel core 1200 MHz.

Two other relevant factors of relatively minor effect are GPU (Low-Medium affect) and Manufacture (Low affect).

The Strategy – combined Edge & Commonality strategy

To reduce the risk you can work in 2 strategies:

Edge Strategy – select phones that have factors that are at the edges of the scale: minimum screen size, maximum screen size, running with OS version of 1.5 and running on the latest version and minimum CPU power.
As we combine them together we can end up with 2 devices:On the lower end:
HTC Hero: with android 1.5, 320×480 HVGA screen and 528 MHz Qualcomm CPU

2. Commonality Strategy – analyze what is the probability each phone has to meet your application and select those with the highest probability.The analysis here is more complex and defendant on your application type, region, target age and more.

There is a huge difference if your application is a business application that is targeted for the US, of if it’s a teenager game for girls in Japan.

A combination of the 2 abovementioned strategies will probably give you the best result

Application developers need to quickly launch their apps on various platforms to cope up with competitive market.

Hardware vendors are developing many different devices based on Android and other mobile OS.

Thousands of mobile devices in the market, different OS/Platforms with different OS versions and competitive market,all these things lead to need of a comprehensive & effective automation tool which can help in cost effective Testing of a Mobile Application or hardware they have developed in quick time.More specifically, they need a test automation tool for real physical mobile devices (not emulators) that has full coverage, ability to integrate into their existing testing environment , is simple & quick to operate and can run the same test on many devices/mobile OS.AND MOST IMPORTANTLY – a downloadable trial version – because there’s nothing like trying it yourself.

Well all these and many other challenges/Issues in Mobile Application Testing will now be addressed by an efficient Mobile Automation tool from Experitest Inc.

Experitest (www.experitest.com) – a strategic partner of both HP and Microsoft – has developed SeeTest – a test automation tool for mobile that meets all these requirements and is deployed in Fortune 500 companies worldwide, such as Microsoft, NYSE, Marvell, Texas Instruments, Clicksoftware, BSkyB, 888, Cisco and many more. To watch a video click here

Requirement 1: Full coverage – all smartphone OS, all functionality

All mobile OS:

All types of OS: There are 5 Android, iOS, Blackberry, WindowsPhone 7 and Symbian. To provide a real coverage of your customer base you need a tool that can test on any of these.All mobile OS versions: OS versions are constantly launched ot the market. There is need ot support all of them.

All mobile device models: phones, tablets. Both are used today by end users. So both need to be tested.

All functionality: Smartphones have a broad range of gestures, system alerts and many other features. All need to be supported. Otherwise no real automation can be achieved:

Gestures:Swipe, drag &drop, zoom in and zoom out, mutlitouch.

System alerts:Security pop ups.

Virtual keyboards: all keyboard configurations

SeeTest is the most comprehensive mobile test tool today in the market. It covers all OS, all functionality. You name it, SeeTest has it.

Requirement 2: Plug into QTP,TestComplete,MSTest,Junit,Perl,Python:-

Mobile testing is “the new guy on the neighborhood”. It is joining into a realty where organizations already have an existing testing environment such as QTP, TestComplete, MSTest, Java, Perl, Python. Naturally, organizations are looking for a solutions that can easily integrate into these existing environments so that they can continue working from their usual testing environment only extend it to cover mobile testing as well.

SeeTest has plugins into all testing environments such as QTP, Testcomplete, MSTest, JUnit, Perl and Pyhton.

Let’s take for example QTP. The SeeTest plugin enables to work from within QTP and simply create tests the regular and usual way tests are created in QTP (Keyword view, Expert view, data driven tests, test results and anaylsis), only that this time it is done on real physical mobile devices connected via a standard USB cable to the tester’s computer. The user can record/edit the test, run it and view reports – all in QTP. Just as he has always done. Same goes for all the HP testing & monitoring tools such as QC and LoadRunner.

The mobile world changes quickly. And so should the ability to create tests easily and efficiently. The SeeTest recorder enables simple, quick test creation.

Requirement 4: Same script running on multiple devices and mobile OS.

There are 5 smartphone OS. There are hundreds of smartphone device models. There are constantly new OS versions. This reality mandates one clear and strong requirement – script once, run on any device/OS.

One of SeeTest’s main strengths is its ability to run the very same test script on any mobile OS. Any physical device. No exceptions. This brings clear and indisputable ROI to the mobile automation project.

Share this:

Like this:

Developing applications for the Android platform is a complicated business. You have to test with multiple operating system versions, hardware vendor interface layers, hardware configurations, and network capabilities. The testing matrix for Android-based applications can be a serious challenge, impacting your product’s quality, time-to-market, and in the end, profitability.

Impetus mAutomate efficiently addresses your Android-based application testing concerns. It is an automated testing framework that runs on a cluster of phones from a variety of vendors, operating system versions, networks, and geographies. Using mAutomate you can execute automated tests on the cloud, on the devices connected over the air to your target operator networks.

mAutomate enables you to:-

‘Record/Execute’ test cases ‘for/on’ multiple devices at a time.

Save test cases and their various versions, for multiple mobile applications, at a single place and then retrieve them later for run on multiple devices.

Test case runs simultaneously on multiple devices.

Receive test results at a predetermined frequency of time or number of test cases/steps of run.

Invoke priority-based product support for the mobile application testing product used within your enterprise.

This framework, integrated with the client application, will automatically scale itself based on application functionality. It will undertake automatic instrumentation in the client application to make it compatible with Impetus’ automation testing engine.

mAutomate framework helps you in generating information about the CPU, Memory, Services, battery, network and various other analytics from the device while running test cases. While mAutomate is currently available only on the Android OS, Impetus is developing test platforms for the iOS, BlackBerry and also supporting Mobile Web.

Another gray area that the framework helps test engineers to deal with is, acquiring performance analytics details of an application. Typically, performance testing helps you to:

Identify bottlenecks and their causes.

Optimize applications based on performance results so that they can run perfectly on maximum configurations.

Verify the reliability of applications under stress.

The Automation Process:-

The complete automation process consists of the following five key steps:

Step 1: Ensures that the Mobile Client Application is re-built with a platform specific library, to create a deployable test application.

Step 2: Executes the test application on the mobile device in record mode so that all the user events and their impact in the normal flow of the application are recorded by the library in form of test scripts.

Step 3: Test scripts generated in step 2 above, are sent to the server and made available on the web Interface for View/editing.

Step 4: Enables the test scripts on the web interface to add more test data or create regression scenarios around them. A push message is then sent to the selected devices on the Cloud which launches the test application, fetches the test scripts from the server and executes them on multiple devices simultaneously.

Step 5: Ensures that all the events and their outcomes are played and sent back to the server along with performance parameters (such as time taken in execution of the test case, memory used, CPU cycles used and impact on battery), and screen shots, where they are processed by the server to produce the test results along with activity performance analysis reports.

Key features:-

Add application/product space.

Create test builds for application/product.

Associate test builds with application/product space.

Add your own remote devices, by getting a small service app installed on them.

Record test cases/scripts/data on a reference device/emulator.

Associate test cases/scripts/data with application/product space.

Maintain test cases/scripts/data for each application/product.

Select devices/emulators to run your test scripts.

Get test results e-mailed to you (after completing the entire run, the fixed number of steps, and after every X units of time) – PDF format supported currently.

Product Benefits:-

Reliability: It allows you to perform the same operation on each iteration, eliminating human error