msc mobile emerging technologies blog

Testing Offline Mobile Enterprise Apps for SAP

Testing is an important part of every software development project - unit tests, integration tests, user acceptance tests and so on. In this blog, we want to discuss the testing of mobile enterprise apps, especially the ones for offline use.

No software is perfect. Rockets are send to Mars and they crash because of bugs. Some airplanes shut down if they are not “rebooted” at least every 248 days (and you ask me why I am afraid of flying).

Having and fixing a bug in one central place like an SAP system can already be a pain (system restart, down time, unhappy users, lost data etc.). Fixing a bug in an application installed on many computers (like for example SAP GUI) is a bit trickier, but still manageable if all machines are in-house and connected to the corporate network.

It gets a lot more difficult to ship a new version of an enterprise offline application that is installed on smartphones and tablets outside the company office. Even if you have a device management solution like Afaria, SAP Mobile Secure, Mobile Iron or Airwatch, it is still a lot of work. Because of this fact, it is important to test enterprise offline apps very carefully. It does not matter which platform you use – Agentry, SMP, MobiLink or something else. Testing must be performed on all layers, including client, middleware and backend. The following is a list of tests that should be performed for every application.

Functional tests

Test of all functionality in both directions - create data in SAP and synchronize it to the mobile device. Create data on the mobile device and send it to SAP. Update records on both sides, delete data and check that it is removed on the opposite system as well.

Test validation on the mobile device - is the user getting the right warning/error message when entering wrong data?

Performance tests - synchronization and on device

Synchronise with the maximum data volume that you expect. Also take into consideration that the data volume could increase over time.

Test with the maximum number of concurrent devices to see how the system can handle the load.

Test the device with the maximum amount of data - how is the performance of lists and detail dialogs? How fluent is the navigation within the app? How fast is the startup time?

Test against the UI/UX Guidelines of the mobile operating system

If you have a native application, is it following the UX guidelines from Apple, Google and Microsoft?

If you have a hybrid/platform independent application, is it following common guidelines (for example the SAP Fiori UX guide)?

Usability Tests

Give the application to users and let them test it. Is it intuitive? Can they figure out how to use it even without or with only little documentation?

Test your solution with users experienced with smartphones/tablets and with those who are not used to modern technology.

If you roll out internationally, test with users from different countries to see how they react to the software.

Test with different languages and location settings

Let the translation be reviewed and tested by a native speaker

Listen to your test users!

Test behind the curtain - not only UI, also the middleware and backend integration

Test client, middleware (if you have any) and backend

Make sure the communication between the three components is working fine.

Test the positive and the negative cases

Test your solution with data that you expect to work

Test your solution with data that you expect to fail

Make sure both cases are covered successfully

Test the application when the middleware server is down? Does it behave like expected? Test it when SAP is not available as well.

Hard kill the app and check if any data is lost

Test outside of the lab/with the real network

Test your application with the real network your users will use later. Is the performance still good enough with GPRS or Edge?

Test the software on the real device, not just a simulator

Test the software with the real device under real conditions. Can you read the screen in bright sunlight?

Test the solution with all the other apps installed - are there any side effects?

Test it in low memory situations – is the app still working? How is the performance in this case? What happens if the app is removed from memory?

Test the app with older versions of the operating system. Also test it with beta versions that will be released in the future!

Write automated test scripts

Let your developers create automatic test cases that can be run before every build.

Test on different devices

If you have a BYOD (bring your own device) setup, try to test the application on different devices. Is the app working on all screen resolutions? If platform independent, is it working on all platforms?

Test the installation/update

Test the installation and upgrade of your mobile app. Can the APK, IPA or XAP be downloaded over the phone connection?

Test the security of the app

Use a network sniffer to see if the encryption that you switched on is active.

Try to open the device database to see if it is really encrypted.

Test the secure storage like for example the iOS keychain – is this data only local or is it send to the cloud?

Test the administration part of the app

Can you create new users, devices etc. in your mobile landscape?

Can you unregister/delete devices

Are the logs setup correctly? Do they have enough information to analyse issues? Is it set low enough that the log is not decreasing the performance?

No matter how long and well you test, there will be bugs slipping through the cracks. The more complex the app is, the more likely that will be. Because of that you should always have a fall-back solution - this could be either an Excel sheet or just a piece of paper. Ideally you never need to use it.

Testing is a hot topic - how much testing is enough? How much is too much? You could argue that one can never test enough, however somebody must pay for it. Like with everything, you need to find the right balance.

Mobile and emerging solutions are changing the business world at a speed that was never experienced before. This blog is covering the mobile revolution. We post industry and company news, technical and functional tips and tricks and much more.

About Us

With everything we do, we want to make the work life of people easier. We believe that people should be able to focus on their work, instead of spending their time on administrative tasks, paper or software.

The way we make the life of people easier is by providing easy to use solutions for the mobile world and the desktop. We implement standard SAP Mobile solutions and develop custom applications.