Why You Should Test in the Real World

One of the most overlooked, yet reliable, ways to truly test your code is to get out of the office.

On a five and a half hour flight to San Francisco, I (re-)learnt this valuable and oft-neglected lesson. During the flight, I figured that it would be the perfect time to ensure that my code works properly in off-line conditions (as there wasn’t any Wi-Fi available). To my surprise, my iPad app, which was near release, crashed frequently when used on the plane.

This was unexpected since we regularly test off-line scenarios in the office to ensure that our apps work in conditions where there is limited or no network access. Of course, all was fine and dandy when testing in the office; but on the plane the app was frequently crashing.

Luckily, I had my laptop with me, so I was able to quickly determine the root cause of the crash and fix it. It turned out that there was a problem when the app was backgrounded and restored when in off-line mode. This usage is something that you wouldn’t necessary test when ensuring that off-line mode works properly while at your desk, but is a frequent scenario that is natural to a regular user of your app.

What does this mean?

The implication is simple: get out of the office and use your app where you think users would.

If the app is going to be used in different lighting conditions, get outside and test in full sunlight.

If off-line mode is a common usage, you may not have the opportunity to fly to San Francisco and back to simply test your app, but you can jump on the subway. (Make sure you test with and without airplane mode on since no or sporadic network can trigger “interesting” behaviours).

If you’re working on a fitness app, go for a run with your phone and try out your app.

You should know the expected behaviors of your users well enough to do this; however, if you want to double-check it even more closely, take your app and test it on some close family or friends who are involved with these activities.

Closing Thoughts

This experience reminded me that the best way to test your app is to actually use it as a user would. Unfortunately, while developing, we tend to have too much insight into the how the code works and don’t necessarily stress our code the way a user would. Don’t forget to occasionally get out of the office and test your app in the real world.