Case Study: User Insights Validate & Optimize The New Nu.nl iPad App

With help of Usabilla Survey, we were able to make important improvements to the design before we launched our new Nu.nl app. Only 13 percent of the participants were able to identify a custom icon correctly. Lots of qualitative input allowed us to change the icon in time.

The company Sanoma Media

Industry: NewsUsing Usabilla Survey since: September 2011

Sanoma Media is part of the Sanoma Group and the largest media company in the Netherlands. Their portfolio includes a wide range of different media, such as magazines, Television channels, events, websites and mobile apps. Among the most popular brands are AutoWeek, Donald Duck, delicious., Flow, Kieskeurig.nl, Libelle, Margriet, Net5, NU.nl, Viva, Vrouwonline.nl, SchoolBANK.nl, Startpagina.nl and Veronica magazine. Sanoma Media strives to inspire, inform, and connect people through different media to help them make informed decisions. Their corporate philosophy is based on five five C’s: Creativity, Content, Context, Contact, and Conversion. Their ambitions are to strengthen and further grow their position as media company in the future.

Combined quantitative and qualitative insights with Usabilla Survey

The Usability team at Sanoma Media is constantly working on the optimization of the usability and conversion of their websites and apps. Their key to creating extraordinary user experiences is to involve their users early and repetitively in the design and optimization process of their products. That is why the Usability team at Sanoma Media makes use of different online tools to gather both quantitative and qualitative insights from their users. Since 2011, Usabilla Survey is part of their standard UX tool box.

Usabilla Survey allows the Usability team at Sanoma Media to involve their users and check whether their assumptions are right. Annegien Bruins Slot, Usability specialist at Sanoma Media, explains that what she likes best about the tool is “the good customer support and the fact that it is easy to involve users for gaining qualitative as well as quantitative data. Qualitative data by asking participants what they like about a design by putting notes on different elements; more quantitative data by letting them ‘perform a task’ by clicking on one specific element.”

In some cases, they also combine various tools in order to find out as much as possible about how their sites and apps are used. For example, feedback and data gathered with help of Usabilla serves as input for qualitative usability interviews that are performed in a Research Lab. In the past, they have also combined Usabilla surveys with different survey tools for even more valuable information about their users. Other tools that complement the UX tool box at Sanoma Media are Clicktale and CrazyEgg. Both are used to analyze clicking behaviour on specific pages. If they show user behaviour that cannot be explained, or that leaves questions, Usabilla tests help the UX team to gain the necessary insights to understand why users behave the way they do.

A real use case with the new Nu.nl iPad app

In one of their latest projects the Usability team at Samona Media used Usabilla Survey to test the new Nu.nl iPad app that was recently launched. There were a number of things they hoped to gain insights on, such as (1) whether or not people know how to change the left menu (the icon bottom right in the menu area), (2) whether or not people know how to return to the frontpage from an article page (tapping the cross), and (3) how people like the frontpage, as this was a major change with more focus on images.

Usually, tests are promoted in one of the newsletters, or directly on the different websites. In this case, the designs were secret before launch of the app, which made it impossible to invite users, or any outsiders as a matter of fact. Eventually, the test was promoted on the intranet of Sanoma Media, asking only iPad users to participate. 93 employees of Sanoma Media took the test.

The results showed that the design of the app in general was very much appreciated, but not all participants were enthusiastic about the new frontpage: some people prefer to read headlines, which is easier when text and images are not combined. While this granted valuable insights into user preferences, keeping user groups with different wishes and demands satisfied is a big issue that is still under consideration.

Other results were more clear and therefore directly actionable. For example, the icon that was designed to change the left menu was hard to find by the participants: Only 13 % managed to tap the right icon. In the feedback that was gathered in a followup question, participants mentioned that the icon was unclear. This was changed before launch of the app, according to the given advice.

Only 13 percent tabbed the correct button to change the left menu.

Going back to the homepage was a task which 52% of respondents ‘completed’ successfully. 40% tapped a wrong area. The advice was given to change the cross on the top right into a button on the top left, as this is what most iPad apps make use of. It became clear that making use of conventions is very important. This advice could not be implemented before launch, but the Usability team at Sanoma Media will take it into account during future updates and also for other design decisions.

The results showed how important it is to make use of design conventions.

In sum, the Usability team at Sanoma Media uses Usabilla Survey to gain combined quantitative and qualitative user feedback. The numbers let them to see how many people actually have different issues with the interface and help them to prioritize improvements. Qualitative feedback on the other hand helps them understand why people run into certain issues and allows them to gather ideas and design suggestions.

With the help of Usabilla Survey, Sanoma Media was able to validate and optimize the redesign for their Nu.nl iPad app before it was launched. This approach has become a best practice for Sanoma Media and will be used for all their other products as well.