Why Testing Designs with Users Is Still a Major Challenge for Companies

The good news: things are slowly but steadily changing, and increasingly more people expect useful and usable business products. The bad news is that user testing remains a major challenge for companies in 2017.

While the issues of legacy technology, bureaucracy, and complex use cases still exist, the status quo is changing. We are undoubtedly in the midst of an enterprise UX renaissance. Startups like Gusto, Stripe, and Slack are setting the expectation that business products should be useful, usable, and satisfying. Meanwhile, large organizations like IBM, GE, and Salesforce are prioritizing design as a competitive advantage by hiring thousands of designers to reshape processes and culture.– UXPin Enterprise UX Industry Report

Out of more than 3,000 people surveyed, over 50% believe that testing designs with users is still a challenge:

This is nothing new to us, since we need to educate our clients in Europe from daily about the real value of regular user testing. What we can contribute to the discussion, is the top 3 reasons WHY this remains such a big problem.

Micro-Feedback at Work

We’ve been surveying our blog readers and customers ourselves for years now, and we automatically send them this email straight after they sign up to receive our latest Usability Hacks:

The 3 reasons why testing with users is still a challenge

As we receive quite a lot of replies to our question, let’s break the answers down to 3 recurring topics:

No. #1: No clue

It seems that many people are afraid to start testing because they believe they lack the necessary expertise to plan, conduct and analyze user tests:

I just don’t know how to write the tasks, which method to use when and how to review a pilot test (testing the test) quickly and effectively.

My biggest challenge is deciding where to even start. What should come first? What tools should I use? There’s so many tests and so many tools out there that I’d love to just have a simple checklist of some solid, basic steps.

How to solve this issue

User testing is not rocket science. In a nutshell, it’s just observing people while they are actually using your product.

You can do almost everything wrong, but as long as you get the basic things right, your test will deliver valuable insights. Don’t rely on external experts to conduct the tests for you – to make user testing a routine in your department, you just need to get started and learn and educate more and more people on the go.

So, what are the basic things to look out for when running your first user test?

It all relies on a good task. We already covered this topic in detail in our blog, so here’s the best on how to write good usability testing tasks:

No.#2: No users

Unsurprisingly, you’ll need 2 things to run a user test: a product and a user who tests the product for you. Finding and recruiting the right users for your test seems to be a big problem for many companies:

My answer would be to find users from the specific target and demographic group the website or product were made for.

How to solve this issue

Demographic factors like gender may be important segmentation variables for marketers, but they have very little impact on the way someone actually uses a product.

This means that it will not make much difference if you don’t test with your exact target audience and basically pick anyone available (there are some exceptions to this rule, of course).

Just follow 2 rules:

1. Never, ever, test with someone from your own team
2. Never, ever, test with someone who has seen your product before

If you want to save a lot of time, you can also use remote usability tools like Userbrain, which offer their own pool of testers, recruit participants for you and send you videos of the tests.

No.#3: No time and/or budget

We get a lot of replies from people who tell us that they actually know the value of testing and would like to integrate things like user testing into their own workflow, but they simply can’t get the high management to approve the expenditure:

Biggest issue? No budget and no time for usability testing…

The biggest challenge I face is getting product managers to spend. It sounds like a cliché problem, but I work with departments who buy into the idea of UX (the bullet point “UX Improvements” tends to appear in many PowerPoint decks), but it’s hard to get people to get people to take action, moving from the idea of doing anything other than screen design. Thinking is fine, but when will the time come when they are done with thinking about testing and start testing with real users?

How to solve this issue

There are hundreds of excellent tips on how to get stakeholder buy-in for usability testing.
But what most people forget, is that buy-in is not enough to get people to change their behavior. It’s always nice to have compelling arguments, but to get people excited about something, you have to talk to them on a more personal level. Speak to them about what they want and explain how usability testing could help them achieve the things they truly want.

That said, it’s not enough to build a general argument for usability testing.

You have to look at each person (stakeholder) individually and find out what they care about.

The status quo is changing

The good news here is that things are changing slowly but steadily. According to UXPin’s study, almost everyone is building prototypes and over 70% of people are involved in User Research or Usability Testing activities:

So, let’s hope for the sake of our users that more and more companies realize the value of integrating users into their processes in 2017 and that every new product is tested and improved before it reaches the end customer.

How does your company deal with user testing? Happy to hear your thoughts…

Rui Pereira

Understanding User testing “think aloud” methodology should be a pre requisite for all UX and UI designers. Should be one of the 1st methodology to be learned. How can designers improve designs if they don’t know how to test them? For me it makes no sense.
Like the article says, it’s not rocket science. You learn more as you do it. The biggest challenge is trying no to lead the user when creating the test scenarios and have “false” findings.

Ana

Think aloud methodology is a qualitative one. I think people should know all the methodologies and know when and why to apply. I see a lot of UXers using and abusing the think aloud protocol and changing the design (and overall business) path mainly based on qualitative results. This is too short and lack of assertiveness. Best practices says to triangulate with qualitative and quantitative.

Markus Pirker

yes – I often experience the same with customers who want to abuse thinking aloud tests to verify business decisions based on the things user’s are saying. You really need to make sure people understand the application of different methods in your UX toolbox.

Ana

Yep. exactly. Unfortunately it seems that most designers / UXers don’t know that research and user testing comes across with a lot of scientific methodologies (hypothesis, experiments, analyzing results, …). Perhaps if companies could see this “scientific side” they would realize that UX works with palpable realities and is not weird black magic 🙂

Markus Pirker

I completely agree – sadly the design study courses I follow have mostly reduced if not completely removed teaching UX basics from their schedule. So you’re educating visual designers who haven’t learned how to question their own design decisions with the help of external user feedback…