Sep 12, 2013

We’re Testing a Digital Tool with Paper??

I admit to feeling very skeptical when I was first told we’d be doing a paper prototype UX test for a digital storytelling tool that my colleagues have been developing. Thoughts in my head included: How can a paper version of a tablet simulate what it’s like to use this tool on an actual iPad? Could users reacting to a paper mock-up possibly give us the feedback we need?

In concept, it sounded silly to have someone touch an image of a button on a piece of paper and then have an ODDI staff member pretend to be the computer and change out the paper with each touch of the ‘screen.’ However, my tune changed on testing day as we went through the process.

i
The Set-up

We had four interns from our building at BBG participate—they were all in their twenties and generally savvy with technology. On our team, we had a moderator to guide the testers through the process, a person acting as ‘the computer,’ and several people on the panel to ask questions afterward. We also had a video camera set up to record all of the action.

During the test, our moderator read from a carefully prepared script that guided each participant through different tasks. While best practice dictates that one strictly follow the script, we were somewhat relaxed with this particular test and occasionally improvised in order to extract the information we needed.

The Results

Despite the fact that we only had a pool of four testers (testing usually requires a minimum of five for a paper prototype), we were able to get some extremely valuable feedback. None of our test subjects knew they would be testing a storytelling tool for breaking news, so it was telling that all of the participants guessed that it was a news application right away—we knew we were on the right track with the general design and feel. All of testers noticed at least one flaw in functionality and offered suggestions on design that continue to inform our decisions as we refine the tool.

i
A Testing Rule to Live By

My colleague Steve Fuchs (head of design at ODDI) taught me an important lesson about testing. He explained why we do paper prototyping at the early stages of the design process by equating it to the ’5-50-5000 rule of print production’.

“An error gets exponentially more expensive to fix the later it is caught in the production process.”

Essentially, he is saying that what we can test for maybe $5 and fix cheaply early in the development of a design project–be it a website, an app or any digital product–would cost more and more as we get further along in the process. The goal is to save time and money by catching things early with a simple and easily implemented UI test.

Take home lesson: in the early stages, don’t worry about creating a fancy UI/UX test that simulates the product perfectly—first do a low-tech test to fix the major flaws.

Paper prototyping is what I now think of as the quick and dirty way to do a first round of UI testing. I’m a convert. I also read a recent blog post from Jamie Tolentino on “The Next Web” that praises guerilla UX testing, further confirming my newfound appreciation.

Do you have any UX or UI testing rules, methodology or tips that you live by?

Erica Malouf

Erica Malouf advises on marketing communications strategy at ODDI/BBG. She also contributes as a blog writer, researcher, UX facilitator, and copy editor. Erica is currently earning her master's at Georgetown University. Follow her on Twitter: @ericamalouf or connect via LinkedIn.