Archive

the problem with paper

A writer for the TC blog, Erick Schonfeld, recently posted a description of an encounter he had with a Stanford student at a drug store trying to recruit users to experiment with a paper prototype. The prototype and study were being carried out as a requirement for an HCI course the student is taking. The TC writer, in short, found the whole experience ridiculous, especially with respect to all of the whiz-bang, interactive demos he is used to seeing. While, as many point out, paper prototyping is a standard technique in HCI, that does not mean that it always works or is always appropriate. In fact, in my experience with early stage prototypes I was overall underwhelmed with paper prototyping. But I realized over time that experience did not reflect a problem with the prototyping tool per se, but rather a lack of understanding of the context of the user.

I conducted countless paper prototypes during my graduate school career, and time-and-time again I encountered users similar to Schonfeld. I saw a lot of strange looks and cocked heads. Because I was in a more constrained setting (a lab as opposed to a drug store) I was usually able to take the time to explain what it was we were doing and why it was important. But users were rarely as invested as they were in later stage testing. This eventually led me to the conclusion that HCI tends to treat users far too monolithically.

In testing a paper prototype it is important for the user to set aside some concerns while focusing on others, and some users are able to do this more readily than others. In particular, I think there are at least four groups (hastily sketched in the diagram below): your design team, your liminal colleagues or (using Rogers’ term) innovators, early adopters, and the early majority. Just as you choose the right experimental technique to fit the current stage of your prototype, you also must choose the right group to test your ideas (this critique is similar to Greenberg’s and Buxton’s). At the earliest stage, when you are just beginning ideation, it is likely that only your design team has the context necessary to understand what it is you are really getting at (that is, only they have the necessary grounding). The next step is to make those paper prototypes, but at that point you can only target other colleagues or people comfortable with taking large leaps of understanding with prototypes. Early stage interactive prototypes can be tested with early adopters. Finally, only when you have something pretty close to a traditional beta release should you approach the early majority (the people you might expect to see at a drug store in Silicon Valley). But doing so beforehand is usually a waste of time.

The other thing to keep in mind is that “innovator”, “early adopter”, etc. are just hats that people put on and they’re not wearing them all the time. Someone who is usually an innovator may be much more conservative if they’ve just stepped off a red eye flight. This means that any quantitative questionnaire to determine which group people are in is nearly useless. It is probably best to catch people when they are likely to be in a certain frame-of-mind (innovators might be more likely to think like innovators at a trade show, for example).