Re: [agile-usability] In Country Web Usability Testing

... If your goal is to measure the current usability of the services, thereby establishing a baseline against which you can measure changes in future versions,

Message 1 of 2
, Sep 5, 2008

0 Attachment

At 04:00 AM 9/5/2008, Harjinder Hothi wrote:

>Our core business is offering secure, reliable and versatile online
>payment and banking services...
>The bank is operation and recently launched in UK, Ireland, Spain,
>Germany, Italy, France and Canada, soon to be launched in many other
>European countries...Ivobank has tested systems for functionality,
>load and performance but now need to test the actual experience from
>within the native country for the real end user experience. How can
>this be best achieved?

If your goal is to measure the current usability of the services,
thereby establishing a baseline against which you can measure changes
in future versions, you need some form of quantitative testing.

The challenge for you is that you'd be doing the testing only after
completing the system, when making changes is likely to be the most
expensive it's been in the entire development cycle. Had you tested
early prototypes that are quick to build and that you can comfortably
change without throwing out the results of considerable development
effort, it's likely you could have made significant improvements to
the user interface for a tiny fraction of what it will cost you now.
When choosing the medium for early prototypes, what's most important
is being able to make changes quickly. Paper prototypes can be
excellent for this, and Carolyn Snyder's book

Regardless of the testing approach you use, you'll need to decide how
many participants to include and from which countries. If you can
afford to be patient, the most cost-effective approach would be to
start with just a couple of the countries you serve, find and address
problems that are likely to be common to all the countries, and only
then do broader testing to uncover problems that are specific to each
country. It's best to conduct test sessions in the native language of
each country, and you'll need to weigh the advantages of

- having fewer test facilitators who work with interpreters, which
lets them see firsthand the differences in different countries

- having facilitators who are fluent in the language of each country
in which you'll test.

Finally, remote usability testing, which uses an online conferencing
service like WebEx or GoToMeeting so that participants and observers
do not need to be at the same location, can greatly reduce the cost
of the testing. It's also more comfortable for participants, who can
join from home or work rather than have to travel to a research
facility. These days, most of the usability tests I conduct are
remote, and with adjustments to the way I facilitate, I estimate that
I'm getting at least 80% of the data from remote tests that I get
from in-person tests. The most important adjustments are

- Ask more questions

- Encourage participants to think aloud even more than in in-person sessions

I can then use verbal cues in place of visual cues to understand what
participants are experiencing. The main reason that we don't all the
data we get from in-person tests is that it takes more time to
establish the necessary rapport with participants and to ask more
questions. I've found that the types and quality of the observations
are very similar to those we get from in-person tests; only the
quantity is different.