3.
Overview of Usability Testing with Morae UIanswers
Preface
About this Overview
This overview is an excerpt of the comprehensive guide, Usability Testing with Morae, which is
now available from TechSmith.
The complete guide represents a collaboration between Dr. Piyusha Paradkar, of UIanswers, and
TechSmith Corporation, developers of Morae. The guide presents Dr. Paradkar’s method for
usability testing, updated to highlight Morae’s rich functionality for usability testing. This proven
methodology has been used by Dr. Paradkar for over five years. It focuses specifically on the
usability testing of software, Web sites, and online documentation.
The complete guide also includes a CD-ROM, which features:
• An interactive PDF version of the entire book
• Sample templates of common usability testing forms
• A sample PowerPoint presentation that you can customize with your own results
• A highlight video illustrating the real-life usability testing example presented in the book
About UIanswers and the Author
UIanswers offers state-of-the-art user experience solutions for improving the usability and
accessibility of products.
Piyusha Paradkar, owner and Principal of UIanswers, has a Ph.D. in Industrial Engineering
from the University of Wisconsin-Madison. She specializes in the field of Human Computer
Interaction, focusing on interface design and analysis. She has led the design and usability testing
of graphical user interfaces, voice response systems, touch-screen interfaces, and networked
communication systems for organizations, including e-mail and voicemail. She helps
organizations define usability effort on cross-functional teams in all stages of product
development, including product concept, initial design analysis, requirements definition,
prototype testing, and online help and user guide development.
Dr. Paradkar has a strong background and expertise in Section 508 accessibility and has guided
product teams on accessibility evaluations and design solutions. She also conducts mentoring
and training workshops on usability evaluation techniques, and assists organizations in setting up
usability testing labs and programs that succeed.
Dr. Paradkar can be contacted at piyusha@uianswers.com.
2 • www.techsmith.com Preface

4.
UIanswers Overview of Usability Testing with Morae
Introduction
What is Usability Testing?
Usability testing is a powerful and exciting technique that lets you determine how well a product
can be used by its intended users. It properly identifies and prioritizes usability issues and
highlights areas that need change, thus eliminating wasteful speculation about where the problem
areas are. This predictive power ultimately permits critical design changes early in development
and prior to product release, which is cost-effective and also enhances customer product
acceptance.
Identifying What Works and What Doesn’t
Usability testing has two primary goals, which are to identify:
1. Areas of a product or product prototype where users have difficulty.
Once identified, you can then either redesign those areas so the user will not have
difficulty, or provide appropriate assistance.
2. Areas of a product that are marketable — that really work!
Once identified, you can provide promotional ideas for the product that extend beyond
product features and the solution the product provides.
Translating Results into Improvements
Once the critical areas needing redesign are identified, these design changes can be made during
development. This process can have the following positive results.
Increased: Decreased:
• Sales revenues • Human error
• Productivity • Development time and costs
• Web site traffic • Technical support calls
• Customer satisfaction and
loyalty
How Morae Simplifies and Improves Usability Testing
Morae is an innovative software-based solution for usability testing that enhances data collection
and speeds up analysis. Its dramatic effectiveness lies in the fact that it actually reduces the
amount of work needed, not only at setup but also during the data logging, analysis and
presentation stages. Morae’s design works well for users new to the field of usability testing, as
well as seasoned specialists with existing usability labs. It is simple to set-up, use, and maintain.
Morae fits well with existing methodologies and processes of usability testing. It improves the
usability testing experience without compromising any established procedures. We present here
an overview of the UIanswers usability testing methodology, updated to make full use of the
rich functionality offered by Morae.
www.techsmith.com Introduction • 3

5.
Overview of Usability Testing with Morae UIanswers
Planning for a Usability Test with Morae
Successful usability testing involves several people performing multiple steps with dependencies
and contingencies. Each usability test needs to be considered a project. The planning phase is the
key to maintaining proper timing, budget, and ultimate success.
This section covers the major planning steps for a successful usability test, describes how Morae
fits into the planning process, and highlights new ways to enhance usability testing using Morae.
Establish Usability Testing Goals
The first step in planning your usability test is to define what you want to know about your
product and establish clear goals for the testing.
Create a vision statement for the usability of your product or site
At the highest level, you need to create a vision statement for the usability of your product or site
if one does not exist. This vision may not be measurable, but it should be related to the business
objective and should help focus the specific measurable aspects of your product or site.
Example vision statement:
“Novice users will be able to complete their taxes in a reasonable time and without major
errors.”
Establish specific, measurable test objectives
The objective of a usability test has to be something you can measure. "Is my product usable?" is
not a good objective. Neither is, “customers will perceive the product or Web site to be easy-to-
learn and efficient-to-use.” Why? Because these attributes cannot be measured. Test objectives
must be expressed in terms of measurable usability attributes.
Example test objective:
“The overall goal is to find out if novice users make any critical errors when using this
software for their taxes.”
Define Test Tasks
The usability test requires understanding the real tasks that real users will be attempting to
accomplish on the Web site or with the software application being tested, and building your
usability test procedure based on this understanding/knowledge.
Choose appropriate tasks
Select test tasks based on the following:
• Tasks that users will perform most frequently in typical usage
• Tasks that users have had difficulty with in current or past versions, as indicated through
support calls or previous testing
4 • www.techsmith.com Planning for a Usability Test with Morae

6.
UIanswers Overview of Usability Testing with Morae
• Tasks based on design areas that have been particularly challenging during design
• Tasks for which you have more than one design option
• Tasks that users perform infrequently but are critical (e.g., software downloads, monthly
reports, troubleshooting routines)
Provide context for tasks through scenarios
UIanswers recommends that you script your tasks for the usability test by including them within
the context of one or more usage scenarios. This will provide perspective for each participant as
they read the instructions and perform the tasks using the product under test.
Select what to measure for each task
Standard task measures include: time on task, errors, task completion, and access of help during
tasks.
How Morae Expands Task Definition
During task definition, it is often frustrating to discover that what you want to measure cannot be
measured because of equipment limitations. For example, the number of clicks taken to complete
a task is an important measure of user success in many applications. However, these measures
are often not included in the usability test because:
• Manually logging this data during live-testing is tedious and error prone.
• Reviewing the video after testing is complete in order to log the data is cumbersome and
time-consuming.
Morae’s Rich Recording Technology gives you more flexibility in what you can measure right at
the task design stage. It dramatically reduces the effort involved and makes this data both easy to
capture and accurate. The following table compares example traditional task measures with
example expanded measures possible using Morae:
Traditional task measure Task measures possible with Morae
Time on task Time on task
Number of steps required to Number of steps required to complete the task,
complete the task with the screens for each step
Number of clicks
For participants who took more than the
estimated number of clicks, exact spots where
the clicks happened on the screen
Number of times user accessed Number of times user accessed help
help
Exact places where user accessed help
Time spent in help
Contents accessed in help
www.techsmith.com Planning for a Usability Test with Morae • 5

7.
Overview of Usability Testing with Morae UIanswers
Setting up the Usability Test
Proper equipment makes the usability test easier, more effective and more meaningful, especially
during analysis. UIanswers recommends that, at a minimum, you record the participant
interacting with the product. While testing software products and Web sites, it is also critical to
record the screen activity of the participant along with the participant video.
How Morae Simplifies Test Setup
Morae is an all-digital, software-based solution for usability testing. An additional microphone
and Web camera are optional (both are inexpensive and often built in). Because Morae is truly
portable, it is perfect for mobile testing. Morae eliminates the need for special usability testing
equipment and dramatically reduces both the complexity and the time involved in setup.
Morae allows you to set up a configuration file, which defines what you want to record, in
advance of the test. You can then put this configuration file on the testing computer ahead of
time so that you are not burdened with setup tasks immediately before or during testing.
Morae offers an easy way to record audio, video and screen activity. By automatically creating a
log of this activity, Morae minimizes the data logging during testing.
Morae’s features include:
Audio and video recording of screen and Automatic and manual data logging
participant capabilities
Picture-in-picture capabilities to show Remote observation of screen activity from
participant video and screen another computer on the network
Synchronized screen recording Software-based, integrated editing
capabilities to create and produce highlight
Live data logging capabilities
video
Using Morae, you will increase your testing and analysis capabilities, increase the overall
productivity of your testing efforts, and eliminate the following complex and expensive
hardware:
Scan converter Picture-in-picture generator
A video splitter device Mixer
Cables Editing equipment for highlight
video
Tapes
Data logging equipment
6 • www.techsmith.com Setting up the Usability Test

8.
UIanswers Overview of Usability Testing with Morae
The Test Team
The critical/essential people in a usability test are the Test Participant, the Facilitator, and the
Primary Data Collector, but there are additional roles that may be involved as well, such as the
Stakeholder Observers.
By facilitating setup and data logging, and by eliminating equipment handling, Morae greatly
reduces the effort required by the test team. It makes the “one-person testing team” more feasible
and less burdensome than it has ever been before. In situations when the Facilitator is the only
person available, Morae permits collection of data that either would not be collected, or would
only be collected during data analysis at a much greater effort.
www.techsmith.com Setting up the Usability Test • 7

9.
Overview of Usability Testing with Morae UIanswers
Conducting the Test
The Pilot Test
The pilot test is a practice run through the test protocol prior to the actual test. Pilot testing
allows you to verify that the tasks work as intended, that the test length is appropriate, and that
the correct information is being properly captured. Run one or two participants through the
protocol before conducting the actual test. Conduct the pilot test exactly as you would conduct
the actual testing. Allow time before the actual test to make any needed changes to the protocol.
The Actual Test
The Test Participant and the application or Web site under study are the primary focus during
testing. The following is a summary of the chronological steps the Facilitator should follow
during the actual test.
Pre-Test Steps
• Greet the participant and establish rapport. Have participant complete the necessary
forms.
• Reinforce their right to take a break or withdraw from the test for any reason if they wish
to do so. Explain how the data will be used. Explain how the video and audio will be
used.
• Explain the participant’s role, explain your role as the Facilitator and how you will
interact during testing, and inform the participant of any audience or observers for the test
and ask consent.
• Provide an introduction and demonstration of the product if it is a part of your test design.
Provide or simulate the testing scenario. Ask the participant if (s)he has any questions.
Test Steps
Hand the first task to the participant and have him/her begin the task. Position your seat 3 to 4
feet away from, and behind, the participant. Continue until all tasks are complete.
Post-Test Steps
Administer the post-test questionnaire. Debrief the participant. Ask clarifying questions about
the participant's interactions with the interface. Solicit any final comments. Thank the
participant, handle reimbursement arrangements, and say good-bye.
8 • www.techsmith.com Conducting the Test

10.
UIanswers Overview of Usability Testing with Morae
How Morae Automates Data Logging during Testing
During a usability test, you can use the automatic data capture capabilities of Morae to record a
majority of the quantitative data. In addition, a primary data logging person or observers may log
qualitative data manually using Morae.
Synchronized Data Logging
Morae creates a recording of the system activity synchronized with participant audio and video.
In addition, Morae automates much of the data that is typically either logged manually during
testing, or logged after testing with time-consuming video reviews.
The automatic data capture:
• Improves accuracy of the data collection
• Reduces the amount of data that needs to be manually logged during live testing
• Frees the facilitator and observer’s time for other activities
Screen video is always captured automatically by Morae. The following table describes the
additional capture options that may be selected within Morae:
This Capture Option: Records:
Camera Video of the user from an attached camera.
Microphone Audio data from an attached microphone.
Keyboard Data related to the activity on the source computer’s keyboard. These data
include all the numeric and alphabetic keys pressed on the keyboard.
Screen text Text being displayed on the source computer’s screen. Screen text includes
every word shown on the screen except for those that are part of an image.
Mouse clicks Mouse clicks (left and right) occurring during a recording.
Application events Windows events and Web page changes that occur with a specified application
during a recording.
Flexibility in Data Logging
When conducting a usability test, unexpected behaviors (e.g., user accessing help, user taking a
different path to complete a task, etc.), which are very important to record, can suddenly change
the data collection and logging requirements. Morae is flexible, and easily accommodates the
extemporaneous events and unexpected paths that are not only common, but also especially
valuable and meaningful in usability testing.
Integrated Live Data Logging
In addition to the task data we plan to capture, it is also important to be able to log points of
interest in the data while testing is in progress. For example, it is important to note areas where
the test participant had difficulty completing a task or where the participant had a particularly
positive experience. Morae allows the Primary Data Collector or Observers using Remote
www.techsmith.com Conducting the Test • 9

11.
Overview of Usability Testing with Morae UIanswers
Viewer to directly add categorized markers during live testing. These are saved and synchronized
directly with the recording for easy access during analysis. Morae also allows association of
audio and text based notes with markers to add more meaning to the markers during analysis.
10 • www.techsmith.com Conducting the Test

12.
UIanswers Overview of Usability Testing with Morae
Analyzing the Results
Usability test data are more than likely comprised of a combination of quantitative and
qualitative data. Quantitative data include measures such as time required to complete a task,
number of errors, and number of clicks. Qualitative data include measures such as user feedback
on surveys, participant comments during and after testing, participants’ facial expressions, as
well as their opinions and 'feelings' about the product. In most cases, the quantitative data are
more effectively understood when combined with the qualitative data and observations. Usually,
for a particular usability problem, the supporting data will have both quantitative and qualitative
elements.
How Morae Simplifies Analysis
Data analysis carries the reputation of being the most difficult and time-consuming part of the
usability test. Testing with Morae significantly reduces the effort involved in data analysis.
Morae allows you to access, view and search for data in new and innovative ways while at the
same time reducing the number of steps required for traditional data analysis.
• Add Markers. You can easily add markers in the recording, both during and after
testing, to mark points of interest, including participant behavior and expressions.
• Create Segments. Using Morae, test tasks performed by the participant can be
segmented by start and end points and saved for analysis and presentation.
• Search for Text or Events. Morae’s powerful search capabilities allow you to quickly
get to the data you need.
• See the Data in Context. Search results are automatically synchronized with the video
recordings, placing the data in context with when they occurred. Simply select an event in
the search results list and the video will move to that point in time, highlighting on the
screen where the event occurred:
• Easily Calculate. Typical data calculations, such as time on task and number of clicks,
are readily available through built-in calculators.
Making Design Changes Based on Results
Usability testing data, typically, does not show what actions to take with the user interface, but
rather where actions need to be taken. UIanswers recommends that you always follow through
with specific recommendations and design improvement suggestions after your test.
www.techsmith.com Analyzing the Results • 11

13.
Overview of Usability Testing with Morae UIanswers
Presenting the Results
While a detailed written report is nice for future reference, UIanswers recommends presenting
the results directly to the audience. This should be done verbally through a presentation, before
or along with any report that is distributed. A report with no presentation/follow-up is more open
to misinterpretation.
Set up a presentation, accompanied by a highlight video, for the stakeholders. Follow this up
with a detailed report to the main stakeholder on the management team, to distribute as
appropriate.
There are four key things that need to be communicated to the development teams after a
usability test:
1. An overall “how it went” factor
2. What worked
3. What did not work
4. What can be done
Summarize findings by priority. Include screenshots where applicable. Group the findings by
general problem areas or by the different areas of the product. Do include positive findings.
UIanswers also recommends coming up with estimates on “time-to-fix” or “degree of difficulty
of fixing” and impact. These should never be made independently, but in collaboration with the
development team lead or appropriate development team members.
How Morae Streamlines the Presentation of Results
A highlight video is a collection of short and meaningful segments of the testing video. Highlight
clips are compelling and effective during presentations. They provide a positive and potentially
eye-opening experience for the audience and the development teams. UIanswers recommends
using several short highlight clips rather than a complete highlight video. Use the video clips
along with the other quantitative data to report a finding in its entirety.
Morae offers a streamlined process for the creation of highlight video clips, with great savings in
the time and effort involved. It carries forward all the segments of interest that you have flagged
in the analysis stage into the presentation stage, allowing you to consider those for highlight
video clips directly. Morae also places the entire recording at your fingertips, to explore and clip.
With Morae, there are several options for presenting your results. You can create a screenshot of
any frame from either the screen or camera video portion of the recording, or a screenshot of a
frame that contains both. You can produce single video clips as AVIs to embed in your
presentation. Or, you can use Morae to produce a polished highlight video with titles.
You can easily order, organize, and preview your video clips on the Storyboard for a better fit
with your presentation.
12 • www.techsmith.com Presenting the Results

14.
UIanswers Overview of Usability Testing with Morae
Flexible Picture-in-Picture
Another compelling feature of Morae is the flexibility you have with the participant video. You
can choose whether or not you wish to display participant video with the screen recording as
picture-in-picture (PIP). In addition, Morae lets you choose where to position the participant
image. You can position it anywhere and even change the location for the different tasks
performed after testing. You can also adjust the opacity (transparency) of the PIP image over the
screen video. Because Morae lets you do this after the recording, you never run into the problem
of participant video obscuring critical portions of the screen video image.
www.techsmith.com Presenting the Results • 13

15.
Overview of Usability Testing with Morae UIanswers
Conclusion
Usability testing is a highly effective technique to measure and validate the user experience. It
can provide specific and timely direction to ensure an appropriate and compelling user
experience. It also saves time and money and ensures a better product.
If you are still on the fence trying to decide how you are going to justify a usability testing
solution or justify upgrading your solution, consider how Morae solves the most common
hurdles to justification:
• Cost of the usability testing equipment and setup.
Morae has successfully removed this hurdle by coming up with a cost-effective, easy to
set up and use, ‘complete’ solution.
• Location for usability testing.
Morae’s portability and ease of set-up on any computer dramatically reduces this hurdle
by expanding the locations where usability testing may be conducted.
• “People” resources and skills to conduct the testing.
With Morae, a smaller (or even a one-person) usability test team is now a reality. In
addition, the complete UIanswers methodology and templates are available with Morae
to assist you in conducting your test. Together, you have the resources and step-by-step
guidance necessary to conduct a successful test.
• Speed of analysis and presentation.
Morae gives you the tools to get to your results faster and to present your conclusions
quickly.
Whether you have access to an existing usability lab or are just in the market to purchase the
minimum to get started, Morae is a solution that will work for your needs. Its design works well
for users new to the field as well as seasoned specialists with existing usability labs. It is simple
to set up, run, use and maintain. It enhances and expands data capture capabilities. Its dramatic
effectiveness lies in the fact that it actually reduces the amount of work needed, not only at set-
up, but also during the data logging, analysis and presentation stages.
It fits in well with existing processes for usability testing as demonstrated with the UIanswers
methodology overview. In addition, Morae offers one of the most portable solutions for user
testing on the market today.
For more information on usability testing, usability process, and establishing a program at your
company, contact Piyusha@uianswers.com.
14 • www.techsmith.com Conclusion