Implementing Usability Testing Techniques

OVERVIEW

I worked in a team to conduct usability testing on various features of the Washington Trails Association website. Our intent was to derive design improvement recommendations based on feedback gathered from the usability test sessions. By implementing our recommendations, WTA can expect to minimize user frustration and improve the stickiness of the website--meaning visitors are more likely to return for future use.

CLIENT BACKGROUND

WTA aims to protect the places people love to hike to enable equal opportunity to access the outdoors. The overall mission of WTA is to preserve, enhance, and promote hiking opportunities in Washington state through the following four main areas:

collaboration

education

advocacy

volunteer trail maintenance

THE PRODUCT

The WTA website is a hub for users to gather information about hikes, volunteer trail maintenance opportunities, and WTA facts. For this study, my team focused on the web features pertaining to hikers. For hikers, the WTA website offers the following:

hiking guide

map and driving directions

trail conditions in trip reports

photography contests

blogs

TARGET AUDIENCE

People of all demographics who are interested in exploring the great Pacific Northwest outdoors.

AUDIENCE ANALYSIS

After conducting user research, we gathered that hikers generally visit the website to seek information about hikes. Based on this finding, we decided to focus our research areas to the following pages:

Homepage

Hiking Guide

Driving Directions

Trip Report

HEURISTIC EVALUATION

To identify key test objectives, my team used Jacob Nielsen's Usability Heuristics to perform a heuristic evaluation. After each member conducted their individual heuristics evaluation, we gathered to rate each heuristic for each WTA website page that we deemed as most relevant to hikers. The rating was done on a pass or fail basis. We eventually narrowed down to three heuristics that we thought were most recurring and pertinent.

consistency

clutter and simplicity

help documentation and error messages

USABILITY PROBLEM

Though abundant in information, the current WTA website is cluttered and disorganized-- resulting in a lack of navigability and intuitiveness.

RESEARCH QUESTIONS

Our research questions were based off of the issues identified from the heuristics evaluation.

1. Can users understand what services are offered to them from the homepage?

Vertical Divider

2. Can users effectively use the website's navigation or search tools to find what they are looking for?

Vertical Divider

3. Can users gather information about a hike using the WTA website?

These research questions inform future improvements of the WTA website because they target the main reason users visit the WTA website: to seek information.

THE 4 TASKS

We came up with 4 tasks that would help understand the research questions and examine the three heuristics that we determined as critical for improvement. These 4 tasks were presented to each participant in the study and informed us of their pain points and areas of confusion.

PARTICIPANTS

To ensure that the target audience was represented, we sought participants with varying hiking experience levels and genders who matched the following inclusion criteria:

have limited experienced using www.wta.org

unregistered on www.wta.org

resides in Washington state

able to come to the study site

between 18-25 years old (this age group was most accessible to us as college students)

Participant

P1

P2

P3

P4

P5

P6

​Gender

Male

Female

Female

Male

Male

Female

​Hiking Experience

Advanced

Beginner

Value

Moderate

Beginner

Beginner

​WTA Familiarity

Some

None

Value

None

None

Some

METHODS USED

1. Screening

2. Pre-Test

3. Tasks

4. Post-Task

5. Post-Test

<

>

​Pre-screening filtered participants for the study. We created an online questionnaire using Google Forms to filter legible participants. The form was distributed across each team member's social media accounts including Facebook and Twitter.

The purpose of the pre-test questionnaire was to refresh the user about what they answered on the screener to ensure that they are qualified to participate in the study. The questionnaire included questions about how many hikes the user went on in the past year, the region they lived in, their hiking level, how they currently find hiking trips, and how familiar they are with the site. Each participant answered the questionnaire in 1-3 minutes.

There were a total of four tasks which focused on four main features surrounding a user’s preparation in hiking: the home page, hiking guide, driving directions, and trip report.​The tasks took on a within-subjects study method which meant that each participant completed the sequential tasks in the same order. They were evaluated by success, learnability, and error rates. The tasks exposed usability deficiencies and gradually shaped the product in question.

​​The post-task questionnaires collected immediate thoughts right after each task was completed. Each task resulted in different past-task questionnaires. In general, we used Likert scales to understand how easy or useful the respective features were.

The post-test questionnaire was intended to gather information about each participant's overall experience with the website. During this time, team members asked any clarifying questions they had while observing the test session.

DATA COLLECTION

Adapting an assessment style usabilty test, my team captured data using the following techiques:

Data Recordings

Voice and screen recordings done through Camtasia

Quantitative + Qualitative Data

Close-ended: multiple choice, likert scales

Open-ended: think-aloud, short answer

Types of Metrics

Success - can the participant complete the task?

Learnability - given the task-at-hand, what is the ease-of-use of the website's tool?

Error Rates - can the participant complete a task with little to no assistance from the moderator?

AFFINITY ANALYSIS

​After the data was compiled and analyzed using affinity analysis, the study revealed a variety of good and bad findings pertaining to each of the four pages that we focused on in our study (homepage, hiking guide, driving directions, and trip report).

FINDINGS

THE GOOD

Community oriented

Clear purpose

High stickiness

Vertical Divider

THE BAD

Cluttered

Missing features

Unclear features

RECOMENDATIONS

My team generated recommendations relating to each of the three heuristics we identifed as most pertinent to the user's experience on the WTA website. The following recommendations are based on data and findings from usability test sessions.

ISSUE 1: CLUTTERED

Reorganize content to emphasize visual and content hierarchy

Remove some content such as the advertisment on the right side bar

ISSUE 2: MISSING FEATURES

Offer filter to narrow down results based on experience level

Designate a section for trail conditions

ISSUE 3: UNCLEAR FEATURES

Reword "view larger map" to make it more clear that clicking on this link will lead users to a Google maps with step-by-step driving directions

FUTURE CONSIDERATIONS

If my group were to conduct repeat this study, we would make the following changes.

EXPERIENCE LEVELS We noticed during the usability test sessions that many participants were unsure how to categorize their hiking experience level. In the future, I would be more clear by recommending that 0-3 hikes in the past year represent beginner level, 4-6 hikes represent intermediate, and 7+ represent expert level.

LOGISTICSThe next time I conduct a usability test involving screen, video, or voice recordings, I will make sure that there is enough storage available because during this study, we ran into the problem of not having enough media space.

PRE-TEST QUESTIONNAIREMany of our participants were unsure where cities belonged to in certain regions within Washington State. This made the task of "find a hike within your experience level and region" difficult because some participants were unfamiliar with geographical locations.