Thursday, March 24, 2016

A Usability Study of GNOME

Last summer, I mentored a usability testing project for Outreachy, working with Gina to examine the usability of GNOME.

Gina did a great job in her usability work. After her internship was complete, we authored a paper together, which FOSS Force has published. Please read A Usability Study of GNOME at FOSS Force.

Here's a brief excerpt from the article, describing the usability test:

During a Summer 2015 internship with Outreachy, an organization that helps underrepresented groups get involved in free and open source software, we conducted a usability test of the GNOME desktop. This was a formal usability test, where we invited a dozen testers to use GNOME and GNOME applications to complete a few simple tasks.

Test volunteers were about evenly divided between men and women (slightly more men than women) predominantly at college age (12 to 25) representing all levels of computer skill but describing mostly “constant” computer use. About half of testers had not used GNOME previously (slightly more had not than had).

Participants were free to choose the language version of GNOME. Five of them chose to test GNOME in French, while the other seven used GNOME in English. The choice of language did not appear to influence the results of the usability test.

In our test, we presented each tester a set of sample tasks, one task at a time. Throughout the test, we watched each volunteer as they completed the sample tasks, and noted any problems they had with the software. We asked the testers to speak aloud during the usability test, to describe what they were looking for; if they were looking for the “Print” button, they should say, “I’m looking for the ‘Print’ button.” After each set of tasks, we took a “comment break” so participants could share their thoughts about the software and the problems they encountered.

The total test duration varied, with the shortest test at thirty minutes and the longest at an hour and a half.