From the archive: A comparative evaluation to explore the effects of document design principles

In our new section ‘From the archive’ we are highlighting papers that are ‘worth going back to’, that we hold in hard copy here in the Department but may be hard to come by in any other way. Some of them are relevant to our current research, and others are just interesting for us. Enjoy!

In this paper, the authors briefly present a model for document design comprising a sequence of stages of iterative design and user feedback.They then demonstrate the model’s effectiveness in a study of a Federal Communications Commission (FCC) manual of regulations for the use of radios in recreational boats. The study compared performance using the original version to a version revised according to document design principles, where the language, length, information structure and typographic presentation were adjusted. The original manual was 49 pages long and required an index to help users search for information; the new version covered the same topics as the original but was organised into 22 rules in an 11-page booklet.

In the study, 53 people with boating experience and 52 without experience used the manual to answer questions about its content. Half the participants in each group were given the original manual, the other half the re-designed version.

The results suggested the re-designed version was significantly easier to use for both experienced and inexperienced boaters: participants with the new rules answered more questions correctly, identified more rules correctly, carrying out the task overall more quickly. Most interestingly, the difference in performance between experienced and inexperienced boaters disappeared with the new rules – that is inexperienced users benefited particularly from the document revision.

The approach used by Redish and colleagues in this study is frequently used in information design research. Particularly powerful effects of information design for people with lower levels of knowledge or confidence seems to be a common theme. For example, a study conducted by Loorbach, Karreman and Steehouder (2007) found the impact of re-design of a phone manual was felt more strongly by users who lacked experience and confidence; similarly Dickinson et al. (2010) found that people who were less confident readers and with lower education attainment benefited most from re-design of a patient information leaflet for medicines; Nielsen (2005) has also found that, while revisions to web sites, such as prioritising key content and simplifying page structure, benefit all users, these benefits are felt particularly by less proficient readers.

The lesson for researchers and practitioners is to include a range of participants, with different levels of expertise, reading abilities and confidence levels, in studies and tests of information design, in order to fully explore the impact of design changes.