Formative and Summative usability testing

October 10, 2017

Formative and Summative usability tests in usability circles offer two distinct advantages. While both deal in assessing the quality of a user interface, these two forms of tests do so from different perspectives.

Formative usability tests are qualitative—they are used to identify the ease or difficulty of design features early in the design phase. It acts as a tool to assist in making an interface more usable. Decisions are made from the data gained from Formative usability tests, which will mold and improve the design of the UI system. Formative usability tests are more about discovery—discovery of problems, issues and difficulties that can be solved and that will move the iteration process forward. The insights gained through Formative usability tests are observational and are conducted with a moderator present as well as others from the product design team. This can be done on-site or in a lab setting.

In comparison, Summative usability tests are quantitative— they act as a final validation that possible problems and issues have been identified and adequately addressed, and that the impact of lingering trouble spots are minimized to an acceptable level. Sometimes referred to as human factors validation testing, Summative usability tests are usually performed for FDA medical products later in the design and development process and help validate that all FDA requirements have been met. The FDA requires stringent documentation of the design process as well as the usability test process and results. Medical device manufacturers must show their medical device reflects good human factors engineering practices, and that it allows users to perform tasks with little to no chance of error that could harm patients or themselves.

In addition, Summative usability testing can ensure the detail logic as well as iconography meet user requirements before the product is put to market. It can also be useful in providing data that validates marketing claims. Summative tests strengthen and justify Formative testing results.

Of course, an important component to both types of tests is defining and determining who the end users will be. Interviewing intended users is of utmost importance and should also be used throughout the entire design and development process. In addition, having the right number of participants per group is essential to achieving valid test results. Since Formative usability testing is as much observational as statistical, a smaller number of participants can yield valuable results as long as those observing the participants know what to look for. Summative testing for the FDA requires a minimum of 15 participants for each user group. Summative test results tend to be more statistical, so the larger number of participants used the more accurate the resulting data.