This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background

Assessments of understanding (AoUs) in clinical trials are often composed of true/false
multiple choice questions, however, these tools can be difficult for volunteers with
limited education or without prior testing experience.

Methods

35 adults were recruited at two research centers in Southern Africa. A within-subjects,
repeated measures design was used, whereby each volunteer served as his /her own control.
An AoU tool with closed- and open-ended questions was administered within a hypothetical
AIDS vaccine trial setting. Performance on closed- and open-ended questions was compared
using correlations and repeated-measure t-tests, limited to 4 complex concepts: false
sense of security, risk of false positive test, need for contraception, and potentially
enhanced susceptibility.

Results

Mean scores of understanding for each concept assessed by closed-ended questions ranged
from 0.73 (need for contraception) to 0.84 (risk of false positive test); and by open-ended
questions from 0.4 (risk of false positive test) – 0.6 (need for contraception). Scores
for the open-ended measure were all lower than the equivalent closed-ended measure.
Correlations between the closed- and open-ended measures were generally low, achieving
significance for false sense of security (r=0.377), potentially enhanced susceptibility
(r=0.393), and total score across concepts (r=0.617). Volunteers’ understanding as
assessed by the closed- and open-ended methods differed significantly: false sense
of security= -3.862; risk of false positive test= -7.210; need for contraception=
-2.303; and potentially enhanced susceptibility= -8.007. The correlation with years
of education was consistently and significantly higher for the open-ended measure
than the true/false questionnaire with the exception of need for contraception.

Conclusion

The results suggest the qualitative measure better assesses understanding than the
quantitative measure. The scores from the two assessment methods have limited interchangeability.
The standard closed-ended questions appear to provide an inflated measure of volunteers’
understanding. An assessment tool with closed- and open-ended questions is better
suited to determine genuine understanding.