Reliability and Validity of Adaptive and Conventional Tests in a Military Recruit Population. Research Report 83-1.

Martin, John T.; And Others

A conventional verbal ability test and a Bayesian adaptive verbal ability test were compared using a variety of psychometric criteria. Tests were administered to 550 Marine recruits, half of whom received two 30-item alternate forms of a conventional test and half of whom received two 30-item alternate forms of a Bayesian adaptive test. Both types of tests were computer administered and were followed by a 50-item conventional verbal ability criterion test. The results of this study supported the feasibility and psychometric superiority of computer-administered adaptive tests as replacements for paper-and-pencil administered conventional tests in a military testing environment. On an item-for-item basis, the adaptive tests took slightly longer than the conventional tests; but with testing time held constant, the adaptive tests obtained substantially higher levels of both reliability and validity than did the conventional tests. The data showed that to obtain equal reliabilities, adaptive tests could administer 50 percent fewer items than the conventional tests; adaptive tests could also achieve the same level of validity as the conventional tests using only one-third the number of items, supporting earlier validity data reported by Thompson and Weiss (1980) on college students. (PN)

Sponsor: Office of Naval Research, Arlington, VA. Personnel and Training Research Programs Office.; Army Research Inst. for the Behavioral and Social Sciences, Arlington, VA.; Air Force Office of Scientific Research, Arlington, VA.