Analyzing the Option Effects of Difficult TOEFL Items with Low Biserials: Methods Developed for Use by Test Assemblers.

Hicks, Marilyn M.

Several exploratory analyses of the fifths data generated by Test of English as a Foreign Language (TOEFL) item analyses were developed in order to evaluate the effects of options on the discriminability of difficult items and to identify difficult items with low, unreliable biserials that had been rejected by test developers, but for which acceptable a-parameters are probably estimable. Intended for use by test assemblers subsequent to an item analysis, the methods were mainly graphical, but included the evaluation of a distance measure and other simple statistics. Localized option effects occur that can impair item discrimination as well as the fit of the item response theory model. The negative impact of these effects on model fit was illustrated, and methods were suggested for analyzing them. An index was also developed to identify very difficult items in which the r-biserial restricts the ability of test developers to construct tests with effective measurement properties at high score levels. Implications of items with nonmonotonic response patterns due to option effects were also discussed. Appendix A presents TOEFL item response functions and Appendix B describes some features of correspondence analysis. (Contains 12 figures and 5 references.) (SLD)