To link to the entire object, paste this link in email, IM or documentTo embed the entire object, paste this HTML in websiteTo link to this page, paste this link in email, IM or documentTo embed this page, paste this HTML in website

A FUNCTIONAL USE OF RESPONSE TIME DATA IN COGNITIVE ASSESSMENT
by
John Janson Prindle
____________________________________________________________________________
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(PSYCHOLOGY)
May 2012
Copyright 2012 John Janson Prindle

The stimulus for the current body of work comes from the desire of researchers to have concise and accurate cognitive tasks implemented in their surveys. The main purpose of this work was to introduce collateral information as a way of making up for lost or forgone information when adaptive frameworks are adopted. The nature of ongoing surveys and the ubiquity of computers provides ample collateral, or nonintrusive, information which can help improve score accuracy. Information such as how long a respondent spends on certain items, their age, education, and other characteristics can improve score prediction beyond simple item responses. ❧ The importance of this work included methods to effectively decrease the number of items given to participants, as well as keep the accuracy high despite the loss in information. In the current study, the Woodcock Johnson – III (WJ-III) Number Series (NS) task was presented with 30 previously unpublished items as stimuli. First, a couple of scoring models were implemented to test for model fit and compare the implications of the fit values. Then methods outlined below systematically adjusted patterns of missingness to mimic reduced and adapted subsets. Once the smaller NS item sets were delineated, several methods of adding predictive accuracy were tested and compared. ❧ In scoring respondents a traditional Item Response Theory (IRT) model as proposed by Rasch (1960) was first used to provide evidence for a uni-dimensional scale and obtain baseline statistics for item difficulty and person abilities. The next model was a Conditionally Independent Response Time (CIRT) model. The latter model includes a response model as well as a joint response time model for scoring. It was shown that with the full item set these two models provide identical ability estimates and item parameters. The response time model of the CIRT framework provides ability scores and speededness scores based on response time patterns. ❧ Next, focus was placed on effectively decreasing the number of items used in scoring each respondent. Methods included item reduction, test forms in which the same item sets were used to score each respondent, and adaptive tests, where each respondent could receive a different item set. Reduced item sets fared better when item difficulties more closely matched sample ability levels (r=0.72-0.90). Adaptive item sets were more consistent in measuring ability (i.e. half-adaptive, block adaptive, fully adaptive), but accuracy was best for the fully adaptive method used (r=0.79-0.91). ❧ The last steps of analysis involved introducing response time and demographic variables as additional predictors of the 30 item scores. Item response, response times, and response/response time interactions provided small improvements in explained variance when used as predictors (1-8%). When CIRT ability and speededness scores were used as predictors, speededness provided limited improvements (<1%) to prediction. The addition of age, education, and gender to response models improved explained variance to a moderate degree (1-5%). ❧ In conclusion, we note that sample had a higher than average ability level for the NS task and this should color our findings for the methods outlined. The item sets that did not match respondent abilities as well were improved more so by response time and demographic data. If one can correctly identify the ability ranges of a sample before administration, then a more focused reduced item set would be advantageous. Adaptive item sets seem advantageous in a more general testing situation where ability levels are more variable. The advantage of using collateral information in predicting cognitive scores is the amount of time saved by omitting items, potentially lowering costs, and allowing researchers to move onto more tasks if desired. While the improvement due to response time in these methods was limited with NS, there is a good foundation for other cognitive tasks administered in computer assisted designs.

The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

A FUNCTIONAL USE OF RESPONSE TIME DATA IN COGNITIVE ASSESSMENT
by
John Janson Prindle
____________________________________________________________________________
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(PSYCHOLOGY)
May 2012
Copyright 2012 John Janson Prindle