Paradata: Its Role in Online Assessment

How Paradata Analysis Helps Recruiters

The 20th annual congress of the society of applied psychology for Germany (the GWP) was held in Wernigerode, Germany and we were delighted to present some of the research by our scientific team. This blog summarises one of these papers.

As innovators in online assessment we are keen to present what we do. Maximilian Jansen presented a literature review and study which explored the insights that could be gained when looking at data sets generated by those completing online assessments.

He argued that one of the possibilities that the move towards greater digitisation and more game-based assessment brings, is the collection of so-called paradata.

This is data generated from the test taker as he or she responds to the assessment. This includes the data relating to the test that is available to the test user – such as display times of certain elements – as well as the data collected by the server hosting the assessment, such as changing the displayed page. Paradata is also available in other forms of assessment such as the interview. For example, the fluidity and speed of speech, and intonation, can now be measured.

The study presented by Maximilian focused on the processing times and button clicks of test takers. We see that both are valuable as they offer us additional and new information to help our understanding of how an assessment works. The research shows that processing times are heavily influenced by the length of an item – and we need to be aware of this when looking at paradata variables. The review also highlighted that need for a specially adapted text analysis tool to be developed. Several authors have called for this before but being able to review text analysis would be of real value in test development scenarios.

The benefits for further investigation are clear. Not only are there upsides of using paradata for quality assurance and in test development, but future use of paradata could enhance data analyses and further inform decision-making.

It’s clearly a fast-developing area of assessment, and one which we are proud to be part of.

About the Author

Richard Justenhoven is the product development director within Aon's Assessment Solutions. A leading organizational psychologist, Richard is an acknowledged expert in the design, implementation and evaluation of online assessments and a sought after speaker about such topics.