Making situational judgement tests mobile

The next step for Situational Judgement Tests

We’re looking forward to presenting at SIOP in Chicago. It’s an opportunity to share our research and findings with the wider organisational, occupation and business psychology world and to hear their thoughts on what we are doing. This year we are delighted to have a number of colleagues taking part in numerous panel discussions and presentations. I’ll showcase some of them in the coming weeks.

Together with Tara Johnson and Eleni Lobene from Aon, I shall be part of a panel of practitioners from some of the leading consulting organisations discussing the current and future state of mobile simulations or situational judgement tests.

We know that increasing numbers of organisations are adopting technology-enabled assessments including media-rich simulations. But creating these mobile-enabled simulations is a complex process, and there is little in the way of research or guidance in the literature.

We’ve been developing our capability over a number of years, finding out what works and what doesn’t. We know that Situational Judgement Tests (SJTs) are popular as an assessment tool because of their criterion-related validity (Lievens, Peeters, & Schollaert, 2008). However, the situations created are artificial to a certain extent, particularly when it comes to social interaction. This affects criterion-related validity (Lievens & Sackett, 2006). Gamification, i.e., incorporating gaming elements into the assessment process (Deterding, Dixon, Khaled, & Nacke, 2011) as well as transforming them into a mobile environment (King, et al., 2015) is likely to make candidates immerse in their role and make scenarios more realistic.

We have transformed situational judgement tests (or SJTs) into a mobile-first simulation called chatAssess. It’s customisable, and fully mobile and has received an excited and positive reaction by our clients. It simulates real-time instant messaging like WhatsApp. Participants take on the role of an employee working with a company and are given their colleagues’ profiles. They receive text and picture messages from different colleagues, but can also receive external links or invitations to additional ability tests. When candidates respond to messages, there is a response from the other person that the candidate can respond to again, which then creates a simulated instant messaging interaction flow in which lots of different elements such as tests or video interviews can be embedded. There are some limitations with regard to the current response pattern and the way forward is likely to be an open response format where candidate’s responses are run through a Natural Language Processing API which uses text analytics to understand the structure of sentences and their meaning and intention through statistical methods and machine learning. But that’s for the future.

Right now, chatAssess is already firmly embedded in the recruitment processes of some great clients. You can learn more about chatAssess here

We hope that our session at SIOP will highlight work that is currently being done in this area, as well as share key learnings, best practices, and discuss what the future holds in this area.

About the Author

Richard Justenhoven is the product development director within Aon's Assessment Solutions. A leading organizational psychologist, Richard is an acknowledged expert in the design, implementation and evaluation of online assessments and a sought after speaker about such topics.