Collaborative Problem Solving

Working with our partners at Digital Artefacts, we developed a novel, simulation-based approach to elicit skill evidence in a task-oriented, scripted agent environment. Our assessment instrument is an immersive puzzle game where a student collaborates with a computer agent to solve a number of challenges presented at locked gates in a maze, and then solve the final challenge of repairing a virtual circuit.

The team recently designed the agent/bot dialog trees and the scoring rules, and we are collecting data via crowdsourced game play. Work with Dr. Yigal Rosen from Harvard University, we’re analyzing the rich process data that comes from the log files using data mining and psychometric methods, then refining the game’s alignment with ACT’s Holistic Framework. In the second part of the study, we will modify the game to allow two students to collaborate directly on solving the same set of puzzles.

“Multimodal Analytics for Teamwork Assessment under Multiple Casualties” is a US Army-funded project in its second year. ACTNext’s Alina von Davier started as the principal investigator, now succeeded by Saad Khan of ETS. The goal of the project is to use sensors, cameras, microphones, and human raters to collect and analyze data from medical military teams under multiple casualty scenarios.