Abstract

Modeling student learning during tutorial interaction is a central problem in intelligent tutoring systems. While many modeling techniques have been developed to address this problem, most of them focus on cognitive models in conjunction with often-complex domain models. This paper presents an analysis suggesting that observing students’ multimodal behaviors may provide deep insight into student learning at critical moments in a tutorial session. In particular, this work examines student facial expression, electrodermal activity, posture, and gesture immediately following inference questions posed by human tutors. The findings show that for human-human task-oriented tutorial dialogue, facial expression and skin conductance response following tutor inference questions are highly predictive of student learning gains. These findings suggest that with multimodal behavior data, intelligent tutoring systems can make more informed adaptive decisions to support students effectively.

Notes

Acknowledgements

The authors wish to thank the members of the LearnDialogue and Intellimedia groups at North Carolina State University for their helpful input. This work is supported in part by the Department of Computer Science at North Carolina State University and the National Science Foundation through Grants IIS-1409639, CNS-1453520, and a Graduate Research Fellowship. Any opinions, findings, conclusions, or recommendations expressed in this report are those of the participants, and do not necessarily represent the official views, opinions, or policy of the National Science Foundation.

Oviatt, S., Cohen, A.: Written and multimodal representations as predictors of expertise and problem-solving success in mathematics. In: Proceedings of the 15th International Conference on Multimodal Interaction, pp. 599–606 (2013)Google Scholar

Graesser, A.C., Olde, B.A.: How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. J. Educ. Psychol. 95(3), 524–536 (2003)CrossRefGoogle Scholar