Masters Thesis Defense “Development of a human-AI teaming based mobile language learning solution for dual language learners in early and special educations” By Saurabh Shukla

Learning English as a secondary language is often an overwhelming challenge for dual language learners (DLLs), whose first language (L1) is not English, especially for children in early education (PreK-3 age group). These early DLLs need to devote a considerable amount of time learning to speak and read the language, in order to gain the language proficiency to function and compete in the classroom.

The process of learning a new language can benefit greatly from the latest computing technologies, such as mobile computing, augmented reality and artificial intelligence. This research focuses on developing a human-AI teaming based mobile learning system for early DLLs. The objective is to provide a supportive and interactive platform for them to develop English reading and pronunciation skills through individual attention and interactive coaching.

In this thesis, we present an AR and AI-based mobile learning tool that provides: 1.) automatic and accurate intelligibility analysis at various levels: letter, word, phrase and sentences, 2.) immediate feedback and multimodal coaching on how to correct pronunciation, and 3.) evidence-based dynamic training curriculum tailored to each individual’s learning patterns and needs, e.g., retention of corrected pronunciation and typical pronunciation errors. The use of visible and interactive virtual expert technology capable of intuitive AR-based interactions will greatly increase a student’s acceptance and retention of a virtual coach. In school or at home, it will readily resemble an expert reading specialist to effectively guide and assist a student in practicing reading and speaking by him-/herself independently, which is particularly important for DLL as many of their parents don’t speak English fluently and cannot offer the necessary help.