Project Description

BioVisualSpeech is a research project that aims to research natural and multimodal
interaction mechanisms for providing bio-feedback in speech therapy through the use of serious
(computer) games. This project proposes a platform for speech therapy that has two main goals:
the first goal is to keep the patients engaged in the exercises in order to have more fruitful
sessions. For this, the platform includes a gamified interactive environment that uses visual
stimuli, several serious games, and a reward system that motivates children to keep exercising.
The environment provides instantaneous bio-feedback, replacing other tools such as a mirror,
and assists the therapist during the session with audio-visual recordings and annotations of
the session. The second goal aims at providing the therapist with a toolset to plan the course
of the ongoing therapy session. For this, the platform will include an application for post-session
analysis where a patient’s audio-visual recordings can be examined through browsing and searching,
and a future session can be planned. The novelty of this platform is the combination of visual
bio-feedback and the gamification of the speech therapy. The gamified interactive environment
is proposed as a key instrument to keep the patient engaged and reactive to the instantaneous
bio-feedback provided by the platform.