Audiovisual Speech to Children

This study is really about how parents speak differently to children verses how they speak to adults, and anyone who has been to a playground for thirty seconds has seen that they’ll use a different speaking style when they are talking to infants and children.

We are trying to understand why parents do this and what this might do to help kids developmentally.

What we are doing in this study is we are having mothers participating with their children in a sort of interactive speech perception task. Mothers and children are communicating via by a very sophisticated version of Skype that we’ve developed here. Mothers and children can see each other on screen and can talk through a microphone and hear each other through headphones. The mothers’ task is to try to get their kids to select the correct item that they see on an iPad screen. They are doing this in a context of noise, so the moms and the kids are hearing noise over the headphones. It is sort of challenging and we’re interested in the sort of acoustical changes that mothers make in their speech to try and highlight differences to make their speech more clear.

When you hear someone talk you are both seeing their face and their face is producing sound, and these things are correlated and we use faces to both product speech but also to express emotion so we are collecting video of the mothers’ speech. We are then analyzing the video using some computer vision software that tracks facial features so we can look at measures of jaw openness and mouth width, eyebrow height and nostril flare and all these different things at a very fine grain level. Using this software we can look at the mother’s facial movements and see whether there are similar exaggerations in the visual part of the signal as we see in the auditory part and so this might be really important when it comes to children with hearing loss because they might have an increased reliance on that visual component. Ultimately I think this work is going to provide some real basic knowledge about how this parent child language learning system works and how it adapts in response to communicative challenges like hearing loss, and I think by understanding how parents can adapt to their children’s developmental needs then we can better guide this sort of remediation strategies that’ll be useful to promote positive outcomes for these children.

Page Content

Caregivers modify the way they speak when talking to infants and young children. A developmental psychologist at Boys Town National Research Hospital, discusses experiments in his lab that examine the acoustical and visible properties of child-directed speech, and their role in speech and language development in children.