BABY GAGA

AI FOR CHILD SPEECH ACQUISITION

Can we empower parents to help their baby develop speech skills? Baby Gaga is an AI system that monitors a child’s progress and gives suggestions to parents— suggestions that are backed by metrics and delivered in a personal and sensitive way designed to empower parents.

Problem

Language: it’s one of the most important aspects of a child’s development. Yet due to lack of time, mental energy, and socio-economic factors, it can be hard for parents to give their child what they need.

OUTCOME

Using an IBM design thinking framework, we designed an AI tool that propels a child’s language development through sensitive and actionable advice to parents, based on an accurate analysis of their child’s speech capabilities.

ROLE

In this team of 6, we all played a part in every aspect of the project: design thinking, primary and secondary research, mockups, prototyping, and presentation.

RESEARCH METHODS

Competitive Analysis \\ Lit Review \\ Interviews

Competitive Analysis

The only product on the market that provides analysis of a child’s speech development is LENA. This is an expensive product ($300) that is prescribed by researchers and therapists to children with development issues. The product is bulky and requires a researcher to take the device back a lab for analysis. This product is inaccessible and raises issues of privacy. The pro of LENA is that it is a highly researched tool, and we were able to use the literature to begin our design process.

Literature Review

We reviewed papers in the fields of 1) child language development, 2) Embedded systems for children, 3) Voice/Speech recognition models and NLP, and 4) Future predictions of where AI language models are taking us in the future. Overall, our research revealed that AI applied to language is a rapidly advancing area of research. The research on speech acquisition for children ages 0-3 has relied largely upon anecdotal evidence until now, as data becomes more available (i.e. Deb Roy at MIT Media Labs).

INTERVIEWS

We conducted 14 interviews with both parents and speech pathologists in order to understand the process of raising a child as they develop speech capabilities— and to understand where the biggest problems are. We also interviewed researchers in the fields of information retrieval, computational linguistics, NLP, and child education.

Interviews - Parents by MURAL

You will be able to edit this mural.

Interviews - SLPs by MURAL

You will be able to edit this mural.

Problem

Based on our interviews, we refined the problems space.

When it comes to speech development of a child, it can be difficult to distinguish which method works. Without guidance, parents can feel lost. The abundance of information on the web and advice from doctors, friends, and family can often overwhelm parents. As a result, parents are left feeling frustrated because they aren’t able to make much progress in their child’s speech development. What we need is an application that can guide the parents to the right direction with the appropriate advice.

Key Insights

We pulled a lot of data from our research process. Below are a few key insights that were vital to understand. Many of these insights were surprising and led to pivot moments leading into our design thinking.

Parents seek detailed information about child development when they notice an issue or delay

“There’s always a bigger milestone to work on. We refer to a child development book when we need help with stuff like potty training or behavior issues, and then we’ll look at all the other information for that age.”

Parents use child comparison as a measurement of success in developmental milestones

”Another kid was winning”, “You’re always comparing to the most advanced kids,” & “It’s hard to know what your standards should be.”

Primary Caregivers want full control of their child’s data

“We returned our Alexa because we weren’t sure if she was recording us or what was happening with our data.”

Analyzed data should be easy to interpret with actionable suggestions

“If I had a curriculum of ‘read from this time to this time, then play with this specific toy to develop this,” & “I feel like I’m always playing catch up on what she should be doing.”

Parents rely heavily on their social and familial networks for developmental milestones

”A lot of our friends have kids so I talk to my friends about what their children were doing at this age.”

DESIGN THINKING

I had never done product development with such a large team with individuals from so many backgrounds. I think what really helped us was starting off with the IBM practitioner short course, which helped to align our team on the main goals of the project. We used a 9 step design thinking framework by IBM that had AI specific tasks. Some tasks worked for our team while others simple didn’t. Based on our feedback, Jennifer Sukis has already been able to make changes to this framework! In particular, I found it necessary to address some of the data privacy, security, and explainability to help lead us to an ethical and transparent product.

USER Profile

The results of our interviews suggested that there were several types of users with different needs. Through accumulating different insights and common patterns, we narrowed our focus to the most common user. With this, we created a user persona, Emily. Focusing on this specific user allowed us to reign in our product design. We constantly returned to as “Is this going to help Emily? How?”

USER JOURNEY

Next, we walked through the process of rearing a child and helping to develop their speech ability. Unlike most user journeys, ours was very broad, and involves many cyclical tasks and feedback loops. This presented the biggest challenge for the project, but by keeping it intentionally broad in our user journey, we were forced to think about the complexities of the bigger picture before narrowing it down later. We also imagined ares in which AI could be applied to the user journey to lessen the burden parents face.

Framing the Problem

To narrow down the frame of the problem, we analyzed potential user and business frameworks. The clearest user is the primary caregiver (the most active parent). There are many ways to revolutionize speech acquisition for children ages 0-3 if we think big— it could eventually evolve a paradigm shift in our communities and way of thinking. For the immediate use of the parent, we decided to use an app as a platform to provide useful features, communication, and reports.

For the rest of our design thinking process, you can see detailed work on our mural. We invite you to take a look!

Design Thinking by MURAL

You will be able to edit this mural.

SOLUTION

Introducing Baby Gaga….Based on our research and design thinking process, we created an AI tool that propels a child’s language development through sensitive and actionable advice to parents, based on an accurate analyses of their child’s speech capabilities. We are still developing our solution, so here you will find low fidelity prototypes that follow the experience principles established through our process.

PITCH

User Experience Principles

These three principles were developed through our research and design thinking process. At each phase of design, we returned to ask whether our decisions reflected principles of sensitivity, control, and empowerment for the parents.

Sensitivity: Parents will be presented with feedback in an encouraging way

Control: Parents get to set the goals (based on well researched suggestions by the AI) and parents have control over the privacy and data.

User Flowchart

Mapping the basic flow of the app forced us to figure each step on the path the users will take throughout the solution. We are currently updating this flowchart to reflect the process of user adoption.

Storyboard

We also storyboarded out the user flow, to provide further context for our designers.

Wireframes

We then sketched out the basic information architecture and began to sketch wireframes. This low fidelity prototype allowed us to quickly pivot as we did user testing.

We are currently in the process of designing the app and are iterating on our design as we do user testing. We had to make a big pivot as we did user testing, which involved the choice to exclude data visual metrics and comparison of a child’s development to other children or national averages. Although parents said they wanted this in their interviews, when confronted with the actual metrics, they found is upsetting— they didn’t want to think of their child as being “measured.” We are currently pivoting towards using notifications (through audio and messages) to inform and encourage parents based on metrics on the backend.

Backend

Speech and voice recognition are a complex thing? We imagine our product being viable in about 20 years, when we have training models that can accurately distinguish between multiple voices, understand contexts based on audio input and other data sources, and develop personalized experiences for users.