RealSense Play: Helping children explore new worlds with old friends.

RealSense Play

Client: Intel

Time Frame: 3 Weeks

Team: Scott Trepper, Shengfan He, and Prafulla Badeanahalli

The Challenge

Intel RealSenseTM is a technology which uses a camera that is able to sense a depth image along with an RGB image. The camera is able to tell where a point is in 3D space along with it's color. Because of this small form factor, the camera can be embedded into various devices like wearable, drones, robots, environments, computers, tablets, phones, etc.

Our goal was to come up with a compelling use case for the Intel RealSenseTM R200 camera as a sensor in any space of our choice in an indoor setting to improve the users experience within or related to that space.

Design Process

We started by analyzing the capabilities of the camera, looked at examples of what had already been done, sketched various ideas, voted down on some while developing others. Conducted primary research to understand how children play with toys. Finally, we designed RealSense Play.

Our Design

Above graphics created by Shengfan (Frank) He

Child Toy Interaction

Sound is emitted by the nearby speakers when a child interacts with a toy. Touching a toy, picking it up, petting it and other different kinds of gestures are able to trigger different sounds.

When Jake picks up his dog and it says "I missed you."

Toy Toy Interaction

Children often play with multiple toys at the same time. New sounds can be triggered when the child interacts with more than one toy to engage them in the stories they create.

Monster growls and the dog barks when Jake brings both these toys together.

Ambient Sensing

Even without any interaction, play can be initiated by the child's behavior. Sitting quietly in a room, reading a story etc all can open up possibilities to play with the toys.

"Children form emotional bonds with their toys. RealSense Play allows them to interact in new ways with their own toys."

Details of the Design Process

My Role

Ideation

We had more time for the ideation phase in this project when compared to other projects. But it was not an easy task. I tried to come up with as many ideas as I could.

Primary research

I took the task of interviewing one of the parents to understand how her child plays with toys.

Team Collaboration

I collaborated with other designers in the team by asking for feedback on my design and providing feedback on their. I contributed my thoughts in the team discussion and tried coming up with alternative course of action when there was a conflict.

Video Editing

I was also responsible for editing the final video. And also wrote the narration for the final video.

Client Interview

We interviewed the client to better understand what they expected from us. They made it clear that they did not want any obvious use case that anyone could have thought of. They were also interested in seeing how the RealSense technology could work in conjunction with other technologies.

Analyzing Capabilities of the Camera

We started by analyzing the technological capabilities and limitations of the RealSense camera. We looked at examples of how Intel and other developers had utilized the camera.

Sketching Concepts

We then sketched ideas. We gave ourselves a time frame and tried to come up with as many concepts as we could. Sketches were used to visually articulate our concepts with each other. As we sketched our ideas, we again analyzed the capabilities and limitations of the camera.

Efficient Placement of Books and Items

Improve Productivity

Remote Meetings

Modelling and Simulation

Real Time Photo Manipulation

Interactive Diary

Improve Handwriting

Shadow Puppets

Affinity Diagramming

We used affinity diagramming to group our ideas. Affinity diagramming helped form organizations and recognize patterns amongst our ideas.

Picking Directions

Rather than picking ideas based on our feeling or fuzzy judgement, we applied a more rational and rigorous method to decide the direction.

We rated each direction by four criteria: Impressiveness, relevance, implementation, and meaningfulness. This process helped us remain objective while weighing important design qualities.

We voted on continuing with the idea of using the RealSense camera in a parent-child setting wherein the RealSense camera would help the parent in creating shadow puppets while telling a story to their kid.

Primary Research

We later conducted primary research to validate our assumptions about how children play with their toys. The main insight that we got from the interview is that children communicate with their toys while playing with them.

"She has 13 to 14 barbies. Each barbie is named. She dresses them up, does their hair, puts make up on them. She then takes the barbie to school and gets them to do homework. She is talking to the barbie while she does all of this."

"She imitates the characters she sees on TV. She imitates what we parents do, such as working on computer, playing, pampering etc. If I scold her, then she does the same with the barbie."

-Parent of a 7 year old

Core Idea

We felt that the use of RealSense camera to generate shadow puppets would not make full use of the capabilities of the technology. So, we discussed and sketched different variations and iterations of the idea. Later, the idea evolved into making use of the camera to turn regular toys (such as a Teddy Bear) smart to engage children to have deeper interactions with their toys.

Implementation Strategy

Intel would sell the RealSense camera along with two to four portable speakers that can be attached to a kid's favorite toy.

The portable speakers can be plugged into a computer to download the required packages related to the toy.

Additional packages are also available, which a parent can install, to facilitate learning and development.

Storyboarding

Finally, we sketched out the storyboard of a kids journey which would later help us in making the video.

Feedback from Intel

They were impressed with the three interaction scenario's (kid-toy, toy-toy, and ambient sensing) that were shown. What they found most interesting was how we made use of a normal stuffed objects such as a toy with a digital object such as a camera. They also gave some recommendations on future directions such as storytelling and learning with the help of toys.