Overview

While students at Austin Center for Design (AC4D), classmate Kylie Jack and I co-founded Kites & Ladders, which developed products to amplify the voices of people on the autism spectrum. Working primarily with nonverbal autistic individuals and their families, we were designing tools to support independence, communication, meaningful relationships, and participation in broader society.

The Team

I conducted the initial research and synthesis and developed an early version of the Visual Voice product system concept along with a storyboard. While we worked closely throughout the prototyping phase, Kylie focused on developing the hardware prototypes while my emphasis was on testing, visual design, and prototyping a tablet app.

The Process

Classmate Eli Robinson and I conducted the initial Contextual Inquiries and Participatory Interviews that seeded Kites & Ladders in fall 2012. We identified key insights, which I used to create a storyboard, a theory of change, and the initial Visual Voice concept to support creative self-expression and communication for nonverbal autistic individuals. Kylie and I worked together to prototype and test the Visual Voice system (which included a wearable biofeedback device, a camera app, and a tablet app), before sharpening our focus on prototyping the wearable device.

Design Research

Research Focus

Our research centered on how people with special needs use technology in school and beyond. We conducted contextual inquiries to understand the landscape and learn about the scope of “adaptive technology.” From there, we narrowed our focus to autism and conducted participatory interviews with moms of nonverbal individuals on the spectrum.

Several nonverbal students use Augmentative and Alternative Communication systems that consist of a speaker board pre-populated with images and words. Teachers and caregivers update the vocabulary based on the student’s needs.

A teacher uses a tactile time-telling system with a student who is deaf and blind. The system helps the student anticipate upcoming activities and reduces his anxiety.

A student uses a calculator for a math game. Even common technology excites and engages students, who change their behavior to keep calculator privileges.

Contextual Inquiries

During contextual inquiries in schools, we observed students using a range of technologies: calculators, computers, specialized communication devices, and more. We were intrigued by the use of an iPad between an autistic student and his aide and focused subsequent research on adaptive technology and autism spectrum disorders.

“[With the iPad,] he gets more interested in talking, overcoming the social impairments of Autism along with the communication impairments.” - Ann

“I can’t think of anything that will be a greater determinate of his future than technology. It could unlock his mind. We can’t survive without it.” - Sara

“The camera...helps you know how he’s experiencing life. It’s an insight into his sense of humor.” - Lynn

Participatory Interviews

We conducted participatory interviews with mothers of non-verbal autistic children. Beginning with a journaling activity, we discussed their experiences with technology and moved into hands-on activities that utilized tangible stimuli such as pictures and words to help the participant tell stories and articulate a more ideal future state.

Jacob: A Portrait of Autism

On one of our research outings, we met Jacob. Jacob is like a lot of other 12 year olds, except he doesn’t speak in words that other people understand. He has been diagnosed on the autism spectrum and has Down’s Syndrome. When he was younger, his mom felt like she didn’t really know him—until he started using a camera. Jacob began taking pictures. Lots of them. He started showing them to family members to communicate how he saw the world.

In our interview with Jacob’s mom, she recounted a trip to the store where Jacob flipped his camera into video mode and placed it on the conveyor belt. When the clerk picked up the camera with a puzzled expression, Jacob burst out laughing.

At this moment, Jacob’s mom realized he had a sense of humor.

Synthesis

Concept Models

We transcribed and externalized our research data onto the wall. Then, using a mix of affinity diagrams and visual diagrams and models, we synthesized our research into insights.

A cultural model visualizes the interactions among students, teachers, and the aide in a Special Education classroom. Lightning bolts indicate breakdowns or workarounds.

A physical model reveals movement of students and teachers in the classroom during our contextual inquiry.

A flow model depicts physical objects used during the class period and how they flowed between students and teacher.

Insights

Several key insights emerged highlighting how technology enables and motivates connection and self-expression, especially for students with communication challenges.

However, a student’s adoption of a new technology is unpredictable and depends on physical abilities and emotional needs. Parents and educators often prefer mainstream devices that are flexible and don’t differentiate the student with special needs from peers. Plus, others will use the device if the autistic individual will not.

Social ConnectionFor a population that is isolated, technology promotes and nurtures social relationships.

Barriers to EntrySmall obstacles and repeated disappointments feel insurmountable in a situation that is already challenging.

Fitting InMainstream and wearable devices can support people with special needs without differentiating them from their peers.

Product System

Theory of Change

We developed a theory of change that identified how people felt during diagnosis and day-to-day management of autism, the desired future state, and characteristics of products that could help achieve the ideal state.

Design Pillars

To guide our product ideation process, we drew from our research and insights to outline Design Pillars. We used these statements to evaluate product and system concepts, and we also returned to the Design Pillars throughout the prototyping process to ensure we were staying true to the empathy-driven research from earlier phases in the design process.

Visual Voice System

We created a storyboard walking through the experience of an autistic individual using the Visual Voice system. Key moments from the story were extracted to illustrate key Visual Voice products.

Wristband: biofeedback device alerts the wearer to emotional triggers and opportunities to share

Camera app: application syncs with wristband data and encourages the user to snap photos from his or her perspective

Prototyping

Harmony Wristband

We created and tested several prototypes for the Harmony wristband. In addition to sketching possible interface designs, we prototyped two wristbands using an Arduino and an iPhone, consulted with an engineer who was bootstrapping open-source biofeedback hardware, and 3-D printed form factors of a design concept.

Visual Voice Application

A boy tried an analog version of the app concept. After taking photos with an iPhone, we printed them on a portable photo printer. He then decorated the photos using markers and stickers. His mom and aide saw a glimpse into his world--his favorite colors and objects, as well as the playful characters he imagines peeking out behind fences.

Testing

We tested both hardware prototypes with autistic individuals and their caregivers. To gather additional baseline biometric data, we also conducted tests and observations with both autistic and non-autistic individuals using a commercial fitness tracker.

On the Visual Voice Application side, we held several analog tests where autistic participants were able to modify and share photos manually, mimicking the self-expression and connection components of the system. A cognitive walk-through with designers helped us gain insights into how to improve the interactive app prototype.

Outcome

Kites & Ladders, LLC

We launched Kites & Ladders, LLC, to recognize and value the experiences of people with autism and to support communication of their unique points of view, which ultimately leads toward independence. Applying user-centered design practices, Kites & Ladders centered on hardware and apps that combine biofeedback and visual communication systems to help autistic people become aware of and express their feelings and worldview to others.

Harmony Wristband

People on the autism spectrum often struggle to communicate when feeling intense emotions or stress. The Harmony wristband uses sensors and biofeedback so wearers can anticipate and express changes in emotional state. Advance notice helps them better understand and respond to sensory and environmental triggers.

Results

Autistic individuals and caretakers continue to express a desire for products like the Harmony wristband and Visual Voice system. However, the cost and technical challenges related to hardware development, the proliferation of wearables, and the relatively small market size led us to shelve the project after completing our studies at AC4D.

Overview: The "Tween" Problem

Sunday School ministries experience a significant attendance drop in Grades 4-6, with some students returning later for confirmation or youth ministry programs. Many curricula don't treat preteens differently, even though kids this age are going through puberty, developing abstract thinking skills, and spending more time with peers. We also saw a trend in churches starting tween-specific programs.

What would it look like to design a curriculum with the particular needs of preteens in mind?

Product Design

Audience

We identified four key audiences who would use this curriculum: preteens, children's ministry directors, preteen ministry directors, and youth ministers.

Creative Jam

We held a "Creative Jam" to rapidly prototype a lesson for proof of concept and initial testing. At the end of two days, the team of seven had identified a unifying visual aesthetic, written a rough video script, brainstormed ideas for a learner activity sheet that would transform into a three-dimensional project, and concepted supplemental sidebar content and activities for a Bible

AUDIENCE

CURRICULUM SYSTEM

SCOPE & SEQUENCE

CREATIVE JAM

PROTOTYPING

CURRICULUM TESTING

SPARKHOUSE TEAM

The Final Product

A Bible overview for grades 5-6 that engages preteens in God's big story through videos, Bible exploration, and hands-on projects.

Spanning 10 units (with six lessons per unit), tweens journey through the Old and New Testaments and discover themes that run throughout Scripture. Every Connect lesson includes three sections: See, Explore, and Make. During each lesson, preteens discover a theme through watching a witty video, explore up to four biblical texts, and create projects that range from games to art and more.

Connect was initially released by unit throughout 2012-2013. Now Connect may be purchased as a print product by unit, a la carte as individual downloadable lessons, or through a sparkhouse Online subscription site that also offers web-exclusive features such as Learner Sheet Tutorial Videos, preteen retreats, and film festivals.

Promotional Videos

To spread the word about Connect, the team developed two overview videos. In addition to reviewing the scripts, another product designer and I served as on-screen talent to provide background on the product system.