Medal of Honor recipient inspires research at USC Institute for Creative Technologies

Shown is Medal of Honor recipient Staff Sgt. Ty Carter as ICT researchers plan to build a 3-D interactive digital version of him that can understand questions and provide relevant answers on topics ranging from his experiences in battle to his definition of heroism. In addition, his body movements were scanned using a Microsoft Kinect in order to help create a digital character that can move just like him.

Medal of Honor recipient Staff Sgt. Ty Carter recently spent the day at the University of Southern California Institute for Creative Technologies answering questions about post-traumatic stress.

"I don't call it post-traumatic stress disorder, because any time I hear disorder I think of a mental deficiency or chemical imbalance," said Carter speaking at the institute. "Post-traumatic stress is your body and mind's instinctive and natural reaction to survive."

The topic is nothing new to Carter, who has been lauded for showing extraordinary courage both in battle and in speaking publically about his struggles with PTS. He has traveled across the country sharing his story, encouraging people to seek care and working to reduce the stigma that surrounds mental health issues.

However, the visit to ICT was different. Carter was not speaking in front of an audience. He was sitting inside Light Stage 6, a nine-meter diameter ball of over 6000 bright LED lights. As he answered questions, 10 video cameras recorded him, capturing his face and body movements from multiple angles. By recording Carter this way, ICT researchers plan to build a 3-D interactive digital version of him that can understand questions and provide relevant answers on topics ranging from his experiences in battle to his definition of heroism. Carter also allowed ICT's team to record him in Light Stage X, a smaller sphere that can recreate facial details down to pores and fine wrinkles. In addition, his body movements were scanned using a Microsoft Kinect in order to help create a digital character that can move just like him.

"We want to help spread his message," said Randall W. Hill Jr., ICT's executive director. "Our aim is to further our computer graphics and natural language research and development efforts to create a virtual Staff Sgt. Carter that could ideally be used to help more people benefit from his story than only those who might be able to meet him in person."

Potential applications of this effort could include a life-sized hologram version of Carter that is projected in public spaces to answer questions about PTS, or a detailed digital representation of him that could live on the Web to answer questions and provide people online resources that can help them get information or care.

ICT, a University Affliated Research Center working in collaboration with the U.S. Army Research Laboratory, has a long history of creating virtual humans, computer-generated characters that look and act like real people. Examples can be seen in the Emergent Leader Interactive Training Enviroment developed in partnership with ARL's Army Simulation and Training Technology Center, located in Orlando, Fla., and Fort Benning's Maneuver Center of Excellence, and also in SimCoach, an intelligent, interactive, online virtual human healthcare guide funded by U.S. Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury.

"This research effort brings together ICT's expertise in interactive digital technologies and Staff Sgt. Carter's mission to reduce stigma around post-traumatic stress," said John Hart, ICT's Army program manager at ARL's Simulation and Training Technology Center. "It is a great example of how Army-funded research taking place in the lab can make a difference in the lives of Soldiers."

Digital doubles created by the ICT Graphics Lab have appeared in major Hollywood movie releases including: Ender's Game, Oblivion and Avatar, as well as in Digital Ira, ICT's collaborations with Activision and Nvida that bring an unprecedented level of realism to real-time video game characters. Researchers have also joined forces with the USC Shoah Foundation to create interactive video recordings of a Holocaust survivor that preserves the ablility for people to ask questions of survivors far into the future.

The idea to recreate Carter came about after he toured ICT in September and was introduced to the Los Angeles-based institute's interactive media prototypes that are being used to treat post-traumatic stress and develop leadership skills.
"Young people today relate to material presented in video games," said Carter during that visit. "Technologies that help Soldiers stay interested so they want to do these types of training or therapies are on the right path."

For the scientists at ICT, the opportunity to have Carter transition from a guest to a scanning subject, provided extra inspiration in their quest to advance their research and development efforts.

"Staff Sgt. Carter is a true hero and it was an honor to work with him," said Paul Debevec, ICT's associate director for graphics research. "His goal of breaking down barriers to care and our goal of improving digital character technologies results in a demonstration that illustrates the power of technology and storytelling to bring about real change in people's lives."