Share

When Mobile Sign Language Systems won an Ann Arbor Spark business plan competition last summer, and as a result got incubator space and other services, the University of Michigan-spawned startup was able to survive a crucial year of development.

“I don’t think that our business would have been moving in the direction it’s going if we had not won that competition,” says Mobile Sign Language Systems CEO Jason Gilbert.

Last month’s news that startup Ambiq Micro was moving to what it considers greener engineering pastures in Texas has people talking, again, about what can be done to stop the brain drain from Michigan. But the case of Mobile Sign Language shows, at least, that some Michigan efforts at nurturing local talent may be paying off.

The company’s aim is to create a smart-phone app that will translate spoken words into sign-language video in real time. As a first (baby) step toward that goal, it will release its first product—an app that will show parents how to communicate with their prelingual babies using simple sign language words—either just before, or just after, the holidays.

Gilbert and cofounder Judy Shau-yuh Yu began working on their project around the end of 2006. But Gilbert’s journey toward this moment really began about 13 years ago, when he volunteered to become a missionary for his church and went to a deaf congregation in California.

Gilbert did not know sign language at all when he started at the congregation, but he spent a couple of years there “completely immersed in deaf culture.” After he did learn to sign, he jumped directly into interpreting for the deaf. So, later, when he began developing his app, he did so not only steeped in the mechanics of interpreting for the deaf, but also with great first-hand knowledge of the way the deaf and hard-of-hearing view themselves.

“Many consider themselves deaf with a capital D,” Gilbert says. “They consider it to be part of their identity. It’s their culture. They don’t see it as a disability at all. And sign language is its own distinct language. It has its own grammar and syntax. It’s not just a manual form of English. In fact, the grammar is very different from English. And they take a lot of pride in being deaf and in having their own distinct culture. It’s a very close-knit, a very tight community.”

That is why Gilbert made sure that he involved the deaf community in development of his app. The University of Michigan has hosted summer camps for deaf and hard-of-hearing children, and he’s let the kids try out early prototypes.

“The aim of this is to increase their access to the hearing world, to ease the communication barrier,” he says.

The app features a three-dimensional avatar on the screen that does the signing. Users can rotate the character around to different angles, speed it up, slow it down, even play the sign backward, if they want to.

Gilbert says it is important that the avatar not be just a cartoon character, but a full motion-capture computer representation of a real human. That is the only way the full syntax of sign language can be achieved, with all its subtleties.

“The grammar of sign language is not entirely in the hands,” Gilbert says. “It’s very much in the body and facial expressions. Facial grammar can actually change the meaning of a handshape. And so we wanted to be able to capture it as realistically as possible.”

All this development requires time in a motion-capture studio. He and Yu did their prototype at the University of Michigan’s 3-D Lab and plan on doing more at a studio at Henry Ford Community College in Dearborn.

And, since Gilbert and Yu are entirely self-funded—and doing this while they also hold down day jobs—progress is sometimes a bit slow. Gilbert has a PhD in atmospheric and space sciences, and he works in the space department at U-M. Gilbert says the company will seek venture funding, but not until they have a fully functioning prototype to shop around.

Because of the time and expense involved in getting into a studio, the full-function app will probably take another year of development. What Mobile Sign Language Systems will release around the holidays is a kind of baby step along the way—in some sense. It’s an app of baby sign language.

It’s been trendy for a few years now for parents to communicate with their prelingual children using sign language. The theory is that babies can make their needs known to their parents using sign language a lot sooner than they can learn words. Educational courses and products have developed around this theory.

Gilbert, a father of three, communicated with his kids that way when they were infants, and he hopes that it is more than just a trend. So, he and Yu put together a short dictionary of baby signs—basicwords like “milk” and “food”—and will test the waters with this baby app. There will be a three-dimensional character on the screen that does the signing for simple words that you either enter into the app, or speak into it (although he is not certain the voice recognition portion will be ready for this first app.) Parents enter the words, and the avatar signs it.

“We’re just trying to play with the technology and get a feel for exactly what our interpreter is going to look like in the end,” Gilbert says. “So, this baby sign app is just sort of a tool that we’re putting out along the way. We hope the people find it entertaining and educational as well.”

The apps, Gilbert says, will go for between $3 and $10, and will be directed at a hearing audience primarily—parents who want to communicate with their pre-lingual children with baby sign, American Sign Language students, interpreters and friends/family of ASL users. The market size for that, Gilbert says, is at least 5 million.

The company’s long-range vision is to develop “virtual interpreters” geared toward two groups. The first is a hand-held version for the deaf and hard of hearing who use American Sign Language as their primary means of communication. Gilbert estimates this market at about 2 million people. The other is a stand-alone, Web-based version for businesses or organizations who would use this to help satisfy compliance with the Americans with Disabilities Act. Gilbert estimates that market at about a half-million people. He says the company is still looking at prices, but more limited versions are marketed by other companies to businesses customers for between $5,000 and $8,000 per installation.

As for the question of whether the company will remain in Michigan, that really is not a question at all, he says. He and Yu are not going anywhere.

“We really like this area and the greater Ann Arbor area is very conducive to the entrepreneurial spirit,” Gilbert says. “We have no plans to move. Hopefully, we’ll be another Ann Arbor success story.”