BEIJING, CHINA – JANUARY 23: A tourist and her daughter wearing the masks visit the Tiananmen Square at dangerous levels of air pollution on January 23, 2013 in Beijing, China. The air quality in Beijing on Wednesday hit serious levels again, as smog blanketed the city. (Photo by Feng Li/Getty Images)

Factory chimneys emit smoke and winter smog makes it hard to see houses between the trees in Christchurch, Canterbury, New Zealand in June 2009. Image: Colin Monteath/Hedgehog House/Minden Pictures/Corbis

Utilizing Markov Chains for Speech/Gesture PredictionsThis research explains a prediction model approach to translation, which inspired us to incorporate a Markov chain prediction model in our system to help with real time visual translation. This will help to improve the speed and accuracy of our translation system.

Record Electricity from Your Muscles!This research gave us a better idea of how electrical impulses spread throughout the body, which inspired us to use an electrode transfer approach to simulate gestures with electrical impulses.

TED: How to Control Someone Else’s Arm with Your BrainScenario: the user moves their arm and it moves someone else’s arm as well. This is done with electrodes.Our system is similar in the fact that it would use brainwaves to move body limbs, but they would be your own limbs and you wouldn’t think where you want to move them: you would think a regular sentence and your limbs would feel appropriate sensations to suggest (or force) movement in that direction. We could incorporate similar electrodes into our system.

The ‘Not Face’ is a universal part of languageIf facial expressions like these, which are crucial in determining the meaning or implication of certain gestures, can be recognized on a wide scale, then this will offer a higher level of accuracy for our system.

* * *

Working diligently

Critiques

Margaret Ellis (CS):

What is the purpose of the sign language route?

Why can’t we just text each other instead of sign language?

Promotes social interaction

Are there any technologies for gesture to sound currently?

Is video capturing enough to read other’s gesture?

Response:

Her questions made us realize we are going for a more educational prototype. Gestures are universal and would stimulate communication unlike texting (which might not even exist in our world since people are quickly avoiding anything that requires reading). Additionally texting would require dependence on a device for communication, and we’re hoping our device will only be needed until the user learns the gestures for themselves.

Zach Duer (ICAT):

Just pollution as a problem is not enough!

There are so many other problems this system can solve. Think bigger.

Problem and solution don’t link up.

Clarify the concept of electrodes.

Why gestures and not a different mode of communication?

What if multiple people converse simultaneously?

Eye tracking!

Devices can’t communicate directly? (Re: no texting)

Response:

His feedback was eyeopening. We realized our system could be utilized for so much more than what we were asking of it. However, given the prompt we don’t think we should change our “problem” because the other things it could be solving are present in this world.

He brought up the possibility of eye tracking (as opposed to our closest person method) for conversation linking which we thought was appropriate since gesture-communication requires you to be looking at someone anyway.

Matt Wisnioski (STS):

What’s your time horizon?

Problem and solution don’t match!

This problem might have a broad scale environmental solutions, which may not require your system!

What’s the core of the system? Gesture?

The time horizon (of 50-100 years) is too far. If your system can be achieved in 10 years, why target for 50 years?

Pollution is too small of a problem.

Strong research, good drawings.

What have you learned from this?

Response

He was very adamant that our system shouldn’t be as far in the future as we planned, however, we disagree because brain wave reading and translation is not, in our opinions, close to feasible in the next ten years (his estimation).

He also suggested we might not need this system if someone else solves it environmentally, which isn’t a part of our world so we didn’t think this was relevant. (I.e. our world is a world in which no one has solved this environmentally).

Also, pollution isn’t just the problem. It’s the lack of communication resulting from the negative health effects from pollution. This was covered, but maybe we need to make it more apparent?

Liesl Baum (ICAT):

Any control over what thoughts get expressed?

People adapt to sign language – system will slowly be unnecessary.

How does the camera detect who is talking?

What about private conversations that need to be censored from people around?

Any thoughts about transmitting one person’s gestures to multiple people?

For public speaking, for instance.

Does the smog interfere with the gesture detecting camera?

Result:

She was the first person to bring up the idea of “gossiping” and it’s effects on our system. She had a very interesting feedback approach. This was helpful because we hadn’t considered how we could handle gossip (i.e. conversations that you are signing but you don’t want other people to be able to see/translate). We don’t think this is possible to avoid, but we also don’t think it’s going to be an issue since you could turn your backs to other people if really needed.

We realized here that we need to make it clear when presenting that multiple people can focus on one person, but one person can only focus on one other person.

Additionally, she thought that it was a good idea to slowly get rid of the system and agreed that with current research the camera should be able to see even in the smog.