Microsoft Demos Spoken English To Chinese 'Universal Translator'

Microsoft's Rick Rashid demonstrated a speech translation system that listens to spoken English, recognizes the words, translates the words into Chinese, and then speaks the translation out loud in a computer-generated voice similar to that of the user. The demonstration was given in Tianjin, China at Microsoft Research Asiaís 21st Century Computing event.

Just over two years ago, researchers at Microsoft Research and the University of Toronto made another breakthrough. By using a technique called Deep Neural Networks, which is patterned after human brain behavior, researchers were able to train more discriminative and better speech recognizers than previous methods.

During my October 25 presentation in China, I had the opportunity to showcase the latest results of this work. We have been able to reduce the word error rate for speech by over 30% compared to previous methods. This means that rather than having one word in 4 or 5 incorrect, now the error rate is one word in 7 or 8. While still far from perfect, this is the most dramatic change in accuracy since the introduction of hidden Markov modeling in 1979, and as we add more data to the training we believe that we will get even better results.

In my presentation, I showed how we take the text that represents my speech and run it through translation- in this case, turning my English into Chinese in two steps. The first takes my words and finds the Chinese equivalents, and while non-trivial, this is the easy part. The second reorders the words to be appropriate for Chinese, an important step for correct translation between languages.

Though it was a limited test, the effect was dramatic, and the audience came alive in response. When I spoke in English, the system automatically combined all the underlying technologies to deliver a robust speech to speech experienceómy voice speaking Chinese. You can see the demo in the video above.

... we may not have to wait until the 22nd century for a usable equivalent of Star Trekís universal translator...

Star Trek fans well remember the famous lizard man translation from the classic Trek episode Arena (start at about 45 seconds).

(Star Trek Arena trailer)

Note the perfect translation of Lizardese into English, while maintaining the Lizard Man's voice characteristics. As you can see, he is holding the Star Trek Universal Translator.