Rosetta Stone for iPhone adds AI to identify objects for live translations

Historically, language translation apps were confined to whatever developers could store in their on-board databases, limitations that were surpassed with internet connections and AI-assisted text recognition tools. Today, language teaching developer Rosetta Stone is taking the next step forward by adding augmented reality and machine learning to its iPhone app, enabling users to identify and translate the names of real-world objects — the first time object recognition technology has been used in a language app in this manner.

The machine learning technology is a new component of Rosetta Stone’s Dynamic Immersion feature, which lets the user simulate being in a foreign language environment, surrounded by new words, phrases, and native speakers’ voices. A scavenger hunt-like game called Seek & Speak has the user point the iPhone camera at an object and see the vocabulary word in a chosen language, then practice a conversation using the word. Seek & Speak is launching in beta for English, French, German, Italian, and Spanish, with plans to add additional languages and challenges during 2019.

These five languages are also getting two more improvements: Embedded Translations, enabling Dynamic Immersion users to long-press any word during the experience to reveal its translation, and an updated Phrasebook — an offline collection of over 164 spoken phrases that can be used in common situations. Rosetta Stone also says it’s using machine learning to continuously improve the performance of its TruAccent speech recognition feature, leveraging thousands of hours of recorded speech data to help users match their pronunciation of words to that of native speakers.

“I’m incredibly proud of our team of software developers behind this product,” said Rosetta Stone president Matt Hulett. “Their use of augmented reality and machine learning creates an incredibly rewarding learning experience for our customers. Our iOS mobile app brings language immersion to the next level, with technologies that are only now widely available to consumers through mobile phones.”

Rosetta Stone for iPhone now supports 30 languages, up from a prior 24, though features vary based on the language. Some of the features depend on an active internet connection, but others — including Phrasebook — are stored within the app so they can be used even when you’re traveling abroad without data services.

The new Rosetta Stone app for iPhones is available now in the iOS App Store and includes iPad and Apple Watch support. As a subscription-based app, it offers a free three-day trial so you can see whether the service meets your needs, after which annual pricing ranges from $84 to $200. Rosetta Stone is offering a 30 percent discount on a one-year subscription through January 31, 2019.