CardboardTranslate

What it does

CardboardTranslate is an Android application that is meant to be used on Google Cardboard. It provides augmented reality by taking the text around you and translating it to another language. For example, if you were in a foreign country, you can use our application to look at the signs/text around you for a quick translation.

The application is also integrated with the Myo for the user to take pictures for translation, and to zoom in/out of pictures for a better view.

How we built it

We built CardboardTranslate as an Android application using Android Studio. For text recognition and translation, our input is gathered from the camera and is processed using Tesseract an open source Optical Character Recognition (OCR) library. To prevent a bad read from the image, we've implemented various techniques to pre-process the image, eliminate special characters, and detect when gibberish is read. The output we get from the OCR is then translated using the Google Translate API. We used the Myo SDK to listen for poses and link them with their appropriate tasks.

Challenges we ran into

The first challenge we ran into is to ensure the accuracy of our text recognition. For example, Tesseract seeing text when there isn't any, or finding special characters.We were able to overcome this by doing pre-processing the image before using Tesseract, and also post-processing the output we get to detect gibberish.

Our second challenge was the accuracy of the Myo. It was hard to get specific actions to register so we were getting behaviour that the user did not intend to create. We were able to make this better by making sure the Myo is put on correctly and to allow multiple gestures for a specific task.