A.R.M. Project

An immersive and educational web application that uses Myo Armband gestures to control Augmented Reality.

Embedded Systems Competition

HackHolyoke was Mount Holyoke College’s second annual 24 hour embedded systems hackathon open to 50 teams from the New England area. During the competition, my team created a web application that combines augmented reality and gesture control (Myo Armband) to create an immersive hands-on experience for both students and teachers. We were the first place winners out of 50 teams and 200 participants. My responsibilities included UI/UX and Front-End Implementation of the javascript library awe.js.

Team Role

UX Design, Front-End Development

Project Type

Wearables Web Application

Duration

24 Hours

Can New Technology & Education Co-Exist?

Scholars have talked about the potential education of augmented reality in the classroom. However, technological and practical constraints have limited its utility. It has been deemed unusable for education.

Understanding the Limits of Augmented Reality

We researched why Augmented Reality has not been successfully integrated in the classroom and then brainstormed ways we could make it impactful.

Teacher Interviews Showed Where Tech Fails

We began our research process by talking with teachers. Our concept was born after several conversations with a teammate’s friend, who was a middle school teacher in San Francisco. She discussed the role of digital technology in the private school system and asserted that it was more of a distraction than an educational benefit in the classroom. Additionally, the teacher referred us to relevant academic literature covering different methods of learning.

Literature Reviews Revealed Opportunities

Our team conducted a literature review on several of Harvard Professor Chris Dede’s papers because he has written extensively about situated learning theory and its implications for using technology in the classroom. Dede argues that AR has great potential for classroom use because it allows digital media to be presented in the context of a lesson and a familiar learning environment. If teachers have total control of augmented reality, they can seamlessly integrate a digital media artifact into his or her lesson.

Our Research Revealed Interesting Insights



Not Enough Due Diligence

School boards often spend money on technology without thorough examination of how that technology will further learning goals. Without proper supervision, these tools often distract from learning more than they add to it.



No Student-Teacher Interaction

Most digital technologies used in classrooms are self contained and highly stimulating. This combination of factors pushes the teacher out of student focus. The teacher is no longer in control of the classroom.

f

More Control Equals Greater Success

The main problem that teachers associate with digital media technologies is that those technologies interfere with their control of their students’ focus. Therefore, digital technologies that allow for teacher greater control may enjoy greater success.

A.R.M. Project MYO Gestures

Wave Left

Rotate 3D Objects Clockwise

Wave Right

Rotate 3D Objects Counterclockwise

Fist

Contract 3D Object

Spread Fingers

Expand 3D Object

MYO Armband + Augmented Reality Equals Total Student Engagement

We feel that the MYO armband is the first natural input device that is both intuitive and mobile enough to be used effectively in the classroom. It allows teachers to move freely around the classroom and to easily control when digital information is displayed and when it is hidden. We have developed a tool that allows a teacher to generate virtual objects in the classroom using a laptop and paper AR tags.

Once an object has been summoned, a teacher can directly manipulate it in three dimensional space using the Myo armband. Teachers can associate any with a tag and easily print out multiple tags to create interactive displays. Furthermore, teachers can easily control when and how objects are displayed with a simple gesture and by moving the paper tags out of view of the camera. This technology has numerous applications.

For this demonstration, we have created a simple diagram of the solar system, an interactive chemistry diagram and an application that allows students creatively combine loop tracks to make their own music. We are the first group to effectively combine this type of augmented reality application with the Myo armband for a classroom audience.