Simulating Haptics in Virtual Reality with Robotics

When people have a Virtual Reality experience they are transported to a virtual world using the Head mounted display and headphones. In such a virtual world they can observe, walk around and interact with objects. But when people interact with objects in the virtual world they cannot feel them in the same way they can feel objects in the real world. This project examines how robotics can be used to provide the missing haptic feedback in virtual reality experiences.

To test this we have developed an interface that allows us to control the industrial robot Baxter from within the game engine Unity that generates the virtual reality experience. In this example users can play a virtual sword fighting game against a virtual opponent, and unlike other virtual reality experiences, feel when the swords cross, which is a tremendously exciting and satisfying experience and we are just at the beginning to explore the possibilities for games, entertainment but also a range of other applications in skills training and industry 4.0 applications.

This project was showcased to the Duke of York during a visit at RMIT University and captured by Channel 9.