A New Dimension In Tactile Feedback For Game Controllers

Monday, March 5, 2012

Engineers at the University of Utah have designed a new kind of video game controller that not only vibrates like existing devices, but pulls and stretches the thumb tips in different directions to simulate the tug of a fishing line, the recoil of a gun or the feeling of ocean waves.

“I’m hoping we can get this into production when the next game consoles come out in a couple of years,” says William Provancher, an associate professor of mechanical engineering who is in Vancouver, British Columbia, demonstrating the new game controller with his students March 5-7.

The first haptic or touch feedback in game controllers came in 1997 with the Nintendo64 system’s “rumble pack” that makes the hands vibrate using an off-balance motor to simulate the feel of driving a race car on a gravel road, flying a jet or dueling with Star Wars light sabers, Provancher and colleagues write.

His new controller does something additional: it delivers directional cues to the player by stretching the skin of the thumb tips in different directions.

“We have developed feedback modes that enhance immersiveness and realism for gaming scenarios such as collision, recoil from a gun, the feeling of being pushed by ocean waves or crawling prone in a first-person shooter game,” Provancher says.

The latest game controller prototype looks like controllers for Microsoft’s Xbox or Sony’s PlayStation but with an addition to the controller’s normal thumb joysticks, on which the thumbs are placed and moved in different directions to control the game. In the new controller, the middle of each ring-shaped thumb stick has a round, red “tactor” that looks like the eraser-head-shaped IBM TrackPoint or pointing stick now found on a number of laptop computer brands.

If a gamer’s avatar runs into a wall, the tactor under the thumb moves back to mimic impact. Both tactors can move from side to side to mimic ocean waves. And when a fish bites in one of the games the researchers tried, “as the fish jerks on the line, you can feel the tactor jerk under your thumb,” Provancher says.

Video games commonly are designed so the left thumb stick controls motion and the right controls the player’s gaze or aim. With the new controller, as a soldier avatar crawls forward, the player pushes the left thumb stick forward and feels the tactors tugging alternately back and forth under both thumbs, mimicking the soldier crawling first with one arm, then the other.

Provancher also hopes to adapt the new game controller design for use as a smart phone peripheral device. A phone would fit into the device with game-controlling thumb sticks and tactors on each side of the phone.

Smart phones provide an analogy for the new advance in game controllers, Provancher says. Early smart phones used vibrations only when the phone rang. Current models use more sophisticated feedback: a vibration with each touch to help users improve their accuracy when they use a touch-screen keyboard.

A study analyzing how gamers perform with the new, thumb-tip feedback device was conducted by Provancher, computer science doctoral student Ashley Guinan, computer science master’s degree student Rebecca Koslover and mechanical engineering master’s student Nathaniel Caswell. The research was funded by the National Science Foundation and the University of Utah.

The study, which Guinan was presenting during the Haptics Symposium, found that gamers’ brains made the necessary mental rotation of the directional tugs so they could perform just as well if their thumbs were angled as if their thumbs were straight.

Provancher says the angled thumb position is better ergonomically, both in terms of comfort and of ease in pushing forward with angled thumbs.

He says that his future research will focus on details of how to provide skin-stretch feedback effectively at the same time a game also produces sounds, sights and vibrations.

Provancher’s earlier study of the steering wheel device that told the fingertips which way to turn showed that information can be conveyed by touch even when auditory and visual information also are present.

This video shows a game controller that integrates skin stretch feedback within the thumbsticks found in current console game controllers. Each skin stretch tactor can move independently or in sync. Skin stretch effect parameters can be dynamically varied by games in real time. This creates greater player immersion and allows for game mechanics that were traditionally only rendered visually or audibly to now be portrayed through tactile feedback.