We present impacto, a device designed to render the haptic sensation of hitting and being hit in virtual reality. The key idea that allows the small and light impacto device to simulate a strong hit is that it decomposes the stimulus: it renders the tactile aspect of being hit by tapping the skin using a solenoid; it adds impulse to the hit by thrusting the user’s arm backwards using electrical muscle stimulation. The device is self-contained, wireless, and small enough for wearable use, and thus leaves the user unencumbered and able to walk around freely in a virtual environment. The device is of generic shape, allowing it to also be worn on legs so as to enhance the experience of kicking, or merged into props, such as a baseball bat. We demonstrate how to assemble multiple impacto units into a simple haptic suit. Participants of our study rated impacts simulated using impacto’s combination of a solenoid hit and electrical muscle stimulation as more realistic than either technique in isolation.

Impacto is designed to render the haptic sensation of hitting and being hit. The key idea that allows the small impacto device to simulate a strong hit is that it decomposes the stimulus. It renders the tactile aspect of being hit by tapping the skin using a solenoid; it adds impulse to the hit by thrust- ing the user’s arm backwards using electrical muscle stimulation. Both technologies are small enough for wearable use.

Impacto: decomposing impact into tactile and force feedback

The figure above depicts the solenoid component in detail. To achieve a compact form factor, the solenoid is mounted parallel to the user’s skin. A lever mechanism redirects its impact by 90 degrees, allowing it to hit the user’s skin at a perpendicular angle. Furthermore, we provide a set of exchangeable 3D printed tips to refine the desired tactile experience, e.g., to simulate boxing without gloves we use a tip that resembles human knuckles (Figure 2c). In addition to the knuckles, Figure 2 shows: (a) a generic surface, e.g., for punching a virtual avatar, (b) a small generic surface, e.g., for receiving a sharp impact, (d) a rounded surface, e.g., for jugging a ball, and (e) a sharp tip, e.g., for getting hit by a fencing weapon.

The figure above shows the electrical muscle stimulation component. Its electrodes are mounted to the specific muscle that is able to render the impulse response that matches the solenoid. Here the solenoid is mounted to the outside of the arm, and therefore matches the impulse that would cause the arm to flex. Hence, we use the muscle that can flex the user’s arm, i.e., we attach the EMS component to the user’s biceps. When activated, the electrodes trigger an involuntary contraction of those muscles, simulating the transfer of impulse by thrusting the arm backwards.

Lastly, the figure above shows the control unit that drives both solenoid and EMS components. We built impacto as a stand-alone and wearable device, with all electronics embedded in a bracelet. The solenoid module features a Velcro closure, allowing the device to be strapped to the user’s upper arm, back of the hand, the user’s leg and so forth.

Impacto in three interactive sport simulators

We have implemented three virtual reality sport simulators to demonstrate the potential use of impacto. All our examples use impacto for haptic feedback, an Oculus Rift for visuals and a Kinect for tracking.

Boxing [left] is a sport for which the notion of impact is crucial. In this simulator, users can fight a virtual avatar by boxing. The avatar keeps its guard up and attacks periodically. Users must choose the right moment to unleash a successful attack. It takes ten successful hits to take down the avatar, which causes a new opponent to appear and the simulation continues. Impacto adds a haptic component to the simulation as it provides haptic feedback when the user blocks the avatar. Also with an additional unit mounted to the knuckles, users can feel the impact of their punches on the avatar (see paper and video for details)

Soccer [middle]. Impacto units can be used on other limbs and muscles, such as the triceps, quads, etc. In Figure 8 we mounted a unit to the user’s calves. This setup points the solenoid component at the user’s in- step (top of the foot) and the EMS unit to the calf muscles (gastrocnemius), as depicted in Figure 10. We operate the unit so as to slightly push the foot backwards at the moment the ball hits the foo

Baseball [right] . The decomposition of impulse and tactile sensation transfers readily to hand-held props. In the baseball simulator, by wearing an impacto unit, the user experiences the impact of an incoming baseball against the bat. To enable the prop, here a stand-in for a baseball bat, we mount the solenoid onto the prop; the EMS unit, in contrast, stays with the user and stimulates the wrist extension muscle.

Talk at UIST 2015

Press Photos

Acknowledgements

We thank our colleagues: Sijing You and Friedrich Horschig for their help with the interactive sport simulators; Maximilian Schneider and Martin Fritzsche for their help with the first study setup; Patrik Jonell for the skeleton tracking. Last, but not least, we thank David Lindlbauer and the TU Berlin for providing their user study facilities.

The Human Computer Interaction Lab

The objective of our work is to unify the virtual world of the computer with the physical world of the user into a single space.