A virtual extension of the handheld part, the so-called stylus, of the haptic device is projected onto the display in addition to a virtual blue 3D cube (3). This virtual extension moves with the stylus and interacts with the cube. In case of collision of both objects, i.e. the extension of the stylus touches the surface of the cube, force feedback is generated by the haptic device. Also acoustic feedback and visual feedback can be generated upon collision using OpenAL and OpenGL features.

The haptic device can be moved on the display surface to be positioned at a suitable location for interaction with the virtual scene. This movement is tracked by the feed of the haptic device touching the touch-sensitive surface. Additional 2D graphics pop up as soon as the new location of the haptic device has been registered and accurate alignment of the virtual extension of the stylus with the real stylus is guaranteed.
A second version of the haptable includes a virtual mirror (4) that is attached to the haptic device. Such a mirror allows for additional, physically restricted views of the virtual objects being projected onto the table top display.

The combination of a haptic force feedback device with the touchscreen allows for a broad variety of applications. The first one, I had in mind, was surgical simulation to train cognitive, technical, as well as communicative skills of surgical teams. Such an interface would be capable of e.g. simulating many critical situations around and on top of the operating table to better train upcoming generations of surgical teams.
The system proposed by Gu Ye, Andrei State and Henry Fuchs in the article „A Practical Multi-viewer Tabletop Autostereoscopic Display“ allows for multiple perspectives on renderings.

Standard tabletop display with simple projection of virtual content are already commercially available e.g. from the companies Evoluce or Microsoft. The company Sectra uses this kind of user interface for rendering 3D medical imaging data.

A combination of haptable with these systems might result in a powerful 3D user interface having the following benefits:

Design: E.g. Working environment for architecture/industrial design. Shape modeling tools and pencils are attached to the haptic input device (1) and the building, room, car, furniture, landscape are pre-sented on table top multi-touch display (2).

Gaming: For instance, a billiard game could use the haptic input device (1) as the billiard cue and balls, holes and playing surface remain virtual.

The haptable apparatus has been filed as two patents by the Technical University of Munich. As agreed by the patent office, TU Munich, the concept is published here. Please find the two official technology offers (pdf) here:

If you are interested in implementing the user interface concept haptable for your own solutions, please contact:

Christoph Bichlmeierchristoph.bichlmeier@medicalar.com

I’m offering my consulting service to technically support you integrating the concept haptable into your application scenario. This includes implementation, integration into your software, planning the adaptation of the concept haptable to your application needs and hardware selection.

powered by

Follow This Blog

Selected Interviews

I’d like to refere to an interesting interview about VR glasses to be used for training surgeons. The interview with Dr .Shafi Ahmed, operating at The Royal London Hospital, has been published at news-medical.net. As Dr. Ahmed sais „VR lets every student…

Helping Autists to Interact with Their Social Environment

Arshya Vahabzadeh, a pediatric psychiatrist, speaks in the interview with The Doctor Weighs In about his work with the company Brain Power developing transformable, wearable technologies for children with autism to learn crucial live skills. With the help of Google Glass…

Google+ Hangout Session about Medical AR

I would like to direct your attention to a very interesting google+ hangout session organized by Christine Pereyand hosted by IEEE. This online discussion having the title „Augmented Reality and the Human Body“ took place on January 22, 2015. Terry Peters,…