Haptic Interaction with Volume Data

Haptic exploration adds an additional dimension to working with 3D data: a sense of touch. This is especially useful in areas such as medical simulation, training and pre-surgical planning, as well as in museum display, sculpting, CAD, military applications, assistive technology for blind and visually impaired people, entertainment and others.

Each haptic rendering frame consists of three stages: collision detection, collision response and force feedback generation. In order to feel the 3D data smoothly, an update rate of at least 1 kHz is required. There exist different surface- and voxel-based haptic rendering methods. Unaddressed practical problems for almost all of them are that no guarantees for collision detection could be given and/or that a special topological structure of the objects is required.

While working on his PhD at Welfenlab, Roman Vlasov proposed a novel and robust approach based on employing the ray casting technique to collision detection and path finding for collision response. The approach is very fast (150~kHz) and does not have the aforementioned drawbacks while guaranteeing nearly constant time complexity independent of data resolution. This is especially important for such delicate procedures as pre-operation planning. The collision response uses an implicit surface representation "on the fly", which can be used with dynamically changing objects. No precalculation is needed.

Further on, Roman Vlasov proposed a flexible deformation framework allowing him to use his haptic rendering approach together with deformation models. Furthermore, he presented a graphics approach which is used to keep the graphics representation of segments up-to-date during the deformation simulation. The challenge here was to reflect deformations of objects interactively.

Further on, he proposed two local deformation simulation approaches based on the method of potential fields. The first approach uses "regular" potential fields. The second approach uses his novel cuboid fields. Further on, he demonstrated that cuboid fields are better suited to haptic rendering of volumetric data. Additionally, he introduced the prototype of the global deformation approach. The resulting haptic rendering approach combined with his proposed approaches for deformation simulation within his deformation framework does not require any pre-calculated structure and works "on the fly". Further on, Roman Vlasov showed that the developed deformation framework can be used for the simulation of drilling and for the simulation of needle insertion. Additionally, his local potential fields model allows simulation and feeling of different tissues.

The developed haptic prototype system was integrated into the YaDiV project - a powerful framework for working with 3D volumetric data. The developed haptic system was presented on the CeBIT international computer expo 2013 and 2015 as a base of the interaction component of the Marie Curie ITN MultiScaleHuman project, which was funded by the European Union.

The proposed deformation framework and all haptic rendering and deformation simulation approaches were fully developedfrom scratch. Their design and development was the main aim of his PhD work and was supported by a grant provided by Siemens/DAAD Postgraduate Programme (DAAD - German Academic Exchange Service).