Andreas Kunz is heading the research group ‘ICVR' (Innovation Center Virtual Reality) at ETH Zurich. His research interests are in the development of user-centered VR-systems for industrial business processes. In particular, his research group works on visualization and collaboration systems, haptic interfaces, human computer interfaces (HCI), as well as on corresponding software.

RedirectedWalking (RDW) is a technique that allows exploring immersive virtual environments by real walking in a small physical room. RDW employs so-called redirection techniques (RETs) to manipulate the userâ€™s real world trajectory in such a way that he remains within the boundaries of the physical room. Different RETs were suggested and evaluated in the past. In addition, steering algorithms were proposed that apply a limited set of RETs to redirect a user away from the physical roomâ€™s boundaries.
Within this paper, a generalized approach to planning and applying RETs is presented. It is capable of dynamically selecting suitable RETs and also controlling parameters like their strengths. The problem of steering a user in a small physical room using RETs is formulated as an optimal control problem. This allows applying an efficient probabilistic planning algorithm to maximize the free walking experience. The proposed algorithm uses a map of the virtual environment to continuously determine the optimal RET that has to be applied next.
The suggested algorithm is evaluated within a user study and compared to a state-of-the-art steering algorithm. Results show that for the given virtual environment, it is able to reduce the number of collisions with the room boundaries by 41% and furthermore reduces the amount of applied redirections significantly.


Microsoft PixelSense is able to detect multitouch input and tagged objects as well, which makes it suitable to be used in net-based brainstorming sessions within small teams. However, any gestures above the table cannot be detected, which makes net-based brainstorming sessions less intuitive. Thus, we present a solution how Kinect can be used together with PixelSense to overcome this limitation without interference between the two devices.

This Special Thematic Session describes the elements of an IT system that allows for a better integration of blind people in a MindMap brainstorming session together with sighted users. Software components will be introduced that convert the parallel information visualization to a serialized information representation, which will then be output on a special blind user interface using screen reader and Braille display. Moreover, non-verbal communication elements are another important carrier of information, which also need to be captured and displayed to the blind user. Thus, this session also describes the necessary hardware components and the overall setup, which will allow for a more efficient teamwork in such a mixed team. 

Today, research focuses on the accessibility of explicit information for blind users. This gives only partly access to the information flow in brainstorming sessions, since non-verbal communication is not supported. Advances in ICT however allow capturing implicit information like hand gestures as important part of non-verbal communication. Thus, we describe a system that allows integrating blind people into a brainstorming session using a mind map. 

Accessible mind maps tools are, due to their visual nature hardly available and, if available, they focus on rendering the structure, not considering nonverbal communication elements in ongoing discussions. In this paper, we describe the need for this type of communication as well as a mind map tool that is capable of processing the respective information, coming from a Leap tracking system attached to the interactive surface. 

VR/AR is used in many application fields in engineering, but also in medicine, architecture, or entertainment. It proved its ability to shorten development cycles, to reduce errors in development, and to visualize complex data. However, it still is a domain of specialists who know how to handle a complex VR system and to work with devices that are not adapted to the human behavior. To overcome this problem, research realizes more intuitive VR/AR systems. This paper gives an overview on these approaches and shows how new application fields could be opened by realizing a more natural and intuitive interaction.


Uneven knowledge distribution is often an issue in remote support systems, creating the occasional need for additional information layers that extend beyond plain videoconference and shared workspaces. This paper introduces SEMarbeta, a remote support system designed for car drivers in need of help from an officebound professional expert. We introduce a design concept and its technical implementation using low-cost hardware and techniques inspired by augmented reality research. In this setup, the driver uses a portable Android tablet PC while the expert mechanic uses a stationary computer equipped with a video camera capturing his gestures and sketches. Hence, verbal instructions can be combined with supportive gestures and sketches added by the expert mechanic to the car_s video display. To validate this concept, we carried out a user study involving two typical automotive repair tasks: checking engine oil and examining fuses. Based on these tasks and following a between-group (drivers and expert mechanics) design, we compared voice-only with additional sketch- and gesture-overlay on video screenshots measuring objective and perceived quality of help. Results indicate that sketch- and gesture-overlay can benefit remote car support in typical breakdown situations.

Modern interactive environments like virtual reality simulators or augmented reality systems often require reliable information about a user's future intention in order to increase their immersion and usefulness. For many of such systems, where human locomotion is an essential way of interaction, knowing a user's future walking direction provides relevant information. This paper explains how head tracking data can be used to retrieve a person's intended direction of walking. The goal is to provide a reliable and stable path prediction of human locomotion that holds for a few seconds. Using 6 degrees of freedom head tracking data, the head orientation and the head's movement direction can be derived. Within a user study it is shown that such raw tracking data provides poor prediction results mainly due to noise from gait oscillations. Hence, smoothing lters have to be applied to the data to increase the reliability and robustness of a predictor. Results of the user study show that double exponential smoothing of a person's walking direction data in combination with an initialization using the head orientation provides a reliable short term path predictor with high robustness. 

Effective handling of location-based data is important to emergency response management (ERM). Expert team members co-located around maps typically discuss events while drawing freeform areas or while using physical placeholders representing incidents. Key ERM functions are filtering data, selecting information recipients, searching datasets, drawing time-dependent freeform areas, and zooming in on one region while leaving others unchanged. Under time pressure the mouse and keyboard could be insufficient; intuitive graspable solutions, such as tangible user interfaces (TUIs), may be better suited for ERM. We present CoTracker, a tangible tabletop system with expected potential for ERM teamwork. On an interactive map expert team members can discuss an operational picture using TUIs like bricks, frames, and pens. With the participation of domain experts for cognitive walk-through studies, we examined how generic and specialized TUIs can support ERM-related functions. We present some insights into the design of ERM-focused tangible tabletops

Effective handling of location-based data is important to
emergency response management (ERM). Expert team members co-located around maps typically discuss events while drawing freeform areas or while using physical placeholders representing incidents. Key ERM functions are filtering data, selecting information recipients, searching datasets, drawing
time-dependent freeform areas, and zooming in on one region while leaving others unchanged. Under time pressure the
mouse and keyboard could be insufficient; intuitive graspable solutions, such as tangible user interfaces (TUIs), may be better suited for ERM. We present CoTracker, a tangible tabletop system with expected pot ential for ERM teamwork. On an interactive map expert team members can discuss an operational picture using TUIs like bricks, frames, and pens. With the participation of domain experts for cognitive walk-through studies, we examined how generic and specialized TUIs can support ERM-related functions. We present some insights into
the design of ERM-focused tangible tabletops

Virtual reality systems supporting real walking as a navigation interface usually lack auditory step feedback, although this could give additional information to the user e.g. about the ground he is walking on. In order to add matching auditory step feedback to virtual environments, we propose a user-independent, calibration-free and easy to use system that can predict the occurrence time of stepping sounds based on human gait data.
Our system is based on the timing of reliably occurring characteristic events in the gait cycle which are detected using foot mounted accelerometers and gyroscopes. This approach not only allows us to detect but to predict the time of an upcoming step sound in real-time. Based on data gathered in an experiment, we compare dierent suitable events that allow a tradeo between the maximum precision of the prediction and the maximum time by which the sound can be predicted.

Virtual reality systems supporting real walking as a navigation interface usually lack auditory step feedback, although this could give additional information to the user e.g. about the ground he is walking on. In order to add matching auditory step feedback to virtual environments, we propose a user-independent, calibration-free and easy to use system that can predict the occurrence time of stepping sounds based on human gait data.
Our system is based on the timing of reliably occurring characteristic events in the gait cycle which are detected using foot mounted accelerometers and gyroscopes. This approach not only allows us to detect but to predict the time of an upcoming step sound in real-time. Based on data gathered in an experiment, we compare di_erent suitable events that allow a tradeo_ between the maximum precision of the prediction and the maximum time by which the sound can be predicted.

Virtual reality systems supporting real walking as a navigation interface usually lack auditory step feedback, although this could give additional information to the user e.g. about the ground he is walking on. In order to add matching auditory step feedback to virtual environments, we propose a user-independent, calibration-free and easy to use system that can predict the occurrence time of stepping sounds based on human gait data.
Our system is based on the timing of reliably occurring characteristic events in the gait cycle which are detected using foot mounted accelerometers and gyroscopes. This approach not only allows us to detect but to predict the time of an upcoming step sound in real-time. Based on data gathered in an experiment, we compare dierent suitable events that allow a tradeo between the maximum precision of the prediction and the maximum time by which the sound can be predicted. 

Effective handling of location-based data is important to emergency response management (ERM). Expert team members co-located around maps typically discuss events while drawing freeform areas or while using physical placeholders representing incidents. Key ERM functions are filtering data, selecting information recipients, searching datasets, drawing time-dependent freeform areas, and zooming in on one region while leaving others unchanged. Under time pressure the mouse and keyboard could be insufficient; intuitive graspable solutions, such as tangible user interfaces (TUIs), may be better suited for ERM. We present CoTracker, a tangible tabletop system with expected potential for ERM teamwork. On an interactive map expert team members can discuss an operational picture using TUIs like bricks, frames, and pens. With the participation of domain experts for cognitive walk-through studies, we examined how generic and specialized TUIs can support ERM-related functions. We present some insights into the design of ERM-focused tangible tabletops.

An increasing number of human computer interaction systems are employing interactive table surfaces. For these horizontally aligned screens, the orientation of text passages and any other 'oriented' graphical content is a common problem. A user will not be able to easily read the
same text from different sides of such a table unless it adapts to his position.
To overcome this problem, we present an interactive system that extends the interaction space from measuring the direct manipulation on the interaction plane to observing the user in the space above the table.
Hence, the content of the graphical user interface can automatically be aligned to the position of the active user, which enables the ergonomic reading of a text.
We present a viewpoint tracking system, which utilizes the Microsoft Kinect depth sensor accessed with the OpenNI framework. This system does not need initial pose calibration and smoothens the vision based tracking data. In a next step, we show the benefit of extending the interaction space for a drawing application that allows multiple users to work on automatically oriented, digital notepads while still being able to freely move around the table. 

When human locomotion is used to interact with virtual or augmented environments, the system's immersion could be improved by providing reliable information about the user's walking intention. Such a prediction can be derived from tracking data to determine the future walking direction.
This paper analyses how tracking data relates to navigation decisions from an egocentric view in order to achieve a reliable and stable path prediction. Since tracking data is noisy, a smoothening is required that eliminates oscillations while still recognizing trends in human locomotion. Thus, we analyze different approaches for path prediction, determine relevant setting values, and verify the results by a user study.
Results indicate that robust short term prediction of human locomotion is possible but care must be taken when designing such a predictor.


Digital collaborative environments enable spatially separated users to access and modify shared data over network. However, transmission delays of the network lead to inconsistent data and reduce the efficiency of collaboration due to interaction conflicts.
In this paper, we present a predictive screen-locking algorithm to avoid interaction collisions on net-based shared interactive screens. A model-based predictor calculates the user's next interaction given his past one. The algorithms locks critical objects to the remote station which is less likely to interact with the object. Although the predictor continuously adapts to the user's interaction behavior, an initial interaction model is needed when the collaboration session is started. Hence, we deduce a reasonable, probabilistic interaction model from a large screen collaboration user study. 

Motion platforms are advanced systems for driving simulators. Studies showed that these systems imitate the real driving behavior of cars very accurately. In low-cost driving simulators, most installations lack motion platforms and miss to simulate real motion forces. Their focus is on high-quality video and audio, or force feedback on steering wheels. We aim to substitute the real motion forces with low-cost actuators triggering the human extremities to create an extended immersion. By this, the quality of driving simulators without any motion platform can be increased. Our full body haptic display concept for low-cost racing car simulators is based on air cushion and pull mechanisms to support longitudinal and lateral forces addressing the human's mechanoreceptive and proprioceptive senses. The concept is analyzed within a user study covering twenty-two participants. 

In this paper, we present CollaBoard, a collaboration system that gives a higher feeling of presence to the local auditory and to the persons on the remote site. By overlaying the remote life-sized video image atop the shared artifacts on the common whiteboard and by keeping the whiteboard's content editable at both sites, it creates a higher involvement of the remote partners into a collaborative teamwork. All deictic gestures of the remote user are shown in the right context with the shared artifacts on the common whiteboard and thus preserve their meaning. The paper describes the hardware setup, as well as the software implementation and the performed user studies with two identical interconnected systems. 

Many interactive screens suffer from the incoherence between image plane and interaction plane. The resulting gap causes parallax errors that hinder a precise interaction with the system. For many reasons, this gap cannot be physically reduced any further, while a software correction is still missing. Thus, this paper introduces an observation model for a continuous automatic recalibration controller of the touch sensitive surface.
First, we show that the overall interaction error stems partly from the parallax error, which depends on the changing viewpoint of the user. Hence, a static calibration cannot overcome this error. Being not directly measurable, a continuously adapting correction controller sets the appropriate correction parameter based on updating the estimate of the user's viewpoint from the history of his interaction errors. To estimate the user's viewpoint in front of the screen based on the interaction error on the screen, we secondly investigate the correlation of the two domains from data of a user study, working on a large interactive screen with a significant gap between image plane and interaction plane. The correlation analysis shows significant differences in the interaction error stemming for different viewpoints, which allows the controller to infer the viewpoint. Finally, we model the results as a discrete observation model for the Partially Observable Markov Decision Processes (POMDP) correction controller. 

This paper presents a method for tracking multiple active tangible devices on tabletops. Most tangible devices for tabletops use infrared to send information about their position, orientation, and state. The method we propose can be realized as a tabletop system using a low-cost camera to
detect position and a low-cost infrared (IR) receiver to detect the state of each device. Since two different receivers (camera and IR-receiver) are used simultaneously we call the method dual mode. Using this method, it is possible to use devices with a large variation of states simultaneously on a tabletop, thus having more interactive devices on the surface.


Tangible tabletops where actively emitting devices (TUIs) are tracked to inform about their identity, position, orientation, and state can provide rich interaction within complex, dynamic scenarios. TUI states assuming values over a highresolution large range requires sending many bits for each update and comes at the cost of system update rate. In the first part of this paper, we present an in-house map application where interaction with time-dependent contour lines may benefit from high-resolution TUI states. Prototypical TUI concepts such as slider, ruler, and knobs further motivate the benefit of high-resolution tracking. In the second part of the paper, we depart from a device tracking overview and then show how tangible devices for tabletops typically use infrared (IR) emitters and a camera to send information about their position, orientation, and state. We propose a method that can be realized as a tabletop system using a low-cost camera detecting position combined with a low-cost infrared receiver detecting the state of each device. Since both kinds of sensors are used simultaneously we call the method _dual mode._ This method offers high-resolution tracking without having to compromise on update rate

Tangible tabletops where actively emitting devices (TUIs) are tracked to inform about their identity, position, orientation, and state can provide rich interaction within complex, dynamic scenarios. TUI states assuming values over a highresolution large range requires sending many bits for each update and comes at the cost of system update rate. In the first part of this paper, we present an in-house map application where interaction with time-dependent contour lines may benefit from high-resolution TUI states. Prototypical TUI concepts such as slider, ruler, and knobs further motivate the benefit of high-resolution tracking. In the second part of the paper, we depart from a device tracking overview and then show how tangible devices for tabletops typically use infrared (IR) emitters and a camera to send information about their position, orientation, and state. We propose a method that can be realized as a tabletop system using a low-cost camera detecting position combined with a low-cost infrared receiver detecting the state of each device. Since both kinds of sensors are used simultaneously we call the method _dual mode._ This method offers high-resolution tracking without having to compromise on update rate

This paper presents a method for tracking multiple active tangible devices on tabletops. Most tangible devices for tabletops use infrared to send information about their position, orientation, and state. The method we propose can be realized as a tabletop system using a low-cost camera to
detect position and a low-cost infrared (IR) receiver to detect the state of each device. Since two different receivers (camera and IR-receiver) are used simultaneously we call the method dual mode. Using this method, it is possible to use devices with a large variation of states simultaneously on a tabletop, thus having more interactive devices on the surface. 

On interactive surfaces, an accurate calibration is crucial for a precise user interaction. Today, geometric distortions are eliminated by a static calibration. However, this calibration is specific to a user's posture and parallax distortions occur if this changes (i.e. when the user moves in front of the screen or multiple users take turns).
In this paper, we describe an approach to apply Partially Observable Markov Decision Processes (POMDPs) for automatic online re-calibration to cope with changing viewpoints, which are indicated by uncertain observations of the user's interaction quality on the surface. 

The need for improved collaboration and interaction in construction projects has grown significantly in recent years, especially as projects have become ever more complex. The early design stage is of particular importance for the final results as most of the building_s lifecycle characteristics are committed at this stage, and the opportunity to influence them decreases rapidly as the cost of making changes, or correcting design errors, increases dramatically. Recent advances in information technology offer methods and tools to meet this need. In view of this, a collaborative whiteboard (CollaBoard) for remote collaboration _ is being developed to support mixed, geographically distributed teams. Interconnected via a network, two or more system setups allow users to interact and share information over a common, interactive vertical whiteboard, allowing experts from different disciplines access to databases through intuitive interfaces in order to integrate and optimize lifecycle-related parameters into a new product. Superimposing the live video of the remote partner _ _people on content_ _ also allows the transfer of meta information, such as gestures, resulting in more intuitively distributed collaborative teamwork.

In this paper we present a mixed presence groupware device called _CollaBoard_. The device improves collaboration between co-located and remote partners by providing a high level of workspace awareness. This is achieved by superimposing a lifesize video showing the entire upper body of remote collaborators atop the displayed shared workspace. By doing so, the CollaBoard enriches the shared workspace with embodiments of remote collaborators. It shows pose, gaze and gestures of collaborators to their remote partners, and preserves the meaning of deictic gestures when pointing at displayed shared artifacts. The separate transmission of video and data allows the shared artifacts to remain editable at both conference sites. The functionality of two interconnected CollaBoard prototypes was verified in a public demonstration, a usability test, and a comparative user study.

On interactive surfaces, a precise calibration of the tracking system is necessary for an exact user interaction. So far, common calibration techniques focus on eliminating geometric distortions. This static calibration is only correct for a specific viewpoint of one single user and parallax error distortions still occur if this changes (i.e. if the user moves in front of the digital screen).
In order to overcome this problem, we present an empirical model of the user_s position and movement in front of a digital screen. With this, a model predictive controller is able to correct the parallax error for future positions of the user. We deduce the model_s parameters from a user study on a large interactive whiteboard, in which we measured the 3D position of the user_s viewpoint during common interaction tasks.

In this paper, we present a device called _CollaBoard_. It was developed in the context of ongoing CSCW research efforts in developing groupware that mediates remote collaboration processes. CollaBoard combines video- and data-conferencing by overlaying a life-sized video showing the entire upper body of remote people in front of the displayed shared content. By doing so, CollaBoard shows pose, gaze, and gestures of remote partners, preserves the meaning of users_ deictic gestures when pointing at displayed shared artifacts, and keeps shared artifacts editable at both conference sites. For this, a new whiteboard software is also introduced, which allows a real-time synchronization of the generated artifacts. Finally, the functionality of two interconnected CollaBoard prototypes was verified in a usability assessment.

The Anoto technology uses a non-repetitive pattern printed on paper to enable a camera-equipped pen to locate its absolute position on that pattern. This technology is also used on projection screens to create large-sized interactive areas, but suffers from the drawbacks such as shadow casting or space requirements. Up to now, no implementation exists that enables a tracking on LC-displays using the Anoto technology. Thus, we introduce Digisketch, which uses special films that can be applied to LC-displays, to back and front projections, or to glass, allowing pattern recognition for the pen_s camera. After describing the technical development of a prototype, we compare this new possibility of using Anoto compatible surfaces with other traditional tracking systems for LC-screens.

In this paper we present a mixed presence groupware device called "CollaBoard". The device improves collaboration between co-located and remote partners by providing a high level of workspace awareness. This is achieved by superimposing a life-size video showing the entire upper body of remote collaborators atop the displayed shared workspace. By doing so, the CollaBoard enriches the shared workspace with embodiments of remote collaborators. It shows pose, gaze and gestures of collaborators to their remote partners, and preserves the meaning of deictic gestures when pointing at displayed shared artifacts. The separate transmission of video and data allows the shared artifacts to remain editable at both conference sites. The functionality of two interconnected CollaBoard prototypes was verified in a public demonstration, a usability test, and a comparative user study.

This chapter presents an overview of the Mixed Reality (MR) paradigm, which proposes to overlay our real-world environment with digital, computer-generated objects. It presents example applications and outlines limitations and solutions for their technical implementation. In MR systems, users perceive both the physical environment around them and digital elements presented through, for example, the use of semitransparent displays. By its very nature, MR is a highly interdisciplinary field engaging signal processing, computer vision, computer graphics, user interfaces, human factors, wearable computing, mobile computing,
information visualization, and the design of displays and sensors. This chapter presents potential MR applications, technical challenges in realizing MR systems, as well as issues related to usability and collaboration in MR. It separately presents a section offering a selection of MR projects which have either been partly or fully undertaken at Swiss universities and rounds off with a section on current challenges and trends. 

Capacitive sensing is used in many different fields of application. It has been implemented in such devices as mobile phones and remote controls. However, up until now the physical sensing area has remained limited despite the widespread use of larger input devices such as keyboards. We present DGTS, which seamlessly integrates keyboard typing and cursor pointing. This input device offers multi-finger operation for scrolling and other specialized input commands. The objective of this work is to replace computer mice and touchpads by integrating capacitive sensing into a layer within the keyboard thereby reducing the space required for pointing devices. This paper gives the technical background, shows our contribution, and concludes with initial tests. 

In this paper, we present a new technology to track multiple active Tangible User Interfaces (TUI) on a projection table. We use a commercial high speed infrared tracking camera with modified firmware. With a special tracking method, we reach update rates of up to 250 Hz with low latencies. At this tracking rate, we are able to track the position, state and the orientation of more than 10 active TUIs on the table. For this, we use specified bit codes which are transmitted by the devices. We developed dedicated hardware (SyncUnit) and software to keep the devices and the high speed camera synchronized. The system of camera, SyncUnit, and devices is fully hardware controlled and delivers event coded tracking data for further usage in interactive applications. 

In this paper, we present TNT (Touch ‘n' Tangibles) - a new combination of several existing hardware technologies, which are integrated into an LC-display. TNT enables users to interact using finger touch and tangible user interfaces at the same time on an active flat panel screen, while maintaining precise identification of all interactive objects and fingers. TNT can accurately distinguish between touch and Tangible User Interfaces input by assigning different time slots to each interactive object using the same sensing technology for both methods. TNT's tracking is not affected by objects on the screen other than fingers and active Tangibles User Interfaces, which makes it ideal for use in brainstorming applications. 

A common problem in optical motion capture is the so-called missing marker problem. The occlusion of markers can lead to significant loss of tracking accuracy unless continuous data flow is guaranteed by computationally demanding interpolation or extrapolation schemes. Since interpolation algorithms require data sampled before and after an occlusion, they cannot be used for real-time applications. Extrapolation algorithms only require data sampled before an occlusion. Other algorithms require statistical data and are designed for post-processing. In order to bridge sampling gaps caused by occluded markers and hence to improve 3D real-time motion capture, we suggest a real-time extrapolation algorithm. The realization of this prediction algorithm does not need statistical data or rely on an underlying cinematic human model with pre-defined marker distances. Under the assumption that natural motion can be linear, circular, or a linear combination of both, a prediction method is suggested and realized. The paper presents linear and circular movement measurements for use when a marker is briefly lost. The suggested extrapolation method seems to behave well for a reasonable number of frames, not exceeding 200 milliseconds.

A common problem in optical motion capture of human-body movement is the so-called missing marker problem. The occlusion of markers can lead to significant problems in tracking accuracy unless a continuous flow of data is guaranteed by interpolation or extrapolation algorithms. Since interpolation algorithms require data sampled before and after an occlusion, they cannot be used for real-time applications. Extrapolation algorithms only require data sampled before an occlusion. Other algorithms require statistical data and are designed for post-processing. In order to bridge sampling gaps caused by occluded markers and hence to improve 3D real-time motion capture, we suggest a computationally cost-efficient extrapolation algorithm partly combined with a so-called constraint matrix. The realization of this prediction algorithm does not require statistical data nor does it rely on an underlying cinematic human model with pre-defined marker distances. Under the assumption that human motion can be linear, circular, or a linear combination of both, a prediction method is realized. The paper presents measurements of a circular movement wherein a marker is briefly lost. The suggested extrapolation method behaves well for a reasonable number of frames, not exceeding around two seconds of time. 

In this paper, we present a new technology to perform multi Tangible User Interface (TUI) tracking on standard LC-displays. A lot of existing technologies for tangible user interface tracking use back- or front-projection setups, but they suffer from poor image quality, shadow casting, non-ergonomic interaction, and/or large installations. Thus, we introduce a principle that allows using the InfrActables_ technology [3] on a large LC-display. It combines simultaneous multiuser input on a display with the advantages of a large flat screen. We use infrared photodiodes (IR-LEDs) mounted behind the display_s LC-matrix to track infrared diodes in front of the screen. After initial tests concerning the infrared transparency and sensor characteristics, we developed a proof of concept consisting of 384 sensors, which are addressed through a modular master-slave circuit. Using several interaction devices, multiuser interaction is possible.

The early stages of the product development process are the most important ones. Decisions made here affect the success of a new product. Due to the complexity of products, a multidisciplinary knowledge is required. In the globalised world the company's internal and external experts are spread all over the world. Time and cost pressure impedes teamwork with personal attendance. As a result, personal meetings are strictly limited and teamwork is often reduced to e-mail or phone communication, combined with shared document repositories. Especially for these early stages in product development, there is no suitable support by means of information technology. Creativity methods like brainstorming have shown to be valuable in this phase. But in order to support them, it is very important not to influence the user's creativity by handling an intricate computer system. Thus, we introduce a system, which allows a net-based collaboration that goes far beyond today's existing technology. In addition to typical audio and video communication, intuitive interfaces like pencils or erasers allow to work simultaneously on a shared digital workspace. Therefore, the presented system does not only support net-based collaboration, but also supports co-located teams to perform team sessions in the early stages on an intuitive digital basis. 

Brainstorming sessions are still performed with paper-based analog tools due to a lack of suitable devices supporting intuitive computer interaction. It is crucial that the user's creativity is not distracted by a complex user interface. Combining the advantages of Tangible User Interfaces and Single Display Groupware, we created InfrActables to support simultaneous multi-user interactions on horizontal workspaces. We developed microprocessor-based interaction devices using infrared tracking technology. Interrogated by the system they respond giving their identity and status by encoded light pulses. Finally, the system is evaluated by a user study. 

In this paper, we present a new portable force feedback device for surgery simulations. Dielectric elastomer spring roll linear actua tors for this device were manufactured, and characterized via pas sive tensile tests and active isometric tests. The actuators exhibited a maximum force of 7.2 N, and a maximum elongation of 31%. Due to the high driving voltage, electrical safety issues were also considered. The results showed that sufficient electrical safety can be provided to the user. Two prototypes were built, which practi cally showed functionalities of the actuator and the proposed force feedback concept with actuators connected between the fingers.

We present InfrActables, a wireless technology for human computer interaction devices. It allows multiple users to interact simultaneously on back-projection displays. It recognizes each device's position, orientation, and identification, but also enables the tools to communicate their states to the application that the user interacts with. This makes it possible to build complex interaction devices for direct manipulation with buttons, sliders, and other input capabilities. The technology was developed for a computer-supported environment, allowing multiple users to interact simultaneously on a surface using multiple styli and other user input devices. The principle of operation is not limited to two-dimensional surfaces, three-dimensional user input devices can also profit from its advantages. 

Miniature spring roll dielectric elastomer actuators for a novel kinematic-free force feedback concept were manufactured and experimentally characterized. The actuators exhibited a maximum blocking force of 7.2 N and a displacement of 5 mm. The theoretical considerations based on the material's incompressibility were discussed in order to estimate the actuator behavior under blocked-strain activation and free-strain activation. One prototype was built for the demonstration of the proposed force feedback concept. 

In this paper, we present a new portable force feedback device for surgery simulations. Dielectric elastomer spring roll linear actua tors for this device were manufactured, and characterized via pas sive tensile tests and active isometric tests. The actuators exhibited a maximum force of 7.2 N, and a maximum elongation of 31%. Due to the high driving voltage, electrical safety issues were also considered. The results showed that sufficient electrical safety can be provided to the user. Two prototypes were built, which practi cally showed functionalities of the actuator and the proposed force feedback concept with actuators connected between the fingers.

In this paper, we present a new portable force feedback device for surgery simulations. Dielectric elastomer spring roll linear actuators for this device were manufactured, and characterized via pas sive tensile tests and active isometric tests. The actuators exhibited a maximum force of 7.2 N, and a maximum elongation of 31%. Due to the high driving voltage, electrical safety issues were also considered. The results showed that sufficient electrical safety can be provided to the user. Two prototypes were built, which practically showed functionalities of the actuator and the proposed force feedback concept with actuators connected between the fingers. 

Lately, we have been proposing a novel concept for a wide area haptic feedback device. It is based on a deformable mechanical structure capable of morphing its shape in order to imitate a desired object. Due to the physical presence of the resulting shape, the latter can intuitively be touched and explored with the whole hand. The prototype has been called SmartMesh [1]. After a short review of the SmartMesh mechanism, this paper focuses on the controllability issue of such a multidegrees of freedom structure and introduces the concept and the implementation of the control algorithm responsible for providing the required parameters for the actuation of the mechanical structure. The results emphasize the amazing deformation capability and expressiveness of a future haptic feedback enabled user interface device based on the SmartMesh concept. 

Applications that run in a projection-based mixed reality environment frequently require simultaneous image acquisition of the user by a camera. For later processing, very often a segmentation of the user's image is required, but along with the user, the camera captures the projected dynamic image in the background of the scene. This dynamic background complicates the later segmentation of the user in real-time. To ease segmentation, we introduce methods allowing to imperceptible blank the projected image for the camera. Four different methods are tested and compared. Acquired video streams are recorded and a successful segmentation of a user in front of a dynamic background is demonstrated. The paper concludes with an outlook on possible applications. 

We present an infrared tracking system, which consists of several identical beacons. These beacons are able to detect each other_s relative position and orientation by transmitting and receiving in defined sectors. This system enables a new way of interaction with available devices in a collaborative workspace, e.g. dragging and dropping data from a laptop to a projection device by moving a personal pointing device into the direction of the desired target.

In this paper we propose a novel attempt to develop a spatial tangible user interface (TUI) based on a deformable structure, the so-called HoverMesh. It consists of a stiff cubical, whose upper wall is composed of a deformable mesh of particle filled inflatable cells. This mesh can be deformed by inflating and/or deflating the cubical while consolidating (evacuating) and/or releasing (inflating) the cells. The HoverMesh is both an input and output device and we see its major benefit in the wide interaction area. The haptic feedback modality is thus embedded as well. The first results in our early experiments sustain the concept of a mesh based on inflatable cells. 

In this article, the setup and working principle of a new telecollaboration system “blue-c” is described. This system is an attempt to meet the rising expectations from industry of an IT-supported telecollaboration system. One basic requirement is that a three-dimensional representation of objects be possible together with three-dimensional representations of the remote users. Since gesture and mimicry represent an important information channel during a discussion, a realistic 3D video representation is used instead of simple animated avatars. A simultaneous projection and image acquisition of the user in a telecollaboration system is necessary to allow simultaneous work of all team members. Thus, in the introduced system, problems had to be overcome such as providing, simultaneously, illumination for the image acquisition by the cameras and darkness for a bright projection to be seen by the user. A new approach was taken to integrate the cameras into the system by placing them behind active projection walls, which can be switched from transparent to opaque electrically. Unlike other systems, the cameras are therefore not visible to the user, who thus behaves more naturally. In addition, since the cameras are placed outside of the projection room, there is more space to move inside the immersive environment. The article describes the technology and functionality of the system, as well as the gathered experiences. 

Kunz, Andreas M., Kennel, Thomas, Mazzone, Andrea and Müller, Stephan (2004): Workplace to support the teamwork in the early stages of the product development process. In Mécanique et Industries, 5 pp. 181-187.

The teamwork characterizes the early stages of the product development process. The teamwork in these early stages is characterized by simultaneous writing and sketching. Since the work is done on paper a later use of the generated ideas with computer programs is hindered. This represents a break in the logical concatenation of the product development process and thus results in a loss of information. This implies that it is necessary to provide the group with a tool that collects the information already during writing and sketching. A technical workplace is described that uses techniques from virtual reality to allow a real teamwork in these early stages. Using picture acquisition and color separation techniques different pencils can be detected simultaneously in real time on a horizontal surface. Since only active parts are detected by the picture acquisition no quantization errors can occur caused by other objects on the surface like paper, elbow or hands. The realized teamwork space offers an A1-size work area onto which a simultaneous input is possible with four pens of different color. For the construction of the device a material has to be evaluated that is suitable for a back-projection. 

In this paper we propose a novel control algorithm for the SmartMesh based on clothes simulation modelled with masses and springs. The SmartMesh is modelled as a tissue or as a blanket which is attracted by the object, that has to be represented, enveloping it. Applying the sampled data (the lengths of the springs) to the actuators of the SmartMesh will result in a deformation of the structure. The SmartMesh has been recently proposed as a novel type of haptic display a®ording the output of 3D shapes with the purpose to give a wide area haptic feedback. The SmartMesh is based on a mechanical structure, more precisely, on a double layer grid of nodes linked by prismatic joints. By elongating the linkages, the structure can be deformed to represent the desired object. 

This paper introduces the SmartMesh, a novel type of active structure capable of deforming actively its shape and thus being able to form objects. It is a new approach to find a solution to the difficulties that are encountered in the field of haptic interaction in virtual environments. The SmartMesh creates in real time the virtual objects as seen through head mounted displays or on projection screens at the according position in space. Thus, the virtual objects are not virtual anymore. The SmartMesh can be embedded into a table, into walls, ceilings and floors. The SmartMesh is actuated by a large number of linear actuators and its resolution depends on the amount of nodes and the length of the actuators. 

After a wrap-up in the human physiology and an overview on physical principles, this paper introduces four novel types of actuators for haptic feedback devices based on electroactive polymers, which were developed and evaluated for their suitability for haptic feedback devices. The electroactive polymers show many promising properties, such as big expansion, high frequency range and big force exertion for instance. Unfortunately they are still in the fledgling state and are not available on the market yet. The actuators presented in this paper are a contribution to the haptic and to the electro active polymer researchers' community. 

We present blue-c, a new generation immersive projection and 3D video acquisition environment for virtual design and collaboration. It combines simultaneous acquisition of multiple live video streams with advanced 3D projection technology in a CAVE™-like environment, creating the impression of total immersion. The blue-c portal currently consists of three rectangular projection screens that are built from glass panels containing liquid crystal layers. These screens can be switched from a whitish opaque state (for projection) to a transparent state (for acquisition), which allows the video cameras to “look through” the walls. Our projection technology is based on active stereo using two LCD projectors per screen. The projectors are synchronously shuttered along with the screens, the stereo glasses, active illumination devices, and the acquisition hardware. From multiple video streams, we compute a 3D video representation of the user in real time. The resulting video inlays are integrated into a networked virtual environment. Our design is highly scalable, enabling blue-c to connect to portals with less sophisticated hardware. 

We introduce the SmartMesh, a novel type of active structure capable of deforming actively its shape and thus being able to form objects. It is a new approach to find a solution to the difficulties that are encountered in the field of haptic interaction in virtual environments. The SmartMesh creates in real time the virtual objects as seen through head mounted displays at the according position in space. Thus, the virtual objects are not virtual anymore. The SmartMesh can be embedded into a table, into walls, ceilings and floors. The SmartMesh is actuated by a large number of linear actuators and its resolution depends on the amount of nodes and the length of the actuators. 

Increased capacity of the network will also provide the possibility to transmit pictures and sound. This is of particular interest if speeches, presentations or lectures have to be recorded or transmitted in order to make them accessible at a different location or to a larger auditorium. Very often additional information has to be transmitted simultaneously as for example the visualization of overhead slides or PowerPoint presentations with additional annotations. Up to now this has not been possible or only with a large number of staff and financial expenditure and thus is refused in many cases. This paper describes the construction and the functionality of a digital lectern named ‘Speaker's Corner' that solves the problems described above and also allows the use of conventional media. 

Virtual Reality allows a simultaneous representation of three dimensional objects in different interconnected visualization installations. This offers new possibilities for distributed collaborative work. Up to now the user remains mostly without consideration. But for an efficient distributed collaborative teamwork the user should be visualized three-dimensionally together with other virtual objects. A special projection installation is described which allows simultaneous projection as well as picture acquisition of the users. 

Virtual reality makes it possible today to realize a distributed collaborative work. In this case objects in visualization installations that are connected with each other over a network can be represented three dimensionally. Up to now the user remained mostly without consideration. But for distributed collaborative teamwork the user should be visualized three dimensionally together with the other virtual objects. In the presented paper a special projection installation is described which allows simultaneous projection as well as picture acquisition of the person using the installation. 

Kunz, Andreas M., Kennel, Thomas, Mazzone, Andrea and Mueller, Stephan (2002): Workplace to Support the Teamwork in the Early Stages of the Product Development Process. In: Virtual Concept October 9-10, 2002, Biarritz, France. .

Teamwork characterizes the early stages of the product development process. Together, ideas are generated and reviewed in the group setting. The most important aid during the teamwork process is a paper based work area upon which the conceptual ideas are created and sketched. Very often, later use of the generated ideas is not possible because they do not exist in digital form. In this paper a workplace is described that uses virtual reality techniques to allow teamwork on the digital product in the conceptual stage. This allows further use of the generated data in later processes, without losing information. For the realization of this workplace materials are evaluated that are suitable for a back-projection. Also, an interpretation software is described that allows a quasi synchronous input of several pens. 

Virtual reality offers completely new possibilities for collaborative work over distributed environments. To enable collaborative work, it is necessary that both, virtual objects as well as the other users, can be seen simultaneously in real-time. A novel technique is necessary to overcome the contradiction of darkness and light for image projection and video acquisition, respectively. A stroboscopic light and a camera system are added to the existing VR-system and a few modifications are made to existing shutter glasses. In such a solution, the method of projection in the VR-system remains unmodified. This paper describes how to resolve this contradiction with the constraint to make as few changes as possible to existing VR-systems. Furthermore the timing of the different devices is discussed. 

The technique of virtual reality has been developed so far that it isn't a privilege of large industries anymore but also smaller enterprises can take advantage of it. In this case the main item is the Digital Product which is used for visualization tasks. The scope of this paper is to demonstrate how the Digital Product and its visualization is used in different enterprise processes and which benefits arise from that. 

Virtual reality (VR) can be extended to create a collaborative distributed environment over a physical network. However, such an environment can only be established if both the virtual objects and the other users can be seen simultaneously in real time. This requires a novel technique to overcome the contradicting requirements of darkness and light for image projection and video acquisition, respectively. In order to make an inexpensive and easy-to-build solution, a camera acquisition system and a stroboscopic light are added to an existing VR system. Modifications are made to existing shutter glasses to handle simultaneous projection and video acquisition. In such a solution, the method of projection in the VR system remains unmodified. This paper describes how to resolve the conflict between the darkness required for projection and sufficient lighting for video acquisition. 

With increasing divulgence of the information technology also possibilities to support the teamwork are arising. In many fields of the product development process an IT-based support is available as for example in the design stage (CAD and PDM). However, the early stages in the product development process still go without the support of the information technology. The topic this paper is to show the causes of that and to show possible solutions. 

Virtual Reality becomes more and more important within the product development process. It enables the engineer to realize constraints or mistakes in the product design at a very early stage by viewing the digital geometric prototype. Beside viewing the design of a product, additional functionalities like simulation of assembling, the physically correct behavior of a machine or the machine control come into focus of interest. Therefore, the interaction modality of haptic feedback gains more and more importance for simulation tasks in virtual environments. However there are only a few portable haptic interfaces with which the user can experience in a natural way the sensation of force feedback.
The scope of this paper is to present a new passive haptic interface that is lightweight and easy to use. Furthermore it has no constraints in the workspace and applies high forces to the fingertips of the user by blocking the natural grasping. 

In this paper, we report ongoing work in a new project, The Blue-C. The goal of this project is to build a collaborative, immersive virtual environment which will eventually integrate real humans, captured by a set of video cameras. Two Blue-Cs will be interconnected via a high-speed network. This will allow for bi-directional collaboration and interaction between two persons sharing virtual spaces. The video streams are used for both texture and geometry extraction. We will generate a 3-D light field inlay enriched with the reconstructed geometry, which will be integrated into the virtual environment. The design and construction of the Blue-C environment, including both hardware and software, is an interdisciplinary effort with participants from the departments of computer science, architecture, product development, and electrical engineering. Parallel to the development of the core system, we are designing new applications in the areas of computer aided architectural design, product reviewing, and medicine, which will highlight the versatility of the Blue-C. 

Haptic rendering is the process of computing and generating forces in response to user interactions with virtual objects. The interaction modality of haptic feedback gains more and more of importance for simulation tasks in virtual environments. However there are only very few portable haptic interfaces, with which the user can experience in a natural way the sensation of force feedback. The scope of this paper is to present a new portable haptic interface using shape memory alloy wires as actuators. Shape memory alloys (SMA), and especially the nickel-titanium compositions, offer a number of engineering properties available in no other material. The ability to respond with significant force and motion to small changes in ambient temperature and the capability to convert heat energy into mechanical work provide the designer with a new array of possibilities. Beside some application specific characterization of the actuator material with respect to energy consumption and dynamic behavior for loading, the mechanical layout of the interface is discussed in detail. To realize a light and compact design of the interface, a flexure hinge is considered as an essential part of interface. 