Digital Museum

In museums, digital technology can be used to tell the background information about their exhibits. In this project, we aim to realize exhibits that merges the exhibits itself and their context, totally support the museum experience, and digital technology for the museum.

Guidance Field guides users to arrive at and gaze at pre-defined targets in virtual environments.

On-Site AR Exhibition System

On-Site AR Exhibition System” is an iOS application to interactively exhibit the past scenery of a certain place by superimposing it onto the current scenery shown on iPad.

Digital Diorama

"Digital Diorama" system is a novel mixed reality museum exhibition system to convey background information related to the exhibit intuitively. The Digital Diorama system aims to realize better function than traditional dioramas in museum exhibitions by using multi modal mixed reality technology. The system superimposes computer generated diorama scene reconstructed from related image/video materials onto real exhibits. In order to generate attractive contents, we are developing image-based rendering techniques and interfaces to induce appreciation activity effectively.

Time-leaping Seat

The Time-leaping Seat enhances a static exhibit toward time axis by 3-D recording visitors and exhibits at museums.

Digital Display Case

This system displays a virtual exhibited object while keeping the interface of conventional display case and effectively conveys background information about the exhibits.

Mob scene filter

Videos are getting high-resolution enough to understand its detail. Improvement of resolution, however, might accelerate privacy problems. High-resolution videos enable people to identify an individual easily. Image processing techniques, such as the mosaic and the blur, have been used to protect privacy. These techniques, however, make people difficult to understand the contexts of the video. So an advanced privacy protecting technique is needed without losing the contexts of the video. We focus on changing the facial appearance of characters on the video for protecting their privacy without failing the information from facial expressions. We propose the system, called Mob Scene Filter that changes not only the facial textures but also the arrangement of the facial regions and their sizes.

Guidance Field

In virtual walkthrough system, many users cannot find important aspects of the virtual environment (VE) and cannot fully enjoy the VE. Guidance Field implicitly guides users to arrive at and gaze at predifined targets in the VE by altering users’ input operations. Demonstration experiments conducted in a real exhibit showed its efficiency at practical level.

FoodLog

"Life Log Infrastructure for Food" project is a cooperative research with Aizawa Lab. and Sasaki Lab. at the Univ. of Tokyo, KDDI R&D Lab., and foo.log Inc. "Food" is one of the most important and regularly consumed factor in our daily life. However, so far, this has rarely been viewed as an object of information processing. In this research, we aim at developing an infrastructure for lifelog, with emphasis on food and food-related activities in our daily life. Especially our laboratory focuses on the method to capture, analyse, visualize and utilize food-related activities.

Vocalog records and analyzes conversations in mealtime, and provides feedback to a user. It enhances mealtime communications.

Yumlog: Improvement of eating habits by social media

We know that healthy dietary habits are important, however high positiveness and strong will are necessary to perform them. That's because we cannot satisfy healthy foods in general. In cognitive psychology area, it is known that what level we satisfy a meal is influenced by others' evaluations. It also follows that we can satisfy healthy food by receiving highly rated evaluations. We can satisfy healthy meals so much that we get apt to eat more healthy meals.

Table Talk Enhancer: A tabletop system for enhancing and balancing mealtime conversations

Table Talk Enhancer is a tabletop system that enhances and balances mealtime conversations by feedback of lights on the surface.

Vocalog

Vocalog records and analyzes conversations in mealtime, and provides feedback to a user. It enhances mealtime communications.

We propose a method for changing the driver's departure time by predicting the future traffic conditions and informing the driver on how long the driver will expect to wait in a traffic jam based on the time at which the driver chooses to depart from the rest area.

You can manage daily tasks reasonably with WillDo, which forecasts your future state by analyzing lifelog. Available for iOS.

Task management using personal lifelog-based future prediction

There are various applications for management of daily tasks. These systems manage the rate of progression of tasks in the moment. We advance the systems and develop the system that forecast the future state of task progression for feedback to users and help efficient task schedule. For forecasting, we use lifelog that include time for tasks, sleep, movement, housework and so on.

PRIMA

PRIMA can record and reconstruct the real space as 3-D data in realtime using four depth cameras.

We propose a method for changing the driver's departure time by predicting the future traffic conditions and informing the driver on how long the driver will expect to wait in a traffic jam based on the time at which the driver chooses to depart from the rest area.

WillDo

You can manage daily tasks reasonably with WillDo, which forecasts your future state by analyzing lifelog. Available for iOS.

Multimodal & Crossmodal Interface

While visual and auditory information are mainly used for exsiting interfaces, other modalities are not used much. We are developing multi-modal interfaces such as haptic, olfactory and gustatory displays in order to realize intuitive virtual reality environments. Moreover, we are focusing on cross-modal interfaces which present ultra-realistic experience with limited sensory information by using cross-modal interactions.

We propose a method for improving the effects of manipulation in redirected walking (RDW) by using haptic cues, particularly for the discrimination of path curvature. We develop an RDW system, which displays a visual representation of a flat wall and users virtually walk straight along it although, in reality, users walk along a convex surface wall with touching it. Using this system, we conduct an experiment, and the results show that our method reduced the amount of perceived curvature in RDW as compared to an only visual method.

Metamorphosis Hand provides users with interactive experiences of altering perceptual body image. You can play the virtual piano with virtual hands that move synchronously with your own hands but whose appearance/movement are far different from your own. That makes you feel like the virtual hands are partly “something like me” and partly “something not like me”.

Visuo-Haptic Redirected Walking

We propose a method for improving the effects of manipulation in redirected walking (RDW) by using haptic cues, particularly for the discrimination of path curvature. We develop an RDW system, which displays a visual representation of a flat wall and users virtually walk straight along it although, in reality, users walk along a convex surface wall with touching it. Using this system, we conduct an experiment, and the results show that our method reduced the amount of perceived curvature in RDW as compared to an only visual method.

We propose a novel method for designing hand-held VR controllers which illusorily represent haptic equivalent of visual shape in VR. Though the actual appearance of the designed controller differs from that of the object in VR, the user perceives as if he handles the object.

ARAtouch: Visuo-Haptic interaction with Mobile Rear Touch Interface

"ARAtouch" is a visuo-haptic system which is able to evoke the haptic sensation on the mobile device, without any haptic devices, only using haptic illusion. Touch panel is a compelling input modality for interactive devices. However, touch input is inaccurate due to occlusion by a user's finger. To solve this problem, the rear touch interface is actively researched and commercialized. Meanwhile, there are increasing numbers of works which focus on passive haptics, which include pseudo-haptics. Pseudo-haptics is a kind of cross modal effect between our visual and haptic sense, which indicates an illusional perception in our haptic sensation evoked by vision. To provoke the pseudo haptic effect, it is need to hide a user’s hand or put it other place. So it is impossible to provoke pseudo-haptic effect, for the normal type of touch panel which we contact directly. Meanwhile, with the rear touch interface, we are able to evoke pseudo-haptic effects because users' hand are hidden behind the screen. To realize a rear touch interface, we combine two mobile tablet back to back. As well, pressure sensors were attached between these two devices to measure the pressure users push. A mirror was set in front of users and a back side device capture a hand image reflected with it with its front camera. To evoke effective haptic illusion, deformed hand shadow composed from captured image is shown in the screen as an alternative to a computer mouse cursor, and the system deforms virtual objects and resize the marker which shows the touching point as to the pressure which was measured by a pressure sensor attached between devices. These methods are critically useful for inducing haptic feedback of the tangential and normal force on the small mobile device for the useful GUI or games.

Augmented Endurance

"Augmented Endurance" is a method for alleviating fatigue during handling medium-weight objects and augmenting our endurance by affecting our weight-perception with augmented reality system. This system changes the visual property of handling objects, in this time, the brightness value to control the weight perception. This visual feedback affects our produced muscle power to handle objects. Then we can reduce excess energy and control our endurance unconsciously by modifying the brightness value of objects.

Tabletop Augmented Satiety

This is a tabletop system for affecting our perception of satiety and controlling energy intakes by controlling a size of a projected image around the food.

Augmented Satiety

"Augmented Satiety" is a system to modify satiety perception implicitly and control nutritional intake interactively according to the meal and its nutritional value by changing the factors with interactive computer graphics techniques, object recognition techniques and augmented reality paradigm. We proposed a method for food-volume augmentation using shape deformation processing in real-time. With this method, we can scale food while ensuring that a subject perceives that they were really holding the food. This "real" feeling of holding the food enables a comparison of the size of a food item and the subject’s hand; this is effective for creating the illusion of apparent food size and the perception of satiety.

MagicPot

"Magic Pot" is a system which enables us to experience a haptic illusion as if users were touching a virtual magic pot shaped by users, even though they are touching only a physically static cylinder in fact. In this system, we compose visual feedback as if we were touching another virtual shape by distorting the movement and the posture of a hand, based on the detection of the point where they are touching the physical static object.

Meta Cookie

"Meta Cookie" is the world's first cross-modal gustatory display to change the perceived taste of a cookie by overlaying visual and olfactory information onto a real cookie. The system induces cross-modal effect among vision, olfaction and gustation to let humans perceive various tastes without changing chemical substances. Experimental results showed that this system can change the perceived taste on 80% of the trials.

User Support of Physical Labor using Pseudo-haptics in a Mixed Reality Space

We managed to enhance the performance of object lifting tasks by modifying the visual movement to manipulate the perception of weight.

Improvement of drinking habit with augmented reality

This research proposes a system for implicitly influencing the satisfaction we experience while drinking a beverage and for controlling beverage consumption.

Subjective VR & Reflex-based Interface

The purpose of this project involved assessing the efficacy of computer-generated emotion and establishing a method for integrating emotional experience. In the field of cognitive science, the ability to alter a bodily response has been shown to unconsciously generate emotions. We therefore hypothesized emotional experience could be manipulated by having people recognize pseudo-generated bodily reactions as changes to their own bodily reactions and are developing emotion evoking interfaces.

"Interactonia Balloon" is an artwork to raise awareness of own respiration, and evokes and enhances tense feeling by synergistic influence of intentional change of own respiratory condition and a paradoxical experience caused by active of the balloon depending on the condition.

Incendiary Reflection aims to create computer-generated emotion by allowing users to recognize pseudo-generated facial expressions as changes to their own facial expressions. The approach manipulates emotional states through feedback of deformed facial expressions (artificial smiley faces and artificial sad faces) in real time. A user study showed that the system can change emotional states; not only positive and negative affect, but also preference decision.

Interactonia Balloon

"Interactonia Balloon" is an artwork to raise awareness of own respiration, and evokes and enhances tense feeling by synergistic influence of intentional change of own respiratory condition and a paradoxical experience caused by active of the balloon depending on the condition.

"Routine++" is a new technique to implement a pre-performance routine in a short period of time by providing a user artificial successful experiences via a simulator.

Digital Public Art

Digital Public Art is a new genre of art produced by bringing information technology into public spaces and evolving media art into public art. We are convinced that by fusing the interactive and global characteristics of media technology - think of computers - with the powerful presence and the ability to transform locations of public art - think of sculpture - we can transcend these separate viewpoints to stimulate people's awareness of media technology.

This system enables to manipulate the transfer path of electricity, using the properties of sound waves.

Sharelog 3D

Sharelog 3D is a piece of interactive art that visualizes the history of trains the user took from contactless fare cards. Up to 20 paths are mapped on the city image, which is made from photographs using Image Based Rendering.

Controllable particle display using water drops

Research of touchable and volumetric visual display using water drops as pixels.

"JUKE Cylinder" is a piece of interactive art installation to augment sound images through hand interaction and to metamorphose hands into a musical instrument. Users are able to control the pitches of the sound like real musical instruments. With our system, people can feel that the sound is generated on their hands, and have an illusion that their hands metamorphose to a musical instrument.

This system enables to manipulate the transfer path of electricity, using the properties of sound waves. Sound waves emitted from parametric loudspeakers are received by a circuit 3cm wide, and are converted to electricity. Because sound waves can be reflected or shielded by objects such as hand and paper, it is easy to manipulate the transfer path of electricity. In our research named VITA, we tested how much people could transfer electricity into these devices by reflecting the sound waves.

We design the interface that enables user to select hearing sound source under the multiple sound circumstance.

Toward inplementing Mosaic-type work for seniors

The objective of this research topic is to exploit the power of senior citizens for revitalizing our society. We propose a mosaic-type work system in which elderly human resources are combined to form a single ``virtual worker'' based on seamless information sharing. This system can enable seniors to work flexibly and the society to ensure stable workforce. Based on social computing technology, we try to systematize virtual workforces through gathering various work profiles of seniors and then matching them to jobs at the seniors' surroundings.

Support system of Time-mosaic formation for seniors

Although the challenges posed by aging populations are great, they can be offset to some degree by harnessing the increasing vitality and productivity of senior citizens in developed countries. To improve work opportunities for seniors and to make use of their abilities, we propose a mosaic-type work system in which elderly human resources are combined to form a single ``virtual worker'' based on seamless information sharing. In this research topic, we particularly focus on part of the mosaic, the time-mosaic, for the stable worker generation in the case of complex irregular work. The objective is to demonstrate the effectiveness of the time-mosaic formation system in e.g. cooperative farms for shift work organizations.

An avator robot for spatial mosaic

In this research topic, we are developing a support system of remote work for seniors, an avator robot system. In particular, a senior-friendly interface is being implemented for help them remotely to tell their knowledge, experience, and skill to young people.

ZooMap: Acoustic Information Zooming Interface

We design the interface that enables user to select hearing sound source under the multiple sound circumstance.

Transcendent Telepresence

Many previous studies on telepresence have extensively investigated conveying distant places exactly. However, it is not necessarily effective to convey exact information. We believe that the capability to transmit information in a modified form is an advantage of telepresence. Thus, we proposed ‘‘Transcendent Telepresence,’’ which uses this advantage positively. We aim to achieve remote communication that is superior to face-to-face communication.

FaceShare is a videochat system which evokes the positive effects of mirroring by generating pseudo-expressions congruently with your partner.

Enhancing the Effects of Social Touch by Changing Gender Impression

A touch in communication (social touch) has various positive effects such as triggering altruistic behavior. However, it is known that these positive effects are limited or become contrary effects in same-gender communication. We are proposing a method to draw the positive effects of social touch between persons of the same gender by changing the operator’s gender impression.

"Transgazer" an Optical Illusion System for Direct and Averted Gazes

Transgazer is a system of robotic eyes that can send averted or direct gazes simultaneously to multiple recipients by switching between convex and hollow eye shapes.Convex eyes can convey gaze direction more correctly whereas hollow eyes can direct a gaze more broadly and eye contact with many people when there is a large audience.

We propose an entertainment system that aims to set the mood for kiss, and enjoy kiss as an entertainment. A kiss is the act to communicate among couples, and has an important role to strengthen bonds among couple. To enhance that action among couples, we developed the digital photo frame, The system detects a physical contact during the kiss by using the electro dermal activity, and photos of couple shown in the digital photo frame are decorated with cute icons by the kiss. We also tested the effectiveness of this system in wedding parties. All of wedding couples kissed with delight, and all of guest were also able to enjoy by using this system as an entertainment for wedding parties.

Desktop Kingdom

"Shikkuri" feeling in interaction in virtual reality space

We aimed to find out and evaluate the factors of "Shikkuri" feelings in doing unreal operations in VR space.

Divisible planar object detection for AR applications

In this research study, we propose a divided planar-object detection method for augmented reality(AR) applications. The proposed system prepares a database of the target object's natural features, and applies progressive sample consen-sus(PROSAC), which is a robust estimation method, for iterativehomography calculation to achieve the multiple planar-object detection. Moreover, the proposed method can detect shapes of pieces by simultaneously using an occlusion detection method. We demonstrate that it is possible to interact with an arbitrarily divided planar object in real time by our method to implement some AR applications.