Nishimura, Masafumi

Research outline

Sound signals, especially speech, contain various kinds of information and we utilize them in our daily lives unconsciously. However, they are not well leveraged by computer systems, yet. This cognitive computing team aims to develop useful and practical systems, which provide effective recording and analysis of sound signals, information extraction including speech recognition, and spoken dialogue management.

Recognition of Daily Activity and Physical Condition by Using Sound Signals

This research project focuses on sound signals, which are not well leveraged by computer systems, for recognition of daily activity and physical condition of aged people. We have already proposed a simple recording system which consists of throat and lavalier microphones. By using this system, we can get various kinds of sounds such as swallowing, drinking, coughing, chewing and snoring besides speech. We continue improving recognition performance of these information for taking care of and supporting aged people more efficiently.

Multi-party interaction analysis

This research project focuses on multi-party interactions. Analysis of multi-party conversation by using speech recognition is in great demand, but it is still very hard task even for the latest ASR such as "Siri" because of simultaneous speech in conversation. We are tackling this problem by combining throat microphone with state-of-the-art speech technologies.

Spoken Dialogue System

This research project aims to develop a spoken dialogue system for aged person, which provides schedule management, cognitive function evaluation, physical condition evaluation and conversation capability, by leveraging not only recognized texts but also para-linguistic and non-linguistic information such as laugh, nod and sigh.