A study of the influence of auralization on speech intelligibility and immersion in multi-party teleconferencing systems using binaural audio

Abstract

The introduction of spatial audio features into conventional teleconferencing systems is a natural way of creating a realistic communication environment and was introduced many years ago. The human auditory system takes advantage of spatial audio cues to locate, separate and understand multiple speakers when they talk simultaneously. As a result, speech intelligibility is significantly improved if the speakers are simulated to be spatially distributed around each participant and also immersion and sense of presence improves. Moreover, it is possible to increase even more this immersion and sense of presence if auralization is introduced in order to simulate the effects of a virtual room where the virtual meeting is taking place. However, if auralization introduces too much reverberation or harmful early echoes, speech intelligibility can be affected. In this paper, we thoughtfully study the influence of auralization on speech intelligibility and sense on immersion in multi-party teleconferencing systems using binaural audio. For carrying out the experiments specific software for multi-party teleconference that employs binaural audio using headphones has been developed. The software runs on smartphones and tablets so touch screens can be used to move and place participants in the virtual room using avatars. Different subjective experiments have been carried out trying to evaluate the influence of the early echoes, the late reverberation, the size of the room and other details related to binaural processing. Results provide interesting guidelines for developing teleconference systems with auralization effects.