This paper presents an IMU (Inertial Measurement Unit) based human gesture recognition system
for a robot instrument player to understand the instructions dictated by an orchestra conductor
and accordingly adapt its musical performance. It is an extension of our previous publications on
natural human-robot musical interaction [28]. With this system, the robot can understand the real-
time variations in musical parameters dictated by the conductor's movements, adding expression to
its performance while being synchronized with all the other human partner musicians. The enhanced
interaction ability would obviously lead to an improvement of the overall live performance, but also
allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance,
thanks to the complete naturalness of the interaction.

Description:

This item is embargoed until 26-02-2015.

Sponsor:

This research was partially supported by a grant by STMicroelectronics, which also provided
the core sensors and the microcontroller. This work was also supported in part by: the Global
COE Program "Global Robot Academia", MEXT, Japan; and the Consolidated Research Insti-
tute for Advanced Science and Medical Care, Waseda University (ASMeW). The authors would
like to express their thanks to the Italian Ministry of Foreign A airs, General Directorate for
Cultural Promotion and Cooperation, for its support to RoboCasa. The authors would also
like to express their gratitude to Life Performance Research, Okino Industries LTD, Japan
ROBOTECH LTD, SolidWorks Corp, Dyden, for their support to the research, and Professor
Suzuki of the University of Tsukuba for the meaningful discussions.