Project Introduction

Diagram SciFi Karaoke is an installation art project focusing on the human utterance. Basically, the karaoke installation consists of a "smart" microphone and a videoprojector that projects texts on the walls. The microphone should invite the audience to participate in the performance (Audience participation), inviting them to react to the projected materials or to other participants in whatever way they feel like (and are inspired to). The microphone is connected to hardware/software that takes the voice signal of the participants as input, processes it in various manners, and gives the processed signal back to the environment so that it becomes part of the performance. The audience will start to explore the response of the system, and the (sometimes unexpected) system response is supposed to trigger the communication between the attendants. The installation is planned to take place at the end of 2004.

Project and Objectives

The central objective of the project is to explore ways by which a system can react to properties of the human voice, most notably the emotional aspects of the voice and the (emotional and semantic) contents of utterances that are triggered by the context, and through its reactions enable forms of communication. Examples: With respect to the emotional aspects, the system might extract acoustic (pitch, volume, speaking rate) correlates of emotion, manipulate the speech signal in particular ways and synthesize utterances that react to the input utterances. With respect to the emotional and semantic content, the system might listen for particular words and generate relevant messages that act as triggers for further interaction.

Learning objectives concerning main Competencies (additional Competencies - Social and Cultural Awareness; Market orientation - may be addressed where applicable): 1. Emphasis in P0302 will be on Idea and Concpet Generation, more specifically on Brainstorming: students will explore several brainstorming techniques and learn to make a motivated choice of techniques give an particular purpose 2. With respect to Competency 3 (User Focus and Perspective), students will get an overview of methods for the identification of Requirements and experience how the Requirements constrain the concept generation stage. In addition, the research and design process makes the students familiar with particular aspects of communication. 3. With respect to Competency 2 (Integrating Technology), students will get insight into properties of human speech and of the human voice in relation to emotion, and apply signal processing techniques relating to speech (automatic speech recognition) and the human voice (signal manipulation techniques) 4. With respect to Competency 6 (Visual Language), students will learn to propose motivated solutions fitting the requirements: choosing expressive elements of a multimedia performance that implement the central objectives of the performance.

Project steps

Orientation stage: what the project is about. Outcome: initial scenarios

Client Information

Ingeborg Houwen has a Master's degree Theaterwetenschap from the University of Amsterdam, with minors in Taalfilosofie, Filosofische Antropologie, Esthetica and Literatuurwetenschap. She is author, cultural entrepreneur, performer with a strong interest for new media, and the initiator of Diagram (see http://www.iice.nl/diagram-site/karaoke.html). She organized cultural exchanges and was one of the organizers of Cultural Center UI in 1997. Houwen wrote texts for the theatre, published in journals (including a serial in Folia Civitates) and is preparing a second novel. Contact: ingeborg@iice.nl