Saturday, September 29, 2012

In about a weeks time the 13th ISMIR (International Society for Music Information Retrieval) conference will be held. This is a conference on the processing, searching, organizing and accessing music-related data. It attracts a research community that is intrigued by the revolution in music distribution and storage brought about by digital technology which generated quite some research activity and interest in academia as well as in industry.

In this discipline, referred to as Music Information Retrieval (or MIR for short), the topic is not so much to understand and model music (like in the field of music cognition), but to design robust and effective methods to locate and retrieve musical information, including tasks like query-by-humming, music recommendation, music recognition, and genre classification.

A common approach in MIR research is to use information-theoretic models to extract information from the musical data, be it the audio recording itself or all kinds of meta-data, such as artist or genre classification. With advanced machine learning techniques, and the availability of so-called ‘ground truth’ data (i.e., annotations made by experts that the algorithm uses to decide on the relevance of the results for a certain query), a model of retrieving relevant musical information is constructed. Overall, this approach is based on the assumption that all relevant information is present in the data and that it can, in principle, be extracted from that data (data-oriented approach).

Several alternatives have been proposed, such as models based on perception-based signal processing or mimetic and gesture-based queries. However, with regard to the cognitive aspects of MIR (the perspective of the listener), some information might be implicit or not present at all in the data. Especially in the design of similarity measures (e.g., ‘search for songs that sound like X’) it becomes clear quite quickly that not all required information is present in the data. Elaborating state-of-the-art MIR techniques with recent findings from music cognition seems therefore a natural next step in improving (exploratory) search engines for music and audio (cognition-based approach) (cf. Honing, 2010).

A creative paper, discussing the differences and overlaps between the two fields in dialog form, is about to appear in the proceedings of the upcoming ISMIR conference. Emanuel Bigand –a well-known music cognition researcher–, and Jean-Julien Aucouturier –MIR researcher–, wrote a fictitious dialog:

“Mel is a MIR researcher (the audio type) who's always been convinced that his field of research had something to contribute to the study of music cognition. His feeling, however, hasn't been much shared by the reviewers of the many psychology journals he tried submitting his views to. Their critics, rejecting his data as irrelevant, have frustrated him - the more he tried to rebut, the more defensive both sides of the debate became. He was close to give up his hopes of interdisciplinary dialog when, in one final and desperate rejection letter, he sensed an unusual touch of interest in the editor's response. She, a cognitive psychologist named Ann, was clearly open to discussion. This was the opportunity that Mel had always hoped for: clarifying what psychologists really think of audio MIR, correcting misconceptions that he himself made about cognition, and maybe, developing a vision of how both fields could work together. The following is the imaginary dialog that ensued. Meet Dr Mel Cepstrum, the MIR researcher, and Prof. Ann Ova, the psychologist.”

Friday, September 14, 2012

Isabelle Peretz, co-director of the International Laboratory for Brain, Music and Sound Research (BRAMS), told me about Mathieu during a workshop at the Université Libre de Bruxelles in November 2009. She was very excited, and was pretty sure she found a 'beat-deaf' person. I couldn’t but share her enthusiasm. In Phillips-Silver et al. (2011) Peretz and her team wrote:

'Mathieu was discovered through a recruitment of subjects who felt they could not keep the beat in music, such as in clapping in time at a concert or dancing in a club. Mathieu was the only clear-cut case among volunteers who reported these problems. Despite a lifelong love of music and dancing, and musical training including lessons over several years in various instruments, voice, dance and choreography, Mathieu complained that he was unable to find the beat in music. Participation in music and dance activities, while pleasurable, had been difficult for him.'

About one year later her group published a journal paper presenting some behavioral evidence that Mathieu was a case of congenital beat deafness.

The questions posted in a blog entry just after the publication of that study resulted in a collaboration in which, next to behavioral, also direct electrophysiological methods were used. Pascale Lidji (also associated with BRAMS) initiated an EEG/ERP experiment, modeled after our earlier Amsterdam experiments, to directly probe Mathieu’s apparent beat-deafness.

Last winter, just a few weeks after the experiments, we had a teleconference discussing the first experimental results (filmed by a Dutch TV crew following our work). The first results suggested that Mathieu’s brain did pick-up the beat, but his conscious perception did not, as several behavioral experiments confirmed. Intriguing, to say the least. And the results will hopefully be published later this year.

See below for some fragments from the teleconference:

For more the documentary De man zonder ritme, see the website of NPO3.