Russian Researchers Develop Virtual Composer Assistant

Experts from the National Research Nuclear University MEPhI's Institute of Cyber Intelligence Systems are currently developing brand new software, a virtual composer assistant, which will be able to analyze the emotional state of composers and follow their logic, the university press service explained to RIA Novosti.

Despite the high level of development of music theory, the process of creating music is still hard to be formally algorithmized, since it is inseparably linked with the emotional experiences of the composer. This aspect of the creative process is of particular interest to experts in machine intelligence.

Scientists can already create synergy between humans and software in a way that allows the computer, which is using the software to gather data on the emotional state and the intentions of the creator, to become a continuation of the human mind and body.

Virtual composer assistant is software that can create human-level music of high aesthetic quality on its own. It is a creative smart assistant that can add to a melody written by the composer its own notes, chords and combinations, employees of the MEPhI's Institute of Cyber Intelligence Systems told RIA Novosti.

In order to achieve this goal, researchers had to analyze the theory of music (identify musical sounds, chords, their combinations), create semantic maps based on this analysis and connect them to models of humans' emotional perception of music.

The new software is fundamentally different from similar developments, since the virtual assistant can maintain emotional contact with the composer, said Professor Alexei Samsonovich with the MEPhI Institute of Cyber Intelligence Systems.

"Most such programs represent neural networks taught to write music or make art," he said. "But our idea was to identify the emotional state of the composer and follow its logic."

Semantic Maps Instead of Neural Networks

This method is based on using semantic maps instead of neural networks, Samsonovich said. In semantic maps, the emotional state of one's mind is represented by a point in the affective space. In this space, every possible action has its own coordinates, and there is a certain rule that allows researchers to connect various coordinates with the others by following the logic of emotional perception. The laws of musical harmony and rhythm impose certain limitations on this process.

"These limitations, however, cannot define everything; they leave some freedom for creativity," Samsonovich explained. "This is the freedom that our software uses, while being guided by the semantic map and the rules of choosing the coordinates on it, and while taking into account the evolution of the emotional state. I would like to point out that we are not talking about machine learning, as the model is based on the principles of psychology (basic psychological aspects of behavior). In this sense, unlike neural networks, models like this can explain the essence of the phenomenon of creativity."

The mutual understanding between the composer and the virtual assistant can be achieved through organizing and storing various types of emotional reactions to musical elements — chords, their sequence, and intervals — in the software environment.

In order to ensure storage and use of the information about the most successful and fitting combinations of sounds, the virtual assistant uses a semantic map, which allows the virtual composer not only to build on a melody set by the user according to his taste, but also monitor his mood, suggesting different versions and creating the proper accompaniment.

Equations for Mood

In the course of the method's development, researchers used a separate application in order to test how participants evaluate various combinations of chords according to three measures, each judged on a scale from one to ten: pleasant-unpleasant, cheerful-sad and exciting-boring. This evaluation system allowed researchers to create three-dimension semantic maps.

"We assumed that one's choice of behavior based on social emotions, feelings and relations can be described by relatively simple dynamic equations," Samsonovich said. "The simple model of behavior based on the semantic map and two equations is indistinguishable from the human's behavior in the simplest of paradigms of social interaction in a virtual environment."

Samsonovich also added that the most promising approach in this field of research is based on cognitive architectures that are created using psychology and neuroscience data — the so-called Biologically Inspired Cognitive Architectures (BICA).

The researchers believe that the new development can be scaled up and used not only in music, but also in many other areas of digital creativity, as well as in human-machine systems. Creating and implementing such innovations is an important step in the study of the socio-emotional aspect of intelligence, which is responsible for the recognition and expression of emotions, intentions, motivations and desires.

The key results of this research were published in the Procedia Computer Science journal.

Hello,
!

We are committed to protecting your personal information and we have updated our Privacy Policy to comply with the General Data Protection Regulation (GDPR), a new EU regulation that went into effect on May 25, 2018.

Please review our Privacy Policy. It contains details about the types of data we collect, how we use it, and your data protection rights.

Since you already shared your personal data with us when you created your personal account, to continue using it, please check the box below:

I agree to the processing of my personal data for the purpose of creating a personal account on this site, in compliance with the Privacy Policy.

If you do not want us to continue processing your data, please click here to delete your account.

promotes the use of narcotic / psychotropic substances, provides information on their production and use;

contains links to viruses and malicious software;

is part of an organized action involving large volumes of comments with identical or similar content ("flash mob");

“floods” the discussion thread with a large number of incoherent or irrelevant messages;

violates etiquette, exhibiting any form of aggressive, humiliating or abusive behavior ("trolling");

doesn’t follow standard rules of the English language, for example, is typed fully or mostly in capital letters or isn’t broken down into sentences.

The administration has the right to block a user’s access to the page or delete a user’s account without notice if the user is in violation of these rules or if behavior indicating said violation is detected.