As a research assistant at the Metacreation Lab, I worked on the sound re-synthesis
problem. Given a target sound, it can be difficult and tedious to find the right set of parameters to
re-synthesize it with a given synthesizer. It is even more complex to come up with a new synthesis
techniques able to synthesize it.

I explored how Evolutionary Computation can be used to automate this task. I published
several papers in international conferences and journals and defended my M.Sc. thesis about this
topic.

Genetic Algorithms and OP-1

The OP-1 is a popular commercial synthesizer designed by the Sweedish company: Teenage Engineering (TE).
The OP-1 contains several synthesis engines, effects and low frequency oscillators, which make the
parameters search space very large and discontinuous.

In collaboration with TE, we addressed the problem of automatically calibrating the parameters of the OP-1
to approximate a given target sound using a multi-objective genetic algorithm.

Genetic Algorithms and ModFM synthesis

Modified FM (Mod FM) is a synthesis technique that can be used to generate harmonic instrument
sounds.

Using a genetic algorithm and a fitness function based on harmonics analysis, our system is able to
automate the calibration of a ModFM synthesis model for the reconstruction of harmonic instrument
tones.

Freepad

Freepad is a piece of software, developed at the Metacreation Lab, that enables anyone with a web camera,
paper and a pen, to draw their own MIDI interfaces. In three simple steps it allows the user to
establish an interface which can send out MIDI notes or trigger keyboard commands for use with your
preferred sampler or software of choice.

I extended this interface to work with the popular RTS game: Starcraft-II and showed that it can improve the gamer's
performance.

Calibrating a sound synthesizer to replicate or approximate a given target sound is a
complex and time consuming task for musicians and sound designers. In the case of the OP1,
a commercial synthesizer developed by Teenage Engineering,
the difficulty is multiple. The OP-1 contains several synthesis engines, effects and low frequency
oscillators, which make the parameters search space very large and discontinuous. Furthermore,
interactions between parameters are common and the OP-1 is not fully deterministic.

We address the problem of automatically calibrating the parameters of the OP-1 to
approximate a given target
sound. We propose and evaluate a solution to this problem using a multi-objective
Non-dominated-Sorting-Genetic-
Algorithm-II. We show that our approach makes it possible to handle the problem complexity, and
returns a small set of presets that best approximate the target sound while covering the Pareto
front of this multi-objective optimization
problem.

A sound synthesizer can be defined as a program that takes a few input parameters and returns a sound.
The general sound synthesis problem could then be formulated as: given a sound (or a set of sounds) what
program and set of input parameters can generate that sound (set of sounds)?

We propose a novel approach
to tackle this problem in which we represent sound synthesizers using Pure Data (Pd), a graphic
programming language for digital signal processing. We search the space of possible sound synthesizers
using Coevolutionary Mixed-typed Cartesian Genetic Programming (MT-CGP), and the set of input parameters
using a standard Genetic Algorithm (GA).

The proposed algorithm coevolves a population of MT-CGP graphs,
representing the functional forms of synthesizers, and a population of GA chromosomes, representing
their inputs parameters. A fitness function based on the Mel-frequency Cepstral Coefficients (MFCC)
evaluates the distance between the target and produced sounds. Our approach is capable of suggesting
novel functional forms and input parameters, suitable to approximate a given target sound (and we hope
in future iterations a set of sounds). Since the resulting synthesizers are presented as Pd patches, the
user can experiment, interact with, and reuse them.

Genetic Algoritms and ModFM - SMC 2012

Many audio synthesis techniques have been successful in reproducing the sounds of
musical instruments. Several of these techniques require parameters calibration. However, this task
can be difficult and time-consuming especially when there is not intuitive correspondence between a
parameter value and the change in the produced sound. Searching the parameter space for a given
synthesis technique is, therefore, a task more naturally suited to an automatic optimization
scheme.

Genetic algorithms (GA) have been used rather extensively for this purpose, and in particular for
calibrating
Classic FM (ClassicFM) synthesis to mimic recorded harmonic sounds. In this work, we use GA to
further explore its modified counterpart, Modified FM (ModFM), which has not been used as widely,
and its ability to produce musical sounds not as fully explored. We completely automize the
calibration of a ModFM synthesis model for the reconstruction of harmonic instrument tones using
GA.

In this algorithm, we refine parameters and operators such
as crossover probability or mutation operator for closer match. As an evaluation, we show that GA
system automatically generates harmonic musical instrument sounds closely matching the target
recordings, a match comparable to the application of GA to ClassicFM synthesis.

Freepad - IHCI 2012

The field of sketching interface design in regards to video game is relatively young and
has not been investigated in great depth. Freepad
is a custom paper-based MIDI musical interface. We describe an extension to Freepad that supports
user customization for real time strategy games. Using only a webcam, a pen and a sheet of paper,
players can design their own interface by drawing shapes and linking them to simple or complex
actions in the game. In an user study, we use this extended Freepad to explore the potential of
sketching interfaces in strategy video games. Our results indicate that using Freepad improves the
efficiency of players and their enjoyment in this kind of games.