SonoMorphis

Interactive Installation with Genetic Graphics and Sound

Authors

Concept

An organic object is projected in front of the visitors. By means of a control mechanism the user can
rotate the object in all directions and observe it from various perspectives. Control sliders allow the
viewer to vary diverse parameters of the object. The graphics and sound are inseparably linked to each
other. In this way the space is always filled with new audiovisual bodies. "SonoMorphis" represents
the attempt to creatively apply the results of our studies in open computer systems.
In this work the interaction follows the evolutionary principle of mutation of the graphic and acoustic
structure and selection from six available variants. On the visual level, specific formal patterns that
have been extracted from the natural world are combined arbitrarily and generate creations that are
both familiar and yet have never been seen before. The momentary state of the graphic objects controls
the sound level in "SonoMorphis". The parameters of the graphics must be interpreted from acoustical
viewpoints in such a way that a musical structure emerges from them. In this way automatic
compositions arise, the results of which are functions of their components and are variable in the
details of their contours, complexity, and their behaviors. The overlapping of visual levels and sound
levels produces an open structure that can be continually and endlessly configured in new ways by each
viewer.

The installation is accessible from two coupled access points. The first of these exists in real space,
visitors interact with a virtual organic projected onto a screen via a special interface box. The second
access point uses the world wide web as its user interface.

In both systems, users evolve a three-dimensional organic object which is created using genetic
algorithms. The organic is defined by a genom, a set of components, which is successively mutated by
the users. Out of six randomly generated mutations users select one, which in the succeeding steps
becomes the starting point for new mutations. This way users choose a thread through a space out of
approximately 1080 possible forms.

In the real space users additionally change the shape and dynamic behavior of the life-like organic
object via an interface box. Both systems are coupled and operate on the same data set constituting the
genom. Actions in the web space effect the real space and vice versa. If a change on the web happens,
the organic in the real space slowly morphs towards the web selection, a change in real space directly
affects the next web action.

Sound

The basic idea is to put the sonic and visual representations of the installation's structure into as close a
relationship as possible. Material, shape, and spatial position of the visible organic correlate with
sound properties like timbre variations, pitch alterations, dynamically moving positions, and the like
of the projected sound in the real space. The correlations rely in part on easily graspable rules for the
sake of clarity, in part on juxtaposition and more sophistacated transformations.

Both sound and projection relate in equal parts to the same underlying abstract structure which they
make palpable to the user. The sound acoustically represents selected properties of the genoms, i.e.
their structure, position, and behavior in a non-arbitrary way. The easiest way to think of this
representation metaphorically is that of a musical instrument: a set of rules with associated variables
by which to generate sound, with the possibility included to control these variables in real-time
according to the underlying genoms' structures.

As one of the installation's aesthetic goals is the bodily impression of the generated object on the user,
a sound synthesis technique was in demand, that is able to both render a visible object's genuine sound
thru all its user-inferred alterations in shape and space in a plausible way, and to be abstract enough
where needed to not duplicate a real-world artefact. The technique of choice is known as physical
modelling which derives the emerging sound from the physical properties of an assumed object, i.e. its
shape, material, excitation mode etc.

Based on associative relationship to the genoms' textures, each acoustic representation has first been
assigned a set of material properties, causing its basic timbre. Second, the genoms' shape is taken into
account, controlling the representations' basic modes of vibration and their reaction to
parameter-induced deformations. Third, the single graphic objects' current spatial positions are
mapped to the sound space, rendering their horizontal movement as well as their proximity to the user.

It is possible and intended to handle the installation as flexible as a musical instrument, consisting of
an image and a sonic component. Observation of the system's behavior during exhibitions has shown
its ability to respond to users' varying approaches, playing styles, and temperaments in a differentiated
and recognizable way.

Implemention

The software runs on an SGI computer for realtime animation of the organic as well as for creation of
the images required for the web pages. A web server running on that machine creates mutations on the
genom and sends them with related images back to web users. The SGI communicates via TCP/IP with
two Macintoshs generating the sound control data.

A real time application cares for a life like animation of the object and reacts on user actions. Users
interact with an interface box. The interface's eight sliders are used to manipulate the object's dynamic
parameters and the camera position. Three additional buttons are used to trigger mutations. The
application and the webserver communicate via an IPC/TCP connection to distribute changes of the
genom.

A genom is composed of components each of which define a single structural or shape property of the
organic. There are components which describe the geometry of a single limb and components which
arrange limbs according to specified algorithms. Components are designed such that they incorporate
form principles observed in nature, e.g the proportion of the golden section is used for the simulation
of spiral phylotaxis.

There is a predefined set of components constituting the gene pool out of which the actual genom is
assembled by random. Assembling operations are insertion, removal and change in connectivity
(crossover) of components. Additionally, each component holds a set of parameters defining details
like thickness, curvature, color, texture etc.

Presentation

SonoMorphis has been shown at the exhibition surrogate1, Nov 1st - Dec 6th 1998 at the ZKM
Institute for Visual Media.
The image was projected onto the front wall of a room of 6m width and 7m
depth. The projection size was 5 x 3m. The interface consists out of a mixing board plus chair which
were installed 4,50m in front of the screen. The graphics ran on an SGI Onyx2 including the webserver
as a graphical application. The image was projected in stereo using Chrystal Eyes Shutter Glasses.

The moving sound sources were projected quadrophonically with two speakers in front and two in the
rear. The height of the speaker mountings approached average ear level. The sound generation ran on a
dedicated configuration of two Apple PPCs and Yamaha sound generators.

SonoMorphis has also been shown during the 11th Stuttgarter Filmwinter, Jan
14th - 17th 1999, in the CAVE
of the IAO, Fraunhofer Gesellschaft in
Stuttgart. The user interface was located in front of the CAVE, at the Media
Summer Festival, Schloss Kapfenburg, 2001, and at the"Art of Immersion"
Festival, CAVE der GMD Bonn, 2002

Software

The software has been developed by the authors and is based on Xfrog and Max.

Production Support

ZKM | Karlsruhe
Institute for Visual Media and Institute for Music and Acoustics