Industrial algorithms inspire interactive contemporary dance

Working in industrial quality control is not generally
associated with creativity. But when German computer engineer Frieder Weiss watched an
inspection system process car parts in 1992, he began to wonder
what would happen if you stuck a person in front of that system.
Then, what would happen if you connected that system up to a
synthesiser and got that person to dance. The answer, he found, was
something quite beautiful: reactive dance that creates its own
music.

Weiss has been experimenting with sound and video motion sensing
technology ever since, and long before, Microsoft made anyone with
a Kinect into a DIY engineering artist. In that time, he's worked
with everyone in the arts from opera groups to Kylie Minogue, and even had a hand in bringing King Kong to life in an
upcoming musical. 19 and 20 October will see the London premiere of
his collaboration with Chunky Move -- an Australian contemporary
dance company. In Mortal Engine dancers compete with a
tilted mechanical stage, their every movement prompting soundbites
from composer Ben Frost's tracks and video and laser projections by
Robin Fox. Having made his Eyecon and Kalypso software to engineer
sound and image reactions to dancers' movements, you'd thing the
days of drawing inspiration from automated manufacturing processes
would be long gone. Not quite.

"It's a huge amount of research, I read papers of people
describing algorithms for manufacturing and I think of ways the
implementation could maybe do something nice," Weiss told
Wired.co.uk. "That's what I'm doing a lot; reviewing algorithms
that have been made for industrial uses in analysing objects and
movement. I'm kind of abusing them for something aesthetical, a
visual output of a new form -- I reuse algorithms for something a
little bit more useless."

Weiss began by using open sound
control networks to measure movement values, and assign a sound
according to how much motion has occurred. He immediately ran into
a pretty big obstacle with his creative applications of the
technology -- the human brain.

"The brain is not really trained to recognise these kinds of
responses," said Weiss. "If you raise your arm and you hear sound
it might just be a car outside, it's not necessarily that your arm
is triggering the sound. It's not really intuitive in that way. The
brain does a good job of blocking things out because sound is
omnidirectional -- you don't have to face it and focus on it, but
the brain is still capable of focusing on a discussion and
filtering out other noises. That's kind of cool, but it's a bit
against what we try to do. Our job is to try to make it connect, in
a really subtle way. We have to do it a number of times to convince
the audience."

When Weiss trailled the effect in the 1990s, the audience's
reaction was one of disbelief: "They'd say, 'why are the dancers
pretending to cause the sound'. They weren't used to
interactivity."

Video presented a more natural way of introducing the
technology. Using different styistic techniques Weiss maps video
projections onto the dancers' bodies, or uses it to stalk them on
stage, creating different effects. "It's more self explanatory," he
explained.

One popular technique is the particle effect -- a swarm-like
haze that traces the dancers' movements and morphs into anything
the designer, or the audience, can imagine. The subtleties of the
effects are achieved because they focus on tracing the dancers'
outline using vecto graphics. "We're not representing an image
through pixels -- there's an outline with these coordinate points.
It's descriptive, like vecto graphics of 20 years ago, but back
then the computers weren't so powerful. It allows you to build
effects that are special for that kind of data."

Mortal Engine is a great transition for Weiss. In a
former Chunky Moves collaboration, Glow, he used a solo
dance to really show off the different tricks that could be
achieved. Weiss called it a "formal study" of the technology. That
all changed when he began to use shadow bodies, which follow the
dancer with a time lag: "It became really emotional for people,
that's what they told me afterward, and I think that's what the
magic is -- it channels emotion". As time went on, it also became
less about the dancer versus the technology, and more about a
synergy between the two.

"When I first did the bit with the shadows, the dancer was
actually looking at the projection, in contact with the projection
and a little bit showing off what the trick was. Now, that's
totally gone now."

The system is so fast, Weiss says, the dancers are no longer
even aware of it. It's not about a projection highlighting a
dancer, or a dancer competing to outdo a video effect -- the two
are working together, and this is something that can only be
achieved when the dancer becomes lost in the world Weiss creates
for them.

"It's like us humans not understanding how the world works --
it's just there," he explained. "It's the limitation of power,
perception and understanding, and that's the situation we put them
in. They don't know about the world they're in. Only the audience
can integrate that and understand that."

The emotional aspect of the dance is possible because it is less
about precision, and more about natural, effortless movements and
an element of surprise.

"Precision is a tricky thing, that's what I've learned about
interaction," said Weiss. "I've seen installations with very
expensive equipment, where every joint is measured. What do you do
with data that precise? The simplest thing is to have virtual
avatar doing the same thing. That's not that exciting. I'm looking
for something that's a little unpredictable and not totally
planned.

It's like human communication -- if it's one to one it's too
predictive. It's not interesting. The same happens with interactive
technology. If it's too predictable, a one to one reproduction of
what you already see, then it doesn't add too much. You want a
certain surprise in it, and that surprise is kind of the story. I'm
also looking for a little friction, some inaccuracy that creates
the magic in these algorithms."

Mortal Engine is on at the Southbank Centre, Queen Elizabeth
Hall, 19 and 20 October.