Art Meets Science

Researchers Push

the Boundaries of their Fields

as They Merge

Creativity

& Technology

By Margarita Venegas

Nearly 10 years before he joined the faculty
at the University of North Texas College of Music, Panayiotis Kokoras won
a competition that earned him a residency
at one of the world’s most distinguished
electroacoustic music studios.
The options included programs in
France, Switzerland, the U.K. — or
Denton, Texas.

Above, Panayiotas Kokoras, assistant professor
of composition, demonstrates his work Sense at
Dallas’ Perot Museum of Nature and Science. At
top, a detail from the work of graduate printmaking
student Jon Vogt serves as a visual response to the
complexities of metabolism.

Photo by: Alyssa Hedenstrom

One of the first college electronic
music centers in the United States,
UNT’s now-expanded Center for Experimental Music and Intermedia is
celebrating its 50th anniversary in 2013-14. It continues to uphold a distinguished
reputation in computer music and
interactive/intermedia performance.

“CEMI has a great history and has
been at the forefront of the international
contemporary and experimental music
scene for 50 years now,” says Kokoras, a
UNT assistant professor of composition
since 2012. “It has excellent facilities,
amazing colleagues and talented students.”

UNT has long been known as an
incubator for creativity, and the work
at CEMI is just one of the ways faculty
and students are exploring new media
and pushing the boundaries of their
fields as they merge art with science
and technology.

Experimental Music

“Composers have always tried to
find new modes of expression,” says
CEMI director Andrew May of the
appeal of using computers and other
media to express sounds and music.

CEMI — which includes faculty in
composition and iARTA, UNT’s Initiative for Advanced Research in Technology and the Arts — was opened in 1963 as the
Electronic Music Center by faculty composer Merrill Ellis. Musicians progressed
from using magnetic tape for compositions
and analog synthesizers for live performances
to incorporating dancers, actors and
visual projections. Today, CEMI students
and faculty produce interactive computer
music, immersive video and sound spatialization,
physical computing and more.

In the 1980s, the center received
worldwide attention when it hosted the
seventh annual International Computer
Music Conference. It will again host the
ICMC in 2015 and this year will host the
International Confederation for Electroacoustic
Music Festival and Conference.

For Kokoras, the appeal of the world
of electroacoustic music is the ability to
capture and manipulate sounds in their
finest detail. That’s the purpose behind
one of his latest works, Sense, for which he
developed a device that allows humans to
“hear” ultrasound and infrasound frequencies
through sound vibrations felt in the
skin, bones and nerves.

The device includes a plate that funnels
ultrasonic sound upward, so that when a
listener places a hand above it, the sound
— the kind heard and used by dolphins in
communication — can be felt on the palm.

Other pieces are attached beneath the
chair of the seated listener to relay infrasound
— the kind created by earthquakes
and volcanoes — in a rumble that can be
felt throughout the body. Headphones
relay the music in the composition that is
within the range of human hearing.

“This is a more holistic way of listening,”
says Kokoras, who will release the
plans for the device so that other composers
may use it.

In other CEMI research, Jon Christopher Nelson, associate dean of operations
and professor of composition, is
working on a computer program that will
make a virtual instrument that can be
modified so it can start off producing a
string sound and end up producing a percussion
sound.

May, who says he was interested at an
early age in programming as well as music
performance on violin, is working to find a
way to make the computer adapt better
in live performances. He creates software
that lets the computer “listen” to live performances
and make musical decisions
emulating those of real musicians.

“I feel fortunate to be part of a program
that has this level of experimentation,”
May says.

Virtual Particle Chamber

David Stout, professor of composition
and coordinator of iARTA, at the computer
second from right, performs at
CEMI’s 50th anniversary concert.

Photo by: Gary Payne

That experimental environment
allows composition professor David Stout to incorporate science into his work as he
researches the extension of sound through
new modes of technology.
“Science and art share many of the
same tools,” notes Stout, coordinator of
iARTA, which brings together faculty
researchers from music, dance, film, art
and engineering.

Stout’s current projects include the
book The Reemergence of Alchemy at the
Intersection of Art, Science and Digital
Media and continued work as a founding
member of the collaborative media performance
and installation group NoiseFold.

His most direct collaboration of art
and science at UNT is Shadow Box
Attractions, a digital media project that
uses software he conceived and created
with the help of three programmers to
form a virtual particle chamber.

Stout can place positively or negatively
charged nodes at various locations within
the chamber, which features reconfigurable
“force fields.” The results are lines or wisps
— or sometimes flashes or dots — of black
particles that float, fall, rise and tumble
around in three white boxes projected
onto separate screens.

The sounds associated with the piece
range from insect-like buzzes to something
resembling shattering glass or static.
“The project shares an ‘imaginative
corollary’ to historic research in contemporary
particle physics,” Stout says, “while
being an artwork that seeks to dramatize
the dynamic interplay of elemental forces.”

Data Visualization

UNT faculty also explore art and
technology in fields outside of music.
Another core iARTA faculty member,
Ruth West, is a new associate professor at
UNT whose expertise spans the areas of
information science, new media, computer
science and biological science.

West, who pioneered cross-resonance
research over the past 10 years and is outfitting
her xREZ ArtScience Laboratory
on campus, works with emerging technologies
to find new ways to visualize and
uncover interconnections within big data.

“In art and in science, human beings
look for or make patterns, then they disturb
those patterns,” West says. “It’s the generation
of the pattern and the disruption of
the pattern that gives us information about
the thing underlying it.”

Her previous work includes Atlas in
silico, an installation developed at the
University of California at San Diego by
an interdisciplinary 19-member team.

It features an interactive virtual environment
driven by data from the Global
Ocean Sampling expedition, a survey of
the diversity of microbes in the oceans.
Each particle in the installation’s virtual
3-D world represents a data record for a
protein sequence that was sampled. The
particles’ movements and sounds reveal
larger patterns within the data.

“The GOS data collection is basically
a snapshot of the micro-organismal biodiversity
of the world’s oceans. It changed
our view of life on Earth,” West says.

“So for Atlas in silico, we developed
this novel visualization of the data, using
algorithms to create visual and auditory
data signatures as a way to provide the
information so that it can be experienced
by a broad public and researchers alike.”

With UNT colleagues, West collaborated
in fall 2013 on an interactive
installation and community engagement
project at the Perot Museum of Nature
and Science
in Dallas that employed the
rePhoto application she helped develop.

The application allows users to align
camera images taken from the same perspective
at different times to show progress
or change over time. It has been used,
for instance, in environmental monitoring
of tree health and growth.

For the Perot event, West joined
fibers faculty members Lesli Robertson and Amie Adelman, Make Art With
Purpose director and artist Janeil
Engelstad, and three students to invite
members of the public to collaborate in
creating geometric fiber sculptures and to
write their responses to the social interaction
as the researchers observed the process.
The installation was documented with
rePhoto as the piece grew and changed.

The idea was for community members
to see how they could work together and
possibly translate that to social and environmental
concerns. The community
engagement angle of the project, West says,
spurs interaction as well as observation.

“These are two important aspects
of scientific research and two of the
elements that drive the rePhoto app,” she
says. “Documentation such as this is vital
to research.”

Robots in Art

Paula Gaetano-Adi, assistant professor
of new media, traveled to a remote
village in the Peruvian Amazon for her
work TZ’IJK, using robots to portray the
Mayan creation myth.

Photo by: Gustavo Crembil

Cultural studies of technoscience are
a focus for artist Paula Gaetano-Adi, assistant professor of new media at UNT.
Funded by a VIDA Art and Artificial
Life Awards’ Artistic Production Incentives
grant, she found herself in a remote
village in the Peruvian Amazon last summer,
sifting through a 30-pound box of
screws for her project TZ’IJK — the
Mayan word for “mud.”

The project, which uses robots to
portray the Mayan creation myth, was
conceived by Gaetano-Adi and her mentor,
Gustavo Crembil, assistant professor
of architecture at Rensselaer Polytechnic
Institute. Constructing 50-inch balls with
a robotic mechanism in the middle, they
would surround them with a polycarbonate
membrane tied with bendable wood
and covered in mud using a pre-colonial
South American building technique.

But, when they got to the Peruvian
village where they were to learn the technique,
they realized that procuring screws
for the prototype robot was not as easy as
going to the local hardware store. The villagers,
who live without many technological advances, brought them the box of
random screws to sift through.

It was one experience that highlighted
that juxtaposition of what Gaetano-Adi
portrays in much of her artwork — seeking
to understand how Western methods,
knowledge and technologies are used and
adapted in the local ways of South
Americans.

In 2006, she won first prize in the
VIDA 9.0 international competition on
Art and Artificial Life for Alexitimia, a
robotic artwork that looks like a lump of
flesh. When the piece is touched, it begins
to “sweat,” emitting water through tiny
pores. While it appears to be a crude entity
with only one way of communication —
touch — it is at the same time a complex
robot made to interact with viewers.

She and Crembil named the prototype
they made for the TZ’IJK project
Mestizo. It recalls the Mayan myth that the
gods made several false starts in creating
man, including making him out of mud,
which led to a sightless, bumbling creature.

Once the artists have full funding for
the project, they plan to make several
mud-covered robotic spheres of varying
sizes that will shake, move and rock, once
again combining high-tech with low-tech.

TZ’IJK is already getting exposure.
Mestizo was one of 10 finalist projects in
the fifth Electronic Arts and Video
Transitio_MX Festival: Biomediations
last fall. Gaetano-Adi and Crembil presented
a paper, “Mestizo Robotics,” which
describes the work they did to create the
prototype, at the Re-New Digital Art
Festival in Copenhagen, Denmark.

“The idea of mestizo as a blending of
races applies to our approach to robotics in
art,” Gaetano-Adi says. “We’re using indigenous
processes to create autonomous
robots, blending craft and technology, traditional
and modern, small-science and
big-science.”

The Science of Art

On the opposite end of the art-technology
spectrum, UNT alumnus Marcus Young, who received Bachelor of Fine Arts
degrees in ceramics and sculpture, is using
technology to evaluate art.

When Young came back to teach at
his alma mater in 2012, it was not in the
College of Visual Arts and Design, but as
an assistant professor of materials science
and engineering. A strong interest in fundamentally
understanding the materials
used in art led Young to pursue a doctorate
in materials science and engineering.

While earning his Ph.D. at Northwestern
University, he worked in art conservation
at the Art Institute of Chicago.
When he returned to UNT, he collaborated
with Dallas Museum of Art chief
conservator Mark Leonard to undertake a
similar project.

The museum has more than 400
American silver and silver-plated pieces
from the late 19th century and 20th century.
The makers often kept their manufacturing
processes closely guarded secrets,
and many are no longer in business.

To help the museum and current
manufacturers know more about these
processes, Young and graduate student
Matthew Carl developed a novel technique
using a focused ion beam system,
which allows for the examination of a
microscopic cross-section of the object.

The technique — which is being
used for the first time on metal museum
artifacts — makes it possible to image
the plating thickness and the plating and
base metal texture of a cross-section
about 30 x 30 x 90 microns. A human hair is roughly 50 to 120 microns thick.

“It is not possible with the human eye
to see where we took the sample, but it’s
enough to get down to the underlying base
metal,” Young says.

The FIB system also is equipped with
a scanning electron microscope with electron
backscattered diffraction and energy
dispersive spectroscopy, which enables the
researchers to see what materials the artist
used to create the object and gives insight
into how the base metal was processed and
how the plating was applied, Young says.

So far, Young and Carl have completed
analysis of about 10 objects, which include
service items such as teapots and utensils.
The project will take a few years to finish,
says Young, whose art background gives
him a greater understanding of the need for
conservation science.

“In materials science, we can often
section objects to make analysis easier.
But, in conservation science, because of
the value and uniqueness of the objects,
analysis is limited to only a handful of
non-destructive techniques,” he says.
“This FIB technique offers one more
powerful tool.”

Student Exhibit

Art and science often can be used
together to help explore complex processes.
Such was the idea behind an exhibition
conceived by Guido Verbeck, associate
professor of chemistry, and Andrew DeCaen, associate professor of art. Verbeck’s
study of chemical processes in metabolic
disease and DeCaen’s artwork
exploring the science of food overlap in
addressing the issue of obesity.

From the scientific perspective, obesity
can lead to metabolic disease. Educating
those with these diseases, such as
diabetes, can be difficult since it’s not easy
for a layperson to understand the science
behind what triggers the diseases and
makes them progress.

To help with that, Verbeck and
DeCaen proposed a collaboration that
educated College of Visual Arts and Design
students through a series of lectures about
metabolic processes and disorders and
invited them to interpret the disease
through a juried art exhibition. Verbeck
opened up his lab to the art students, who
were encouraged to ask questions of chemistry
students in order to develop ideas for
their artworks.

“It was exciting to see so much
exchange of ideas,” DeCaen says.
Twenty-nine of 43 artworks submitted
were displayed in November 2013 at UNT
on the Square. The work varied from
prints to sculptures, photographs, drawings
and more, and the winning students
received UNT scholarships.

While science is objective, art can be
more subjective and allows for interpretation
of concepts in a variety of ways, says Jon
Vogt, a second-year graduate printmaking
student. His prints showing the complexity
of metabolism in the body and the vast
number of intricate information systems at
work won one of the Best of Show prizes.

Seeing an exhibition that mixed science
and art was a way to start a conversation
about a topic that might not otherwise be
addressed, says Aaron Flynn, a second-year
graduate ceramics student who replicated
the process of diabetes in his Best of Show
ceramics piece.

For Caitlin Odneal, the Best of Show
winner whose photos depict sugar crystals
under a microscope, the outreach to the
community was important, but she also
was grateful for the opportunity to show
in the exhibition.

“This is what I want to do in my
career — mix science and art, maybe
working for a hospital,” says the senior
photography student.

The sentiment of uniting the arts
with science and technology is one echoed
by students and faculty across campus,
says CEMI director May.

“Working with technology gives artists
a chance to collaborate in more ways
than ever before,” he says. “This is a changing
environment — and a very exciting
place to be.”

More Features

Biocultural Conservation
The sub-Antarctic provides a model environment for
studying the diversities of life.By Adrienne Nettles

Plant Discoveries
Improved crops and new bioproducts grow from the research of Richard Dixon.By Leslie Wimmer