Adaptation, Part III: Art as Research (Braille tattoos! Socially-adept handbags!)

New media art self-consciously reworks technology into culture, and rereads technology as culture. What’s more, it does so in a concrete, applied way; it manipulates the technology itself, with a nonindustrial latitude that admits misapplication and adaptation, rewiring and hacking, pseudofunctionality and accident. New media art also fractures that technocultural material into millions of heterogeneous interests and agendas, specific investigations, aesthetics, approaches, and projects.

In Part II of this series, I ended by quoting Whitelaw about this “nonindustrial latitude” that artists bring to our own work, and ideally to the larger cultural conversation. Here I want to explore that notion: the value of playful experimentation and conjecture in making new knowledge. I’ll detail, with drawings, my own collaborative research in the last year—research that makes its own process visible, including the impractical and imaginative research that is especially valuable in art and design. I’ll also point to some others’ work where imaginative conjecture yields good ideas about adaptive design.

A little over a year ago, a colleague and longtime friend suggested I get in touch with philosopher Brian Glenney, head researcher for the Philosophical Psychology Lab at Gordon College. Glenney has been working for some time with the Seeing with Sound/vOICe technology, a device that transforms visual data into sound waves. With his lab, he’s been training human subjects to voluntarily blind themselves with opaque goggles, and rely on a small mounted camera on those goggles for their “vision.” The camera feed transforms its intake of light and dark (or in another version, the basic color spectrum) into specific aural tones. If these subjects can match and memorize the sounds with the visual cues, relying only on what they hear, they can (more or less) navigate their environments without the benefit of sight.

The most promising application of the device would be another way for people who are blind to gain more autonomy in moving through the world. Training oneself to “hear” colors would mean more information, more independence.

Glenney is interested in this practical application, but he has also long been interested in the cognitive and philosophical experience of substituting one sense for another. Whether we’re sighted or non-sighted, can we activate the visual cortex using sound, if we can trick our brains into adapting this skill? And if so, what does this tell us about our assumptions that our senses work in very discrete ways? Is our sensory processing more fluid than we might think?

At our first conversation, he told me about these experiments. In turn, I showed him a bunch of new art work in adaptive technology, and told him of my interest in collaborating with people outside of my field, in creating both practical and aesthetic projects around disability and adaptive tech.

So we started talking. I joined the lab in experimenting with the device:

You can read more about these experiments here. And we kept talking, and I started drawing:

Sensory interdependence, in the form of wheels and pulleys, with debts to Rube Goldberg machines. “The mind is a mousetrap!” said Glenney.

And I drew and thought more about Goldberg, here thinking about animality, reason, and cognition. How do we know what we know?

I re-imagined the goggles:

And the entire apparatus:

And, naturally, we got to speculating about a cyborg-ian future:

This, I would argue, is research made visible. It involves, as Whitelaw says, “misapplication and adaptation…pseudofunctionality and accident,” and it is often best when collaborative. It’s playful and practical.

Part of the work, of course, has been gathering data in a manner that’s familiar in a lab setting: testing how subjects learn to “see” the sounds, how much training they need to be effective, how long it takes to become “fluent.” And part of the work is the conversation about the conceptual reach of the ideas—the plasticity of the brain, the fluidity of the senses.

And part of the work is the iteration and (deliberate or accidental) inversion of the ideas, whether by drawing, or building, or through a narrated form, like this short film we made:

With the film especially, we’re speculating about the Mobius-loop kind of feedback between a device and its wearer: How does the machine shape perceptions about the human using it? Might the design have important visual registers that should be considered for future re-models of the equipment? Who does this person become in a public setting (i.e., weak or strong, dependent or super-human)?

It’s that last set of questions that has been the subject of our collaborative conjecture.

But why does this matter, as art?

Artist and writer Svetlana Boym has written of the “estrangement” that an artist can bring to otherwise self-evident phenomena. By all this work/play of conjecture, the inversions, the hacking, we see the world re-envisioned. And estrangement is more than just exercised curiosity:

The device of estrangement places emphasis on the process rather than the product of art, on retardation and deferral of dénoument, on cognitive ambivalence and play. By making things strange, the artist does not simply displace them from an everyday context into an artistic framework; he also helps to “return sensation” to life itself, to reinvent the world, to experience it anew. (From Boym’s Architecture of the Off-Modern)

Nor is this simply a reworking of the avant-garde strategy, where artists shock and de-center their viewers in order to deliver their manifest genius. Especially when collaborative with fields outside the visual arts, estrangement is indeed the work itself, experienced by the creators and viewers, not a privileged knowledge.*

Estrangement includes the scientific parsing, categorizing—and the meta-questions, the wonder at the whole. By approaching this subject with an eye toward both the details and the larger registers for disability communities and politics, we are researching not only with a direct gaze at the problem. Our research includes the sideways-glances in the form of conjecture, of play, so that, in my own case here, we can work alongside the science of prosthetics without being solely efficiency-and-utility minded. We can raise questions about disability in the culture, questions of politics, of cognition and perception, of community.

I’ll end with a few other pieces of work that are at play in just this manner.

There’s Klara Jirkova’s haptic tattoo, made of Braille lettering. (I’ve got an interview with Klara in the works; coming soon!)

Here’s the Touchstone, two halves of a rock-like device. When one half is held or touched, it glows with light, and the other half—given to another person, perhaps in another room, or in another country—also glows. “Thinking about you,” communicated in simple, affective technology.

2 thoughts on “Adaptation, Part III: Art as Research (Braille tattoos! Socially-adept handbags!)”

I think I’m really catching on Sara. These articles are helping me form up an adaptive technology “hypothesis”:

“Through technology we become estranged to ourselves. Then through continued use of that technology, we find ourselves anew.”

What I’m now realizing (thanks Sara!) is that the both the luddite criticism of technology and the transhumanist critique of “human nature”, both of which I am paradoxically attracted to, is softened by this adaptive technology hypothesis. I do not like technology because it distances me from myself, but I’m enamored by the idea of becoming renewed. With technology viewed in an adaptive context I can have it both ways.

Thanks for these fruitful thoughts and the visual media. You’ve help me connect a lot of dots.

Abler is run by Sara Hendren. I'm tracking and commenting on art, adaptive technologies and prosthetics, the future of human bodies in the built environment, and related ideas.

I'm an artist and design researcher. I also teach design for engineers-in-training, and I've just started the Adaptation + Ability Group, a research lab at Olin College. We'll be documenting our work here starting in 2015.
You can see some of my own projects—making, writing, speaking—here.