Apes With Apps

Using tablets and customized keyboards, bonobos can become great communicators

Posted 29 Jun 2012 | 18:32 GMT

By Ken Schweller

Advertisement

Photos: Gregg Segal
Can we talk? Kanzi, a 31-year-old bonobo, can converse with humans by selecting “lexigram” symbols on his Motorola Xoom tablet.

Have you ever watched a toddler play with an iPhone?

Most likely, the child was completely captivated and surprisingly adept at manipulating the tiny icons. Two-year-old Teco is no different. Sitting with his Motorola Xoom tablet, he’s rapt, his dark eyes fixed on the images, fingers pecking away at the touch screen. He can’t speak, but with the aid of the tablet app I created for him, he’s building a vocabulary that will likely total several thousand words. What’s more, he’ll be able to string those words together into simple sentences and ask questions, tell jokes, and carry on conversations.

Such talents wouldn’t seem exceptional in a human child, but Teco is an ape—a bonobo, to be precise. To the uninitiated, bonobos look very much like chimpanzees, but they are in fact a separate species with distinct physical and behavioral traits. More collaborative and sociable than their chimp cousins, bonobos also seem to be more adept at learning human language. And they are endangered, found in the wild only in the Democratic Republic of the Congo. Recent estimates put the wild bonobo population at between 10 000 and 50 000. Fewer than 150 live in captivity. Along with the chimpanzee, they are our species’ closest relatives.

For more than three decades, researchers have been working with a small group of bonobos, including Teco, to explore their amazing cognitive and linguistic abilities. Teco’s father, Kanzi, is the group’s most famous member: Anderson Cooper has interviewed him, and he’s played piano with Paul McCartney and Peter Gabriel. Animal lovers worldwide have marveled at his ability to communicate by pointing to abstract symbols. He recognizes nearly 500 of these “lexigrams,” which he uses to make requests, answer questions, and compose short sentences. The spoken words he understands number in the thousands.

Even so, many people question these abilities. Indeed, for more than a century scientists have debated whether apes could ever truly comprehend human language. Many researchers argue that language is the exclusive domain of humans, and several influential studies in the 1980s concluded that supposedly “talking” apes were merely demonstrating their capacity for imitation, with lots of unintentional cuing by the animals’ handlers. Linguist Noam Chomsky has likewise argued that the human brain contains a species-specific “language acquisition device,” which allows humans, and only humans, to acquire language.

But the bonobo research I’ve been involved with, led by primatologist Sue Savage-Rumbaugh at the Bonobo Hope Great Ape Trust Sanctuary, in Des Moines, strongly suggests otherwise. Today, the wide availability of touch screens, tablet computers, digital recording, and wireless networking is giving researchers the world over powerful new ways to study and unambiguously document ape communication. The results of these studies are in turn helping to spark a renaissance of technology-aided research into primate development and cognition and shedding light on the origins of culture, language, tools, and intelligence.

It’s a typical workday, and Panbanisha, Kanzi’s younger sister, is sitting before a 42-inch touch-screen display. She’s doing a match-to-sample task: When she presses a green button in the middle of the screen, the computer’s text-to-speech synthesizer says “apple,” and then the lexigram for the word apple appears on screen, along with two other lexigrams, for dog and sorry. Panbanisha touches the one for apple. After five correct responses, she gets to pick a short video to watch. She selects one of her favorites: a clip from Tarzan the Ape Man.

Our research group is using tasks like this to measure the bonobos’ vocabularies. We estimate that Panbanisha, like her brother, understands several thousand words. These match-to-sample experiments are enabling us to determine the exact number and should also help dispel criticisms that the bonobos are simply displaying the “Clever Hans effect.” Clever Hans was a horse that became renowned at the turn of the last century for solving arithmetic problems, telling time, and reading and understanding German. Later it was revealed that his trainer was subconsciously nodding whenever the horse tapped out the correct answer. Hans was indeed clever—clever at reading subtle cues from his trainer, not at adding and subtracting.

To avoid the Clever Hans effect, the bonobos aren’t encouraged to use sign language, because it leaves too much open to interpretation. Instead they “talk” to us almost exclusively with lexigrams; the computer helps remove any ambiguity. One day, for instance, Savage-Rumbaugh was using the match-to-sample program to familiarize Panbanisha with new words. The computer’s synthesized voice spoke the word “carrot” and then its screen displayed the lexigrams for carrot, carry, and potato. Panbanisha was about to hit the lexigram for carrot, but Savage-Rumbaugh, who’d misheard the word as “carry,” told the ape she was mistaken. The ape, though, knew better and selected the carrot lexigram anyway.

For more free-form communication, the apes can use their lexigram program, which displays up to 600 symbols on screen [for examples, see the image “True Meaning”]. The bonobos can tap multiple keys to construct a sentence, and each sentence they write is time-stamped and recorded for further analysis.

In 1971 a primatologist named Duane Rumbaugh (Savage-Rumbaugh’s ex-husband) came up with the idea of teaching language to apes by displaying abstract geometrical symbols on a computer screen. The first set of 120 symbols was then designed by Ernst von Glasersfeld [PDF], who also coined the word lexigram. Each symbol represented a noun, verb, adjective, or name. The lexigram lexicon was later expanded to 384 symbols, which were displayed on a keyboard. Researchers also used (and sometimes still use) a folding poster-board keyboard when greater mobility was required.

The latest version of the keyboard is created with software on a touch screen. These keyboards are easier to update and much less expensive to make than their hardware predecessors. Written in Java, the program will run on any reasonably up-to-date desktop or laptop. The keyboard software can also be wirelessly shared among several computers so that more than one researcher can communicate with a single bonobo. For easier translation, the researcher’s keyboard displays the English word just below each lexigram. Or the researcher can type in a word or sentence in English, and the software does its best to translate it into a meaningful string of lexigrams. For example, there is no lexigram for pizza, so the program translates that word into the three-lexigram sequence for bread cheese tomato, a description the bonobos came up with themselves. [To view the complete set of lexigrams, see the interactive lexigram keyboard created by the Bonobo Hope Great Ape Trust Sanctuary. As you hover over each symbol, its English-language meaning will pop up.]

One of the newest improvements to the keyboard software allows new lexigrams to be defined on the fly. Unlike the original lexigrams, the newer symbols aren’t limited to abstract shapes; they often include the English word, too, to make it easier for human users to understand. Here’s an example: Let’s say one of the bonobos has a toothache. There is no lexigram for toothache, so the researcher calls up a standard English keyboard on screen and types in “toothache.” A second later, a new lexigram spelling the word in colorful letters appears on the bonobo’s keyboard, and when the animal taps the key, the computer’s flat, synthetic voice says “toothache.” The researcher can then explain the link between what the ape is feeling and the new lexigram. Being able to add new lexigrams at will lets us reinforce new words, when the word and its meaning are fresh in the ape’s mind. If the bonobo didn’t have a toothache, it would be nearly impossible to explain the concept.

Another way the keyboard can be used is in picture mode. By pressing a lexigram key, the user can call up an image of the object, action, or concept that the lexigram describes.

I’ve spent many an hour watching the bonobos use the touch-screen keyboards, and they are incredibly good at operating them. Indeed, over the years, they’ve adapted to whatever new computer technology we’ve introduced. Before the touch screen, they used off-the-shelf gaming joysticks to control the computer cursor. The apes mastered that device in no time and soon graduated to using the joystick for its intended purpose: Kanzi is an expert at Pac-Man, while Panbanisha prefers “Sesame Street” video games.

Of course, studies of ape language didn’t start with the advent of computers. Beginning in the early 1900s, several attempts were made to teach chimpanzees to speak human languages. Such efforts proved largely fruitless, however, because chimps cannot produce human vocal sounds.

Experimenters then turned to sign language. In the 1970s, for instance, the psychologists Beatrix and Allen Gardner reported in Science that they had raised a female chimp named Washoe [PDF] to use and understand 85 signs. What’s more, she could combine signs in novel and meaningful ways. For example, she referred to the refrigerator as “open food drink,” even though the scientists themselves never used that phrase.

David Premack, a psychologist, and Ann Premack, a science writer, adopted another approach: They used colored plastic tokens to represent different words and concepts. In a series of experiments, a chimpanzee named Sarah learned to use the tokens to answer sometimes quite abstract questions about objects and their relations. For example, she was taught that a blue triangle token represented an apple; when she was later shown the token and asked what shape and color the object was, she replied that it was round and red, not triangular and blue.

Almost as soon as affordable computers came on the market in the 1960s, primatologists eagerly applied them to their work. As mentioned earlier, Duane Rumbaugh designed a computer-controlled lexigram keyboard for what came to be known as the Language Analogue, or LANA, Project. Lana is also the name of the first chimpanzee to use Rumbaugh’s keyboard. It consisted of three aluminum panels, each about 30 by 60 centimeters, mounted side by side on the wall of Lana’s cage. Each lexigram was embossed on a small square Lucite key, which was inserted into slots on the panel. The panel itself was connected by patch cords to a DEC PDP-8/E, an early minicomputer; its magnetic core memory could store a whopping 12 000 words. When a key was pressed, it glowed. The computer monitored the sequence of lexigrams Lana pressed and projected them above the keyboard.

Lana learned to use dozens of lexigrams. She could put these symbols in a grammatical sequence to generate sentences, sometimes quite complex ones. To request a treat from her trainer Tim, for instance, she might press the lexigrams for Lana want Tim give M&M.

Image: Bonobo Hope Great Ape Trust Sanctuary

True Meaning: The 600 or so lexigrams that the bonobos use to communicate represent nouns, verbs, adjectives, and other parts of speech. The newer icons incorporate the spelled-out word, to make it easier for human users to understand. The latest version of the lexigram software also allows new symbols to be created on the fly. To view the complete set of lexigrams, see the
interactive lexigram keyboard created by the Bonobo Hope Great Ape Trust Sanctuary. As you hover over each symbol, its English-language meaning will pop up.
Click on the image for a larger view.

Work with Kanzi and his fellow bonobos has taken our appreciation of ape language to a new level. Interestingly, Kanzi was never taught to use human language: He acquired it as children do, by being exposed to it. The process began when he was only 6 months old, while researchers were trying to teach lexigrams to his mother, Matata, a bonobo who had been raised in the wild. Baby Kanzi always accompanied Matata during her training sessions and so was in the perfect position to eavesdrop.

For two years, nobody suspected that Kanzi was paying even the slightest attention to the lexigram training, although he clearly liked the lights on the keyboard and the blinking projections above. It was only when Matata was taken away for a few weeks for breeding that researchers discovered how much Kanzi had picked up. After searching in vain for his mother, he spontaneously began using her keyboard to communicate with his caretakers. What is more, he understood the spoken words that the lexigrams represented, and he could locate their representations on the keyboard.

That event marked a paradigm shift in ape language studies. Previously, researchers had worked from a behaviorist psychology tradition, which held that mental events are products of reinforced training. So a scientist would show a chimp an apple, say “apple,” and then make the sign for the word apple. If the chimp signed back with apple, he’d be rewarded with an apple. Kanzi showed us that bonobos don’t really learn language that way; neither, of course, do people.

It now appears that 2-year-old Teco will equal or maybe even surpass his father and aunt in linguistic ability. Since birth, he has been totally immersed in human language—and to a much greater extent than Kanzi and Panbanisha have been. I built Teco his first toddler’s keyboard app on the Motorola Xoom tablet with just a handful of oversize lexigrams: grape, dog, go, cereal, and milk. He’s since graduated to a 25-lexigram keyboard.

At the age of 4 months, Teco recognized his first lexigram. Savage-Rumbaugh recalls that on the day it happened, the baby bonobo had been eating grapes. When the researcher told him she would give him a grape, he reached out his hand and touched the grape lexigram. She waited briefly and then asked him if he was ready for the grape. This time, he touched the grape symbol, but with his mouth instead of his hand.

What these and other incidents add up to is a rich picture of bonobo behavior. In the wild, the bonobos live in the rain forest of the Congo Basin, where, until the fairly recent arrival of logging crews, they had no natural predators and enjoyed an abundance of food. As a result, bonobos have gentle dispositions and are extremely sociable and collaborative. They are typically eager to interact with humans—and thus quite receptive to adopting human language and culture. What we now believe is that language, rather than being a uniquely human trait, is something other species can develop to varying degrees under the right circumstances—not to our level of sophistication but certainly to the point where we can communicate intelligently with them.

Photo: Jay Benedict/Buena Vista University

I, Robo-Bonobo: This squirt-gun-packing mobile robot, built by the author and his students, is designed to let the bonobos have fun with their visitors.
Click on image to enlarge.

If you’re worried that the bonobos I work with are spending too much time staring at computer screens, rest assured: They have plenty of opportunity for more natural interactions and for just goofing around. One of the first projects I worked on was a 60-centimeter-tall mobile robot, dubbed Robo-Bonobo, which the apes can control using a wireless joystick. The bot, about the size of a small garbage can, is mounted on wheels and has an onboard camera and an animatronic chimp head that can be raised and lowered. The idea is to provide the bonobos with a safe way to interact with visitors and have some fun—the robot comes equipped with a squirt gun. Of course, there’s also a scientific purpose: to study the apes’ ability to solve problems that require them to take another observer’s perspective.

My students have gotten very creative in developing new high-tech tools for this research. They’ve developed dozens of computer games, including puzzles and mazes, as well as a program called Keyboard Trainer, which helps people learn the hundreds of bonobo lexigrams. Students love working with the bonobos. On one visit, we watched Panbanisha tapping on a touch screen. Seeing us, she grabbed her poster-board keyboard and used the lexigrams to invite her visitors to have some juice. We all stopped working and sat down outside her glass enclosure to enjoy a drink together.

These days, much of my effort is directed toward finding better ways to document experiments and collect data. For example, it’s helpful to know who exactly is using a particular touch-screen keyboard, whether human or bonobo. My colleagues and I are considering using thumbprint or retinal scanners, RFID bracelets, or face-recognition software to identify who’s at each computer. We’re also exploring ways to allow the apes to control their own environment—using their keyboards to open doors and windows, access vending machines, control cameras, and so on. And we’re writing lots of apps for use with the bonobos’ wireless and GPS-enabled tablets, which will allow us to collect data in the field. One app we’re working on will let the apes alert security guards to any suspicious activity outside their enclosure, like the presence of wild dogs or other intruders.

Where might all this research lead? Beyond showing to what extent apes can use language, we anticipate that our studies will also shed light on people’s development and use of language, the impact of culture on cognition, and the cognitive structures that must be in place for language to blossom. Our work also has implications for how to handle sentient animals such as apes in captivity. If apes can communicate with their keepers, for instance, they can ask for changes to their environment that would make it better suited to their needs. Such possibilities suggest that apes’ cognitive abilities are a lot closer to humans’ than anyone imagined just a short time ago. But maybe that shouldn’t be so surprising, given the genetic similarities between the various great apes, a formal grouping that includes humans.

While the bonobo species still survives, we believe it’s our obligation to learn as much as we can about these extraordinary animals. They are fascinating in their own right, and they are also a window into our not-too-distant evolutionary past. By studying them, we learn ultimately about ourselves.

To Probe Further

For more about the author, Ken Schweller, see the Back Story, “The Magic Word.”