SMARTER THAN YOU THINK; Computers That Listen to You Make Strides in Talking Back

By STEVE LOHR and JOHN MARKOFF

Published: June 25, 2010

CORRECTION APPENDED

''Hi, thanks for coming,'' the medical assistant says, greeting a mother with her 5-year-old son. ''Are you here for your child or yourself?''

The boy, the mother replies. He has diarrhea.

''Oh no, sorry to hear that,'' she says, looking down at the boy.

The assistant asks the mother about other symptoms, including fever (''slight'') and abdominal pain (''He hasn't been complaining'').

She turns again to the boy. ''Has your tummy been hurting?'' Yes, he replies.

After a few more questions, the assistant declares herself ''not that concerned at this point.'' She schedules an appointment with a doctor in a couple of days. The mother leads her son from the room, holding his hand. But he keeps looking back at the assistant, fascinated, as if reluctant to leave.

Maybe that is because the assistant is the disembodied likeness of a woman's face on a computer screen -- a no-frills avatar. Her words of sympathy are jerky, flat and mechanical. But she has the right stuff -- the ability to understand speech, recognize pediatric conditions and reason according to simple rules -- to make an initial diagnosis of a childhood ailment and its seriousness. And to win the trust of a little boy.

''Our young children and grandchildren will think it is completely natural to talk to machines that look at them and understand them,'' said Eric Horvitz, a computer scientist at Microsoft's research laboratory who led the medical avatar project, one of several intended to show how people and computers may communicate before long.

For decades, computer scientists have been pursuing artificial intelligence -- the use of computers to simulate human thinking. But in recent years, rapid progress has been made in machines that can listen, speak, see, reason and learn, in their way. The prospect, according to scientists and economists, is not only that artificial intelligence will transform the way humans and machines communicate and collaborate, but will also eliminate millions of jobs, create many others and change the nature of work and daily routines.

The artificial intelligence technology that has moved furthest into the mainstream is computer understanding of what humans are saying. People increasingly talk to their cellphones to find things, instead of typing. Both Google's and Microsoft's search services now respond to voice commands. More drivers are asking their cars to do things like find directions or play music.

The number of American doctors using speech software to record and transcribe accounts of patient visits and treatments has more than tripled in the past three years to 150,000. The progress is striking. A few years ago, supraspinatus (a rotator cuff muscle) got translated as ''fish banana.'' Today, the software transcribes all kinds of medical terminology letter perfect, doctors say. It has more trouble with other words and grammar, requiring wording changes in about one of every four sentences, doctors say.

''It's unbelievably better than it was five years ago,'' said Dr. Michael A. Lee, a pediatrician in Norwood, Mass., who now routinely uses transcription software. ''But it struggles with 'she' and 'he,' for some reason. When I say 'she,' it writes 'he.' The technology is sexist. It likes to write 'he.' ''

Meanwhile, translation software being tested by the Defense Advanced Research Projects Agency is fast enough to keep up with some simple conversations. With some troops in Iraq, English is translated to Arabic and Arabic to English. But there is still a long way to go. When a soldier asked a civilian, ''What are you transporting in your truck?'' the Arabic reply was that the truck was ''carrying tomatoes.'' But the English translation became ''pregnant tomatoes.'' The speech software understood ''carrying,'' but not the context.

Yet if far from perfect, speech recognition software is good enough to be useful in more ways all the time. Take call centers. Today, voice software enables many calls to be automated entirely. And more advanced systems can understand even a perplexed, rambling customer with a misbehaving product well enough to route the caller to someone trained in that product, saving time and frustration for the customer. They can detect anger in a caller's voice and respond accordingly -- usually by routing the call to a manager.

So the outlook is uncertain for many of the estimated four million workers in American call centers or the nation's 100,000 medical transcriptionists, whose jobs were already threatened by outsourcing abroad. ''Basic work that can be automated is in the bull's-eye of both technology and globalization, and the rise of artificial intelligence just magnifies that reality,'' said Erik Brynjolfsson, an economist at the Sloan School of Management at the Massachusetts Institute of Technology.

Still, Mr. Brynjolfsson says artificial intelligence will also spur innovation and create opportunities, both for individuals and entrepreneurial companies, just as the Internet has led to new businesses like Google and new forms of communication like blogs and social networking. Smart machines, experts predict, will someday tutor students, assist surgeons and safely drive cars.

The Digital Assistant

''Hi, are you looking for Eric?'' asks the receptionist outside the office of Eric Horvitz at Microsoft.

This article has been revised to reflect the following correction: An article on June 25 about progress in speech recognition technology misidentified the company that used the technology to identify increased use of the phrase ''cash for clunkers.'' The company was a customer of Verint, which supplies speech analysis software to call centers; it was not Aetna Insurance. (Verint would not identify the customer.)