When he performs a life-saving operation, David Yuh
puts his head inside a console, views a screen with a
three-dimensional image of the heart, and moves a
computerized "wrist" that sends commands to a robotic arm
inside the patient. A scalpel at the end of that arm, under
Yuh's command, will repair damage to the mitral valve and
other areas of the heart.

Yuh, an associate professor of surgery and director of
Johns Hopkins Medicine's minimally invasive
cardiac surgery unit, can quickly rattle off the
advantages of using the daVinci machine, a robot designed
to assist surgeons who perform delicate procedures on the
heart and prostate. Besides allowing him to move in small
spaces, the robot eliminates the need to make a large
incision and crack and cut through the sternum to work on
the heart, Yuh says. Instead of doing such drastic damage
to the body, the daVinci enables Yuh to thread a robotic
arm through a small incision — about 6 to 8
centimeters. As a result, patients bleed less, recover much
more quickly, and spend fewer days in a costly hospital bed
than if they had been fully opened up. The daVinci also
gives Yuh and other surgeons the ability to make minute
movements, such as tying sutures or removing bits of
damaged or abnormal tissue, that would be more chancy if
done by hand alone.

But there's one problem: Though the daVinci is outfitted
with a camera arm so that surgeons can see what they are
doing, they can't feel what's happening inside a
patient's body. "It's like eating without your sense of
smell," Yuh says. "You have to adapt to that."

Lacking the critical sense of touch when using the robot,
surgeons tend to move more slowly, to worry more about
breaking sutures or injuring healthy tissue — both of
which can lengthen an operation and put heart patients at
greater risk of bleeding, heart failure, and stroke. "The
more time a surgeon spends being cautious, the more
dangerous it is for the patient," Yuh says. "If a surgeon
breaks a suture or inadvertently damages tissue, he has to
take time to repair it. That can mean worse outcomes for
patients."

According to a study concluded late last year and submitted
to The Journal of Thoracic and Cardiovascular
Surgery, doctors new to the daVinci would operate more
safely if the machine were able to deliver sensory feedback
— if it could replicate to some degree what surgeons
would feel if they had their hands inside a patient.
Doctors could move more confidently and comfortably, the
study's authors say, and surgeons new to the daVinci could
learn to use it more quickly.

This is where Allison M. Okamura comes in. An associate
professor of mechanical
engineering and director of Johns Hopkins' Haptics
Exploration Laboratory, Okamura has made it her mission to
figure out how to infuse robots with a human-like
sensitivity to touch — and to help robot-assisted
surgeons like Yuh practice safer medicine.

For those who have lost their ability to feel, even
simple tasks — dialing a telephone, using a tool, or
moving about — can be nightmarish.

Okamura started the lab from scratch seven years ago, and
it has since grown in both size and stature. The lab's
researchers consult with surgeons and scientists from a
variety of the university's medical and engineering
departments on a multitude of projects: finding better ways
to provide haptics information on video screens,
replicating a sense of feel through sight; looking at how
to refine the feedback that machines like the daVinci
transmit to surgeons to make the robots less costly to use
and to give surgeons precisely the feelings they need to
operate; and creating "haptic scissors" that would send
sensations to surgeons as they cut.

"They're really on the right track," Yuh says. "They
understand that for a surgeon, the holy grail of haptics is
to feel force as if he were using a regular, hand-held
instrument. It's still years away, but it's on the
horizon."

Haptics, from the Greek haphe, means related
to the sense of touch. Like our other senses, touch tends
to be taken for granted. But for those who have lost their
ability to feel, even simple tasks — like dialing a
telephone, using a tool, or moving about — can become
nightmarish.

When applied to machines, haptics usually involves feedback
— augmenting robotic arms or hand-held devices with
the ability to transmit sensation so humans can find more
and better uses for them. For a futuristic example, think
"the feelies" in Brave New World, Aldous Huxley's
1932 novel. In that sci-fi classic, characters who grabbed
metal knobs on their chairs not only saw and heard movies,
but felt them on their skin — right down to the fur
of the bearskin rug on which a pair of actors made love.

Haptic feedback's real-world beginnings have a bit of a
sci-fi twist as well: Humans who used robots to handle
radioactive waste and materials in the 1950s were aided by
haptics sensors that let them know if containers had been
grasped properly. Robots that explored space and the bottom
of oceans also delivered a rudimentary kind of haptic
feedback.

It wasn't until the 1980s, however, that haptics research
began to snowball. Since then, advances in computer
technology have speeded up the electronic systems needed to
accurately and quickly transmit "feelings" from sensors to
users.

"It's all motors and sensors now," says Okamura. "The
mechanical side hasn't really improved, but with the
development of the virtual world and the increase in
computational speed, there's much, much more we can do."

In our brave new world, haptic feedback straddles the
entertainment and medical realms. It's used to re-create
touch in everything from video game joysticks to the BMW
iDrive control knob, designed to help drivers adjust air
conditioning, music, and the navigation system by feel. The
control knob transmits haptic information to the hand so
that, for example, as the driver moves from one temperature
setting to another, she feels the knob kick back slightly
with each adjustment. She can feel what she has done
without taking her eyes off the road.

With about 2,000 haptic feedback researchers in North
America, the field is booming. At Hopkins, much of the work
being done aims to help surgeons perform difficult
procedures more efficiently and safely, or to invent an
exoskeleton that would help people with nerve disorders
feel again, or to create artificial limbs that replicate
the functions of missing arms or legs.

Robotics, along with stem-cell research, will drive much of
the innovation in medicine in the coming decades. Mohsen
Mahvash Mohammady, an assistant research professor at the
Engineering Research
Center for Computer-Integrated Surgical Systems and
Technology (ERC CISST) at Johns Hopkins, and a fixture
in the haptics lab, says that collaboration is the key to
the lab's success. "Without a doctor's input, I would be
able to develop a nicely controlled robot, but I wouldn't
be able to incorporate what surgeons need," says Mohammady,
who is working on developing haptic scissors, as well as
finding the best ways to retrofit the daVinci with the most
useful types of force feedback.

Surgeons have told Okamura that they only need a few of the
seven degrees of freedom — the seven ways the
daVinci's arm can move — to operate effectively. By
eliminating the types of movement that doctors don't need,
the machine will become simplified, she says, and expensive
parts — some of which cost thousands of dollars and
must be replaced regularly — will be eliminated.

In the back of the lab, where a daVinci surgical robot sits
awaiting modifications, Lawton Verner does the hands-on
work to make it more efficient. (Hopkins is the only
academic research institution to feature a nearly complete
system. There are about 400 daVinci machines worldwide.)
The 26-year-old PhD candidate in mechanical engineering
came to Hopkins from Nebraska, where he had worked with a
surgeon to research the possibilities of creating a machine
that could "read" what surgeons are doing by the movements
they make.

Carol Reiley maneuvers one robotic arm by moving
another. Behind her, Verner wears a head-mounted display, a
visual system used in some robot-assisted
surgeries.

These days, Verner looks to instill the daVinci with
partial-force feedback — a system that takes the
force surgeons feel in their hands from the robotic arm and
limits it to what they actually need to feel to complete a
procedure. "The more factors we figure out are important,
the more we can take out, so there are fewer sensors and
motor devices," says Verner. "It makes the machine less
costly, as well as more sensitive to what the surgeon
actually needs. As an engineer, part of my job is to not
just make it a strong surgical tool, but to make it
practical. Better tools lead to better outcomes. They
reduce patient suffering and recovery time."

The possibility of participating in such breakthroughs,
along with some encouragement from Okamura, is what landed
Verner at the haptics lab. Matching robots with haptics
will become a major focus of his engineering career, he
hopes.

"It's a really good way to affect medicine right now,"
Verner says.

During a lull in research on a day during winter
break, Okamura explains her work to a local radio station
team on hand to tape a feature story. But the lack of an
audible dimension makes it tricky: The robots Okamura and
her students work with might save lives, but they don't
bleep or blorp. They're not battlebots that pound and
screech, or pet-like robots (list price: $129.95) that
wheeze as they vacuum the linoleum.

A sound technician unsuccessfully tries to coax some
decibels out of the robotic arm of the daVinci. The arm
picks at a Petri-sized dish full of plastic replicas of
human tissue that look like polyps — the dish
resembles something you might find growing at the bottom of
a sophomore's refrigerator. As the technician fecklessly
pokes around with his audio boom, Okamura stands in front
of a board on which phrases like "redundancy algorithm" and
"gravity compensation" stand out in blue, and translates
into plain English some of the basics of haptic feedback.

"Robots are a physical way to use information," Okamura
says. "And the daVinci is a large robot. The larger a robot
is, the harder it is to get haptic feedback."

The public's fascination with robots has no doubt driven
much of the media attention given to the lab; the formula
looks something like this: "space age" + "cutting edge" =
news. Okamura, who explains things clearly and in layman's
terms, is getting used to being a media darling — a
strange place for an investigator/engineer to be.

Russ Taylor, Engr '70, helped hire Okamura, now 34, to work
with the center's other engineers on robots and information
technology that could be used in minimally invasive
surgery. Taylor is a professor of computer science at the
Whiting School of
Engineering and director of its ERC CISST, a consortium
of universities that do high-level medical robotics work.
The $30 million center, which is based at Hopkins, features
scientists who develop "machines working with humans to do
things neither could do alone," says Taylor, who is known
as "the godfather of medical robotics" to his peers.

Intuitive Surgical Inc., a Silicon Valley company that
developed the daVinci surgical system, chose Hopkins to
develop haptics for it — in part because Taylor had
patented some of the daVinci technology. The company also
chose Hopkins over several innovative West Coast
universities because of its medical reputation. "While it's
true that there are many excellent schools nearby, we are
attempting to provide a world-class solution to our
customers' needs," says William C. Nowlin, senior director
of research and software systems development at Intuitive
Surgical. "We certainly view their work as
high-caliber."

Because of Hopkins' pre-eminent role in medicine, it has
become a natural landing place for engineers like Okamura
and Taylor, and for research that marries engineering with
care for patients. "I came here because of the center
— the whole idea of surgeons and radiologists mixing
with engineers excited me," Okamura says. "The quality of
the people here and the interaction with the medical school
allows us to do things that are unparalleled elsewhere."
Then a 27-year-old wunderkind, she chose Hopkins over
several other universities after receiving her PhD from
Stanford.

The daughter of PhD chemists in Riverside, California,
Okamura pursued physics as a high school student. But soon
she began to look elsewhere for inspiration. "I liked the
idea of physics, but after high school, I thought that if I
got into physics I would be dealing with things that are
too large — like the cosmos — or too small,"
she says. "I like working with things that are my scale. I
like working with my hands."

During her undergraduate years at the University of
California-Berkeley, Okamura began to design and build
machines that performed tasks. By her first year of
graduate school, she was doing research regularly. "The
idea that you could get a robot to affect its environment
in an intelligent way was fascinating to me," she says. At
the time, engineers who wanted their machines to act on
their own dominated robotics research. Okamura's doctoral
thesis was built around creating a robot that would explore
an object unknown to it, then determine what it was by its
shape, weight, and texture. "It didn't do so well," she
says.

Okamura chose her side in the ongoing debate in robotics
— autonomous machines vs. ones designed to interact
with humans — after working for Immersion
Corporation, a San Josť company that develops haptic
devices. "The frustration of doing autonomous things made
me think I should go in a new direction," she says. "I was
intrigued by the sense of touch."

Okamura built Hopkins' haptics lab from scratch. Shortly
after arriving here, she wanted to work on medical robotics
but, she says, "I had no idea how to get started." Mentors
from the ERC taught her how best to write grant proposals
and integrate work on robotics with researchers from other
disciplines. She apprenticed for two years under a host of
Hopkins engineers. After a pair of cardiac surgeons —
Yuh, and Vincent Gott, a since-retired professor of surgery
— approached her in 2001 with their concerns about
breaking fine sutures while performing delicate surgery,
Okamura and crew were off and running.

The haptics lab gradually has built up the grants it
receives from the National Institutes of Health, the
National Science Foundation, and DARPA (a Pentagon-based
research agency) to about $500,000 a year — enough to
stay on the leading edge of medical haptics research.

"In order to succeed in a research field, you need a
critical mass. Allison being here helped make that happen
in haptics," says Taylor, citing the support the lab has
received for its work.

"The idea that you could get a robot to affect its
environment in an intelligent way was fascinating to me,"
says Okamura.

As a result, student researchers like Carol Reiley, a
24-year-old PhD candidate in computer science from
Vancouver, Washington, can apply her knowledge of
information technology and engineering to the field. Reiley
deals in "augmented reality," what might be called "virtual
haptics" — visual representations of the landscape
inside a patient that a surgeon has to work in. Her
approach to giving doctors haptic feedback is one sense
removed from the robot-centered one. Surgeons who use the
system Reiley is developing won't receive force feedback
through their hands. Instead, they'll get haptic
information delivered to their eyes during surgery via
flashing lights on a computer screen.

Like many other haptics lab students, Reiley works with the
daVinci system, trying to eliminate its drawbacks. The
sensors that relay the feedback from the daVinci's arm to
the surgeon have to be sterilized, a process that causes
them to lose their sensitivity after 10 or so uses. They
must then be replaced — at a cost of as much as
$6,000 each.

In Reiley's augmented reality system, surgeons can see how
much pressure their surgical device is exerting on tissue
— they can tell by watching the color of a
quarter-sized dot on the monitor. Working with thresholds
set by Yuh, Reiley applied the traffic-light standard to
her system: Green means the device is applying little or no
pressure; yellow, moderate pressure; and red, a surgeon is
in danger of constricting a vessel.

"Visual representation is still a form of haptics, but it's
less direct," explains Reiley. "We thought about using
auditory cues instead, but with all the noises you have in
a typical operating room, it wasn't as attractive." Her
system uses gauges that cost pennies instead of the pricey
sensors. If she could come up with a way to sterilize the
gauges, daVinci-aided operations would be much less
expensive. Ideally, Reiley adds, she'd like her system to
work with more machines than just the daVinci. Surgeons who
have tried her system have broken fewer sutures. "It's
clear that they're gauging the force correctly from the
visual cues," Reiley says.

Okamura and crew are darlings not only of the media,
but of the Hopkins development staff as well. Tours for
alumni and prospective students are regular features at the
haptics lab, where, despite its nondescript look and sound,
the wow factor from playing around with touch-sensitive
robots makes it a natural spot for the curious.

"The fact that the lab is hands-on and deals in robotics
in general makes it attractive," says Rob Spiller,
associate dean of development and alumni relations at the
Whiting School. "The lab is an intersection of different
disciplines working together to benefit society. We're
proud of it."

Spiller says that his department gives at least 10 alumni
tours of the haptics lab each year. What's more, Okamura
has taken part in several national speaking tours, along
with other Hopkins robotic professors. She leaves her
audience happy, Spiller says. "They're impressed with the
work the lab does, that it deals with life-and-death
issues. But they're also impressed with Allison and her
wonderful ability to make things understood without
eliminating all the sophistication that comes with doing
cutting-edge work," he adds.

That work will continue to grow, Okamura says, as we better
understand how humans process haptic information. She has
been exploring those questions as she collaborates with
other Hopkins scientists, including Amy Bastian, associate
professor of neurology, and Steven Hsiao, associate
professor of neuroscience at the Zanvyl Krieger Mind/Brain
Institute. Among the areas they are investigating: how
humans experience texture, vibration, and shape; how we
perceive softness; and if it is possible to develop human
models based on haptics. "It's important for me to know how
humans use haptics to perform tasks, so I can apply that in
my work," says Okamura, who adds that this research is
still in its early stages.

She'd also like to figure out a way that robots could
create a model of a patient's tissue, so haptics could then
be adapted to the density of it. And there's the ultimate
hope of drawing up a haptic model of a person, so surgeons
would know the nature of the patient's tissue, blood
oxygenation level, and several other factors. "It would be
great to have a haptic human," Okamura enthuses.

Okamura is also working with Nitish Thakor, a professor of
biomedical
engineering at Hopkins, to develop an artificial arm
that would use sensors to replicate a sense of touch.
Another approach involves creating vibrations in a foot to
simulate the sensations made when a prosthetic finger runs
over a texture. Thakor and Okamura are also looking at ways
to infuse prosthetic devices with proprioception —
the wearer's ability to know what position limbs and joints
are in without looking at them.

Verner, Okamura, Reiley, and Mohammady outside
the engineering lab.

"Our lab has done research and deemed that proprioception
is necessary" for people to use prosthetics successfully,"
says Okamura. "If you can judge where the arm and hand are
using an intuitive method, then it will help them. Also, it
will make them want to use the prosthesis."

Sometimes, lab investigators will veer slightly away from
straight-on haptics research if investigators see a medical
need they can address. One of the more intriguing cases
involves creating a "steerable needle" that a surgeon could
thread through a patient without damaging organ tissue or
nicking blood vessels. By making the needle asymmetrical
and steering it by its beveled tip, a surgeon could
conceivably make it twist more accurately through the body
— potentially a considerable aid in performing
biopsies or injecting substances into tumors to slow or
stop their growth.

One of the admirers of Okamura and her crew's work on the
steerable needle is Taylor. "It's an extremely promising
technique, but they need to develop a theory as to how you
model tissue density so a surgeon would know how to control
the needle," he says. "Overall, it's a wonderful example of
how to do important research into how you combine modeling
with medical applications. There are only a few places in
the world that do this kind of work."

As she churns out her latest round of grant proposals to
investigate some of these new areas, Okamura looks forward
to moving the lab to a larger, state-of-the-art facility.
This fall, the $36 million Computational Science and
Engineering Building will open on the new Decker
Quadrangle. Okamura is excited about the move. "It'll give
a huge shot of adrenaline to our whole robotics group,"
Okamura says of the lab space-to-be.

"It's just too bad they rejected my idea for a fire pole
from my office down to the lab."

Baltimore-based freelancer Mike Anft wrote about
political science professor Matt Crenson in the November
issue of Johns Hopkins Magazine.