It’s a Sunday afternoon and the Ben Franklin Racing Team has come to practice precision driving maneuvers in the empty parking lot at Boeing, out past the Philadelphia airport.

Their Toyota Prius is packed with engineers: mechanical-engineering student Tully Foote GEng’08 at the wheel, with electrical- and systems-engineering Ph.D. candidate Paul Vernaza in the backseat next to Alex Stewart, a Ph.D. candidate in electrical engineering. Heteen Choxi, a Lockheed Martin employee and recent Drexel computer-science grad, rides shotgun, wearing the bright yellow-and-black team jacket. As the car rolls to a gentle stop, Foote gets out and pops the hatchback to reveal a mess of wires snaking over the back seat and onto the roof. Half of the trunk’s floor is covered by Plexiglas, and more wires and boxes are visible in the wheel well. Foote pulls up a command prompt on an LCD screen installed in the trunk, and soon a satellite image of the parking lot pops into view. The three passengers remain belted in the car, each hunched over a laptop. Driving practice begins.

The team is building a robot car capable of driving itself through urban terrain. In November, they will compete in the DARPA (Defense Advanced Research Projects Agency) Urban Challenge. Their car, “Little Ben,” will drive itself through an as-yet-unspecified urban location in the western United States. No remote controls, no pre-programmed paths through the cityjust 89 autonomous vehicles trying to drive down streets, around corners, through intersections, and around each other. A $2 million prize goes to the fastest finisher, $1 million and $500,000 to the runners-up.

The DARPA Urban Challenge developed in response to a congressional mandate that by 2015, one-third of U.S. ground combat vehicles should be unmanned. From a military perspective, autonomous vehicles can save troops: A robot car could carry out supply runs and ordinary errands, especially through enemy terrain, without incurring casualties. However, as with most robot technology, the ideas are grander than the reality. Nobody completed the first DARPA Grand Challenge, a race through the desert, in 2004. In 2005, five out of fifteen teams finished the course; Stanford’s car, “Stanley,” won. This year is the first time the robot cars will attempt city driving.

Robot-car technology is already assisting everyday drivers: Lexus recently released a car that can parallel-park itself.

“Today, all of the high-end cars have features like adaptive cruise control, or parking assistance. It’s getting more and more automated,” explains Dan Lee, associate professor of engineering and the team’s adviser. “Now, to do it fully, the car has to have a complete awareness of the surrounding world. These are the hard problems of robotics: computer vision, having computers ‘hear’ sounds, having computers understand what’s happening in the world around them. This is a good environment to test these things.”

For Little Ben to “see” an obstacle and drive around it, the automated driving and GPS navigation have to work properly, and the laser sensors on the roof rack have to observe the object. Then Little Ben has to identify the object as an obstacle and develop a path around it. The goal for this day’s practice is to work out the kinks so that Little Ben can steer clear of other cars.

“The system is complicated enough that there are a lot of unforeseen consequences,” says Foote. “If one thing runs slow, something else crashes. In software development in general, the standard is that you spend three-fourths of your time debugging. In a project like this, it’s more like nine-tenths of the time debugging.” The car has an internal network and supplemental power supply that all three programmers are plugged into. Senior electrical-engineering major Alex Kushleyev arrives at the lot with a remote control of the type used for toy cars. This is the emergency stop button. Two additional buttons are duct-taped to the rear side panels of the car and connected to the server rack of Mac Minis that make up its “brain.” The team has spent about $100,000 on the project so far, through Penn’s General Robotics, Automation, Sensing and Perception (GRASP) Laboratory. Lockheed Martin Advanced Technology Laboratories in Cherry Hill, N.J., and Maryland-based Thales Communications also sponsor the team.

“The Prius gives us more maneuverability, and since it is a hybrid car, it has a big on-board battery. We run a lot of computers, a lot of sensors, a lot of motors in addition to the car, so we need that extra power,” says Lee. An electric motor controls Little Ben’s gas, brakes, and steering; all functions, from turn signals to wipers, can be controlled by buttons on a panel mounted above the gearshift, not unlike the customization used by disabled drivers who use their hands instead of their legs to drive. The car can be driven in the ordinary manner, or it can be driven using the hand controls. When the autopilot is engaged, there’s no need for a driver at all; a safety driver sits in the passenger’s seat with one hand on the emergency stop button. It’s unsettling, but thrilling, to see the car driving ahead with its steering wheel moving in front of an empty driver’s seat.

As the battery makes a low hum, Kushleyev takes the wheel and drives across the parking lot at 15 mph. In the Urban Challenge, Little Ben will have to navigate intersections and curbs, and make decisions about how to react to stop signs, other cars, and stray dogs at a maximum speed of 30 mph.

For the moment, Little Ben is learning about parking-lot obstacles. Kushleyev turns on the automated driving mechanism and the car advances a few feetthen lurches wildly to the left, to the right, and jumps off its trajectory. “Gain control!” Stewart shouts from the backseat. Kushleyev hits override, and the brakes. The car stops. Directly ahead is a light pole.

“Only a GPS reading away from death,” Stewart mutters cheerfully. The engineers debate the swerving problem: The car is making a big wiggle where it’s supposed to make a small, smooth turn. The laser sensors are scanning the area ahead of the car, but the software isn’t registering the light pole as an obstacle. This seems to be affecting the steering, causing the car to jerk instead of turning smoothly.

Foote and Stewart confer. They are robot-car veterans, having worked on two Grand Challenge robot cars as undergrads at Caltech. Their last autonomous vehicle was “Alice,” a Ford E350 van developed for the 2005 Grand Challenge. In the desert race, Alice drove herself about seven miles before heading intoand overa barrier separating the media tent from the race course. Judges disabled Alice before she made headlines.

Little Ben’s steering wheel moves on its own a few times; Stewart and Vernaza are controlling it from the backseat. Code problem solved, Kushleyev drives the car across the parking lot again and engages the autopilot. The steering jerks, and the car heads toward an enormous snowplow parked at the edge of the lot as a grating sound screams out of the engine.

“Bugger,” says Stewart.

“Maybe it’s Sheep?” says Vernaza, naming one of the programs controlling the car.

“This is high up on my list of things I don’t want to fix today,” says Stewart.

More coding ensues. Soon it’s time to practice driving curves. The course is a huge oval laid out in the parking lot. Kushleyev drives around it once, varying his speed. Then it’s Little Ben’s turn. The car moves smoothly now, driving between the lane markers and taking the curves. At such a slow speed, it feels like a carnival ride, but without the security of a track beneath the car. Little Ben loops around the course once, twice, then stops just past a large puddle. The car’s tires are almost perfectly aligned with the tracks from its first trip around. It’s almost possible to imagine the scene at the Urban Challenge: 89 robot cars quietly moving down the street together, pausing at a stop light, turning left, not a driver in sight.

“After a while, you kind of forget the cool factor,” says Stewart, shrugging. He turns back to his laptop, and the rest of the team gathers around the car like a high-tech pit crew. With an hour of sunlight left, there is time for more troubleshooting. The Ben Franklin Racing Team has a robot race to win.