Robots stalking robots stalking robots

It's a robot version of spy versus spy.

Stanford computer scientists have equipped "observer" robots with video cameras and successfully programmed them to track "target" robots. The researchers now are working on the more difficult problem of programming their observers to stalk targets that are attempting to evade pursuit.

Naturally, the Army is interested, and is providing financial support for the project. Observer robots that can automatically track potentially hostile targets could be real lifesavers for soldiers fighting in conditions of limited visibility, such as urban environments.

Autonomous observers (AOs) ­ as computer science Professor Jean-Claude Latombe and his students call their electromechanical creations ­ have other potential uses. In operating rooms they could automatically keep video displays which surgeons depend upon for delicate operations ­ zeroed in on key tissues despite the obstructions created by the movements of people and machinery. AOs could perform search and rescue operations in potentially hostile environments; monitor operations in remote assembly plants; and supervise automated construction efforts in outer space.

"Originally, we called them intelligent observers, but they aren't really that intelligent," says Hector H. Gonzalez-Banos, a doctoral student in electrical engineering who is working on the project. "So we named them autonomous observers instead."

Computer science Professor Jean-Claude Latombe, left, and graduate student Hector Gonzalez-Banos are framed by an autonomous observer on the right and a target robot on the left.

The autonomous observer may not be all that intelligent, but it does more than simply follow its target around at a constant distance. It must constantly calculate the positions that it needs to assume to ensure that the target doesn't disappear behind a column or down a hallway.

This kind of robotic motion planning is anything but simple. The robot first must be familiar with the area in which it is operating, so Latombe's group gave its observer robot the ability to map a new area when it first enters. The robot moves about continually measuring the distances to walls and furniture with a horizontal laser range sensor and it uses this information to create a two-dimensional floor plan. Next, the robot uses a horizontal video camera to create a series of overlapping three-dimensional views of the space. Finally, it combines these into a 3-D rendering of the space.

The observer robot carries a second camera that is focused on the ceiling. This helps it keep track of its position by recognizing block patterns that the researchers have attached to the ceiling in a grid pattern. "We could program the robot to recognize the tile pattern, but this is a lot easier," says Gonzalez-Banos apologetically.

The target robot also sports its own black-and-white pattern stenciled on every side. The observer robot uses this to identify it. In an associated project with Professor Ruzena Bajcsy's group at the University of Pennsylvania, Latombe and his students are working on a technique that will allow the robot to identify and track unmarked robots and people.

Human operators have a choice of two views on their computer screens. In one, they can see the view from the AO's horizontal camera. But when the robot is zipping hither and yon and panning quickly back and forth, this viewpoint can leave people pretty woozy, Gonzalez-Banos says. So, in most cases, the operators choose the 3-D view that shows the position of the observer and target robots within the representation of the space that the AO initially created.

So far, the researchers have been experimenting with a single observer and a single target. The robots, which were built by Nomadic Technologies of Mountain View, Calif., are about 4 feet tall and resemble an upright tank vacuum cleaner without the hose. Shortly, a new grant from the Army will give the researchers four additional robots. The new, smaller robots will allow the researchers to devise methods for deploying multiple observers.

Latombe's group also is working with Steven La Valle of Iowa State University on the problem of how to deal with a target that is trying to escape from surveillance.

One of the group's goals is to program groups of autonomous observers to work together to locate designated targets. This would give AOs the ability to "sweep" a given area for a target, robotic or otherwise, that is trying to avoid detection.

Given the broad range of possible applications for autonomous observers, the idea that led to their development was surprisingly modest. "We were just looking for a way to share robots with researchers in other laboratories," says Latombe. The cost of building and maintaining research robots is extremely high, so finding ways to spread that cost was quite appealing.

The Internet gives researchers an inexpensive way to program and operate robots from afar, but it didn't have a good way to view exactly what the remote robot is doing when the researchers are running it. The autonomous observer was Latombe's solution. By using one of their robots to keep a second, target robot continuously in view and then feeding this view back over the Internet, the long-distance researchers can observe what the target robot does as it follows the instructions that they send via the Internet.

At the Monterrey Institute of Technology (ITESM) in Mexico, researchers headed by Professor Jose Luis Gordillo are collaborating with Latombe's group on this project. The Mexican scientists have programmed one of their robots to identify empty Coke cans sitting on tables and sweep them into a waste basket that it carries.

To test the value of the autonomous observer, Latombe and his students modified one of their mobile robots so that it can perform the same task. The Monterrey researchers then wrote a program for the Stanford robot that allows it, at their command from Mexico, to collect Coke cans placed around the Stanford lab. As the target robot wanders about the Stanford lab searching for empty Coke cans (it only collects Diet Coke empties because it can only recognize cans with white labels), the observer robot zips around keeping it in sight. That allows the Monterrey researchers to see just how well their program is working.

This research is funded by the Army Research Office, the Office of Naval Research and the National Science Foundation.